Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

The modem's IPA: the hypervisor takes one SMMU match register per stream, and the handshake is Android's, not Linux's

shimoku74a19cf5ab2027
diff --git a/.gitignore b/.gitignore
index 92cca1f..8fe3fbb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -79,3 +79,4 @@ __pycache__/
!/xnu-deps/build/lzbitmap/libzbitmap.c
!/xnu-deps/build/lzbitmap/libzbitmap.h
/tools/android/f2fs-resize
+/tools/macos-bin/platform-tools/
diff --git a/README.md b/README.md
index 91e5047..7466371 100644
--- a/README.md
+++ b/README.md
@@ -2300,6 +2300,7 @@ SpringBoard спрашивает десять тысяч раз за все се
каждые 15 секунд, того не стоит. Заглушить первый драйвер — работа того же рода, что `RedwoodBattery`; второй —
это модем Apple на PCIe, а модем этого телефона сидит на самом кристалле и говорит по QMI с подсистемой
Qualcomm, так что за драйвером нет и того, к чему он ведёт.
+Текущий план подключения MPSS и проверка аппаратных ресурсов: [docs/cellular.md](docs/cellular.md).
**Остальные из того же списка «никогда не пробовали».** `cloudd` и `mobileassetd` не падают, и 42 ненайденных
обращения за сеанс с ними исчезают (`com.apple.cloudd` — 26, `com.apple.mobileassetd.v2` — 16). И всё-таки
diff --git a/docs/cellular.md b/docs/cellular.md
new file mode 100644
index 0000000..dee9610
--- /dev/null
+++ b/docs/cellular.md
@@ -0,0 +1,185 @@
+# Сотовая связь: состояние и план порта
+
+Сейчас сотовая связь в HackiOS **не работает**, но модем уже запускается из XNU (см. «21 октября» ниже). Wi-Fi использует WPSS и свой GLINK/QRTR; модем MPSS — отдельный процессор с другим каналом. `redwood_qmi.c` кодирует сообщения QMI и теперь позволяет привязать клиент к заданному транспорту QRTR, но канал к MPSS ещё не реализован. У iOS также нет драйверов AppleBaseband и AppleIPAppender для этого аппаратного пути. Запуск `CommCenter` без них уже проверялся: он завершается аварийно (см. README, раздел «Телефония»).
+
+Проверка `python3 tools/modem_firmware.py --dtb prebuilt/uefi/redwood.dtb --firmware-dir fw-modem --node /soc/qcom,mss@4080000` сопоставила 28 сегментов штатной прошивки с областью `mpss@8b800000` (PAS ID 4). `MODEM=1 SYSTEM=1 ... userland/build.sh` теперь может положить эти локальные файлы в системный APFS после проверки. Это только подготовка образа: наличие прошивки в `/fw` не запускает MPSS. Исходник `redwood_nas.c` содержит клиент QMI NAS для заданного транспорта QRTR: он ищет службу, запрашивает регистрацию и силу сигнала, разбирает ответы и сбрасывает состояние при исчезновении службы. Уровень `bars` остаётся нулевым без подтверждённой регистрации. MPSS-транспорта в XNU пока нет, поэтому этот клиент ещё не получает сообщения с телефона и не подключён к статус-бару iOS. Форматы NAS сверены с [описанием libqmi](https://github.com/linux-mobile-broadband/libqmi/blob/main/data/qmi-service-nas.json).
+
+## Проверка на телефоне, 20 октября 2026
+
+На redwood через `fastboot boot` временно запущены xnuloader и образ iOS 16.5. Системный APFS загружен в ОЗУ; разделы телефона не прошивались. `launchd` запустил `SpringBoard`, и на дисплее открылся интерфейс «Настройки». Это подтверждает загрузку iOS и работу экрана, но не работу модема. В журнале `mg_test` показывает `telephony = true` и `cellular-data = false`; запросы `com.apple.commcenter.xpc` и `com.apple.basebandd.xpc` остаются без службы. Значение `telephony = true` — ответ MobileGestalt о возможностях устройства, а не регистрация в сети. Тест выполнен с готовым ядром из `prebuilt`, поэтому изменения QMI в исходниках XNU в этой загрузке **не проверялись**. Дальнейший аппаратный тест требует сборки ядра с MPSS/QRTR и интеграции с iOS.
+
+## Что установлено по redwood
+
+`prebuilt/uefi/redwood.dtb` содержит зарезервированную память `mpss@8b800000`, `qcom,smp2p-modem`, `qcom,glink/modem/qcom,modem_qrtr`, `qcom,ipa@1e00000` и `qcom,rmnet-ipa`. Это разные части тракта: запуск прошивки модема, сигнализация, управляющий QMI-канал и передача IP-пакетов. Дерево Android [redwood](https://github.com/Redwood-AOSP/android_device_xiaomi_redwood) включает `full_base_telephony.mk`; его `init.qcom.rc` запускает `vendor.qcrild` и `vendor.qcrild2`. [Ветка araxys с ядром/DTB](https://github.com/Redwood-AOSP/android_device_xiaomi_redwood-kernel/tree/araxys) пригодна как аппаратный справочник. Бинарники Android RIL нельзя просто запустить в iOS: им нужны Android binder, HAL, службы и ABI.
+
+Проверка по USB 20 октября 2026 года: fastboot определил продукт `redwood` и разблокированный загрузчик; Android сообщил `ro.boot.hardware.sku=redwood`, `ro.soc.model=SM7325`. Под Android подсистема `modem` и прошивка `yupik_ipa_fws` были `ONLINE`, `vendor.qcrild`, `vendor.qcrild2`, `vendor.qrtr-ns` — `running`; присутствовали `rmnet_ipa0` и `rmnet_data0`–`rmnet_data5`. Состояние SIM: `ABSENT,LOADED` для двух слотов. Временная загрузка `xnuloader` через `fastboot boot` и его `selftest` прошли (`selftest-ok`); это проверка загрузчика, а не сотовой связи в XNU.
+
+Android `dmesg` уточняет последовательность: сначала поднимается `yupik_ipa_fws`, затем MPSS загружает `modem.mdt` и сегменты в `0x8b800000..0x9ae00000`, получает `Power/Clock ready`, после чего QRTR фиксирует готовность QMI. Штатные `modem.mdt`/`modem.b*` (около 85 МиБ) сохранены локально в игнорируемом Git каталоге `fw-modem/` и не входят в исходники. Их ELF-заголовок содержит 30 программных заголовков и 28 загружаемых сегментов; все файлы совпали по размеру, а сегменты лежат в зарезервированном окне. Загрузчик WPSS ограничен 16 сегментами; путь ADSP рассчитан на 48, но использует собственные ресурсы памяти, SMP2P и GLINK и не может запускать MPSS простым изменением имени прошивки.
+
+## Границы общей реализации
+
+Общими для устройств должны быть контракт транспорта QRTR, разбор QMI, состояние SIM/сети и интерфейс передачи IP-пакетов. Запуск прошивки MPSS, ресурсы памяти, прерывания, SMMU, IPA и прошивки остаются в профиле и драйвере конкретного SoC/устройства. По наличию `qcom,glink` нельзя объявлять поддержку всех Qualcomm или других телефонов.
+
+Порядок доведения до работающей связи:
+
+1. На подключённом Android собрать отчёт `python3 tools/cellular_probe.py --adb` и сверить DTB, версию прошивки и активные службы. Скрипт читает только открытые свойства и не собирает IMEI, IMSI и номера SIM.
+2. Поднять MPSS в XNU с собственным GLINK/QRTR-каналом, отдельным от WPSS; проверить обнаружение QMI-служб, не отправляя команд регистрации или передачи данных.
+3. Реализовать QMI DMS/UIM/NAS/WDS и состояние SIM/сети; отдельно включить путь IPA/rmnet для мобильных данных.
+4. Подключить эти возможности к CoreTelephony/CommCenter через совместимый сервис и аппаратный драйвер, затем проверить SIM, регистрацию, входящие/исходящие вызовы, SMS и мобильные данные на телефоне. VoLTE/IMS — отдельная работа.
+
+Пока пункт 2 не подтверждён на устройстве, не включать узел `baseband` и не переключать систему на cellular `CommCenter`: это создаёт аварийные циклы без радио.
+
+## Команды до и после подключения
+
+```sh
+python3 tools/cellular_probe.py > modem-dtb.json
+# После подключения телефона с разрешённой USB-отладкой:
+python3 tools/cellular_probe.py --adb > modem-android.json
+```
+
+Для другого устройства передать его DTB через `--dtb PATH`; прошивку на телефон инструмент не записывает.
+
+## 21 октября 2026: модем запущен из XNU
+
+Ядро теперь собирается на этом Mac (см. `xnu/README.md`, «Сборка на macOS»), и драйвер
+`bsd/dev/redwood_mpss.c` — копия пути аудио‑DSP (`redwood_adsp.c`) с числами модема — проверен
+на телефоне через `xl run` с `modem_step=11 modem_return=1`:
+
+- шаги 1–10 проходят: прошивка `modem.*` из `/fw` системного образа (85 МБ, 28 сегментов)
+ записана в `0x8b800000` и прочитана обратно без расхождений; TrustZone принял подпись
+ (образ 4) и область `0xf600000` байт; шины CX и MSS запрошены у RPMh на уровне 384; AOP
+ получил `modem: on`; через 63 мс после старта модем сказал `slave-kernel = 0x6` (поднялся и
+ взял питание сам); GLINK согласован, канал `IPCRTR` открыт в обе стороны; QRTR: hello в обе
+ стороны, модем — узел 0;
+- модем объявил только две службы: 43 (SSCTL, экземпляр 18) и 66 (реестр служб). DMS и NAS
+ не появились за 40 с, поэтому шаг 11 (DMS «online», опрос NAS) не выполнен;
+- по просьбе модем остановился за 43 мс (`0x10e`), TrustZone выключил его, AOP получил
+ `modem: off`; телефон вернулся в загрузчик штатно.
+
+Причина отсутствия DMS/NAS — у модема нет доступа к своим файловым системам: на Android их
+отдаёт демон `rmtfs` (служба QMI 14 поверх QRTR, [linux-msm/rmtfs](https://github.com/linux-msm/rmtfs)).
+В драйвер добавлена такая служба: четыре раздела (`modemst1`, `modemst2`, `fsc`, `fsg`)
+читаются при старте шагов целиком в память ядра через `redwood_ufs_read_partition()`
+(нужен `ufs=2` в аргументах ядра — блочная служба загрузчика), модем открывает их по путям
+`/boot/modem_fs1` и т. д., получает буфер 2,5 МБ (`qcom,rmtfs_sharedmem@0`), в который
+TrustZone пускает модем вызовом `SCM MP ASSIGN` (HLOS + MSS_MSA, чтение и запись), и читает
+или пишет секторы через него. Записи остаются в памяти ядра: **разделы телефона не
+записываются**. Служба объявляется модему в ответ на hello и на lookup (наша сторона
+сервиса имён QRTR). Это ещё **не проверено на телефоне**: после предыдущего прогона телефон
+исчез с USB (сторожевой таймер загрузчика), нужен ручной перезапуск в fastboot.
+
+Запуск теста:
+
+```sh
+./xl enter && ./xl ramdisk userland/system.apfs # образ с MODEM=1 SYSTEM=1 (один раз после входа в загрузчик)
+XL_CMDLINE="debug=0x8 rd=md0 hv_wdt_pet=1 bootcpu=7 cpus=8 cpufreq=1 sb_delay=600 ufs=2 modem_step=11 modem_return=1 modem_new=12 modem_nas_wait=60" XL_SECS=300 XL_FOLLOW=3 ./xl run
+```
+
+`modem=1` в обычной загрузке делает всё то же на своём потоке и оставляет модем включённым,
+опрашивая NAS каждые две секунды; результат — в `sysctl kern.redwood.cellular_up`,
+`cellular_registered`, `cellular_bars`, `cellular_dbm`, `cellular_radio`. Палочки в
+статус‑баре iOS из этого ещё не получаются: нужна либо часть CommCenter, либо подмена
+элемента статус‑бара в `libredwood-sb.dylib` по этим sysctl (следующий шаг).
+
+## 23 октября 2026: буфер RMTFS и перезапуски телефона
+
+Ночные прогоны (`tools/macos-bin`: adb из platform-tools r34 — сервер adb 37.0.1 на этом Mac
+убивается через две секунды после запуска; `xl` возвращает телефон в Android после каждого
+прогона) показали:
+
+- буфер RMTFS должен лежать **ниже 4 ГБ**: адреса в запросах модема 32‑битные (с буфером
+ `0x197508000` модем пишет «xpu lock failed» по усечённому адресу и сдаётся);
+- память CDSP (`0x9c700000`) не годится: после её передачи TrustZone отказывается запускать
+ модем (`-3145799`); память видеокодека (`0x8ae00000`) и `ramoops` (`0xb0000000`) —
+ модем стартует, но телефон сбрасывается при первой нашей записи сектора; свободная память
+ UEFI (`0xa8000000`, `modem_buffer=`) — то же, пока буфер отдан только HLOS и MSS;
+- Android отдаёт буфер трём владельцам: HLOS, MSS_MSA (15) и NAV (0x2b) — в DTB телефона у
+ `qcom,rmtfs_sharedmem@0` стоит `qcom,vm-nav-path`. С тремя владельцами запись в буфер после
+ «замка» модема проходит без сброса (цикл 2: сектор записан, модему отказано — ядро вернулось
+ штатно). Обратная передача (`{HLOS, MSS, NAV} → {HLOS}`) при остановке отказывает
+ (`-2`, ответ `0x56`): видимо, пока держится замок модема; на перезагрузку это не влияет;
+- опыты включаются ключами `modem_share_test=1` (пробная запись после передачи),
+ `modem_rmtfs_rw_off=1` (отказ в чтении), `modem_rmtfs_rw_fake=1` (запись есть, ответ «нет»),
+ `modem_rmtfs_delay=S` (пауза перед первым чтением), `modem_share_off=1`, `modem_no_nav=1`.
+
+Паника `copyio.c:89 Assertion failed: pan != 0` в пользовательских процессах iOS встречалась в
+двух прогонах с модемом; её причина не найдена.
+
+## 23 октября 2026, вечер: модем читает файлы, выходит в ONLINE
+
+С буфером в `0xa8000000`, отданным HLOS, MSS и NAV, модем читает свои файловые системы и за
+~30 с объявляет все службы (88 объявлений: DMS 2, NAS 3, UIM 11, WDS 1, ...). Дальше он стоит в
+режиме **OFFLINE** и на `DMS Set Operating Mode = online` отвечает «устройство не готово» (52),
+на `low power` — «недопустимый переход» (60): он ждёт двух служб процессора приложений, которые
+на Android дают демоны `pd-mapper` и `tqftpserv`:
+
+- **реестр служб** (QMI 64, версия 257): модем спрашивает, где `tms/servreg` и другие; ответ —
+ домены (`msm/modem/root_pd`, экземпляр 180) из `*.jsn` раздела прошивки;
+- **TFTP поверх QRTR** (QMI 4096): модем пишет `/readwrite/server_check.txt`, читает
+ `/readonly/firmware/image/modem_pr/so/665_0_0.mbn` кусками (опции `rsize`, `seek`, `wsize`,
+ `blksize`), просит `/readwrite/ota_firewall/ruleset` (нет — не страшно).
+
+Обе службы теперь в `redwood_mpss.c` (разделы «TFTP over QRTR» и «the service registry's
+locator»): файлы берутся из `/fw/image` системного образа (`fw-modem/image` — копия
+`/vendor/firmware_mnt/image` телефона, снятая `adb pull`; `MODEM=1 SYSTEM=1 userland/build.sh`
+кладёт её в образ), записи модема хранятся в памяти ядра на время прогона. Сервер имён QRTR
+объявляет наши службы модему при hello и отвечает на его lookup.
+
+Результат (прогон 23:00): после обслуживания файлов модем **согласился перейти в ONLINE**, NAS
+ответил «не зарегистрирован, CS/PS отключены», и через несколько секунд модем упал в задаче
+`enl2_dl` (LTE L2, downlink) с нулевым адресом. Предположительно при выходе в сеть он обращается
+к IPA, которого с нашей стороны нет; проверяется ответ «IPA недоступен» через SMP2P
+(`modem_ipa=1`), дальше — либо запуск прошивки IPA и его тактов, либо иной обход.
+
+## 24 октября 2026: IPA — прошивка, такт, SMMU
+
+Падение `enl2_dl` повторяется в каждом прогоне через несколько секунд после ONLINE. Гипотеза —
+модему нужен работающий IPA (пакетный ускоритель, через него идут данные LTE), и в
+`redwood_mpss.c` появился его минимальный драйвер («the IPA» в файле), по образцу
+`drivers/net/ipa` Linux (версия IPA 4.11, sc7280):
+
+- **такт**: без голоса RPMh за BCM `IP0` (`ipa_clock()`) TrustZone отвечает на
+ `AUTH_AND_RESET` образа 15 «Unhandled implementation specific error»; с голосом прошивка
+ `yupik_ipa_fws` (перемещаемый ELF, кладётся от `0x8b700000`) стартует, микроконтроллер
+ отвечает «booted» (`ipa_status()`);
+- **память IPA**: `ipa_memory_init()` обнуляет локальную память (0x3000), пишет канарейки и базу
+ контекстов обработки; раскладка — `ipa_data-v4.11.c`;
+- **QMI 0x31**: наш хост (экземпляр 1) посылает модемному драйверу IPA `INIT_DRIVER` с раскладкой
+ памяти (таблиц маршрутов 15, иначе модем падает на `num_indices >= rtng_tbl_max`), модем
+ подписывается на индикацию; ответ `INIT_COMPLETE` ещё не дождались;
+- **SMMU**: IPA ходит в память через SMMU процессора приложений потоками `0x480` (AP) и `0x482`
+ (микроконтроллер). Поток без совпадения даёт отказ, обход (bypass) гипервизор не даёт.
+ Контекст с нашими таблицами (тождественное отображение гигабайта DDR от `0x80000000` и imem
+ `0x146a8000`) гипервизор принял (`sctlr` стал `0x127`, `s2cr` — наш), но **регистр
+ совпадения с потоком `0x480` и маской `0x2` он сбросил в 0**. У Wi-Fi (`0x1c00`, маска 1,
+ как в дереве устройств) прошло. В дереве устройств у потоков IPA маска 0 — гипервизор, видимо,
+ сверяет пару (поток, маска) со списком хозяев. Прогон 21: по регистру совпадения на каждый
+ поток, маска 0.
+
+Телефон после каждого прогона возвращается в Android (`cycle.sh`); разделы не пишутся.
+
+### 24 октября, ночь: что говорит сам Android
+
+Android телефона рутован (KernelSU), и его `dmesg`, debugfs (`/d/ipa`, `/d/gsi`) и IPC-логи
+(`/d/ipc_logging/ipa/log`, `gsi/log`) читаются по adb. По ним:
+
+- SMMU: прогон 21 подтвердил догадку — **гипервизор принял по регистру совпадения на поток**
+ (`0x480` и `0x482`, маска 0, оба в контекст 7), отказов IPA не было;
+- модем **не отвечал на INIT_DRIVER** вовсе (13 с до падения `enl2_dl`). У Android ответ
+ приходит через 11 мс, и запрос другой: `platform_type 3` (мы слали 1 — «data card»),
+ `num_indices 7` (мы 14), область модема `0x27d8` размером `0x800` (мы `0x1f18/0x100c`,
+ раскладка upstream v4.11 не совпадает с раскладкой `ipa_4_11_mem_part` из techpack/dataipa),
+ контексты обработки `0xad0` размером `0xac0`, статистика квот `0x1d00/0x30`, дропов
+ `0x27b0/0x20`; TLV 0x18 (`is_ssr_bootup`) при первой загрузке не посылается;
+- после ответа модем шлёт `INIT_MODEM_DRIVER_CMPLT` (0x35), `CONFIG` (0x27),
+ `INSTALL_FILTER_RULE` (0x23), `FILTER_INSTALLED_NOTIF` (0x24) — Android отвечает успехом;
+ индикацию `INIT_COMPLETE` (0x22) шлёт сразу после ответа на INIT_DRIVER;
+- микроконтроллер IPA загружает **модем** (его события «booted» приходят после ответа), нам его
+ область (0x0..0x288) трогать не надо;
+- прошивка IPA у Android кладётся с `0x8b710000` (reserved `ipa_fw@8b700000` 0x10000,
+ `ipa_gsi@8b710000` 0xa000); окно SRAM — `ipa-base + 0x40000 + 0x10000 = 0x1e50000`, 0x3000
+ байт (блок в драйвере был отображён на 4 КБ — исправлено на 0x4000).
+
+Источник раскладки и обработчиков: `techpack/dataipa/drivers/platform/msm/ipa/ipa_v3`
+(`ipa_utils.c: ipa_4_11_mem_part`, `ipa_qmi_service.c`), ядро
+Atom-X-Devs/android_kernel_xiaomi_sm7325 (у MiCode ветка `redwood-s-oss`).
diff --git a/ios165/fetch.sh b/ios165/fetch.sh
index ef6f43d..ee15b25 100755
--- a/ios165/fetch.sh
+++ b/ios165/fetch.sh
@@ -39,7 +39,7 @@ unzbm_all() { # unzbm_all DIR: expand every LZBITMAP file below DIR
find "$1" -type f -name '*:com.apple.decmpfs' | sed 's/:com\.apple\.decmpfs$//' | tr '\n' '\0' | xargs -0 -r -n 200 $UNZBM
local left
left=$(find "$1" -name '*:com.apple.*' | wc -l)
- [ "$left" = 0 ] || { echo "fetch.sh: $left files in $1 are still compressed"; exit 1; }
+ [ "$left" -eq 0 ] || { echo "fetch.sh: $left files in $1 are still compressed"; exit 1; }
}
# --- the root filesystem --------------------------------------------------------------------------
@@ -82,7 +82,7 @@ if [ ! -f .done-cryptex ]; then
unzbm_all cryptex.cache
mv cryptex.cache/System/Library/Caches/com.apple.dyld $CACHE
rm -rf cryptex.cache
- [ "$(ls $CACHE | wc -l)" = 43 ] || { echo "fetch.sh: the dyld shared cache is incomplete"; exit 1; }
+ [ "$(ls $CACHE | wc -l)" -eq 43 ] || { echo "fetch.sh: the dyld shared cache is incomplete"; exit 1; }
touch .done-cryptex
[ -n "$KEEP" ] || rm -f cryptex-os.dmg
fi
diff --git a/tools/cellular_probe.py b/tools/cellular_probe.py
new file mode 100755
index 0000000..c6ccc76
--- /dev/null
+++ b/tools/cellular_probe.py
@@ -0,0 +1,133 @@
+#!/usr/bin/env python3
+"""Read-only modem inventory from an FDT and, optionally, Android over ADB.
+
+This is a discovery tool. It does not start MPSS or implement a cellular driver.
+The output intentionally omits IMEI, IMSI, phone numbers and SIM identifiers.
+"""
+
+import argparse
+import json
+import pathlib
+import struct
+import subprocess
+
+FDT_MAGIC = 0xD00DFEED
+BEGIN_NODE, END_NODE, PROP, NOP, END = 1, 2, 3, 4, 9
+KEYWORDS = ("modem", "mpss", "mss", "ipa", "qrtr", "glink", "smem", "smp2p", "remoteproc")
+PROPERTY_NAMES = {"compatible", "status", "memory-region", "qcom,pas-id", "qcom,smem", "qcom,smem-id", "reg", "interrupts", "phandle", "linux,phandle"}
+ANDROID_PROPERTIES = (
+ "ro.product.device", "ro.boot.hardware.sku", "ro.board.platform", "ro.boot.hardware", "ro.boot.baseband",
+ "gsm.version.baseband", "init.svc.vendor.qcrild", "init.svc.vendor.qcrild2",
+ "init.svc.vendor.qrtr-ns", "init.svc.vendor.ssgqmigd", "gsm.sim.state",
+)
+
+
+def fdt_nodes(data):
+ if len(data) < 40:
+ raise ValueError("DTB header is too short")
+ magic, total, off_struct, off_strings, _, version, _, _, size_strings, size_struct = struct.unpack_from(">10I", data)
+ if magic != FDT_MAGIC or version < 16 or total > len(data):
+ raise ValueError("invalid or unsupported DTB")
+ if off_struct + size_struct > total or off_strings + size_strings > total:
+ raise ValueError("DTB sections exceed file size")
+ strings = data[off_strings:off_strings + size_strings]
+ pos, limit, stack, indexes, nodes = off_struct, off_struct + size_struct, [], [], []
+ while pos + 4 <= limit:
+ token = struct.unpack_from(">I", data, pos)[0]
+ pos += 4
+ if token == BEGIN_NODE:
+ stop = data.find(b"\0", pos, limit)
+ if stop < 0:
+ raise ValueError("unterminated DTB node")
+ name = data[pos:stop].decode("ascii", "replace")
+ stack.append(name)
+ nodes.append({"path": "/" + "/".join(x for x in stack if x), "properties": {}})
+ indexes.append(len(nodes) - 1)
+ pos = (stop + 4) & ~3
+ elif token == END_NODE:
+ if not stack:
+ raise ValueError("unbalanced DTB nodes")
+ stack.pop()
+ indexes.pop()
+ elif token == PROP:
+ if pos + 8 > limit or not nodes or not stack:
+ raise ValueError("invalid DTB property")
+ length, nameoff = struct.unpack_from(">II", data, pos)
+ pos += 8
+ if pos + length > limit or nameoff >= len(strings):
+ raise ValueError("DTB property exceeds section")
+ name_end = strings.find(b"\0", nameoff)
+ if name_end < 0:
+ raise ValueError("unterminated DTB property name")
+ name = strings[nameoff:name_end].decode("ascii", "replace")
+ if name in PROPERTY_NAMES:
+ raw = data[pos:pos + length]
+ if name in ("compatible", "status"):
+ value = [x.decode("ascii", "replace") for x in raw.rstrip(b"\0").split(b"\0")]
+ else:
+ value = raw.hex()
+ nodes[indexes[-1]]["properties"][name] = value
+ pos = (pos + length + 3) & ~3
+ elif token == NOP:
+ continue
+ elif token == END:
+ if stack:
+ raise ValueError("unclosed DTB nodes")
+ return nodes
+ else:
+ raise ValueError(f"invalid DTB token {token}")
+ raise ValueError("DTB has no end token")
+
+
+def modem_nodes(data):
+ nodes = fdt_nodes(data)
+ return [node for node in nodes if any(
+ word in (node["path"].rsplit("/", 1)[-1] + " " + " ".join(node["properties"].get("compatible", []))).lower()
+ for word in KEYWORDS
+ )]
+
+
+def adb_call(serial, *args):
+ command = ["adb"] + (["-s", serial] if serial else []) + list(args)
+ result = subprocess.run(command, capture_output=True, text=True, timeout=15, check=False)
+ if result.returncode:
+ raise RuntimeError(result.stderr.strip() or f"adb exited {result.returncode}")
+ return result.stdout.strip()
+
+
+def android_inventory(serial):
+ devices = [line.split()[0] for line in adb_call(None, "devices").splitlines()[1:]
+ if len(line.split()) >= 2 and line.split()[1] == "device"]
+ if serial is None:
+ if len(devices) != 1:
+ raise RuntimeError(f"expected one authorized Android device, found {len(devices)}; use --serial")
+ serial = devices[0]
+ elif serial not in devices:
+ raise RuntimeError(f"Android device {serial!r} is not authorized or connected")
+ properties = {key: adb_call(serial, "shell", "getprop", key) for key in ANDROID_PROPERTIES}
+ subsystem_lines = adb_call(serial, "shell", "for d in /sys/bus/msm_subsys/devices/subsys*; do "
+ "[ -r \"$d/name\" ] && [ -r \"$d/state\" ] && "
+ "printf '%s:%s\\n' \"$(cat \"$d/name\")\" \"$(cat \"$d/state\")\"; done")
+ subsystems = dict(line.split(":", 1) for line in subsystem_lines.splitlines() if ":" in line)
+ interfaces = adb_call(serial, "shell", "ls", "/sys/class/net").split()
+ return {"properties": properties, "subsystems": subsystems,
+ "cellular_interfaces": [name for name in interfaces if name.startswith(("rmnet", "wwan", "ipa"))]}
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--dtb", type=pathlib.Path, default=pathlib.Path(__file__).resolve().parents[1] / "prebuilt/uefi/redwood.dtb")
+ parser.add_argument("--adb", action="store_true", help="also query a connected Android phone without changing it")
+ parser.add_argument("--serial", help="ADB serial when more than one phone is connected")
+ args = parser.parse_args()
+ try:
+ report = {"dtb": str(args.dtb), "modem_related_nodes": modem_nodes(args.dtb.read_bytes())}
+ if args.adb:
+ report["android"] = android_inventory(args.serial)
+ except (OSError, ValueError, RuntimeError, subprocess.TimeoutExpired) as error:
+ parser.exit(1, f"cellular_probe: {error}\n")
+ print(json.dumps(report, ensure_ascii=False, indent=2))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/macos-bin/README.md b/tools/macos-bin/README.md
new file mode 100755
index 0000000..78656e4
--- /dev/null
+++ b/tools/macos-bin/README.md
@@ -0,0 +1,4 @@
+What `xl` needs of Linux on a Mac: `flock`, GNU `timeout` and `stat` (MacPorts' gtimeout and gstat),
+and Android's `fastboot` and `adb` (platform-tools, which `xl` finds here or in $PATH). Put the
+platform-tools directory next to this file as `platform-tools/` (not in the repository).
+r34 platform-tools (adb 34.0.5): the newer 37.0.1 adb server is killed within seconds on this Mac
diff --git a/tools/macos-bin/flock b/tools/macos-bin/flock
new file mode 100755
index 0000000..8ce9fff
--- /dev/null
+++ b/tools/macos-bin/flock
@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+# flock(1) for macOS, as much of util-linux's as xl uses: flock [-n] [-w SECS] FILE COMMAND...,
+# flock [-n] FD (the lock is on the open file description, which the shell keeps open), flock [-n] FILE -c CMD
+import fcntl, os, subprocess, sys, time
+args = sys.argv[1:]
+nonblock = False; wait = None
+while args and args[0].startswith('-'):
+ a = args.pop(0)
+ if a in ('-n', '--nonblock', '--nb'): nonblock = True
+ elif a in ('-w', '--wait', '--timeout'): wait = float(args.pop(0))
+ elif a in ('-x', '--exclusive', '-e'): pass
+ elif a in ('-s', '--shared'): pass
+ else: sys.exit('flock: unknown option ' + a)
+if not args: sys.exit('flock: usage: flock [-n] [-w secs] FILE|FD [command...]')
+target = args.pop(0)
+if target.isdigit() and not args:
+ fd = int(target)
+else:
+ fd = os.open(target, os.O_RDONLY | os.O_CREAT, 0o644)
+deadline = None if wait is None else time.time() + wait
+while True:
+ try:
+ fcntl.flock(fd, fcntl.LOCK_EX | (fcntl.LOCK_NB if (nonblock or deadline is not None) else 0))
+ break
+ except OSError:
+ if nonblock or (deadline is not None and time.time() >= deadline): sys.exit(1)
+ time.sleep(0.1)
+if not args: sys.exit(0)
+sys.exit(subprocess.call(args))
diff --git a/tools/macos-bin/stat b/tools/macos-bin/stat
new file mode 100755
index 0000000..0424613
--- /dev/null
+++ b/tools/macos-bin/stat
@@ -0,0 +1,2 @@
+#!/bin/sh
+exec /opt/local/bin/gstat "$@"
diff --git a/tools/macos-bin/timeout b/tools/macos-bin/timeout
new file mode 100755
index 0000000..30d68d4
--- /dev/null
+++ b/tools/macos-bin/timeout
@@ -0,0 +1,2 @@
+#!/bin/sh
+exec /opt/local/bin/gtimeout "$@"
diff --git a/tools/modem_firmware.py b/tools/modem_firmware.py
new file mode 100755
index 0000000..e93825a
--- /dev/null
+++ b/tools/modem_firmware.py
@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+"""Validate a Qualcomm MDT firmware set against a device tree modem region.
+
+This is an offline check. It does not authenticate, start, or contact the modem.
+"""
+
+import argparse
+import json
+import pathlib
+import struct
+
+from cellular_probe import fdt_nodes
+
+
+def cell(prop):
+ raw = bytes.fromhex(prop)
+ if len(raw) != 4:
+ raise ValueError("expected one 32-bit device tree cell")
+ return struct.unpack(">I", raw)[0]
+
+
+def inspect(dtb_path, firmware_dir, node_path):
+ nodes = fdt_nodes(dtb_path.read_bytes())
+ by_path = {node["path"]: node for node in nodes}
+ if node_path not in by_path:
+ raise ValueError(f"modem node {node_path} is absent")
+ props = by_path[node_path]["properties"]
+ if "memory-region" not in props or "qcom,pas-id" not in props:
+ raise ValueError("modem node lacks memory-region or qcom,pas-id")
+ memory_phandle = cell(props["memory-region"])
+ matches = [node for node in nodes if any(
+ cell(node["properties"][name]) == memory_phandle
+ for name in ("phandle", "linux,phandle") if name in node["properties"])]
+ if len(matches) != 1:
+ raise ValueError(f"memory-region phandle {memory_phandle:#x} is not unique")
+ region = matches[0]
+ raw_reg = bytes.fromhex(region["properties"].get("reg", ""))
+ if len(raw_reg) != 16:
+ raise ValueError("expected a two-cell base and size for modem reserved memory")
+ base, size = struct.unpack(">QQ", raw_reg)
+ if size == 0:
+ raise ValueError("empty modem reserved memory")
+
+ header = (firmware_dir / "modem.mdt").read_bytes()
+ if len(header) < 52 or header[:5] != b"\x7fELF\x01" or header[5] != 1:
+ raise ValueError("modem.mdt is not little-endian ELF32")
+ program_offset = struct.unpack_from("<I", header, 28)[0]
+ entry_size, count = struct.unpack_from("<HH", header, 42)
+ if entry_size != 32 or not 1 <= count <= 128 or program_offset + count * entry_size > len(header):
+ raise ValueError("invalid program header table")
+ segments = []
+ total = 0
+ for index in range(count):
+ kind, _, _, address, file_size, memory_size, flags, _ = struct.unpack_from(
+ "<8I", header, program_offset + index * entry_size)
+ if kind != 1 or (flags >> 24) & 7 == 2 or memory_size == 0:
+ continue
+ if file_size > memory_size or address < base or address + memory_size > base + size:
+ raise ValueError(f"segment {index} exceeds the modem reserved memory")
+ name = f"modem.b{index:02d}"
+ if file_size:
+ actual = (firmware_dir / name).stat().st_size
+ if actual != file_size:
+ raise ValueError(f"{name}: {actual} bytes, expected {file_size}")
+ total += actual
+ segments.append({"index": index, "address": address, "file_bytes": file_size,
+ "memory_bytes": memory_size})
+ if not segments or min(item["address"] for item in segments) != base:
+ raise ValueError("firmware has no segments starting at the modem memory base")
+ return {"modem_node": node_path, "memory_node": region["path"], "memory_base": base,
+ "memory_size": size, "pas_id": cell(props["qcom,pas-id"]), "program_headers": count,
+ "loadable_segments": len(segments), "file_bytes": total, "segments": segments}
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--dtb", type=pathlib.Path, required=True)
+ parser.add_argument("--firmware-dir", type=pathlib.Path, required=True)
+ parser.add_argument("--node", required=True, help="modem remoteproc node in this device's DTB")
+ args = parser.parse_args()
+ try:
+ print(json.dumps(inspect(args.dtb, args.firmware_dir, args.node), indent=2))
+ except (OSError, ValueError) as error:
+ parser.exit(1, f"modem_firmware: {error}\n")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/qmi-test/run-nas.sh b/tools/qmi-test/run-nas.sh
new file mode 100755
index 0000000..482b750
--- /dev/null
+++ b/tools/qmi-test/run-nas.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+set -eu
+root=$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)
+out="$root/xnu-deps/build/qmi-test/nas-test"
+mkdir -p "$(dirname "$out")"
+${CC:-cc} -std=c11 -O1 -g -Wall -Wextra -Werror -fsanitize=address,undefined \
+ -I"$root/xnu/files/bsd" -o "$out" \
+ "$root/tools/qmi-test/test_nas.c" "$root/xnu/files/bsd/dev/redwood_nas.c"
+"$out"
diff --git a/tools/qmi-test/test_nas.c b/tools/qmi-test/test_nas.c
new file mode 100644
index 0000000..7c3f24a
--- /dev/null
+++ b/tools/qmi-test/test_nas.c
@@ -0,0 +1,92 @@
+#include <assert.h>
+#include <stdio.h>
+#include <sys/redwood_nas.h>
+#include <sys/redwood_qrtr.h>
+
+/* Literal NAS frames using the public QMI TLV layout. */
+static const unsigned char registered[] = {
+ 2, 1, 0, 0x24, 0, 15, 0,
+ 2, 4, 0, 0, 0, 0, 0,
+ 1, 5, 0, 1, 2, 2, 2, 0
+};
+static const unsigned char disconnected[] = {
+ 4, 1, 0, 0x24, 0, 8, 0,
+ 1, 5, 0, 0, 3, 3, 0, 0
+};
+static const unsigned char lte_signal[] = {
+ 2, 2, 0, 0x4f, 0, 16, 0,
+ 2, 4, 0, 0, 0, 0, 0,
+ 0x14, 6, 0, 0xba, 0xf6, 0xa1, 0xff, 0x64, 0
+};
+static const unsigned char failed_signal[] = {
+ 2, 2, 0, 0x4f, 0, 16, 0,
+ 2, 4, 0, 1, 0, 1, 0,
+ 0x14, 6, 0, 0xba, 0xf6, 0xa1, 0xff, 0x64, 0
+};
+
+static redwood_qrtr_receive_t received;
+static redwood_qrtr_server_t server;
+static void *callback_context;
+static int requests, updates;
+
+static struct redwood_qrtr_port *open_port(redwood_qrtr_receive_t receive, redwood_qrtr_server_t on_server, void *context)
+{
+ received = receive; server = on_server; callback_context = context;
+ return (struct redwood_qrtr_port *)(uintptr_t)1;
+}
+static int send_packet(struct redwood_qrtr_port *port, uint32_t node, uint32_t peer,
+ const void *data, size_t length)
+{
+ const unsigned char *bytes = data;
+ assert(port && node == 7 && peer == 9 && length == 7 && bytes[0] == 0);
+ assert((requests == 0 && bytes[3] == 0x24) || (requests == 1 && bytes[3] == 0x4f));
+ requests++;
+ return 0;
+}
+static int lookup_port(struct redwood_qrtr_port *port, uint32_t service, uint32_t version, uint32_t instance)
+{
+ assert(port && service == 3 && version == 0 && instance == 0);
+ server(callback_context, true, 3, 1, 0, 7, 9);
+ return 0;
+}
+static void close_port(struct redwood_qrtr_port *port) { assert(port); }
+static void changed(void *context, const struct redwood_nas_state *state)
+{
+ assert(context && state);
+ updates++;
+}
+static const struct redwood_qrtr_ops transport = {
+ .open = open_port, .send = send_packet, .lookup = lookup_port, .close = close_port
+};
+
+int main(void)
+{
+ struct redwood_nas_state state = {0};
+ assert(redwood_nas_apply(&state, lte_signal, sizeof(lte_signal)) == 1);
+ assert(state.has_signal && !state.registered && state.bars == 0);
+ assert(redwood_nas_apply(&state, registered, sizeof(registered)) == 1);
+ assert(state.registered && state.bars == 0);
+ assert(redwood_nas_apply(&state, lte_signal, sizeof(lte_signal)) == 1);
+ assert(state.radio_interface == 8 && state.dbm == -95 && state.bars == 4);
+ assert(redwood_nas_apply(&state, failed_signal, sizeof(failed_signal)) == 0);
+ assert(state.bars == 4);
+ for (size_t n = 0; n < sizeof(lte_signal); n++) {
+ assert(redwood_nas_apply(&state, lte_signal, n) == -1);
+ assert(state.bars == 4);
+ }
+ assert(redwood_nas_apply(&state, disconnected, sizeof(disconnected)) == 1);
+ assert(!state.registered && !state.has_signal && state.bars == 0);
+ struct redwood_nas_client client;
+ assert(redwood_nas_client_open(&client, &transport, changed, &client) == 0);
+ assert(requests == 2);
+ received(callback_context, 7, 9, registered, sizeof(registered));
+ received(callback_context, 7, 9, lte_signal, sizeof(lte_signal));
+ assert(client.state.registered && client.state.bars == 4 && updates == 2);
+ received(callback_context, 8, 9, disconnected, sizeof(disconnected));
+ assert(client.state.registered && updates == 2);
+ server(callback_context, false, 3, 1, 0, 7, 9);
+ assert(!client.state.registered && client.state.bars == 0 && updates == 3);
+ redwood_nas_client_close(&client);
+ puts("nas-test: registration gates bars; QRTR lookup, polling and disconnect verified");
+ return 0;
+}
diff --git a/tools/qmi-test/test_qmi.c b/tools/qmi-test/test_qmi.c
index 1e432ff..9021950 100644
--- a/tools/qmi-test/test_qmi.c
+++ b/tools/qmi-test/test_qmi.c
@@ -944,6 +944,9 @@ test_helpers(void)
/* no router: no handle */
redwood_qmi_set_transport(NULL);
check(qmi_handle_init(&qmi, 8192, &handle_ops, handle_handlers) == -ENXIO && qmi.sock == NULL, "a handle is made with no router there");
+ check(redwood_qmi_handle_init_transport(&qmi, 8192, &handle_ops, handle_handlers, &router_ops) == 0 &&
+ qmi.sock != NULL, "an explicit transport cannot be used without the default router");
+ qmi_handle_release(&qmi);
redwood_qmi_set_transport(&router_ops);
memset(&seen, 0, sizeof(seen));
diff --git a/userland/build.sh b/userland/build.sh
index 41570fe..6388c9d 100755
--- a/userland/build.sh
+++ b/userland/build.sh
@@ -53,6 +53,8 @@
# kernel's adsp_step=N, bsd/dev/redwood_adsp.c, which runs before the root is
# mounted). Not beside GL=1: the two do not fit the 64 MB the loader takes with
# the kernel. adsp=1 needs none of it: the system image has the firmware in /fw.
+# MODEM=1 stage checked local MPSS firmware from ../fw-modem in system.apfs /fw.
+# This does not start the modem; the XNU MPSS driver is still required.
# system.apfs the system image, the root filesystem (the resident RAM disk, /dev/md0), an
# APFS container: the part of iOS 16.5 chosen by ../ios165/select-system.py at
# its real paths, the dyld shared cache, and src/stage0.c as /sbin/launchd (the
@@ -336,9 +338,10 @@ if [ -d $IOS/rootfs/usr/lib ]; then
cp $IOS/var.skel root/etc/var.skel # the directories init creates in /private/var
# time zones: the template's links go in a circle until the time zone daemon has run (/usr/share/zoneinfo
# points back into /var); point them at the data that ships with the system, and use this machine's zone
- zone=$(timedatectl show -p Timezone --value 2>/dev/null); zone=${zone:-UTC}
- sed -i -e 's|^l\t/usr/share/zoneinfo\t/private/var/db/timezone/zoneinfo$|l\t/usr/share/zoneinfo.default\t/private/var/db/timezone/zoneinfo|' \
+ zone=$(timedatectl show -p Timezone --value 2>/dev/null || true); zone=${zone:-UTC}
+ sed -i.bak -e 's|^l\t/usr/share/zoneinfo\t/private/var/db/timezone/zoneinfo$|l\t/usr/share/zoneinfo.default\t/private/var/db/timezone/zoneinfo|' \
-e "s|^l\t/var/db/timezone/zoneinfo/[^\t]*\t/private/var/db/timezone/localtime\$|l\t/var/db/timezone/zoneinfo/$zone\t/private/var/db/timezone/localtime|" root/etc/var.skel
+ rm -f root/etc/var.skel.bak
# ... and the files of ours it puts there (var/): those under mobile/ belong to the user mobile
cp -r var root/var
(cd var && find . -type f | sed 's|^\./||' | sort) | while read -r f; do
@@ -586,6 +589,21 @@ if [ -f $IOS/system.manifest ] && [ -f $CACHE/dyld_shared_cache_arm64 ] &&
mkdir -p sysroot/fw
cp ../fw-audio/adsp.mdt ../fw-audio/adsp.b0[2-9] ../fw-audio/adsp.b[1-9][0-9] sysroot/fw/
fi
+ # MPSS firmware is much too large for the small boot image. Keep it in the system APFS,
+ # only when explicitly requested, after checking every loadable segment against this DTB.
+ # This stages the files; a separate XNU MPSS driver is still required to start the modem.
+ if [ -n "$MODEM" ]; then
+ [ -f ../fw-modem/modem.mdt ] || { echo "build.sh: MODEM=1 needs ../fw-modem/modem.mdt" >&2; exit 1; }
+ ../tools/modem_firmware.py --dtb ../prebuilt/uefi/redwood.dtb --firmware-dir ../fw-modem \
+ --node /soc/qcom,mss@4080000 >/dev/null
+ mkdir -p sysroot/fw
+ cp ../fw-modem/modem.mdt ../fw-modem/modem.b[0-9][0-9] sysroot/fw/
+ # ... and the firmware partition's image/ (its mcfg configurations and the registry's *.jsn, pulled from
+ # the phone's /vendor/firmware_mnt/image into ../fw-modem/image), which the modem reads over TFTP (redwood_mpss.c)
+ if [ -d ../fw-modem/image ]; then
+ rm -rf sysroot/fw/image && cp -R ../fw-modem/image sysroot/fw/image
+ fi
+ fi
# ... and the speaker amplifiers' configuration, the phone's /vendor/etc/aw882xx_acf.bin (bsd/dev/redwood_speaker.c,
# for sound=3): Android's own register values, which are not in the repository either
if [ -f ../fw-audio/aw882xx_acf.bin ]; then
diff --git a/userland/src/inject_sb.c b/userland/src/inject_sb.c
index c739bf6..6b31ec4 100644
--- a/userland/src/inject_sb.c
+++ b/userland/src/inject_sb.c
@@ -1066,6 +1066,38 @@ redwood_springboard_init(void)
snapshot_format ? "as JPEG" : "NOT changed", original_lock_long_press ? "checked" : "NOT checked",
battery_lvl ? "from QG" : "NOT replaced", battery_st ? "from QG" : "NOT replaced",
poster_switcher ? "not offered" : "NOT withheld");
+ /*
+ * Cellular, a survey first (the modem's bars come from the kernel: sysctl kern.redwood.cellular_*,
+ * bsd/dev/redwood_mpss.c): where UIKit makes the status bar's cellular entry from the raw data,
+ * and with which signature -- said in the log with the unslid address, for the hook to come.
+ */
+ {
+ static const char *const classes[] = { "_UIStatusBarDataCellularEntry", "_UIStatusBarData", "_UIStatusBarDataEntry" };
+ const char *selector = "_cellularEntryWithData:signalStrengthEnabled:serviceEnabled:dataNetworkEnabled:serviceString:"
+ "serviceCrossfadeString:serviceBadgeString:serviceContentType:dataNetworkType:gsmSignalStrengthRaw:"
+ "gsmSignalStrengthBars:callForwardingType:lowDataModeActive:";
+
+ for (unsigned c = 0; c < sizeof(classes) / sizeof(classes[0]); c++) {
+ Class cls = objc_getClass(classes[c]);
+ Method plus = cls ? class_getClassMethod(cls, sel_registerName(selector)) : NULL;
+ Method minus = cls ? class_getInstanceMethod(cls, sel_registerName(selector)) : NULL;
+
+ if (plus) {
+ say("redwood: cellular: +[%s %s] at %#lx, types \"%s\"\n", classes[c], selector,
+ (uintptr_t)method_getImplementation(plus) - slide, method_getTypeEncoding(plus));
+ }
+ if (minus) {
+ say("redwood: cellular: -[%s %s] at %#lx, types \"%s\"\n", classes[c], selector,
+ (uintptr_t)method_getImplementation(minus) - slide, method_getTypeEncoding(minus));
+ }
+ if (cls && !plus && !minus) {
+ say("redwood: cellular: %s has no such method\n", classes[c]);
+ }
+ }
+ say("redwood: cellular: the kernel says up=%d registered=%d bars=%d dbm=%d radio=%d\n", sysctl_int("kern.redwood.cellular_up"),
+ sysctl_int("kern.redwood.cellular_registered"), sysctl_int("kern.redwood.cellular_bars"),
+ sysctl_int("kern.redwood.cellular_dbm"), sysctl_int("kern.redwood.cellular_radio"));
+ }
say("redwood: battery UI: shows%% %s, set-shows%% %s, bolt %s, chargePercent %s, setChargePercent %s, lockscreen ring %s\n",
original_battery_view_shows_percentage ? "ok" : "NO",
original_battery_view_set_shows_percentage ? "ok" : "NO",
diff --git a/xl b/xl
index 11a79d5..1a3849e 100755
--- a/xl
+++ b/xl
@@ -44,6 +44,8 @@
# when the last upload was not from this copy. The record starts empty -- an upload by a copy without
# this change leaves none -- and `run` says so rather than assuming.
cd "$(dirname "$(readlink -f "$0")")" || exit 1
+# On a Mac: flock, GNU timeout and stat, and Android's platform-tools, from tools/macos-bin (see its README)
+[ "$(uname)" = Darwin ] && PATH=$PWD/tools/macos-bin:$PWD/tools/macos-bin/platform-tools:$PATH
# the kernel and the loader built here, or else the ones that come with the repository
K=xnu-src/xnu-8796.121.2/BUILD/obj/DEVELOPMENT_ARM64_BCM2837/mach.development.bcm2837
[ -f $K ] || K=prebuilt/mach.development.bcm2837
diff --git a/xnu-build.sh b/xnu-build.sh
index 8f851bb..dc3c7c4 100755
--- a/xnu-build.sh
+++ b/xnu-build.sh
@@ -6,11 +6,15 @@ cd "$XNU_SRC" || exit 1
# does not see it change, so the linked kernel is thrown away when it has.
LX=$XNUDEPS/build/ath11k/out/ath11k.o
STAMP=BUILD/obj/redwood-objects
-if [ -f "$LX" ]; then now="$LX $(stat -c %Y "$LX")"; else LX= now=; fi
+mtime() { if [ "$XNU_HOST_OS" = Darwin ]; then stat -f %m "$1"; else stat -c %Y "$1"; fi; }
+if [ -f "$LX" ]; then now="$LX $(mtime "$LX")"; else LX= now=; fi
+# the build tools of SETUP/ are Apple's: on macOS they build as they are, on Linux with a compatibility header
+if [ "$XNU_HOST_OS" = Darwin ]; then HOSTCC=clang; else
+HOSTCC="clang -include $XNUDEPS/hostinc/hostcompat.h -I$XNUDEPS/hostinc -Wno-unused-command-line-argument -Wno-error=implicit-function-declaration"; fi
if [ -d BUILD/obj ] && [ "$(cat $STAMP 2>/dev/null)" != "$now" ]; then
rm -f BUILD/obj/*/mach.*.unstripped
echo "$now" > $STAMP
fi
exec make REDWOOD_OBJECTS="$LX" SDKROOT=iphoneos ARCH_CONFIGS=ARM64 MACHINE_CONFIGS=${MACHINE:-BCM2837} KERNEL_CONFIGS=DEVELOPMENT \
ARCH_STRING_FOR_CURRENT_MACHINE_CONFIG=arm64 BUILD_WERROR=0 BUILD_LTO=0 DO_CTFMERGE=0 \
- HOST_CC="clang -include $XNUDEPS/hostinc/hostcompat.h -I$XNUDEPS/hostinc -Wno-unused-command-line-argument -Wno-error=implicit-function-declaration" HOST_FLEX=flex HOST_BISON=bison HOST_GM4=m4 CC=xnu-clang CXX=xnu-clang++ LOGCOLORS=n "$@"
+ HOST_CC="$HOSTCC" HOST_FLEX=flex HOST_BISON=bison HOST_GM4=m4 CC=xnu-clang CXX=xnu-clang++ LOGCOLORS=n "$@"
diff --git a/xnu-deps/host/bin/xcrun b/xnu-deps/host/bin/xcrun
index c74c918..e29b378 100755
--- a/xnu-deps/host/bin/xcrun
+++ b/xnu-deps/host/bin/xcrun
@@ -1,6 +1,9 @@
#!/bin/bash
# Minimal xcrun replacement for building XNU on Linux
HB="$(cd "$(dirname "$0")" && pwd)"
+# On macOS the host side (the SDK and tools the build tools of SETUP/ are made with) is the real
+# xcrun's business; only the kernel's own SDK is ours.
+case " $* " in *" -sdk macosx"*) [ "$(uname)" = Darwin ] && exec /usr/bin/xcrun "$@" ;; esac
sdk=""; find=""; show=""
while [ $# -gt 0 ]; do case "$1" in
-sdk) sdk="$2"; shift 2;; -verbose) shift;;
@@ -18,6 +21,7 @@ case "$show" in
esac
if [ -n "$find" ]; then
for d in "$HB" /usr/bin; do [ -x "$d/$find" ] && { echo "$d/$find"; exit 0; }; done
+ [ "$(uname)" = Darwin ] && exec /usr/bin/xcrun -find "$find"
exit 1
fi
[ $# -gt 0 ] && exec "$@"
diff --git a/xnu-deps/hostinc/xnu_clang_compat.h b/xnu-deps/hostinc/xnu_clang_compat.h
index dd781d3..c95cd53 100644
--- a/xnu-deps/hostinc/xnu_clang_compat.h
+++ b/xnu-deps/hostinc/xnu_clang_compat.h
@@ -16,6 +16,8 @@
#else
#define __builtin_xnu_types_compatible(a, b) __builtin_types_compatible_p(a, b)
#endif
+#else
+#define XNU_NO_TYPE_BUILTINS 0 /* Apple's clang: the real ones */
#endif
#endif
#endif
diff --git a/xnu-deps/setup.sh b/xnu-deps/setup.sh
index 005dd06..ebf44f7 100755
--- a/xnu-deps/setup.sh
+++ b/xnu-deps/setup.sh
@@ -26,7 +26,7 @@ set -e -o pipefail
HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
source "$HERE/../xnu-env.sh"
cd "$XNUDEPS"
-JOBS=${JOBS:-$(nproc)}
+JOBS=${JOBS:-$(nproc 2>/dev/null || sysctl -n hw.ncpu)}
CCTOOLS=$THEOS/toolchain/linux/iphone/bin
IOS_SDK=$THEOS/sdks/iPhoneOS16.5.sdk
MACOS_SDK_URL=https://github.com/joseluisq/macosx-sdks/releases/download/13.3/MacOSX13.3.sdk.tar.xz
@@ -58,9 +58,46 @@ update() {
if cmp -s "$1.new" "$1"; then rm "$1.new"; else mv "$1.new" "$1"; fi
}
+# macOS: no theos, but Xcode has the same cctools and the SDK can be had from github.com/theos/sdks.
+# theos-mac/ is laid out like theos so that the rest of this script does not care.
+theos_mac() {
+ local tool
+ mkdir -p "$THEOS/toolchain/linux/iphone/bin" "$THEOS/sdks"
+ for tool in ld nm otool strip lipo libtool nmedit ctf_insert codesign_allocate install_name_tool; do
+ link "$(/usr/bin/xcrun -f $tool)" "$THEOS/toolchain/linux/iphone/bin/$tool"
+ done
+ if [ ! -d "$THEOS/sdks-git/iPhoneOS16.5.sdk/usr/include" ]; then
+ log "fetching iPhoneOS16.5.sdk from github.com/theos/sdks (about 220 MB)"
+ rm -rf "$THEOS/sdks-git"
+ git clone -q --depth 1 --filter=blob:none --sparse https://github.com/theos/sdks.git "$THEOS/sdks-git" &&
+ git -C "$THEOS/sdks-git" sparse-checkout set iPhoneOS16.5.sdk || die "cannot fetch the SDK"
+ fi
+ link ../sdks-git/iPhoneOS16.5.sdk "$THEOS/sdks/iPhoneOS16.5.sdk"
+}
+
+# linktree FROM TO: TO becomes a tree of links to the files of FROM (cp -rs, which BSD cp lacks)
+linktree() {
+ if [ "$XNU_HOST_OS" = Darwin ]; then
+ python3 - "$1" "$2" <<-'PY'
+ import os, sys
+ src, dst = sys.argv[1], sys.argv[2]
+ for root, dirs, files in os.walk(src):
+ rel = os.path.relpath(root, src)
+ out = dst if rel == '.' else os.path.join(dst, rel)
+ os.makedirs(out, exist_ok=True)
+ for name in files:
+ os.symlink(os.path.join(root, name), os.path.join(out, name))
+ PY
+ else
+ cp -rs "$1" "$2"
+ fi
+}
+
stage1() {
local missing= tool
- for tool in git curl tar xz make cc clang clang++ flex bison m4 perl python3 autoreconf automake; do
+ local tools="git curl tar xz make cc clang clang++ flex bison m4 perl python3 autoreconf automake"
+ [ "$XNU_HOST_OS" = Darwin ] && { theos_mac; tools="git curl tar make cc clang clang++ flex bison m4 perl python3"; }
+ for tool in $tools; do
command -v $tool >/dev/null || missing="$missing $tool"
done
[ -z "$missing" ] || die "not installed:$missing"
@@ -84,13 +121,21 @@ stage1() {
if [ ! -d MacOSX13.3.sdk/System/Library/Frameworks/Kernel.framework/Headers/DriverKit ]; then
log "downloading the macOS 13.3 SDK for its DriverKit headers"
curl -L --fail --retry 3 -s -o macos-sdk.tar.xz "$MACOS_SDK_URL" || die "download failed: $MACOS_SDK_URL"
- tar -xJf macos-sdk.tar.xz --wildcards '*MacOSX13.3.sdk/System/Library/Frameworks/Kernel.framework*' ||
+ local wild=--wildcards; [ "$XNU_HOST_OS" = Darwin ] && wild= # bsdtar: patterns as they are
+ tar -xJf macos-sdk.tar.xz $wild '*MacOSX13.3.sdk/System/Library/Frameworks/Kernel.framework*' ||
die "cannot unpack the macOS SDK"
rm -f macos-sdk.tar.xz
fi
# --- host tools ----------------------------------------------------------------------------
mkdir -p host/bin
+ if [ "$XNU_HOST_OS" = Darwin ]; then
+ # macOS has it all: mig (Xcode's; it takes $MIGCC and $MIGCOM from the kernel build), unifdef, csh
+ link "$(/usr/bin/xcrun -f mig)" host/bin/mig
+ link "$(/usr/bin/xcrun -f migcom)" host/bin/migcom
+ link /usr/bin/unifdef host/bin/unifdef
+ link /bin/tcsh host/bin/tcsh
+ fi
# mig: Apple's own does not build on glibc; this is a port that does
if [ ! -x host/bin/migcom ] || [ ! -x host/bin/mig ]; then
log "building mig"
@@ -120,7 +165,12 @@ stage1() {
for tool in nm otool strip lipo libtool nmedit ctf_insert codesign_allocate install_name_tool; do
link "$CCTOOLS/$tool" host/bin/$tool
done
- link "$CCTOOLS/ld" host/bin/ld64
+ if [ "$XNU_HOST_OS" = Darwin ]; then
+ # (not a link: Xcode's ld looks itself up by the name it was called by, and knows no "ld64")
+ printf '#!/bin/sh\nexec "%s" "$@"\n' "$(/usr/bin/xcrun -f ld)" > host/bin/ld64.new && chmod +x host/bin/ld64.new && mv host/bin/ld64.new host/bin/ld64
+ else
+ link "$CCTOOLS/ld" host/bin/ld64
+ fi
link xnu-clang host/bin/xnu-clang++
# --- the SDK -------------------------------------------------------------------------------
@@ -129,7 +179,7 @@ stage1() {
if [ ! -d "$XNU_SDK_PATH/usr/include" ]; then
log "making the SDK: links to $IOS_SDK"
rm -rf "$XNU_SDK_PATH"
- cp -rs "$(cd "$IOS_SDK" && pwd)" "$XNU_SDK_PATH"
+ linktree "$(cd "$IOS_SDK" && pwd -P)" "$XNU_SDK_PATH"
fi
local sdk_local=$XNU_SDK_PATH/usr/local
mkdir -p $sdk_local/include/kernel/os $sdk_local/include/os $sdk_local/lib/kernel $sdk_local/libexec
diff --git a/xnu-env.sh b/xnu-env.sh
index f2c6b83..4a2179d 100644
--- a/xnu-env.sh
+++ b/xnu-env.sh
@@ -3,7 +3,14 @@
# iOS SDK; XNU_SRC is the kernel tree (xnu/apply.sh creates it).
DARWIN_REDWOOD=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
export DARWIN_REDWOOD
-export THEOS=${THEOS:-$HOME/theos}
+export XNU_HOST_OS=$(uname)
+# On macOS there is no theos: xnu-deps/setup.sh lays out xnu-deps/theos-mac like one (links to the
+# cctools of Xcode and to the iPhoneOS 16.5 SDK of github.com/theos/sdks).
+if [ "$XNU_HOST_OS" = Darwin ]; then
+ export THEOS=${THEOS:-$DARWIN_REDWOOD/xnu-deps/theos-mac}
+else
+ export THEOS=${THEOS:-$HOME/theos}
+fi
export XNU_SRC=${XNU_SRC:-$DARWIN_REDWOOD/xnu-src/xnu-8796.121.2}
export XNUDEPS=$DARWIN_REDWOOD/xnu-deps
export PATH=$XNUDEPS/host/bin:$PATH
diff --git a/xnu/README.md b/xnu/README.md
index e3caa7b..da37ad0 100644
--- a/xnu/README.md
+++ b/xnu/README.md
@@ -6,7 +6,7 @@
- `files/` — новые файлы по их путям в дереве XNU (37 файлов);
- `apply.sh` — берёт исходники у Apple и накладывает всё это; `export.sh` — обратное действие.
-Собирается на Linux стоковым clang, без macOS и Xcode.
+Собирается на Linux стоковым clang, без macOS и Xcode, — и на macOS с Xcode (ниже, «Сборка на macOS»).
## Сборка
@@ -41,6 +41,16 @@ xnu-deps/setup.sh # ~30 с: исходники зависимостей,
`xnu-deps/gen-iig-stubs.py xnu-deps/iig-impl/undefined-all.txt xnu-deps/iig-impl qemu/kernel.syms`, затем `rm -rf BUILD`.
- Для `qemu/sym.py` после пересборки: `nm -n …/mach.development.bcm2837.unstripped > qemu/kernel.syms`.
+## Сборка на macOS
+
+Те же три команды. theos не нужен: `xnu-deps/setup.sh` на Darwin сам складывает `xnu-deps/theos-mac`
+(ссылки на cctools из Xcode, iPhoneOS16.5.sdk из github.com/theos/sdks частичным клоном, ~220 МБ)
+и берёт mig, unifdef и csh системные. Нужны Xcode (проверено с 26.5, Apple clang 21) и MacPorts с
+coreutils (`gstat`, `gtimeout` — для `xl`). `host/bin/ld64` там — обёртка, а не ссылка: ld из Xcode
+ищет себя по имени, под которым вызван. Ядро, собранное так, загружается на телефоне и возвращается в
+загрузчик как и собранное на Linux. `xl` на Mac берёт `flock`, `timeout`, `stat` и platform-tools из
+`tools/macos-bin` (см. README там).
+
## Работа с деревом
Правки делаются прямо в `xnu-src/xnu-8796.121.2` (обычное git-дерево Apple на ветке `redwood`). Перед коммитом
@@ -79,6 +89,8 @@ xnu-deps/setup.sh # ~30 с: исходники зависимостей,
| `bsd/dev/redwood_adreno.c`, `iokit/Kernel/RedwoodGPU.cpp` | GPU телефона (Adreno 642L): пошаговое включение (`adreno=N`: питание, GMU, zap-шейдер, микрокод), буферы и очередь команд; клиент IOKit с подмножеством msm DRM для Mesa |
| `bsd/dev/redwood_wifi.c`, `bsd/sys/redwood_qrtr.h` | Wi-Fi телефона (WCN6750), начало пути (`wifi=N`): общая память процессоров Qualcomm (SMEM: чтение и выделение под её замком), сигнальные биты между нами и процессором Wi-Fi (SMP2P, обе стороны), почтовый ящик процессора питания, загрузка прошивки процессора Wi-Fi, его запуск через TrustZone и остановка по просьбе, GLINK (очереди в общей памяти, канал IPCRTR) и QRTR поверх него; `redwood_qrtr.h` — порт датаграмм для остального ядра (открыть, послать, найти службу, закрыть) |
| `bsd/dev/redwood_qmi.c`, `bsd/sys/redwood_qmi.h` | QMI — протокол сообщений служб на процессорах Qualcomm: кодирование по таблицам, запросы с ответами, оповещения; интерфейс как у Linux (`qmi_*`), чтобы `qmi.c` из ath11k собирался без правок, реализация своя; проверяется на компьютере (`tools/qmi-test/run.sh`) |
+| `bsd/dev/redwood_nas.c`, `bsd/sys/redwood_nas.h` | клиент QMI NAS поверх заданного QRTR: регистрация в сети и уровень сигнала; не показывает сигнал без регистрации. Проверка на компьютере: `tools/qmi-test/run-nas.sh`. Для работы на телефоне требуются отдельный транспорт MPSS и подключение к CoreTelephony |
+| `bsd/dev/redwood_mpss.c` | модем (MPSS): путь аудио-DSP с числами модема (`modem=N`, `modem_step=N`): прошивка из `/fw`, TrustZone (образ 4), шины CX/MSS, SMP2P, GLINK, QRTR с портами для QMI; наша служба RMTFS (QMI 14: файловые системы модема из разделов UFS, в памяти, без записи на флеш) и сервис имён QRTR; DMS «online» и клиент NAS → `sysctl kern.redwood.cellular_*`. Проверено: модем стартует и отвечает; RMTFS ещё не проверен (docs/cellular.md) |
| `bsd/dev/redwood_wlfw.c` | временная замена `qmi.c` из ath11k: разговор со службой прошивки Wi-Fi вручную, в его порядке и его сообщениями (шаг 15 `wifi=N`); описания сообщений берутся из исходников ath11k при сборке (`tools/mk-wlfw.sh`) и в git не попадают |
| `bsd/dev/redwood_audio.c`, `redwood_virtio_sound.c`, `redwood_speaker.c` | звук: `/dev/redwood-audio` (куда наше устройство вывода в mediaserverd пишет то, что играет iOS), запись его в память, virtio-sound для QEMU; первый взгляд на усилители динамиков телефона |
| `bsd/pthread/pthread_kext_static.c` | pthread kext, слинкованный в ядро |
diff --git a/xnu/export.sh b/xnu/export.sh
index 0940781..8d3868c 100755
--- a/xnu/export.sh
+++ b/xnu/export.sh
@@ -26,7 +26,7 @@ mkdir "$HERE/files.new"
while IFS= read -r f; do
[ -f "$f" ] || [ -L "$f" ] || continue # added in a commit, deleted since
mkdir -p "$HERE/files.new/$(dirname "$f")"
- cp -P --preserve=mode "$f" "$HERE/files.new/$f"
+ if [ "$(uname)" = Darwin ]; then cp -pP "$f" "$HERE/files.new/$f"; else cp -P --preserve=mode "$f" "$HERE/files.new/$f"; fi
done < "$HERE/files.list"
rm -rf "$HERE/files" "$HERE/files.list"
mv "$HERE/files.new" "$HERE/files"
diff --git a/xnu/files/bsd/dev/redwood_input.c b/xnu/files/bsd/dev/redwood_input.c
index 8076cf0..e5ec5cb 100644
--- a/xnu/files/bsd/dev/redwood_input.c
+++ b/xnu/files/bsd/dev/redwood_input.c
@@ -224,4 +224,5 @@ redwood_input_init(void)
redwood_adreno_init();
redwood_wifi_init();
redwood_adsp_init();
+ redwood_mpss_init();
}
diff --git a/xnu/files/bsd/dev/redwood_mpss.c b/xnu/files/bsd/dev/redwood_mpss.c
new file mode 100644
index 0000000..086900f
--- /dev/null
+++ b/xnu/files/bsd/dev/redwood_mpss.c
@@ -0,0 +1,5438 @@
+/*
+ * The modem of the POCO X5 Pro (redwood), started, and asked about the network: step 1 of
+ * cellular. Every address below belongs to this phone, and nothing here is touched unless a
+ * boot-arg asks for it.
+ *
+ * The modem ("MPSS", Android's "modem") is a Hexagon of its own inside the SoC, exactly as the
+ * Wi-Fi's and the audio's are (redwood_wifi.c, redwood_adsp.c): it stays in reset until somebody
+ * loads its signed image -- modem.mdt and modem.b00 to modem.b29 from /vendor/firmware_mnt/image
+ * of its Android, 85 MB, in the system image under /fw (MODEM=1 SYSTEM=1 userland/build.sh, from
+ * fw-modem/) -- into the 246 MB reserved for it and asks TrustZone to authenticate and start it.
+ * From there everything is conversation through the shared memory: SMP2P for "up", "given up",
+ * "stopped"; GLINK for channels; QRTR on the channel "IPCRTR" for the modem's services, which
+ * speak QMI (bsd/dev/redwood_qmi.c knows the protocol; here only two of its messages are made by
+ * hand): the device management service is asked to put the modem online, and the network access
+ * service what network it is on and how strong the signal is (bsd/dev/redwood_nas.c). What this
+ * learns goes to sysctl kern.redwood.cellular_* for the rest of the system; calls, SMS and data
+ * are for the steps after this one.
+ *
+ * The way is the modem's, which is the Wi-Fi DSP's: redwood_adsp.c's comments say why each
+ * thing is done so; this file is that one with the modem's numbers (dt: qcom,mss@4080000,
+ * qcom,smp2p-modem, qcom,glink/modem, reserved-memory mpss@8b800000):
+ * - processor 1 in the shared memory; its SMP2P items 435 (its) and 428 (ours); its last words
+ * in item 421; GLINK's three items in our partition with it
+ * - its doorbells are IPCC client 2 (MPSS): signal 0 GLINK, signal 2 SMP2P
+ * - TrustZone knows it as image 4; its memory is 0x8b800000, 246 MB, of which the image takes
+ * the first 85 (TrustZone is told of all of it: the modem uses the rest)
+ * - its rails are CX and MSS (cx.lvl and mss.lvl), asked for at level 384 while it boots
+ * - the always-on processor is told of it as "modem"
+ * - the channel is IPCRTR only; the modem's others (DS, DATA*, glink_ssr) are left unanswered
+ * - the ports of QRTR are the Wi-Fi driver's (sys/redwood_qrtr.h): what the NAS client needs
+ * The modem and the Wi-Fi DSP may run at the same time: the mailbox to the always-on
+ * processor and the lock for allocating in the shared memory are shared, each behind a lock.
+ *
+ * Boot-arg modem_step=N performs steps 1 to N and stops; modem=1 is all of them, on a thread
+ * beside the system's start, and the modem stays on, its network state asked every two seconds.
+ *
+ * 1 reads the shared memory: our partition with the modem, its items, whether it has run
+ * 2 reads the firmware on the root (or in the small image): is it whole, does it fit
+ * 3 writes the always-on processor's mailbox open, unless it is
+ * 4 asks TrustZone to take the image's header (image 4) and where the image will be; a
+ * modem that a kernel before this one left running is asked to stop and shut down first
+ * 5 writes the image into the modem's memory, and reads it back
+ * 6 writes our SMP2P item (428) into the shared memory, and our two marks of GLINK to zero
+ * 7 asks RPMh for the modem's rails, writes "modem: on" to the always-on processor, asks
+ * TrustZone to authenticate and start the modem, and watches for its first words
+ * 8 reads GLINK's queues
+ * 9 writes our side of GLINK: the versions, the channel "IPCRTR" opened both ways, our intents;
+ * from here a thread of ours listens to the modem
+ * 10 talks QRTR on that channel: hello both ways, and the services the modem tells of
+ * 11 talks QMI: DMS "operating mode online" (modem_online=0: not), then NAS "serving system"
+ * and "signal info" every second for modem_nas_wait seconds (default 30), or until
+ * the modem says it is registered and how strong the signal is
+ *
+ * What is started is stopped: when the steps are over (unless modem_keep=1, and then when the run
+ * ends), and on a panic's way back to the loader. More boot-args: modem_new=K, modem_pause=MS,
+ * modem_wait=S (step 7's watch, default 10), modem_listen=S (step 10, default 3), modem_again=1,
+ * modem_return=1, as the modem's.
+ *
+ * Nothing here writes a clock or changes a pin. One request goes to RPMh (the two rails), as
+ * Linux's remoteproc makes it; the SIM is not touched, no call is made, no data session opened.
+ */
+#include <kern/clock.h>
+#include <kern/kalloc.h>
+#include <kern/locks.h>
+#include <kern/thread.h>
+#include <libkern/libkern.h>
+#include <os/atomic_private.h>
+#include <mach/mach_types.h>
+#include <mach/vm_param.h>
+#include <vm/memory_types.h>
+#include <pexpert/pexpert.h>
+#include <sys/errno.h>
+#include <sys/kauth.h>
+#include <sys/stat.h>
+#include <sys/systm.h>
+#include <sys/vnode.h>
+#include <sys/vnode_internal.h>
+#include <vm/vm_kern.h>
+#include <sys/redwood_input.h>
+#include <sys/redwood_nas.h>
+#include <sys/redwood_qrtr.h>
+#include <sys/sysctl.h>
+
+extern vm_offset_t ml_io_map(vm_offset_t phys_addr, vm_size_t size);
+extern vm_offset_t ml_io_map_unmappable(vm_offset_t phys_addr, vm_size_t size, unsigned int flags);
+extern void ml_io_unmap(vm_offset_t addr, vm_size_t sz);
+extern uint64_t kvtophys(vm_offset_t va);
+extern void loader_return(void);
+extern kern_return_t thread_terminate(thread_t thread);
+extern vm_offset_t pe_arm_get_soc_base_phys(void);
+extern dev_t mdevlookup(int devid);
+extern int mdevgetkernelrange(dev_t dev, vm_offset_t *base, uint64_t *size);
+extern void FlushPoC_DcacheRegion(vm_offset_t va, size_t length); /* osfmk/arm64/caches_asm.s */
+
+/* What the Wi-Fi's DSP and this one both use (redwood_wifi.c). */
+extern lck_mtx_t redwood_aop_mailbox_lock;
+extern lck_mtx_t redwood_smem_alloc_lock;
+
+static LCK_GRP_DECLARE(mpss_lock_group, "redwood-mpss");
+
+/* the modem's file systems, served from here (rmtfs, below) */
+static bool rmtfs_buffer_make(void);
+static bool ipa_firmware_load(void);
+static void ipa_firmware_stop(void);
+static bool ipa_loaded; /* the IPA's firmware runs (ipa_firmware_load()) */
+static uint32_t ipa_mem_offset; /* where the IPA's local memory begins, as its register says */
+static bool rmtfs_share(bool in);
+static bool rmtfs_start(void);
+static void rmtfs_stop(void);
+
+#define STEPS 11
+#define REDWOOD_SOC_BASE 0x00400000ul /* where the loader says the phone's peripherals begin */
+
+static uint32_t last_step, pause_ms = 2200, listen_seconds = 3;
+static uint32_t step; /* the one being performed */
+static uint32_t new_from; /* the first of the steps that have not been seen to work: they wait */
+static char last_write[96] = "nothing written";
+static bool leaving; /* on the way back to the loader after a panic: no lock is taken, nothing sleeps */
+
+/*
+ * Every line goes to the log the loader reads from another core as well (as in redwood_wifi.c);
+ * with modem=1 only the few that say how it went (tell()), the rest to the kernel's own log.
+ */
+extern void loader_log_putc(char c);
+
+static LCK_MTX_DECLARE(say_lock, &mpss_lock_group);
+static bool quiet;
+
+static void
+say_line(const char *line, bool aloud)
+{
+ if (leaving) {
+ for (const char *c = line; *c != 0; c++) {
+ loader_log_putc(*c);
+ }
+ return;
+ }
+ lck_mtx_lock(&say_lock);
+ if (!quiet || aloud) {
+ for (const char *c = line; *c != 0; c++) {
+ loader_log_putc(*c);
+ }
+ }
+ printf("%s", line);
+ lck_mtx_unlock(&say_lock);
+}
+
+static int say(const char *format, ...) __printflike(1, 2);
+static int tell(const char *format, ...) __printflike(1, 2);
+
+static int
+say(const char *format, ...)
+{
+ char line[400];
+ va_list list;
+ int length;
+
+ va_start(list, format);
+ length = vsnprintf(line, sizeof(line), format, list);
+ va_end(list);
+ say_line(line, false);
+ return length;
+}
+
+static int
+tell(const char *format, ...)
+{
+ char line[400];
+ va_list list;
+ int length;
+
+ va_start(list, format);
+ length = vsnprintf(line, sizeof(line), format, list);
+ va_end(list);
+ say_line(line, true);
+ return length;
+}
+#define printf say
+
+static void
+sleep_us(uint32_t us)
+{
+ delay_for_interval(us, NSEC_PER_USEC);
+}
+
+static void
+sleep_ms(uint32_t ms)
+{
+ delay_for_interval(ms, NSEC_PER_MSEC);
+}
+
+static uint64_t
+now_us(void)
+{
+ uint64_t nanoseconds;
+
+ absolutetime_to_nanoseconds(mach_absolute_time(), &nanoseconds);
+ return nanoseconds / 1000;
+}
+
+/* Until every write to memory so far has been carried out. */
+static void
+settle(void)
+{
+ __asm__ volatile ("dsb sy");
+}
+
+static bool
+option(const char *name)
+{
+ uint32_t value = 0;
+
+ return PE_parse_boot_argn(name, &value, sizeof(value)) && value != 0;
+}
+
+/* ---- register access (as in redwood_wifi.c) ------------------------------------------------------- */
+
+static uint32_t __attribute__((noinline))
+rd(vm_offset_t address)
+{
+ uint32_t value;
+
+ __asm__ volatile ("ldr %w0, [%1]" : "=r" (value) : "r" (address) : "memory");
+ return value;
+}
+
+static void __attribute__((noinline))
+wr(vm_offset_t address, uint32_t value)
+{
+ __asm__ volatile ("str %w0, [%1]" : : "r" (value), "r" (address) : "memory");
+}
+
+struct block {
+ const char *name;
+ uint32_t phys, size;
+ vm_offset_t base;
+};
+
+static void
+need(struct block *b)
+{
+ if (b->base == 0) {
+ b->base = ml_io_map(b->phys, b->size);
+ }
+}
+
+/* Time for the loader to note what has just been said, where it is not yet known to be harmless. */
+static void
+hold(void)
+{
+ if (step >= new_from && pause_ms && !leaving) {
+ sleep_ms(pause_ms);
+ }
+}
+
+static void
+announce(const char *what)
+{
+ printf("mpss: step %u: %s\n", step, what);
+ hold();
+}
+
+static uint32_t
+get(struct block *b, const char *name, uint32_t offset)
+{
+ uint32_t value;
+
+ need(b);
+ value = rd(b->base + offset);
+ printf("mpss: %u: %s %s (%#x) = %#x\n", step, b->name, name, b->phys + offset, value);
+ return value;
+}
+
+static void
+put(struct block *b, const char *name, uint32_t offset, uint32_t value)
+{
+ need(b);
+ printf("mpss: %u: about to write %s %s at %#x: %#x\n", step, b->name, name, b->phys + offset, value);
+ snprintf(last_write, sizeof(last_write), "step %u wrote %#x to %s %s at %#x", step, value, b->name, name, b->phys + offset);
+ wr(b->base + offset, value);
+}
+
+/* ---- the shared memory (redwood_wifi.c, step 1) --------------------------------------------------- */
+
+#define SMEM_PHYS 0x80900000UL
+#define SMEM_SIZE 0x00200000u
+#define SMEM_US 0
+#define SMEM_MPSS 1 /* dt: qcom,remote-pid of qcom,smp2p-modem and qcom,glink/modem */
+#define SMEM_GLOBAL 0xfffe
+#define SMEM_CANARY 0xa5a5
+
+struct smem_table_entry {
+ uint32_t offset, size, flags;
+ uint16_t host0, host1;
+ uint32_t cacheline, reserved[7];
+};
+struct smem_table {
+ uint8_t magic[4];
+ uint32_t version, count, reserved[5];
+};
+struct smem_partition {
+ uint8_t magic[4];
+ uint16_t host0, host1;
+ uint32_t size, free_uncached, free_cached, reserved[3];
+};
+struct smem_entry {
+ uint16_t canary, item;
+ uint32_t size;
+ uint16_t padding_data, padding_header;
+ uint32_t reserved;
+};
+struct smem_item {
+ uint32_t item, offset, size;
+};
+
+/*
+ * The DSP's items (dt: qcom,smem of qcom,smp2p-modem, <435 428>: the item the DSP writes first, then
+ * ours -- the Wi-Fi's <617 616> are in the same order; taken the other way round, the DSP made
+ * our item its own and never said "up" --, and qcom,smem-id of qcom,mss@4080000).
+ */
+#define ITEM_SMP2P_OURS 428
+#define ITEM_SMP2P_THEIRS 435
+#define ITEM_CRASH_REASON 421
+
+static const volatile uint8_t *smem;
+
+static bool
+smem_peek(uint64_t offset, void *to, uint32_t size)
+{
+ if (smem == NULL) {
+ smem = (const volatile uint8_t *)ml_io_map_unmappable(SMEM_PHYS, SMEM_SIZE, VM_WIMG_WCOMB);
+ }
+ if (offset + size > SMEM_SIZE) {
+ return false;
+ }
+ for (uint32_t i = 0; i < size; i++) {
+ ((uint8_t *)to)[i] = smem[offset + i];
+ }
+ return true;
+}
+
+static void
+smem_poke(uint32_t offset, uint32_t value)
+{
+ wr((vm_offset_t)smem + offset, value);
+}
+
+static uint32_t
+smem_word(uint32_t offset)
+{
+ return rd((vm_offset_t)smem + offset);
+}
+
+static uint32_t
+smem_partition(uint16_t one, uint16_t other, uint32_t *cacheline, uint32_t *size)
+{
+ struct smem_table table;
+ struct smem_table_entry entry;
+ struct smem_partition header;
+
+ if (!smem_peek(SMEM_SIZE - 0x1000, &table, sizeof(table)) || memcmp(table.magic, "$TOC", 4) != 0 || table.version != 1 ||
+ table.count > 64) {
+ return 0;
+ }
+ for (uint32_t i = 0; i < table.count; i++) {
+ if (!smem_peek(SMEM_SIZE - 0x1000 + sizeof(table) + i * sizeof(entry), &entry, sizeof(entry)) || entry.offset == 0 ||
+ entry.size == 0) {
+ continue;
+ }
+ if (!((entry.host0 == one && entry.host1 == other) || (entry.host0 == other && entry.host1 == one))) {
+ continue;
+ }
+ if ((uint64_t)entry.offset + entry.size > SMEM_SIZE - 0x1000 || !smem_peek(entry.offset, &header, sizeof(header)) ||
+ memcmp(header.magic, "$PRT", 4) != 0 || header.size != entry.size) {
+ return 0;
+ }
+ *cacheline = entry.cacheline;
+ *size = entry.size;
+ return entry.offset;
+ }
+ return 0;
+}
+
+static unsigned
+smem_items(uint32_t partition, uint32_t cacheline, struct smem_item *found, unsigned max)
+{
+ struct smem_partition header;
+ struct smem_entry entry;
+ uint32_t at, step_back;
+ unsigned count = 0;
+
+ if (partition == 0 || !smem_peek(partition, &header, sizeof(header)) || header.free_uncached > header.size ||
+ header.free_cached > header.size) {
+ return 0;
+ }
+ for (at = sizeof(header); at + sizeof(entry) <= header.free_uncached;) {
+ if (!smem_peek(partition + at, &entry, sizeof(entry)) || entry.canary != SMEM_CANARY || entry.size > header.size) {
+ break;
+ }
+ if (count < max) {
+ found[count].item = entry.item;
+ found[count].offset = partition + at + sizeof(entry) + entry.padding_header;
+ found[count].size = entry.size - entry.padding_data;
+ }
+ count++;
+ at += sizeof(entry) + entry.padding_header + entry.size;
+ }
+ if (cacheline == 0 || (cacheline & (cacheline - 1))) {
+ return count;
+ }
+ step_back = ((uint32_t)sizeof(entry) + cacheline - 1) & ~(cacheline - 1);
+ for (at = header.size - step_back; at > header.free_cached && at < header.size;) {
+ if (!smem_peek(partition + at, &entry, sizeof(entry)) || entry.canary != SMEM_CANARY || entry.size > at) {
+ break;
+ }
+ if (count < max) {
+ found[count].item = entry.item;
+ found[count].offset = partition + at - entry.size;
+ found[count].size = entry.size - entry.padding_data;
+ }
+ count++;
+ if (entry.size + step_back > at) {
+ break;
+ }
+ at -= entry.size + step_back;
+ }
+ return count;
+}
+
+/* An item the DSP and we share: in the partition of the two, or else in the global one. Where it is; 0: nowhere. */
+static uint32_t
+smem_item(uint32_t item, uint32_t *size)
+{
+ static struct smem_item found[256];
+ static const uint16_t hosts[2][2] = { { SMEM_US, SMEM_MPSS }, { SMEM_GLOBAL, SMEM_GLOBAL } };
+
+ for (unsigned h = 0; h < 2; h++) {
+ uint32_t cacheline = 0, partition_size = 0;
+ uint32_t partition = smem_partition(hosts[h][0], hosts[h][1], &cacheline, &partition_size);
+ unsigned count = smem_items(partition, cacheline, found, 256);
+
+ for (unsigned i = 0; i < count && i < 256; i++) {
+ if (found[i].item == item) {
+ *size = found[i].size;
+ return found[i].offset;
+ }
+ }
+ }
+ return 0;
+}
+
+static void
+show_partition(void)
+{
+ static struct smem_item found[256];
+ struct smem_partition header;
+ uint32_t cacheline = 0, size = 0, partition = smem_partition(SMEM_US, SMEM_MPSS, &cacheline, &size);
+ unsigned count;
+ char line[320];
+
+ if (partition == 0 || !smem_peek(partition, &header, sizeof(header))) {
+ printf("mpss: %u: there is no partition for us and the modem (0 and 1)\n", step);
+ return;
+ }
+ count = smem_items(partition, cacheline, found, 256);
+ printf("mpss: %u: the partition for us and the modem (0 and 1): at %#lx, %#x bytes, of which %#x free; %u items\n", step,
+ SMEM_PHYS + partition, size, header.free_cached - header.free_uncached, count);
+ for (unsigned i = 0; i < count && i < 256;) {
+ int at = 0;
+
+ for (unsigned n = 0; n < 12 && i < count && i < 256; n++, i++) {
+ at += snprintf(line + at, sizeof(line) - (size_t)at, " %u (%u)", found[i].item, found[i].size);
+ }
+ printf("mpss: %u: items (bytes):%s\n", step, line);
+ }
+}
+
+/* ---- SMP2P ---------------------------------------------------------------------------------------- */
+
+struct smp2p {
+ uint32_t magic;
+ uint32_t version;
+ uint16_t local, remote, total, valid;
+ uint32_t flags;
+ struct {
+ char name[16];
+ uint32_t value;
+ } entry[16];
+};
+#define SMP2P_MAGIC 0x504d5324
+#define SMP2P_VERSION 1
+#define SMP2P_RESTART_ACKS 1u
+#define SMP2P_RESTART_DONE (1u << 0)
+#define SMP2P_RESTART_ACK (1u << 1)
+#define SMP2P_AT_VERSION 4
+#define SMP2P_AT_PROCESSORS 8
+#define SMP2P_AT_COUNTS 12
+#define SMP2P_AT_FLAGS 16
+#define SMP2P_AT_NAME(i) (20 + 20 * (i))
+#define SMP2P_AT_VALUE(i) (36 + 20 * (i))
+
+/* The bits of the DSP's entry "slave-kernel" (dt: the interrupts of qcom,lpass@3700000). */
+#define DSP_FATAL (1u << 0)
+#define DSP_READY (1u << 1)
+#define DSP_HANDOVER (1u << 2)
+#define DSP_STOP_ACK (1u << 3)
+#define DSP_SHUTDOWN_ACK (1u << 7)
+
+/* Our entry (dt: the child "master-kernel" of qcom,smp2p-adsp; "qcom,force-stop" of qcom,lpass@3700000). */
+#define OUR_ENTRY_STOP "master-kernel"
+#define DSP_STOP (1u << 0)
+
+static bool
+dsp_words(struct smp2p *words)
+{
+ static uint32_t at;
+ uint32_t size = 0;
+
+ if (at == 0) {
+ at = smem_item(ITEM_SMP2P_THEIRS, &size);
+ if (at == 0 || size < sizeof(*words)) {
+ at = 0;
+ return false;
+ }
+ }
+ return smem_peek(at, words, sizeof(*words)) && words->magic == SMP2P_MAGIC;
+}
+
+static int
+smp2p_entry(const struct smp2p *words, const char *name)
+{
+ for (unsigned i = 0; i < words->valid && i < 16; i++) {
+ if (strncmp(words->entry[i].name, name, sizeof(words->entry[i].name)) == 0) {
+ return (int)i;
+ }
+ }
+ return -1;
+}
+
+static uint32_t
+dsp_state(const struct smp2p *words)
+{
+ int entry = smp2p_entry(words, "slave-kernel");
+
+ return entry < 0 ? 0 : words->entry[entry].value;
+}
+
+static void
+show_words(const char *prefix, const struct smp2p *words)
+{
+ char line[320];
+ int at = 0;
+
+ line[0] = 0;
+ for (unsigned i = 0; i < words->valid && i < 16 && at < (int)sizeof(line) - 40; i++) {
+ at += snprintf(line + at, sizeof(line) - (size_t)at, " %.16s = %#x;", words->entry[i].name, words->entry[i].value);
+ }
+ printf("mpss: %u: %s: SMP2P version %u, features %#x, from processor %u to %u, flags %#x, %u of %u entries:%s\n", step, prefix,
+ words->version & 0xff, words->version >> 8, words->local, words->remote, words->flags, words->valid, words->total, line);
+}
+
+static void
+show_crash_reason(void)
+{
+ char text[257];
+ uint32_t size = 0, at = smem_item(ITEM_CRASH_REASON, &size);
+
+ if (at == 0 || size == 0) {
+ printf("mpss: %u: item %u, the DSP's last words: not there\n", step, ITEM_CRASH_REASON);
+ return;
+ }
+ bzero(text, sizeof(text));
+ smem_peek(at, text, size < sizeof(text) - 1 ? size : sizeof(text) - 1);
+ if (text[0] == 0) {
+ printf("mpss: %u: item %u, the DSP's last words (%u bytes): none\n", step, ITEM_CRASH_REASON, size);
+ } else {
+ printf("mpss: %u: THE MODEM'S LAST WORDS (item %u, %u bytes): \"%s\"\n", step, ITEM_CRASH_REASON, size, text);
+ }
+}
+
+/* ---- the doorbells, and the mailbox to the always-on processor (redwood_wifi.c, steps 2 and 7) ------ */
+
+#define QMP_MAGIC 0x00
+#define QMP_MAGIC_VALUE 0x4d41494c
+#define QMP_VERSION 0x04
+#define QMP_THEIR_LINK 0x0c
+#define QMP_THEIR_LINK_ACK 0x10
+#define QMP_THEIR_CHANNEL 0x14
+#define QMP_THEIR_CHANNEL_ACK 0x18
+#define QMP_OUR_LINK 0x24
+#define QMP_OUR_LINK_ACK 0x28
+#define QMP_OUR_CHANNEL 0x2c
+#define QMP_OUR_CHANNEL_ACK 0x30
+#define QMP_OUR_BOX_SIZE 0x34
+#define QMP_OUR_BOX 0x38
+#define QMP_UP 0x0000ffffu
+#define QMP_DOWN 0xffff0000u
+#define QMP_MESSAGE 64
+
+static struct block aoss = { "mailbox", 0x0c300000, 0x400 };
+
+#define IPCC_SEND 0x0c
+#define IPCC_RECEIVED 0x10
+#define IPCC_ENABLE 0x14
+#define IPCC_DISABLE 0x18
+#define IPCC_CLEAR 0x1c
+#define IPCC_NOBODY 0xffffffffu
+#define IPCC_AOP 0u
+#define IPCC_MPSS 2u /* dt: mboxes of qcom,smp2p-modem and qcom,glink/modem */
+#define IPCC_GLINK 0u
+#define IPCC_SMP2P 2u
+
+static struct block ipcc = { "ipcc", 0x00408000, 0x1000 };
+
+static struct {
+ uint32_t glink, smp2p, said;
+ uint64_t since;
+} doorbell;
+
+/*
+ * The modem's rings, counted and cleared. Another's ring (the Wi-Fi DSP's, the always-on
+ * processor's) is left for whoever listens for it: IPCC names the same one until it is cleared,
+ * so the look ends there.
+ */
+static void
+doorbells(void)
+{
+ for (unsigned i = 0; i < 8; i++) {
+ uint32_t who = rd(ipcc.base + IPCC_RECEIVED);
+
+ if (who == IPCC_NOBODY || who >> 16 != IPCC_MPSS) {
+ return;
+ }
+ wr(ipcc.base + IPCC_CLEAR, who);
+ doorbell.glink += (who & 0xffff) == IPCC_GLINK;
+ doorbell.smp2p += (who & 0xffff) == IPCC_SMP2P;
+ if (doorbell.said < 24) {
+ uint64_t now = now_us() - doorbell.since;
+
+ doorbell.said++;
+ printf("mpss: doorbell: +%llu.%03llu ms: signal %u of the modem; heard and cleared\n", now / 1000, now % 1000,
+ who & 0xffff);
+ }
+ }
+}
+
+static uint32_t our_box; /* where in the message memory our mailbox is; 0: not open */
+
+static bool
+mailbox_wait(uint32_t offset, uint32_t value, uint32_t ms)
+{
+ for (uint32_t i = 0; i < ms * 10; i++) {
+ if (rd(aoss.base + offset) == value) {
+ return true;
+ }
+ sleep_us(100);
+ }
+ return rd(aoss.base + offset) == value;
+}
+
+static void
+ring_aop(void)
+{
+ wr(ipcc.base + IPCC_SEND, IPCC_AOP << 16 | 0);
+}
+
+/* Linux's qmp_open(), as redwood_wifi.c does it; what is done already is found done. Under the mailbox's lock. */
+static bool
+mailbox_open(void)
+{
+ uint32_t box, size;
+
+ need(&aoss);
+ need(&ipcc);
+ if (get(&aoss, "magic", QMP_MAGIC) != QMP_MAGIC_VALUE || get(&aoss, "version", QMP_VERSION) != 1) {
+ printf("mpss: %u: not a mailbox this knows\n", step);
+ return false;
+ }
+ box = get(&aoss, "our mailbox", QMP_OUR_BOX);
+ size = get(&aoss, "our mailbox's size", QMP_OUR_BOX_SIZE);
+ if (size < QMP_MESSAGE + 4 || box < 0x3c || box + size > aoss.size || (box & 3)) {
+ printf("mpss: %u: our mailbox is not where a message fits\n", step);
+ return false;
+ }
+ if (rd(aoss.base + QMP_OUR_LINK) == QMP_UP && rd(aoss.base + QMP_OUR_LINK_ACK) == QMP_UP &&
+ rd(aoss.base + QMP_OUR_CHANNEL) == QMP_UP && rd(aoss.base + QMP_THEIR_CHANNEL) == QMP_UP &&
+ rd(aoss.base + QMP_OUR_CHANNEL_ACK) == QMP_UP) {
+ printf("mpss: %u: the mailbox is open already (the Wi-Fi's start, or a kernel before this one, opened it)\n", step);
+ our_box = box;
+ return true;
+ }
+ put(&aoss, "their link, seen by us", QMP_THEIR_LINK_ACK, rd(aoss.base + QMP_THEIR_LINK));
+ put(&aoss, "our link", QMP_OUR_LINK, QMP_UP);
+ ring_aop();
+ if (!mailbox_wait(QMP_OUR_LINK_ACK, QMP_UP, 1000)) {
+ printf("mpss: %u: THE ALWAYS-ON PROCESSOR DOES NOT ACKNOWLEDGE OUR LINK within a second; taken down again\n", step);
+ put(&aoss, "our link", QMP_OUR_LINK, QMP_DOWN);
+ ring_aop();
+ return false;
+ }
+ put(&aoss, "our channel", QMP_OUR_CHANNEL, QMP_UP);
+ ring_aop();
+ if (!mailbox_wait(QMP_THEIR_CHANNEL, QMP_UP, 1000)) {
+ printf("mpss: %u: THE ALWAYS-ON PROCESSOR DOES NOT OPEN ITS CHANNEL within a second; ours is taken down again\n", step);
+ put(&aoss, "our channel", QMP_OUR_CHANNEL, QMP_DOWN);
+ put(&aoss, "our link", QMP_OUR_LINK, QMP_DOWN);
+ ring_aop();
+ return false;
+ }
+ put(&aoss, "their channel, seen by us", QMP_THEIR_CHANNEL_ACK, QMP_UP);
+ ring_aop();
+ if (!mailbox_wait(QMP_OUR_CHANNEL_ACK, QMP_UP, 1000)) {
+ printf("mpss: %u: THE ALWAYS-ON PROCESSOR DOES NOT ACKNOWLEDGE OUR CHANNEL within a second; taken down again\n", step);
+ put(&aoss, "our channel", QMP_OUR_CHANNEL, QMP_DOWN);
+ put(&aoss, "our link", QMP_OUR_LINK, QMP_DOWN);
+ ring_aop();
+ return false;
+ }
+ printf("mpss: %u: THE MAILBOX TO THE ALWAYS-ON PROCESSOR IS OPEN\n", step);
+ our_box = box;
+ return true;
+}
+
+/* A message to the always-on processor (Linux's qmp_send()), under the lock the Wi-Fi's start takes too. */
+static bool
+mailbox_send(const char *text)
+{
+ uint32_t words[QMP_MESSAGE / 4] = { 0 };
+ bool sent = false;
+
+ if (!leaving) {
+ lck_mtx_lock(&redwood_aop_mailbox_lock);
+ }
+ if (our_box == 0 && (leaving || !mailbox_open())) {
+ goto out;
+ }
+ strlcpy((char *)words, text, sizeof(words));
+ printf("mpss: %u: about to tell the always-on processor \"%s\"\n", step, text);
+ snprintf(last_write, sizeof(last_write), "step %u sent the always-on processor a message", step);
+ if (rd(aoss.base + our_box) != 0) {
+ printf("mpss: %u: our mailbox still holds a message of %u bytes: not sent\n", step, rd(aoss.base + our_box));
+ goto out;
+ }
+ for (unsigned i = 0; i < QMP_MESSAGE / 4; i++) {
+ wr(aoss.base + our_box + 4 + 4 * i, words[i]);
+ }
+ wr(aoss.base + our_box, QMP_MESSAGE);
+ (void)rd(aoss.base + our_box);
+ ring_aop();
+ if (!mailbox_wait(our_box, 0, 1000)) {
+ printf("mpss: %u: THE ALWAYS-ON PROCESSOR DOES NOT TAKE THE MESSAGE within a second; it is taken back\n", step);
+ wr(aoss.base + our_box, 0);
+ goto out;
+ }
+ printf("mpss: %u: the always-on processor has taken it\n", step);
+ sent = true;
+out:
+ if (!leaving) {
+ lck_mtx_unlock(&redwood_aop_mailbox_lock);
+ }
+ return sent;
+}
+
+/* ---- RPMh: the DSP's two rails, asked for while it boots ------------------------------------------------ */
+
+/*
+ * The modem's logic and memories run from LPASS's own rails, LCX and LMX (dt: vdd_cx and
+ * vdd_mx of qcom,lpass@3700000 are PM7325 L5 and L4 "level" regulators, RPMh resources cx.lvl and
+ * mss.lvl), which the boot loader leaves at retention (16). Linux and Android ask for level 384
+ * while the DSP boots, until it says it has asked for its own (bit 2, "proxy-unvote"); without it
+ * the DSP rang once and left "SFR Init: wdog or kernel error suspected." in its last words.
+ *
+ * A request goes to RPMh through the application processor's sequencer (dt: rsc@18200000, DRV 2,
+ * qcom,tcs-offset 0xd00, qcom,tcs-config: TCSs 0 and 1 are for requests made at once), as Linux's
+ * rpmh-rsc.c writes one: the command's message id, address and value into a TCS that is idle, the
+ * command enabled and marked to be waited for, the TCS put in "active" mode and triggered, and its
+ * bit in the status of completions looked for. A rail asked for by level is asked by position in
+ * the list of its levels that the boot loader's command database keeps (cmd-db.c), the first
+ * position at or above the level wanted.
+ */
+#define CMD_DB_PHYS 0x80860000UL
+#define CMD_DB_SIZE 0x20000UL
+
+struct cmd_db_slave {
+ uint16_t id, header_offset, data_offset, count, version, reserved[3];
+};
+struct cmd_db_header {
+ uint32_t version;
+ uint8_t magic[4];
+ struct cmd_db_slave slave[8];
+ uint32_t checksum, reserved;
+ uint8_t data[];
+};
+struct cmd_db_entry {
+ char id[8];
+ uint32_t priority[2];
+ uint32_t address;
+ uint16_t length, offset;
+};
+
+static const struct cmd_db_header *cmd_db;
+
+static struct block rsc = { "rsc", 0x18220000, 0x10000 };
+#define RSC_TCS_BASE 0xd00
+#define RSC_IRQ_STATUS (RSC_TCS_BASE + 0x04) /* a bit per TCS: done */
+#define RSC_IRQ_CLEAR (RSC_TCS_BASE + 0x08)
+#define RSC_TCS(n) (RSC_TCS_BASE + 672 * (n))
+#define TCS_WAIT_FOR_DONE 0x10
+#define TCS_CONTROL 0x14
+#define TCS_STATUS 0x18 /* not zero: idle */
+#define TCS_ENABLED 0x1c
+#define TCS_COMMAND(c) (0x30 + 20 * (c)) /* message id, address, value, status */
+#define TCS_ACTIVE_MODE (1u << 16)
+#define TCS_TRIGGER (1u << 24)
+#define MSGID_WRITE_WAITED (8u | 1u << 8 | 1u << 16) /* four bytes, an answer wanted, a write */
+#define RPMH_TCS 0 /* the first of the two for requests made at once */
+
+/* The level position RPMh takes for that rail and level; -1: not found. */
+static int
+rail_position(const char *name, uint32_t level, uint32_t *address)
+{
+ char id[8] = { 0 };
+
+ if (cmd_db == NULL) {
+ cmd_db = (const struct cmd_db_header *)ml_io_map_unmappable(CMD_DB_PHYS, CMD_DB_SIZE, VM_WIMG_WCOMB);
+ }
+ if (!(cmd_db->magic[0] == 0xdb && cmd_db->magic[1] == 0x30 && cmd_db->magic[2] == 0x03 && cmd_db->magic[3] == 0x0c)) {
+ printf("mpss: %u: the boot loader's command database is not there\n", step);
+ return -1;
+ }
+ strlcpy(id, name, sizeof(id));
+ for (unsigned s = 0; s < 8; s++) {
+ const struct cmd_db_slave *slave = &cmd_db->slave[s];
+ const struct cmd_db_entry *entry = (const struct cmd_db_entry *)(cmd_db->data + slave->header_offset);
+
+ if (slave->id == 0 || slave->count > 512) {
+ continue;
+ }
+ for (unsigned i = 0; i < slave->count; i++) {
+ const uint8_t *data;
+ char levels[160];
+ int at;
+
+ if (memcmp(entry[i].id, id, sizeof(id)) != 0) {
+ continue;
+ }
+ if ((unsigned)slave->data_offset + entry[i].offset + entry[i].length > CMD_DB_SIZE - 0x1000) {
+ return -1;
+ }
+ data = cmd_db->data + slave->data_offset + entry[i].offset;
+ at = snprintf(levels, sizeof(levels), "%s at %#x, levels:", name, entry[i].address);
+ for (unsigned l = 0; l < entry[i].length / 2u && l < 16; l++) {
+ unsigned value = data[2 * l] | data[2 * l + 1] << 8;
+
+ if (l > 0 && value == 0) {
+ break;
+ }
+ at += snprintf(levels + at, sizeof(levels) - (size_t)at, " %u", value);
+ }
+ printf("mpss: %u: RPMh has %s\n", step, levels);
+ for (unsigned l = 0; l < entry[i].length / 2u && l < 16; l++) {
+ if ((uint32_t)(data[2 * l] | data[2 * l + 1] << 8) >= level) {
+ *address = entry[i].address;
+ return (int)l;
+ }
+ }
+ return -1;
+ }
+ }
+ printf("mpss: %u: RPMh has no \"%s\"\n", step, name);
+ return -1;
+}
+
+/* The two rails at that level (Linux's proxy request), or given back (level 0). True: RPMh has said done. */
+static bool
+rails(uint32_t level)
+{
+ static const char *const names[2] = { "cx.lvl", "mss.lvl" };
+ uint32_t addresses[2] = { 0, 0 }, values[2] = { 0, 0 }, done = 0;
+ uint64_t began;
+
+ for (unsigned r = 0; r < 2; r++) {
+ int position = level == 0 ? 0 : rail_position(names[r], level, &addresses[r]);
+
+ if (level == 0) {
+ rail_position(names[r], 0, &addresses[r]);
+ }
+ if (position < 0 || addresses[r] == 0) {
+ printf("mpss: %u: no position for %s at level %u: the rails are not asked\n", step, names[r], level);
+ return false;
+ }
+ values[r] = (uint32_t)position;
+ }
+ need(&rsc);
+ /* (the boot loader leaves the trigger set after its last request, done: idle is what the status says) */
+ if (rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_STATUS) == 0) {
+ printf("mpss: %u: TCS %u of the sequencer is busy (status %#x, control %#x): the rails are not asked\n", step, RPMH_TCS,
+ rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_STATUS), rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL));
+ return false;
+ }
+ printf("mpss: %u: about to ask RPMh, through TCS %u, for cx.lvl (%#x) = %u and mss.lvl (%#x) = %u (level %u%s)\n", step,
+ RPMH_TCS, addresses[0], values[0], addresses[1], values[1], level, level == 0 ? ": given back" : "");
+ snprintf(last_write, sizeof(last_write), "step %u asked RPMh for the modem's rails at level %u", step, level);
+ hold();
+ wr(rsc.base + RSC_IRQ_CLEAR, 1u << RPMH_TCS);
+ for (unsigned r = 0; r < 2; r++) {
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(r), MSGID_WRITE_WAITED);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(r) + 4, addresses[r]);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(r) + 8, values[r]);
+ }
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_WAIT_FOR_DONE, 3);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_ENABLED, 3);
+ settle();
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) & ~TCS_TRIGGER);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) | TCS_ACTIVE_MODE);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) | TCS_TRIGGER);
+ began = now_us();
+ while (now_us() - began < 100000) {
+ done = rd(rsc.base + RSC_IRQ_STATUS) & (1u << RPMH_TCS);
+ if (done) {
+ break;
+ }
+ sleep_us(50);
+ }
+ printf("mpss: %u: RPMh %s after %llu us; the commands' status %#x %#x, the TCS's %#x\n", step, done ? "HAS DONE IT" :
+ "HAS NOT SAID DONE", now_us() - began, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(0) + 12),
+ rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(1) + 12), rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_STATUS));
+ wr(rsc.base + RSC_IRQ_CLEAR, 1u << RPMH_TCS);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) & ~(TCS_TRIGGER | TCS_ACTIVE_MODE));
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_ENABLED, 0);
+ return done != 0;
+}
+
+
+/*
+ * One command to RPMh through the sequencer, as rails() sends its two: a resource's address from
+ * the command database and the word for it. True: RPMh has said done. The IPA's clock is such a
+ * resource, a "bus clock manager" named IP0 (Linux's clk-rpmh.c: BCM votes are a word of commit
+ * (bit 30), valid (bit 29) and a vote of 14 bits).
+ */
+static bool
+rpmh_command(const char *name, uint32_t value)
+{
+ uint32_t address = 0, done = 0;
+ uint64_t began;
+
+ if (rail_position(name, 0, &address) < 0 || address == 0) {
+ printf("mpss: %u: RPMh has no \"%s\": nothing is asked\n", step, name);
+ return false;
+ }
+ need(&rsc);
+ if (rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_STATUS) == 0) {
+ printf("mpss: %u: TCS %u of the sequencer is busy: \"%s\" is not asked\n", step, RPMH_TCS, name);
+ return false;
+ }
+ printf("mpss: %u: about to ask RPMh, through TCS %u, for %s (%#x) = %#x\n", step, RPMH_TCS, name, address, value);
+ snprintf(last_write, sizeof(last_write), "step %u asked RPMh for %s = %#x", step, name, value);
+ hold();
+ wr(rsc.base + RSC_IRQ_CLEAR, 1u << RPMH_TCS);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(0), MSGID_WRITE_WAITED);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(0) + 4, address);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(0) + 8, value);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_WAIT_FOR_DONE, 1);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_ENABLED, 1);
+ settle();
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) & ~TCS_TRIGGER);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) | TCS_ACTIVE_MODE);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) | TCS_TRIGGER);
+ began = now_us();
+ while (now_us() - began < 100000) {
+ done = rd(rsc.base + RSC_IRQ_STATUS) & (1u << RPMH_TCS);
+ if (done) {
+ break;
+ }
+ sleep_us(50);
+ }
+ printf("mpss: %u: RPMh %s after %llu us for %s; the command's status %#x\n", step, done ? "HAS DONE IT" : "HAS NOT SAID DONE",
+ now_us() - began, name, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_COMMAND(0) + 12));
+ wr(rsc.base + RSC_IRQ_CLEAR, 1u << RPMH_TCS);
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL, rd(rsc.base + RSC_TCS(RPMH_TCS) + TCS_CONTROL) & ~(TCS_TRIGGER | TCS_ACTIVE_MODE));
+ wr(rsc.base + RSC_TCS(RPMH_TCS) + TCS_ENABLED, 0);
+ return done != 0;
+}
+
+#define BCM_VOTE(on, vote) (1u << 30 | (uint32_t)(on) << 29 | ((vote) & 0x3fffu))
+
+/* The IPA's clock, on or off: the BCM "IP0" (dt of Linux: clocks = <&rpmhcc RPMH_IPA_CLK>). */
+static bool
+ipa_clock(bool on)
+{
+ return rpmh_command("IP0", BCM_VOTE(on, on ? 0x3fffu : 0));
+}
+
+/* ---- step 1: the shared memory --------------------------------------------------------------------- */
+
+static bool
+step_survey(void)
+{
+ struct smp2p words;
+ uint32_t size = 0;
+
+ announce("read our partition with the modem in the shared memory (0x80900000), and its items");
+ show_partition();
+ if (dsp_words(&words)) {
+ show_words("the DSP's item (435) is there: it has run since the phone was started", &words);
+ } else {
+ printf("mpss: %u: the DSP's SMP2P item (435) is not there: it has not run since the phone was started\n", step);
+ }
+ printf("mpss: %u: our SMP2P item (428): %s; GLINK's descriptor (478): %s, our queue (479): %s, the DSP's (480): %s\n", step,
+ smem_item(ITEM_SMP2P_OURS, &size) ? "there" : "not there", smem_item(478, &size) ? "there" : "NOT THERE",
+ smem_item(479, &size) ? "there" : "NOT THERE", smem_item(480, &size) ? "there" : "not there (the DSP makes it)");
+ show_crash_reason();
+ need(&ipcc);
+ printf("mpss: %u: the doorbells say %#x\n", step, rd(ipcc.base + IPCC_RECEIVED));
+ return smem_partition(SMEM_US, SMEM_MPSS, &size, &size) != 0;
+}
+
+/* ---- step 2: the firmware -------------------------------------------------------------------------- */
+
+#define MPSS_MEMORY 0x8b800000u /* dt: reserved-memory mpss@8b800000, memory-region of qcom,mss@4080000 */
+#define MPSS_MEMORY_SIZE 0x0f600000u
+#define PAS_MPSS 4 /* dt: qcom,pas-id */
+
+
+/*
+ * A file of /fw on the root file system, read whole into memory of ours: the system image, where
+ * userland/build.sh puts fw-modem/ when it makes that image. It is compressed there, which the file
+ * system undoes and the reading of the small image's memory below does not; and it is there only
+ * once the root is mounted, so only modem=1, which waits for that on its own thread, finds it so.
+ */
+static const uint8_t *
+root_file(const char *name, uint64_t *size)
+{
+ char path[256];
+ vfs_context_t context = vfs_context_kernel();
+ vnode_t vnode = NULLVP;
+ struct vnode_attr attributes;
+ uint8_t *data;
+ int left = 0, error;
+
+ if (rootvnode == NULLVP) {
+ return NULL;
+ }
+ snprintf(path, sizeof(path), "/fw/%s", name);
+ if (vnode_lookup(path, 0, &vnode, context) != 0) {
+ return NULL;
+ }
+ VATTR_INIT(&attributes);
+ VATTR_WANTED(&attributes, va_data_size);
+ if (vnode_getattr(vnode, &attributes, context) != 0 || attributes.va_data_size == 0 ||
+ attributes.va_data_size > MPSS_MEMORY_SIZE) {
+ vnode_put(vnode);
+ return NULL;
+ }
+ data = kalloc_data(attributes.va_data_size, Z_WAITOK);
+ if (data == NULL) {
+ vnode_put(vnode);
+ return NULL;
+ }
+ error = vn_rdwr(UIO_READ, vnode, (caddr_t)data, (int)attributes.va_data_size, 0, UIO_SYSSPACE, 0,
+ vfs_context_ucred(context), &left, vfs_context_proc(context));
+ vnode_put(vnode);
+ if (error != 0 || left != 0) {
+ printf("mpss: %u: %s: read error %d, %d bytes left\n", step, path, error, left);
+ kfree_data(data, attributes.va_data_size);
+ return NULL;
+ }
+ *size = attributes.va_data_size;
+ return data;
+}
+
+static bool from_root; /* the image is in memory of ours (root_file()), to be given back once written */
+
+/* A file of fw/ in the small image (MODEM=1 userland/build.sh puts fw-modem/ there), else of /fw on the root (above). */
+extern const uint8_t *redwood_image_file(const char *dir, const char *name, uint64_t *size, int *disk_out);
+static const uint8_t *
+image_file(const char *name, uint64_t *size)
+{
+ {
+ const uint8_t *data = redwood_image_file("fw", name, size, NULL); /* (redwood_image.c) */
+
+ if (data != NULL) {
+ return data;
+ }
+ }
+ {
+ const uint8_t *data = root_file(name, size);
+
+ if (data != NULL) {
+ from_root = true;
+ return data;
+ }
+ }
+ printf("mpss: %u: no fw/%s in the small image nor on the root: put the phone's /vendor/firmware_mnt/image/%s into "
+ "fw-modem/ and run SYSTEM=1 userland/build.sh (or MODEM=1, for the small image)\n", step, name, name);
+ return NULL;
+}
+
+
+#define MDT_HASH(flags) (((flags) >> 24 & 7) == 2)
+#define MAX_SEGMENTS 48
+
+struct elf_segment {
+ uint32_t type, offset, vaddr, paddr, file_size, memory_size, flags, align;
+};
+
+static struct {
+ const uint8_t *header;
+ uint64_t header_size;
+ struct {
+ uint32_t at;
+ uint32_t file_size, memory_size;
+ const uint8_t *file;
+ } segment[MAX_SEGMENTS];
+ unsigned segments;
+ uint32_t size;
+} image;
+
+/* The image's files given back, if they were read into memory of ours. */
+static void
+image_forget(void)
+{
+ if (!from_root) {
+ return;
+ }
+ if (image.header != NULL) {
+ void *data = (void *)(uintptr_t)image.header;
+
+ kfree_data(data, image.header_size);
+ }
+ for (unsigned s = 0; s < image.segments; s++) {
+ if (image.segment[s].file != NULL) {
+ void *data = (void *)(uintptr_t)image.segment[s].file;
+
+ kfree_data(data, image.segment[s].file_size);
+ }
+ }
+ bzero(&image, sizeof(image));
+ from_root = false;
+}
+
+static unsigned rmtfs_load(void);
+static void servreg_load(void);
+
+static bool
+step_firmware(void)
+{
+ struct elf_segment segment;
+ const uint8_t *header;
+ uint64_t size = 0, files = 0;
+ uint32_t count, hashes = 0, lowest = ~0u, highest = 0;
+
+ announce("find the modem's image in the small image or on the root, and check it against its own header");
+ image_forget();
+ image.segments = 0;
+ header = image_file("modem.mdt", &size);
+ if (header == NULL) {
+ return false;
+ }
+ image.header = header;
+ image.header_size = size;
+ count = size >= 52 + 2 * 32 ? header[44] | header[45] << 8 : 0;
+ if (memcmp(header, "\177ELF\1", 5) != 0 || count < 3 || count > MAX_SEGMENTS || 52 + 32 * count > size || size > PAGE_SIZE) {
+ printf("mpss: %u: fw/modem.mdt is not such a header (%02x %02x %02x %02x, %u segments, %llu bytes)\n", step, header[0],
+ header[1], header[2], header[3], count, size);
+ goto fail;
+ }
+ for (uint32_t i = 0; i < count; i++) {
+ char name[16];
+
+ memcpy(&segment, header + 52 + 32 * i, sizeof(segment));
+ if (MDT_HASH(segment.flags)) {
+ hashes = segment.file_size;
+ }
+ if (segment.type != 1 || MDT_HASH(segment.flags) || segment.memory_size == 0) {
+ continue;
+ }
+ if (segment.paddr < MPSS_MEMORY || segment.file_size > segment.memory_size ||
+ (uint64_t)segment.paddr + segment.memory_size > (uint64_t)MPSS_MEMORY + MPSS_MEMORY_SIZE) {
+ printf("mpss: %u: segment %u (%#x, %#x bytes) is not within the DSP's memory\n", step, i, segment.paddr,
+ segment.memory_size);
+ goto fail;
+ }
+ image.segment[image.segments].at = segment.paddr - MPSS_MEMORY;
+ image.segment[image.segments].file_size = segment.file_size;
+ image.segment[image.segments].memory_size = segment.memory_size;
+ image.segment[image.segments].file = NULL;
+ if (segment.file_size != 0) {
+ uint64_t file_size = 0;
+
+ snprintf(name, sizeof(name), "modem.b%02u", i);
+ image.segment[image.segments].file = image_file(name, &file_size);
+ if (image.segment[image.segments].file == NULL) {
+ goto fail;
+ }
+ if (file_size != segment.file_size) {
+ printf("mpss: %u: fw/%s has %llu bytes, the header says %u\n", step, name, file_size, segment.file_size);
+ image.segments++; /* (for its file to be given back) */
+ goto fail;
+ }
+ files += file_size;
+ }
+ lowest = segment.paddr < lowest ? segment.paddr : lowest;
+ highest = segment.paddr + segment.memory_size > highest ? segment.paddr + segment.memory_size : highest;
+ image.segments++;
+ }
+ if (image.segments == 0 || lowest != MPSS_MEMORY || 52 + 32 * count + (uint64_t)hashes != size) {
+ printf("mpss: %u: not what TrustZone takes: %u segments to load, the lowest at %#x; the header file has %llu bytes, "
+ "the header %u and the hashes %u\n", step, image.segments, lowest, size, 52 + 32 * count, hashes);
+ goto fail;
+ }
+ image.header = header;
+ image.header_size = size;
+ image.size = ((highest + 0xfff) & ~0xfffu) - MPSS_MEMORY;
+ printf("mpss: %u: the image is whole (%s): a header of %llu bytes (%u of hashes), %u segments, %llu bytes of files, %#x "
+ "bytes of the DSP's %#x from %#x\n", step, from_root ? "from /fw on the root" : "from the small image", size, hashes,
+ image.segments, files, image.size, MPSS_MEMORY_SIZE, MPSS_MEMORY);
+ announce("read the modem's four file systems from the phone's storage into memory of ours (rmtfs below)");
+ {
+ unsigned loaded = rmtfs_load();
+
+ printf("mpss: %u: %u of the modem's 4 file systems are here%s\n", step, loaded, loaded == 4 ? "" :
+ ": WITHOUT THEM ALL THE MODEM OFFERS NO SERVICES BUT ITS OWN (DMS, NAS... stay away)");
+ servreg_load();
+ }
+ return true;
+fail:
+ image_forget();
+ image.segments = 0;
+ return false;
+}
+
+/* ---- step 3: the mailbox --------------------------------------------------------------------------- */
+
+static bool
+step_mailbox(void)
+{
+ bool open;
+
+ announce("open the mailbox to the always-on processor, unless it is open: words of its message memory, and its doorbell");
+ lck_mtx_lock(&redwood_aop_mailbox_lock);
+ open = mailbox_open();
+ lck_mtx_unlock(&redwood_aop_mailbox_lock);
+ return open;
+}
+
+/* ---- TrustZone, and stopping the DSP --------------------------------------------------------------- */
+
+#define SCM_64 (1ull << 30)
+#define SCM_SIP (2ull << 24)
+#define SCM_CALL(service, command) ((uint64_t)(service) << 8 | (command))
+#define SCM_PAS_INIT_IMAGE SCM_CALL(2, 1)
+#define SCM_PAS_MEM_SETUP SCM_CALL(2, 2)
+#define SCM_PAS_AUTH_AND_RESET SCM_CALL(2, 5)
+#define SCM_PAS_SHUTDOWN SCM_CALL(2, 6)
+#define SCM_ONE_VALUE 1
+#define SCM_THREE_VALUES 3
+#define SCM_VALUE_AND_BUFFER (2 | 2 << 6)
+#define SCM_INTERRUPTED 1
+
+static LCK_MTX_DECLARE(dsp_lock, &mpss_lock_group); /* the DSP being started or stopped */
+static bool rails_asked; /* our request for its rails stands (until it asks for its own, or is shut down) */
+static bool dsp_started;
+static bool stopped_by_its_word;
+static uint32_t our_item;
+static bool smp2p_agreed, smp2p_restart_acks, smp2p_restart_acked;
+
+static int32_t
+scm(const char *what, uint64_t function, uint64_t arguments, uint64_t first, uint64_t second, uint64_t third, uint64_t fourth,
+ uint64_t *answer)
+{
+ uint64_t session = 0;
+
+ function |= SCM_64 | SCM_SIP;
+ printf("mpss: %u: about to ask TrustZone %s: smc %#llx, arguments %#llx, %#llx, %#llx\n", step, what, function, first,
+ second, third);
+ snprintf(last_write, sizeof(last_write), "step %u asked TrustZone %#llx(%#llx, %#llx)", step, function, first, second);
+ for (int i = 0; i < 100; i++) {
+ register uint64_t x0 __asm__("x0") = i == 0 ? function : SCM_INTERRUPTED;
+ register uint64_t x1 __asm__("x1") = arguments;
+ register uint64_t x2 __asm__("x2") = first;
+ register uint64_t x3 __asm__("x3") = second;
+ register uint64_t x4 __asm__("x4") = third;
+ register uint64_t x5 __asm__("x5") = fourth;
+ register uint64_t x6 __asm__("x6") = session;
+ register uint64_t x7 __asm__("x7") = 0;
+
+ __asm__ volatile ("smc #0" : "+r" (x0), "+r" (x1), "+r" (x2), "+r" (x3), "+r" (x4), "+r" (x5), "+r" (x6),
+ "+r" (x7) : : "x8", "x9", "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", "memory");
+ session = x6;
+ if (x0 != SCM_INTERRUPTED) {
+ printf("mpss: %u: TrustZone returns %d, answer %#llx\n", step, (int32_t)x0, x1);
+ *answer = x1;
+ return (int32_t)x0;
+ }
+ }
+ printf("mpss: %u: TrustZone keeps saying the call was interrupted\n", step);
+ return -1;
+}
+
+static uint32_t
+our_item_find(void)
+{
+ uint32_t size = 0, at = smem_item(ITEM_SMP2P_OURS, &size);
+
+ return at != 0 && size >= sizeof(struct smp2p) ? at : 0;
+}
+
+/* What Linux does when the DSP rings for SMP2P: agree on the features once, and answer a restart. */
+static void
+dsp_listen(const struct smp2p *theirs)
+{
+ struct smp2p ours;
+
+ if (our_item == 0 || !smem_peek(our_item, &ours, sizeof(ours)) || ours.magic != SMP2P_MAGIC) {
+ return;
+ }
+ if (!smp2p_agreed) {
+ uint32_t features = (ours.version >> 8) & (theirs->version >> 8);
+
+ if ((theirs->version & 0xff) != (ours.version & 0xff)) {
+ return;
+ }
+ if (features != ours.version >> 8) {
+ smem_poke(our_item + SMP2P_AT_VERSION, features << 8 | (ours.version & 0xff));
+ }
+ smp2p_restart_acks = features & SMP2P_RESTART_ACKS;
+ smp2p_agreed = true;
+ printf("mpss: %u: the DSP and we agree on SMP2P: version %u, features %#x\n", step, ours.version & 0xff, features);
+ }
+ if (smp2p_restart_acks && ((theirs->flags & SMP2P_RESTART_DONE) != 0) != smp2p_restart_acked) {
+ smp2p_restart_acked = !smp2p_restart_acked;
+ printf("mpss: %u: the DSP's flags say it has restarted (%#x): answered in our flags (bit 1 = %u), and its SMP2P doorbell\n",
+ step, theirs->flags, smp2p_restart_acked);
+ smem_poke(our_item + SMP2P_AT_FLAGS, (ours.flags & ~SMP2P_RESTART_ACK) | (smp2p_restart_acked ? SMP2P_RESTART_ACK : 0));
+ settle();
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_SMP2P);
+ }
+}
+
+/* Linux's qcom_q6v5_request_stop(): our bit "stop", the DSP's "stop ack" waited for, our bit taken back. */
+static bool
+dsp_ask_to_stop(uint32_t bound_ms)
+{
+ struct smp2p ours, theirs;
+ uint32_t at = our_item_find(), state, value_at;
+ uint64_t asked, now;
+ int entry = -1;
+
+ if (at == 0 || !smem_peek(at, &ours, sizeof(ours)) || ours.magic != SMP2P_MAGIC || (ours.version & 0xff) != SMP2P_VERSION ||
+ (entry = smp2p_entry(&ours, OUR_ENTRY_STOP)) < 0) {
+ printf("mpss: %u: the DSP is not asked to stop first: there is no SMP2P item of ours to ask it with\n", step);
+ return false;
+ }
+ if (!dsp_words(&theirs) || smp2p_entry(&theirs, "slave-kernel") < 0) {
+ printf("mpss: %u: the DSP is not asked to stop first: its item has no entry to answer in\n", step);
+ return false;
+ }
+ state = dsp_state(&theirs);
+ if (state & DSP_STOP_ACK) {
+ printf("mpss: %u: the DSP is not asked to stop: its entry says %#x, that it has stopped when asked before\n", step, state);
+ return true;
+ }
+ if (!(state & DSP_READY) || (state & DSP_FATAL)) {
+ printf("mpss: %u: the DSP is not asked to stop first: its entry says %#x, not \"up\"\n", step, state);
+ return false;
+ }
+ need(&ipcc);
+ value_at = at + SMP2P_AT_VALUE((uint32_t)entry);
+ printf("mpss: %u: about to ask the DSP to stop: bit 0 of our entry \"%s\" at %#lx, and its SMP2P doorbell\n", step,
+ OUR_ENTRY_STOP, SMEM_PHYS + value_at);
+ snprintf(last_write, sizeof(last_write), "step %u asked the modem to stop", step);
+ smem_poke(value_at, ours.entry[entry].value | DSP_STOP);
+ settle();
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_SMP2P);
+ asked = now = now_us();
+ while (now - asked < bound_ms * 1000ull) {
+ if (dsp_words(&theirs) && (dsp_state(&theirs) & (DSP_STOP_ACK | DSP_FATAL))) {
+ break;
+ }
+ sleep_us(200);
+ now = now_us();
+ }
+ state = dsp_words(&theirs) ? dsp_state(&theirs) : 0;
+ printf("mpss: %u: %s after %llu.%03llu ms; its entry says %#x\n", step, state & DSP_STOP_ACK ?
+ "THE MODEM ACKNOWLEDGES THE STOP" : "THE MODEM DOES NOT ACKNOWLEDGE THE STOP", (now - asked) / 1000,
+ (now - asked) % 1000, state);
+ smem_poke(value_at, ours.entry[entry].value & ~DSP_STOP);
+ settle();
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_SMP2P);
+ return (state & DSP_STOP_ACK) != 0;
+}
+
+static void link_stop(void);
+
+/* Asked to stop, then shut down by TrustZone, then the always-on processor told that the image is gone. */
+static bool
+dsp_shutdown(const char *why)
+{
+ uint64_t answer = ~0ull;
+ int32_t result;
+
+ printf("mpss: %u: %s: the modem is asked to stop, then TrustZone to shut it down\n", step, why);
+ if (!leaving) {
+ link_stop();
+ }
+ stopped_by_its_word = dsp_ask_to_stop(leaving ? 2000 : 5000);
+ result = scm("to shut the modem down", SCM_PAS_SHUTDOWN, SCM_ONE_VALUE, PAS_MPSS, 0, 0, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT SHUT THE MODEM DOWN: the call returns %d, its answer is %#llx\n", step, result,
+ answer);
+ return false;
+ }
+ printf("mpss: %u: THE MODEM IS SHUT DOWN\n", step);
+ if (ipa_loaded) {
+ ipa_firmware_stop();
+ ipa_loaded = false;
+ }
+ if (!leaving) {
+ rmtfs_share(false);
+ }
+ need(&ipcc);
+ wr(ipcc.base + IPCC_DISABLE, IPCC_MPSS << 16 | IPCC_SMP2P);
+ doorbells();
+ dsp_started = false;
+ mailbox_send("{class: image, res: load_state, name: modem, val: off}");
+ if (rails_asked) {
+ rails(0);
+ rails_asked = false;
+ }
+ return true;
+}
+
+/* ---- step 4: TrustZone takes the header ------------------------------------------------------------ */
+
+static bool
+step_header(void)
+{
+ struct smp2p words;
+ vm_offset_t page = 0;
+ uint64_t answer = ~0ull;
+ int32_t result;
+
+ if (image.segments == 0) {
+ return false;
+ }
+ if (dsp_words(&words)) {
+ show_words("the DSP has left words in the shared memory", &words);
+ announce("ask TrustZone to shut down an modem that a kernel before this one may have left running");
+ if (!dsp_shutdown("it may be running")) {
+ return false;
+ }
+ if (!stopped_by_its_word && !option("modem_again")) {
+ printf("mpss: %u: THE MODEM IS NOT STARTED AGAIN: it was shut down without having acknowledged a stop, and "
+ "with the Wi-Fi's DSP a start after that reset the phone (modem_again=1 to try all the same)\n", step);
+ return false;
+ }
+ } else {
+ printf("mpss: %u: the DSP has left nothing in the shared memory: it has not run since the phone was started\n", step);
+ }
+
+ announce("hand the image's header to TrustZone to check its signature (image 4), in a page of ours");
+ if (kmem_alloc(kernel_map, &page, PAGE_SIZE, KMA_KOBJECT | KMA_ZERO | KMA_DATA, VM_KERN_MEMORY_IOKIT) != KERN_SUCCESS) {
+ printf("mpss: %u: no memory\n", step);
+ return false;
+ }
+ memcpy((void *)page, image.header, image.header_size);
+ for (vm_offset_t line = page; line < page + PAGE_SIZE; line += 64) {
+ __asm__ volatile ("dc civac, %0" : : "r" (line));
+ }
+ settle();
+ result = scm("to take the modem's header", SCM_PAS_INIT_IMAGE, SCM_VALUE_AND_BUFFER, PAS_MPSS, kvtophys(page), 0, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT TAKE THE HEADER: the call returns %d, its answer is %#llx\n", step, result, answer);
+ return false;
+ }
+ printf("mpss: %u: TRUSTZONE ACCEPTS THE MODEM'S SIGNATURE FROM THIS KERNEL\n", step);
+
+ announce("tell TrustZone where the image will be: the whole of the modem's memory, as Linux does");
+ result = scm("where the image will be", SCM_PAS_MEM_SETUP, SCM_THREE_VALUES, PAS_MPSS, MPSS_MEMORY, MPSS_MEMORY_SIZE, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT TAKE THE PLACE: the call returns %d, its answer is %#llx\n", step, result, answer);
+ return false;
+ }
+ printf("mpss: %u: TrustZone takes the place: %#x bytes at %#x (the image takes %#x of them)\n", step, MPSS_MEMORY_SIZE, MPSS_MEMORY,
+ image.size);
+ return true;
+}
+
+/* ---- step 5: the image, in the DSP's memory --------------------------------------------------------- */
+
+static bool image_written;
+
+static uint64_t
+segment_word(unsigned s, uint32_t at)
+{
+ uint64_t word = 0;
+
+ for (unsigned b = 0; b < 8 && at + b < image.segment[s].file_size; b++) {
+ word |= (uint64_t)image.segment[s].file[at + b] << (8 * b);
+ }
+ return word;
+}
+
+/* A segment written at its place: words of eight where it begins on one, bytes where it does not or at its end. */
+static void
+segment_put(volatile uint8_t *dsp_memory, unsigned s)
+{
+ uint32_t at = image.segment[s].at, size = image.segment[s].memory_size, i = 0;
+
+ if ((at & 7) == 0) {
+ volatile uint64_t *to = (volatile uint64_t *)(dsp_memory + at);
+
+ for (; i + 8 <= size; i += 8) {
+ to[i / 8] = segment_word(s, i);
+ }
+ }
+ for (; i < size; i++) {
+ dsp_memory[at + i] = i < image.segment[s].file_size ? image.segment[s].file[i] : 0;
+ }
+}
+
+/* ... and read back: how many words of eight (or bytes, where it is not written by words) differ. */
+static uint64_t
+segment_check(volatile uint8_t *dsp_memory, unsigned s)
+{
+ uint32_t at = image.segment[s].at, size = image.segment[s].memory_size, i = 0;
+ uint64_t wrong = 0;
+
+ if ((at & 7) == 0) {
+ volatile uint64_t *from = (volatile uint64_t *)(dsp_memory + at);
+
+ for (; i + 8 <= size; i += 8) {
+ wrong += from[i / 8] != segment_word(s, i);
+ }
+ }
+ for (; i < size; i++) {
+ wrong += dsp_memory[at + i] != (i < image.segment[s].file_size ? image.segment[s].file[i] : 0);
+ }
+ return wrong;
+}
+
+static bool
+step_image(void)
+{
+ volatile uint8_t *dsp_memory;
+ uint64_t written = 0, cleared = 0, wrong = 0, began;
+
+ if (image.segments == 0) {
+ return false;
+ }
+ announce("put the image's segments into the modem's memory (0x8b800000, 246 MB: memory of nobody's until TrustZone starts "
+ "the modem), with zeroes where a segment is longer than its file, and read them back");
+ dsp_memory = (volatile uint8_t *)ml_io_map_unmappable(MPSS_MEMORY, image.size, VM_WIMG_WCOMB);
+ if (dsp_memory == NULL) {
+ printf("mpss: %u: the modem's memory cannot be mapped (%#x bytes)\n", step, image.size);
+ return false;
+ }
+ snprintf(last_write, sizeof(last_write), "step %u wrote the image at %#x", step, MPSS_MEMORY);
+ began = now_us();
+ for (unsigned s = 0; s < image.segments; s++) {
+ segment_put(dsp_memory, s);
+ written += image.segment[s].file_size;
+ cleared += image.segment[s].memory_size - image.segment[s].file_size;
+ }
+ settle();
+ for (unsigned s = 0; s < image.segments; s++) {
+ wrong += segment_check(dsp_memory, s);
+ }
+ printf("mpss: %u: %u segments, %llu bytes written and %llu cleared in %llu ms; read back, %llu words or bytes differ\n", step,
+ image.segments, written, cleared, (now_us() - began) / 1000, wrong);
+ settle();
+ ml_io_unmap((vm_offset_t)dsp_memory, image.size);
+ image_written = wrong == 0;
+ image_forget(); /* (it is in the modem's memory now: what was read of the root, given back) */
+ return image_written;
+}
+
+/* ---- step 6: our SMP2P item ------------------------------------------------------------------------ */
+
+static struct block smem_lock = { "shared memory's lock", 0x01f43000, 0x1000 };
+#define LOCK_OWNER 0x000
+#define LOCK_US 1u
+
+/* Our item made in the partition, as Linux's qcom_smem_alloc_private() makes one (redwood_wifi.c: our_item_make()). */
+static bool
+our_item_make(uint32_t partition, uint32_t item, uint32_t bytes)
+{
+ struct smem_partition header;
+ struct smem_entry entry;
+ const uint32_t item_size = (bytes + 7) & ~7u, need_bytes = sizeof(entry) + item_size;
+ uint32_t owner = ~0u, at;
+ bool made = false;
+
+ lck_mtx_lock(&redwood_smem_alloc_lock); /* (the Wi-Fi's start allocates under the same hardware lock) */
+ need(&smem_lock);
+ printf("mpss: %u: about to take the lock (%#x), write an entry for item %u with %u bytes behind it at the partition's mark, "
+ "move the mark by %u, and release the lock\n", step, smem_lock.phys + LOCK_OWNER, item, item_size, need_bytes);
+ snprintf(last_write, sizeof(last_write), "step %u took the shared memory's lock", step);
+ for (uint32_t ms = 0; ms <= 1000; ms++) {
+ wr(smem_lock.base + LOCK_OWNER, LOCK_US);
+ owner = rd(smem_lock.base + LOCK_OWNER);
+ if (owner == LOCK_US) {
+ break;
+ }
+ sleep_ms(1);
+ }
+ if (owner != LOCK_US) {
+ printf("mpss: %u: THE LOCK IS HELD BY PROCESSOR %u AND STAYS SO FOR A SECOND: nothing is written\n", step, owner);
+ lck_mtx_unlock(&redwood_smem_alloc_lock);
+ return false;
+ }
+ if (!smem_peek(partition, &header, sizeof(header)) || header.free_uncached > header.free_cached ||
+ header.free_cached > header.size) {
+ printf("mpss: %u: the partition's marks make no sense: nothing is written\n", step);
+ goto out;
+ }
+ for (at = sizeof(header); at < header.free_uncached;) {
+ if (!smem_peek(partition + at, &entry, sizeof(entry)) || entry.canary != SMEM_CANARY || entry.item == item ||
+ entry.size > header.size) {
+ break;
+ }
+ at += sizeof(entry) + entry.padding_header + entry.size;
+ }
+ if (at != header.free_uncached) {
+ printf("mpss: %u: the partition's entries do not end at its mark (+%#x, the mark is +%#x), or the item has appeared: "
+ "nothing is written\n", step, at, header.free_uncached);
+ goto out;
+ }
+ if (at + need_bytes > header.free_cached) {
+ printf("mpss: %u: no room: %u bytes wanted, %u free: nothing is written\n", step, need_bytes, header.free_cached - at);
+ goto out;
+ }
+ smem_poke(partition + at, SMEM_CANARY | item << 16);
+ smem_poke(partition + at + 4, item_size);
+ smem_poke(partition + at + 8, item_size - bytes);
+ for (uint32_t z = 0; z < item_size; z += 4) {
+ smem_poke(partition + at + (uint32_t)sizeof(entry) + z, 0);
+ }
+ smem_poke(partition + at + 12, 0);
+ settle();
+ smem_poke(partition + __offsetof(struct smem_partition, free_uncached), at + need_bytes);
+ settle();
+ made = true;
+out:
+ wr(smem_lock.base + LOCK_OWNER, 0);
+ printf("mpss: %u: the lock is released (it reads %u)\n", step, rd(smem_lock.base + LOCK_OWNER));
+ lck_mtx_unlock(&redwood_smem_alloc_lock);
+ return made;
+}
+
+static void glink_reset(void);
+
+static bool
+step_our_item(void)
+{
+ struct smem_partition header;
+ struct smp2p ours;
+ uint32_t cacheline = 0, size = 0, partition, name[4] = { 0 };
+
+ announce("look for an SMP2P item of ours (428) in our partition with the modem, make it if there is none, and fill it in");
+ partition = smem_partition(SMEM_US, SMEM_MPSS, &cacheline, &size);
+ if (partition == 0 || !smem_peek(partition, &header, sizeof(header))) {
+ printf("mpss: %u: there is no such partition\n", step);
+ return false;
+ }
+ our_item = our_item_find();
+ if (our_item != 0) {
+ printf("mpss: %u: item %u is there, at %#lx: a kernel before this one made it, and it is used again\n", step,
+ ITEM_SMP2P_OURS, SMEM_PHYS + our_item);
+ } else {
+ if (!our_item_make(partition, ITEM_SMP2P_OURS, sizeof(struct smp2p))) {
+ return false;
+ }
+ our_item = our_item_find();
+ if (our_item == 0) {
+ printf("mpss: %u: THE ITEM IS NOT FOUND where it was just made\n", step);
+ return false;
+ }
+ printf("mpss: %u: item %u made, at %#lx\n", step, ITEM_SMP2P_OURS, SMEM_PHYS + our_item);
+ }
+ /* zeroes; from processor 0 to 1, room for 16 entries, "master-kernel" at zero, feature bit 0; last the version */
+ snprintf(last_write, sizeof(last_write), "step %u filled in our SMP2P item", step);
+ for (uint32_t at = 0; at < sizeof(ours); at += 4) {
+ smem_poke(our_item + at, 0);
+ }
+ settle();
+ smem_poke(our_item, SMP2P_MAGIC);
+ smem_poke(our_item + SMP2P_AT_PROCESSORS, SMEM_US | SMEM_MPSS << 16);
+ strlcpy((char *)name, OUR_ENTRY_STOP, sizeof(name));
+ for (uint32_t word = 0; word < 4; word++) {
+ smem_poke(our_item + SMP2P_AT_NAME(0) + 4 * word, name[word]);
+ }
+ /*
+ * A second entry, "ipa" (dt: qcom,smp2p-ipa-1-out of qcom,smp2p-modem): the IPA driver's answer to the
+ * modem's question whether the IPA's clock is on, as Linux's ipa_smp2p.c gives it: bit 0 "the answer
+ * is valid", bit 1 "the clock is on". The modem, told so, keeps away from the IPA (there is no IPA
+ * driver here). Off unless modem_ipa=1 (valid, off) or 3 (on): with the entry the run of 23 October hung.
+ */
+ {
+ uint32_t ipa = 0;
+
+ PE_parse_boot_argn("modem_ipa", &ipa, sizeof(ipa));
+ if (ipa != 0) {
+ bzero(name, sizeof(name));
+ strlcpy((char *)name, "ipa", sizeof(name));
+ for (uint32_t word = 0; word < 4; word++) {
+ smem_poke(our_item + SMP2P_AT_NAME(1) + 4 * word, name[word]);
+ }
+ smem_poke(our_item + SMP2P_AT_VALUE(1), ipa);
+ printf("mpss: %u: our entry \"ipa\" = %#x\n", step, ipa);
+ }
+ smem_poke(our_item + SMP2P_AT_COUNTS, 16 | (ipa != 0 ? 2u : 1u) << 16);
+ }
+ smem_poke(our_item + SMP2P_AT_VERSION, SMP2P_RESTART_ACKS << 8);
+ settle();
+ smem_poke(our_item + SMP2P_AT_VERSION, SMP2P_RESTART_ACKS << 8 | SMP2P_VERSION);
+ settle();
+ smp2p_agreed = smp2p_restart_acks = smp2p_restart_acked = false;
+ need(&ipcc);
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_SMP2P);
+ if (!smem_peek(our_item, &ours, sizeof(ours)) || ours.magic != SMP2P_MAGIC || smp2p_entry(&ours, OUR_ENTRY_STOP) < 0) {
+ printf("mpss: %u: OUR ITEM DOES NOT READ BACK AS WRITTEN\n", step);
+ return false;
+ }
+ show_words("our item reads back", &ours);
+ /*
+ * The IPA's item (dt of Linux: qcom,smem-id 497, qcom,smem-size 0x2000 of the IPA): memory the
+ * application processor's IPA driver allocates in this partition for the modem's filter tables.
+ * There is no IPA driver here, but the modem looks for the item when it goes online; made, zeroed,
+ * so that it finds it (modem_ipa_item=0: not).
+ */
+ {
+ uint32_t ipa_item = 0, ipa_size = 0; /* (497 made the modem give up before any service: off; modem_ipa_item=497 to try) */
+
+ PE_parse_boot_argn("modem_ipa_item", &ipa_item, sizeof(ipa_item));
+ if (ipa_item != 0) {
+ if (smem_item(ipa_item, &ipa_size) != 0) {
+ printf("mpss: %u: the IPA's item %u is there already (%u bytes)\n", step, ipa_item, ipa_size);
+ } else if (our_item_make(partition, ipa_item, 0x2000)) {
+ printf("mpss: %u: the IPA's item %u made, 0x2000 bytes, zeroed\n", step, ipa_item);
+ } else {
+ printf("mpss: %u: the IPA's item %u could not be made\n", step, ipa_item);
+ }
+ }
+ }
+ glink_reset();
+ return true;
+}
+
+
+/* ---- the IPA's firmware, loaded before the modem (Android's pil "ipa_fws") -------------------------- */
+
+/*
+ * The IPA (the SoC's packet engine, the modem's path for data) has a small firmware of its own,
+ * which Android's IPA driver has TrustZone load before the modem starts (dt: qcom,ipa_fws, image
+ * 15, yupik_ipa_fws.mdt + .b*, into reserved-memory ipa_fw@8b700000 and ipa_gsi@8b710000). A modem
+ * that goes online without it dies in its LTE downlink task (23 October: "EX:kernel:0x0:enl2_dl",
+ * a null pointer, twice of two). Nothing else of the IPA is done here yet: no clock, no tables.
+ * modem_ipa_fw=0 leaves it out.
+ */
+#define PAS_IPA 15
+#define IPA_FW_MEMORY 0x8b700000u
+#define IPA_FW_MEMORY_SIZE 0x1a000u /* ipa_fw (0x10000) and ipa_gsi (0xa000), one after the other */
+
+
+/*
+ * What the IPA says of itself, read with its clock on: its shared memory register (where the local
+ * memory is), the microcontroller's area at the start of that memory (Linux's ipa_uc.c: command,
+ * response -- 1 once it has booted --, event, error, hardware state, interface version) and GSI's
+ * status (bit 0: the GSI firmware is enabled) and error log, for our execution environment.
+ */
+static struct block ipa_regs = { "ipa", 0x01e40000, 0x1000 };
+static struct block ipa_sram = { "ipa-shared", 0x01e50000, 0x4000 }; /* the local memory: 0x3000 bytes (SHARED_MEM_SIZE) */
+static struct block gsi_ee0 = { "gsi-ee0", 0x01e16000, 0x1000 };
+
+static void
+ipa_status(const char *when)
+{
+ uint32_t shared, uc0, uc3, uc5, uc7, uc8, gsi, error;
+
+ need(&ipa_regs);
+ need(&ipa_sram);
+ need(&gsi_ee0);
+ shared = rd(ipa_regs.base + 0x54);
+ uc0 = rd(ipa_sram.base + 0x00); /* command (byte 0) */
+ uc3 = rd(ipa_sram.base + 0x0c); /* response (byte 0) */
+ uc5 = rd(ipa_sram.base + 0x14); /* event (byte 0) */
+ uc7 = rd(ipa_sram.base + 0x1c); /* first error address */
+ uc8 = rd(ipa_sram.base + 0x20); /* hw_state (byte 0), warning counter (byte 1) */
+ gsi = rd(gsi_ee0.base + 0x000); /* GSI_STATUS */
+ error = rd(gsi_ee0.base + 0x200); /* ERROR_LOG */
+ tell("mpss: ipa: %s: shared memory %#x (offset %u, %u bytes); uC command %#x response %#x (1: booted) event %#x error address %#x "
+ "hw_state %u warnings %u, interface %#x; GSI status %#x (%s), error log %#x\n", when, shared, 8 * (shared >> 16),
+ 8 * (shared & 0xffff), uc0 & 0xff, uc3 & 0xff, uc5 & 0xff, uc7, uc8 & 0xff, (uc8 >> 8) & 0xff, rd(ipa_sram.base + 0x24) & 0xffff,
+ gsi, gsi & 1 ? "ENABLED" : "NOT ENABLED", error);
+ ipa_mem_offset = 8 * (shared >> 16);
+}
+
+
+/*
+ * The IPA's local memory as Linux's ipa_mem_config() and ipa_mem_setup() leave it before the modem
+ * is told of it: zeroed, with "canaries" (0xdeadbeef) in the words before the regions that have
+ * them (IPA v4.11: ipa_data-v4.11.c), and the register that says where the modem's processing
+ * contexts begin. The modem reads this memory for the microcontroller's state and its own tables;
+ * what it finds there after power-on is whatever was left.
+ */
+static void
+ipa_memory_init(void)
+{
+ static const struct {
+ uint16_t offset, canaries;
+ } regions[] = {
+ /* Android's _ipa_init_sram_v3() for IPA 4.11 (its ipa_4_11_mem_part): two before each table and the modem's
+ * headers and processing contexts, one 16 bytes before the NAT table (0x17a0) and the modem's quota statistics
+ * (0x1d00), two before the modem's region (0x27d8), one at the end */
+ { 0x0288, 2 }, { 0x0308, 2 }, { 0x0388, 2 }, { 0x0408, 2 }, { 0x0488, 2 }, { 0x0508, 2 }, { 0x0588, 2 }, { 0x0608, 2 },
+ { 0x0688, 2 }, { 0x0ad0, 2 }, { 0x1794, 1 }, { 0x1cf4, 1 }, { 0x27d8, 2 }, { 0x3000, 1 },
+ };
+ const uint32_t size = 0x3000;
+
+ need(&ipa_sram);
+ need(&ipa_regs);
+ /* from 0x288: the microcontroller's own area (its shared words at 0, its info at 0x80) is left alone, as Linux leaves it */
+ for (uint32_t at = 0x288; at < size; at += 4) {
+ wr(ipa_sram.base + ipa_mem_offset + at, 0);
+ }
+ for (unsigned r = 0; r < sizeof(regions) / sizeof(regions[0]); r++) {
+ for (unsigned c = 1; c <= regions[r].canaries; c++) {
+ wr(ipa_sram.base + ipa_mem_offset + regions[r].offset - 4 * c, 0xdeadbeef);
+ }
+ }
+ wr(ipa_regs.base + 0x1e8, ipa_mem_offset + 0x0ad0); /* LOCAL_PKT_PROC_CNTXT: the modem's processing contexts */
+ settle();
+ printf("mpss: ipa: its local memory zeroed (0x288..%#x at offset %#x), the canaries written, the processing contexts' base set\n",
+ size, ipa_mem_offset);
+}
+
+
+/*
+ * What Android's IPA driver programs into the IPA after its firmware starts and before the modem
+ * boots (techpack/dataipa, ipa3_post_init(): ipa3_init_hw(), ipa3_set_resorce_groups_min_max_limits(),
+ * ipa3_cfg_route()), as read from its source for IPA 4.11 and from the live registers of the phone's
+ * Android (debugfs gen_reg): the QSB masters' outstanding reads and writes, the Qtime timers, the
+ * GSI's in-order bits of COMP_CFG, the idle indication for clock division, the resource groups'
+ * limits (without them nothing can be processed: the modem's own commands to the IPA would never
+ * complete, and its answer to INIT_DRIVER never come), and the default route. Nothing of the
+ * application processor's own pipes: there is no data path on this side yet.
+ */
+#define IPA_COMP_CFG 0x3c
+#define IPA_ROUTE 0x48
+#define IPA_QSB_MAX_WRITES 0x74
+#define IPA_QSB_MAX_READS 0x78
+#define IPA_IDLE_INDICATION_CFG 0x240
+#define IPA_QTIME_TIMESTAMP_CFG 0x24c
+#define IPA_TIMERS_XO_CLK_DIV_CFG 0x250
+#define IPA_TIMERS_PULSE_GRAN_CFG 0x254
+#define IPA_SRC_RSRC_GRP_01(n) (0x400 + 0x20 * (n)) /* groups 0 and 1: x_min [5:0] x_max [13:8] y_min [21:16] y_max [29:24] */
+#define IPA_SRC_RSRC_GRP_23(n) (0x404 + 0x20 * (n))
+#define IPA_DST_RSRC_GRP_01(n) (0x500 + 0x20 * (n))
+#define IPA_DST_RSRC_GRP_23(n) (0x504 + 0x20 * (n))
+#define RSRC(xmin, xmax, ymin, ymax) ((uint32_t)(xmin) | (uint32_t)(xmax) << 8 | (uint32_t)(ymin) << 16 | (uint32_t)(ymax) << 24)
+
+static void
+ipa_hardware_init(void)
+{
+ static const uint8_t source[5] = { 6, 8, 18, 2, 15 }; /* packet contexts, descriptor lists, descriptor buffers, HPS DMARs, ACK entries: group 0 (UL/DL) */
+ uint32_t value;
+
+ need(&ipa_regs);
+ /* QSB: 12 outstanding writes, 13 reads of 120 beats on the DDR master (ipa3_qmb_outstanding[IPA_4_11]) */
+ value = rd(ipa_regs.base + IPA_QSB_MAX_WRITES);
+ wr(ipa_regs.base + IPA_QSB_MAX_WRITES, (value & ~0xfu) | 12);
+ value = rd(ipa_regs.base + IPA_QSB_MAX_READS);
+ wr(ipa_regs.base + IPA_QSB_MAX_READS, (value & ~0x00ff000fu) | 13 | 120u << 16);
+ /* Qtime: the divider off, the timestamps and pulse generators as Android leaves them, the divider 192 on */
+ wr(ipa_regs.base + IPA_TIMERS_XO_CLK_DIV_CFG, 0);
+ wr(ipa_regs.base + IPA_QTIME_TIMESTAMP_CFG, 0x180e8e);
+ wr(ipa_regs.base + IPA_TIMERS_PULSE_GRAN_CFG, 0x123);
+ wr(ipa_regs.base + IPA_TIMERS_XO_CLK_DIV_CFG, 0xbf);
+ wr(ipa_regs.base + IPA_TIMERS_XO_CLK_DIV_CFG, 0x800000bf);
+ /* COMP_CFG: GSI_MULTI_INORDER_RD_DIS and _WR_DIS */
+ value = rd(ipa_regs.base + IPA_COMP_CFG);
+ wr(ipa_regs.base + IPA_COMP_CFG, value | 1u << 7 | 1u << 8);
+ /* dynamic clock division: enter idle after 256, no constant non-idle */
+ wr(ipa_regs.base + IPA_IDLE_INDICATION_CFG, 256);
+ /* the resource groups: the sources' five types for group 0 alone; the destinations' data sectors 3 for group 0
+ * and 25 for group 2 (DRB_IP), DPS DMARs 2 for group 0 */
+ for (unsigned n = 0; n < 5; n++) {
+ wr(ipa_regs.base + IPA_SRC_RSRC_GRP_01(n), RSRC(source[n], source[n], 0, 0));
+ }
+ wr(ipa_regs.base + IPA_DST_RSRC_GRP_01(0), RSRC(3, 3, 0, 0));
+ wr(ipa_regs.base + IPA_DST_RSRC_GRP_23(0), RSRC(25, 25, 0, 0));
+ wr(ipa_regs.base + IPA_DST_RSRC_GRP_01(1), RSRC(2, 2, 0, 0));
+ wr(ipa_regs.base + IPA_DST_RSRC_GRP_23(1), 0);
+ /* the default route as Android's: not disabled, default pipe 9, header table 1, fragments to pipe 9, header retained */
+ wr(ipa_regs.base + IPA_ROUTE, 0x1120052);
+ settle();
+ printf("mpss: ipa: the hardware set up as Android's driver does before the modem: QSB %#x %#x, COMP_CFG %#x, idle %#x, route %#x, "
+ "source group 0 %u/%u/%u/%u/%u, destination sectors %#x %#x\n", rd(ipa_regs.base + IPA_QSB_MAX_WRITES),
+ rd(ipa_regs.base + IPA_QSB_MAX_READS), rd(ipa_regs.base + IPA_COMP_CFG), rd(ipa_regs.base + IPA_IDLE_INDICATION_CFG),
+ rd(ipa_regs.base + IPA_ROUTE), rd(ipa_regs.base + IPA_SRC_RSRC_GRP_01(0)) & 0x3f, rd(ipa_regs.base + IPA_SRC_RSRC_GRP_01(1)) & 0x3f,
+ rd(ipa_regs.base + IPA_SRC_RSRC_GRP_01(2)) & 0x3f, rd(ipa_regs.base + IPA_SRC_RSRC_GRP_01(3)) & 0x3f,
+ rd(ipa_regs.base + IPA_SRC_RSRC_GRP_01(4)) & 0x3f, rd(ipa_regs.base + IPA_DST_RSRC_GRP_01(0)), rd(ipa_regs.base + IPA_DST_RSRC_GRP_23(0)));
+}
+
+
+/*
+ * The IPA reaches memory through the application processors' SMMU (dt of Linux: iommus 0x480 and
+ * 0x482 of apps_smmu), where a stream nobody matches faults, and the hypervisor grants no bypass
+ * (tried on 24 October: the S2CR stayed "fault"). So, as the Wi-Fi's radio has (redwood_wifi.c,
+ * radio_space_on(), where the formats and the order of programming are explained): a translation
+ * context of our own, with the first free match register and the lowest free context, and page
+ * tables of ours. The IPA, though, is shown memory as it is: one table of 2 MB blocks maps the
+ * gigabyte of DDR from 0x80000000 (the shared memory, the modem's memory, our buffers) onto
+ * itself, uncached, and one block the SoC's internal memory at 0x146a8000 (Linux's ipa_mem_data:
+ * imem), as a device. Put back when the IPA is stopped (modem_ipa_smmu_off=1: not done at all).
+ *
+ * The hypervisor checks what is written to a match register against the masters it knows (the
+ * device tree's "iommus" entries, stream and mask): on 24 October one register with stream 0x480
+ * and the mask 0x2 (both streams at once) was refused -- the S2CR took, the SMR read back 0 --
+ * while the Wi-Fi's register, whose mask 1 is the device tree's, was granted. The device tree
+ * gives the IPA's streams the mask 0, so each takes a match register of its own, with the mask 0,
+ * both sending to the one context.
+ */
+#define IPA_STREAM 0x480u
+#define IPA_STREAM_UC 0x482u /* the IPA's microcontroller: its own match register */
+#define IPA_SPACE_BASE 0x80000000ull /* a gigabyte from here, as it is */
+#define IPA_IMEM 0x14600000ull /* the 2 MB block that holds 0x146a8000 */
+static struct block apps_smmu = { "apps smmu", 0x15000000, 0x2000 };
+static struct block apps_smmu_second = { "apps smmu", 0, 0x1000 };
+static struct block apps_smmu_context = { "apps smmu, the IPA's context", 0, 0x1000 };
+#define SMMU_SCR0 0x000
+#define SMMU_IDR0 0x020
+#define SMMU_IDR1 0x024
+#define SMMU_SMR(n) (0x800 + 4 * (n))
+#define SMMU_S2CR(n) (0xc00 + 4 * (n))
+#define SMMU_SGFSR 0x048
+#define SMMU_SGFSYNR0 0x050
+#define SMMU_SGFSYNR1 0x054
+#define SMMU_CBAR(n) (4 * (n))
+#define SMMU_CBAR_STAGE_1 (1u << 16 | 3u << 8 | 0xfu << 12)
+#define SMMU_CBA2R(n) (0x800 + 4 * (n))
+#define SMMU_CBA2R_64_BIT 1
+#define SMMU_CB_SCTLR 0x00
+#define SMMU_CB_TCR2 0x10
+#define SMMU_CB_TTBR0 0x20
+#define SMMU_CB_TTBR1 0x28
+#define SMMU_CB_TCR 0x30
+#define SMMU_CB_MAIR0 0x38
+#define SMMU_CB_MAIR1 0x3c
+#define SMMU_CB_FSR 0x58
+#define SMMU_CB_FAR 0x60
+#define SMMU_CB_FSYNR0 0x68
+#define SMMU_CB_TLBIASID 0x610
+#define SMMU_CB_TLBSYNC 0x7f0
+#define SMMU_CB_TLBSTATUS 0x7f4
+#define SMMU_SCTLR_ON (1u << 0 | 1u << 1 | 1u << 2 | 1u << 5 | 1u << 8 | 1u << 12)
+#define SMMU_TCR_39_BITS (25u | 1u << 8 | 1u << 10 | 3u << 12 | 1u << 23)
+#define SMMU_TCR2 (1u | 1u << 4 | 7u << 15)
+#define SMMU_MAIR0 0xf404ff44 /* 0 uncached, 1 cached, 2 device */
+#define SMMU_FSR_ALL 0xc00001fe
+#define IPA_BLOCK (1ull | 0ull << 2 | 1ull << 6 | 2ull << 8 | 1ull << 10 | 1ull << 11) /* a 2 MB block, uncached, RW, shared */
+#define IPA_BLOCK_DEVICE (1ull | 2ull << 2 | 1ull << 6 | 2ull << 8 | 1ull << 10 | 1ull << 11)
+#define IPA_TABLE 3ull /* a table entry of the root */
+extern lck_mtx_t redwood_smmu_lock;
+static struct {
+ bool on;
+ uint32_t page, pages, match, match_uc, context, old_smr, old_s2cr, old_smr_uc, old_s2cr_uc, old_sctlr;
+ vm_offset_t tables; /* three pages: the root, the DDR gigabyte's blocks, the SoC's */
+} ipa_space;
+
+static void
+ipa_space_faults(const char *when)
+{
+ uint32_t status, global;
+
+ if (!ipa_space.on) {
+ return;
+ }
+ status = rd(apps_smmu_context.base + SMMU_CB_FSR);
+ global = rd(apps_smmu.base + SMMU_SGFSR);
+ if ((status & SMMU_FSR_ALL) == 0 && global == 0) {
+ printf("mpss: ipa: %s: the SMMU has refused the IPA nothing (context %u)\n", when, ipa_space.context);
+ return;
+ }
+ tell("mpss: ipa: %s: THE SMMU HAS REFUSED A REQUEST OF THE IPA'S: context %u: status %#x, address %#llx, syndrome %#x; the SMMU "
+ "as a whole: status %#x, syndromes %#x %#x\n", when, ipa_space.context, status,
+ (uint64_t)rd(apps_smmu_context.base + SMMU_CB_FAR + 4) << 32 | rd(apps_smmu_context.base + SMMU_CB_FAR),
+ rd(apps_smmu_context.base + SMMU_CB_FSYNR0), global, rd(apps_smmu.base + SMMU_SGFSYNR0), rd(apps_smmu.base + SMMU_SGFSYNR1));
+ wr(apps_smmu_context.base + SMMU_CB_FSR, SMMU_FSR_ALL);
+}
+
+static bool
+ipa_space_on(void)
+{
+ bool used[256] = { false };
+ uint32_t scr0, idr0, idr1, matches, contexts, sees = 1u << 31 | IPA_STREAM, sees_uc = 1u << 31 | IPA_STREAM_UC;
+ uint64_t *root, *ddr, *soc, ttbr;
+
+ if (ipa_space.on) {
+ return true;
+ }
+ lck_mtx_lock(&redwood_smmu_lock);
+ need(&apps_smmu);
+ scr0 = rd(apps_smmu.base + SMMU_SCR0);
+ idr0 = rd(apps_smmu.base + SMMU_IDR0);
+ idr1 = rd(apps_smmu.base + SMMU_IDR1);
+ matches = idr0 & 0xff;
+ contexts = idr1 & 0xff;
+ ipa_space.page = idr1 >> 31 ? 0x10000 : 0x1000;
+ ipa_space.pages = 2u << (idr1 >> 28 & 7);
+ if ((scr0 & 1u << 3) || 2 * ipa_space.pages * ipa_space.page != 0x100000) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ printf("mpss: ipa: the SMMU is not of the shape this knows (scr0 %#x, idr1 %#x)\n", scr0, idr1);
+ return false;
+ }
+ ipa_space.match = ipa_space.match_uc = ~0u;
+ for (uint32_t n = 0; n < matches && SMMU_SMR(n) < 0xc00; n++) {
+ uint32_t smr = rd(apps_smmu.base + SMMU_SMR(n)), s2cr = rd(apps_smmu.base + SMMU_S2CR(n));
+
+ if (smr >> 31) {
+ used[s2cr & 0xff] = true;
+ } else if (ipa_space.match == ~0u) {
+ ipa_space.match = n;
+ ipa_space.old_smr = smr;
+ ipa_space.old_s2cr = s2cr;
+ } else if (ipa_space.match_uc == ~0u) {
+ ipa_space.match_uc = n;
+ ipa_space.old_smr_uc = smr;
+ ipa_space.old_s2cr_uc = s2cr;
+ }
+ }
+ for (ipa_space.context = 0; ipa_space.context < contexts && used[ipa_space.context]; ipa_space.context++) {
+ }
+ if (ipa_space.match == ~0u || ipa_space.match_uc == ~0u || ipa_space.context >= contexts) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ printf("mpss: ipa: the SMMU has not two match registers or no context left\n");
+ return false;
+ }
+ apps_smmu_second.phys = apps_smmu.phys + ipa_space.page;
+ apps_smmu_context.phys = apps_smmu.phys + (ipa_space.pages + ipa_space.context) * ipa_space.page;
+ need(&apps_smmu_second);
+ need(&apps_smmu_context);
+ ipa_space.old_sctlr = rd(apps_smmu_context.base + SMMU_CB_SCTLR);
+ if (ipa_space.old_sctlr & 1) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ printf("mpss: ipa: context %u of the SMMU is switched on although no stream goes through it: left alone\n", ipa_space.context);
+ return false;
+ }
+ if (ipa_space.tables == 0 &&
+ kmem_alloc(kernel_map, &ipa_space.tables, 3 * 4096, KMA_KOBJECT | KMA_ZERO | KMA_DATA, VM_KERN_MEMORY_IOKIT) != KERN_SUCCESS) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ return false;
+ }
+ root = (uint64_t *)ipa_space.tables;
+ ddr = (uint64_t *)(ipa_space.tables + 4096);
+ soc = (uint64_t *)(ipa_space.tables + 8192);
+ bzero(root, 3 * 4096);
+ for (unsigned i = 0; i < 512; i++) {
+ ddr[i] = (IPA_SPACE_BASE + (uint64_t)i * 0x200000) | IPA_BLOCK;
+ }
+ soc[(IPA_IMEM >> 21) & 0x1ff] = IPA_IMEM | IPA_BLOCK_DEVICE;
+ root[IPA_SPACE_BASE >> 30] = kvtophys((vm_offset_t)ddr) | IPA_TABLE;
+ root[IPA_IMEM >> 30] = kvtophys((vm_offset_t)soc) | IPA_TABLE;
+ FlushPoC_DcacheRegion(ipa_space.tables, 3 * 4096);
+ settle();
+ ttbr = kvtophys(ipa_space.tables) | (uint64_t)ipa_space.context << 48;
+ printf("mpss: ipa: THE SMMU IS ABOUT TO BE WRITTEN: context %u gets our tables (root %#llx: DDR from %#llx as it is, 1 GB, and "
+ "the SoC's memory at %#llx) and match registers %u and %u send streams %#x and %#x through it (before: smr %#x s2cr %#x, "
+ "smr %#x s2cr %#x, sctlr %#x)\n", ipa_space.context, ttbr & 0x0000fffffffff000ull, IPA_SPACE_BASE, IPA_IMEM, ipa_space.match,
+ ipa_space.match_uc, IPA_STREAM, IPA_STREAM_UC, ipa_space.old_smr, ipa_space.old_s2cr, ipa_space.old_smr_uc, ipa_space.old_s2cr_uc,
+ ipa_space.old_sctlr);
+ wr(apps_smmu_context.base + SMMU_CB_SCTLR, 0);
+ wr(apps_smmu_second.base + SMMU_CBA2R(ipa_space.context), SMMU_CBA2R_64_BIT);
+ wr(apps_smmu_second.base + SMMU_CBAR(ipa_space.context), SMMU_CBAR_STAGE_1);
+ wr(apps_smmu_context.base + SMMU_CB_TCR2, SMMU_TCR2);
+ wr(apps_smmu_context.base + SMMU_CB_TCR, SMMU_TCR_39_BITS);
+ wr(apps_smmu_context.base + SMMU_CB_TTBR0, (uint32_t)ttbr);
+ wr(apps_smmu_context.base + SMMU_CB_TTBR0 + 4, (uint32_t)(ttbr >> 32));
+ wr(apps_smmu_context.base + SMMU_CB_TTBR1, 0);
+ wr(apps_smmu_context.base + SMMU_CB_TTBR1 + 4, 0);
+ wr(apps_smmu_context.base + SMMU_CB_MAIR0, SMMU_MAIR0);
+ wr(apps_smmu_context.base + SMMU_CB_MAIR1, 0);
+ wr(apps_smmu_context.base + SMMU_CB_FSR, SMMU_FSR_ALL);
+ wr(apps_smmu_context.base + SMMU_CB_TLBIASID, ipa_space.context);
+ wr(apps_smmu_context.base + SMMU_CB_TLBSYNC, 1);
+ for (unsigned ms = 0; rd(apps_smmu_context.base + SMMU_CB_TLBSTATUS) & 1; ms++) {
+ if (ms == 100) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ printf("mpss: ipa: the SMMU does not finish forgetting old translations\n");
+ return false;
+ }
+ sleep_ms(1);
+ }
+ wr(apps_smmu_context.base + SMMU_CB_SCTLR, SMMU_SCTLR_ON);
+ ipa_space.on = true;
+ wr(apps_smmu.base + SMMU_S2CR(ipa_space.match), ipa_space.context);
+ wr(apps_smmu.base + SMMU_SMR(ipa_space.match), sees);
+ wr(apps_smmu.base + SMMU_S2CR(ipa_space.match_uc), ipa_space.context);
+ wr(apps_smmu.base + SMMU_SMR(ipa_space.match_uc), sees_uc);
+ if (rd(apps_smmu.base + SMMU_SMR(ipa_space.match)) != sees ||
+ (rd(apps_smmu.base + SMMU_S2CR(ipa_space.match)) & 0x300ff) != ipa_space.context ||
+ rd(apps_smmu.base + SMMU_SMR(ipa_space.match_uc)) != sees_uc ||
+ (rd(apps_smmu.base + SMMU_S2CR(ipa_space.match_uc)) & 0x300ff) != ipa_space.context ||
+ !(rd(apps_smmu_context.base + SMMU_CB_SCTLR) & 1) || rd(apps_smmu_context.base + SMMU_CB_TTBR0) != (uint32_t)ttbr) {
+ lck_mtx_unlock(&redwood_smmu_lock);
+ printf("mpss: ipa: THE SMMU DID NOT TAKE WHAT WAS WRITTEN: streams %#x and %#x are not both going through context %u (smr %#x "
+ "s2cr %#x, smr %#x s2cr %#x, sctlr %#x)\n", IPA_STREAM, IPA_STREAM_UC, ipa_space.context,
+ rd(apps_smmu.base + SMMU_SMR(ipa_space.match)), rd(apps_smmu.base + SMMU_S2CR(ipa_space.match)),
+ rd(apps_smmu.base + SMMU_SMR(ipa_space.match_uc)), rd(apps_smmu.base + SMMU_S2CR(ipa_space.match_uc)),
+ rd(apps_smmu_context.base + SMMU_CB_SCTLR));
+ return false;
+ }
+ lck_mtx_unlock(&redwood_smmu_lock);
+ tell("mpss: ipa: THE IPA'S STREAMS GO THROUGH CONTEXT %u OF THE SMMU, with our tables (match registers %u and %u)\n", ipa_space.context,
+ ipa_space.match, ipa_space.match_uc);
+ return true;
+}
+
+static void
+ipa_space_off(void)
+{
+ if (!ipa_space.on) {
+ return;
+ }
+ ipa_space_faults("at the end");
+ lck_mtx_lock(&redwood_smmu_lock);
+ wr(apps_smmu.base + SMMU_SMR(ipa_space.match), ipa_space.old_smr);
+ wr(apps_smmu.base + SMMU_S2CR(ipa_space.match), ipa_space.old_s2cr);
+ wr(apps_smmu.base + SMMU_SMR(ipa_space.match_uc), ipa_space.old_smr_uc);
+ wr(apps_smmu.base + SMMU_S2CR(ipa_space.match_uc), ipa_space.old_s2cr_uc);
+ wr(apps_smmu_context.base + SMMU_CB_SCTLR, ipa_space.old_sctlr);
+ lck_mtx_unlock(&redwood_smmu_lock);
+ ipa_space.on = false;
+ printf("mpss: ipa: the IPA's streams and context %u of the SMMU are put back as they were found\n", ipa_space.context);
+}
+
+static bool
+ipa_firmware_load(void)
+{
+ const uint8_t *header, *segment_file[8] = { NULL };
+ uint64_t header_size = 0, segment_size[8] = { 0 }, answer = ~0ull;
+ struct elf_segment segments[8];
+ volatile uint8_t *memory;
+ vm_offset_t page = 0;
+ uint32_t count, loaded = 0, highest = 0, size;
+ int32_t result;
+ bool ok = false;
+
+ announce("turn the IPA's clock on (RPMh), load the IPA's firmware (image 15) into its memory at 0x8b700000 and have TrustZone "
+ "start it, as Android does before the modem");
+ if (!ipa_clock(true)) {
+ printf("mpss: %u: the IPA's clock is not on: its firmware is not loaded (a start without the clock ends the kernel)\n", step);
+ return false;
+ }
+ if (!option("modem_ipa_smmu_off")) {
+ ipa_space_on();
+ }
+ header = root_file("image/yupik_ipa_fws.mdt", &header_size);
+ if (header == NULL) {
+ printf("mpss: %u: no fw/image/yupik_ipa_fws.mdt: the IPA's firmware is not loaded\n", step);
+ return false;
+ }
+ count = header_size >= 52 ? header[44] | header[45] << 8 : 0;
+ if (memcmp(header, "\177ELF\1", 5) != 0 || count == 0 || count > 8 || 52 + 32 * count > header_size) {
+ printf("mpss: %u: yupik_ipa_fws.mdt is not such a header (%u segments)\n", step, count);
+ goto out;
+ }
+ /* (the image is relocatable, bit 27 of the flags: its addresses count from the lowest, which goes at the region's start) */
+ for (uint32_t i = 0; i < count; i++) {
+ memcpy(&segments[i], header + 52 + 32 * i, sizeof(segments[i]));
+ }
+ {
+ uint32_t lowest = ~0u;
+ bool relocatable = false;
+
+ for (uint32_t i = 0; i < count; i++) {
+ if (segments[i].type == 1 && !MDT_HASH(segments[i].flags) && segments[i].memory_size != 0) {
+ relocatable |= (segments[i].flags & (1u << 27)) != 0;
+ lowest = segments[i].paddr < lowest ? segments[i].paddr : lowest;
+ }
+ }
+ if (relocatable && lowest != ~0u) {
+ for (uint32_t i = 0; i < count; i++) {
+ segments[i].paddr = segments[i].paddr - lowest + IPA_FW_MEMORY;
+ }
+ }
+ }
+ for (uint32_t i = 0; i < count; i++) {
+ char name[48];
+
+ if (segments[i].type != 1 || MDT_HASH(segments[i].flags) || segments[i].memory_size == 0) {
+ continue;
+ }
+ if (segments[i].paddr < IPA_FW_MEMORY || (uint64_t)segments[i].paddr + segments[i].memory_size > IPA_FW_MEMORY + IPA_FW_MEMORY_SIZE) {
+ printf("mpss: %u: the IPA's segment %u (%#x, %#x bytes) is not within its memory\n", step, i, segments[i].paddr,
+ segments[i].memory_size);
+ goto out;
+ }
+ if (segments[i].file_size != 0) {
+ snprintf(name, sizeof(name), "image/yupik_ipa_fws.b%02u", i);
+ segment_file[i] = root_file(name, &segment_size[i]);
+ if (segment_file[i] == NULL || segment_size[i] != segments[i].file_size) {
+ printf("mpss: %u: fw/%s: %s\n", step, name, segment_file[i] == NULL ? "not there" : "not the size the header says");
+ goto out;
+ }
+ }
+ if (segments[i].paddr + segments[i].memory_size > highest) {
+ highest = segments[i].paddr + segments[i].memory_size;
+ }
+ loaded++;
+ }
+ size = ((highest + 0xfff) & ~0xfffu) - IPA_FW_MEMORY;
+ printf("mpss: %u: the IPA's image: %u segments to load, %#x bytes from %#x\n", step, loaded, size, IPA_FW_MEMORY);
+ if (kmem_alloc(kernel_map, &page, PAGE_SIZE, KMA_KOBJECT | KMA_ZERO | KMA_DATA, VM_KERN_MEMORY_IOKIT) != KERN_SUCCESS) {
+ goto out;
+ }
+ memcpy((void *)page, header, header_size < PAGE_SIZE ? header_size : PAGE_SIZE);
+ for (vm_offset_t line = page; line < page + PAGE_SIZE; line += 64) {
+ __asm__ volatile ("dc civac, %0" : : "r" (line));
+ }
+ settle();
+ result = scm("to take the IPA's header", SCM_PAS_INIT_IMAGE, SCM_VALUE_AND_BUFFER, PAS_IPA, kvtophys(page), 0, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT TAKE THE IPA'S HEADER (%d, %#llx)\n", step, result, answer);
+ goto out;
+ }
+ result = scm("where the IPA's image will be", SCM_PAS_MEM_SETUP, SCM_THREE_VALUES, PAS_IPA, IPA_FW_MEMORY, size, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT TAKE THE IPA'S PLACE (%d, %#llx)\n", step, result, answer);
+ goto out;
+ }
+ memory = (volatile uint8_t *)ml_io_map_unmappable(IPA_FW_MEMORY, size, VM_WIMG_WCOMB);
+ if (memory == NULL) {
+ goto out;
+ }
+ for (uint32_t i = 0; i < count; i++) {
+ uint32_t at = segments[i].paddr - IPA_FW_MEMORY;
+
+ if (segments[i].type != 1 || MDT_HASH(segments[i].flags) || segments[i].memory_size == 0) {
+ continue;
+ }
+ for (uint32_t b = 0; b < segments[i].memory_size; b++) {
+ memory[at + b] = b < segments[i].file_size ? segment_file[i][b] : 0;
+ }
+ }
+ settle();
+ ml_io_unmap((vm_offset_t)memory, size);
+ result = scm("to authenticate the IPA's image and start it", SCM_PAS_AUTH_AND_RESET, SCM_ONE_VALUE, PAS_IPA, 0, 0, 0, &answer);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT START THE IPA'S FIRMWARE (%d, %#llx)\n", step, result, answer);
+ goto out;
+ }
+ tell("mpss: %u: THE IPA'S FIRMWARE IS LOADED AND STARTED (image 15, %u segments)\n", step, loaded);
+ sleep_ms(50);
+ if (!option("modem_ipa_quiet")) {
+ ipa_status("after its firmware's start");
+ }
+ if (!option("modem_ipa_raw")) {
+ ipa_memory_init();
+ ipa_hardware_init();
+ }
+ ok = true;
+out:
+ if (page != 0) {
+ kmem_free(kernel_map, page, PAGE_SIZE);
+ }
+ {
+ uint8_t *gone = __DECONST(uint8_t *, header);
+
+ kfree_data(gone, header_size);
+ }
+ for (uint32_t i = 0; i < 8; i++) {
+ if (segment_file[i] != NULL) {
+ uint8_t *gone = __DECONST(uint8_t *, segment_file[i]);
+
+ kfree_data(gone, segment_size[i]);
+ }
+ }
+ return ok;
+}
+
+static void
+ipa_firmware_stop(void)
+{
+ uint64_t answer = ~0ull;
+
+ (void)scm("to shut the IPA's firmware down", SCM_PAS_SHUTDOWN, SCM_ONE_VALUE, PAS_IPA, 0, 0, 0, &answer);
+ if (!leaving) {
+ ipa_space_off();
+ ipa_clock(false);
+ }
+}
+
+/* ---- step 7: the DSP, started ---------------------------------------------------------------------- */
+
+/*
+ * Android's kernel (this phone, dmesg): the IPA's firmware first, then modem.mdt and its segments
+ * into 0x8b800000..0x9ae00000, "Power/Clock ready", and QRTR's "ready" after that. Here the modem
+ * is watched for as the audio DSP is (redwood_adsp.c, step 7): its SMP2P item, its doorbells, its
+ * "slave-kernel" word, and its last words in item 421 if it gives up.
+ */
+static bool
+step_start(void)
+{
+ struct smp2p words, last;
+ uint64_t answer = ~0ull, started, now, ready_at = 0, limit;
+ uint32_t state = 0, rang, changes = 0, wait_seconds = 10;
+ int32_t result;
+ bool have, spoke = false, handed_over = false;
+
+ if (!image_written) {
+ printf("mpss: %u: the image is not in place\n", step);
+ return false;
+ }
+ PE_parse_boot_argn("modem_wait", &wait_seconds, sizeof(wait_seconds));
+ bzero(&last, sizeof(last));
+ have = dsp_words(&last);
+ announce("listen for the modem's doorbells, ask RPMh for its rails, tell the always-on processor that its image is "
+ "loaded, and ask TrustZone to authenticate the image and let the DSP out of reset");
+ need(&ipcc);
+ for (unsigned i = 0; i < 8 && (rang = rd(ipcc.base + IPCC_RECEIVED)) != IPCC_NOBODY && rang >> 16 == IPCC_MPSS; i++) {
+ put(&ipcc, "heard", IPCC_CLEAR, rang);
+ }
+ put(&ipcc, "listened for", IPCC_ENABLE, IPCC_MPSS << 16 | IPCC_SMP2P);
+ put(&ipcc, "listened for", IPCC_ENABLE, IPCC_MPSS << 16 | IPCC_GLINK);
+ bzero(&doorbell, sizeof(doorbell));
+ lck_mtx_lock(&dsp_lock);
+ {
+ uint32_t ipa_fw = 1;
+
+ PE_parse_boot_argn("modem_ipa_fw", &ipa_fw, sizeof(ipa_fw));
+ ipa_loaded = ipa_fw != 0 && ipa_firmware_load();
+ }
+ if (option("modem_share_off")) {
+ printf("mpss: %u: the buffer is not shared with the modem (modem_share_off)\n", step);
+ } else if (!rmtfs_buffer_make() || !rmtfs_share(true)) {
+ printf("mpss: %u: THE MODEM GETS NO BUFFER FOR ITS FILES: it is started all the same, and will offer little\n", step);
+ }
+ if (!rails(384)) {
+ lck_mtx_unlock(&dsp_lock);
+ return false;
+ }
+ rails_asked = true;
+ if (!mailbox_send("{class: image, res: load_state, name: modem, val: on}")) {
+ rails(0);
+ rails_asked = false;
+ lck_mtx_unlock(&dsp_lock);
+ return false;
+ }
+ result = scm("to authenticate the modem's image and start it", SCM_PAS_AUTH_AND_RESET, SCM_ONE_VALUE, PAS_MPSS, 0, 0, 0,
+ &answer);
+ doorbell.since = started = now_us();
+ image_written = false;
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT START THE MODEM: the call returns %d, its answer is %#llx\n", step, result,
+ answer);
+ mailbox_send("{class: image, res: load_state, name: modem, val: off}");
+ rails(0);
+ rails_asked = false;
+ lck_mtx_unlock(&dsp_lock);
+ return false;
+ }
+ dsp_started = true;
+ lck_mtx_unlock(&dsp_lock);
+ printf("mpss: %u: TRUSTZONE HAS AUTHENTICATED THE IMAGE AND LET THE MODEM OUT OF RESET; watching for %u s\n", step,
+ wait_seconds);
+ limit = started + wait_seconds * 1000000ull;
+ for (;;) {
+ now = now_us();
+ if (now >= limit || !dsp_started) {
+ break;
+ }
+ doorbells();
+ spoke = spoke || doorbell.glink + doorbell.smp2p != 0;
+ if (dsp_words(&words) && (!have || memcmp(&words, &last, sizeof(words)) != 0)) {
+ char when[64];
+
+ snprintf(when, sizeof(when), "+%llu.%03llu ms: the DSP's item %s", (now - started) / 1000, (now - started) % 1000,
+ have ? "changes" : "APPEARS");
+ if (changes++ < 40) {
+ show_words(when, &words);
+ }
+ last = words;
+ have = spoke = true;
+ dsp_listen(&words);
+ state = dsp_state(&words);
+ if ((state & DSP_READY) && ready_at == 0) {
+ ready_at = now;
+ limit = now + 3000000ull;
+ printf("mpss: %u: THE MODEM SAYS IT IS UP: \"slave-kernel\" says %#x, %llu.%03llu ms after TrustZone started "
+ "it\n", step, state, (now - started) / 1000, (now - started) % 1000);
+ }
+ if ((state & DSP_HANDOVER) && !handed_over) {
+ handed_over = true;
+ printf("mpss: %u: the DSP says it has asked for its own power (bit 2), %llu.%03llu ms after the start: ours is "
+ "given back\n", step, (now - started) / 1000, (now - started) % 1000);
+ lck_mtx_lock(&dsp_lock);
+ if (rails_asked) {
+ rails(0);
+ rails_asked = false;
+ }
+ lck_mtx_unlock(&dsp_lock);
+ }
+ if (state & DSP_FATAL) {
+ printf("mpss: %u: THE MODEM SAYS IT HAS GIVEN UP\n", step);
+ show_crash_reason();
+ break;
+ }
+ }
+ sleep_us(200);
+ }
+ printf("mpss: %u: the watch is over after %llu ms; its doorbells heard: GLINK %u, SMP2P %u\n", step,
+ (now_us() - started) / 1000, doorbell.glink, doorbell.smp2p);
+ show_partition();
+ if (our_item != 0 && smem_peek(our_item, &words, sizeof(words))) {
+ show_words("our item (428) now", &words);
+ }
+ {
+ /* (what is at the bottom of our partition, entry by entry, as it is now) */
+ uint32_t cacheline = 0, size = 0, partition = smem_partition(SMEM_US, SMEM_MPSS, &cacheline, &size);
+ struct smem_partition header;
+ struct smem_entry entry;
+
+ if (partition != 0 && smem_peek(partition, &header, sizeof(header))) {
+ printf("mpss: %u: the partition's header: hosts %u and %u, size %#x, free from the bottom at +%#x, from the top at "
+ "+%#x; the table's line size %u\n", step, header.host0, header.host1, header.size, header.free_uncached,
+ header.free_cached, cacheline);
+ for (uint32_t at = sizeof(header), n = 0; at + sizeof(entry) <= header.size && n < 8; n++) {
+ smem_peek(partition + at, &entry, sizeof(entry));
+ printf("mpss: %u: +%#x: canary %#x item %u size %u padding %u/%u\n", step, at, entry.canary, entry.item,
+ entry.size, entry.padding_data, entry.padding_header);
+ if (entry.canary != SMEM_CANARY) {
+ break;
+ }
+ at += sizeof(entry) + entry.padding_header + entry.size;
+ }
+ }
+ }
+ if (state & DSP_FATAL) {
+ return false;
+ }
+ if (ready_at == 0) {
+ printf("mpss: %u: %s\n", step, spoke ? "THE MODEM RUNS BUT HAS NOT SAID \"UP\"" : "NOT A WORD FROM THE MODEM");
+ show_crash_reason();
+ }
+ return ready_at != 0;
+}
+
+/* ---- GLINK (redwood_wifi.c; items 478 to 480 of our partition with the modem) ------------------- */
+
+#define ITEM_GLINK_DESCRIPTOR 478
+#define ITEM_GLINK_OUR_QUEUE 479
+#define ITEM_GLINK_THEIR_QUEUE 480
+#define GLINK_OUR_TAIL 0
+#define GLINK_OUR_HEAD 4
+#define GLINK_THEIR_TAIL 8
+#define GLINK_THEIR_HEAD 12
+#define GLINK_RESERVE 16
+
+enum {
+ GLINK_VERSION = 0, GLINK_VERSION_ACK = 1, GLINK_OPEN = 2, GLINK_CLOSE = 3, GLINK_OPEN_ACK = 4, GLINK_INTENT = 5,
+ GLINK_DONE = 6, GLINK_INTENT_REQUEST = 7, GLINK_INTENT_REQUEST_ACK = 8, GLINK_DATA = 9, GLINK_CLOSE_ACK = 11,
+ GLINK_DATA_MORE = 12, GLINK_READ_NOTIFY = 13, GLINK_DONE_REUSE = 14, GLINK_SIGNALS = 15,
+};
+#define GLINK_VERSION_1 1
+#define GLINK_INTENT_REUSE 1u
+
+struct glink_message {
+ uint16_t command, param1;
+ uint32_t param2;
+};
+struct glink_packet {
+ struct glink_packet *next;
+ uint32_t length;
+ uint8_t data[];
+};
+
+/* The one channel we answer when the modem opens it: IPCRTR, which carries QRTR (dt: qcom,intents of qcom,modem_qrtr). */
+enum { CHANNEL_QRTR, CHANNELS };
+
+struct glink_channel {
+ const char *name;
+ uint32_t ours; /* our number for it */
+ struct {
+ uint32_t size, count;
+ } intents[3];
+ uint32_t theirs; /* the DSP's number for it; 0: it has not opened it */
+ bool asked, acknowledged, open; /* our open sent; acknowledged; both ways done and our intents told */
+ bool request_answered, request_granted; /* our last request for an intent */
+ struct {
+ uint32_t id, size;
+ bool there, in_use;
+ } their_intents[64]; /* what we may send into */
+ struct {
+ uint32_t size, fill;
+ uint8_t *data;
+ } our_intents[24]; /* what the DSP may send into; an intent's number is its place here + 1 */
+ uint32_t our_intent_count;
+ struct glink_packet *first, *last; /* received, in order */
+};
+
+static LCK_MTX_DECLARE(glink_lock, &mpss_lock_group);
+static struct {
+ uint32_t descriptor, ours, ours_size, theirs, theirs_size;
+ bool up, broken;
+ uint32_t features;
+ bool agreed;
+ struct glink_channel channel[CHANNELS];
+ uint32_t said, heard, trace;
+ char others[4][33]; /* the channels the DSP opens that nobody here speaks (for the log) */
+} glink = {
+ .channel = {
+ [CHANNEL_QRTR] = { .name = "IPCRTR", .ours = 1, .intents = { { 0x800, 5 }, { 0x2000, 3 }, { 0x4400, 2 } } },
+ },
+};
+
+static struct glink_channel *
+glink_theirs(uint32_t theirs)
+{
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ if (glink.channel[c].theirs != 0 && glink.channel[c].theirs == theirs) {
+ return &glink.channel[c];
+ }
+ }
+ return NULL;
+}
+
+static bool
+glink_find(void)
+{
+ uint32_t size = 0;
+
+ glink.descriptor = smem_item(ITEM_GLINK_DESCRIPTOR, &size);
+ if (glink.descriptor == 0 || size != 32) {
+ return false;
+ }
+ glink.ours = smem_item(ITEM_GLINK_OUR_QUEUE, &glink.ours_size);
+ glink.theirs = smem_item(ITEM_GLINK_THEIR_QUEUE, &glink.theirs_size);
+ return glink.ours != 0 && glink.theirs != 0 && glink.ours_size >= 1024 && glink.theirs_size >= 1024 &&
+ !(glink.ours_size & 7) && !(glink.theirs_size & 7);
+}
+
+static void
+glink_reset(void)
+{
+ uint32_t size = 0, descriptor = smem_item(ITEM_GLINK_DESCRIPTOR, &size);
+
+ if (descriptor == 0 || size != 32) {
+ printf("mpss: %u: GLINK's descriptor (item %u) is not there: nothing to reset\n", step, ITEM_GLINK_DESCRIPTOR);
+ return;
+ }
+ printf("mpss: %u: GLINK's descriptor at %#lx: our queue read to %#x and written to %#x, the DSP's read to %#x and written "
+ "to %#x\n", step, SMEM_PHYS + descriptor, smem_word(descriptor + GLINK_OUR_TAIL), smem_word(descriptor + GLINK_OUR_HEAD),
+ smem_word(descriptor + GLINK_THEIR_TAIL), smem_word(descriptor + GLINK_THEIR_HEAD));
+ if (smem_word(descriptor + GLINK_OUR_HEAD) != 0 || smem_word(descriptor + GLINK_THEIR_TAIL) != 0) {
+ printf("mpss: %u: about to write our two marks there (+4, +8): 0\n", step);
+ smem_poke(descriptor + GLINK_OUR_HEAD, 0);
+ smem_poke(descriptor + GLINK_THEIR_TAIL, 0);
+ settle();
+ }
+}
+
+static uint32_t
+glink_waiting(void)
+{
+ uint32_t head = smem_word(glink.descriptor + GLINK_THEIR_HEAD), tail = smem_word(glink.descriptor + GLINK_THEIR_TAIL);
+
+ if (head >= glink.theirs_size || tail >= glink.theirs_size) {
+ return 0;
+ }
+ return head >= tail ? head - tail : glink.theirs_size - tail + head;
+}
+
+static void
+glink_peek(void *to, uint32_t offset, uint32_t count)
+{
+ uint32_t at = (smem_word(glink.descriptor + GLINK_THEIR_TAIL) + offset) % glink.theirs_size;
+
+ for (uint32_t i = 0; i < count; i++) {
+ ((uint8_t *)to)[i] = smem[glink.theirs + (at + i) % glink.theirs_size];
+ }
+}
+
+static void
+glink_advance(uint32_t count)
+{
+ smem_poke(glink.descriptor + GLINK_THEIR_TAIL, (smem_word(glink.descriptor + GLINK_THEIR_TAIL) + count) % glink.theirs_size);
+}
+
+static uint32_t
+glink_room(void)
+{
+ uint32_t head = smem_word(glink.descriptor + GLINK_OUR_HEAD), tail = smem_word(glink.descriptor + GLINK_OUR_TAIL), room;
+
+ if (head >= glink.ours_size || tail >= glink.ours_size) {
+ return 0;
+ }
+ room = tail <= head ? glink.ours_size - head + tail : tail - head;
+ return room < GLINK_RESERVE ? 0 : room - GLINK_RESERVE;
+}
+
+static bool
+glink_put(const void *first, uint32_t first_length, const void *second, uint32_t second_length)
+{
+ volatile uint8_t *queue = (volatile uint8_t *)(uintptr_t)smem + glink.ours;
+ uint32_t head = smem_word(glink.descriptor + GLINK_OUR_HEAD), length = first_length + second_length;
+ uint32_t padded = (length + 7) & ~7u;
+
+ if (!glink.up || glink_room() < padded) {
+ return false;
+ }
+ for (uint32_t i = 0; i < padded; i++) {
+ queue[(head + i) % glink.ours_size] = i < first_length ? ((const uint8_t *)first)[i] :
+ i < length ? ((const uint8_t *)second)[i - first_length] : 0;
+ }
+ settle();
+ smem_poke(glink.descriptor + GLINK_OUR_HEAD, (head + padded) % glink.ours_size);
+ settle();
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_GLINK);
+ glink.said++;
+ return true;
+}
+
+static const char *
+glink_name(unsigned command)
+{
+ static const char *const names[] = {
+ "version", "version-ack", "open", "close", "open-ack", "intent", "done", "intent-request", "intent-request-ack",
+ "data", "10", "close-ack", "data-more", "read-notify", "done-reuse", "signals"
+ };
+
+ return command < sizeof(names) / sizeof(names[0]) ? names[command] : "?";
+}
+
+static void
+glink_say(unsigned command, uint32_t param1, uint32_t param2, const void *more, uint32_t more_length)
+{
+ struct glink_message message = { .command = (uint16_t)command, .param1 = (uint16_t)param1, .param2 = param2 };
+ bool sent = glink_put(&message, sizeof(message), more, more_length);
+
+ if (glink.trace > 0 || !sent) {
+ glink.trace -= glink.trace > 0;
+ printf("mpss: glink: > %s %u %#x%s\n", glink_name(command), param1, param2, sent ? "" : ": NO ROOM IN OUR QUEUE, not sent");
+ }
+}
+
+static bool
+glink_offer(struct glink_channel *channel, uint32_t size)
+{
+ uint32_t words[2] = { size, channel->our_intent_count + 1 }, i = channel->our_intent_count;
+
+ if (i >= sizeof(channel->our_intents) / sizeof(channel->our_intents[0]) || size == 0 || size > 0x10000) {
+ return false;
+ }
+ if (channel->our_intents[i].data == NULL || channel->our_intents[i].size != size) {
+ channel->our_intents[i].data = kalloc_data(size, Z_WAITOK | Z_ZERO);
+ }
+ if (channel->our_intents[i].data == NULL) {
+ return false;
+ }
+ channel->our_intents[i].size = size;
+ channel->our_intents[i].fill = 0;
+ channel->our_intent_count++;
+ glink_say(GLINK_INTENT, channel->ours, 1, words, sizeof(words));
+ return true;
+}
+
+/* Everything the DSP has put into its queue, taken out and acted on (Linux's qcom_glink_native_rx()). With glink_lock held. */
+static void
+glink_pump(void)
+{
+ struct glink_message message;
+ struct glink_channel *channel;
+ uint32_t words[4];
+ char name[33];
+
+ while (glink.up && !glink.broken) {
+ uint32_t waiting = glink_waiting(), length = sizeof(message);
+
+ if (waiting < sizeof(message)) {
+ return;
+ }
+ glink_peek(&message, 0, sizeof(message));
+ if (message.command == GLINK_OPEN) {
+ length += message.param2;
+ } else if (message.command == GLINK_INTENT) {
+ length += 8 * message.param2;
+ } else if (message.command == GLINK_DATA || message.command == GLINK_DATA_MORE) {
+ if (waiting < 16) {
+ return;
+ }
+ glink_peek(words, 8, 8);
+ length = 16 + words[0];
+ }
+ length = (length + 7) & ~7u;
+ if (length > glink.theirs_size - 8) {
+ printf("mpss: glink: A MESSAGE OF %u BYTES (%s %u %#x): not believed, and nothing more is read\n", length,
+ glink_name(message.command), message.param1, message.param2);
+ glink.broken = true;
+ return;
+ }
+ if (waiting < length) {
+ return;
+ }
+ glink.heard++;
+ if (glink.trace > 0 && message.command != GLINK_DATA && message.command != GLINK_DATA_MORE &&
+ message.command != GLINK_INTENT && message.command != GLINK_DONE_REUSE) {
+ glink.trace--;
+ printf("mpss: glink: < %s %u %#x\n", glink_name(message.command), message.param1, message.param2);
+ }
+ switch (message.command) {
+ case GLINK_VERSION:
+ if (message.param1 == GLINK_VERSION_1) {
+ glink.features &= message.param2;
+ }
+ glink_advance(length);
+ glink_say(GLINK_VERSION_ACK, GLINK_VERSION_1, glink.features, NULL, 0);
+ break;
+ case GLINK_VERSION_ACK:
+ glink_advance(length);
+ if (message.param1 == GLINK_VERSION_1 && message.param2 == glink.features) {
+ glink.agreed = true;
+ } else if (message.param1 != 0) {
+ glink.features &= message.param2;
+ glink_say(GLINK_VERSION, GLINK_VERSION_1, glink.features, NULL, 0);
+ }
+ break;
+ case GLINK_OPEN:
+ bzero(name, sizeof(name));
+ glink_peek(name, sizeof(message), message.param2 < sizeof(name) - 1 ? message.param2 : sizeof(name) - 1);
+ glink_advance(length);
+ channel = NULL;
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ if (strcmp(name, glink.channel[c].name) == 0) {
+ channel = &glink.channel[c];
+ }
+ }
+ if (channel == NULL) {
+ printf("mpss: glink: the DSP opens \"%s\" as its channel %u: left unanswered, nobody here speaks it\n", name,
+ message.param1);
+ for (unsigned i = 0; i < 4; i++) {
+ if (glink.others[i][0] == 0) {
+ strlcpy(glink.others[i], name, sizeof(glink.others[i]));
+ break;
+ }
+ }
+ break;
+ }
+ printf("mpss: glink: the DSP opens \"%s\" as its channel %u: acknowledged, and opened from our side as channel %u\n",
+ name, message.param1, channel->ours);
+ channel->theirs = message.param1;
+ glink_say(GLINK_OPEN_ACK, channel->theirs, 0, NULL, 0);
+ if (!channel->asked) {
+ uint32_t size = (uint32_t)strlen(channel->name) + 1;
+
+ bzero(name, sizeof(name));
+ strlcpy(name, channel->name, sizeof(name));
+ glink_say(GLINK_OPEN, channel->ours, size, name, (size + 7) & ~7u);
+ channel->asked = true;
+ }
+ break;
+ case GLINK_OPEN_ACK:
+ glink_advance(length);
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ if (message.param1 == glink.channel[c].ours) {
+ glink.channel[c].acknowledged = true;
+ }
+ }
+ break;
+ case GLINK_INTENT:
+ channel = glink_theirs(message.param1);
+ for (uint32_t i = 0; channel != NULL && i < message.param2; i++) {
+ glink_peek(words, 8 + 8 * i, 8);
+ for (unsigned n = 0; n < sizeof(channel->their_intents) / sizeof(channel->their_intents[0]); n++) {
+ if (!channel->their_intents[n].there || channel->their_intents[n].id == words[1]) {
+ channel->their_intents[n].there = true;
+ channel->their_intents[n].in_use = false;
+ channel->their_intents[n].size = words[0];
+ channel->their_intents[n].id = words[1];
+ break;
+ }
+ }
+ }
+ glink_advance(length);
+ break;
+ case GLINK_DONE:
+ case GLINK_DONE_REUSE:
+ channel = glink_theirs(message.param1);
+ for (unsigned n = 0; channel != NULL && n < sizeof(channel->their_intents) / sizeof(channel->their_intents[0]); n++) {
+ if (channel->their_intents[n].there && channel->their_intents[n].id == message.param2) {
+ channel->their_intents[n].in_use = false;
+ channel->their_intents[n].there = message.command == GLINK_DONE_REUSE;
+ }
+ }
+ glink_advance(length);
+ break;
+ case GLINK_INTENT_REQUEST:
+ glink_advance(length);
+ if ((channel = glink_theirs(message.param1)) != NULL) {
+ bool granted = glink_offer(channel, message.param2);
+
+ glink_say(GLINK_INTENT_REQUEST_ACK, channel->ours, granted, NULL, 0);
+ }
+ break;
+ case GLINK_INTENT_REQUEST_ACK:
+ glink_advance(length);
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ if (message.param1 == glink.channel[c].theirs || message.param1 == glink.channel[c].ours) {
+ glink.channel[c].request_answered = true;
+ glink.channel[c].request_granted = message.param2 != 0;
+ }
+ }
+ break;
+ case GLINK_DATA:
+ case GLINK_DATA_MORE: {
+ uint32_t intent = message.param2 - 1;
+
+ channel = glink_theirs(message.param1);
+ if (channel == NULL || intent >= channel->our_intent_count ||
+ words[0] > channel->our_intents[intent].size - channel->our_intents[intent].fill) {
+ printf("mpss: glink: < data of %u bytes on channel %u for intent %u: NOT OURS OR TOO MUCH, dropped\n", words[0],
+ message.param1, message.param2);
+ glink_advance(length);
+ break;
+ }
+ glink_peek(channel->our_intents[intent].data + channel->our_intents[intent].fill, 16, words[0]);
+ channel->our_intents[intent].fill += words[0];
+ glink_advance(length);
+ if (words[1] == 0) {
+ struct glink_packet *packet = kalloc_data(sizeof(*packet) + channel->our_intents[intent].fill, Z_WAITOK);
+
+ if (packet != NULL) {
+ packet->next = NULL;
+ packet->length = channel->our_intents[intent].fill;
+ memcpy(packet->data, channel->our_intents[intent].data, packet->length);
+ if (channel->last != NULL) {
+ channel->last->next = packet;
+ } else {
+ channel->first = packet;
+ }
+ channel->last = packet;
+ }
+ channel->our_intents[intent].fill = 0;
+ glink_say(GLINK_DONE_REUSE, channel->ours, message.param2, NULL, 0);
+ }
+ break;
+ }
+ case GLINK_READ_NOTIFY:
+ glink_advance(length);
+ wr(ipcc.base + IPCC_SEND, IPCC_MPSS << 16 | IPCC_GLINK);
+ break;
+ case GLINK_CLOSE:
+ glink_advance(length);
+ glink_say(GLINK_CLOSE_ACK, message.param1, 0, NULL, 0);
+ if ((channel = glink_theirs(message.param1)) != NULL) {
+ printf("mpss: glink: THE DSP CLOSES \"%s\"\n", channel->name);
+ channel->open = false;
+ channel->theirs = 0;
+ }
+ break;
+ case GLINK_CLOSE_ACK:
+ case GLINK_SIGNALS:
+ glink_advance(length);
+ break;
+ default:
+ printf("mpss: glink: A MESSAGE THIS DOES NOT KNOW (%u, %u, %#x): nothing more is read\n", message.command,
+ message.param1, message.param2);
+ glink.broken = true;
+ return;
+ }
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ channel = &glink.channel[c];
+ if (channel->theirs != 0 && channel->acknowledged && !channel->open) {
+ channel->open = true;
+ printf("mpss: glink: THE CHANNEL \"%s\" IS OPEN both ways (ours %u, the DSP's %u); telling the DSP our intents\n",
+ channel->name, channel->ours, channel->theirs);
+ for (unsigned i = 0; i < 3; i++) {
+ for (unsigned n = 0; n < channel->intents[i].count; n++) {
+ glink_offer(channel, channel->intents[i].size);
+ }
+ }
+ }
+ }
+ }
+}
+
+/* One whole message to the DSP on a channel; waits up to five seconds for an intent of the DSP's that it fits. */
+static int
+glink_send(struct glink_channel *channel, const void *data, uint32_t length)
+{
+ struct {
+ struct glink_message message;
+ uint32_t size, left;
+ } header = { { GLINK_DATA, (uint16_t)channel->ours, 0 }, length, 0 };
+ bool asked = false, fits;
+ int found = -1;
+
+ lck_mtx_lock(&glink_lock);
+ if (glink.up && length + sizeof(header) + 8 > glink.ours_size - GLINK_RESERVE) {
+ lck_mtx_unlock(&glink_lock);
+ return EMSGSIZE;
+ }
+ for (unsigned waited = 0;; waited++) {
+ glink_pump();
+ if (!glink.up || !channel->open || glink.broken) {
+ lck_mtx_unlock(&glink_lock);
+ return ENOTCONN;
+ }
+ found = -1;
+ fits = false;
+ for (unsigned n = 0; n < sizeof(channel->their_intents) / sizeof(channel->their_intents[0]); n++) {
+ if (!channel->their_intents[n].there || channel->their_intents[n].size < length) {
+ continue;
+ }
+ fits = true;
+ if (!channel->their_intents[n].in_use &&
+ (found < 0 || channel->their_intents[n].size < channel->their_intents[found].size)) {
+ found = (int)n;
+ }
+ }
+ if (found >= 0 && glink_room() >= ((length + sizeof(header) + 7) & ~7u)) {
+ break;
+ }
+ if (!fits && !asked) {
+ channel->request_answered = false;
+ glink_say(GLINK_INTENT_REQUEST, channel->ours, length, NULL, 0);
+ asked = true;
+ }
+ if (waited >= 5000 || (!fits && asked && channel->request_answered && !channel->request_granted)) {
+ lck_mtx_unlock(&glink_lock);
+ return found < 0 ? ENOBUFS : EAGAIN;
+ }
+ lck_mtx_unlock(&glink_lock);
+ sleep_ms(1);
+ lck_mtx_lock(&glink_lock);
+ }
+ channel->their_intents[found].in_use = true;
+ header.message.param2 = channel->their_intents[found].id;
+ glink_put(&header, sizeof(header), data, length);
+ lck_mtx_unlock(&glink_lock);
+ return 0;
+}
+
+/* ---- QRTR: the router's datagrams, and ports of ours (redwood_wifi.c) ------------------------------ */
+
+/*
+ * What goes over the channel: datagrams of Qualcomm's IPC router, each with a header that says
+ * who sends it (a node, a port) and to whom, and of which kind. The control port hears of the
+ * services the modem offers ("new server": a service, an instance, a node, a port), and later
+ * their going. We are node 1 (Linux's default, and what this phone's Android is); the modem says
+ * who it is in what it sends. The interface for the rest of the kernel is sys/redwood_qrtr.h:
+ * the four calls at the end of this section, which the NAS client (redwood_nas.c) and the QMI
+ * helpers (redwood_qmi.c) take.
+ */
+#define REDWOOD_QRTR_MAX 8192 /* the most a datagram holds, either way */
+#define QRTR_US 1u
+#define QRTR_CONTROL 0xfffffffeu /* the control port */
+#define QRTR_FIRST_PORT 0x4000u /* where Linux begins numbering ports that ask for no number */
+enum { QRTR_DATA = 1, QRTR_HELLO = 2, QRTR_BYE = 3, QRTR_NEW_SERVER = 4, QRTR_DEL_SERVER = 5, QRTR_DEL_CLIENT = 6, QRTR_RESUME_TX = 7,
+ QRTR_NEW_LOOKUP = 10, QRTR_DEL_LOOKUP = 11 };
+
+struct qrtr_header { /* version 1, which Linux sends; version 2 is read as well */
+ uint32_t version, type, from_node, from_port, confirm, size, to_node, to_port;
+};
+struct qrtr_control {
+ uint32_t command;
+ uint32_t words[4]; /* a server: service, instance, node, port; a client: node, port */
+};
+
+struct redwood_qrtr_port {
+ uint32_t number; /* 0: free */
+ redwood_qrtr_receive_t receive;
+ redwood_qrtr_server_t server;
+ void *context;
+ struct {
+ uint32_t service, instance; /* the instance as on the wire: version | instance << 8; 0: any */
+ bool asked;
+ bool fresh; /* not yet told of the services that were there before it */
+ } lookups[4];
+};
+
+static LCK_MTX_DECLARE(qrtr_lock, &mpss_lock_group); /* the tables below */
+static LCK_MTX_DECLARE(qrtr_calling, &mpss_lock_group); /* held by the link's thread around a callback */
+static struct redwood_qrtr_port qrtr_ports[16];
+static struct {
+ uint32_t service, instance, node, port;
+ bool there;
+} qrtr_servers[64];
+static uint32_t qrtr_peer; /* the modem's node; 0: it has not said */
+static uint32_t qrtr_next_port = QRTR_FIRST_PORT, qrtr_in, qrtr_out, qrtr_hellos, qrtr_lookups;
+static thread_t link_thread_is;
+
+/* The name service's side: our servers, told of on hello and on lookup (the one there is: rmtfs, below). */
+static struct {
+ uint32_t service, instance, port;
+} our_servers[4];
+static uint32_t rmtfs_requests, rmtfs_reads, rmtfs_writes, rmtfs_sectors_read, rmtfs_sectors_written, rmtfs_refused;
+static uint32_t tftp_requests, tftp_served, tftp_refused, servreg_requests;
+
+/* One datagram out, of any type. */
+static int
+qrtr_out_packet(uint32_t type, uint32_t from_port, uint32_t to_node, uint32_t to_port, const void *data, uint32_t length)
+{
+ uint32_t padded = (length + 3) & ~3u;
+ struct qrtr_header *packet;
+ int error;
+
+ if (length == 0 || length > REDWOOD_QRTR_MAX) {
+ return EMSGSIZE;
+ }
+ packet = kalloc_data(sizeof(*packet) + padded, Z_WAITOK | Z_ZERO);
+ if (packet == NULL) {
+ return ENOMEM;
+ }
+ packet->version = 1;
+ packet->type = type;
+ packet->from_node = QRTR_US;
+ packet->from_port = from_port;
+ packet->size = length;
+ packet->to_node = to_node;
+ packet->to_port = to_port;
+ memcpy(packet + 1, data, length);
+ error = glink_send(&glink.channel[CHANNEL_QRTR], packet, (uint32_t)sizeof(*packet) + padded);
+ kfree_data(packet, sizeof(*packet) + padded);
+ qrtr_out += error == 0;
+ return error;
+}
+
+static bool
+qrtr_matches(uint32_t service, uint32_t instance, unsigned server)
+{
+ return qrtr_servers[server].there && qrtr_servers[server].service == service &&
+ (instance == 0 || qrtr_servers[server].instance == instance);
+}
+
+/* Everybody who looked a service up is told that it has come, or gone. From the link's thread. */
+static void
+qrtr_tell(uint32_t service, uint32_t instance, uint32_t node, uint32_t port, bool arrived)
+{
+ for (unsigned p = 0; p < sizeof(qrtr_ports) / sizeof(qrtr_ports[0]); p++) {
+ for (unsigned l = 0; l < 4; l++) {
+ redwood_qrtr_server_t server = NULL;
+ void *context;
+
+ lck_mtx_lock(&qrtr_lock);
+ if (qrtr_ports[p].number != 0 && qrtr_ports[p].lookups[l].asked && !qrtr_ports[p].lookups[l].fresh &&
+ qrtr_ports[p].lookups[l].service == service &&
+ (qrtr_ports[p].lookups[l].instance == 0 || qrtr_ports[p].lookups[l].instance == instance)) {
+ server = qrtr_ports[p].server;
+ }
+ context = qrtr_ports[p].context;
+ lck_mtx_unlock(&qrtr_lock);
+ if (server != NULL) {
+ server(context, arrived, service, instance & 0xff, instance >> 8, node, port);
+ }
+ }
+ }
+}
+
+static void qrtr_announce(uint32_t node, unsigned server, bool end);
+static void qrtr_announce_all(uint32_t node);
+
+/* The control port's part: Linux's name service, with our one service to tell of. */
+static void
+qrtr_control(const struct qrtr_header *header, const struct qrtr_control *control)
+{
+ struct qrtr_control hello = { .command = QRTR_HELLO };
+ int found = -1, error;
+
+ switch (control->command) {
+ case QRTR_HELLO:
+ if (qrtr_hellos++ != 0) {
+ break; /* (answered once: two that answer every hello never end) */
+ }
+ error = qrtr_out_packet(QRTR_HELLO, QRTR_CONTROL, header->from_node, QRTR_CONTROL, &hello, sizeof(hello));
+ printf("mpss: qrtr: HELLO FROM NODE %u; ours to it: %s\n", header->from_node, error == 0 ? "SENT" : "NOT SENT");
+ qrtr_announce_all(header->from_node); /* (the name service's part: what we offer) */
+ break;
+ case QRTR_NEW_SERVER:
+ lck_mtx_lock(&qrtr_lock);
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]); i++) {
+ if (qrtr_servers[i].there && qrtr_servers[i].node == control->words[2] && qrtr_servers[i].port == control->words[3]) {
+ found = (int)i;
+ break;
+ }
+ if (!qrtr_servers[i].there && found < 0) {
+ found = (int)i;
+ }
+ }
+ if (found >= 0) {
+ qrtr_servers[found].there = true;
+ qrtr_servers[found].service = control->words[0];
+ qrtr_servers[found].instance = control->words[1];
+ qrtr_servers[found].node = control->words[2];
+ qrtr_servers[found].port = control->words[3];
+ }
+ lck_mtx_unlock(&qrtr_lock);
+ printf("mpss: qrtr: a service: %u (%#x), version %u, instance %u, at node %u port %u\n", control->words[0],
+ control->words[0], control->words[1] & 0xff, control->words[1] >> 8, control->words[2], control->words[3]);
+ qrtr_tell(control->words[0], control->words[1], control->words[2], control->words[3], true);
+ break;
+ case QRTR_DEL_SERVER:
+ case QRTR_BYE:
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]); i++) {
+ bool gone;
+
+ lck_mtx_lock(&qrtr_lock);
+ gone = qrtr_servers[i].there && (control->command == QRTR_BYE ? qrtr_servers[i].node == header->from_node :
+ qrtr_servers[i].node == control->words[2] && qrtr_servers[i].port == control->words[3]);
+ qrtr_servers[i].there = qrtr_servers[i].there && !gone;
+ lck_mtx_unlock(&qrtr_lock);
+ if (gone) {
+ printf("mpss: qrtr: service %u at node %u port %u has gone\n", qrtr_servers[i].service, qrtr_servers[i].node,
+ qrtr_servers[i].port);
+ qrtr_tell(qrtr_servers[i].service, qrtr_servers[i].instance, qrtr_servers[i].node, qrtr_servers[i].port, false);
+ }
+ }
+ break;
+ case QRTR_NEW_LOOKUP: {
+ /* the modem looks for a service: ours that match, then the empty "new server" that ends the answer */
+ unsigned matches = 0;
+
+ for (unsigned i = 0; i < sizeof(our_servers) / sizeof(our_servers[0]); i++) {
+ if (our_servers[i].port != 0 && (control->words[0] == 0 || our_servers[i].service == control->words[0]) &&
+ (control->words[1] == 0 || our_servers[i].instance == control->words[1])) {
+ qrtr_announce(header->from_node, i, false);
+ matches++;
+ }
+ }
+ qrtr_announce(header->from_node, 0, true);
+ if (qrtr_lookups++ < 16) {
+ printf("mpss: qrtr: the modem looks for service %u (%#x), instance %u: %u of ours\n", control->words[0],
+ control->words[0], control->words[1], matches);
+ }
+ break;
+ }
+ default:
+ break; /* (a client's going, and what else there is: nothing here wants it) */
+ }
+}
+
+/* A message from the channel: one datagram. From the link's thread. */
+static void
+qrtr_input(const uint8_t *data, uint32_t length)
+{
+ struct qrtr_header header;
+ redwood_qrtr_receive_t receive = NULL;
+ void *context = NULL;
+ uint32_t at;
+
+ if (length >= sizeof(header) && data[0] == 1) {
+ memcpy(&header, data, sizeof(header));
+ at = sizeof(header);
+ } else if (length >= 16 && data[0] == 3) {
+ /* version 2: version, type, flags, a length of options; the size; then four numbers of 16 bits */
+ bzero(&header, sizeof(header));
+ header.type = data[1];
+ header.confirm = data[2] & 1;
+ memcpy(&header.size, data + 4, 4);
+ header.from_node = (uint32_t)(data[8] | data[9] << 8);
+ header.from_port = (uint32_t)(data[10] | data[11] << 8);
+ header.to_node = (uint32_t)(data[12] | data[13] << 8);
+ header.to_port = (uint32_t)(data[14] | data[15] << 8);
+ header.from_port = header.from_port == 0xfffe ? QRTR_CONTROL : header.from_port;
+ header.to_port = header.to_port == 0xfffe ? QRTR_CONTROL : header.to_port;
+ at = 16 + data[3];
+ } else {
+ printf("mpss: qrtr: %u bytes that are no datagram (they begin %#x)\n", length, length ? data[0] : 0);
+ return;
+ }
+ if (header.size == 0 || at + header.size > length) {
+ printf("mpss: qrtr: a datagram of %u bytes in a message of %u: dropped\n", header.size, length);
+ return;
+ }
+ qrtr_in++;
+ if (qrtr_peer == 0) {
+ qrtr_peer = header.from_node;
+ }
+ if (header.type == QRTR_RESUME_TX) {
+ return; /* (we never ask for one: GLINK's intents hold us back already) */
+ }
+ if (header.confirm) {
+ /* the sender counts what it has sent to this port, and waits for this before it sends many more */
+ struct qrtr_control resume = { .command = QRTR_RESUME_TX, .words = { header.to_node, header.to_port } };
+
+ qrtr_out_packet(QRTR_RESUME_TX, header.to_port, header.from_node, header.from_port, &resume, sizeof(resume));
+ }
+ if (header.to_port == QRTR_CONTROL || header.type == QRTR_HELLO || header.type == QRTR_BYE ||
+ header.type == QRTR_NEW_SERVER || header.type == QRTR_DEL_SERVER || header.type == QRTR_NEW_LOOKUP) {
+ struct qrtr_control control = { 0 };
+
+ memcpy(&control, data + at, header.size < sizeof(control) ? header.size : sizeof(control));
+ qrtr_control(&header, &control);
+ return;
+ }
+ lck_mtx_lock(&qrtr_lock);
+ for (unsigned p = 0; p < sizeof(qrtr_ports) / sizeof(qrtr_ports[0]); p++) {
+ if (qrtr_ports[p].number == header.to_port && header.type == QRTR_DATA) {
+ receive = qrtr_ports[p].receive;
+ context = qrtr_ports[p].context;
+ }
+ }
+ lck_mtx_unlock(&qrtr_lock);
+ if (receive != NULL) {
+ receive(context, header.from_node, header.from_port, data + at, header.size);
+ } else if (qrtr_in <= 64) {
+ printf("mpss: qrtr: a datagram of %u bytes from node %u port %u to our port %u: nobody here listens there\n", header.size,
+ header.from_node, header.from_port, header.to_port);
+ }
+}
+
+/* Lookups made since the last look are told of the services that were there before them. From the link's thread. */
+static void
+qrtr_fresh(void)
+{
+ for (unsigned p = 0; p < sizeof(qrtr_ports) / sizeof(qrtr_ports[0]); p++) {
+ for (unsigned l = 0; l < 4; l++) {
+ redwood_qrtr_server_t server;
+ uint32_t service, instance;
+ void *context;
+
+ lck_mtx_lock(&qrtr_lock);
+ if (qrtr_ports[p].number == 0 || !qrtr_ports[p].lookups[l].fresh) {
+ lck_mtx_unlock(&qrtr_lock);
+ continue;
+ }
+ qrtr_ports[p].lookups[l].fresh = false;
+ server = qrtr_ports[p].server;
+ service = qrtr_ports[p].lookups[l].service;
+ instance = qrtr_ports[p].lookups[l].instance;
+ context = qrtr_ports[p].context;
+ lck_mtx_unlock(&qrtr_lock);
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]) && server != NULL; i++) {
+ if (qrtr_matches(service, instance, i)) {
+ server(context, true, qrtr_servers[i].service, qrtr_servers[i].instance & 0xff,
+ qrtr_servers[i].instance >> 8, qrtr_servers[i].node, qrtr_servers[i].port);
+ }
+ }
+ }
+ }
+}
+
+static struct redwood_qrtr_port *
+qrtr_open(redwood_qrtr_receive_t receive, redwood_qrtr_server_t server, void *context)
+{
+ struct redwood_qrtr_port *port = NULL;
+
+ lck_mtx_lock(&qrtr_lock);
+ for (unsigned p = 0; p < sizeof(qrtr_ports) / sizeof(qrtr_ports[0]) && port == NULL; p++) {
+ if (qrtr_ports[p].number == 0) {
+ port = &qrtr_ports[p];
+ bzero(port, sizeof(*port));
+ port->number = qrtr_next_port++;
+ port->receive = receive;
+ port->server = server;
+ port->context = context;
+ }
+ }
+ lck_mtx_unlock(&qrtr_lock);
+ return port;
+}
+
+static int
+qrtr_send(struct redwood_qrtr_port *port, uint32_t node, uint32_t to, const void *data, size_t length)
+{
+ if (port == NULL || port->number == 0 || length > REDWOOD_QRTR_MAX) {
+ return EINVAL;
+ }
+ return qrtr_out_packet(QRTR_DATA, port->number, node, to, data, (uint32_t)length);
+}
+
+static int
+qrtr_lookup(struct redwood_qrtr_port *port, uint32_t service, uint32_t version, uint32_t instance)
+{
+ int error = ENOSPC;
+
+ lck_mtx_lock(&qrtr_lock);
+ for (unsigned l = 0; l < 4 && error != 0 && port != NULL && port->number != 0; l++) {
+ if (!port->lookups[l].asked) {
+ port->lookups[l].service = service;
+ port->lookups[l].instance = instance != 0 ? (version & 0xff) | instance << 8 : 0;
+ port->lookups[l].asked = true;
+ port->lookups[l].fresh = true;
+ error = 0;
+ }
+ }
+ lck_mtx_unlock(&qrtr_lock);
+ return error;
+}
+
+static void
+qrtr_close(struct redwood_qrtr_port *port)
+{
+ struct qrtr_control gone = { .command = QRTR_DEL_CLIENT, .words = { QRTR_US, port ? port->number : 0 } };
+ bool outside = current_thread() != link_thread_is;
+
+ if (port == NULL || port->number == 0) {
+ return;
+ }
+ if (outside) {
+ lck_mtx_lock(&qrtr_calling); /* (a callback of the port's that is running ends first) */
+ }
+ lck_mtx_lock(&qrtr_lock);
+ port->number = 0;
+ lck_mtx_unlock(&qrtr_lock);
+ if (outside) {
+ lck_mtx_unlock(&qrtr_calling);
+ }
+ if (qrtr_peer != 0 && glink.channel[CHANNEL_QRTR].open) {
+ qrtr_out_packet(QRTR_DEL_CLIENT, gone.words[1], qrtr_peer, QRTR_CONTROL, &gone, sizeof(gone));
+ }
+}
+
+static const struct redwood_qrtr_ops qrtr_ops = { .open = qrtr_open, .send = qrtr_send, .lookup = qrtr_lookup, .close = qrtr_close };
+
+/* The modem's QRTR for the rest of the kernel: NULL until the link is up (sys/redwood_qrtr.h). */
+const struct redwood_qrtr_ops *
+redwood_mpss_qrtr(void)
+{
+ return glink.channel[CHANNEL_QRTR].open ? &qrtr_ops : NULL;
+}
+
+/* ---- the link's thread ----------------------------------------------------------------------------- */
+
+static volatile bool link_wanted, link_runs;
+
+/* Everything the modem says once it runs: its SMP2P item, its queue, and from that the datagrams, handed on from here. */
+static void
+link_thread(__unused void *parameter, __unused wait_result_t result)
+{
+ struct smp2p words, last;
+ struct glink_packet *packet;
+ bool gave_up = false;
+
+ bzero(&last, sizeof(last));
+ dsp_words(&last);
+ while (link_wanted) {
+ sleep_us(1000);
+ doorbells();
+ if (dsp_words(&words) && memcmp(&words, &last, sizeof(words)) != 0) {
+ last = words;
+ dsp_listen(&words);
+ if ((dsp_state(&words) & DSP_FATAL) && !gave_up) {
+ gave_up = true;
+ printf("mpss: link: THE MODEM SAYS IT HAS GIVEN UP (\"slave-kernel\" = %#x)\n", dsp_state(&words));
+ show_crash_reason();
+ }
+ }
+ for (;;) {
+ struct glink_channel *channel = &glink.channel[CHANNEL_QRTR];
+
+ lck_mtx_lock(&glink_lock);
+ glink_pump();
+ packet = channel->first;
+ if (packet != NULL) {
+ channel->first = packet->next;
+ channel->last = channel->first == NULL ? NULL : channel->last;
+ }
+ lck_mtx_unlock(&glink_lock);
+ if (packet == NULL) {
+ break;
+ }
+ lck_mtx_lock(&qrtr_calling);
+ qrtr_input(packet->data, packet->length);
+ lck_mtx_unlock(&qrtr_calling);
+ kfree_data(packet, sizeof(*packet) + packet->length);
+ }
+ lck_mtx_lock(&qrtr_calling);
+ qrtr_fresh();
+ lck_mtx_unlock(&qrtr_calling);
+ }
+ link_runs = false;
+ thread_terminate(current_thread());
+}
+
+/* The link given up, before the modem is stopped: Linux too takes GLINK down first and tells the modem nothing of it. */
+static void
+link_stop(void)
+{
+ struct glink_packet *packet;
+
+ if (link_runs) {
+ link_wanted = false;
+ for (unsigned ms = 0; ms < 2000 && link_runs; ms++) {
+ sleep_ms(1);
+ }
+ printf("mpss: link: %s; GLINK said %u and heard %u messages, QRTR sent %u and received %u datagrams; its doorbells heard: "
+ "GLINK %u, SMP2P %u; rmtfs: %u requests, %u reads (%u sectors), %u writes (%u sectors), %u refused\n", link_runs ?
+ "THE THREAD DOES NOT END" : "down", glink.said, glink.heard, qrtr_out, qrtr_in, doorbell.glink, doorbell.smp2p,
+ rmtfs_requests, rmtfs_reads, rmtfs_sectors_read, rmtfs_writes, rmtfs_sectors_written, rmtfs_refused);
+ printf("mpss: link: tftp: %u requests, %u served whole, %u refused; the registry: %u questions\n", tftp_requests, tftp_served,
+ tftp_refused, servreg_requests);
+ }
+ lck_mtx_lock(&glink_lock);
+ glink.up = false;
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ struct glink_channel *channel = &glink.channel[c];
+
+ channel->open = false;
+ while ((packet = channel->first) != NULL) {
+ channel->first = packet->next;
+ kfree_data(packet, sizeof(*packet) + packet->length);
+ }
+ channel->last = NULL;
+ }
+ lck_mtx_unlock(&glink_lock);
+ lck_mtx_lock(&qrtr_lock);
+ bzero(qrtr_servers, sizeof(qrtr_servers));
+ qrtr_peer = 0;
+ lck_mtx_unlock(&qrtr_lock);
+}
+
+/* ---- the modem's file systems: RMTFS, a service of ours on QRTR ----------------------------------- */
+
+/*
+ * The modem keeps its own files -- its settings, its calibration, what it has learnt of the
+ * networks -- on four partitions of the phone's storage, which it cannot reach itself: the
+ * application processor reads and writes them for it, through QMI service 14, "RMTFS" (Linux:
+ * github.com/linux-msm/rmtfs, which this follows). Until that service is there the modem offers
+ * nothing beyond SSCTL and its service registry (service 43 and 66: what the first run saw), and
+ * waits. The modem opens a file by path (/boot/modem_fs1 and so on), asks for a buffer (2.5 MB
+ * of memory both can reach: dt qcom,rmtfs_sharedmem@0; TrustZone is asked to let the modem into
+ * it, as Linux's rmtfs_mem and Android's sharedmem do, and out again when it is shut down), and
+ * then reads and writes sectors of 512 bytes through that buffer.
+ *
+ * Here the four partitions are read whole into memory of ours when the steps begin (the phone's
+ * storage through the loader, redwood_ufs.c: ufs=2, which the ordinary run has), the modem is
+ * served out of that, and what it writes goes into that memory and no further: the storage is
+ * not written. The modem is content with that for a run; what it would have saved is lost when
+ * the run ends, as the next run reads the partitions afresh.
+ *
+ * QRTR's name service is the application processor's job as well (Linux: net/qrtr/ns.c): the
+ * modem is told of our service when it says hello and when it looks for one, with an empty
+ * "new server" to end an answer to a lookup, as Linux ends it.
+ */
+#define RMTFS_SERVICE 14
+#define RMTFS_VERSION 1
+#define RMTFS_INSTANCE 0
+#define RMTFS_SECTOR 512
+#define RMTFS_BUFFER_SIZE 0x280000u /* dt: reg of qcom,rmtfs_sharedmem@0 */
+#define RMTFS_OPEN 1
+#define RMTFS_CLOSE 2
+#define RMTFS_RW_IOVEC 3
+#define RMTFS_ALLOC_BUFF 4
+#define RMTFS_GET_DEV_ERROR 5
+#define QMI_REQUEST 0
+#define QMI_RESPONSE 2
+#define QMI_RESULT_FAILURE 1
+#define QMI_ERR_INTERNAL 1
+#define QMI_ERR_MALFORMED 2
+#define SCM_MP_ASSIGN SCM_CALL(0xc, 0x16)
+#define SCM_ASSIGN_ARGUMENTS 0x1117 /* seven: a buffer, a value, a buffer, a value, a buffer, two values */
+#define VMID_HLOS 3
+#define VMID_MSS_MSA 15
+#define VMID_NAV 0x2b /* the modem's navigation side; dt: qcom,vm-nav-path of qcom,rmtfs_sharedmem@0 */
+#define VM_PERM_RW 6
+
+static struct {
+ const char *path, *partition;
+ uint8_t *data;
+ uint64_t size;
+ bool open;
+} rmtfs_files[4] = {
+ { "/boot/modem_fs1", "modemst1" }, { "/boot/modem_fs2", "modemst2" }, { "/boot/modem_fsc", "fsc" },
+ { "/boot/modem_fsg", "fsg" },
+};
+static struct redwood_qrtr_port *rmtfs_port;
+static vm_offset_t rmtfs_buffer; /* the buffer, as this kernel sees it; 0: none yet */
+static volatile uint8_t *rmtfs_window; /* ... and the uncached one it is used through */
+static uint64_t rmtfs_buffer_pa;
+static bool rmtfs_shared; /* the modem has been let into it */
+
+static unsigned
+rmtfs_load(void)
+{
+ unsigned loaded = 0;
+
+ for (unsigned f = 0; f < 4; f++) {
+ uint64_t size = 0;
+ int error = redwood_ufs_read_partition(rmtfs_files[f].partition, 0, NULL, 0, &size);
+
+ if (error != 0 || size == 0 || size > (64u << 20)) {
+ printf("mpss: %u: the partition %s: %s (%d)\n", step, rmtfs_files[f].partition, error == ENXIO ?
+ "no storage to read it from (the loader was not asked for ufs=2)" : "not read", error);
+ continue;
+ }
+ if (rmtfs_files[f].data == NULL || rmtfs_files[f].size != size) {
+ if (rmtfs_files[f].data != NULL) {
+ kfree_data(rmtfs_files[f].data, rmtfs_files[f].size);
+ }
+ rmtfs_files[f].data = kalloc_data(size, Z_WAITOK | Z_ZERO);
+ rmtfs_files[f].size = rmtfs_files[f].data == NULL ? 0 : size;
+ }
+ if (rmtfs_files[f].data == NULL) {
+ printf("mpss: %u: no memory for %s (%llu bytes)\n", step, rmtfs_files[f].partition, size);
+ continue;
+ }
+ error = redwood_ufs_read_partition(rmtfs_files[f].partition, 0, rmtfs_files[f].data, size, NULL);
+ if (error != 0) {
+ printf("mpss: %u: the partition %s: read error %d\n", step, rmtfs_files[f].partition, error);
+ continue;
+ }
+ printf("mpss: %u: the partition %s (%s): %llu bytes read into memory of ours\n", step, rmtfs_files[f].partition,
+ rmtfs_files[f].path, size);
+ loaded++;
+ }
+ return loaded;
+}
+
+/*
+ * The buffer, once. The modem's requests carry addresses of 32 bits, so it has to be below 4 GB,
+ * and the kernel's free memory there is not to be had in one piece of 2.5 MB; so it is the first
+ * 2.5 MB of the memory reserved for the compute DSP (dt: reserved-memory cdsp@9c700000, 30 MB,
+ * no-map: nothing of ours uses it, and the CDSP is never started here), through an uncached
+ * window, as the modem reads it.
+ */
+#define RMTFS_BUFFER_PA 0x9c700000ull
+
+static bool
+rmtfs_buffer_make(void)
+{
+ if (rmtfs_window != NULL) {
+ return true;
+ }
+ rmtfs_buffer_pa = RMTFS_BUFFER_PA;
+ PE_parse_boot_argn("modem_buffer", &rmtfs_buffer_pa, sizeof(rmtfs_buffer_pa)); /* (another region, to try) */
+ rmtfs_window = (volatile uint8_t *)ml_io_map_unmappable((vm_offset_t)rmtfs_buffer_pa, RMTFS_BUFFER_SIZE, VM_WIMG_WCOMB);
+ if (rmtfs_window == NULL) {
+ printf("mpss: %u: the modem's buffer at %#llx cannot be mapped\n", step, rmtfs_buffer_pa);
+ return false;
+ }
+ rmtfs_buffer = (vm_offset_t)rmtfs_window;
+ for (uint32_t i = 0; i < RMTFS_BUFFER_SIZE; i += 8) {
+ *(volatile uint64_t *)(rmtfs_window + i) = 0;
+ }
+ settle();
+ printf("mpss: %u: the modem's buffer: %u bytes at %#llx (the compute DSP's memory, unused here)\n", step, RMTFS_BUFFER_SIZE,
+ rmtfs_buffer_pa);
+ return true;
+}
+
+/*
+ * TrustZone asked to let the modem into the buffer (Linux's qcom_scm_assign_mem(): the memory, who
+ * has it now, who is to have it and how), or, with `in` false, out of it again.
+ */
+static bool
+rmtfs_share(bool in)
+{
+ struct {
+ uint64_t address, size; /* +0: the memory */
+ uint64_t pad0[6];
+ uint32_t sources[3]; /* +64: who has it */
+ uint32_t pad1[13];
+ struct { /* +128: who is to have it */
+ uint32_t vmid, permission;
+ uint64_t context;
+ uint32_t context_size, unused;
+ } destinations[3];
+ uint8_t pad2[40];
+ uint64_t more[7]; /* +240: the arguments beyond the registers */
+ } *page;
+ vm_offset_t memory = 0;
+ uint64_t answer = ~0ull, pa;
+ unsigned sources, destinations;
+ int32_t result;
+
+ if (rmtfs_buffer == 0 || rmtfs_shared == in) {
+ return rmtfs_buffer != 0;
+ }
+ if (kmem_alloc(kernel_map, &memory, PAGE_SIZE, KMA_KOBJECT | KMA_ZERO | KMA_DATA, VM_KERN_MEMORY_IOKIT) != KERN_SUCCESS) {
+ return false;
+ }
+ page = (void *)memory;
+ page->address = rmtfs_buffer_pa;
+ page->size = RMTFS_BUFFER_SIZE;
+ if (in) {
+ page->sources[0] = VMID_HLOS;
+ sources = 1;
+ page->destinations[0].vmid = VMID_HLOS;
+ page->destinations[0].permission = VM_PERM_RW;
+ page->destinations[1].vmid = VMID_MSS_MSA;
+ page->destinations[1].permission = VM_PERM_RW;
+ page->destinations[2].vmid = VMID_NAV;
+ page->destinations[2].permission = VM_PERM_RW;
+ destinations = option("modem_no_nav") ? 2 : 3;
+ } else {
+ page->sources[0] = VMID_HLOS;
+ page->sources[1] = VMID_MSS_MSA;
+ page->sources[2] = VMID_NAV;
+ sources = option("modem_no_nav") ? 2 : 3;
+ page->destinations[0].vmid = VMID_HLOS;
+ page->destinations[0].permission = VM_PERM_RW;
+ destinations = 1;
+ }
+ pa = kvtophys(memory);
+ page->more[0] = sources * 4; /* argument 3: the size of the sources */
+ page->more[1] = pa + 128; /* 4: the destinations */
+ page->more[2] = destinations * sizeof(page->destinations[0]); /* 5: their size */
+ page->more[3] = 0; /* 6: spare */
+ for (vm_offset_t line = memory; line < memory + PAGE_SIZE; line += 64) {
+ __asm__ volatile ("dc civac, %0" : : "r" (line));
+ }
+ settle();
+ announce(in ? "ask TrustZone to let the modem into our buffer for its files (the memory assigned to HLOS and MSS, both "
+ "read and write)" : "ask TrustZone to take the buffer back from the modem (assigned to HLOS alone)");
+ result = scm(in ? "to share the buffer with the modem" : "to take the buffer back", SCM_MP_ASSIGN, SCM_ASSIGN_ARGUMENTS, pa, 16,
+ pa + 64, pa + 240, &answer);
+ kmem_free(kernel_map, memory, PAGE_SIZE);
+ if (result != 0 || answer != 0) {
+ printf("mpss: %u: TRUSTZONE DOES NOT %s: the call returns %d, its answer is %#llx\n", step, in ? "SHARE THE BUFFER" :
+ "TAKE THE BUFFER BACK", result, answer);
+ return false;
+ }
+ rmtfs_shared = in;
+ printf("mpss: %u: %s\n", step, in ? "THE MODEM MAY USE THE BUFFER" : "the buffer is ours alone again");
+ if (in && option("modem_share_test")) {
+ /* (an experiment: is the buffer still ours to write, now that it is shared? said aloud before and after) */
+ tell("mpss: %u: about to write a word into the shared buffer at %#llx, and read it back\n", step, rmtfs_buffer_pa + 0x200);
+ *(volatile uint32_t *)(rmtfs_window + 0x200) = 0x52545346;
+ settle();
+ tell("mpss: %u: the shared buffer takes our write: it reads %#x\n", step, *(volatile uint32_t *)(rmtfs_window + 0x200));
+ }
+ return true;
+}
+
+/* A TLV of a QMI message: its value and length, or NULL. */
+static const uint8_t *
+qmi_tlv(const uint8_t *m, size_t length, uint8_t type, uint16_t *size)
+{
+ for (size_t at = 7; at + 3 <= length;) {
+ uint16_t n = (uint16_t)(m[at + 1] | m[at + 2] << 8);
+
+ if (at + 3 + n > length) {
+ return NULL;
+ }
+ if (m[at] == type) {
+ *size = n;
+ return m + at + 3;
+ }
+ at += 3 + n;
+ }
+ return NULL;
+}
+
+static uint32_t
+le32(const uint8_t *p)
+{
+ return (uint32_t)(p[0] | p[1] << 8 | p[2] << 16 | p[3] << 24);
+}
+
+/* The modem's request for its files: answered, the one TLV the answer carries besides the result made here. From the link's thread. */
+static void
+rmtfs_receive(__unused void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data, *value;
+ uint8_t answer[64], *tlv;
+ uint16_t message, size = 0, result = 0, error = 0, extra = 0;
+ unsigned file = 4;
+
+ if (length < 7 || m[0] != QMI_REQUEST) {
+ return;
+ }
+ message = (uint16_t)(m[3] | m[4] << 8);
+ rmtfs_requests++;
+ bzero(answer, sizeof(answer));
+ answer[0] = QMI_RESPONSE;
+ answer[1] = m[1];
+ answer[2] = m[2];
+ answer[3] = m[3];
+ answer[4] = m[4];
+ tlv = answer + 7 + 7; /* (the result TLV, 7 bytes, comes first) */
+ switch (message) {
+ case RMTFS_OPEN:
+ /* (the path: the TLV's value is the string itself, with no length in front -- the modem sends it so) */
+ value = qmi_tlv(m, length, 1, &size);
+ for (unsigned f = 0; value != NULL && f < 4; f++) {
+ if (size == strlen(rmtfs_files[f].path) && memcmp(value, rmtfs_files[f].path, size) == 0) {
+ file = f;
+ }
+ }
+ if (file == 4 || rmtfs_files[file].data == NULL) {
+ printf("mpss: rmtfs: the modem opens \"%.*s\": NOT HERE\n", value != NULL ? (int)(size < 64 ? size : 64) : 0,
+ value != NULL ? (const char *)value : "");
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_INTERNAL;
+ rmtfs_refused++;
+ break;
+ }
+ rmtfs_files[file].open = true;
+ printf("mpss: rmtfs: the modem opens %s (%s, %llu bytes): caller %u\n", rmtfs_files[file].path, rmtfs_files[file].partition,
+ rmtfs_files[file].size, file + 1);
+ tlv[0] = 0x10; /* the caller's id: the file's number plus one */
+ tlv[1] = 4;
+ tlv[2] = 0;
+ tlv[3] = (uint8_t)(file + 1);
+ extra = 7;
+ break;
+ case RMTFS_CLOSE:
+ value = qmi_tlv(m, length, 1, &size);
+ file = value != NULL && size == 4 && le32(value) >= 1 && le32(value) <= 4 ? le32(value) - 1 : 4;
+ if (file < 4) {
+ rmtfs_files[file].open = false;
+ printf("mpss: rmtfs: the modem closes %s\n", rmtfs_files[file].path);
+ }
+ break;
+ case RMTFS_ALLOC_BUFF:
+ value = qmi_tlv(m, length, 2, &size);
+ if (value == NULL || size != 4 || le32(value) > RMTFS_BUFFER_SIZE || rmtfs_buffer == 0 || !rmtfs_shared) {
+ printf("mpss: rmtfs: the modem asks for a buffer of %u bytes: NOT GIVEN (ours is %u, %s)\n", value != NULL && size == 4 ?
+ le32(value) : 0, RMTFS_BUFFER_SIZE, rmtfs_shared ? "shared" : "NOT SHARED");
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_INTERNAL;
+ rmtfs_refused++;
+ break;
+ }
+ tell("mpss: rmtfs: the modem asks for a buffer of %u bytes: given ours at %#llx\n", le32(value), rmtfs_buffer_pa);
+ tlv[0] = 0x10;
+ tlv[1] = 8;
+ tlv[2] = 0;
+ for (unsigned b = 0; b < 8; b++) {
+ tlv[3 + b] = (uint8_t)(rmtfs_buffer_pa >> (8 * b));
+ }
+ extra = 11;
+ break;
+ case RMTFS_RW_IOVEC: {
+ const uint8_t *direction = qmi_tlv(m, length, 2, &size), *entries;
+ uint16_t direction_size = size, entries_size = 0;
+ bool write;
+ unsigned count;
+
+ value = qmi_tlv(m, length, 1, &size);
+ entries = qmi_tlv(m, length, 3, &entries_size);
+ file = value != NULL && size == 4 && le32(value) >= 1 && le32(value) <= 4 ? le32(value) - 1 : 4;
+ if (file == 4 || !rmtfs_files[file].open || direction == NULL || direction_size != 1 || entries == NULL || entries_size < 1 ||
+ entries_size != 1 + 12 * entries[0] || rmtfs_window == NULL) {
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_MALFORMED;
+ rmtfs_refused++;
+ break;
+ }
+ write = direction[0] != 0;
+ count = entries[0];
+ {
+ /* (an experiment: the first request waited on, said aloud before and after, for the log to catch what follows) */
+ static bool first = true;
+ uint32_t delay = 0;
+
+ if (first && PE_parse_boot_argn("modem_rmtfs_delay", &delay, sizeof(delay)) && delay != 0) {
+ tell("mpss: rmtfs: the modem's first request (%s %u pieces of %s, from sector %u, %u sectors, to %#x): waiting %u s "
+ "before it is served\n", write ? "write" : "read", count, rmtfs_files[file].partition, le32(entries + 1),
+ le32(entries + 9), le32(entries + 5), delay);
+ for (uint32_t ms = 0; ms < delay * 1000; ms++) {
+ sleep_ms(1);
+ }
+ tell("mpss: rmtfs: serving it now\n");
+ }
+ first = false;
+ }
+ if (option("modem_rmtfs_rw_off")) {
+ /* (an experiment: the modem is told no, and the buffer is not touched from here) */
+ printf("mpss: rmtfs: the modem %s %u pieces of %s: REFUSED (modem_rmtfs_rw_off)\n", write ? "writes" : "reads", count,
+ rmtfs_files[file].partition);
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_INTERNAL;
+ rmtfs_refused++;
+ break;
+ }
+ for (unsigned e = 0; e < count && result == 0; e++) {
+ uint64_t sector = le32(entries + 1 + 12 * e), physical = le32(entries + 5 + 12 * e), sectors = le32(entries + 9 + 12 * e);
+ uint64_t bytes = sectors * RMTFS_SECTOR;
+
+ if (physical < rmtfs_buffer_pa || physical - rmtfs_buffer_pa + bytes > RMTFS_BUFFER_SIZE ||
+ sector * RMTFS_SECTOR + bytes > rmtfs_files[file].size) {
+ printf("mpss: rmtfs: %s %llu sectors from %llu of %s at %#llx: OUTSIDE THE FILE OR THE BUFFER\n", write ? "write" :
+ "read", sectors, sector, rmtfs_files[file].partition, physical);
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_MALFORMED;
+ rmtfs_refused++;
+ break;
+ }
+ if (write) {
+ for (uint64_t i = 0; i < bytes; i++) {
+ rmtfs_files[file].data[sector * RMTFS_SECTOR + i] = rmtfs_window[physical - rmtfs_buffer_pa + i];
+ }
+ rmtfs_sectors_written += sectors;
+ } else {
+ for (uint64_t i = 0; i < bytes; i++) {
+ rmtfs_window[physical - rmtfs_buffer_pa + i] = rmtfs_files[file].data[sector * RMTFS_SECTOR + i];
+ }
+ rmtfs_sectors_read += sectors;
+ }
+ }
+ settle();
+ if (option("modem_rmtfs_rw_fake")) {
+ /* (an experiment: the buffer is filled as asked, but the modem is told no: does the filling alone end the run?) */
+ tell("mpss: rmtfs: the buffer is filled for the modem's %s of %s, and it is told NO (modem_rmtfs_rw_fake)\n",
+ write ? "write" : "read", rmtfs_files[file].partition);
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_INTERNAL;
+ break;
+ }
+ rmtfs_reads += !write;
+ rmtfs_writes += write;
+ if (rmtfs_reads + rmtfs_writes <= 12) {
+ tell("mpss: rmtfs: the modem %s %u pieces of %s%s: served\n", write ? "writes" : "reads", count, rmtfs_files[file].partition,
+ write ? " (kept in memory, not on the storage)" : "");
+ }
+ break;
+ }
+ case RMTFS_GET_DEV_ERROR:
+ tlv[0] = 0x10; /* the device's error status: none */
+ tlv[1] = 1;
+ tlv[2] = 0;
+ tlv[3] = 0;
+ extra = 4;
+ break;
+ default:
+ printf("mpss: rmtfs: a request this does not know (%#x, %zu bytes)\n", message, length);
+ result = QMI_RESULT_FAILURE;
+ error = QMI_ERR_MALFORMED;
+ break;
+ }
+ answer[5] = (uint8_t)(7 + extra);
+ answer[6] = (uint8_t)((7 + extra) >> 8);
+ answer[7] = 2; /* the result TLV */
+ answer[8] = 4;
+ answer[9] = 0;
+ answer[10] = (uint8_t)result;
+ answer[11] = (uint8_t)(result >> 8);
+ answer[12] = (uint8_t)error;
+ answer[13] = (uint8_t)(error >> 8);
+ if (rmtfs_port != NULL) {
+ int sent = qrtr_send(rmtfs_port, node, port, answer, 7 + 7 + extra);
+
+ if (sent != 0) {
+ printf("mpss: rmtfs: the answer to %#x is NOT SENT (%d)\n", message, sent);
+ }
+ }
+}
+
+/* Our service announced to a node (its control port): a "new server", or the empty one that ends an answer to a lookup. */
+static void
+qrtr_announce(uint32_t node, unsigned server, bool end)
+{
+ struct qrtr_control packet = { .command = QRTR_NEW_SERVER };
+
+ if (!end) {
+ packet.words[0] = our_servers[server].service;
+ packet.words[1] = our_servers[server].instance;
+ packet.words[2] = QRTR_US;
+ packet.words[3] = our_servers[server].port;
+ }
+ qrtr_out_packet(QRTR_NEW_SERVER, QRTR_CONTROL, node, QRTR_CONTROL, &packet, sizeof(packet));
+}
+
+static void
+qrtr_announce_all(uint32_t node)
+{
+ for (unsigned i = 0; i < sizeof(our_servers) / sizeof(our_servers[0]); i++) {
+ if (our_servers[i].port != 0) {
+ qrtr_announce(node, i, false);
+ }
+ }
+}
+
+static void tftp_receive(void *context, uint32_t node, uint32_t port, const void *data, size_t length);
+static void servreg_receive(void *context, uint32_t node, uint32_t port, const void *data, size_t length);
+static struct redwood_qrtr_port *tftp_port, *servreg_port;
+static bool ipa_qmi_start(void);
+static void ipa_qmi_stop(void);
+static struct redwood_qrtr_port *ipa_host_port;
+
+/* The port, and the service on it; the partitions are read in step 2, the buffer shared in step 7. */
+static bool
+rmtfs_start(void)
+{
+ if (rmtfs_port == NULL) {
+ rmtfs_port = qrtr_open(rmtfs_receive, NULL, NULL);
+ }
+ if (rmtfs_port == NULL) {
+ return false;
+ }
+ our_servers[0].service = RMTFS_SERVICE;
+ our_servers[0].instance = RMTFS_VERSION | RMTFS_INSTANCE << 8;
+ our_servers[0].port = rmtfs_port->number;
+ if (!option("modem_no_tftp")) {
+ if (tftp_port == NULL) {
+ tftp_port = qrtr_open(tftp_receive, NULL, NULL);
+ }
+ if (tftp_port != NULL) {
+ our_servers[1].service = 4096; /* TQFTP (below) */
+ our_servers[1].instance = 1;
+ our_servers[1].port = tftp_port->number;
+ }
+ if (servreg_port == NULL) {
+ servreg_port = qrtr_open(servreg_receive, NULL, NULL);
+ }
+ if (servreg_port != NULL) {
+ our_servers[2].service = 64; /* the service registry's locator (below) */
+ our_servers[2].instance = 257;
+ our_servers[2].port = servreg_port->number;
+ }
+ if (ipa_qmi_start()) {
+ our_servers[3].service = 0x31; /* the IPA's host service (below) */
+ our_servers[3].instance = 0x101;
+ our_servers[3].port = ipa_host_port->number;
+ }
+ }
+ return true;
+}
+
+static void
+rmtfs_stop(void)
+{
+ for (unsigned i = 0; i < sizeof(our_servers) / sizeof(our_servers[0]); i++) {
+ our_servers[i].port = 0;
+ }
+ if (rmtfs_port != NULL) {
+ qrtr_close(rmtfs_port);
+ rmtfs_port = NULL;
+ }
+ if (tftp_port != NULL) {
+ qrtr_close(tftp_port);
+ tftp_port = NULL;
+ }
+ if (servreg_port != NULL) {
+ qrtr_close(servreg_port);
+ servreg_port = NULL;
+ }
+ ipa_qmi_stop();
+ for (unsigned f = 0; f < 4; f++) {
+ rmtfs_files[f].open = false;
+ }
+}
+
+/* ---- TFTP over QRTR: the modem's files of configuration (tqftpserv) --------------------------------- */
+
+/*
+ * Besides its file systems, the modem reads files of its firmware partition -- the carrier
+ * configurations (mcfg), shared objects -- through TFTP carried in QRTR datagrams, from a
+ * service numbered 4096 that Linux's tqftpserv and Android's provide; until it has them it
+ * stays offline ("device not ready" to every request to go online). Here the partition's
+ * directory image/ is in the system image under /fw/image (userland/build.sh: fw-modem/image,
+ * pulled from the phone's /vendor/firmware_mnt/image), and served read-only: a read request
+ * names a file as /readonly/firmware/image/..., its options (RFC 2347: blksize, tsize, timeoutms;
+ * Qualcomm's wsize, a window of blocks sent without waiting, rsize, how much of the file, and
+ * seek, from where) are answered in an OACK, and the blocks go from a port of ours opened for the
+ * transfer, as TFTP wants. Writes (/readwrite/...) are refused: nothing of the modem's is kept
+ * here between runs.
+ */
+#define TQFTP_SERVICE 4096
+#define TQFTP_INSTANCE 1 /* version 1, instance 0 */
+#define TFTP_RRQ 1
+#define TFTP_WRQ 2
+#define TFTP_DATA 3
+#define TFTP_ACK 4
+#define TFTP_ERROR 5
+#define TFTP_OACK 6
+#define TFTP_ENOENT 1
+#define TFTP_EACCESS 2
+#define TFTP_EBADOP 4
+#define TFTP_END_OF_TRANSFER 9
+#define TFTP_MAX_BLOCK 8000 /* a datagram of ours holds 8192; the modem's intents 0x4400 */
+#define TFTP_MAX_WINDOW 4
+
+struct tftp_transfer {
+ struct redwood_qrtr_port *port;
+ uint32_t node, peer;
+ const uint8_t *data; /* a read: the file, whole, in memory of ours (root_file(), or tftp_files) */
+ uint64_t size;
+ uint32_t blksize, wsize, rsize, seek;
+ uint32_t sent; /* blocks sent so far */
+ bool done, writing, borrowed; /* borrowed: data is a tftp_files entry's, not to be freed */
+ uint8_t *written; /* a write: what has come, up to TFTP_WRITE_MAX */
+ uint32_t written_size, expected;
+ char name[96];
+};
+static struct tftp_transfer tftp_transfers[4];
+
+/* What the modem writes, kept for the run (it reads some of it back); not on the storage. */
+#define TFTP_WRITE_MAX (1u << 20)
+static struct {
+ char name[96];
+ uint8_t *data;
+ uint32_t size;
+} tftp_files[8];
+
+static void
+tftp_error(struct redwood_qrtr_port *from, uint32_t node, uint32_t port, uint16_t code, const char *text)
+{
+ uint8_t packet[64];
+ size_t n = strlen(text) + 1;
+
+ if (n > sizeof(packet) - 4) {
+ n = sizeof(packet) - 4;
+ }
+ packet[0] = 0;
+ packet[1] = TFTP_ERROR;
+ packet[2] = (uint8_t)(code >> 8);
+ packet[3] = (uint8_t)code;
+ memcpy(packet + 4, text, n);
+ packet[4 + n - 1] = 0;
+ qrtr_send(from, node, port, packet, 4 + n);
+}
+
+static void
+tftp_transfer_end(struct tftp_transfer *t)
+{
+ if (t->data != NULL && !t->borrowed) {
+ uint8_t *data = __DECONST(uint8_t *, t->data);
+
+ kfree_data(data, t->size);
+ }
+ t->data = NULL;
+ if (t->written != NULL) {
+ kfree_data(t->written, TFTP_WRITE_MAX);
+ t->written = NULL;
+ }
+ if (t->port != NULL) {
+ qrtr_close(t->port);
+ t->port = NULL;
+ }
+ t->done = true;
+}
+
+/* Blocks `from` + 1 onward, a window of them: TFTP's blocks are numbered from 1. */
+static void
+tftp_send_window(struct tftp_transfer *t, uint32_t from)
+{
+ uint8_t *packet = kalloc_data(t->blksize + 4, Z_WAITOK);
+
+ if (packet == NULL) {
+ return;
+ }
+ for (uint32_t w = 0; w < t->wsize; w++) {
+ uint32_t block = from + 1 + w;
+ uint64_t offset = (uint64_t)t->seek + (uint64_t)(block - 1) * t->blksize;
+ uint32_t n = offset >= t->size ? 0 : (uint32_t)(t->size - offset < t->blksize ? t->size - offset : t->blksize);
+
+ if (t->rsize != 0 && (uint64_t)(block - 1) * t->blksize >= t->rsize) {
+ break; /* (as much as was asked for has gone) */
+ }
+ if (t->rsize != 0 && (uint64_t)block * t->blksize > t->rsize) {
+ uint32_t left = (uint32_t)(t->rsize - (uint64_t)(block - 1) * t->blksize);
+
+ n = left < n ? left : n; /* (as much as was asked for, and no more than the file has) */
+ }
+ packet[0] = 0;
+ packet[1] = TFTP_DATA;
+ packet[2] = (uint8_t)(block >> 8);
+ packet[3] = (uint8_t)block;
+ if (n != 0) {
+ memcpy(packet + 4, t->data + offset, n);
+ }
+ if (qrtr_send(t->port, t->node, t->peer, packet, 4 + n) != 0) {
+ break;
+ }
+ t->sent = block;
+ if (n < t->blksize) {
+ break; /* the last block */
+ }
+ }
+ kfree_data(packet, t->blksize + 4);
+}
+
+/* What the modem says to a transfer's port: acknowledgements, or an error that ends it. From the link's thread. */
+static void
+tftp_transfer_receive(void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ struct tftp_transfer *t = context;
+ const uint8_t *m = data;
+ uint16_t opcode, block;
+
+ if (t->done || node != t->node || port != t->peer || length < 4) {
+ return;
+ }
+ opcode = (uint16_t)(m[0] << 8 | m[1]);
+ block = (uint16_t)(m[2] << 8 | m[3]);
+ if (t->writing && opcode == TFTP_DATA) {
+ uint32_t n = (uint32_t)(length - 4);
+ uint8_t ack[4] = { 0, TFTP_ACK, (uint8_t)(block >> 8), (uint8_t)block };
+
+ if (block != t->expected) {
+ tftp_error(t->port, node, port, TFTP_EBADOP, "block out of sequence");
+ tftp_transfer_end(t);
+ return;
+ }
+ if (t->written_size + n > TFTP_WRITE_MAX) {
+ tftp_error(t->port, node, port, 3, "too much");
+ tftp_transfer_end(t);
+ return;
+ }
+ memcpy(t->written + t->written_size, m + 4, n);
+ t->written_size += n;
+ t->expected++;
+ if (block % t->wsize == 0 || n < t->blksize) {
+ qrtr_send(t->port, node, port, ack, sizeof(ack));
+ }
+ if (n < t->blksize) {
+ /* the last block: the file is kept, under its name, for the run */
+ unsigned f, free_slot = 8;
+
+ for (f = 0; f < 8; f++) {
+ if (tftp_files[f].data != NULL && strcmp(tftp_files[f].name, t->name) == 0) {
+ break;
+ }
+ if (tftp_files[f].data == NULL && free_slot == 8) {
+ free_slot = f;
+ }
+ }
+ if (f == 8) {
+ f = free_slot;
+ }
+ if (f < 8) {
+ if (tftp_files[f].data != NULL) {
+ kfree_data(tftp_files[f].data, tftp_files[f].size);
+ }
+ tftp_files[f].data = kalloc_data(t->written_size ? t->written_size : 1, Z_WAITOK);
+ if (tftp_files[f].data != NULL) {
+ memcpy(tftp_files[f].data, t->written, t->written_size);
+ tftp_files[f].size = t->written_size;
+ strlcpy(tftp_files[f].name, t->name, sizeof(tftp_files[f].name));
+ }
+ }
+ printf("mpss: tftp: the modem has written \"%s\": %u bytes, kept%s\n", t->name, t->written_size, f < 8 ? "" :
+ " NOWHERE (no room)");
+ tftp_served++;
+ tftp_transfer_end(t);
+ }
+ return;
+ }
+ if (opcode == TFTP_ERROR) {
+ if (block != TFTP_END_OF_TRANSFER) {
+ printf("mpss: tftp: %s: the modem ends the transfer with error %u \"%.*s\"\n", t->name, block, (int)(length - 4), m + 4);
+ }
+ tftp_transfer_end(t);
+ return;
+ }
+ if (opcode != TFTP_ACK) {
+ tftp_error(t->port, node, port, TFTP_EBADOP, "expected ACK");
+ tftp_transfer_end(t);
+ return;
+ }
+ if ((uint64_t)t->seek + (uint64_t)block * t->blksize >= t->size || (t->rsize != 0 && (uint64_t)block * t->blksize >= t->rsize)) {
+ /* the acknowledged block was the last: over */
+ tftp_served++;
+ tftp_transfer_end(t);
+ return;
+ }
+ tftp_send_window(t, block);
+}
+
+static const char *
+tftp_file_of(const char *path)
+{
+ static const char *const prefixes[] = { "/readonly/firmware/image/", "/readonly/vendor/firmware_mnt/image/",
+ "/readonly/vendor/firmware/", "/readonly/firmware/" };
+
+ for (unsigned i = 0; i < sizeof(prefixes) / sizeof(prefixes[0]); i++) {
+ size_t n = strlen(prefixes[i]);
+
+ if (strncmp(path, prefixes[i], n) == 0 && strnstr(path + n, "..", strlen(path + n)) == NULL) {
+ return path + n;
+ }
+ }
+ return NULL;
+}
+
+/* A request to our TFTP port: a read starts a transfer on a port of its own; anything else is refused. From the link's thread. */
+static void
+tftp_receive(__unused void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data, *end = m + length;
+ const char *name, *mode, *p;
+ struct tftp_transfer *t = NULL;
+ uint32_t blksize = 512, wsize = 1, rsize = 0, seek = 0;
+ bool tsize = false, options = false;
+ char fwname[160];
+ uint64_t size = 0;
+ const uint8_t *file;
+ uint8_t oack[160];
+ unsigned at;
+
+ bool writing;
+
+ if (length < 4) {
+ return;
+ }
+ tftp_requests++;
+ writing = (m[0] << 8 | m[1]) == TFTP_WRQ;
+ if ((m[0] << 8 | m[1]) != TFTP_RRQ && !writing) {
+ printf("mpss: tftp: the modem's request %u: not a read nor a write, refused\n", m[0] << 8 | m[1]);
+ tftp_error(tftp_port, node, port, TFTP_EBADOP, "not a read nor a write");
+ tftp_refused++;
+ return;
+ }
+ name = (const char *)m + 2;
+ p = name;
+ while (p < (const char *)end && *p != 0) {
+ p++;
+ }
+ if (p >= (const char *)end) {
+ return;
+ }
+ mode = p + 1;
+ p = mode;
+ while (p < (const char *)end && *p != 0) {
+ p++;
+ }
+ if (p >= (const char *)end) {
+ return;
+ }
+ p++;
+ /* the options: name, value, name, value..., each ended by a zero */
+ while (p < (const char *)end) {
+ const char *option = p, *value;
+
+ while (p < (const char *)end && *p != 0) {
+ p++;
+ }
+ if (p >= (const char *)end) {
+ break;
+ }
+ value = ++p;
+ while (p < (const char *)end && *p != 0) {
+ p++;
+ }
+ if (p >= (const char *)end) {
+ break;
+ }
+ p++;
+ options = true;
+ if (strcmp(option, "blksize") == 0) {
+ blksize = (uint32_t)strtoul(value, NULL, 10);
+ } else if (strcmp(option, "wsize") == 0) {
+ wsize = (uint32_t)strtoul(value, NULL, 10);
+ } else if (strcmp(option, "rsize") == 0) {
+ rsize = (uint32_t)strtoul(value, NULL, 10);
+ } else if (strcmp(option, "seek") == 0) {
+ seek = (uint32_t)strtoul(value, NULL, 10);
+ } else if (strcmp(option, "tsize") == 0) {
+ tsize = true;
+ }
+ }
+ if (blksize < 8) {
+ blksize = 8;
+ }
+ if (blksize > TFTP_MAX_BLOCK) {
+ blksize = TFTP_MAX_BLOCK;
+ }
+ if (wsize < 1) {
+ wsize = 1;
+ }
+ if (wsize > TFTP_MAX_WINDOW) {
+ wsize = TFTP_MAX_WINDOW;
+ }
+ for (unsigned i = 0; i < sizeof(tftp_transfers) / sizeof(tftp_transfers[0]) && t == NULL; i++) {
+ if (tftp_transfers[i].port == NULL) {
+ t = &tftp_transfers[i];
+ }
+ }
+ if (t == NULL) {
+ printf("mpss: tftp: the modem %s \"%s\": NO FREE TRANSFER\n", writing ? "writes" : "reads", name);
+ tftp_error(tftp_port, node, port, TFTP_ENOENT, "busy");
+ tftp_refused++;
+ return;
+ }
+ if (writing) {
+ bzero(t, sizeof(*t));
+ t->written = kalloc_data(TFTP_WRITE_MAX, Z_WAITOK);
+ t->port = t->written == NULL ? NULL : qrtr_open(tftp_transfer_receive, NULL, t);
+ if (t->port == NULL) {
+ tftp_transfer_end(t);
+ tftp_error(tftp_port, node, port, TFTP_ENOENT, "no port");
+ return;
+ }
+ t->node = node;
+ t->peer = port;
+ t->writing = true;
+ t->expected = 1;
+ t->blksize = blksize;
+ t->wsize = wsize;
+ strlcpy(t->name, name, sizeof(t->name));
+ printf("mpss: tftp: the modem writes \"%s\": blocks of %u, window %u; taken into memory\n", name, blksize, wsize);
+ if (!options) {
+ uint8_t ack[4] = { 0, TFTP_ACK, 0, 0 };
+
+ qrtr_send(t->port, node, port, ack, sizeof(ack));
+ return;
+ }
+ size = 0;
+ goto oack;
+ }
+ /* a read: what the modem wrote earlier under that name, else the firmware partition's file */
+ file = NULL;
+ for (unsigned f = 0; f < 8 && file == NULL; f++) {
+ if (tftp_files[f].data != NULL && strcmp(tftp_files[f].name, name) == 0) {
+ file = tftp_files[f].data;
+ size = tftp_files[f].size;
+ t->borrowed = true;
+ }
+ }
+ if (file == NULL && tftp_file_of(name) == NULL) {
+ printf("mpss: tftp: the modem reads \"%s\": NOT A PATH SERVED HERE\n", name);
+ tftp_error(tftp_port, node, port, TFTP_ENOENT, "file not found");
+ tftp_refused++;
+ return;
+ }
+ if (file == NULL) {
+ snprintf(fwname, sizeof(fwname), "image/%s", tftp_file_of(name));
+ file = root_file(fwname, &size);
+ } else {
+ strlcpy(fwname, "(written by the modem)", sizeof(fwname));
+ }
+ if (file == NULL) {
+ printf("mpss: tftp: the modem reads \"%s\" (fw/%s): NOT HERE\n", name, fwname);
+ tftp_error(tftp_port, node, port, TFTP_ENOENT, "file not found");
+ tftp_refused++;
+ return;
+ }
+ {
+ bool borrowed = t->borrowed;
+
+ bzero(t, sizeof(*t));
+ t->borrowed = borrowed;
+ }
+ t->port = qrtr_open(tftp_transfer_receive, NULL, t);
+ if (t->port == NULL) {
+ if (!t->borrowed) {
+ uint8_t *gone = __DECONST(uint8_t *, file);
+
+ kfree_data(gone, size);
+ }
+ tftp_error(tftp_port, node, port, TFTP_ENOENT, "no port");
+ return;
+ }
+ t->node = node;
+ t->peer = port;
+ t->data = file;
+ t->size = size;
+ t->blksize = blksize;
+ t->wsize = wsize;
+ t->rsize = rsize;
+ t->seek = seek;
+ strlcpy(t->name, name, sizeof(t->name));
+ if (tftp_requests <= 40) {
+ printf("mpss: tftp: the modem reads \"%s\": %llu bytes; blocks of %u, window %u, %u asked for, from %u%s\n", name, size, blksize,
+ wsize, rsize, seek, options ? "" : " (no options)");
+ }
+ if (!options) {
+ tftp_send_window(t, 0);
+ return;
+ }
+oack:
+ /* OACK: the options as taken, as tqftpserv answers them */
+ oack[0] = 0;
+ oack[1] = TFTP_OACK;
+ at = 2;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "blksize") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "%u", blksize) + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "timeoutms") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "1000") + 1;
+ if (tsize && !t->writing) {
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "tsize") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "%llu", size) + 1;
+ }
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "wsize") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "%u", wsize) + 1;
+ if (rsize != 0) {
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "rsize") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "%u", rsize) + 1;
+ }
+ if (seek != 0) {
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "seek") + 1;
+ at += (unsigned)snprintf((char *)oack + at, sizeof(oack) - at, "%u", seek) + 1;
+ }
+ qrtr_send(t->port, node, port, oack, at);
+}
+
+/* ---- the service registry's locator (pd-mapper) ---------------------------------------------------- */
+
+/*
+ * The modem's protection domains register their services with a registry whose locator (QMI
+ * service 64) is the application processor's: asked for a service by name ("tms/servreg",
+ * "gps/gps_service"), it answers with the domains that provide it ("msm/modem/root_pd") and their
+ * QMI instance. Linux's pd-mapper reads the table from the *.jsn files beside the firmware; here
+ * they are read from /fw/image (the phone's own), each one domain with its services.
+ */
+#define SERVREG_SERVICE 64
+#define SERVREG_INSTANCE 257 /* what pd-mapper publishes: version 257 */
+#define SERVREG_GET_DOMAIN_LIST 0x21
+#define SERVREG_PFR 0x24
+
+static struct {
+ char service[48]; /* provider/service */
+ char domain[48]; /* soc/domain/subdomain */
+ uint32_t instance;
+} servreg_map[32];
+static unsigned servreg_maps;
+
+/* The text between the quotes after "key": in a JSON file of the registry's simple shape; NULL if there is none. */
+static const char *
+jsn_string(const char *from, const char *end, const char *key, char *out, size_t out_size)
+{
+ char quoted[48];
+ const char *at;
+
+ snprintf(quoted, sizeof(quoted), "\"%s\"", key);
+ at = strnstr(from, quoted, (size_t)(end - from));
+ if (at == NULL) {
+ return NULL;
+ }
+ at += strlen(quoted);
+ while (at < end && (*at == ':' || *at == ' ' || *at == '\t' || *at == '\n' || *at == '\r')) {
+ at++;
+ }
+ if (at >= end) {
+ return NULL;
+ }
+ if (*at == '"') {
+ const char *close = at + 1;
+
+ while (close < end && *close != '"') {
+ close++;
+ }
+ if (close >= end) {
+ return NULL;
+ }
+ snprintf(out, out_size, "%.*s", (int)(close - at - 1), at + 1);
+ return close + 1;
+ }
+ /* a number */
+ {
+ const char *close = at;
+
+ while (close < end && ((*close >= '0' && *close <= '9') || *close == '-')) {
+ close++;
+ }
+ snprintf(out, out_size, "%.*s", (int)(close - at), at);
+ return close;
+ }
+}
+
+static void
+servreg_load(void)
+{
+ static const char *const files[] = { "modemr.jsn", "adspr.jsn", "adsps.jsn", "adspua.jsn", "battmgr.jsn", "cdspr.jsn", "wpssr.jsn" };
+
+ servreg_maps = 0;
+ for (unsigned f = 0; f < sizeof(files) / sizeof(files[0]); f++) {
+ char name[64], soc[24], domain[24], subdomain[24], instance[16], provider[24], service[24], full[48];
+ uint64_t size = 0;
+ const uint8_t *data;
+ const char *text, *end, *at;
+
+ snprintf(name, sizeof(name), "image/%s", files[f]);
+ data = root_file(name, &size);
+ if (data == NULL) {
+ continue;
+ }
+ text = (const char *)data;
+ end = text + size;
+ if (jsn_string(text, end, "soc", soc, sizeof(soc)) == NULL || jsn_string(text, end, "domain", domain, sizeof(domain)) == NULL ||
+ jsn_string(text, end, "subdomain", subdomain, sizeof(subdomain)) == NULL ||
+ jsn_string(text, end, "qmi_instance_id", instance, sizeof(instance)) == NULL) {
+ uint8_t *gone = __DECONST(uint8_t *, data);
+
+ printf("mpss: servreg: fw/%s: not of the shape this reads\n", name);
+ kfree_data(gone, size);
+ continue;
+ }
+ snprintf(full, sizeof(full), "%s/%s/%s", soc, domain, subdomain);
+ at = text;
+ while (servreg_maps < sizeof(servreg_map) / sizeof(servreg_map[0]) &&
+ (at = jsn_string(at, end, "provider", provider, sizeof(provider))) != NULL &&
+ jsn_string(at, end, "service", service, sizeof(service)) != NULL) {
+ snprintf(servreg_map[servreg_maps].service, sizeof(servreg_map[0].service), "%s/%s", provider, service);
+ strlcpy(servreg_map[servreg_maps].domain, full, sizeof(servreg_map[0].domain));
+ servreg_map[servreg_maps].instance = (uint32_t)strtoul(instance, NULL, 10);
+ servreg_maps++;
+ }
+ {
+ uint8_t *gone = __DECONST(uint8_t *, data);
+
+ kfree_data(gone, size);
+ }
+ }
+ printf("mpss: %u: the service registry: %u services of %u files\n", step, servreg_maps, (unsigned)(sizeof(files) / sizeof(files[0])));
+ for (unsigned i = 0; i < servreg_maps; i++) {
+ printf("mpss: %u: %s at %s, instance %u\n", step, servreg_map[i].service, servreg_map[i].domain, servreg_map[i].instance);
+ }
+}
+
+/* A question to the locator, answered. From the link's thread. */
+static void
+servreg_receive(__unused void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data, *value;
+ uint8_t answer[512];
+ uint16_t message, size = 0;
+ unsigned at = 7 + 7, found = 0;
+ char wanted[64] = "";
+
+ if (length < 7 || m[0] != QMI_REQUEST) {
+ return;
+ }
+ message = (uint16_t)(m[3] | m[4] << 8);
+ servreg_requests++;
+ bzero(answer, sizeof(answer));
+ answer[0] = QMI_RESPONSE;
+ answer[1] = m[1];
+ answer[2] = m[2];
+ answer[3] = m[3];
+ answer[4] = m[4];
+ answer[7] = 2; /* result: success */
+ answer[8] = 4;
+ if (message == SERVREG_GET_DOMAIN_LIST) {
+ value = qmi_tlv(m, length, 1, &size);
+ if (value != NULL) {
+ snprintf(wanted, sizeof(wanted), "%.*s", (int)(size < 63 ? size : 63), (const char *)value);
+ }
+ for (unsigned i = 0; i < servreg_maps; i++) {
+ found += strcmp(servreg_map[i].service, wanted) == 0;
+ }
+ /* TLV 0x10: how many domains in all; 0x11: the table's revision; 0x12: the list, each a string with a
+ * byte of length, the instance (4 bytes), and the service data (a byte of validity, 4 bytes) */
+ answer[at++] = 0x10;
+ answer[at++] = 2;
+ answer[at++] = 0;
+ answer[at++] = (uint8_t)found;
+ answer[at++] = (uint8_t)(found >> 8);
+ answer[at++] = 0x11;
+ answer[at++] = 2;
+ answer[at++] = 0;
+ answer[at++] = 1;
+ answer[at++] = 0;
+ if (found != 0) {
+ unsigned list = at;
+
+ answer[at++] = 0x12;
+ at += 2; /* the length, filled in below */
+ answer[at++] = (uint8_t)found;
+ for (unsigned i = 0; i < servreg_maps && at + 64 < sizeof(answer); i++) {
+ if (strcmp(servreg_map[i].service, wanted) != 0) {
+ continue;
+ }
+ answer[at++] = (uint8_t)strlen(servreg_map[i].domain);
+ memcpy(answer + at, servreg_map[i].domain, strlen(servreg_map[i].domain));
+ at += (unsigned)strlen(servreg_map[i].domain);
+ for (unsigned b = 0; b < 4; b++) {
+ answer[at++] = (uint8_t)(servreg_map[i].instance >> (8 * b));
+ }
+ answer[at++] = 0; /* no service data */
+ at += 4;
+ }
+ answer[list + 1] = (uint8_t)(at - list - 3);
+ answer[list + 2] = (uint8_t)((at - list - 3) >> 8);
+ }
+ printf("mpss: servreg: the modem asks where \"%s\" is: %u domain%s\n", wanted, found, found == 1 ? "" : "s");
+ } else if (message == SERVREG_PFR) {
+ value = qmi_tlv(m, length, 1, &size);
+ printf("mpss: servreg: the modem reports a failure of \"%.*s\"\n", value ? (int)(size < 63 ? size : 63) : 0,
+ value ? (const char *)value : "");
+ } else {
+ printf("mpss: servreg: a request this does not know (%#x)\n", message);
+ answer[10] = QMI_RESULT_FAILURE;
+ answer[12] = QMI_ERR_MALFORMED;
+ }
+ answer[5] = (uint8_t)(at - 7);
+ answer[6] = (uint8_t)((at - 7) >> 8);
+ qrtr_send(servreg_port, node, port, answer, at);
+}
+
+/* ---- the IPA's QMI handshake with the modem (Linux's ipa_qmi.c) ------------------------------------- */
+
+/*
+ * The modem's IPA driver and the application processor's agree on the IPA's shared memory over
+ * QMI before the modem uses the IPA: the application processor hosts service 0x31 instance 1 (the
+ * modem registers for an indication there, 0x20, and says when its driver is up, 0x35), and asks
+ * the modem's service 0x31 instance 2 to initialize its driver (0x21) with where the tables are in
+ * the IPA's local memory -- the layout of IPA v4.11 (Linux: ipa_data-v4.11.c) --, then tells it
+ * that all is complete (indication 0x22). Without this the modem's LTE downlink task dies on a
+ * null pointer the moment the modem goes online (23 October: enl2_dl, three of three).
+ */
+#define IPA_QMI_SERVICE 0x31
+#define IPA_HOST_INSTANCE 0x101 /* version 1, instance 1: ours */
+#define IPA_MODEM_INSTANCE 0x201 /* version 1, instance 2: the modem's */
+#define IPA_QMI_INDICATION_REGISTER 0x20
+#define IPA_QMI_INIT_DRIVER 0x21
+#define IPA_QMI_INIT_COMPLETE 0x22
+#define IPA_QMI_INSTALL_FILTER_RULE 0x23
+#define IPA_QMI_FILTER_INSTALLED_NOTIF 0x24
+#define IPA_QMI_CONFIG 0x27
+#define IPA_QMI_DRIVER_INIT_COMPLETE 0x35
+#define QMI_INDICATION 4
+
+static struct redwood_qrtr_port *ipa_client_port;
+static uint32_t ipa_modem_node, ipa_modem_port; /* the modem's IPA service; 0: not seen */
+static uint32_t ipa_register_node, ipa_register_port; /* who asked for the indication */
+static bool ipa_indication_wanted, ipa_indication_sent, ipa_driver_ready, ipa_uc_ready;
+static uint16_t ipa_transaction;
+
+static unsigned
+qmi_put_u32(uint8_t *at, uint8_t type, uint32_t value)
+{
+ at[0] = type;
+ at[1] = 4;
+ at[2] = 0;
+ for (unsigned b = 0; b < 4; b++) {
+ at[3 + b] = (uint8_t)(value >> (8 * b));
+ }
+ return 7;
+}
+
+static unsigned
+qmi_put_pair(uint8_t *at, uint8_t type, uint32_t first, uint32_t second)
+{
+ at[0] = type;
+ at[1] = 8;
+ at[2] = 0;
+ for (unsigned b = 0; b < 4; b++) {
+ at[3 + b] = (uint8_t)(first >> (8 * b));
+ at[7 + b] = (uint8_t)(second >> (8 * b));
+ }
+ return 11;
+}
+
+/* A QMI message with only the result TLV: an answer (flags 2, the transaction echoed) or an indication (flags 4). */
+static void
+ipa_qmi_result(struct redwood_qrtr_port *from, uint32_t node, uint32_t port, uint8_t flags, uint16_t transaction, uint16_t message)
+{
+ uint8_t packet[14] = { flags, (uint8_t)transaction, (uint8_t)(transaction >> 8), (uint8_t)message, (uint8_t)(message >> 8), 7, 0,
+ 2, 4, 0, 0, 0, 0, 0 };
+
+ qrtr_send(from, node, port, packet, sizeof(packet));
+}
+
+static void
+ipa_init_complete(void)
+{
+ if (ipa_indication_wanted && !ipa_indication_sent && ipa_driver_ready) {
+ ipa_qmi_result(ipa_host_port, ipa_register_node, ipa_register_port, QMI_INDICATION, 0, IPA_QMI_INIT_COMPLETE);
+ ipa_indication_sent = true;
+ tell("mpss: ipa: the modem is told that the IPA is all set up (indication %#x)\n", IPA_QMI_INIT_COMPLETE);
+ }
+}
+
+/* The modem's requests to our IPA service. From the link's thread. */
+static void
+ipa_host_receive(__unused void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data;
+ uint16_t message, transaction;
+
+ if (length < 7 || m[0] != QMI_REQUEST) {
+ return;
+ }
+ transaction = (uint16_t)(m[1] | m[2] << 8);
+ message = (uint16_t)(m[3] | m[4] << 8);
+ switch (message) {
+ case IPA_QMI_INDICATION_REGISTER:
+ ipa_qmi_result(ipa_host_port, node, port, QMI_RESPONSE, transaction, message);
+ ipa_register_node = node;
+ ipa_register_port = port;
+ ipa_indication_wanted = true;
+ printf("mpss: ipa: the modem registers for the IPA's indication (node %u port %u)\n", node, port);
+ ipa_init_complete();
+ break;
+ case IPA_QMI_DRIVER_INIT_COMPLETE:
+ ipa_qmi_result(ipa_host_port, node, port, QMI_RESPONSE, transaction, message);
+ ipa_uc_ready = true;
+ printf("mpss: ipa: the modem says its IPA driver is up\n");
+ ipa_init_complete();
+ break;
+ case IPA_QMI_CONFIG:
+ case IPA_QMI_INSTALL_FILTER_RULE:
+ case IPA_QMI_FILTER_INSTALLED_NOTIF:
+ default:
+ /*
+ * After the handshake the modem asks for the IPA's configuration (0x27), has its uplink filter rules
+ * installed (0x23) and tells that they are (0x24) -- Android's driver answers each with success and
+ * little else. Everything it asks is answered so here (the filter rules are not programmed: there is
+ * no data path yet).
+ */
+ ipa_qmi_result(ipa_host_port, node, port, QMI_RESPONSE, transaction, message);
+ printf("mpss: ipa: the modem's request %#x (%s) answered with success\n", message,
+ message == IPA_QMI_CONFIG ? "configuration" : message == IPA_QMI_INSTALL_FILTER_RULE ? "install filter rules" :
+ message == IPA_QMI_FILTER_INSTALLED_NOTIF ? "filters installed" : "not known here");
+ break;
+ }
+}
+
+/* The modem's answer to our INIT_DRIVER. From the link's thread. */
+static void
+ipa_client_receive(__unused void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data;
+ uint16_t size = 0;
+ const uint8_t *value;
+
+ /* every datagram on this port is shown: on 24 October the modem's answer was waited for in vain, if it came at all */
+ printf("mpss: ipa: from node %u port %u, %u bytes:", node, port, (unsigned)length);
+ for (size_t i = 0; i < length && i < 24; i++) {
+ printf(" %02x", m[i]);
+ }
+ printf("%s\n", length > 24 ? " ..." : "");
+ if (length < 7 || m[0] != QMI_RESPONSE || node != ipa_modem_node || port != ipa_modem_port) {
+ return;
+ }
+ value = qmi_tlv(m, length, 2, &size);
+ if ((m[3] | m[4] << 8) == IPA_QMI_INIT_DRIVER && value != NULL && size == 4) {
+ uint16_t result = (uint16_t)(value[0] | value[1] << 8), error = (uint16_t)(value[2] | value[3] << 8);
+
+ tell("mpss: ipa: the modem %s our INIT_DRIVER (result %u, error %u)\n", result == 0 ? "ACCEPTS" : "REFUSES", result, error);
+ ipa_driver_ready = result == 0;
+ ipa_init_complete();
+ }
+}
+
+/* The IPA's local memory as IPA v4.11 lays it out (Linux: ipa_data-v4.11.c), told to the modem. */
+static void
+ipa_init_driver(void)
+{
+ uint8_t packet[200];
+ unsigned at = 7;
+ uint32_t routes = 8; /* the route tables' entries the modem may use: indices 0..7 of 15 (Android's driver:
+ * v4_modem_rt_index_hi 7; its log "num_indices 7"); modem_ipa_routes=N */
+ const uint32_t o = ipa_mem_offset;
+
+ PE_parse_boot_argn("modem_ipa_routes", &routes, sizeof(routes));
+
+ ipa_transaction = (uint16_t)(ipa_transaction + 1 == 0 ? 1 : ipa_transaction + 1);
+ packet[0] = QMI_REQUEST;
+ packet[1] = (uint8_t)ipa_transaction;
+ packet[2] = (uint8_t)(ipa_transaction >> 8);
+ packet[3] = IPA_QMI_INIT_DRIVER;
+ packet[4] = 0;
+ at += qmi_put_u32(packet + at, 0x10, 3); /* platform: 3 = MSM Android (Android's log: "platform_type 3"; 1 is a data card) */
+ at += qmi_put_pair(packet + at, 0x11, o + 0x0688, o + 0x0688 + 0x0240 - 1); /* the modem's headers */
+ at += qmi_put_pair(packet + at, 0x12, o + 0x0508, routes - 1); /* IPv4 routes */
+ at += qmi_put_pair(packet + at, 0x13, o + 0x0608, routes - 1); /* IPv6 routes */
+ at += qmi_put_u32(packet + at, 0x14, o + 0x0308); /* IPv4 filters */
+ at += qmi_put_u32(packet + at, 0x15, o + 0x0408); /* IPv6 filters */
+ at += qmi_put_pair(packet + at, 0x16, o + 0x27d8, 0x800); /* the modem's own region (Android: modem_ofst 0x27d8, 0x800) */
+ at += qmi_put_u32(packet + at, 0x17, 16); /* our endpoint for its control messages (AP_MODEM_RX) */
+ {
+ /*
+ * TLV 0x18 is "is_ssr_bootup" to Android's driver (skip_uc_load to Linux's): whether the modem is
+ * being restarted with the IPA's microcontroller already loaded. Android leaves it out on the first
+ * boot (its log: "is_ssr_bootup 0", not sent), and the modem then loads the microcontroller itself
+ * (its "booted" events follow the answer). Sent as 1 with modem_ipa_skip_uc=1 only.
+ */
+ uint32_t skip = 0;
+
+ PE_parse_boot_argn("modem_ipa_skip_uc", &skip, sizeof(skip));
+ if (skip != 0) {
+ packet[at++] = 0x18;
+ packet[at++] = 1;
+ packet[at++] = 0;
+ packet[at++] = 1;
+ }
+ }
+ at += qmi_put_pair(packet + at, 0x19, o + 0x0ad0, o + 0x0ad0 + 0x0ac0 - 1); /* the modem's processing contexts (Android: 0xad0, 0xac0) */
+ at += qmi_put_pair(packet + at, 0x1b, o + 0x0488, routes - 1); /* hashed IPv4 routes */
+ at += qmi_put_pair(packet + at, 0x1c, o + 0x0588, routes - 1); /* hashed IPv6 routes */
+ at += qmi_put_u32(packet + at, 0x1d, o + 0x0288); /* hashed IPv4 filters */
+ at += qmi_put_u32(packet + at, 0x1e, o + 0x0388); /* hashed IPv6 filters */
+ at += qmi_put_u32(packet + at, 0x1f, o + 0x1d00); /* the modem's quota statistics (Android: stats_quota_q6 0x1d00, 0x30) */
+ at += qmi_put_u32(packet + at, 0x20, 0x30);
+ at += qmi_put_u32(packet + at, 0x21, o + 0x27b0); /* drop statistics (Android: stats_drop 0x27b0, 0x20) */
+ at += qmi_put_u32(packet + at, 0x22, 0x20);
+ packet[5] = (uint8_t)(at - 7);
+ packet[6] = (uint8_t)((at - 7) >> 8);
+ printf("mpss: ipa: about to ask the modem's IPA driver (node %u port %u) to initialize with the IPA's memory layout (%u bytes, "
+ "offset %#x)\n", ipa_modem_node, ipa_modem_port, at, o);
+ if (qrtr_send(ipa_client_port, ipa_modem_node, ipa_modem_port, packet, at) != 0) {
+ printf("mpss: ipa: INIT_DRIVER not sent\n");
+ }
+ if (!option("modem_ipa_no_probe")) {
+ /*
+ * A probe: a request of a message the service has no handler for. A QMI service answers it with an
+ * error (invalid message id) at once -- or, if the modem's IPA driver is stuck before its service's
+ * loop, not at all. Whether anything comes back tells which (24 October: INIT_DRIVER got no answer).
+ */
+ uint8_t probe[7] = { QMI_REQUEST, 0x7e, 0x00, 0xff, 0x7f, 0, 0 };
+
+ if (qrtr_send(ipa_client_port, ipa_modem_node, ipa_modem_port, probe, sizeof(probe)) == 0) {
+ printf("mpss: ipa: a probe sent to the modem's IPA service too (message 0x7fff, transaction 0x7e): an error back means it listens\n");
+ }
+ }
+}
+
+static void
+ipa_client_server(__unused void *context, bool arrived, uint32_t service, __unused uint32_t version, uint32_t instance, uint32_t node,
+ uint32_t port)
+{
+ if (service != IPA_QMI_SERVICE || instance != 2) {
+ return;
+ }
+ if (arrived) {
+ ipa_modem_node = node;
+ ipa_modem_port = port;
+ ipa_init_driver();
+ } else {
+ ipa_modem_node = ipa_modem_port = 0;
+ ipa_driver_ready = ipa_indication_sent = ipa_indication_wanted = false;
+ }
+}
+
+static bool
+ipa_qmi_start(void)
+{
+ if (option("modem_no_ipa_qmi")) {
+ return false;
+ }
+ if (ipa_host_port == NULL) {
+ ipa_host_port = qrtr_open(ipa_host_receive, NULL, NULL);
+ }
+ if (ipa_client_port == NULL) {
+ ipa_client_port = qrtr_open(ipa_client_receive, ipa_client_server, NULL);
+ if (ipa_client_port != NULL) {
+ qrtr_lookup(ipa_client_port, IPA_QMI_SERVICE, 1, 2);
+ }
+ }
+ ipa_driver_ready = ipa_indication_sent = ipa_indication_wanted = ipa_uc_ready = false;
+ return ipa_host_port != NULL && ipa_client_port != NULL;
+}
+
+static void
+ipa_qmi_stop(void)
+{
+ if (ipa_host_port != NULL) {
+ qrtr_close(ipa_host_port);
+ ipa_host_port = NULL;
+ }
+ if (ipa_client_port != NULL) {
+ qrtr_close(ipa_client_port);
+ ipa_client_port = NULL;
+ }
+ ipa_modem_node = ipa_modem_port = 0;
+}
+
+/* ---- step 8: GLINK's queues, looked at -------------------------------------------------------------- */
+
+static bool
+step_queues(void)
+{
+ struct glink_message message;
+ uint32_t head, tail;
+
+ announce("find GLINK's descriptor and the two queues, and read what the modem has put into its own");
+ if (!glink_find()) {
+ printf("mpss: %u: GLINK's items are not all there, or not of a shape this knows (descriptor %#x, ours %#x of %u bytes, "
+ "the DSP's %#x of %u)\n", step, glink.descriptor, glink.ours, glink.ours_size, glink.theirs, glink.theirs_size);
+ return false;
+ }
+ head = smem_word(glink.descriptor + GLINK_THEIR_HEAD);
+ tail = smem_word(glink.descriptor + GLINK_THEIR_TAIL);
+ printf("mpss: %u: our queue: %u bytes at %#lx, written to %#x, read by the DSP to %#x; the DSP's: %u bytes at %#lx, written "
+ "to %#x, read by us to %#x\n", step, glink.ours_size, SMEM_PHYS + glink.ours, smem_word(glink.descriptor + GLINK_OUR_HEAD),
+ smem_word(glink.descriptor + GLINK_OUR_TAIL), glink.theirs_size, SMEM_PHYS + glink.theirs, head, tail);
+ for (uint32_t at = 0, shown = 0; at + sizeof(message) <= head && head < glink.theirs_size && tail == 0 && shown < 8; shown++) {
+ char name[33] = "";
+ uint32_t length = sizeof(message);
+
+ glink_peek(&message, at, sizeof(message));
+ if (message.command == GLINK_OPEN) {
+ glink_peek(name, at + 8, message.param2 < 32 ? message.param2 : 32);
+ length += message.param2;
+ }
+ printf("mpss: %u: in the DSP's queue at %#x: %s %u %#x %s\n", step, at, glink_name(message.command), message.param1,
+ message.param2, name);
+ if (message.command != GLINK_VERSION && message.command != GLINK_OPEN) {
+ break;
+ }
+ at += (length + 7) & ~7u;
+ }
+ return true;
+}
+
+/* ---- step 9: GLINK, spoken -------------------------------------------------------------------------- */
+
+static bool
+step_link(void)
+{
+ thread_t thread;
+ uint64_t began;
+
+ if (!dsp_started || glink.descriptor == 0) {
+ return false;
+ }
+ announce("take up GLINK with the modem: our version, then whatever its messages call for; a thread of ours listens "
+ "from here on, every millisecond");
+ snprintf(last_write, sizeof(last_write), "step %u took up GLINK", step);
+ if (!rmtfs_start()) {
+ printf("mpss: %u: no port for the modem's file systems\n", step);
+ }
+ lck_mtx_lock(&glink_lock);
+ glink.features = GLINK_INTENT_REUSE;
+ glink.agreed = glink.broken = false;
+ glink.said = glink.heard = 0;
+ for (unsigned c = 0; c < CHANNELS; c++) {
+ struct glink_channel *channel = &glink.channel[c];
+
+ channel->asked = channel->acknowledged = channel->open = false;
+ channel->theirs = channel->our_intent_count = 0;
+ bzero(channel->their_intents, sizeof(channel->their_intents));
+ }
+ bzero(glink.others, sizeof(glink.others));
+ glink.trace = 60;
+ glink.up = true;
+ qrtr_hellos = qrtr_in = qrtr_out = qrtr_lookups = 0;
+ doorbell.since = began = now_us();
+ glink_say(GLINK_VERSION, GLINK_VERSION_1, glink.features, NULL, 0);
+ lck_mtx_unlock(&glink_lock);
+ link_wanted = link_runs = true;
+ if (kernel_thread_start(link_thread, NULL, &thread) != KERN_SUCCESS) {
+ link_wanted = link_runs = false;
+ printf("mpss: %u: no thread\n", step);
+ return false;
+ }
+ link_thread_is = thread;
+ thread_deallocate(thread);
+ for (unsigned ms = 0; ms < 5000 && !glink.channel[CHANNEL_QRTR].open && !glink.broken; ms++) {
+ sleep_ms(1);
+ }
+ sleep_ms(50);
+ printf("mpss: %u: %llu ms after our version: versions %s (features %#x); the channel \"IPCRTR\" is %s; the modem also "
+ "opened \"%s\" \"%s\" \"%s\" \"%s\"\n", step, (now_us() - began) / 1000, glink.agreed ? "agreed" : "NOT AGREED", glink.features,
+ glink.channel[CHANNEL_QRTR].open ? "OPEN" : "NOT OPEN", glink.others[0], glink.others[1], glink.others[2], glink.others[3]);
+ return glink.channel[CHANNEL_QRTR].open;
+}
+
+/* ---- step 10: QRTR ---------------------------------------------------------------------------------- */
+
+static bool
+step_qrtr(void)
+{
+ unsigned services = 0;
+
+ if (!glink.channel[CHANNEL_QRTR].open) {
+ return false;
+ }
+ PE_parse_boot_argn("modem_listen", &listen_seconds, sizeof(listen_seconds));
+ announce("listen to QRTR on the channel: the DSP's hello (answered from the link's thread), and the services it tells of");
+ for (unsigned ms = 0; ms < listen_seconds * 1000; ms++) {
+ sleep_ms(1);
+ }
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]); i++) {
+ services += qrtr_servers[i].there;
+ }
+ printf("mpss: %u: after %u s: %s; the DSP is node %u and has told of %u services; it looked for %u of ours; QRTR received %u "
+ "and sent %u datagrams\n", step, listen_seconds, qrtr_hellos ? "HELLO BOTH WAYS" : "NO HELLO FROM THE DSP", qrtr_peer,
+ services, qrtr_lookups, qrtr_in, qrtr_out);
+ return qrtr_hellos != 0;
+}
+
+/* ---- step 11: the modem's services: online, and what the network says ------------------------------ */
+
+/*
+ * QMI, by hand: a message is a byte of flags (0 a request, 2 an answer, 4 a notice), a transaction
+ * of two bytes, the message's number and the length of what follows, then TLVs (a type, a length
+ * of two bytes, the value). The device management service (DMS, 2) is asked to put the modem
+ * online -- message 0x2e, TLV 1 = 0 --: it starts in low power and stays there until somebody
+ * does, which on Android is the RIL's first act. The network access service (NAS, 3) is then
+ * asked, by redwood_nas.c, for the serving system (0x24: registered or not) and the signal (0x4f:
+ * dBm, per radio); both are answers to questions, asked every second here and every two seconds
+ * afterwards (modem=1), since no indication is asked for. What comes back is the kernel's to show:
+ * sysctl kern.redwood.cellular_up, _registered, _bars, _dbm, _radio.
+ */
+#define QMI_DMS 2
+#define QMI_NAS 3
+#define DMS_SET_OPERATING_MODE 0x002e
+#define DMS_GET_OPERATING_MODE 0x002d
+#define DMS_ONLINE 0
+
+static struct redwood_nas_client nas;
+static bool nas_opened;
+static struct redwood_qrtr_port *dms_port;
+static uint32_t dms_node, dms_port_number; /* where DMS is; 0: not seen */
+static volatile uint32_t dms_answers;
+static volatile uint32_t dms_result, dms_error;
+static uint16_t dms_transaction;
+static int cellular_up, cellular_registered, cellular_bars, cellular_dbm, cellular_radio;
+static uint32_t nas_polls, nas_changes;
+
+SYSCTL_DECL(_kern_redwood);
+SYSCTL_INT(_kern_redwood, OID_AUTO, cellular_up, CTLFLAG_RD | CTLFLAG_LOCKED, &cellular_up, 0,
+ "the modem runs and its network access service answers");
+SYSCTL_INT(_kern_redwood, OID_AUTO, cellular_registered, CTLFLAG_RD | CTLFLAG_LOCKED, &cellular_registered, 0,
+ "registered with a network");
+SYSCTL_INT(_kern_redwood, OID_AUTO, cellular_bars, CTLFLAG_RD | CTLFLAG_LOCKED, &cellular_bars, 0, "signal, 0 to 4 bars");
+SYSCTL_INT(_kern_redwood, OID_AUTO, cellular_dbm, CTLFLAG_RD | CTLFLAG_LOCKED, &cellular_dbm, 0, "signal, dBm; 0: none read");
+SYSCTL_INT(_kern_redwood, OID_AUTO, cellular_radio, CTLFLAG_RD | CTLFLAG_LOCKED, &cellular_radio, 0,
+ "the radio (QMI NAS): 4 GSM, 5 UMTS, 8 LTE, 12 NR");
+
+static const char *
+radio_name(uint8_t radio)
+{
+ return radio == 4 ? "GSM" : radio == 5 ? "UMTS" : radio == 8 ? "LTE" : radio == 12 ? "NR" : "?";
+}
+
+/* From the link's thread, when what NAS says has changed. */
+static void
+nas_changed(__unused void *context, const struct redwood_nas_state *state)
+{
+ nas_changes++;
+ cellular_registered = state->registered;
+ cellular_bars = state->bars;
+ cellular_dbm = state->has_signal ? state->dbm : 0;
+ cellular_radio = state->has_signal ? state->radio_interface : 0;
+ tell("mpss: nas: %s; signal %s%d dBm (%s): %u bars\n", state->registered ? "REGISTERED WITH A NETWORK" : "not registered",
+ state->has_signal ? "" : "none read, ", state->has_signal ? state->dbm : 0, radio_name(state->radio_interface), state->bars);
+}
+
+/* A datagram for our DMS port: the answer to our one question. From the link's thread. */
+static void
+dms_receive(__unused void *context, __unused uint32_t node, __unused uint32_t port, const void *data, size_t length)
+{
+ const uint8_t *m = data;
+ uint32_t result = ~0u, error = ~0u;
+
+ if (length < 7 || m[0] != 2) {
+ return;
+ }
+ for (size_t at = 7; at + 3 <= length;) {
+ uint16_t size = (uint16_t)(m[at + 1] | m[at + 2] << 8);
+
+ if (m[at] == 2 && size == 4 && at + 7 <= length) {
+ result = (uint32_t)(m[at + 3] | m[at + 4] << 8);
+ error = (uint32_t)(m[at + 5] | m[at + 6] << 8);
+ }
+ at += 3 + size;
+ }
+ printf("mpss: dms: < message %#x, transaction %u: result %u, error %u\n", m[3] | m[4] << 8, m[1] | m[2] << 8, result, error);
+ if ((m[3] | m[4] << 8) == DMS_GET_OPERATING_MODE) {
+ /* the mode (TLV 1: 0 online, 1 low power, 3 offline, 6 persistent low power, 7 mode-only low power), why offline (TLV 0x10,
+ * bits: host image, PRI image, PRI version, memory full), and whether the hardware restricts it (TLV 0x11) */
+ static const char *const modes[] = { "ONLINE", "LOW POWER", "factory test", "OFFLINE", "resetting", "shutting down",
+ "PERSISTENT LOW POWER", "mode-only low power", "network test" };
+ uint32_t mode = ~0u, reason = ~0u, restricted = ~0u;
+
+ for (size_t at = 7; at + 3 <= length;) {
+ uint16_t size = (uint16_t)(m[at + 1] | m[at + 2] << 8);
+
+ if (at + 3 + size > length) {
+ break;
+ }
+ if (m[at] == 1 && size >= 1) {
+ mode = m[at + 3];
+ } else if (m[at] == 0x10 && size >= 2) {
+ reason = (uint32_t)(m[at + 3] | m[at + 4] << 8);
+ } else if (m[at] == 0x11 && size >= 1) {
+ restricted = m[at + 3];
+ }
+ at += 3 + size;
+ }
+ tell("mpss: dms: THE MODEM'S OPERATING MODE IS %u (%s); offline reason %#x%s; hardware restricted %u\n", mode,
+ mode < sizeof(modes) / sizeof(modes[0]) ? modes[mode] : "?", reason, reason == ~0u ? " (not told)" : "", restricted);
+ }
+ dms_result = result;
+ dms_error = error;
+ dms_answers++;
+}
+
+static void
+dms_server(__unused void *context, bool arrived, uint32_t service, __unused uint32_t version, __unused uint32_t instance,
+ uint32_t node, uint32_t port)
+{
+ if (service != QMI_DMS) {
+ return;
+ }
+ if (arrived) {
+ dms_node = node;
+ dms_port_number = port;
+ } else if (node == dms_node && port == dms_port_number) {
+ dms_node = dms_port_number = 0;
+ }
+}
+
+/* DMS, once it is there, asked to put the modem online, and its answer waited for. */
+static bool
+dms_set_mode(uint32_t mode)
+{
+ uint8_t request[11] = { 0, 0, 0, DMS_SET_OPERATING_MODE & 0xff, DMS_SET_OPERATING_MODE >> 8, 4, 0, 1, 1, 0, (uint8_t)mode };
+ uint32_t before;
+ int error;
+
+ if (dms_port == NULL || dms_port_number == 0) {
+ return false;
+ }
+ {
+ /* first, what mode the modem is in and why (the answer is said by dms_receive()) */
+ uint8_t question[7] = { 0, 0, 0, DMS_GET_OPERATING_MODE & 0xff, DMS_GET_OPERATING_MODE >> 8, 0, 0 };
+
+ dms_transaction = (uint16_t)(dms_transaction + 1 == 0 ? 1 : dms_transaction + 1);
+ question[1] = (uint8_t)dms_transaction;
+ question[2] = (uint8_t)(dms_transaction >> 8);
+ before = dms_answers;
+ if (qrtr_send(dms_port, dms_node, dms_port_number, question, sizeof(question)) == 0) {
+ for (unsigned ms = 0; ms < 3000 && dms_answers == before; ms++) {
+ sleep_ms(1);
+ }
+ }
+ }
+ dms_transaction = (uint16_t)(dms_transaction + 1 == 0 ? 1 : dms_transaction + 1);
+ request[1] = (uint8_t)dms_transaction;
+ request[2] = (uint8_t)(dms_transaction >> 8);
+ before = dms_answers;
+ printf("mpss: %u: about to ask DMS (node %u port %u) for operating mode %u (0 online, 1 low power): message %#x\n", step, dms_node,
+ dms_port_number, mode, DMS_SET_OPERATING_MODE);
+ snprintf(last_write, sizeof(last_write), "step %u asked the modem to go online", step);
+ hold();
+ error = qrtr_send(dms_port, dms_node, dms_port_number, request, sizeof(request));
+ if (error != 0) {
+ printf("mpss: %u: NOT SENT (%d)\n", step, error);
+ return false;
+ }
+ for (unsigned ms = 0; ms < 5000 && dms_answers == before; ms++) {
+ sleep_ms(1);
+ }
+ if (dms_answers == before) {
+ printf("mpss: %u: DMS DOES NOT ANSWER within 5 s\n", step);
+ return false;
+ }
+ tell("mpss: %u: mode %u: %s (result %u, error %u)\n", step, mode, dms_result == 0 ? "THE MODEM AGREES" : "THE MODEM REFUSES",
+ dms_result, dms_error);
+ return dms_result == 0;
+}
+
+/*
+ * Online. A modem that boots offline (as this one does) does not go straight online: Qualcomm's
+ * allow offline -> low power -> online (ModemManager and qmicli users meet the same refusal, error
+ * 52, and go the long way), so that is tried when the short way is refused.
+ */
+static bool
+dms_online(void)
+{
+ if (dms_set_mode(DMS_ONLINE)) {
+ return true;
+ }
+ if (dms_error == 52 && dms_set_mode(1)) {
+ sleep_ms(2000);
+ return dms_set_mode(DMS_ONLINE);
+ }
+ return false;
+}
+
+static bool
+step_nas(void)
+{
+ uint32_t wait_seconds = 90, online = 1;
+ uint64_t began, asked_online = 0;
+ bool registered = false, is_online = false;
+
+ if (!glink.channel[CHANNEL_QRTR].open) {
+ return false;
+ }
+ PE_parse_boot_argn("modem_nas_wait", &wait_seconds, sizeof(wait_seconds));
+ PE_parse_boot_argn("modem_online", &online, sizeof(online));
+ announce("look the modem's services up as they come (they take some 30 s): DMS asked to put the modem online as soon as it is "
+ "there (modem_online=0: not), NAS asked for the serving system and the signal every second, until the modem is registered "
+ "with a signal read or modem_nas_wait seconds (default 90) are over");
+ dms_port = qrtr_open(dms_receive, dms_server, NULL);
+ if (dms_port == NULL || qrtr_lookup(dms_port, QMI_DMS, 0, 0) != 0) {
+ printf("mpss: %u: no port for DMS\n", step);
+ return false;
+ }
+ if (redwood_nas_client_open(&nas, &qrtr_ops, nas_changed, NULL) != 0) {
+ printf("mpss: %u: no port for NAS\n", step);
+ return false;
+ }
+ nas_opened = true;
+ began = now_us();
+ while (now_us() - began < wait_seconds * 1000000ull) {
+ if (online && !is_online && dms_port_number != 0 && now_us() - asked_online > 5000000ull) {
+ asked_online = now_us();
+ is_online = dms_online(); /* (asked again every five seconds until the modem agrees) */
+ }
+ if (nas.connected) {
+ nas_polls += redwood_nas_client_poll(&nas) == 0;
+ }
+ sleep_ms(1000);
+ if (nas.state.registered && nas.state.has_signal) {
+ registered = true;
+ break;
+ }
+ }
+ cellular_up = nas.connected;
+ if (ipa_loaded && !option("modem_ipa_quiet")) {
+ ipa_status("at the end of step 11");
+ ipa_space_faults("at the end of step 11");
+ }
+ {
+ struct smp2p words;
+
+ if (dsp_words(&words)) {
+ show_words("the modem's item now", &words);
+ }
+ }
+ tell("mpss: %u: after %llu ms: DMS %s, the modem %s; NAS %s (node %u port %u), asked %u times, answered %u, %u changes; %s; "
+ "signal %s%d dBm (%s), %u bars\n", step, (now_us() - began) / 1000, dms_port_number ? "there" : "NOT THERE", is_online ?
+ "online" : "NOT PUT ONLINE", nas.connected ? "there" : "NOT THERE", nas.peer_node, nas.peer_port, nas_polls, nas.answers,
+ nas_changes, registered ? "REGISTERED WITH A NETWORK" : nas.state.registered ? "registered" : "not registered",
+ nas.state.has_signal ? "" : "none read, ", nas.state.has_signal ? nas.state.dbm : 0, radio_name(nas.state.radio_interface),
+ nas.state.bars);
+ return nas.connected;
+}
+
+/* With modem=1: NAS asked every two seconds while the modem runs, on a thread of its own. */
+static volatile bool nas_wanted, nas_runs;
+
+static void
+nas_thread(__unused void *parameter, __unused wait_result_t result)
+{
+ while (nas_wanted) {
+ if (nas_opened && nas.connected) {
+ nas_polls += redwood_nas_client_poll(&nas) == 0;
+ }
+ for (unsigned ms = 0; ms < 2000 && nas_wanted; ms += 100) {
+ sleep_ms(100);
+ }
+ }
+ nas_runs = false;
+ thread_terminate(current_thread());
+}
+
+static void
+nas_start(void)
+{
+ thread_t thread;
+
+ nas_wanted = nas_runs = true;
+ if (kernel_thread_start(nas_thread, NULL, &thread) != KERN_SUCCESS) {
+ nas_wanted = nas_runs = false;
+ return;
+ }
+ thread_deallocate(thread);
+}
+
+/* The questions end before the link does. */
+static void
+nas_stop(void)
+{
+ if (nas_runs) {
+ nas_wanted = false;
+ for (unsigned ms = 0; ms < 3000 && nas_runs; ms++) {
+ sleep_ms(1);
+ }
+ }
+ if (nas_opened) {
+ redwood_nas_client_close(&nas);
+ nas_opened = false;
+ }
+ if (dms_port != NULL) {
+ qrtr_close(dms_port);
+ dms_port = NULL;
+ }
+ rmtfs_stop();
+ cellular_up = cellular_registered = cellular_bars = cellular_dbm = cellular_radio = 0;
+}
+
+/* ---- the steps, and their end ----------------------------------------------------------------------- */
+
+static bool (*const steps[STEPS])(void) = {
+ step_survey, step_firmware, step_mailbox, step_header, step_image, step_our_item, step_start, step_queues, step_link,
+ step_qrtr, step_nas,
+};
+
+static bool keep; /* the DSP and the link stay up while the system runs (modem_keep, modem=1) */
+
+static void
+steps_over(uint32_t done)
+{
+ lck_mtx_lock(&dsp_lock);
+ if (dsp_started && done == last_step && keep) {
+ printf("mpss: THE MODEM STAYS ON while the system runs%s\n", link_runs ? ", and the link with it" : "");
+ } else if (dsp_started) {
+ dsp_shutdown("the steps are over");
+ }
+ lck_mtx_unlock(&dsp_lock);
+}
+
+/* When the run ends in order: a DSP that runs is shut down, for the next kernel to start again. */
+void
+redwood_mpss_stop(void)
+{
+ nas_stop(); /* the questions first, while the modem still listens */
+ lck_mtx_lock(&dsp_lock);
+ if (dsp_started) {
+ unsigned services = 0;
+ uint32_t node = qrtr_peer;
+ bool down;
+
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]); i++) {
+ services += qrtr_servers[i].there;
+ }
+ down = dsp_shutdown("the run is about to end");
+ /* (said again at the end: a long run's log keeps its beginning and its end, and the middle may be gone) */
+ tell("mpss: the modem is off: it had been up%s (node %u, %u services; NAS %s, %s, %u bars), it %s, and TrustZone %s\n",
+ node ? " and linked" : ", NOT LINKED", node, services, nas.connected ? "there" : "NOT THERE", nas.state.registered ?
+ "registered" : "not registered", nas.state.bars, stopped_by_its_word ? "stopped when asked" :
+ "DID NOT SAY IT HAD STOPPED", down ? "has shut it down" : "DID NOT SHUT IT DOWN");
+ }
+ lck_mtx_unlock(&dsp_lock);
+}
+
+/* The same on a panic's way back to the loader: one processor, interrupts off, no lock, no sleep. */
+void
+redwood_mpss_quiesce(void)
+{
+ if (!dsp_started) {
+ return;
+ }
+ leaving = true;
+ new_from = ~0u;
+ dsp_shutdown("the kernel is on its way back to the loader");
+}
+
+static void
+run_steps(void)
+{
+ uint32_t done = 0;
+
+ for (step = 1; step <= last_step; step++) {
+ if (!steps[step - 1]()) {
+ printf("mpss: step %u: FAILED; the steps after it are not attempted\n", step);
+ break;
+ }
+ printf("mpss: step %u: done\n", step);
+ done = step;
+ }
+ step = done;
+ new_from = STEPS + 1;
+ steps_over(done);
+ printf("mpss: over after step %u; %s\n", done, last_write);
+}
+
+/* modem_step=N: the steps once the root is there, then back to the loader if modem_return=1 says so. */
+static void
+steps_thread(__unused void *parameter, __unused wait_result_t result)
+{
+ for (unsigned waited = 0; waited < 300 && rootvnode == NULLVP; waited++) {
+ sleep_ms(200);
+ }
+ if (rootvnode == NULLVP) {
+ printf("mpss: no root file system within a minute: the steps are not performed\n");
+ } else {
+ run_steps();
+ }
+ if (option("modem_return")) {
+ printf("mpss: back to the loader (modem_return)\n");
+ loader_return();
+ }
+ thread_terminate(current_thread());
+}
+
+static void
+whole_thread(__unused void *parameter, __unused wait_result_t result)
+{
+ uint64_t began = now_us();
+ unsigned services = 0;
+
+ /* (the firmware may be on the root file system, which is not mounted yet when this begins: a minute at most) */
+ for (unsigned waited = 0; waited < 300 && rootvnode == NULLVP; waited++) {
+ sleep_ms(200);
+ }
+ run_steps();
+ for (unsigned i = 0; i < sizeof(qrtr_servers) / sizeof(qrtr_servers[0]); i++) {
+ services += qrtr_servers[i].there;
+ }
+ if (step == STEPS && dsp_started) {
+ tell("mpss: THE MODEM IS UP AND LINKED, %llu ms after the kernel began on it (the root's waited for included): node %u, %u "
+ "services; NAS %s, %s, %u bars (%d dBm, %s); it stays on, asked every two seconds\n", (now_us() - began) / 1000, qrtr_peer,
+ services, nas.connected ? "there" : "NOT THERE", nas.state.registered ? "REGISTERED" : "not registered", nas.state.bars,
+ nas.state.has_signal ? nas.state.dbm : 0, radio_name(nas.state.radio_interface));
+ nas_start();
+ } else {
+ tell("mpss: THE MODEM DID NOT COME UP: step %u failed (%s); see the kernel's own log for the steps\n", step + 1,
+ last_write);
+ }
+ thread_terminate(current_thread());
+}
+
+/* Called while the system starts (bsd/dev/redwood_input.c), after the Wi-Fi's start has been set going. */
+void
+redwood_mpss_init(void)
+{
+ uint32_t on = 0;
+ thread_t thread;
+
+ PE_parse_boot_argn("modem_step", &last_step, sizeof(last_step));
+ if (last_step == 0 && (!PE_parse_boot_argn("modem", &on, sizeof(on)) || on != 1)) {
+ return;
+ }
+ if (pe_arm_get_soc_base_phys() != REDWOOD_SOC_BASE) {
+ printf("mpss: modem=1 and modem_step=N are for the phone's modem, and this is not the phone\n");
+ return;
+ }
+ if (last_step == 0) {
+ /* all of it, on a thread of its own, and left up */
+ last_step = STEPS;
+ keep = true;
+ quiet = true;
+ pause_ms = 0;
+ listen_seconds = 1;
+ new_from = STEPS + 1;
+ if (kernel_thread_start(whole_thread, NULL, &thread) != KERN_SUCCESS) {
+ printf("mpss: no thread: the modem is not brought up\n");
+ return;
+ }
+ thread_deallocate(thread);
+ tell("mpss: the modem is being brought up beside the system's start (modem=1)\n");
+ return;
+ }
+ if (last_step > STEPS) {
+ last_step = STEPS;
+ }
+ keep = option("modem_keep");
+ new_from = last_step;
+ PE_parse_boot_argn("modem_new", &new_from, sizeof(new_from));
+ PE_parse_boot_argn("modem_pause", &pause_ms, sizeof(pause_ms));
+ printf("mpss: steps up to %u; those from %u on are new and wait %u ms after each announcement\n", last_step, new_from,
+ pause_ms);
+ /*
+ * The image is on the root (85 MB: too much for the small image), which is not mounted yet: the
+ * steps wait for it on a thread of their own, while the system goes on starting (sb_delay=N
+ * keeps SpringBoard out of the way meanwhile).
+ */
+ if (kernel_thread_start(steps_thread, NULL, &thread) != KERN_SUCCESS) {
+ printf("mpss: no thread: the steps are not performed\n");
+ return;
+ }
+ thread_deallocate(thread);
+ printf("mpss: the steps wait for the root file system, and the system goes on starting\n");
+}
diff --git a/xnu/files/bsd/dev/redwood_nas.c b/xnu/files/bsd/dev/redwood_nas.c
new file mode 100644
index 0000000..4ea9105
--- /dev/null
+++ b/xnu/files/bsd/dev/redwood_nas.c
@@ -0,0 +1,178 @@
+/*
+ * Small QMI NAS parser. Wire IDs and TLVs follow libqmi's qmi-service-nas.json:
+ * Serving System (0x24), Get Signal Info (0x4f), Signal Info indication (0x51).
+ * It only consumes reports. It never changes modem registration or radio power.
+ */
+#include <sys/redwood_nas.h>
+#include <sys/systm.h>
+#include <sys/redwood_qrtr.h>
+
+static uint16_t
+le16(const uint8_t *p)
+{
+ return (uint16_t)(p[0] | p[1] << 8);
+}
+
+static uint8_t
+bars_for(int16_t dbm)
+{
+ if (dbm < -120) return 1;
+ if (dbm < -110) return 2;
+ if (dbm < -100) return 3;
+ return 4;
+}
+
+int
+redwood_nas_apply(struct redwood_nas_state *state, const void *frame, size_t length)
+{
+ const uint8_t *data = frame;
+ struct redwood_nas_state next;
+ uint16_t message, payload_size;
+ size_t at;
+ bool serving = false, signal = false, success = false, result_seen = false;
+ int16_t level = 0;
+ uint8_t radio = 0;
+
+ if (state == NULL || data == NULL || length < 7) return -1;
+ if (data[0] != 2 && data[0] != 4) return 0;
+ message = le16(data + 3);
+ if (message != 0x24 && message != 0x4f && message != 0x51) return 0;
+ if ((message == 0x4f && data[0] != 2) || (message == 0x51 && data[0] != 4)) return 0;
+ payload_size = le16(data + 5);
+ if ((size_t)payload_size != length - 7) return -1;
+ next = *state;
+ for (at = 7; at < length;) {
+ uint8_t type;
+ uint16_t size;
+ const uint8_t *value;
+ if (length - at < 3) return -1;
+ type = data[at];
+ size = le16(data + at + 1);
+ at += 3;
+ if (size > length - at) return -1;
+ value = data + at;
+ at += size;
+ if (type == 2 && data[0] == 2) {
+ if (size != 4) return -1;
+ result_seen = true;
+ success = le16(value) == 0;
+ } else if (message == 0x24 && type == 1) {
+ static unsigned shown;
+
+ if (size < 5 || size < (size_t)5 + value[4]) return -1;
+ if (shown++ < 4) {
+ printf("nas: serving system: registration %u (0 none, 1 registered, 2 searching, 3 denied, 4 unknown), "
+ "CS %u, PS %u, network %u, %u radios%s%u\n", value[0], value[1], value[2], value[3], value[4],
+ value[4] ? ", the first " : "", value[4] ? value[5] : 0);
+ }
+ next.registered = value[0] == 1;
+ if (!next.registered) {
+ next.has_signal = false;
+ next.bars = 0;
+ }
+ serving = true;
+ } else if ((message == 0x4f || message == 0x51) && type == 0x17 && size >= 4) {
+ level = (int16_t)le16(value); radio = 12; signal = true;
+ } else if ((message == 0x4f || message == 0x51) && type == 0x14 && size >= 6 && radio != 12) {
+ level = (int16_t)le16(value + 2); radio = 8; signal = true;
+ } else if ((message == 0x4f || message == 0x51) && type == 0x13 && size >= 3 && !signal) {
+ level = (int8_t)value[0]; radio = 5; signal = true;
+ } else if ((message == 0x4f || message == 0x51) && type == 0x12 && size >= 1 && !signal) {
+ level = (int8_t)value[0]; radio = 4; signal = true;
+ }
+ }
+ if (data[0] == 2 && (!result_seen || !success)) return 0;
+ if (signal && level >= -150 && level <= -30) {
+ next.has_signal = true;
+ next.dbm = level;
+ next.radio_interface = radio;
+ next.bars = next.registered ? bars_for(level) : 0;
+ }
+ if (!serving && !signal) return 0;
+ if (next.registered == state->registered && next.has_signal == state->has_signal &&
+ next.radio_interface == state->radio_interface && next.dbm == state->dbm && next.bars == state->bars) return 0;
+ *state = next;
+ return 1;
+}
+
+static void
+nas_receive(void *context, uint32_t node, uint32_t port, const void *data, size_t length)
+{
+ struct redwood_nas_client *client = context;
+
+ if (!client->connected || node != client->peer_node || port != client->peer_port) return;
+ client->answers++;
+ if (redwood_nas_apply(&client->state, data, length) == 1 && client->changed != NULL)
+ client->changed(client->context, &client->state);
+}
+
+static void
+nas_server(void *context, bool arrived, uint32_t service, uint32_t version, uint32_t instance,
+ uint32_t node, uint32_t port)
+{
+ struct redwood_nas_client *client = context;
+ (void)version;
+ (void)instance;
+ if (service != 3) return; /* QMI NAS */
+ if (arrived && !client->connected) {
+ client->connected = true;
+ client->peer_node = node;
+ client->peer_port = port;
+ if (client->port != NULL) (void)redwood_nas_client_poll(client);
+ } else if (!arrived && client->connected && node == client->peer_node && port == client->peer_port) {
+ client->connected = false;
+ client->state = (struct redwood_nas_state){0};
+ if (client->changed != NULL) client->changed(client->context, &client->state);
+ }
+}
+
+int
+redwood_nas_client_open(struct redwood_nas_client *client, const struct redwood_qrtr_ops *transport,
+ void (*changed)(void *, const struct redwood_nas_state *), void *context)
+{
+ int error;
+ if (client == NULL || transport == NULL || transport->open == NULL || transport->lookup == NULL ||
+ transport->send == NULL || transport->close == NULL) return -1;
+ *client = (struct redwood_nas_client){0};
+ client->transport = transport;
+ client->changed = changed;
+ client->context = context;
+ client->port = transport->open(nas_receive, nas_server, client);
+ if (client->port == NULL) return -1;
+ error = transport->lookup(client->port, 3, 0, 0);
+ if (error != 0) {
+ redwood_nas_client_close(client);
+ return error;
+ }
+ return 0;
+}
+
+int
+redwood_nas_client_poll(struct redwood_nas_client *client)
+{
+ uint8_t request[7] = {0};
+ const uint16_t messages[2] = {0x24, 0x4f};
+ int error;
+ if (client == NULL || client->port == NULL || !client->connected) return -1;
+ for (unsigned i = 0; i < 2; i++) {
+ client->transaction++;
+ if (client->transaction == 0) client->transaction++;
+ request[1] = (uint8_t)client->transaction;
+ request[2] = (uint8_t)(client->transaction >> 8);
+ request[3] = (uint8_t)messages[i];
+ request[4] = (uint8_t)(messages[i] >> 8);
+ error = client->transport->send(client->port, client->peer_node, client->peer_port, request, sizeof(request));
+ if (error != 0) return error;
+ }
+ return 0;
+}
+
+void
+redwood_nas_client_close(struct redwood_nas_client *client)
+{
+ if (client != NULL && client->port != NULL) {
+ client->transport->close(client->port);
+ client->port = NULL;
+ client->connected = false;
+ }
+}
diff --git a/xnu/files/bsd/dev/redwood_qmi.c b/xnu/files/bsd/dev/redwood_qmi.c
index e88ec7b..130154d 100644
--- a/xnu/files/bsd/dev/redwood_qmi.c
+++ b/xnu/files/bsd/dev/redwood_qmi.c
@@ -759,9 +759,9 @@ qmi_server(void *context, bool arrived, uint32_t service, uint32_t version, uint
}
int
-qmi_handle_init(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *ops, const struct qmi_msg_handler *handlers)
+redwood_qmi_handle_init_transport(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *ops,
+ const struct qmi_msg_handler *handlers, const struct redwood_qrtr_ops *transport)
{
- const struct redwood_qrtr_ops *transport = qmi_transport;
struct qmi_state *s;
qmi->sock = NULL;
@@ -791,6 +791,12 @@ qmi_handle_init(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *op
return 0;
}
+int
+qmi_handle_init(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *ops, const struct qmi_msg_handler *handlers)
+{
+ return redwood_qmi_handle_init_transport(qmi, largest, ops, handlers, qmi_transport);
+}
+
/* Nobody may be waiting in qmi_txn_wait() on this handle any more. */
void
qmi_handle_release(struct qmi_handle *qmi)
diff --git a/xnu/files/bsd/dev/redwood_ufs.c b/xnu/files/bsd/dev/redwood_ufs.c
index efa2fda..6d2d6da 100644
--- a/xnu/files/bsd/dev/redwood_ufs.c
+++ b/xnu/files/bsd/dev/redwood_ufs.c
@@ -1062,6 +1062,41 @@ ufs_entry(const char *name)
return BLOCK_MAX_PARTS;
}
+/*
+ * For the kernel's own drivers (the modem's file systems, bsd/dev/redwood_mpss.c): bytes
+ * [offset, offset + len) of the partition of that name (or of its _a or _b slot), and how big it
+ * is. ENXIO without the storage, ENOENT without the partition; nothing is written.
+ */
+int
+redwood_ufs_read_partition(const char *name, uint64_t offset, uint8_t *dst, uint64_t len, uint64_t *bytes)
+{
+ uint32_t part;
+
+ if (ufs_page == NULL) {
+ return ENXIO;
+ }
+ part = ufs_entry(name);
+ for (const char *slot = "_a"; part == BLOCK_MAX_PARTS && slot[0] != 0; slot = slot[1] == 'a' ? "_b" : "") {
+ char slotted[48];
+
+ snprintf(slotted, sizeof(slotted), "%s%s", name, slot);
+ part = ufs_entry(slotted);
+ }
+ if (part == BLOCK_MAX_PARTS) {
+ return ENOENT;
+ }
+ if (bytes != NULL) {
+ *bytes = ufs_part_bytes(part);
+ }
+ if (len == 0) {
+ return 0;
+ }
+ if (offset + len > ufs_part_bytes(part)) {
+ return EINVAL;
+ }
+ return ufs_read(part, offset, dst, len);
+}
+
/*
* Which logical unit each unit the loader lists is (their device paths do not say): unit k is
* taken to be LUN k (ordinary logical units only: never a well-known one), and it is so only if
diff --git a/xnu/files/bsd/sys/redwood_input.h b/xnu/files/bsd/sys/redwood_input.h
index ff84b74..57c5122 100644
--- a/xnu/files/bsd/sys/redwood_input.h
+++ b/xnu/files/bsd/sys/redwood_input.h
@@ -45,6 +45,7 @@ void redwood_buttons_init(void); /* boot-arg buttons=1: the phone's power
void redwood_ufs_init(void); /* ufs=2 to xnuloader: the phone's storage, read only, through the loader (ufs_shared_pa=) */
void redwood_ufs_report(const char *when); /* where reading the storage has spent its time so far */
bool redwood_ufs_dev(dev_t dev); /* one of the storage's devices */
+int redwood_ufs_read_partition(const char *name, uint64_t offset, uint8_t *dst, uint64_t len, uint64_t *bytes); /* for drivers */
void redwood_battery_init(void); /* boot-arg battery=1: Qualcomm UEFI charger state shared by xnuloader */
int redwood_battery_temperature(void); /* the battery's temperature in C, -273 while it is not known */
/* one transfer at a time through the PMIC arbiter for this system (bsd/dev/redwood_buttons.c) */
@@ -78,6 +79,7 @@ void redwood_adreno_stop(void); /* before that, where the ending is no p
void redwood_adreno_minute(void); /* once a minute: a line about how warm the GPU is, if it is in use */
void redwood_wifi_init(void); /* boot-arg wifi=1: the phone's Wi-Fi; wifi_step=N: the first N steps of it */
void redwood_adsp_init(void); /* boot-arg adsp=1: the phone's audio DSP started and linked; adsp_step=N: the first N steps */
+void redwood_mpss_init(void); /* boot-arg modem=1: the phone's modem started and linked; modem_step=N: the first N steps */
void redwood_wifi_stop(void); /* where a run ends in order: the Wi-Fi DSP, if it was started, shut down */
int redwood_wifi_say(const char *format, ...) __printflike(1, 2); /* a line of the Wi-Fi steps' log */
int redwood_wifi_tell(const char *format, ...) __printflike(1, 2); /* ... one that is said even with wifi=1 */
diff --git a/xnu/files/bsd/sys/redwood_nas.h b/xnu/files/bsd/sys/redwood_nas.h
new file mode 100644
index 0000000..9fe7677
--- /dev/null
+++ b/xnu/files/bsd/sys/redwood_nas.h
@@ -0,0 +1,43 @@
+/* Minimal, transport-independent QMI NAS network state for cellular UI clients. */
+#ifndef _SYS_REDWOOD_NAS_H_
+#define _SYS_REDWOOD_NAS_H_
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+
+struct redwood_nas_state {
+ bool registered;
+ bool has_signal;
+ uint8_t radio_interface; /* QMI NAS: 4 GSM, 5 UMTS, 8 LTE, 12 NR */
+ int16_t dbm; /* LTE/NR RSRP; GSM/UMTS RSSI */
+ uint8_t bars; /* 0 until registration and a valid signal reading */
+};
+
+/* Apply one complete QMI NAS frame. 1: state changed; 0: ignored; -1: malformed. */
+int redwood_nas_apply(struct redwood_nas_state *state, const void *frame, size_t length);
+
+struct redwood_qrtr_ops;
+struct redwood_qrtr_port;
+
+/* One NAS service client on a specified processor's QRTR transport. Callbacks are serialized
+ * by that transport; the caller must synchronize a state read on any other thread. */
+struct redwood_nas_client {
+ const struct redwood_qrtr_ops *transport;
+ struct redwood_qrtr_port *port;
+ struct redwood_nas_state state;
+ void (*changed)(void *context, const struct redwood_nas_state *state);
+ void *context;
+ uint32_t peer_node, peer_port;
+ uint16_t transaction;
+ uint32_t answers; /* NAS frames received, changed or not */
+ bool connected;
+};
+
+int redwood_nas_client_open(struct redwood_nas_client *client, const struct redwood_qrtr_ops *transport,
+ void (*changed)(void *, const struct redwood_nas_state *), void *context);
+/* Get Serving System and Get Signal Info. Poll regularly until indications are configured. */
+int redwood_nas_client_poll(struct redwood_nas_client *client);
+void redwood_nas_client_close(struct redwood_nas_client *client);
+
+#endif
diff --git a/xnu/files/bsd/sys/redwood_qmi.h b/xnu/files/bsd/sys/redwood_qmi.h
index 2be5586..62c3b81 100644
--- a/xnu/files/bsd/sys/redwood_qmi.h
+++ b/xnu/files/bsd/sys/redwood_qmi.h
@@ -151,6 +151,8 @@ struct qmi_handle {
struct socket *sock; /* what kernel_connect() is given; not NULL while the handle is in use */
};
+struct redwood_qrtr_ops;
+
struct qmi_txn {
struct qmi_handle *qmi;
uint16_t id;
@@ -166,6 +168,9 @@ struct qmi_txn {
* errno: -ENXIO where there is no router (redwood_qmi_set_transport() has not been called).
*/
int qmi_handle_init(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *ops, const struct qmi_msg_handler *handlers);
+/* Bind a handle to a specific processor's QRTR transport (e.g. WPSS or MPSS). */
+int redwood_qmi_handle_init_transport(struct qmi_handle *qmi, size_t largest, const struct qmi_ops *ops,
+ const struct qmi_msg_handler *handlers, const struct redwood_qrtr_ops *transport);
void qmi_handle_release(struct qmi_handle *qmi);
/* Ask for a service: ops->new_server is called for every server of it there is and will be. */
int qmi_add_lookup(struct qmi_handle *qmi, unsigned int service, unsigned int version, unsigned int instance);
diff --git a/xnu/files/osfmk/arm64/loader_return.c b/xnu/files/osfmk/arm64/loader_return.c
index a058bb5..bde30ee 100644
--- a/xnu/files/osfmk/arm64/loader_return.c
+++ b/xnu/files/osfmk/arm64/loader_return.c
@@ -51,6 +51,8 @@ extern void redwood_wifi_stop(void);
extern void redwood_wifi_quiesce(void);
extern void redwood_adsp_stop(void);
extern void redwood_adsp_quiesce(void);
+extern void redwood_mpss_stop(void);
+extern void redwood_mpss_quiesce(void);
extern void redwood_ddr_quiesce(void);
extern void redwood_haptic_quiesce(void);
extern void redwood_torch_quiesce(void);
@@ -373,6 +375,7 @@ loader_return(void)
redwood_adreno_quiesce(); /* the GPU block, if it was started, has tables in memory that is about to be reused */
redwood_wifi_quiesce(); /* the Wi-Fi DSP, if it still runs (a panic), is not left to itself */
redwood_adsp_quiesce(); /* nor the audio DSP */
+ redwood_mpss_quiesce(); /* nor the modem */
redwood_ddr_quiesce(); /* and the vote for the memory's bandwidth taken away */
redwood_haptic_quiesce(); /* the vibration motor's chip back into reset, if it was let out */
redwood_torch_quiesce(); /* the flashlight off, if it was lit */
@@ -440,6 +443,7 @@ loader_return_timer(__unused thread_call_param_t p0, __unused thread_call_param_
redwood_adreno_stop(); /* a GPU kept for the system is given back while there is a thread to wait in */
redwood_wifi_stop(); /* and the Wi-Fi DSP, if it runs, shut down */
redwood_adsp_stop(); /* and the audio DSP */
+ redwood_mpss_stop(); /* and the modem */
}
if (PE_parse_boot_argn("return_cpu", &cpu, sizeof(cpu)) && cpu <= (uint32_t)ml_get_max_cpu_number() &&
cpu_xcall((int)cpu, loader_return_test, (void *)(uintptr_t)(as_panic ? 2 : 1)) == KERN_SUCCESS) {
diff --git a/xnu/redwood.patch b/xnu/redwood.patch
index 8f7dd87..2f6b425 100644
--- a/xnu/redwood.patch
+++ b/xnu/redwood.patch
@@ -62,10 +62,10 @@ index a4d25040f38f..aad72993d139 100644
kern_debug.o_INCFLAGS += -I$(SRCROOT)/osfmk
diff --git a/bsd/conf/files b/bsd/conf/files
-index 44274016a4cf..9cd24b42caca 100644
+index 44274016a4cf..08878fedb36e 100644
--- a/bsd/conf/files
+++ b/bsd/conf/files
-@@ -109,6 +109,37 @@ bsd/dev/munge.c optional config_requires_u32_munging
+@@ -109,6 +109,39 @@ bsd/dev/munge.c optional config_requires_u32_munging
bsd/dev/unix_startup.c standard
bsd/dev/monotonic.c optional monotonic
@@ -91,19 +91,21 @@ index 44274016a4cf..9cd24b42caca 100644
+bsd/dev/redwood_mic.c standard
+bsd/dev/redwood_wifi.c standard
+bsd/dev/redwood_adsp.c standard
++bsd/dev/redwood_mpss.c standard
+bsd/dev/redwood_image.c standard
+bsd/dev/redwood_virtio_blk.c standard
+bsd/dev/redwood_sparse.c standard
+bsd/dev/redwood_apfs_check.c optional apfs
+bsd/dev/redwood_sound.c standard
+bsd/dev/redwood_qmi.c standard
++bsd/dev/redwood_nas.c standard
+bsd/dev/redwood_wlfw.c standard
+bsd/dev/redwood_lx.c standard
+bsd/dev/redwood_sta.c standard
bsd/dev/dev_perfmon.c optional config_perfmon
bsd/libkern/crc32.c standard
-@@ -155,6 +186,14 @@ bsd/miscfs/devfs/devfs_vfsops.c optional devfs
+@@ -155,6 +188,14 @@ bsd/miscfs/devfs/devfs_vfsops.c optional devfs
bsd/miscfs/mockfs/mockfs_fsnode.c optional mockfs
bsd/miscfs/mockfs/mockfs_vfsops.c optional mockfs
bsd/miscfs/mockfs/mockfs_vnops.c optional mockfs
@@ -118,7 +120,7 @@ index 44274016a4cf..9cd24b42caca 100644
bsd/miscfs/routefs/routefs_ops.c optional routefs
-@@ -361,6 +400,7 @@ bsd/security/audit/audit_syscalls.c standard
+@@ -361,6 +402,7 @@ bsd/security/audit/audit_syscalls.c standard
bsd/security/audit/audit_worker.c optional config_audit
bsd/pthread/pthread_shims.c standard
@@ -126,7 +128,7 @@ index 44274016a4cf..9cd24b42caca 100644
bsd/pthread/pthread_priority.c standard
bsd/pthread/pthread_workqueue.c standard
-@@ -383,6 +423,7 @@ bsd/kern/kern_debug.c standard
+@@ -383,6 +425,7 @@ bsd/kern/kern_debug.c standard
bsd/kern/kern_symfile.c standard
bsd/kern/kern_descrip.c standard
bsd/kern/kern_guarded.c standard
@@ -1865,7 +1867,7 @@ index 5bd6f23dd10c..14e0f17409f5 100644
ifneq ($(filter T6000 T8101 VMAPPLE T6000 T8101 VMAPPLE,$(CURRENT_MACHINE_CONFIG)),)
LDFLAGS_KERNEL_ONLY += -rdynamic -Wl,-force_load,$(KDKROOT)/System/Library/KernelSupport/lib$(CURRENT_MACHINE_CONFIG).os.$(CURRENT_KERNEL_CONFIG).a
diff --git a/makedefs/MakeInc.top b/makedefs/MakeInc.top
-index 0316792a3f42..bbef0a4b26d4 100644
+index 0316792a3f42..b9c195b09cd1 100644
--- a/makedefs/MakeInc.top
+++ b/makedefs/MakeInc.top
@@ -285,7 +285,7 @@ endif
@@ -1873,7 +1875,7 @@ index 0316792a3f42..bbef0a4b26d4 100644
# $(warning VARIANT_CONFIGS is $(VARIANT_CONFIGS))
-MEMORY_SIZE := $(shell /usr/sbin/sysctl -n hw.memsize)
-+MEMORY_SIZE := $(shell awk '/MemTotal/{printf "%d", $$2*1024}' /proc/meminfo)
++MEMORY_SIZE := $(shell if [ -r /proc/meminfo ]; then awk '/MemTotal/{printf "%d", $$2*1024}' /proc/meminfo; else sysctl -n hw.memsize; fi)
# Assume LTO scaling by default, unless it is being explicitly passed on the command-line
LARGE_BUILD_FOOTPRINT := $(if $(BUILD_LTO),$(BUILD_LTO),1)