Run a virtual iPhone on an Apple Silicon Mac.
Swift
docs(research): record the archive measurements and where this branch got to
Lakr56b4e3198aba
diff --git a/research/archive_extraction_contracts.md b/research/archive_extraction_contracts.mdnew file mode 100644index 00000000..97a2e1bb--- /dev/null+++ b/research/archive_extraction_contracts.md@@ -0,0 +1,129 @@+# VPhoneArchive: what was measured, and what bit++> 2026-09-23, branch `vphone-intg-update`. libarchive 3.8.9 from+> `Lakr233/libarchive.xcframework` 0.1.1, macOS 26 (26A428).+>+> Companion to `research/libarchive_xcframework_validation.md`, which covers+> whether libzstd and the liblzma MT encoder are compiled in. This one is+> about behaviour on disk.++## Why one program replaces four++`gtar`, `bsdtar`, `unzip` and `zstd` all go away. zstd is the one that forces+the issue: **both the system tar and GNU tar shell out to a `zstd(1)` for that+filter**, and libzstd is in neither `/usr/lib` nor the SDK. On a machine+without Homebrew, every `.zst` step of a CFW install fails. Side by side, with+no zstd reachable:++```+$ env -i PATH=/usr/bin:/bin vphone-archive extract -f t.tzst -C out+extracted 5 entries to .../out++$ env -i PATH=/usr/bin:/bin /usr/bin/tar -tf t.tzst+tar: Error opening archive: Can't initialize filter; unable to run program "zstd -d -qq"+```++## `--no-overwrite-dir`: measured, and it corrected me twice++GNU tar has the flag; libarchive has no equivalent, and neither of the two+things it does offer is the same thing.++**First measurement said the flag was unnecessary.** An existing directory's+mode came back unchanged with the flag off, which would have meant+`man 3 archive_write_disk`'s "existing directories will have their permissions+updated" no longer held and `noOverwriteDir` was decorative.++**That reading was wrong, and the way it was wrong is worth remembering.** The+extraction under test was failing partway — `ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATHS`+was rejecting the rewritten paths — so nothing was being applied to anything.+A measurement taken from a failing code path measures the failure. With+extraction actually working: **0700 in, 0777 out.** The manual is right, and+the flag is load-bearing.++`ARCHIVE_EXTRACT_NO_OVERWRITE` is still not a substitute, and it is wrong in+the opposite direction: it skips *any* existing object, so a regular file that+should be replaced silently is not. Both halves are pinned in tests — files+under a skipped directory still get written, and an existing regular file+still gets replaced.++The implementation is an `lstat` before `archive_write_header` on directory+entries. `lstat`, not `stat`: a symlink pointing at a directory is a symlink,+and leaving it alone is a different decision. Measured: with a symlink planted+where a directory entry would land, `SECURE_SYMLINKS` removes it and creates a+real directory, and nothing is written through it.++## Two macOS path traps++Both cost real time, and both come from the same place: `/tmp` and `/var` are+symlinks into `/private`.++### `SECURE_SYMLINKS` inspects every component of an absolute path++Give it `/var/folders/.../dest/file` and it refuses the whole extraction with+"Cannot extract through symlink /var/...", because `/var` is one. The fix is to+resolve the destination once, up front, so only components *below* it are left+for the flag to judge — which is the part worth judging.++### Foundation's path resolution goes the wrong way++`URL.resolvingSymlinksInPath()` and `.standardizedFileURL` both turn+`/private/tmp/x` into `/tmp/x` — they *strip* a `/private` prefix rather than+resolving the symlink. That is the opposite of what is needed here, and it+broke two separate things:++- extraction, via the trap above;+- packing, where a source under `/private` never matched its own standardised+ root, so **every member was stored under an absolute path**. The test suite+ missed this entirely, because `FileManager.temporaryDirectory` is+ `/var/folders/...`, which has no `/private` to strip. It showed up the first+ time the CLI was run by hand from a `/private/tmp` scratch directory.++`VPhoneArchivePaths.resolved` uses `realpath(3)` and exists so this decision is+made in one place. **Do not replace it with the Foundation equivalents.**++For the path-containment check, `URL.standardized` is the right one: it+collapses `.` and `..` without touching symlinks, so it does not undo the+`realpath` above.++## `SECURE_NOABSOLUTEPATHS` is deliberately not set++The migration plan asks for all three `SECURE_*` flags unconditionally. Two are+set; this one is not, and the reason is structural rather than a preference.++`archive_write_disk` has no destination-directory concept — it writes relative+to the process's working directory. Reaching an arbitrary destination therefore+means one of:++1. **chdir there.** What tar does, and it was tried. It makes extraction+ process-global and non-reentrant, and it broke immediately under parallel+ tests, leaving files in the repository root.+2. **Give each entry an absolute target.** Then this flag rejects the entry we+ just computed.++So option 2, without the flag, plus an explicit check: each target is+normalised and required to sit under the destination before anything is+written. That is **stronger** than the flag, which only asks whether a stored+path begins with a slash; the check asks where the path actually lands. There+is a test with a hand-built `../escaped` member, checksum and all.++## Known wart++libarchive leaves zero-byte `tar.XXXXXXXX` files in the process's **working+directory** while restoring macOS metadata — a relative `mkstemp` template in+its own sources, which is why `TMPDIR` does not redirect them. Harmless, and+gitignored, but worth knowing that `vphone-archive` running as root during an+install will leave them wherever the installer's cwd is.++## Not yet done++- **Call sites are not switched.** The `$TAR` invocations in `cfw_install*.sh`+ write to a **mounted guest volume** as root, with ownership restored by+ number. Nothing in a unit test covers that, and getting it wrong produces a+ guest that does not boot. It wants a real install and the tree-fingerprint+ comparison from the plan (uid/gid, ACLs, xattrs, hardlink grouping,+ `st_blocks`) before it is switched.+- **`vm export` / `import`** still go through the two-stage tar pipe. The format+ contract there — gnutar, `.tzst` at zstd 3, `.txz` at xz 9 — is expressed in+ `VPhoneArchiveFormat` and `VPhoneArchiveCompression` but not yet wired in,+ and the bidirectional compatibility check (old export read by new import, and+ new export read by `/usr/bin/tar`) has not been run.diff --git a/research/intg_update_status.md b/research/intg_update_status.mdnew file mode 100644index 00000000..aa104c67--- /dev/null+++ b/research/intg_update_status.md@@ -0,0 +1,83 @@+# `vphone-intg-update` — where this got to++> 2026-09-23. Branch off `qof-update-26-fall` @ `6d5ce7d`, pushed.+>+> Plan: `~/Desktop/vphone-cli-migration-plan.md`, and the approved execution+> order in `~/.claude/plans/mellow-weaving-gem.md`.++## Done++| step | what | state |+| --- | --- | --- |+| S0 | libzstd static in the xcframework? | **yes** — proven at runtime |+| S2 | liblzma MT encoder? | **yes** — 4.84x at 1 GiB |+| 1 | entitlements moved off `vphone-cli` onto `vphone-vm` | done, verified |+| 2 | `vphone-letmein` in, `amfidont` scripts out, docs in 6 languages | done |+| 3 | P0's three Python files | done; two deleted, one kept as reference |+| 4 | `custom-firmware-kit` → `cfw-kit/` | done, as-is |+| 5a | admission gates 1–3 (`make check-aux`) | done; fails on bundled ldid, as designed |+| 5b | `VPhoneArchive` + `vphone-archive` | library and binary done; **call sites not switched** |++Tests: `VPhoneCoreTests` 152/152, `VPhoneArchiveTests` 13/13. The 14+`FirmwarePatcherTests` failures are pre-existing — they need+`ipsws/patch_refactor_input/`, which is not in the repo.++## Needs you, and a machine++Nothing below could be done without root or a real guest.++1. **Can `vphone-vm` start a VM holding the entitlements alone?** Everything+ in step 1 rests on this, and none of it is proven until a guest boots.+2. **`vphone-letmein` end to end** — the sudo prompt, `--hold 10` restoring+ while the guest keeps running, Ctrl-C reaching the guest. The window+ defaults to 10 seconds, which is a guess; measure it.+3. **Location and TouchID**, which depend on TCC attributing the usage+ strings to `vphone-vm`. It is `CFBundleExecutable`, so it should — worth+ confirming rather than assuming.+4. **Bridged networking**, now validated at boot instead of at config time.+5. **`cfw flip-snapshot` against a real `Disk.img`.** The byte comparison+ against the Python passed on a synthetic fixture; repeat it once on a real+ image, then delete `tools/apfs_snap_rename.py`.++## Next, in order++1. **Switch the archive call sites.** `vphone-archive` is built, bundled and+ tested, and nothing calls it yet. The IPSW unzip in `fw_prepare.sh` and the+ host-side temp extractions are low risk. The `$TAR` calls in+ `cfw_install*.sh` are not: they write to a mounted guest volume as root,+ and want the tree-fingerprint comparison (uid/gid, ACLs, xattrs, hardlink+ grouping, `st_blocks`) against GNU tar before being switched. See+ `research/archive_extraction_contracts.md`.+2. **`VPhoneSign`**, which is what clears the last two admission-gate+ failures. `ldid` is the only binary we ship that is already+ non-self-contained. The plan's §3.11 has the measurements; the+ `signcert.p12` needs re-wrapping with a password first, and the old+ empty-password copy has to stay for the `--use-ldid` escape hatch.+3. **`vm export` / `import`** onto `VPhoneArchive`, keeping gnutar, `.tzst`+ at zstd 3 and `.txz` at xz 9, and checking compatibility both ways.+4. **Gate 4** — a machine with no Homebrew. Still the only thing that can+ support "it works elsewhere"; gates 1–3 are necessary and not sufficient,+ which the script says out loud.++## Three things the plan got wrong++Recorded because they were measured, not reasoned about.++- **`ipsw` / `aea` / `ldid` are called by absolute `/opt/homebrew` path from+ Swift**, not through `PATH`. §1.4 counts the shell call sites. Gate 2 lists+ all of them.+- **`sources/vphone.entitlements` has 7 keys**, not the 4 the plan says or the+ 5 `CLAUDE.md` said. Two of them — location and BiometricKit — belong to+ `Devices/`, which is why they all landed on `vphone-vm`.+- **The admission rule caught a bug the plan did not predict**: signing+ `vphone-vm` first sealed the bundle over its siblings in an earlier state,+ and `codesign -v` reported "nested code is modified or invalid". The main+ executable is signed last now, and both build paths verify the seal.++## And one I nearly got wrong++The first measurement of `--no-overwrite-dir` said libarchive already leaves+an existing directory's mode alone, which would have made the flag+decorative — and it was taken from an extraction that was failing partway and+applying nothing. A measurement from a failing code path measures the failure.+It is 0700 in, 0777 out when extraction actually works, and the flag matters.