The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
Preserve a T2 Mac’s wireless firmware before future installation (#841)
Dee995647d318d8
diff --git a/.github/workflows/os-t2-firmware.yml b/.github/workflows/os-t2-firmware.ymlnew file mode 100644index 000000000..271c1eccb--- /dev/null+++ b/.github/workflows/os-t2-firmware.yml@@ -0,0 +1,63 @@+name: Harness OS Apple firmware preservation++on:+ workflow_dispatch:+ push:+ branches: ['os-polish/t2-firmware']+ paths:+ - 'os/platforms/apple-t2/firmware_names.py'+ - 'os/tools/prepare-t2-firmware.py'+ - 'os/tests/test_t2_firmware.py'+ - 'os/tests/t2_firmware_reference.py'+ - '.github/workflows/os-t2-firmware.yml'++permissions:+ contents: read++concurrency:+ group: harness-t2-firmware-${{ github.ref }}+ cancel-in-progress: false++jobs:+ preservation:+ strategy:+ fail-fast: false+ matrix:+ os: [ubuntu-24.04, macos-15]+ runs-on: ${{ matrix.os }}+ timeout-minutes: 6+ steps:+ - uses: actions/checkout@v7+ - uses: actions/setup-python@v6+ with:+ python-version: '3.14.7'+ - name: Verify local export, archive refusal and private staging+ run: |+ python3 - <<'PY'+ import json+ from pathlib import Path+ folder = Path('.harness/validation')+ folder.mkdir(parents=True, exist_ok=True)+ plan = {'reason': 'T2 firmware preservation on the native host; fixture bytes only, no disk, driver or system writes.', 'minimum_free_gib': 1,+ 'checks': [{'name': 'firmware-contract', 'argv': ['python3', '-m', 'unittest', 'discover', '-s', 'os/tests', '-p', 'test_t2_firmware.py', '-v'], 'timeout_seconds': 60}]}+ (folder / 't2-firmware.json').write_text(json.dumps(plan))+ PY+ python3 scripts/validate-change.py .harness/validation/t2-firmware.json+ - name: Compare naming and bytes with reviewed upstream conversion+ run: |+ mkdir -p os/work/t2-firmware-reference+ curl --fail --location --proto '=https' --proto-redir '=https' --max-time 45 \+ https://raw.githubusercontent.com/t2linux/wiki/11fc0a8d8cfb61affd0cb9d1ac245c1b6c16d3cd/docs/tools/firmware.sh \+ --output os/work/t2-firmware-reference/upstream.sh+ python3 os/tests/t2_firmware_reference.py \+ --upstream os/work/t2-firmware-reference/upstream.sh \+ --output os/test-results/t2-firmware/reference.json+ - name: Retain source and host evidence without firmware+ if: always()+ uses: actions/upload-artifact@v6+ with:+ name: harness-t2-firmware-${{ matrix.os }}+ path: |+ .harness/validation/+ os/test-results/t2-firmware/+ retention-days: 7diff --git a/os/platforms/apple-t2/README.md b/os/platforms/apple-t2/README.mdindex c9455a5a7..e560486cd 100644--- a/os/platforms/apple-t2/README.md+++ b/os/platforms/apple-t2/README.md@@ -24,6 +24,45 @@ built-in input, Wi-Fi, audio, graphics and suspend on real Macs. In particular,iMac Pro firmware needs the macOS export route. Never add a moving unsignedrepository or copy Apple firmware into the public ISO as a shortcut.+## Preserve wireless firmware++`os/tools/prepare-t2-firmware.py` is a separate preparation tool. It reads the+Intel Mac's own `/usr/share/firmware` while macOS is running and creates a local+archive. It does not install Harness, mount or change a disk, or enable the T2+installer. It is not added to the generic PC image.++From a checkout on the Intel Mac, with Python 3.10 or later already available:++```sh+python3 os/tools/prepare-t2-firmware.py export --output ~/Downloads/harness-apple-firmware.tar+```++Keep that archive with the computer. It contains Apple's firmware and should not+be committed or uploaded to a public release. The command never overwrites an+existing export. No macOS account files, serial numbers or network passwords are+collected. An iMac Pro export also reads its calibration filenames from IORegistry;+it stops if those files cannot be identified. Three BCM4377 models additionally+require the Bluetooth firmware available in macOS Monterey or later.++The future T2 installer can validate the archive against the detected model and+stage it in a fresh RAM-backed directory before erasing the source disk. The+tool's `verify` and `stage` commands implement that data boundary; `--model` is+explicit for development, not an installer override. Archive checks reject links,+special files, unexpected paths, duplicate entries, excessive sizes and altered+inventories. Every staged file has a size and SHA-256 in the retained manifest.+These checks establish local copy integrity, not a signature from Apple or proof+that a physical radio works. Restoring into the installed system and preserving+the firmware through updates/recovery remain part of the T2 installer integration.++The naming rules derive from the MIT-licensed upstream conversion script at+`t2linux/wiki@11fc0a8d8cfb61affd0cb9d1ac245c1b6c16d3cd`; its full SHA-256 is+`c1c1d8aa25bb5f089e46ccd0d9738fc13bfbd784f499aa924466c690f059961e`.+The **Harness OS Apple firmware preservation** workflow checks local CLI round+trips and malformed-input refusal on Linux and macOS, then compares filenames and+bytes with that pinned converter. Fixtures contain invented bytes only. No+physical Mac's firmware is stored in CI. The existing T2 installation refusal+remains in place until the USB kernel, installer and recovery path are complete.+Upstream references: [maintained kernel](https://github.com/NoaHimesaka1873/linux-t2-arch),[early input and kernel parameters](https://wiki.t2linux.org/guides/postinstall/),[wireless firmware](https://wiki.t2linux.org/guides/wifi-bluetooth/), anddiff --git a/os/platforms/apple-t2/firmware_names.py b/os/platforms/apple-t2/firmware_names.pynew file mode 100644index 000000000..7e004065e--- /dev/null+++ b/os/platforms/apple-t2/firmware_names.py@@ -0,0 +1,137 @@+"""Translate local Apple wireless firmware names into Linux driver requests.++Adapted from the naming/pruning rules in t2linux's firmware.sh, revision+11fc0a8d8cfb61affd0cb9d1ac245c1b6c16d3cd (2026-10-06 review).+https://github.com/t2linux/wiki/blob/11fc0a8d8cfb61affd0cb9d1ac245c1b6c16d3cd/docs/tools/firmware.sh+The original Python implementation is based on work by the Asahi Linux Contributors.++Copyright (C) 2024 Aditya Garg <[email protected]>+Copyright (C) 2024 Orlando Chamberlain <[email protected]>+Copyright (C) 2024 Sharpened Blade <[email protected]>++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to deal+in the Software without restriction, including without limitation the rights+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell+copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in all+copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE+SOFTWARE.+"""+import re+++WIFI_FOLDERS = {'C-4355__s-C1', 'C-4364__s-B2', 'C-4364__s-B3', 'C-4377__s-B3'}+EXTENSIONS = {'trx': 'bin', 'txt': 'txt', 'clmb': 'clm_blob', 'txcb': 'txcap_blob'}+DIMENSIONS = ('C', 's', 'P', 'M', 'V', 'm', 'A')+++def nvram(data):+ lines = []+ for line in data.decode('ascii').split('\n'):+ if not line:+ continue+ key, separator, value = line.partition('=')+ if not separator or not key.strip():+ raise ValueError('Invalid Apple Wi-Fi NVRAM.')+ lines.append(key.strip() + '=' + value + '\n')+ return ''.join(lines).encode('ascii')+++class Node:+ def __init__(self):+ self.data = None+ self.children = {}++ def __eq__(self, other):+ return isinstance(other, Node) and self.data == other.data and self.children == other.children++ def prune(self, depth=0):+ for child in self.children.values():+ child.prune(depth + 1)+ if self.data is None and self.children and depth > 3:+ first = next(iter(self.children.values()))+ if all(child == first for child in self.children.values()):+ self.data = first.data+ if self.data is not None and all(child.data == self.data for child in self.children.values()):+ self.children = {}++ def files(self, ident=()):+ if self.data is not None:+ yield ident, self.data+ for key, child in sorted(self.children.items()):+ yield from child.files(ident + (key,))+++def put(output, name, data):+ if name in output and output[name] != data:+ raise ValueError('Conflicting Apple firmware variants: ' + name)+ output[name] = data+++def convert(files):+ """Pure conversion of already bounded, regular input files; no filesystem I/O."""+ tree, result = Node(), {}+ for path, data in sorted(files.items()):+ parts = path.split('/')+ if len(parts) == 3 and parts[0] == 'wifi' and parts[1] in WIFI_FOLDERS:+ stem, separator, ext = parts[2].rpartition('.')+ if not separator or ext not in EXTENSIONS:+ continue+ if ext != 'txt':+ stem = 'P-' + stem+ properties = {}+ for component in (parts[1] + '_' + stem).split('_'):+ if not component:+ continue+ key, separator, value = component.partition('-')+ if not separator or not value or key not in DIMENSIONS or key in properties:+ raise ValueError('Invalid Apple firmware dimensions: ' + path)+ if key == 'P' and '-' in value:+ value, antenna = value.split('-', 1)+ properties['A'] = antenna+ properties[key] = value+ ident = (ext,) + tuple(properties[key] for key in DIMENSIONS if key in properties)+ if ext == 'txt':+ data = nvram(data)+ node = tree+ for key in ident:+ node = node.children.setdefault(key, Node())+ if node.data is not None and node.data != data:+ raise ValueError('Conflicting Apple firmware dimensions: ' + path)+ node.data = data+ elif len(parts) == 2 and parts[0] == 'bluetooth':+ stem, _, ext = parts[1].rpartition('.')+ if ext not in {'bin', 'ptb'} or '_DEV' in stem:+ continue+ words = stem.split('_')+ chip = re.fullmatch(r'bcm(43[0-9]{2})([a-z][0-9])', words[0].lower())+ if not chip or chip[1] != '4377' or 'PCIE' not in words:+ continue # UART and Apple Silicon radios are not this platform.+ index = words.index('PCIE') + 1+ if index < len(words) and words[index] == 'macOS':+ index += 1+ vendors = {value for key, value in {'MUR': 'm', 'USI': 'u', 'GEN': None}.items() if key in words}+ if index >= len(words) or len(vendors) != 1:+ raise ValueError('Ambiguous Apple Bluetooth firmware: ' + path)+ board = words[index].removesuffix('ES2').lower()+ if not re.fullmatch(r'[a-z0-9]+', board):+ raise ValueError('Invalid Apple Bluetooth board.')+ vendor = vendors.pop()+ name = 'brcmbt' + ''.join(chip.groups()) + '-apple,' + board+ put(result, name + ('-' + vendor if vendor else '') + '.' + ext, data)+ tree.prune()+ for ident, data in tree.files():+ ext, chip, revision, *rest = ident+ name = 'brcmfmac' + chip + revision.lower() + '-pcie.apple'+ put(result, name + (',' + '-'.join(rest) if rest else '') + '.' + EXTENSIONS[ext], data)+ return resultdiff --git a/os/tests/t2_firmware_reference.py b/os/tests/t2_firmware_reference.pynew file mode 100644index 000000000..c2e8c7035--- /dev/null+++ b/os/tests/t2_firmware_reference.py@@ -0,0 +1,75 @@+#!/usr/bin/env python3+"""Compare synthetic firmware conversion against pinned, reviewed upstream code."""+import argparse+import hashlib+import json+import platform+from pathlib import Path+import subprocess+import sys+import tarfile+import tempfile+import time++from test_t2_firmware import firmware, source_fixture+++UPSTREAM_SHA = 'c1c1d8aa25bb5f089e46ccd0d9738fc13bfbd784f499aa924466c690f059961e'+UPSTREAM_COMMIT = '11fc0a8d8cfb61affd0cb9d1ac245c1b6c16d3cd'+++def compare(upstream, output):+ started = time.time()+ source = upstream.read_bytes()+ if hashlib.sha256(source).hexdigest() != UPSTREAM_SHA:+ raise ValueError('The upstream source differs from the reviewed conversion script.')+ marker = b'python3 - "$@" <<\'EOF\'\n'+ if source.count(marker) != 1:+ raise ValueError('Unexpected upstream Python entry point.')+ python = source.split(marker, 1)[1].split(b'\nEOF\n', 1)[0]+ checks = []+ with tempfile.TemporaryDirectory() as temporary:+ root = Path(temporary)+ reference = root / 'reference.py'+ reference.write_bytes(python)+ raw = root / 'source'+ source_fixture(raw)+ # Upstream also converts Apple Silicon Bluetooth. Our T2-only boundary+ # intentionally excludes it; compare only the shared supported inputs.+ (raw / 'bluetooth/BCM4378B1_PCIE_macOS_J314_MUR.bin').unlink()+ for scenario in ('distinct-calibration', 'shared-calibration', 'missing-bluetooth'):+ if scenario == 'shared-calibration':+ for path in (raw / 'wifi').glob('*/P-*.txt'):+ path.write_bytes(b'boardrev =fixture\nvalue=1\n')+ if scenario == 'missing-bluetooth':+ for path in (raw / 'bluetooth').iterdir():+ path.unlink()+ (raw / 'bluetooth').rmdir()+ archive = root / (scenario + '.tar')+ subprocess.run([sys.executable, str(reference), str(raw), str(archive)], check=True,+ timeout=20, stdout=subprocess.PIPE, stderr=subprocess.PIPE)+ # Read the upstream hardlinks as archive data; never extract them.+ with tarfile.open(archive) as handle:+ expected = {entry.name: handle.extractfile(entry).read() for entry in handle}+ actual = firmware.names.convert(firmware.source_files(raw))+ if actual != expected:+ raise ValueError('Linux firmware filenames or bytes differ from upstream: ' + scenario)+ checks.append({'scenario': scenario, 'files': len(actual),+ 'bytes': sum(len(data) for data in actual.values())})+ receipt = {'status': 'passed', 'source': subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip(),+ 'upstream_commit': UPSTREAM_COMMIT, 'upstream_sha256': UPSTREAM_SHA,+ 'platform': platform.platform(), 'python': sys.version,+ 'checks': checks, 'duration_seconds': round(time.time() - started, 3),+ 'limits': ['Invented fixture bytes; no actual Apple firmware is read, downloaded or published.',+ 'No claim of radio loading, macOS IORegistry discovery, or physical T2 support.']}+ output.parent.mkdir(parents=True, exist_ok=True)+ output.write_text(json.dumps(receipt, indent=2) + '\n')+ print(json.dumps(receipt, indent=2))+++if __name__ == '__main__':+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument('--upstream', type=Path, required=True)+ parser.add_argument('--output', type=Path, required=True)+ args = parser.parse_args()+ compare(args.upstream, args.output)diff --git a/os/tests/test_t2_firmware.py b/os/tests/test_t2_firmware.pynew file mode 100644index 000000000..cda0e82d8--- /dev/null+++ b/os/tests/test_t2_firmware.py@@ -0,0 +1,252 @@+import importlib.util+import io+import json+import os+from pathlib import Path+import plistlib+import stat+import subprocess+import sys+import tarfile+import tempfile+import unittest+from unittest.mock import patch+++TOOL = Path(__file__).parents[1] / 'tools/prepare-t2-firmware.py'+spec = importlib.util.spec_from_file_location('harness_t2_firmware', TOOL)+firmware = importlib.util.module_from_spec(spec)+spec.loader.exec_module(firmware)+++def source_fixture(root):+ """Invented sentinel bytes only; never distribute Apple's firmware in tests."""+ files = {}+ for folder in sorted(firmware.names.WIFI_FOLDERS):+ for ext in ('trx', 'clmb', 'txcb'):+ files['wifi/' + folder + '/fiji.' + ext] = (folder + ':' + ext + ':fixture\x00').encode()+ for vendor, value in [('m', b'1'), ('u', b'2')]:+ files['wifi/' + folder + '/P-fiji_M-SPPR_V-' + vendor + '__m-3.1.txt'] = b'boardrev =fixture\nvalue=' + value + b'\n'+ for vendor in ('MUR', 'USI'):+ for extension in ('bin', 'ptb'):+ files['bluetooth/BCM4377B3_PCIE_macOS_HawaiiES2_' + vendor + '.' + extension] = (vendor + ':' + extension + ':fixture').encode()+ # These must never get copied into a T2 bundle.+ files['bluetooth/BCM4378B1_PCIE_macOS_J314_MUR.bin'] = b'Apple Silicon fixture'+ files['bluetooth/BCM4377B3_PCIE_macOS_HawaiiES2_MUR_DEV.bin'] = b'development fixture'+ files['wifi/C-4378__s-B1/j314.trx'] = b'Apple Silicon fixture'+ for name, data in files.items():+ path = root / name+ path.parent.mkdir(parents=True, exist_ok=True)+ path.write_bytes(data)+ return files+++class T2FirmwareTests(unittest.TestCase):+ def setUp(self):+ self.temp = tempfile.TemporaryDirectory()+ self.addCleanup(self.temp.cleanup)+ self.root = Path(self.temp.name)+ self.source = self.root / 'macos-firmware'+ self.inputs = source_fixture(self.source)+ self.bundle = self.root / 'firmware.tar'+ self.model = 'MacBookAir9,1'++ def prepare(self, **kwargs):+ return firmware.prepare(self.source, self.bundle, kwargs.pop('model', self.model), **kwargs)++ def archive(self, entries):+ with tarfile.open(self.bundle, 'w', format=tarfile.USTAR_FORMAT) as output:+ for name, value in entries:+ if isinstance(name, tarfile.TarInfo):+ entry = name+ else:+ entry = tarfile.TarInfo(name)+ entry.mode, entry.size = 0o644, len(value)+ output.addfile(entry, io.BytesIO(value))++ def entries(self):+ with tarfile.open(self.bundle) as archive:+ return [(entry.name, archive.extractfile(entry).read()) for entry in archive]++ def test_mac_export_linux_names_and_private_staging_preserve_exact_data(self):+ before = dict(self.inputs)+ manifest = self.prepare()+ verified, values = firmware.verify(self.bundle, self.model)+ self.assertEqual(verified, manifest)+ self.assertEqual(values['brcmfmac4377b3-pcie.apple,fiji.bin'], b'C-4377__s-B3:trx:fixture\x00')+ self.assertEqual(values['brcmfmac4377b3-pcie.apple,fiji-SPPR-m.txt'], b'boardrev=fixture\nvalue=1\n')+ self.assertEqual(values['brcmbt4377b3-apple,hawaii-m.ptb'], b'MUR:ptb:fixture')+ self.assertFalse(any('4378' in name or '_DEV' in name for name in values))+ output = self.root / 'staged'+ self.assertEqual(firmware.stage(self.bundle, output, self.model), manifest)+ self.assertEqual(stat.S_IMODE(output.stat().st_mode), 0o700)+ self.assertEqual({p.name: p.read_bytes() for p in (output / 'brcm').iterdir()}, values)+ self.assertTrue(all(stat.S_IMODE(p.stat().st_mode) == 0o644 and not p.is_symlink() for p in (output / 'brcm').iterdir()))+ self.assertEqual({name: (self.source / name).read_bytes() for name in before}, before)++ def test_archive_is_deterministic_and_never_overwrites_existing_output(self):+ self.prepare()+ first = self.bundle.read_bytes()+ self.bundle.unlink()+ with patch('time.time', return_value=9876543):+ self.prepare()+ self.assertEqual(self.bundle.read_bytes(), first)+ with self.assertRaises(FileExistsError):+ self.prepare()+ self.assertEqual(self.bundle.read_bytes(), first)+ self.assertEqual(list(self.root.glob('.harness-firmware-*')), [])+ output = self.root / 'existing'+ output.mkdir()+ (output / 'work').write_bytes(b'preserve')+ with self.assertRaises(FileExistsError):+ firmware.stage(self.bundle, output, self.model)+ self.assertEqual((output / 'work').read_bytes(), b'preserve')++ def test_calibration_is_exported_for_the_exact_imac_pro(self):+ nvram = 'wifi/C-4364__s-B2/P-fiji_M-SPPR_V-m__m-3.1.txt'+ txcap = 'wifi/C-4364__s-B2/fiji.txcb'+ requested = plistlib.dumps([{'IORegistryEntryChildren': [{'RequestedFiles': {+ 'NVRAM': '/usr/share/firmware/' + nvram, 'TxCap': '/' + txcap}}]}])+ self.prepare(model='iMacPro1,1', requested_files=requested)+ _, files = firmware.verify(self.bundle, 'iMacPro1,1')+ self.assertEqual(files['brcmfmac4364b2-pcie.txt'], self.inputs[nvram])+ self.assertEqual(files['brcmfmac4364b2-pcie.txcap_blob'], self.inputs[txcap])+ self.bundle.unlink()+ for invalid in (plistlib.dumps([]), plistlib.dumps({'RequestedFiles': '../../private/calibration'})):+ with self.subTest(invalid=invalid), self.assertRaises(ValueError):+ self.prepare(model='iMacPro1,1', requested_files=invalid)+ self.assertFalse(self.bundle.exists())++ def test_missing_wifi_bluetooth_or_wrong_model_fails_before_staging(self):+ self.prepare()+ output = self.root / 'staged'+ for model in ('MacBookPro16,1', 'Mac14,2', '../escape'):+ with self.subTest(model=model), self.assertRaises(ValueError):+ firmware.stage(self.bundle, output, model)+ self.assertFalse(output.exists())+ _, files = firmware.verify(self.bundle, self.model)+ for remove in ('brcmbt4377b3-apple,hawaii-m.ptb', 'brcmfmac4377b3-pcie.apple,fiji.bin'):+ missing = dict(files)+ missing.pop(remove)+ with self.subTest(remove=remove), self.assertRaises(ValueError):+ firmware.validate_files(missing, self.model)++ def test_source_symlink_special_file_and_size_limits_do_not_publish(self):+ path = self.source / 'wifi/C-4377__s-B3/fiji.trx'+ path.unlink()+ path.symlink_to(self.root / 'outside')+ (self.root / 'outside').write_bytes(b'private')+ with self.assertRaises((OSError, ValueError)):+ self.prepare()+ path.unlink()+ os.mkfifo(path)+ with self.assertRaises(ValueError):+ self.prepare()+ path.unlink()+ path.write_bytes(b'oversized')+ with patch.object(firmware, 'MAX_FILE', 2), self.assertRaises(ValueError):+ self.prepare()+ self.assertFalse(self.bundle.exists())+ original = self.source / 'bluetooth'+ original.rename(self.source / 'elsewhere')+ original.symlink_to(self.source / 'elsewhere', target_is_directory=True)+ with self.assertRaisesRegex(ValueError, 'symbolic links'):+ self.prepare()++ def test_memory_and_file_count_budgets_are_enforced(self):+ for name, value in [('MAX_TOTAL', 1), ('MAX_FILES', 1)]:+ with self.subTest(name=name), patch.object(firmware, name, value), self.assertRaises(ValueError):+ self.prepare()+ self.assertFalse(self.bundle.exists())++ def test_failed_staging_removes_only_its_new_directory(self):+ self.prepare()+ (self.root / 'keep').write_bytes(b'existing work')+ output = self.root / 'staged'+ with patch.object(firmware.os, 'fsync', side_effect=OSError('fixture disk full')), self.assertRaises(OSError):+ firmware.stage(self.bundle, output, self.model)+ self.assertFalse(output.exists())+ self.assertEqual((self.root / 'keep').read_bytes(), b'existing work')+ firmware.verify(self.bundle, self.model)++ def test_corrupt_missing_or_extra_contents_cannot_be_materialized(self):+ self.prepare()+ original = self.entries()+ broken = [original[:-1], original + [('brcm/brcmfmac4377b3-pcie.apple,other.bin', b'unlisted')],+ original[:1] + [(original[1][0], b'corruption')] + original[2:]]+ for entries in broken:+ self.archive(entries)+ with self.subTest(entries=len(entries)), self.assertRaises(ValueError):+ firmware.stage(self.bundle, self.root / 'staged', self.model)+ self.assertFalse((self.root / 'staged').exists())++ def test_archive_links_duplicates_traversal_special_and_large_entries_fail(self):+ self.prepare()+ original = self.entries()+ bad_entries = []+ for name in ('../outside', '/absolute', 'brcm/../outside', 'brcm/a/b', 'manifest.json'):+ bad_entries.append((name, b'x'))+ for kind in (tarfile.SYMTYPE, tarfile.LNKTYPE, tarfile.FIFOTYPE, tarfile.CHRTYPE, tarfile.DIRTYPE):+ entry = tarfile.TarInfo('brcm/link')+ entry.mode, entry.type, entry.linkname = 0o644, kind, '/outside'+ bad_entries.append((entry, b''))+ for bad in bad_entries:+ self.archive(original + [bad])+ with self.subTest(entry=bad[0]), self.assertRaises(ValueError):+ firmware.verify(self.bundle, self.model)+ self.archive(original)+ with patch.object(firmware, 'MAX_ARCHIVE', 8), self.assertRaises(ValueError):+ firmware.verify(self.bundle, self.model)+ with patch.object(firmware, 'MAX_FILE', 4), self.assertRaises(ValueError):+ firmware.verify(self.bundle, self.model)++ def test_manifest_requires_unique_exact_fields_and_integrity(self):+ self.prepare()+ original = self.entries()+ manifest = json.loads(original[0][1])+ mutations = [{'model': 'MacBookPro16,1'}, {'schema': True}, {'schema': 2},+ {'files': {}}, {'extra': 'ignored?'}, {'platform': 'x86_64'}]+ for change in mutations:+ altered = dict(manifest, **change)+ self.archive([('manifest.json', json.dumps(altered).encode())] + original[1:])+ with self.subTest(change=change), self.assertRaises(ValueError):+ firmware.verify(self.bundle, self.model)+ duplicate = original[0][1].rstrip().removesuffix(b'}') + b', "schema": 1}'+ self.archive([('manifest.json', duplicate)] + original[1:])+ with self.assertRaisesRegex(ValueError, 'Duplicate'):+ firmware.verify(self.bundle, self.model)++ def test_conflicting_dimensions_and_malformed_nvram_fail(self):+ for raw in (b'broken nvram', b'=empty key\n', b'\xff=non-ascii\n'):+ files = {'wifi/C-4377__s-B3/P-fiji.txt': raw}+ with self.subTest(raw=raw), self.assertRaises(ValueError):+ firmware.names.convert(files)+ with self.assertRaises(ValueError):+ firmware.names.convert({'wifi/C-4377__s-B3/P-fiji_P-other.txt': b'x=1\n'})++ def test_cli_checks_and_stages_on_the_host_without_system_paths(self):+ self.prepare()+ for action in ('verify', 'stage'):+ args = [sys.executable, str(TOOL), action, '--bundle', str(self.bundle), '--model', self.model]+ if action == 'stage':+ args += ['--output', str(self.root / 'cli-staged')]+ result = subprocess.run(args, check=True, capture_output=True, text=True, timeout=15)+ self.assertEqual(json.loads(result.stdout)['status'], 'verified')+ result = subprocess.run([sys.executable, str(TOOL), 'verify', '--bundle', str(self.bundle), '--model', 'Mac14,2'],+ capture_output=True, text=True, timeout=15)+ self.assertNotEqual(result.returncode, 0)+ self.assertNotIn('Traceback', result.stderr)++ def test_export_command_rejects_non_intel_hosts_without_reading_firmware(self):+ for operating_system, architecture in [('Linux', 'x86_64'), ('Darwin', 'arm64')]:+ with patch.object(sys, 'argv', ['prepare-t2-firmware.py', 'export', '--output', str(self.bundle)]), \+ patch.object(firmware.platform, 'system', return_value=operating_system), \+ patch.object(firmware.platform, 'machine', return_value=architecture), \+ patch.object(firmware, 'source_files') as read, self.assertRaises(SystemExit) as error:+ firmware.main()+ self.assertEqual(error.exception.code, 1)+ read.assert_not_called()+++if __name__ == '__main__':+ unittest.main()diff --git a/os/tools/prepare-t2-firmware.py b/os/tools/prepare-t2-firmware.pynew file mode 100644index 000000000..a2765c520--- /dev/null+++ b/os/tools/prepare-t2-firmware.py@@ -0,0 +1,308 @@+#!/usr/bin/env python3+"""Export and verify a Mac's own wireless firmware, without changing its disks."""+import argparse+import hashlib+import importlib.util+import io+import json+import os+from pathlib import Path+import platform+import plistlib+import re+import shutil+import stat+import subprocess+import tarfile+import tempfile+++CONVERTER = Path(__file__).parents[1] / 'platforms/apple-t2/firmware_names.py'+spec = importlib.util.spec_from_file_location('harness_apple_firmware_names', CONVERTER)+names = importlib.util.module_from_spec(spec)+spec.loader.exec_module(names)++MODELS = {'MacBookAir8,1', 'MacBookAir8,2', 'MacBookAir9,1',+ 'MacBookPro15,1', 'MacBookPro15,2', 'MacBookPro15,3', 'MacBookPro15,4',+ 'MacBookPro16,1', 'MacBookPro16,2', 'MacBookPro16,3', 'MacBookPro16,4',+ 'Macmini8,1', 'MacPro7,1', 'iMacPro1,1', 'iMac20,1', 'iMac20,2'}+BT_MODELS = {'MacBookAir9,1', 'MacBookPro15,4', 'MacBookPro16,3'}+MAX_FILE = 16 * 1024**2+MAX_TOTAL = 128 * 1024**2+MAX_FILES = 4096+MAX_MANIFEST = 2 * 1024**2+MAX_ARCHIVE = MAX_TOTAL + MAX_FILES * 1024 + MAX_MANIFEST+SOURCE_NAME = re.compile(r'[A-Za-z0-9_,.+-]{1,180}')+WIFI_NAME = re.compile(r'brcmfmac(4355c1|4364b2|4364b3|4377b3)-pcie\.apple(?:,[A-Za-z0-9,.+-]+)?\.(bin|txt|clm_blob|txcap_blob)')+BT_NAME = re.compile(r'brcmbt4377[a-z][0-9]-apple,[a-z0-9]+(?:-[mu])?\.(bin|ptb)')+IMAC_FILES = {'brcmfmac4364b2-pcie.txt', 'brcmfmac4364b2-pcie.txcap_blob'}+++def identity(data):+ return {'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest()}+++def check_model(model):+ if model not in MODELS:+ raise ValueError('This firmware export requires an Intel Mac with a T2 chip.')+++def regular_bytes(path, limit):+ """Reject links and special files before reading; never wait on a FIFO."""+ fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)+ with os.fdopen(fd, 'rb') as source:+ before = os.fstat(source.fileno())+ if not stat.S_ISREG(before.st_mode) or not 0 < before.st_size <= limit:+ raise ValueError('Invalid firmware file size or type: ' + str(path))+ data = source.read(limit + 1)+ after = os.fstat(source.fileno())+ if len(data) != before.st_size or (before.st_size, before.st_mtime_ns) != (after.st_size, after.st_mtime_ns):+ raise ValueError('Firmware changed while reading: ' + str(path))+ return data+++def source_files(source):+ source = source.resolve(strict=True)+ result, size = {}, 0+ folders = ['wifi/' + folder for folder in sorted(names.WIFI_FOLDERS)] + ['bluetooth']+ for relative in folders:+ folder = source+ missing = False+ for part in relative.split('/'):+ folder = folder / part+ if folder.is_symlink():+ raise ValueError('Firmware directories cannot be symbolic links.')+ if not folder.exists():+ missing = True+ break+ if not folder.is_dir():+ raise ValueError('Invalid firmware directory.')+ if missing:+ continue+ for path in sorted(folder.iterdir()):+ extensions = {'.bin', '.ptb'} if relative == 'bluetooth' else {'.' + ext for ext in names.EXTENSIONS}+ if path.suffix not in extensions:+ continue+ if not SOURCE_NAME.fullmatch(path.name):+ raise ValueError('Invalid Apple firmware filename.')+ data = regular_bytes(path, MAX_FILE)+ size += len(data)+ if size > MAX_TOTAL or len(result) >= MAX_FILES:+ raise ValueError('Apple firmware exceeds the preservation budget.')+ result[relative + '/' + path.name] = data+ return result+++def imac_requested_files(raw):+ """Read iMac Pro calibration filenames from IORegistry data, never shell text."""+ if len(raw) > MAX_FILE:+ raise ValueError('IORegistry output is too large.')+ candidates = set()++ def strings(value):+ if isinstance(value, str):+ yield value+ elif isinstance(value, dict):+ for child in value.values():+ yield from strings(child)+ elif isinstance(value, list):+ for child in value:+ yield from strings(child)++ def visit(value):+ if isinstance(value, dict):+ for key, child in value.items():+ if key == 'RequestedFiles':+ for text in strings(child):+ for match in re.finditer(r'(?:^|/)C-4364__s-B2/([A-Za-z0-9_,.+-]+\.(?:txt|txcb))(?=$|["\s])', text):+ candidates.add('wifi/C-4364__s-B2/' + match[1])+ else:+ visit(child)+ elif isinstance(value, list):+ for child in value:+ visit(child)++ visit(plistlib.loads(raw))+ if len(candidates) != 2 or {Path(name).suffix for name in candidates} != {'.txt', '.txcb'}:+ raise ValueError('Cannot identify this iMac Pro’s Wi-Fi calibration. Keep macOS until it is exported.')+ return candidates+++def validate_files(files, model):+ check_model(model)+ if not files or len(files) > MAX_FILES or sum(len(data) for data in files.values()) > MAX_TOTAL:+ raise ValueError('Invalid preserved firmware inventory.')+ for name, data in files.items():+ if len(name) > 200 or not (WIFI_NAME.fullmatch(name) or BT_NAME.fullmatch(name) or+ model == 'iMacPro1,1' and name in IMAC_FILES):+ raise ValueError('Unexpected Linux firmware filename: ' + name)+ if not 0 < len(data) <= MAX_FILE:+ raise ValueError('Invalid preserved firmware size.')+ # These are completeness checks, not proof of radio operation. Hardware+ # acceptance must still verify brcmfmac loads the particular board variant.+ wifi = [WIFI_NAME.fullmatch(name) for name in files if WIFI_NAME.fullmatch(name)]+ chips = {match[1] for match in wifi}+ if not chips or any(not {'bin', 'txt'} <= {match[2] for match in wifi if match[1] == chip} for chip in chips):+ raise ValueError('Wi-Fi firmware is incomplete; keep macOS until it is exported.')+ if model in BT_MODELS:+ bases = {name.rsplit('.', 1)[0] for name in files if BT_NAME.fullmatch(name)}+ if not bases or any(base + '.bin' not in files or base + '.ptb' not in files for base in bases):+ raise ValueError('This Mac needs Bluetooth firmware from macOS Monterey or later.')+ if '4377b3' not in chips:+ raise ValueError('This Mac’s BCM4377 Wi-Fi firmware is missing.')+ if model == 'iMacPro1,1' and not IMAC_FILES <= files.keys():+ raise ValueError('This iMac Pro’s Wi-Fi calibration is missing.')+++def prepare(source, output, model, requested_files=None):+ check_model(model)+ inputs = source_files(source)+ files = names.convert(inputs)+ if model == 'iMacPro1,1':+ for name in imac_requested_files(requested_files or b''):+ if name not in inputs:+ raise ValueError('Requested iMac Pro calibration is not present in macOS.')+ destination = 'brcmfmac4364b2-pcie.' + ('txt' if name.endswith('.txt') else 'txcap_blob')+ files[destination] = inputs[name]+ validate_files(files, model)+ manifest = {'schema': 1, 'platform': 'apple-t2', 'model': model,+ 'source': 'local-macos-firmware',+ 'files': {name: identity(data) for name, data in sorted(files.items())}}+ payload = {'manifest.json': (json.dumps(manifest, sort_keys=True, indent=2) + '\n').encode()}+ payload.update({'brcm/' + name: data for name, data in sorted(files.items())})+ if len(payload['manifest.json']) > MAX_MANIFEST:+ raise ValueError('Firmware manifest is too large.')+ # Output is a fresh, local archive, never an EFI write or a package install.+ # Linking a completed private file makes publication create-only even if+ # another process creates the requested output while conversion is running.+ fd, temporary = tempfile.mkstemp(prefix='.harness-firmware-', dir=output.parent)+ try:+ with os.fdopen(fd, 'wb') as destination:+ with tarfile.open(fileobj=destination, mode='w', format=tarfile.USTAR_FORMAT) as archive:+ for name, data in payload.items():+ entry = tarfile.TarInfo(name)+ entry.mode, entry.size = 0o644, len(data)+ archive.addfile(entry, io.BytesIO(data))+ destination.flush()+ os.fsync(destination.fileno())+ verify(Path(temporary), model)+ os.link(temporary, output)+ finally:+ os.unlink(temporary)+ return manifest+++def unique_object(pairs):+ result = {}+ for key, value in pairs:+ if key in result:+ raise ValueError('Duplicate firmware manifest key.')+ result[key] = value+ return result+++def verify(bundle, model):+ check_model(model)+ data = regular_bytes(bundle, MAX_ARCHIVE)+ files, manifest = {}, None+ size = 0+ seen = set()+ with tarfile.open(fileobj=io.BytesIO(data), mode='r:') as archive:+ for entry in archive:+ if (entry.name in seen or len(seen) >= MAX_FILES + 1 or not entry.isreg() or+ entry.mode != 0o644 or entry.uid != 0 or entry.gid != 0 or entry.pax_headers or+ entry.linkname or entry.sparse is not None):+ raise ValueError('Invalid preserved firmware archive entry.')+ seen.add(entry.name)+ limit = MAX_MANIFEST if entry.name == 'manifest.json' else MAX_FILE+ if not 0 < entry.size <= limit:+ raise ValueError('Invalid preserved firmware archive size.')+ if entry.name != 'manifest.json' and (not entry.name.startswith('brcm/') or entry.name.count('/') != 1):+ raise ValueError('Unexpected preserved firmware archive path.')+ value = archive.extractfile(entry).read(limit + 1)+ if len(value) != entry.size:+ raise ValueError('Truncated preserved firmware archive.')+ if entry.name == 'manifest.json':+ manifest = json.loads(value, object_pairs_hook=unique_object)+ else:+ size += len(value)+ if size > MAX_TOTAL:+ raise ValueError('Preserved firmware exceeds the memory budget.')+ files[entry.name.removeprefix('brcm/')] = value+ if not isinstance(manifest, dict) or set(manifest) != {'schema', 'platform', 'model', 'source', 'files'}:+ raise ValueError('Missing or invalid firmware manifest.')+ if (type(manifest['schema']) is not int or manifest['schema'] != 1 or+ manifest['platform'] != 'apple-t2' or manifest['model'] != model or+ manifest['source'] != 'local-macos-firmware'):+ raise ValueError('This firmware bundle belongs to another Mac model or format.')+ validate_files(files, model)+ if manifest['files'] != {name: identity(value) for name, value in sorted(files.items())}:+ raise ValueError('Preserved firmware checksum or inventory mismatch.')+ return manifest, files+++def stage(bundle, output, model):+ """Materialize verified data in a new private directory for a future installer.++ This does not copy into a running system, mount a disk, unload a driver, or+ enable T2 installation. The eventual installer must choose a RAM-backed+ staging location before erasure and preserve this data during recovery.+ """+ manifest, files = verify(bundle, model) # Reject everything before any write.+ output.mkdir(mode=0o700, exist_ok=False)+ try:+ (output / 'brcm').mkdir(mode=0o700)+ for name, data in files.items():+ path = output / 'brcm' / name+ with path.open('xb') as destination:+ destination.write(data)+ destination.flush()+ os.fsync(destination.fileno())+ path.chmod(0o644)+ if identity(regular_bytes(path, MAX_FILE)) != manifest['files'][name]:+ raise ValueError('Staged firmware readback differs from its manifest.')+ manifest_path = output / 'manifest.json'+ with manifest_path.open('x') as destination:+ destination.write(json.dumps(manifest, sort_keys=True, indent=2) + '\n')+ destination.flush()+ os.fsync(destination.fileno())+ manifest_path.chmod(0o600)+ except BaseException:+ shutil.rmtree(output)+ raise+ return manifest+++def main():+ parser = argparse.ArgumentParser(description=__doc__)+ commands = parser.add_subparsers(dest='command', required=True)+ export = commands.add_parser('export', help='Read this Intel Mac’s own firmware from macOS.')+ export.add_argument('--output', type=Path, required=True)+ for name in ('verify', 'stage'):+ command = commands.add_parser(name)+ command.add_argument('--bundle', type=Path, required=True)+ command.add_argument('--model', required=True)+ if name == 'stage':+ command.add_argument('--output', type=Path, required=True)+ args = parser.parse_args()+ try:+ if args.command == 'export':+ if platform.system() != 'Darwin' or platform.machine() != 'x86_64':+ raise ValueError('Export firmware while running macOS on the Intel Mac that will receive Harness.')+ model = subprocess.check_output(['sysctl', '-n', 'hw.model'], timeout=10, text=True).strip()+ check_model(model)+ requested = subprocess.check_output(['ioreg', '-a', '-l'], timeout=20) if model == 'iMacPro1,1' else None+ manifest = prepare(Path('/usr/share/firmware'), args.output.absolute(), model, requested)+ elif args.command == 'stage':+ manifest = stage(args.bundle, args.output, args.model)+ else:+ manifest, _ = verify(args.bundle, args.model)+ print(json.dumps({'model': manifest['model'], 'files': len(manifest['files']),+ 'bytes': sum(entry['bytes'] for entry in manifest['files'].values()),+ 'status': 'verified'}, sort_keys=True))+ except (OSError, ValueError, tarfile.TarError, plistlib.InvalidFileException, subprocess.SubprocessError) as error:+ parser.exit(1, str(error) + '\n')+++if __name__ == '__main__':+ main()