Play your own Switch games on iPhone/iPad — one-click macOS app, no terminal. | 把 Switch 游戏装进 iPhone:图形界面一键构建,不用命令行。Bring your own game.
Swift apple-siliconcustom-formathomebrewiosipaipadipadosiphonemetalnative-portnintendo-switchnxnativeromfsswitch-games
Release carries the engine tree again, signed latest.json, experimental channel
v0lumehi1a487bdeeb15
diff --git a/.github/workflows/build-mac-app.yml b/.github/workflows/build-mac-app.ymlindex 610350c3..4d833422 100644--- a/.github/workflows/build-mac-app.yml+++ b/.github/workflows/build-mac-app.yml@@ -8,7 +8,12 @@# "up to date" — a silent failure of the update path for every existing install. Copies from before# the app was renamed (NS Port Builder) look for NS-Port-Builder-macOS-arm64.zip, so that one goes up too.#-# The app carries no engine (issue #214): the launcher runs `tools/engine.py sync` on the first start and after updates, from GitHub. It carries its build tools (zstd, glslangValidator, spirv-cross, xcodegen) and a Python with the packages the build scripts import,+# The app carries the engine (issues #214 and #227): `tools/engine.py tree` builds the engine tree (seedds/nxnative at the pin plus the patch+# series, no git history) into Contents/Resources/engine, and the launcher copies it to Application Support on first use. The same tree goes up+# as a release asset (NX2Apple-engine.zip) together with latest.json and its Ed25519 signature latest.json.sig (tools/release_manifest.py), which+# is what the launcher reads to update the engine without an app release. The signing key is the repository secret ENGINE_SIGNING_KEY: a tag build+# fails at its first step when the secret is missing or is not the key in frontend/engine-signing.pub (compiled into the launcher). Other builds+# run the same steps as a dry run with a throwaway key and upload nothing. It also carries its build tools (zstd, glslangValidator, spirv-cross, xcodegen) and a Python with the packages the build scripts import,# so a player installs neither Homebrew nor Python (frontend/tools/bundle_tools.sh builds them from pinned, checksummed sources; check_bundle.sh# runs each from the assembled bundle with a clean PATH, checks the load paths and the signature).#@@ -21,7 +26,7 @@ on:push:tags: ['v*']pull_request:- paths: ['frontend/**', 'tools/engine.py', '.github/workflows/build-mac-app.yml']+ paths: ['frontend/**', 'tools/engine.py', 'tools/release_manifest.py', 'tools/requirements-release.txt', 'patches/**', '.upstream-ref', '.github/workflows/build-mac-app.yml']workflow_dispatch:permissions:@@ -33,6 +38,30 @@ jobs:steps:- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0+ with:+ python-version: '3.14'+ - name: Install the signing tool's package+ run: python -m pip install -r tools/requirements-release.txt++ - name: Signing key (a release without a matching key must not ship)+ if: startsWith(github.ref, 'refs/tags/')+ env:+ ENGINE_SIGNING_KEY: ${{ secrets.ENGINE_SIGNING_KEY }}+ run: |+ # Fails when the secret is missing, or when its public key is not the one in frontend/engine-signing.pub, the value the launcher+ # compiles in (EngineManifest.verify): a manifest signed with any other key would be rejected by every installed app.+ python tools/release_manifest.py check-key --pub frontend/engine-signing.pub++ - name: Engine tree (tools/engine.py tree)+ shell: bash+ run: |+ set -euo pipefail+ python tools/engine.py sync+ # The tree as it goes into the app, and its zip, byte for byte the same on every run (the SHA-256 in latest.json).+ python tools/engine.py tree --out "$RUNNER_TEMP/engine-tree" --zip "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip"+ du -sh "$RUNNER_TEMP/engine-tree" "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip"+- name: Version from the tagid: vershell: bash@@ -73,6 +102,13 @@ jobs:# The build tools and the Python go into Resources before the bundle is signed (bundle_tools.sh signs each nested Mach-O file# itself: --deep below does not reach code that sits in Resources).tools/bundle_tools.sh "$APP/Contents/Resources" "$RUNNER_TEMP/bundle-tools-work"+ # The engine tree (Contents/ is read-only once signed: the launcher copies it to ~/Library/Application Support/NX2Apple/ on first+ # use). Its Mach-O files (the shader translator, spirv-cross) are signed on their own like the bundled tools: --deep does not reach+ # code in Resources. The zip asset keeps them as upstream built them.+ ditto "$RUNNER_TEMP/engine-tree" "$APP/Contents/Resources/engine"+ while IFS= read -r -d '' f; do+ if file -b "$f" | grep -q 'Mach-O'; then codesign --force --sign - "$f" 2> /dev/null; fi+ done < <(find "$APP/Contents/Resources/engine" -type f -perm -u+x -print0)V="${{ steps.ver.outputs.version }}"cat > "$APP/Contents/Info.plist" <<PLIST<?xml version="1.0" encoding="UTF-8"?>@@ -108,6 +144,13 @@ jobs:# lz4, capstone, fontTools, zbic (system 22.0.0+ modules) and nsz, so a release without one of them fails here; no Mach-O file links into /opt/homebrew; the signature verifies.tools/check_bundle.sh NX2Apple.appcodesign -v --deep --strict NX2Apple.app+ # The engine tree inside the bundle is the one the zip asset holds: same identity, the files the launcher runs by path.+ E="NX2Apple.app/Contents/Resources/engine"+ cmp "$E/.engine-id" "$RUNNER_TEMP/engine-tree/.engine-id"+ cmp "$E/.upstream-ref" ../.upstream-ref+ test -f "$E/scripts/build_ipa_from_nsp.py" && test -f "$E/dm/tools/romfs_image.py" && test -f "$E/dm/fork_patches.txt"+ test ! -e "$E/.git"+ echo "engine in the bundle: $(cat "$E/.engine-id")"- name: Packageworking-directory: frontend@@ -153,3 +196,35 @@ jobs:frontend/NX2Apple-macOS-arm64.zip \frontend/nxnative-macOS-arm64.zip \frontend/NS-Port-Builder-macOS-arm64.zip --clobber++ - name: Publish the engine (NX2Apple-engine.zip, latest.json, latest.json.sig)+ if: startsWith(github.ref, 'refs/tags/')+ shell: bash+ env:+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}+ ENGINE_SIGNING_KEY: ${{ secrets.ENGINE_SIGNING_KEY }}+ run: |+ # latest.json: version, pin, engine_id, the engine and app assets with SHA-256 and size, sources (this release's download URL),+ # built, channel stable. latest.json.sig: the Ed25519 signature over its exact bytes. The zip goes first, the signature last.+ python tools/release_manifest.py publish \+ --engine-zip "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip" --app-zip frontend/NX2Apple-macOS-arm64.zip \+ --version "${{ steps.ver.outputs.version }}" --channel stable --tag "$GITHUB_REF_NAME" \+ --out "$RUNNER_TEMP/publish" --upload++ - name: Engine publishing dry run (a throwaway key, nothing uploaded)+ if: ${{ !startsWith(github.ref, 'refs/tags/') }}+ shell: bash+ run: |+ python tools/release_manifest.py publish \+ --engine-zip "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip" --app-zip frontend/NX2Apple-macOS-arm64.zip \+ --version "${{ steps.ver.outputs.version }}" --channel stable --tag "v${{ steps.ver.outputs.version }}" \+ --out "$RUNNER_TEMP/publish" --ephemeral-key+ cat "$RUNNER_TEMP/publish/latest.json"+ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2+ if: ${{ !startsWith(github.ref, 'refs/tags/') }}+ with:+ name: engine-latest-json-dry-run+ path: |+ ${{ runner.temp }}/publish/latest.json+ ${{ runner.temp }}/publish/latest.json.sig+ if-no-files-found: errordiff --git a/.github/workflows/sync-upstream.yml b/.github/workflows/sync-upstream.ymlindex f73a9ce3..09c1f6d3 100644--- a/.github/workflows/sync-upstream.yml+++ b/.github/workflows/sync-upstream.yml@@ -10,12 +10,24 @@# A PR opened with the default GITHUB_TOKEN does not start other workflows (GitHub's rule), so this job# runs the checks itself before it opens the PR. A repository secret SYNC_TOKEN (a fine-grained token# with contents, pull requests and issues write) makes the PR start tests.yml as well.+#+# The experimental engine channel (LNPANDY/NX2Apple#227): after the sync job passed its checks (the series applies, the tests, the host build)+# the publish-experimental job builds the engine tree of the bumped branch (tools/engine.py tree), writes latest.json with channel+# "experimental" and "built", signs it with the repository secret ENGINE_SIGNING_KEY (tools/release_manifest.py) and replaces the assets+# (NX2Apple-engine.zip, latest.json, latest.json.sig) of the pre-release tag engine-experimental, which it creates when missing and keeps a+# pre-release that is never "latest". A workflow artifact cannot serve: the launcher has no token. It only publishes after a green sync job,+# and its failure (a missing or wrong key) never opens the "sync blocked" issue. `force_experimental` publishes main's engine by hand.name: Sync upstreamon:schedule:- cron: '30 21 * * *' # 21:30 UTC = 5:30 Beijingworkflow_dispatch:+ inputs:+ force_experimental:+ description: "Also publish the engine of main to the engine-experimental tag (the first time, or after the tag was removed)"+ type: boolean+ default: falsepermissions:contents: write@@ -25,6 +37,8 @@ permissions:jobs:sync:runs-on: macos-latest # the host build needs clang + macOS headers+ outputs:+ branch: ${{ steps.pr.outputs.branch }} # the pushed sync branch, empty when nothing changedenv:GH_TOKEN: ${{ secrets.SYNC_TOKEN || secrets.GITHUB_TOKEN }}steps:@@ -64,6 +78,7 @@ jobs:make -C engine/dm/host GAME=$GAME dmhost -j4 2>&1 | tail -20- name: Commit to a sync branch and open a pull request+ id: prif: steps.bump.outputs.changed == 'yes'# main is protected (a pull request is required, force pushes and deletions are# blocked, and the rules apply to everyone including the owner and this bot), so the@@ -79,6 +94,7 @@ jobs:git checkout -b "sync/upstream-$SHORT"git commit -m "Sync upstream $SHORT"git push -u origin "sync/upstream-$SHORT"+ echo "branch=sync/upstream-$SHORT" >> "$GITHUB_OUTPUT"{echo "The patch queue re-applied on upstream \`${NEW:0:12}\` (was \`${OLD:0:12}\`): every patch applied, and \`tools/engine.py check --tree\`, the tests and a macOS host build passed. Merging brings seedds/nxnative to \`$NEW\`."echo@@ -113,3 +129,50 @@ jobs:elsegh issue create -R "${{ github.repository }}" --title "$TITLE" --body-file issue.md || echo "issue creation failed; the log is in the workflow run"fi++ publish-experimental:+ # Only after a green sync job (its checks are the "CI is green" of this channel). Separate from it so that a publishing problem+ # (no key, no release permission) does not read as a blocked sync above.+ needs: sync+ if: needs.sync.result == 'success' && (needs.sync.outputs.branch != '' || inputs.force_experimental)+ runs-on: ubuntu-latest+ timeout-minutes: 20+ concurrency: engine-experimental # two runs must not replace the assets at the same time+ permissions:+ contents: write+ actions: read+ env:+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}+ ENGINE_SIGNING_KEY: ${{ secrets.ENGINE_SIGNING_KEY }}+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ with:+ ref: ${{ needs.sync.outputs.branch || github.event.repository.default_branch }}+ persist-credentials: false+ - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0+ with:+ python-version: '3.14'+ - run: python -m pip install -r tools/requirements-release.txt++ - name: Signing key (fails when the secret is missing or is not the key in frontend/engine-signing.pub)+ run: python tools/release_manifest.py check-key --pub frontend/engine-signing.pub++ - name: "Main's tests must be green (a bump branch was just checked, a hand-started publish of main was not)"+ if: needs.sync.outputs.branch == ''+ run: |+ c=$(gh run list -R "${{ github.repository }}" --workflow tests.yml --branch "${{ github.event.repository.default_branch }}" --event push --limit 1 --json conclusion --jq '.[0].conclusion')+ echo "latest tests.yml on main: ${c:-none}"+ [ "$c" = "success" ] || { echo "::error::tests.yml on main is not green: not publishing"; exit 1; }++ - name: Build the engine tree+ run: |+ python tools/engine.py sync+ python tools/engine.py tree --out "$RUNNER_TEMP/engine-tree" --zip "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip"++ - name: Publish to the engine-experimental pre-release+ run: |+ APPV=$(sed -n 's/.*static let appVersion = "\(.*\)".*/\1/p' frontend/Sources/NSPortBuilder/ContentView.swift)+ python tools/release_manifest.py publish \+ --engine-zip "$RUNNER_TEMP/engine-out/NX2Apple-engine.zip" \+ --version "$APPV-experimental" --channel experimental --tag engine-experimental \+ --out "$RUNNER_TEMP/publish" --uploaddiff --git a/.github/workflows/tests.yml b/.github/workflows/tests.ymlindex c08f3f18..b7ba7f7d 100644--- a/.github/workflows/tests.yml+++ b/.github/workflows/tests.yml@@ -160,8 +160,10 @@ jobs:}- name: Install ripgrep (the engine/ search test)run: sudo apt-get install -y -qq ripgrep > /dev/null- - name: Patch tool tests- run: python -m unittest -v tools/tests/test_patches.py tools/tests/test_engine.py+ - name: Install the signing tool's package (tools/release_manifest.py)+ run: python -m pip install -r tools/requirements-release.txt+ - name: Patch tool and release tool tests+ run: python -m unittest -v tools/tests/test_patches.py tools/tests/test_engine.py tools/tests/test_release_manifest.py- name: Dry bump to seedds main (report only)run: python tools/engine.py bump --dry --to main || truediff --git a/frontend-qt/nspb_qt/engine.py b/frontend-qt/nspb_qt/engine.pyindex d3ddeab6..0d530534 100644--- a/frontend-qt/nspb_qt/engine.py+++ b/frontend-qt/nspb_qt/engine.py@@ -2,8 +2,8 @@Since #214 the engine is built into `<checkout>/engine/` (seedds/nxnative at `.upstream-ref` plus `patches/`). Until thecommitted copy goes (step 2), a checkout has both, and one from before the switch has only the committed copy. The rule, the-same as the Mac launcher's (frontend/Sources/NSPortBuilder/Engine.swift). A release carries no engine: the sync downloads it from-GitHub on the first start and after updates, and a failed one becomes a failure row (`failure_kind`):+same as the Mac launcher's (frontend/Sources/NSPortBuilder/Engine.swift). The sync is for checkouts (a release carries the engine tree+instead, #227): it downloads the pinned commit from GitHub, and a failed one becomes a failure row (`failure_kind`):- engine/ when it is synced and current,- else the committed tree (dm/, scripts/ at the checkout's root) when the checkout has one,diff --git a/frontend/Sources/NSPortBuilder/BuildFailureView.swift b/frontend/Sources/NSPortBuilder/BuildFailureView.swiftindex 4ca1e3c0..c228e1d5 100644--- a/frontend/Sources/NSPortBuilder/BuildFailureView.swift+++ b/frontend/Sources/NSPortBuilder/BuildFailureView.swift@@ -82,7 +82,7 @@ struct BuildFailureCard: View {}}-/// Above every page of the launcher while the engine is fetched and patched (tools/engine.py sync, run on the first start and after+/// Above every page of the launcher while the engine is fetched and patched (tools/engine.py sync, run for a checkout at launch and after/// updates), and its failure row when that stops: what happened, what to do, and Try again.struct EngineSyncBanner: View {@ObservedObject var m: BuildModeldiff --git a/frontend/Sources/NSPortBuilder/ContentView.swift b/frontend/Sources/NSPortBuilder/ContentView.swiftindex c804cb77..89daa93f 100644--- a/frontend/Sources/NSPortBuilder/ContentView.swift+++ b/frontend/Sources/NSPortBuilder/ContentView.swift@@ -629,7 +629,7 @@ final class BuildModel: ObservableObject {@Published var engineFailure: BuildFailure?/// `tools/engine.py sync` when engine/ is stale (#214, Engine.swift): at launch, after a clone, a pull or an adoption, and before a- /// build that has no other engine. A release app carries no engine, so the first start downloads it here. A no-op for a checkout+ /// build that has no other engine. Checkouts only: a release app carries its engine tree (#227) and never syncs. A no-op for a checkout/// from before #214 or a current engine/. `then(ok)` runs on the main queue.func syncEngine(then: ((Bool) -> Void)? = nil) {if engineSyncWaiting != nil { engineSyncAgain = true; if let then { engineSyncWaiting?.append(then) }; return }diff --git a/frontend/Sources/NSPortBuilder/Engine.swift b/frontend/Sources/NSPortBuilder/Engine.swiftindex bbeae806..f9914505 100644--- a/frontend/Sources/NSPortBuilder/Engine.swift+++ b/frontend/Sources/NSPortBuilder/Engine.swift@@ -12,8 +12,10 @@ import Foundation/// - else `engine/` as it is (stale or missing: the build syncs it first).////// `stampKey` is tools/engine.py's digest of the pin and the series, compared with the stamp sync wrote into engine/.git, so the-/// launcher tells a stale engine without starting Python. A release app carries no engine: the launcher runs the sync on the first-/// start and after updates, which downloads it from GitHub (about 80 MB), and shows a failure row when that does not work.+/// launcher tells a stale engine without starting Python. The sync is for a repository checkout (a developer's,+/// or one a launcher cloned): it downloads the pinned engine from GitHub (about 80 MB) and shows a failure row when that does not work.+/// A release app carries the engine tree instead (`Contents/Resources/engine`, built by `tools/engine.py tree`, #227) and takes updates+/// from the release's `latest.json`: no git, no sync.enum EngineLayout {/// The file that tells an engine tree (BuildModel.isRepo always looked for it).static let marker = "dm/tools/nx_extract/Program.cs"diff --git a/frontend/engine-signing.pub b/frontend/engine-signing.pubnew file mode 100644index 00000000..72b5ab9b--- /dev/null+++ b/frontend/engine-signing.pub@@ -0,0 +1,3 @@+# The public half of the key that signs latest.json (Ed25519, base64 of the 32 raw bytes), one line below. Empty until the key exists.+# The launcher (EngineManifest.verify's publicKey) compiles in the same value, and the release workflow fails when the repository+# secret ENGINE_SIGNING_KEY does not belong to it (tools/release_manifest.py check-key). Lines starting with # are comments.diff --git a/patches/README.md b/patches/README.mdindex 3462880f..69803617 100644--- a/patches/README.md+++ b/patches/README.md@@ -107,6 +107,23 @@ leaves the conflict in a worktree.The nightly workflow does the first step only: it opens a pull request on `sync/upstream` when every patch applies, and an issuenaming the blocking patch when one does not. Whoever owns the patch refreshes it.+## The engine tree and how it is published (#227)++Players get no git: a release carries the engine as a plain tree. `python3 tools/engine.py tree --out DIR [--zip FILE]` writes it from+a synced `engine/` (`dm/`, `scripts/`, `docs/`, `ref/`, `AGENTS.md`, `LICENSE`, the patches applied, `dm/fork_patches.txt`, no history) plus+`.upstream-ref` and `.engine-id`, the tree's identity `<pin>-<first 12 hex of the series digest>` (the series changes without the pin moving, so+the pin alone does not name a tree). The zip is deterministic (sorted, fixed timestamps), so its SHA-256 is reproducible.++`python3 tools/release_manifest.py publish ...` writes `latest.json` (version, pin, `engine_id`, channel, built, the engine and app assets with+SHA-256 and size, sources) and `latest.json.sig` (a detached Ed25519 signature over its exact bytes, base64), and with `--upload` puts+`NX2Apple-engine.zip`, `latest.json` and `latest.json.sig` on a GitHub release. Without `--upload` it is a dry run. The key is the repository+secret `ENGINE_SIGNING_KEY` (base64 of the 32 raw private bytes), checked against `frontend/engine-signing.pub`, which the launcher compiles in+(`python3 tools/release_manifest.py keygen` makes a pair). `build-mac-app.yml` does this for a release tag (stable channel, the tree also goes+into `Contents/Resources/engine`), `sync-upstream.yml` for the pre-release tag `engine-experimental` after a green nightly bump.++Locally: `python3 tools/engine.py sync`, then `tree --out /tmp/t --zip /tmp/e/NX2Apple-engine.zip`, then `publish --engine-zip /tmp/e/NX2Apple-engine.zip+--version 0.0.0 --channel experimental --tag engine-experimental --out /tmp/p --ephemeral-key` runs every step except the upload.+## Patch policyUpstream's compatibility is the baseline. A patch does not change it silently.diff --git a/tools/engine.py b/tools/engine.pyindex 71313782..bc5cd726 100755--- a/tools/engine.py+++ b/tools/engine.py@@ -16,6 +16,7 @@ Subcommands (Python 3.9+, stdlib and git):export [REV] turn commits made in engine/ (default: everything after the last sync) into patches with theUpstream-Status / Reason / Issue header, and add them to patches/seriesstatus the patches (Upstream-Status, Reason, age) and the state of engine/+ tree --out DIR [--zip FILE] the engine tree for players, the release's engine (see below)check [--tree] headers valid, no patch touches the shader chain or the generated files, the series appliescleanly to the pin with no empty patch. --tree also compares the committed vendored tree withpin + series, as long as the repository still commits it (step 1 of #214)@@ -24,8 +25,14 @@ Subcommands (Python 3.9+, stdlib and git):refresh WORKTREE --to SHA after a conflicting bump was resolved by hand in WORKTREE: its commits become the seriesThe upstream URL is https://github.com/seedds/nxnative.git, or $NX2APPLE_ENGINE_URL (a mirror, or a local clone that holds the-pin, which needs no network). A release app carries no engine: the launcher runs `sync` on the first start and after updates.+pin, which needs no network). `sync` is for developers, CI and checkouts: a release carries the engine tree (`tree`) instead.Scratch worktrees go to $PATCHES_TMP (or --tmp), else the system's temporary folder.++The engine tree (`tree`): what a player's launcher runs the build from, with no git. DIR gets the files of engine/ at its last sync+(TREE_PATHS, the patches applied, dm/fork_patches.txt included), `.upstream-ref` and `.engine-id`, the engine's identity+`<pin>-<first 12 hex of the series digest>`: the pin alone does not name a tree, since the series changes without the pin moving.+The zip lists its entries sorted, with fixed timestamps and permissions (0755 or 0644), so the same tree gives the same bytes+(for one zlib). tools/release_manifest.py publishes it."""import argparseimport contextlib@@ -34,6 +41,7 @@ import importlibimport ioimport osimport re+import shutilimport subprocessimport sysimport time@@ -236,6 +244,97 @@ def cmd_hook(a):return 0+# ---- tree: the engine for players ----++TREE_PATHS = ('dm', 'scripts', 'docs', 'ref', 'AGENTS.md', 'LICENSE') # what a player's build reads, of what engine/ tracks+ENGINE_ID_FILE = '.engine-id'+ZIP_TIME = (2020, 1, 1, 0, 0, 0) # fixed, so the zip's bytes depend on the content only+++def engine_id(root):+ """`<pin>-<first 12 hex of the series digest>`: names a tree (the series changes without the pin moving). [0-9a-f-], 53 characters."""+ return f'{pin(root)}-{stamp_key(root)[:12]}'+++def extract_tree(eng, out):+ """The files of engine/'s HEAD under TREE_PATHS into `out`: `git archive`, so no .git and nothing untracked or ignored."""+ import tarfile+ names = [n for n in TREE_PATHS if git('cat-file', '-e', f'HEAD:{n}', cwd=eng, check=False).returncode == 0]+ if not names: raise Fail(f'{eng} holds none of {", ".join(TREE_PATHS)}: not an engine checkout')+ proc = subprocess.Popen(['git', 'archive', '--format=tar', 'HEAD', '--', *names], cwd=eng, stdout=subprocess.PIPE, stderr=subprocess.PIPE)+ count = 0+ with tarfile.open(fileobj=proc.stdout, mode='r|') as tar:+ for m in tar:+ dest = out / m.name+ if m.isdir(): dest.mkdir(parents=True, exist_ok=True)+ elif m.isreg():+ dest.parent.mkdir(parents=True, exist_ok=True)+ with tar.extractfile(m) as src, open(dest, 'wb') as f: shutil.copyfileobj(src, f)+ dest.chmod(0o755 if m.mode & 0o111 else 0o644)+ count += 1+ elif m.issym():+ dest.parent.mkdir(parents=True, exist_ok=True)+ os.symlink(m.linkname, dest)+ count += 1+ proc.stdout.close()+ if proc.wait(): raise Fail(f'git archive failed: {proc.stderr.read().decode(errors="replace").strip()[-500:]}')+ return count+++def write_zip(tree, zpath):+ """Every file of `tree` (the tree itself is the archive's root), sorted by path, fixed time, 0755 or 0644, deflate level 6."""+ import stat+ import zipfile+ entries = []+ for dirpath, dirnames, files in os.walk(tree):+ dirnames.sort()+ for n in files + [d for d in dirnames if os.path.islink(os.path.join(dirpath, d))]:+ full = os.path.join(dirpath, n)+ entries.append((Path(full).relative_to(tree).as_posix(), full))+ entries.sort()+ zpath = Path(zpath)+ zpath.parent.mkdir(parents=True, exist_ok=True)+ tmp = zpath.with_name(zpath.name + '.part')+ with zipfile.ZipFile(tmp, 'w', zipfile.ZIP_DEFLATED, compresslevel=6) as z:+ for rel, full in entries:+ zi = zipfile.ZipInfo(rel, ZIP_TIME)+ zi.create_system = 3+ if os.path.islink(full):+ zi.external_attr = (stat.S_IFLNK | 0o755) << 16+ z.writestr(zi, os.readlink(full))+ continue+ zi.compress_type = zipfile.ZIP_DEFLATED+ zi.external_attr = (stat.S_IFREG | (0o755 if os.access(full, os.X_OK) else 0o644)) << 16+ with open(full, 'rb') as f: z.writestr(zi, f.read(), compresslevel=6)+ os.replace(tmp, zpath)+ return len(entries)+++def cmd_tree(a):+ root = a.root+ eng = engine_dir(root)+ reason = why_stale(root)+ if reason: raise Fail(f'engine/ is not a current sync ({reason}): run `python3 tools/engine.py sync` first')+ work = local_work(eng, read_stamp(eng))+ if work: raise Fail('engine/ has changes the release must not carry: ' + '; '.join(work) + '. Export them as patches or sync --force.')+ out = Path(a.out).resolve()+ if out.exists() and any(out.iterdir()):+ if not (out / ENGINE_ID_FILE).is_file(): raise Fail(f'{out} exists and is not an engine tree made by this command: choose an empty folder')+ shutil.rmtree(out)+ out.mkdir(parents=True, exist_ok=True)+ n = extract_tree(eng, out)+ eid = engine_id(root)+ (out / '.upstream-ref').write_text(pin(root) + '\n')+ (out / ENGINE_ID_FILE).write_text(eid + '\n')+ if not (out / P.FORK_PATCHES).is_file(): raise Fail(f'{P.FORK_PATCHES} is missing from engine/: sync again')+ say(f'engine tree: {eid} ({n} files) in {out}')+ if a.zip:+ k = write_zip(out, a.zip)+ h = hashlib.sha256(Path(a.zip).read_bytes()).hexdigest()+ say(f'engine zip: {a.zip} ({k} entries, {Path(a.zip).stat().st_size} bytes, sha256 {h})')+ return 0++# ---- export ----def cmd_export(a):@@ -423,6 +522,7 @@ def main(argv=None):s = sub.add_parser('hook'); s.set_defaults(f=cmd_hook, force=False, quiet=True)s = sub.add_parser('export'); s.add_argument('rev', nargs='?', help='a commit or a range in engine/ (default: everything after the last sync)')s.add_argument('--status', default='pending'); s.add_argument('--reason', default='TODO: why the fork carries this'); s.add_argument('--issue', default='TODO: LNPANDY/NX2Apple#<n>'); s.set_defaults(f=cmd_export)+ s = sub.add_parser('tree'); s.add_argument('--out', required=True, help='the folder for the tree (emptied when it is an earlier tree)'); s.add_argument('--zip', help='also write the deterministic zip of the tree'); s.set_defaults(f=cmd_tree)sub.add_parser('status').set_defaults(f=cmd_status)s = sub.add_parser('check'); s.add_argument('--tree', action='store_true', help='also compare the committed vendored tree (step 1 of #214)'); s.set_defaults(f=cmd_check)s = sub.add_parser('bump'); s.add_argument('--to', required=True, help='the new upstream commit, or a branch such as main'); s.add_argument('--dry', action='store_true'); s.set_defaults(f=cmd_bump)diff --git a/tools/release_manifest.py b/tools/release_manifest.pynew file mode 100755index 00000000..9904b784--- /dev/null+++ b/tools/release_manifest.py@@ -0,0 +1,256 @@+#!/usr/bin/env python3+"""latest.json for the engine, signed (LNPANDY/NX2Apple#227): what the launcher reads to update the engine without an app release.++The manifest has the shape of EngineManifest.Manifest (frontend/Sources/NSPortBuilder/EngineManifest.swift), plus `engine_id`:++ {"version": "0.8.0", "pin": "<seedds commit>", "engine_id": "<pin>-<12 hex of the series digest>", "channel": "stable",+ "built": "2026-10-31T02:00:00Z",+ "engine": {"asset": "NX2Apple-engine.zip", "sha256": "<64 hex>", "size": 91553652},+ "app": {"asset": "NX2Apple-macOS-arm64.zip", "sha256": "...", "size": 1234}, (stable only, optional)+ "sources": ["https://github.com/LNPANDY/NX2Apple/releases/download/v0.8.0/"]}++`pin` and `engine_id` come from the engine zip itself (its .upstream-ref and .engine-id, written by `tools/engine.py tree`), so the+manifest cannot disagree with the asset it describes. The signature is a detached Ed25519 signature over the exact bytes of+latest.json, base64, in latest.json.sig next to it (EngineManifest.verify(manifest:signature:publicKey:)).++The private key is the repository secret ENGINE_SIGNING_KEY: base64 of the 32 raw bytes. Its public key must equal the one in+frontend/engine-signing.pub, which the launcher compiles in. `check-key` fails when the secret is missing or belongs to another key,+so a release can never ship with signing off or with a key the launcher does not trust.++Subcommands (Python 3.9+, `pip install -r tools/requirements-release.txt`):+ keygen print a new key pair (private base64 for the secret, public base64 for engine-signing.pub)+ check-key [--pub FILE] exit 0 when $ENGINE_SIGNING_KEY is set and matches the public key file+ publish --engine-zip Z --version V --channel stable|experimental --tag T --out DIR [--app-zip A] [--upload]+ write latest.json and latest.json.sig (and copies of the zips) to DIR and verify them. With --upload,+ put the zip, latest.json and latest.json.sig on the GitHub release TAG (gh). Without it nothing is+ uploaded: that is the dry run. --ephemeral-key signs with a throwaway key (dry runs only, never uploads)+ verify --manifest F --sig F [--pub FILE] check a signature the way the launcher does+"""+import argparse+import base64+import datetime+import hashlib+import json+import os+import re+import shutil+import subprocess+import sys+import zipfile+from pathlib import Path++REPO = Path(__file__).resolve().parent.parent+PUB_FILE = REPO / 'frontend' / 'engine-signing.pub'+KEY_ENV = 'ENGINE_SIGNING_KEY'+MANIFEST = 'latest.json'+SIGNATURE = 'latest.json.sig'+GITHUB_REPO = 'LNPANDY/NX2Apple'+EXPERIMENTAL_TAG = 'engine-experimental'+ENGINE_ID = re.compile(r'[0-9a-f]{40}-[0-9a-f]{12}')+++class Fail(Exception):+ pass+++def say(*a):+ print(*a, flush=True)+++def ed25519():+ try:+ from cryptography.hazmat.primitives.asymmetric import ed25519+ from cryptography.exceptions import InvalidSignature+ except ImportError:+ raise Fail('the Python package `cryptography` is missing: python3 -m pip install -r tools/requirements-release.txt')+ return ed25519, InvalidSignature+++def raw32(b64, what):+ try: data = base64.b64decode(b64.strip(), validate=True)+ except Exception: raise Fail(f'{what} is not base64')+ if len(data) != 32: raise Fail(f'{what} must be the base64 of 32 raw bytes (is {len(data)} bytes)')+ return data+++def private_key(b64):+ return ed25519()[0].Ed25519PrivateKey.from_private_bytes(raw32(b64, KEY_ENV))+++def public_b64(priv):+ from cryptography.hazmat.primitives import serialization+ raw = priv.public_key().public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw)+ return base64.b64encode(raw).decode()+++def read_pub(path):+ """The first line of the file that is not blank and not a comment."""+ path = Path(path)+ if not path.is_file(): raise Fail(f'{path} is missing: it holds the public key the launcher compiles in')+ for line in path.read_text().splitlines():+ line = line.strip()+ if line and not line.startswith('#'):+ raw32(line, str(path))+ return line+ raise Fail(f'{path} holds no public key yet: generate a key pair (`python3 tools/release_manifest.py keygen`), store the private '+ f'half as the repository secret {KEY_ENV}, put the public half in this file and compile the same value into the launcher')+++def signing_key(pub_file, ephemeral=False):+ """The private key to sign with, after checking it belongs to the committed public key. (key, its public base64)."""+ if ephemeral:+ ed, _ = ed25519()+ k = ed.Ed25519PrivateKey.generate()+ return k, public_b64(k)+ secret = os.environ.get(KEY_ENV, '').strip()+ if not secret: raise Fail(f'the repository secret {KEY_ENV} is not set: a release must be signed')+ key = private_key(secret)+ want = read_pub(pub_file)+ have = public_b64(key)+ if have != want:+ raise Fail(f'{KEY_ENV} does not belong to the public key in {pub_file} (its public key is {have}, the file has {want}): '+ f'the launcher would reject every manifest')+ return key, have+++def verify_signature(manifest_bytes, sig_text, pub_b64):+ ed, Invalid = ed25519()+ key = ed.Ed25519PublicKey.from_public_bytes(raw32(pub_b64, 'the public key'))+ try: sig = base64.b64decode(sig_text.strip(), validate=True)+ except Exception: return False+ try: key.verify(sig, manifest_bytes)+ except Invalid: return False+ return True+++# ---- the manifest ----++def sha256_file(path):+ h = hashlib.sha256()+ with open(path, 'rb') as f:+ for chunk in iter(lambda: f.read(1 << 20), b''): h.update(chunk)+ return h.hexdigest()+++def asset(path):+ path = Path(path)+ if not path.is_file(): raise Fail(f'{path} is missing')+ return {'asset': path.name, 'sha256': sha256_file(path), 'size': path.stat().st_size}+++def engine_identity(zip_path):+ """(pin, engine_id) as the engine zip itself says."""+ try:+ with zipfile.ZipFile(zip_path) as z:+ eid = z.read('.engine-id').decode().strip()+ pin = z.read('.upstream-ref').decode().strip()+ except (KeyError, zipfile.BadZipFile, OSError) as e:+ raise Fail(f'{zip_path} is not an engine tree zip from `tools/engine.py tree` (.engine-id or .upstream-ref missing: {e})')+ if not ENGINE_ID.fullmatch(eid): raise Fail(f'.engine-id in {zip_path} is not <pin>-<12 hex>: {eid!r}')+ if eid[:40] != pin: raise Fail(f'.engine-id ({eid}) and .upstream-ref ({pin}) in {zip_path} disagree about the pin')+ return pin, eid+++def build_manifest(engine_zip, version, channel, tag, app_zip=None, repo=GITHUB_REPO, built=None, extra_sources=()):+ if channel not in ('stable', 'experimental'): raise Fail(f'channel must be stable or experimental, not {channel!r}')+ if not version.strip(): raise Fail('--version is empty')+ pin, eid = engine_identity(engine_zip)+ built = built or datetime.datetime.now(datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')+ m = {'version': version, 'pin': pin, 'engine_id': eid, 'channel': channel, 'built': built, 'engine': asset(engine_zip)}+ if app_zip: m['app'] = asset(app_zip)+ m['sources'] = [f'https://github.com/{repo}/releases/download/{tag}/', *extra_sources]+ return (json.dumps(m, indent=2) + '\n').encode()+++# ---- publishing ----++def gh(*a, check=True):+ r = subprocess.run(['gh', *a], capture_output=True, text=True)+ if check and r.returncode: raise Fail(f'gh {" ".join(a[:3])} failed: {(r.stderr or r.stdout).strip()[-600:]}')+ return r+++def upload(files, tag, repo, channel, notes):+ """Engine zip first, the manifest and its signature last, so a launcher that reads the new manifest finds its zip."""+ exists = gh('release', 'view', tag, '-R', repo, check=False).returncode == 0+ if channel == 'experimental':+ # A pre-release under a fixed tag, never "latest", so it never becomes what the app's own update looks at.+ if exists: gh('release', 'edit', tag, '-R', repo, '--prerelease', '--latest=false', '--notes', notes)+ else: gh('release', 'create', tag, '-R', repo, '--prerelease', '--latest=false', '--title', 'Engine (experimental)', '--notes', notes)+ elif not exists:+ raise Fail(f'the release {tag} does not exist: the release workflow creates it before it publishes the engine')+ for f in files:+ gh('release', 'upload', tag, str(f), '-R', repo, '--clobber')+ say(f'uploaded {Path(f).name} to {repo} {tag}')+++def cmd_publish(a):+ if a.upload and a.ephemeral_key: raise Fail('--ephemeral-key signs with a throwaway key: it never uploads')+ key, pub = signing_key(a.pub, a.ephemeral_key)+ out = Path(a.out)+ out.mkdir(parents=True, exist_ok=True)+ zsrc = Path(a.engine_zip)+ data = build_manifest(zsrc, a.version, a.channel, a.tag, a.app_zip, a.repo, a.built, a.source or ())+ files = [out / zsrc.name, out / MANIFEST, out / SIGNATURE]+ if zsrc.resolve() != files[0].resolve(): shutil.copyfile(zsrc, files[0])+ files[1].write_bytes(data)+ files[2].write_text(base64.b64encode(key.sign(data)).decode()) # no newline: Data(base64Encoded:) rejects one+ if not verify_signature(files[1].read_bytes(), files[2].read_text(), pub): raise Fail('the signature just made does not verify')+ m = json.loads(data)+ say(f'{MANIFEST}: {m["channel"]} {m["version"]} engine_id {m["engine_id"]} built {m["built"]}, signed ({"ephemeral key" if a.ephemeral_key else "ENGINE_SIGNING_KEY"})')+ if a.upload:+ notes = (f'Engine {m["engine_id"]} (seedds/nxnative {m["pin"][:12]} + the patch series), built {m["built"]}. '+ + ('Experimental: built by the nightly sync and checked by CI only, not played on hardware. The launcher offers it as a manual, opt-in update.'+ if a.channel == 'experimental' else 'The engine of this release.'))+ upload(files, a.tag, a.repo, a.channel, notes)+ else:+ say(f'dry run: nothing uploaded; the files are in {out}')+ return 0+++def cmd_check_key(a):+ key, pub = signing_key(a.pub)+ say(f'{KEY_ENV} matches {a.pub} ({pub})')+ return 0+++def cmd_verify(a):+ pub = read_pub(a.pub)+ ok = verify_signature(Path(a.manifest).read_bytes(), Path(a.sig).read_text(), pub)+ say('signature valid' if ok else 'signature INVALID')+ return 0 if ok else 1+++def cmd_keygen(a):+ ed, _ = ed25519()+ from cryptography.hazmat.primitives import serialization+ k = ed.Ed25519PrivateKey.generate()+ raw = k.private_bytes(serialization.Encoding.Raw, serialization.PrivateFormat.Raw, serialization.NoEncryption())+ say(f'private (the repository secret {KEY_ENV}, keep it out of the repository): {base64.b64encode(raw).decode()}')+ say(f'public (frontend/engine-signing.pub and the launcher): {public_b64(k)}')+ return 0+++def main(argv=None):+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ sub = ap.add_subparsers(dest='cmd', required=True)+ sub.add_parser('keygen').set_defaults(f=cmd_keygen)+ s = sub.add_parser('check-key'); s.add_argument('--pub', default=str(PUB_FILE)); s.set_defaults(f=cmd_check_key)+ s = sub.add_parser('verify'); s.add_argument('--manifest', required=True); s.add_argument('--sig', required=True); s.add_argument('--pub', default=str(PUB_FILE)); s.set_defaults(f=cmd_verify)+ s = sub.add_parser('publish')+ s.add_argument('--engine-zip', required=True); s.add_argument('--app-zip'); s.add_argument('--version', required=True)+ s.add_argument('--channel', required=True, choices=('stable', 'experimental')); s.add_argument('--tag', required=True)+ s.add_argument('--out', required=True); s.add_argument('--repo', default=os.environ.get('GITHUB_REPOSITORY') or GITHUB_REPO)+ s.add_argument('--built', help='UTC timestamp, 2026-10-31T02:00:00Z (default: now)'); s.add_argument('--source', action='append', help='another base URL for the assets (a mirror), tried after GitHub')+ s.add_argument('--pub', default=str(PUB_FILE)); s.add_argument('--upload', action='store_true'); s.add_argument('--ephemeral-key', action='store_true')+ s.set_defaults(f=cmd_publish)+ a = ap.parse_args(argv)+ try:+ return a.f(a)+ except Fail as e:+ print(f'release_manifest.py: {e}', file=sys.stderr)+ return 2+++if __name__ == '__main__':+ sys.exit(main())diff --git a/tools/requirements-release.txt b/tools/requirements-release.txtnew file mode 100644index 00000000..4e65bf54--- /dev/null+++ b/tools/requirements-release.txt@@ -0,0 +1,2 @@+# What tools/release_manifest.py needs to sign latest.json (Ed25519). Only the release and the nightly publish jobs install it.+cryptography==46.0.3diff --git a/tools/tests/test_engine.py b/tools/tests/test_engine.pyindex bdeac34f..82f812d2 100644--- a/tools/tests/test_engine.py+++ b/tools/tests/test_engine.py@@ -255,6 +255,80 @@ class Search(unittest.TestCase):self.assertEqual(r.stdout.strip(), './engine/dm/needle.c', r.stderr)+class Tree(Fixture):+ """The engine tree for players: engine/ at its last sync, no git, with the identity and a zip that is the same every time."""+ def make(self, name='out', zipname=None):+ out = self.tmp / name+ args = ['tree', '--out', str(out)] + (['--zip', str(self.tmp / zipname)] if zipname else [])+ return out, self.tool(*args)++ def test_the_tree_is_the_patched_engine_without_git_and_names_its_identity(self):+ self.tool('sync')+ out, said = self.make()+ self.assertEqual((out / 'dm/a.c').read_text(), 'line1\nline2 fork\nline3\nline4\nline5\nline6\nline7\n') # the patch is in+ self.assertEqual((out / '.upstream-ref').read_text(), self.up1 + '\n')+ eid = (out / '.engine-id').read_text().strip()+ self.assertEqual(eid, self.up1 + '-' + E.stamp_key(self.r)[:12])+ self.assertRegex(eid, r'^[0-9a-f]{40}-[0-9a-f]{12}$')+ self.assertEqual(eid, E.engine_id(self.r)); self.assertIn(eid, said)+ self.assertIn('line-2', (out / 'dm/fork_patches.txt').read_text())+ self.assertTrue((out / 'AGENTS.md').is_file())+ self.assertFalse((out / '.git').exists()); self.assertFalse((out / '.gitignore').exists()) # no history, no repository files+ self.assertFalse((out / 'tools').exists()); self.assertFalse((out / 'patches').exists()) # nothing of the fork++ def test_a_series_change_without_a_pin_move_changes_the_identity(self):+ self.tool('sync'); out, _ = self.make('one')+ self.add_patch('0002-b.patch', 'dm/b.c', 'b1\nb2 fork\n', 'Change b2', header=HEADER.replace('#1', '#2'))+ self.tool('sync'); out2, _ = self.make('two')+ a, b = (out / '.engine-id').read_text().strip(), (out2 / '.engine-id').read_text().strip()+ self.assertEqual(a[:40], b[:40]); self.assertNotEqual(a, b)++ def test_the_zip_is_deterministic_and_unpacks_to_the_tree(self):+ import hashlib, time, zipfile+ self.tool('sync')+ out, _ = self.make('one', 'one.zip')+ time.sleep(1.1) # other modification times, same bytes+ for f in out.rglob('*'): os.utime(f, (time.time(), time.time()))+ out2, _ = self.make('two', 'two.zip')+ h = lambda n: hashlib.sha256((self.tmp / n).read_bytes()).hexdigest()+ self.assertEqual(h('one.zip'), h('two.zip'))+ with zipfile.ZipFile(self.tmp / 'one.zip') as z:+ names = z.namelist()+ self.assertEqual(names, sorted(names)); self.assertIn('.engine-id', names); self.assertIn('dm/a.c', names)+ self.assertEqual({i.date_time for i in z.infolist()}, {E.ZIP_TIME})+ self.assertEqual(z.read('.engine-id').decode(), (out / '.engine-id').read_text())++ def test_executable_bits_and_symlinks_survive(self):+ import stat, zipfile+ self.wu('dm/run.sh', '#!/bin/sh\n'); os.chmod(self.up / 'dm/run.sh', 0o755)+ os.symlink('a.c', self.up / 'dm/link.c')+ up2 = self.up_commit('dm/c.c', 'c\n', 'upstream 2')+ (self.r / '.upstream-ref').write_text(up2 + '\n')+ self.tool('sync'); out, _ = self.make('out', 'e.zip')+ self.assertTrue(os.access(out / 'dm/run.sh', os.X_OK)); self.assertFalse(os.access(out / 'dm/a.c', os.X_OK))+ self.assertTrue((out / 'dm/link.c').is_symlink())+ with zipfile.ZipFile(self.tmp / 'e.zip') as z:+ self.assertEqual(stat.S_IMODE(z.getinfo('dm/run.sh').external_attr >> 16), 0o755)+ self.assertTrue(stat.S_ISLNK(z.getinfo('dm/link.c').external_attr >> 16))++ def test_a_stale_or_dirty_engine_makes_no_tree(self):+ self.tool('tree', '--out', str(self.tmp / 'o'), expect=2) # no engine/ at all+ self.tool('sync')+ self.add_patch('0002-b.patch', 'dm/b.c', 'b1\nb2 fork\n', 'Change b2', header=HEADER.replace('#1', '#2'))+ self.assertIn('not a current sync', self.tool('tree', '--out', str(self.tmp / 'o'), expect=2))+ self.tool('sync')+ (self.eng / 'dm/a.c').write_text('local edit\n')+ self.assertIn('changes the release must not carry', self.tool('tree', '--out', str(self.tmp / 'o'), expect=2))+ self.assertFalse((self.tmp / 'o/.engine-id').exists())++ def test_it_will_not_empty_a_folder_that_is_not_an_earlier_tree(self):+ self.tool('sync')+ keep = self.tmp / 'precious'; keep.mkdir(); (keep / 'x').write_text('mine')+ self.assertIn('not an engine tree', self.tool('tree', '--out', str(keep), expect=2))+ self.assertEqual((keep / 'x').read_text(), 'mine')+ self.make('again'); self.make('again') # an earlier tree is replaced++class StampKey(Fixture):def test_the_qt_front_end_computes_the_same_key(self):sys.path.insert(0, str(REPO / 'frontend-qt'))diff --git a/tools/tests/test_release_manifest.py b/tools/tests/test_release_manifest.pynew file mode 100644index 00000000..7a6ad6bd--- /dev/null+++ b/tools/tests/test_release_manifest.py@@ -0,0 +1,228 @@+"""tools/release_manifest.py (LNPANDY/NX2Apple#227): latest.json has the shape EngineManifest.Manifest decodes, the signature is a+detached Ed25519 signature over its exact bytes, the release fails without the right key, and uploads go in the right order with a+fake `gh`. The keys are generated here: nothing secret is used and nothing touches the network."""+import base64+import hashlib+import importlib+import json+import os+import stat+import subprocess+import sys+import tempfile+import unittest+import zipfile+from pathlib import Path++TOOLS = Path(__file__).resolve().parents[1]+sys.path.insert(0, str(TOOLS))+R = importlib.import_module('release_manifest')++try:+ from cryptography.hazmat.primitives import serialization+ from cryptography.hazmat.primitives.asymmetric import ed25519+except ImportError:+ ed25519 = None++PIN = 'ee6648cb2bb46c19255c1d04f88e9c40d36ef6b4'+EID = PIN + '-d4abe1fa2485'+++def new_key():+ k = ed25519.Ed25519PrivateKey.generate()+ raw = k.private_bytes(serialization.Encoding.Raw, serialization.PrivateFormat.Raw, serialization.NoEncryption())+ return k, base64.b64encode(raw).decode(), R.public_b64(k)++[email protected](ed25519 is None, 'needs `cryptography` (tools/requirements-release.txt)')+class Manifest(unittest.TestCase):+ def setUp(self):+ self.tmp = Path(tempfile.mkdtemp(prefix='test-manifest-'))+ self.addCleanup(__import__('shutil').rmtree, self.tmp, True)+ self.key, self.secret, self.pub = new_key()+ self.pubfile = self.tmp / 'engine-signing.pub'+ self.pubfile.write_text(f'# comment\n\n{self.pub}\n')+ self.engine = self.zip('NX2Apple-engine.zip', {'.engine-id': EID + '\n', '.upstream-ref': PIN + '\n', 'dm/a.c': 'x'})+ self.app = self.tmp / 'NX2Apple-macOS-arm64.zip'; self.app.write_bytes(b'app bytes')+ self.env = dict(os.environ, **{R.KEY_ENV: self.secret})+ self.old_env = dict(os.environ)+ os.environ[R.KEY_ENV] = self.secret+ self.addCleanup(self.restore)++ def restore(self):+ os.environ.clear(); os.environ.update(self.old_env)++ def zip(self, name, files):+ p = self.tmp / name+ with zipfile.ZipFile(p, 'w') as z:+ for n, t in files.items(): z.writestr(n, t)+ return p++ def publish(self, *extra, channel='stable', tag='v0.8.0', out='out', expect=0):+ args = ['publish', '--engine-zip', str(self.engine), '--version', '0.8.0', '--channel', channel, '--tag', tag,+ '--out', str(self.tmp / out), '--pub', str(self.pubfile), '--built', '2026-10-31T02:00:00Z', *extra]+ self.assertEqual(R.main(args), expect)+ return self.tmp / out++ # ---- the shape the launcher decodes ----++ def test_the_manifest_has_every_field_the_launcher_decodes(self):+ out = self.publish('--app-zip', str(self.app))+ m = json.loads((out / 'latest.json').read_text())+ self.assertEqual(m['version'], '0.8.0'); self.assertEqual(m['pin'], PIN); self.assertEqual(m['engine_id'], EID)+ self.assertEqual(m['channel'], 'stable'); self.assertEqual(m['built'], '2026-10-31T02:00:00Z')+ self.assertEqual(m['sources'], ['https://github.com/LNPANDY/NX2Apple/releases/download/v0.8.0/'])+ self.assertEqual(set(m['engine']), {'asset', 'sha256', 'size'})+ self.assertEqual(m['engine']['asset'], 'NX2Apple-engine.zip')+ self.assertEqual(m['engine']['sha256'], hashlib.sha256(self.engine.read_bytes()).hexdigest())+ self.assertEqual(m['engine']['size'], self.engine.stat().st_size)+ self.assertEqual(m['app'], {'asset': 'NX2Apple-macOS-arm64.zip', 'sha256': hashlib.sha256(b'app bytes').hexdigest(), 'size': 9})+ for a in (m['engine'], m['app']): self.assertRegex(a['sha256'], r'^[0-9a-f]{64}$') # EngineManifest.isDigest+ self.assertRegex(m['engine_id'], r'^[0-9a-f]{40}-[0-9a-f]{12}$')++ def test_the_experimental_channel_has_no_app_and_its_own_source(self):+ out = self.publish(channel='experimental', tag='engine-experimental')+ m = json.loads((out / 'latest.json').read_text())+ self.assertEqual(m['channel'], 'experimental'); self.assertNotIn('app', m)+ self.assertEqual(m['sources'], ['https://github.com/LNPANDY/NX2Apple/releases/download/engine-experimental/'])++ def test_the_manifest_bytes_are_the_same_for_the_same_input(self):+ a = (self.publish(out='a') / 'latest.json').read_bytes(); b = (self.publish(out='b') / 'latest.json').read_bytes()+ self.assertEqual(a, b)++ def test_a_mirror_is_added_as_a_source_after_github(self):+ out = self.publish('--source', 'https://mirror.example/engine/')+ self.assertEqual(json.loads((out / 'latest.json').read_text())['sources'][1], 'https://mirror.example/engine/')++ def test_a_zip_that_is_not_an_engine_tree_is_refused(self):+ self.engine = self.zip('bad.zip', {'dm/a.c': 'x'})+ self.publish(expect=2)+ self.engine = self.zip('bad2.zip', {'.engine-id': 'ABC\n', '.upstream-ref': PIN + '\n'})+ self.publish(expect=2)+ self.engine = self.zip('bad3.zip', {'.engine-id': PIN[:-1] + '0-d4abe1fa2485\n', '.upstream-ref': PIN + '\n'})+ self.publish(expect=2)++ # ---- the signature ----++ def test_the_signature_is_detached_ed25519_over_the_exact_bytes(self):+ out = self.publish()+ data = (out / 'latest.json').read_bytes(); sig = (out / 'latest.json.sig').read_text()+ self.assertFalse(sig.endswith('\n')) # Data(base64Encoded:) rejects a newline+ raw = base64.b64decode(sig, validate=True); self.assertEqual(len(raw), 64)+ self.key.public_key().verify(raw, data) # what CryptoKit's isValidSignature checks+ self.assertTrue(R.verify_signature(data, sig, self.pub))+ self.assertFalse(R.verify_signature(data + b' ', sig, self.pub)) # one byte more: invalid+ self.assertFalse(R.verify_signature(data.replace(b'0.8.0', b'0.8.1'), sig, self.pub))+ other = new_key()[2]+ self.assertFalse(R.verify_signature(data, sig, other))+ self.assertFalse(R.verify_signature(data, 'not base64!', self.pub))++ def test_verify_subcommand(self):+ out = self.publish()+ args = ['verify', '--manifest', str(out / 'latest.json'), '--sig', str(out / 'latest.json.sig'), '--pub', str(self.pubfile)]+ self.assertEqual(R.main(args), 0)+ (out / 'latest.json').write_bytes((out / 'latest.json').read_bytes().replace(b'stable', b'rolling'))+ self.assertEqual(R.main(args), 1)++ # ---- the key check ----++ def test_a_release_fails_without_the_secret(self):+ del os.environ[R.KEY_ENV]+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)+ self.publish(expect=2)+ os.environ[R.KEY_ENV] = ' '+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)++ def test_a_release_fails_when_the_key_is_not_the_one_in_the_repository(self):+ os.environ[R.KEY_ENV] = new_key()[1]+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)+ self.publish(expect=2)+ self.assertFalse((self.tmp / 'out' / 'latest.json').exists())++ def test_a_release_fails_while_the_repository_has_no_public_key(self):+ self.pubfile.write_text('# nothing yet\n')+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)+ self.pubfile.unlink()+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)++ def test_malformed_keys_are_refused(self):+ os.environ[R.KEY_ENV] = 'not base64!'+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)+ os.environ[R.KEY_ENV] = base64.b64encode(b'short').decode()+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)+ os.environ[R.KEY_ENV] = self.secret+ self.pubfile.write_text(base64.b64encode(b'short').decode() + '\n')+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 2)++ def test_the_matching_key_passes(self):+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 0)++ def test_a_dry_run_can_sign_with_a_throwaway_key_but_never_upload(self):+ del os.environ[R.KEY_ENV]+ self.pubfile.write_text('# none\n')+ out = self.publish('--ephemeral-key')+ data = (out / 'latest.json').read_bytes(); sig = (out / 'latest.json.sig').read_text()+ self.assertFalse(R.verify_signature(data, sig, self.pub)) # not the repository's key+ self.publish('--ephemeral-key', '--upload', out='out2', expect=2)++ def test_keygen_makes_a_pair_the_check_accepts(self):+ import contextlib, io+ buf = io.StringIO()+ with contextlib.redirect_stdout(buf): self.assertEqual(R.main(['keygen']), 0)+ lines = buf.getvalue().splitlines()+ priv, pub = lines[0].rsplit(': ', 1)[1], lines[1].rsplit(': ', 1)[1]+ os.environ[R.KEY_ENV] = priv; self.pubfile.write_text(pub + '\n')+ self.assertEqual(R.main(['check-key', '--pub', str(self.pubfile)]), 0)++ # ---- upload, with a fake gh ----++ def fake_gh(self, exists):+ """A `gh` that logs its arguments and answers `release view` as asked."""+ d = self.tmp / 'bin'; d.mkdir(exist_ok=True)+ log = self.tmp / 'gh.log'+ gh = d / 'gh'+ gh.write_text(f'#!/bin/sh\necho "$@" >> "{log}"\n[ "$1 $2" = "release view" ] && exit {0 if exists else 1}\nexit 0\n')+ gh.chmod(gh.stat().st_mode | stat.S_IXUSR)+ os.environ['PATH'] = f'{d}:{os.environ["PATH"]}'+ return log++ def calls(self, log):+ return [l.split() for l in log.read_text().splitlines()]++ def test_the_stable_release_uploads_the_zip_first_and_the_signature_last(self):+ log = self.fake_gh(exists=True)+ self.publish('--app-zip', str(self.app), '--upload', '--repo', 'LNPANDY/NX2Apple')+ ups = [c for c in self.calls(log) if c[:2] == ['release', 'upload']]+ self.assertEqual([Path(c[3]).name for c in ups], ['NX2Apple-engine.zip', 'latest.json', 'latest.json.sig'])+ self.assertTrue(all(c[2] == 'v0.8.0' and '--clobber' in c and c[4:6] == ['-R', 'LNPANDY/NX2Apple'] for c in ups))+ self.assertFalse([c for c in self.calls(log) if c[:2] in (['release', 'create'], ['release', 'edit'])])++ def test_the_stable_release_needs_its_release_to_exist(self):+ log = self.fake_gh(exists=False)+ self.publish('--upload', expect=2)+ self.assertFalse([c for c in self.calls(log) if c[:2] == ['release', 'upload']])++ def test_the_experimental_tag_is_created_as_a_prerelease_that_is_never_latest(self):+ log = self.fake_gh(exists=False)+ self.publish('--upload', channel='experimental', tag='engine-experimental')+ create = [c for c in self.calls(log) if c[:2] == ['release', 'create']]+ self.assertEqual(len(create), 1)+ self.assertEqual(create[0][2], 'engine-experimental'); self.assertIn('--prerelease', create[0]); self.assertIn('--latest=false', create[0])+ self.assertEqual(len([c for c in self.calls(log) if c[:2] == ['release', 'upload']]), 3)++ def test_an_existing_experimental_release_is_kept_a_prerelease_and_its_assets_replaced(self):+ log = self.fake_gh(exists=True)+ self.publish('--upload', channel='experimental', tag='engine-experimental')+ edit = [c for c in self.calls(log) if c[:2] == ['release', 'edit']]+ self.assertEqual(len(edit), 1); self.assertIn('--prerelease', edit[0]); self.assertIn('--latest=false', edit[0])+ self.assertFalse([c for c in self.calls(log) if c[:2] == ['release', 'create']])+ self.assertTrue(all('--clobber' in c for c in self.calls(log) if c[:2] == ['release', 'upload']))++ def test_a_dry_run_calls_gh_never(self):+ log = self.fake_gh(exists=True)+ self.publish()+ self.assertFalse(log.exists())+++if __name__ == '__main__':+ unittest.main()