Skip to content

Play your own Switch games on iPhone/iPad — one-click macOS app, no terminal. | 把 Switch 游戏装进 iPhone:图形界面一键构建,不用命令行。Bring your own game.

32.7 KB
Raw Blame
#!/usr/bin/env python3
"""The engine in engine/: seedds/nxnative at the pin plus the patch queue, fetched and patched where it is needed
(LNPANDY/NX2Apple#214).

The repository keeps the pin (.upstream-ref) and the patches (patches/series, patches/*.patch). engine/ is a real git
repository, ignored by the outer one: upstream's commit at the pin (a shallow fetch of that one commit), then one commit per
patch (`git am -3`), then one commit with the generated files (the indexes of tools/patches.py's GENERATED and
dm/fork_patches.txt). A stamp in engine/.git records the pin and the series it was built from, so `sync` is a no-op while
neither changed.

Subcommands (Python 3.9+, stdlib and git):
  sync [--force]               make engine/ match the pin and the series. A no-op when the stamp matches. Refuses to throw
                               away commits or changes made in engine/ (export them first, or --force keeps them on a branch)
  stale                        exit 0 when engine/ matches the pin and the series, 1 when a sync is needed (prints why)
  hook                         for an agent's session start (.claude/settings.json): sync when stale, print one line, always exit 0
  export [REV]                 turn commits made in engine/ (default: everything after the last sync) into patches with the
                               Upstream-Status / Reason / Issue header, and add them to patches/series
  status                       the patches (Upstream-Status, Reason, age) and the state of engine/
  tree --out DIR [--zip FILE]  the engine tree for players, the release's engine (see below)
  check                        headers valid, no patch touches the shader chain or the generated files, the series applies
                               cleanly to the pin with no empty patch
  bump --to SHA|main [--dry]   move the pin: re-apply the series on the new commit, drop the patches upstream has, write
                               patches/ and .upstream-ref, then print the candidate list (below; also with --dry)
  candidates [--from SHA] --to SHA|main
                               the upstream commits between two pins that probably change what the front ends and the players
                               depend on, as a Markdown table (tools/bump_candidates.py, LNPANDY/NX2Apple#464). A list of
                               candidates, not a verdict; needs `gh` for the commits' files, never fails the bump
  refresh WORKTREE --to SHA    after a conflicting bump was resolved by hand in WORKTREE: its commits become the series
  relink                       make the links to seedds/nxnative in the README and the other pages point at the pinned commit (bump and
                               refresh do it, check fails while a page names another commit)

The upstream URL is https://github.com/seedds/nxnative.git, or $NX2APPLE_ENGINE_URL (a mirror, or a local clone that holds the
pin, which needs no network). `sync` is for developers, CI and checkouts: a release carries the engine tree (`tree`) instead.
Scratch worktrees go to $PATCHES_TMP (or --tmp), else the system's temporary folder.

The engine tree (`tree`): what a player's launcher runs the build from, with no git. DIR gets the files of engine/ at its last sync
(TREE_PATHS, the patches applied, dm/fork_patches.txt included), `.upstream-ref` and `.engine-id`, the engine's identity
`<pin>-<first 12 hex of the series digest>`: the pin alone does not name a tree, since the series changes without the pin moving.
The zip lists its entries sorted, with fixed timestamps and permissions (0755 or 0644), so the same tree gives the same bytes
(for one zlib). tools/release_manifest.py publishes it.
"""
import argparse
import contextlib
import hashlib
import importlib
import io
import os
import re
import shutil
import subprocess
import sys
import time
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parent))
P = importlib.import_module('patches')   # tools/patches.py: the headers, apply_series, generate, the chain rule
C = importlib.import_module('bump_candidates')   # tools/bump_candidates.py: the commits of a pin move that front ends depend on (#464)

REPO = Path(__file__).resolve().parent.parent
UPSTREAM_URL = 'https://github.com/seedds/nxnative.git'
STAMP = 'nx2apple-stamp'          # in engine/.git: invisible to `git status` in engine/
BRANCH = 'nx2apple'               # the branch sync builds: the pin, the patches, the generated files
GENERATED_SUBJECT = 'NX2Apple: generated files (indexes, dm/fork_patches.txt)'
IDENT = ('-c', 'user.name=tools/engine.py', '-c', 'user.email=engine@localhost', '-c', 'commit.gpgsign=false', '-c', 'core.autocrlf=false')

Fail = P.Fail
say = P.say


def git(*a, cwd, check=True, binary=False):
    return P.git(*a, cwd=cwd, check=check, binary=binary)


HOOKS = '.githooks'


def install_hooks(root):
    """core.hooksPath -> .githooks (the pre-push and pre-commit checks), unless it is set. A path the user chose is kept, with a warning.
    The setting lives in the repository's config, so every worktree of it has the hooks."""
    if not (root / HOOKS / 'pre-push').is_file(): return
    cur = git('config', '--get', 'core.hooksPath', cwd=root, check=False).stdout.strip()
    if cur.rstrip('/') == HOOKS: return
    if cur:
        say(f'WARNING: core.hooksPath is {cur!r}, not {HOOKS!r}, so the pre-push checks do not run here. '
            f'Install them: git config core.hooksPath {HOOKS}')
    elif git('config', '--local', 'core.hooksPath', HOOKS, cwd=root, check=False).returncode == 0:
        say(f'git hooks: set core.hooksPath to {HOOKS} (pre-commit and pre-push checks, tools/githooks.py). Never bypass them with --no-verify.')


def engine_dir(root):
    return root / 'engine'


def pin(root):
    f = root / '.upstream-ref'
    if not f.is_file(): raise Fail(f'{f} is missing: this is not an NX2Apple checkout')
    sha = f.read_text(encoding='utf-8').strip()
    if not re.fullmatch(r'[0-9a-f]{40}', sha): raise Fail(f'.upstream-ref must hold a full commit id (is: {sha!r})')
    return sha


def stamp_key(root):
    """The digest of what engine/ is built from: the pin, and each patch of the series by name and content, in order.
    frontend/Sources/NXCore/EngineLayout.swift and frontend-qt/nspb_qt/engine.py compute the same, so the launchers can
    tell a stale engine/ without starting Python."""
    h = hashlib.sha256()
    h.update(b'pin ' + pin(root).encode() + b'\n')
    for name in P.read_series(root):
        f = root / 'patches' / name
        if not f.is_file(): raise Fail(f'listed in patches/series but missing: {name}')
        h.update(b'patch ' + name.encode() + b'\n' + hashlib.sha256(f.read_bytes()).hexdigest().encode() + b'\n')
    return h.hexdigest()


def read_stamp(eng):
    try: text = (eng / '.git' / STAMP).read_text(encoding='utf-8')
    except OSError: return {}
    return dict(l.split(' ', 1) for l in text.splitlines() if ' ' in l)


def write_stamp(eng, root, head, key=None):
    (eng / '.git' / STAMP).write_text(f'nx2apple-engine 1\npin {pin(root)}\nkey {key or stamp_key(root)}\nhead {head}\n', encoding='utf-8')


def is_repo(eng):
    if not (eng / '.git').is_dir(): return False
    top = git('rev-parse', '--show-toplevel', cwd=eng, check=False).stdout.strip()
    return bool(top) and Path(top).resolve() == eng.resolve()


def head(eng):
    r = git('rev-parse', '--verify', '--quiet', 'HEAD', cwd=eng, check=False)
    return r.stdout.strip() if r.returncode == 0 else ''


SYNC_COMMITTERS = ('patches@localhost', 'engine@localhost')   # `git am` in patches.py and the generated-files commit above


def made_by_sync(eng, h, pins):
    """True when the commits from HEAD down to a pin were all committed by sync itself: what a sync that failed halfway
    (older versions did not roll back) leaves behind. A commit made by a person has another committer."""
    pins = {p for p in pins if p}
    n = 0
    while h not in pins:
        r = git('log', '-1', '--format=%P%n%ce', h, cwd=eng, check=False).stdout.split('\n')
        if len(r) < 2 or len(r[0].split()) != 1 or r[1] not in SYNC_COMMITTERS: return False
        h, n = r[0], n + 1
    return n > 0


def local_work(eng, st, pins=()):
    """What sync would throw away: commits after the last sync, and changes to tracked or new files (ignored build products
    such as dm/host/gen are not work and survive a sync)."""
    out = []
    h = head(eng)
    if st.get('head') and h and h != st['head'] and not made_by_sync(eng, h, pins):
        n = git('rev-list', '--count', f'{st["head"]}..{h}', cwd=eng, check=False).stdout.strip() or '?'
        out.append(f'{n} commit(s) after the last sync (HEAD {h[:12]})')
    dirty = git('status', '--porcelain', cwd=eng, check=False).stdout.splitlines()
    if dirty: out.append(f'{len(dirty)} changed or new file(s), e.g. {dirty[0][3:]}')
    return out


def why_stale(root):
    """'' when engine/ matches the pin and the series, else the reason in a few words."""
    eng = engine_dir(root)
    if not is_repo(eng): return 'engine/ is missing'
    st = read_stamp(eng)
    if not st: return 'engine/ has no stamp (not made by tools/engine.py sync)'
    if st.get('key') != stamp_key(root):
        return 'the pin moved' if st.get('pin') != pin(root) else 'patches/ changed'
    if st.get('head') != head(eng): return 'engine/ has commits after the last sync'
    return ''


# ---- getting the pinned commit ----

def have_commit(eng, sha):
    return git('cat-file', '-e', f'{sha}^{{commit}}', cwd=eng, check=False).returncode == 0


def fetch(eng, sha, url=None):
    """Bring the commit into engine/ from upstream, one commit deep."""
    if have_commit(eng, sha): return 'already here'
    url = url or os.environ.get('NX2APPLE_ENGINE_URL') or UPSTREAM_URL
    r = git('fetch', '--quiet', '--no-tags', '--depth', '1', url, sha, cwd=eng, check=False)
    if r.returncode or not have_commit(eng, sha):
        err = (r.stderr or '').strip().splitlines()
        raise Fail(f'cannot fetch {sha[:12]} from {url}: {err[-1] if err else "unknown error"}\n'
                   f'  Offline or GitHub unreachable? Check the connection and run `python3 tools/engine.py sync` again. '
                   f'Behind a mirror or with a local clone that has the commit: set NX2APPLE_ENGINE_URL.')
    return f'from {url}'


def ensure_repo(eng):
    if is_repo(eng):
        git('config', 'core.autocrlf', 'false', cwd=eng)
        return
    if eng.exists() and any(eng.iterdir()):
        if not any(f.is_file() for f in eng.rglob('*')):
            # An empty tree is the leftover of a failed first sync: on Windows the engine sync
            # needs git, and the failure hit right after this folder was made, so every retry
            # then stopped here forever ("move it away"). Start over instead of asking.
            shutil.rmtree(eng, ignore_errors=True)
        else:
            raise Fail(f'{eng} exists but is not a git repository made by tools/engine.py: move it away, then run sync again')
    eng.mkdir(parents=True, exist_ok=True)
    git('init', '--quiet', cwd=eng)
    git('config', 'advice.detachedHead', 'false', cwd=eng)
    git('config', 'core.autocrlf', 'false', cwd=eng)
    git('remote', 'add', 'seedds', UPSTREAM_URL, cwd=eng, check=False)   # for `git fetch seedds` and pull requests upstream


# ---- sync ----

def build(root, eng, patches, sha):
    """Check out the pin on BRANCH (keeping ignored build products), apply the series, commit the generated files."""
    git('checkout', '--quiet', '--force', '-B', BRANCH, sha, cwd=eng)
    git('clean', '--quiet', '-fd', cwd=eng)        # untracked files, not ignored ones (dm/host/gen and other build products stay)
    applied, landed, conflicts = P.apply_series(eng, patches)
    if conflicts:
        p, msg = conflicts[0]
        git('am', '--abort', cwd=eng, check=False)
        raise Fail(f'{p.file} does not apply to the pin {sha[:12]}\n  Reason: {p.reason}\n  Upstream-Status: {p.status}\n  {msg}\n'
                   f'  The series is broken on this pin: see patches/README.md, "Bumping, and what a conflict is".')
    if landed:
        say('empty on the pin (upstream has them, `tools/engine.py bump` drops them): ' + ', '.join(p.file for p in landed))
    for out, tool in P.GENERATED:
        if (eng / tool).is_file():
            r = subprocess.run([sys.executable, '-X', 'utf8', str(eng / tool)], cwd=eng, capture_output=True, text=True, encoding='utf-8', errors='replace', check=False, env=P.clean_env())
            if r.returncode: raise Fail(f'{tool} failed on the patched tree: {(r.stderr.strip().splitlines() or [str(r.returncode)])[-1]}')
    (eng / P.FORK_PATCHES).write_text(P.fork_patch_list([p for p, _ in applied]), encoding='utf-8')
    git('add', '--force', '--', P.FORK_PATCHES, *(o for o, _ in P.GENERATED if (eng / o).exists()), cwd=eng)
    git(*IDENT, 'commit', '--quiet', '--allow-empty', '-m', GENERATED_SUBJECT, cwd=eng)
    return applied, landed


def restore(eng, rev, branch):
    git('am', '--abort', cwd=eng, check=False)
    git('checkout', '--quiet', '--force', '-B', BRANCH, rev, cwd=eng)
    if branch and branch != BRANCH: git('checkout', '--quiet', '--force', branch, cwd=eng)


def cmd_sync(a):
    root = a.root
    install_hooks(root)
    eng = engine_dir(root)
    reason = why_stale(root)
    if not reason and not a.force:
        if not a.quiet: say(f'engine: up to date (upstream {pin(root)[:12]}, {len(P.read_series(root))} patches) in {eng}')
        return 0
    patches = P.load(root)
    ensure_repo(eng)
    st = read_stamp(eng)
    work = local_work(eng, st, (pin(root), st.get('pin'))) if head(eng) else []
    if work and st and not a.force:
        raise Fail('engine/ has work a sync would throw away: ' + '; '.join(work) + '.\n'
                   '  Turn commits into patches with `python3 tools/engine.py export`, then sync. Or keep them on a branch and '
                   'sync anyway: `python3 tools/engine.py sync --force`.')
    if head(eng) and (work or not st):
        keep = f'nx2apple-backup-{time.strftime("%Y%m%d-%H%M%S")}'
        git(*IDENT, 'stash', '--include-untracked', '--quiet', cwd=eng, check=False)
        git('branch', '--force', keep, 'HEAD', cwd=eng, check=False)
        say(f'engine: kept the previous HEAD on branch {keep} (and uncommitted changes in `git -C engine stash list`)')
    sha = pin(root)
    got = fetch(eng, sha)
    say(f'engine: building {sha[:12]} + {len(patches)} patches ({reason or "forced"}; upstream commit {got})')
    before = head(eng)
    branch = git('symbolic-ref', '--quiet', '--short', 'HEAD', cwd=eng, check=False).stdout.strip()
    try:
        applied, _ = build(root, eng, patches, sha)
    except Fail:
        if before: restore(eng, before, branch)    # a failed sync leaves engine/ as it was: the next sync must not see commits nobody made
        raise
    write_stamp(eng, root, head(eng))
    say(f'engine: ready, {len(applied)} patches on upstream {sha[:12]} in {eng}')
    return 0


def cmd_stale(a):
    reason = why_stale(a.root)
    if reason:
        say(f'engine: stale ({reason}): run `python3 tools/engine.py sync`'); return 1
    if not a.quiet: say('engine: up to date')
    return 0


def cmd_hook(a):
    """One line for a session's context, whatever happens: a failed sync must never block the session."""
    buf = io.StringIO()
    try:
        install_hooks(a.root)
        reason = why_stale(a.root)
        if reason:
            with contextlib.redirect_stdout(buf):
                cmd_sync(a)
        n = len(P.read_series(a.root))
        done = f'synced ({reason})' if reason else 'up to date'
        say(f'engine/: {done}, upstream {pin(a.root)[:12]} + {n} patches. The engine guide is engine/AGENTS.md. '
            f'Engine work: commit in engine/, then `python3 tools/engine.py export`.')
    except (Fail, OSError, ValueError, subprocess.SubprocessError) as e:   # a failed fetch, a missing git: say it in one line
        msg = ' '.join(str(e).split())[:300]
        say(f'engine/: sync failed ({msg}). Run `python3 tools/engine.py sync` to see why.')
    return 0


# ---- tree: the engine for players ----

TREE_PATHS = ('dm', 'scripts', 'docs', 'ref', 'AGENTS.md', 'LICENSE')   # what a player's build reads, of what engine/ tracks
ENGINE_ID_FILE = '.engine-id'
ZIP_TIME = (2020, 1, 1, 0, 0, 0)    # fixed, so the zip's bytes depend on the content only


def engine_id(root):
    """`<pin>-<first 12 hex of the series digest>`: names a tree (the series changes without the pin moving). [0-9a-f-], 53 characters."""
    return f'{pin(root)}-{stamp_key(root)[:12]}'


def extract_tree(eng, out):
    """The files of engine/'s HEAD under TREE_PATHS into `out`: `git archive`, so no .git and nothing untracked or ignored."""
    import tarfile
    names = [n for n in TREE_PATHS if git('cat-file', '-e', f'HEAD:{n}', cwd=eng, check=False).returncode == 0]
    if not names: raise Fail(f'{eng} holds none of {", ".join(TREE_PATHS)}: not an engine checkout')
    proc = subprocess.Popen(['git', 'archive', '--format=tar', 'HEAD', '--', *names], cwd=eng, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=P.clean_env())
    count = 0
    with tarfile.open(fileobj=proc.stdout, mode='r|') as tar:
        for m in tar:
            dest = out / m.name
            if m.isdir(): dest.mkdir(parents=True, exist_ok=True)
            elif m.isreg():
                dest.parent.mkdir(parents=True, exist_ok=True)
                with tar.extractfile(m) as src, open(dest, 'wb') as f: shutil.copyfileobj(src, f)
                dest.chmod(0o755 if m.mode & 0o111 else 0o644)
                count += 1
            elif m.issym():
                dest.parent.mkdir(parents=True, exist_ok=True)
                os.symlink(m.linkname, dest)
                count += 1
    proc.stdout.close()
    if proc.wait(): raise Fail(f'git archive failed: {proc.stderr.read().decode(errors="replace").strip()[-500:]}')
    return count


def write_zip(tree, zpath):
    """Every file of `tree` (the tree itself is the archive's root), sorted by path, fixed time, 0755 or 0644, deflate level 6."""
    import stat
    import zipfile
    entries = []
    for dirpath, dirnames, files in os.walk(tree):
        dirnames.sort()
        for n in files + [d for d in dirnames if os.path.islink(os.path.join(dirpath, d))]:
            full = os.path.join(dirpath, n)
            entries.append((Path(full).relative_to(tree).as_posix(), full))
    entries.sort()
    zpath = Path(zpath)
    zpath.parent.mkdir(parents=True, exist_ok=True)
    tmp = zpath.with_name(zpath.name + '.part')
    with zipfile.ZipFile(tmp, 'w', zipfile.ZIP_DEFLATED, compresslevel=6) as z:
        for rel, full in entries:
            zi = zipfile.ZipInfo(rel, ZIP_TIME)
            zi.create_system = 3
            if os.path.islink(full):
                zi.external_attr = (stat.S_IFLNK | 0o755) << 16
                z.writestr(zi, os.readlink(full))
                continue
            zi.compress_type = zipfile.ZIP_DEFLATED
            zi.external_attr = (stat.S_IFREG | (0o755 if os.access(full, os.X_OK) else 0o644)) << 16
            with open(full, 'rb') as f: z.writestr(zi, f.read(), compresslevel=6)
    os.replace(tmp, zpath)
    return len(entries)


def cmd_tree(a):
    root = a.root
    eng = engine_dir(root)
    reason = why_stale(root)
    if reason: raise Fail(f'engine/ is not a current sync ({reason}): run `python3 tools/engine.py sync` first')
    work = local_work(eng, read_stamp(eng))
    if work: raise Fail('engine/ has changes the release must not carry: ' + '; '.join(work) + '. Export them as patches or sync --force.')
    out = Path(a.out).resolve()
    if out.exists() and any(out.iterdir()):
        if not (out / ENGINE_ID_FILE).is_file(): raise Fail(f'{out} exists and is not an engine tree made by this command: choose an empty folder')
        shutil.rmtree(out)
    out.mkdir(parents=True, exist_ok=True)
    n = extract_tree(eng, out)
    eid = engine_id(root)
    (out / '.upstream-ref').write_text(pin(root) + '\n', encoding='utf-8')
    (out / ENGINE_ID_FILE).write_text(eid + '\n', encoding='utf-8')
    if not (out / P.FORK_PATCHES).is_file(): raise Fail(f'{P.FORK_PATCHES} is missing from engine/: sync again')
    say(f'engine tree: {eid} ({n} files) in {out}')
    if a.zip:
        k = write_zip(out, a.zip)
        h = hashlib.sha256(Path(a.zip).read_bytes()).hexdigest()
        say(f'engine zip: {a.zip} ({k} entries, {Path(a.zip).stat().st_size} bytes, sha256 {h})')
    return 0


# ---- export ----

def cmd_export(a):
    root = a.root
    install_hooks(root)
    eng = engine_dir(root)
    if not is_repo(eng): raise Fail('engine/ is missing: run `python3 tools/engine.py sync`, commit your change in engine/, then export')
    st = read_stamp(eng)
    rev = a.rev or (f'{st["head"]}..HEAD' if st.get('head') else None)
    if not rev: raise Fail('engine/ has no stamp: name the commits, e.g. `tools/engine.py export HEAD~2..HEAD`')
    spec = rev if '..' in rev else f'{rev}^..{rev}'
    commits = git('rev-list', '--reverse', spec, cwd=eng).stdout.split()
    if not commits: raise Fail(f'no commits in {spec} (commit your change in engine/ first)')
    existing = P.load(root)
    pd = root / 'patches'
    n = max([int(m.group(1)) for p in existing for m in [re.match(r'(\d+)-', p.file)] if m] or [0])
    series = (pd / 'series').read_text()
    if series and not series.endswith('\n'): series += '\n'
    skip = {P.FORK_PATCHES, *(o for o, _ in P.GENERATED)}
    made = []
    for c in commits:
        if git('log', '-1', '--format=%s', c, cwd=eng).stdout.strip() == GENERATED_SUBJECT: continue
        text = git('format-patch', '--binary', '--zero-commit', '--no-signature', '--no-numbered', '--stdout', f'{c}^..{c}', '--', '.',
                   *(f':(exclude){o}' for o in sorted(skip)), cwd=eng, binary=True).stdout.decode(errors='surrogateescape')
        if not re.search(r'(?m)^diff --git ', text):
            say(f'skipped {c[:10]}: it changes only generated files'); continue
        files = sorted({f for m in re.finditer(r'(?m)^diff --git a/(\S+) b/(\S+)$', text) for f in m.groups()})
        hits = P.chain_hits(files, eng)
        if hits:
            raise Fail(f'{c[:10]} touches the shader chain or the tool pins ({", ".join(hits)}): patches must not change them (patches/README.md, Patch policy)')
        head_, sep, rest = text.partition('\n---\n')
        add = [f'{k}: {v}' for k, v in (('Upstream-Status', a.status), ('Reason', a.reason), ('Issue', a.issue))
               if not re.search(r'(?m)^' + k + ':', head_)]
        if add: head_ = head_.rstrip('\n') + '\n\n' + '\n'.join(add) + '\n'
        text = head_ + sep + rest
        m = re.search(r'(?m)^Subject: (?:\[PATCH[^\]]*\] )?(.*)$', text)
        subj = m.group(1) if m else c[:10]
        n += 1
        name = f'{n:04d}-{P.slug(subj)}.patch'
        (pd / name).write_bytes(text.encode(errors='surrogateescape'))
        series += name + '\n'; made.append(name)
    (pd / 'series').write_text(series)
    for m in made: say('wrote patches/' + m)
    if made:
        # engine/ already holds these changes: record that, so the next sync rebuilds it from the series instead of refusing
        if st: write_stamp(eng, root, head(eng), key=st.get('key'))
        say('Fill in Reason and Issue in the new headers, then `python3 tools/engine.py check`. `python3 tools/engine.py sync` '
            'rebuilds engine/ from the series (your commits stay on a backup branch).')
    return 0


# ---- status, check ----

def cmd_status(a):
    P.cmd_status(a)
    eng = engine_dir(a.root)
    reason = why_stale(a.root)
    if not reason: state = 'up to date'
    else: state = f'stale ({reason}): run `python3 tools/engine.py sync`'
    say(f'engine/: {state}. Pin: upstream {pin(a.root)[:12]}')
    if is_repo(eng):
        work = local_work(eng, read_stamp(eng))
        if work: say('engine/ work not in patches/: ' + '; '.join(work) + ' (`tools/engine.py export`)')
    return 0


def scratch_root(root):
    """The repository the scratch worktrees branch from: engine/, made (and fetched) on demand."""
    eng = engine_dir(root)
    ensure_repo(eng)
    return eng


def cmd_check(a):
    root = a.root
    install_hooks(root)
    patches = P.load(root)
    bad = []
    pd = root / 'patches'
    stray = sorted(f.name for f in pd.glob('*.patch') if f.name not in {p.file for p in patches})
    if stray: bad.append('patch files not in patches/series: ' + ', '.join(stray))
    old = stale_links(root)
    if old: bad.append('links into seedds/nxnative name another commit than .upstream-ref in ' + ', '.join(p.name for p in old) + ': run `python3 tools/engine.py relink`')
    eng = scratch_root(root)
    sha = pin(root)
    fetch(eng, sha)
    with P.Scratch(eng, sha, a.tmp) as s:
        # The chain rule reads the engine's AGENTS.md: the pin's own copy is close enough for the globs.
        for p in patches:
            for m in p.problems(): bad.append(f'{p.file}: {m}')
            gen = sorted(f for f in p.files if f in {o for o, _ in P.GENERATED})
            if gen: bad.append(f'{p.file}: touches generated files ({", ".join(gen)}); sync regenerates them, so a patch must not carry them')
            hits = P.chain_hits(p.files, s.dir)
            if hits: bad.append(f'{p.file}: touches the shader chain or the tool pins, which patches must not change ({", ".join(hits)}); see patches/README.md, Patch policy')
        if bad:
            say('patch queue: invalid\n  ' + '\n  '.join(bad)); return 1
        _, landed, conflicts = P.apply_series(s.dir, patches)
        if conflicts:
            p, msg = conflicts[0]
            say(f'patch queue: {p.file} does not apply to the pinned upstream {sha[:12]}\n  Reason: {p.reason}\n  Upstream-Status: {p.status}\n  {msg}'); return 1
        if landed:
            say('patch queue: patches that are empty on the pinned upstream (it has them already; run bump to drop them): ' + ', '.join(p.file for p in landed)); return 1
    say(f'patch queue: ok. {len(patches)} patches apply cleanly to upstream {sha[:12]}')
    return 0


# ---- links into the engine's documents ----

# The pages of this repository that link to the engine's documents and images (the engine is not committed here, #214): they point at
# seedds/nxnative at the pinned commit, and move with the pin.
LINK_PINNED = re.compile(r'(https://(?:github\.com/seedds/nxnative/(?:blob|tree)|raw\.githubusercontent\.com/seedds/nxnative))/([0-9a-f]{40})(?=/)')


def link_pages(root):
    pages = [p for p in sorted(root.glob('*.md')) if not p.name.startswith('CHANGELOG')]
    pages += [root / 'frontend' / 'README.md', root / 'frontend-qt' / 'README.md']
    return [p for p in pages if p.is_file()]


def stale_links(root):
    """The pages that link to seedds/nxnative at a commit other than the pin."""
    sha = pin(root)
    return [p for p in link_pages(root) if any(m.group(2) != sha for m in LINK_PINNED.finditer(p.read_text(encoding='utf-8')))]


def relink(root, sha=None):
    """Point every link into seedds/nxnative at the pin (or `sha`). Returns the pages it changed."""
    sha = sha or pin(root)
    changed = []
    for p in link_pages(root):
        text = p.read_text(encoding='utf-8')
        new = LINK_PINNED.sub(lambda m: f'{m.group(1)}/{sha}', text)
        if new != text:
            p.write_text(new, encoding='utf-8'); changed.append(p)
    return changed


def cmd_relink(a):
    changed = relink(a.root)
    say(f'links into seedds/nxnative now point at {pin(a.root)[:12]} in ' + (', '.join(p.name for p in changed) if changed else 'no page (already current)'))
    return 0


# ---- bump, refresh ----

def resolve_target(eng, to):
    if re.fullmatch(r'[0-9a-f]{40}', to): return to
    url = os.environ.get('NX2APPLE_ENGINE_URL') or UPSTREAM_URL
    r = git('ls-remote', url, to, cwd=eng, check=False)
    line = next((l for l in r.stdout.splitlines() if l.endswith((f'refs/heads/{to}', f'\t{to}'))), '')
    if not line: raise Fail(f'{to} is neither a full commit id nor a branch of {url}')
    return line.split()[0]


def cmd_bump(a):
    root = a.root
    patches = P.load(root)
    eng = scratch_root(root)
    old = pin(root)
    fetch(eng, old)                      # the old pin's blobs let `git am -3` merge three ways
    sha = resolve_target(eng, a.to)
    fetch(eng, sha)
    s = P.Scratch(eng, sha, a.tmp, keep=not a.dry)
    try:
        applied, landed, conflicts = P.apply_series(s.dir, patches, stop=not a.dry)
        P.report(old, sha, applied, landed, conflicts)
        if conflicts:
            if a.dry: s.close(); return 1
            p = conflicts[0][0]
            say(f'\nStopped at {p.file}. The conflict is left in {s.dir}:\n'
                f'  resolve it there, `git am --continue`, apply the rest with `git am -3 <patches>`, then run\n'
                f'  tools/engine.py refresh {s.dir} --to {sha[:12]}')
            return 1
        if a.dry: say('\n' + C.report(root, old, sha, eng)); return 0
        P.write_series(root, s.dir, applied, landed, patches)
        (root / '.upstream-ref').write_text(sha + '\n')
        s.close()
        pages = relink(root)
        say('wrote patches/ and .upstream-ref' + (f', and moved the links into seedds/nxnative to the new pin in {", ".join(p.name for p in pages)}' if pages else '') +
            '; `python3 tools/engine.py sync` builds engine/ on the new pin')
        say('\n' + C.report(root, old, sha, eng))   # last, under a heading: the sync workflow lifts it into the pull request
        return 0
    except BaseException:
        s.close(); raise


def cmd_candidates(a):
    root = a.root
    old = a.old or pin(root)
    sha = a.to if re.fullmatch(r'[0-9a-f]{40}', a.to) else resolve_target(scratch_root(root), a.to)
    say(C.report(root, old, sha, scratch_root(root)))
    return 0


def cmd_refresh(a):
    root, wt = a.root, Path(a.worktree)
    sha = a.to if re.fullmatch(r'[0-9a-f]{40}', a.to) else git('rev-parse', '--verify', f'{a.to}^{{commit}}', cwd=wt).stdout.strip()
    old = P.load(root)
    commits = git('rev-list', '--reverse', f'{sha}..HEAD', cwd=wt).stdout.split()
    if len(commits) > len(old): raise Fail('more commits than patches')
    by_subject = {p.subject: p for p in old}
    applied, used = [], set()
    for c in commits:
        subj = git('log', '-1', '--format=%s', c, cwd=wt).stdout.strip()
        p = by_subject.get(subj)
        if not p: raise Fail(f'commit "{subj}" matches no patch subject')
        applied.append((p, c)); used.add(p.file)
    landed = [p for p in old if p.file not in used]
    say('landed upstream, dropped: ' + (', '.join(p.name for p in landed) or 'none'))
    P.write_series(root, wt, applied, landed, old)
    (root / '.upstream-ref').write_text(sha + '\n')
    relink(root)
    say('wrote patches/ and .upstream-ref; `python3 tools/engine.py sync` builds engine/ on the new pin')
    return 0


def main(argv=None):
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument('--repo', default=str(REPO), help="the checkout (default: this script's)")
    ap.add_argument('--tmp', help='folder for scratch worktrees (default $PATCHES_TMP, else the system temporary folder)')
    ap.add_argument('-q', '--quiet', action='store_true', help='print nothing when there is nothing to do')
    sub = ap.add_subparsers(dest='cmd', required=True)
    s = sub.add_parser('sync'); s.add_argument('--force', action='store_true', help='rebuild even when up to date; local work goes to a backup branch'); s.set_defaults(f=cmd_sync)
    sub.add_parser('stale').set_defaults(f=cmd_stale)
    s = sub.add_parser('hook'); s.set_defaults(f=cmd_hook, force=False, quiet=True)
    s = sub.add_parser('export'); s.add_argument('rev', nargs='?', help='a commit or a range in engine/ (default: everything after the last sync)')
    s.add_argument('--status', default='pending'); s.add_argument('--reason', default='TODO: why the fork carries this'); s.add_argument('--issue', default='TODO: LNPANDY/NX2Apple#<n>'); s.set_defaults(f=cmd_export)
    s = sub.add_parser('tree'); s.add_argument('--out', required=True, help='the folder for the tree (emptied when it is an earlier tree)'); s.add_argument('--zip', help='also write the deterministic zip of the tree'); s.set_defaults(f=cmd_tree)
    sub.add_parser('status').set_defaults(f=cmd_status)
    sub.add_parser('check').set_defaults(f=cmd_check)
    s = sub.add_parser('bump'); s.add_argument('--to', required=True, help='the new upstream commit, or a branch such as main'); s.add_argument('--dry', action='store_true'); s.set_defaults(f=cmd_bump)
    s = sub.add_parser('candidates'); s.add_argument('--from', dest='old', help='the old pin (default: .upstream-ref)'); s.add_argument('--to', required=True, help='the new upstream commit, or a branch such as main'); s.set_defaults(f=cmd_candidates)
    sub.add_parser('relink').set_defaults(f=cmd_relink)
    s = sub.add_parser('refresh'); s.add_argument('worktree'); s.add_argument('--to', required=True); s.set_defaults(f=cmd_refresh)
    a = ap.parse_args(argv)
    a.root = Path(a.repo).resolve()
    try:
        return a.f(a)
    except Fail as e:
        print(f'engine.py: {e}', file=sys.stderr)
        return 2


if __name__ == '__main__':
    sys.exit(main())