Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Reuse Desktop candidates across unrelated component merges (#652)

Deecaa646aa4256
diff --git a/.github/workflows/release-desktop.yml b/.github/workflows/release-desktop.yml
index 8f9a4c7d9..4009b1051 100644
--- a/.github/workflows/release-desktop.yml
+++ b/.github/workflows/release-desktop.yml
@@ -587,7 +587,7 @@ jobs:
# Same build/signing path and production updater settings. Only the storage
# destination differs. A successful receipt binds six verified generations to
- # this complete source tree, version, workflow run and attempt.
+ # its full producer source tree, Desktop build inputs, version, run and attempt.
candidate:
needs: [version, build-macos, build-linux]
if: ${{ inputs.prepare_only }}
diff --git a/AGENTS.md b/AGENTS.md
index 15b18e356..9896acfbc 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -30,4 +30,5 @@ validation and shipping, use [docs/validation-and-release.md](docs/validation-an
from the final pushed PR branch alongside validation and review. It prepares
verified packages without publishing; merge and release only after checks pass.
Avoid starting candidates while implementation is still changing. The release
- automatically reuses a matching source tree/version and otherwise builds normally.
+ automatically reuses matching Desktop build inputs/version and otherwise builds
+ normally; unrelated CLI, firmware or documentation merges do not force a rebuild.
diff --git a/desktop/RELEASE.md b/desktop/RELEASE.md
index b85d4f858..82e75e62e 100644
--- a/desktop/RELEASE.md
+++ b/desktop/RELEASE.md
@@ -29,8 +29,9 @@ branch, while the required tests and review run. It selects the next version by
as a release, builds/signs/notarizes all platforms in CI, and verifies all six downloads at an
isolated candidate path. It creates neither a release tag nor a product manifest/Release page.
After checks pass, merge and release that same version normally. CI reuses the candidate only
-when the complete source tree and version match; a clean squash is supported. Changed code,
-build inputs or version require a new build. Candidate packaging does not replace application
+when the Desktop build inputs and version match; unrelated component merges and a clean
+squash are supported. Changed build inputs or version require a new build.
+Candidate packaging does not replace application
tests or authorize publication. See the [validation guide](../docs/validation-and-release.md#package-desktop-while-final-checks-run)
for identity checks, bounded waiting, expiration and disposable promotion validation.
diff --git a/docs/validation-and-release.md b/docs/validation-and-release.md
index 6873996a5..68b4ebf9d 100644
--- a/docs/validation-and-release.md
+++ b/docs/validation-and-release.md
@@ -87,9 +87,21 @@ candidate prefix, verifies all six downloads, and retains a small receipt in
GitHub Actions. It neither tags nor updates the product manifest or Release page.
After validation and review pass, merge and release that version normally. The
-workflow automatically looks for a candidate with the same version and complete
-Git source tree. A clean squash preserves that tree; any source, dependency,
-toolchain-pin or workflow change requires another build. The workflow checks the
+workflow automatically looks for a candidate with the same version and Desktop
+build inputs. It compares Git objects for the whole `desktop/` and `scripts/`
+trees, `.github/actions/`, the release workflow (including its SDK/runner pins),
+`mobile/pubspec.lock` when present, and root `.gitattributes`, `.gitignore` and
+`.gitmodules` files. File contents, additions, removals and executable modes are
+covered. Extend this input contract before a build starts reading another component;
+the process suite checks that it covers every native builder's sparse checkout.
+
+Unrelated CLI, firmware and documentation merges, or a clean squash, can preserve
+those inputs even when the full repository tree changes. Any Desktop source, asset,
+dependency, native code, packaging helper, toolchain-pin or workflow change requires
+another build. The receipt still identifies the producer's actual commit/full tree;
+reuse records both source identities and the matching input fingerprint. GitHub's
+immutable tree objects must independently confirm the producer's inputs. Truncated
+or malformed responses cannot authorize reuse. The workflow checks the
producer run's successful completion, repository, workflow, source, attempt and
immutable receipt digest. Promotion copies only the exact object generations
whose sizes and SHA-256 hashes were verified, with atomic no-overwrite conditions.
diff --git a/scripts/desktop-release-candidate.py b/scripts/desktop-release-candidate.py
index 478b0e14a..ad150c26f 100644
--- a/scripts/desktop-release-candidate.py
+++ b/scripts/desktop-release-candidate.py
@@ -39,6 +39,20 @@ FILES = dict(zip(verification.KEYS, (
PREFIX = r'harness/desktop-candidates/([1-9][0-9]*)-([1-9][0-9]*)-[0-9a-f]{32}'
RETENTION = timedelta(days=7)
MAX_AGE_SECONDS = 1200
+# Whole component/helper trees include new files, removals and executable modes.
+# These cover every native sparse checkout, the workflow/toolchain pin, and the
+# root Git files that can affect materialization. Extend this contract whenever
+# a build starts reading another component; the checkout coverage test guards it.
+BUILD_INPUTS = {
+ 'desktop': 'tree',
+ 'scripts': 'tree',
+ '.github/actions': 'tree',
+ WORKFLOW: 'blob',
+ 'mobile/pubspec.lock': None, # included when present in the shared pub-cache key
+ '.gitattributes': None,
+ '.gitignore': None,
+ '.gitmodules': None,
+}
class CandidatePendingError(RuntimeError):
@@ -60,6 +74,62 @@ def git(value):
return subprocess.check_output(['git', 'rev-parse', value], text=True, timeout=10).strip()
+def local_tree(sha, cwd=None):
+ records = subprocess.check_output(['git', 'ls-tree', '--full-tree', '-z', sha], cwd=cwd, text=True, timeout=10)
+ entries = []
+ for record in records.split('\0'):
+ if record:
+ metadata, path = record.split('\t', 1)
+ mode, kind, object_sha = metadata.split()
+ entries.append(dict(path=path, mode=mode, type=kind, sha=object_sha))
+ return dict(sha=sha, tree=entries, truncated=False)
+
+
+def build_inputs(tree, read_tree):
+ """Read complete Git objects, including optional-file absence, without blobs."""
+ cache = {}
+ def entries(sha):
+ require(isinstance(sha, str) and re.fullmatch(r'[0-9a-f]{40}', sha), 'invalid input tree')
+ if sha not in cache:
+ data = read_tree(sha)
+ require(isinstance(data, dict) and data.get('sha') == sha and data.get('truncated') is False
+ and isinstance(data.get('tree'), list), 'incomplete input tree response')
+ indexed = {}
+ for entry in data['tree']:
+ require(isinstance(entry, dict), 'invalid input tree entry')
+ path = entry['path']
+ require(isinstance(path, str) and path and '/' not in path and path not in indexed,
+ 'invalid or duplicate input tree entry')
+ indexed[path] = entry
+ cache[sha] = indexed
+ return cache[sha]
+
+ result = {}
+ for path, required_type in BUILD_INPUTS.items():
+ parent = tree
+ parts = path.split('/')
+ for index, name in enumerate(parts):
+ entry = entries(parent).get(name)
+ if entry is None:
+ require(required_type is None, f'missing build input: {path}')
+ result[path] = None
+ break
+ kind = 'tree' if index < len(parts) - 1 else required_type or 'blob'
+ require(entry.get('type') == kind and entry.get('mode') in
+ (('040000',) if kind == 'tree' else ('100644', '100755')),
+ f'unsupported build input type: {path}')
+ require(isinstance(entry.get('sha'), str) and re.fullmatch(r'[0-9a-f]{40}', entry['sha']),
+ f'invalid build input object: {path}')
+ parent = entry['sha']
+ if index == len(parts) - 1:
+ result[path] = {key: entry[key] for key in ('mode', 'type', 'sha')}
+ return result
+
+
+def input_fingerprint(inputs):
+ return hashlib.sha256(json.dumps(inputs, sort_keys=True, separators=(',', ':')).encode()).hexdigest()
+
+
def context():
repository = os.environ['GITHUB_REPOSITORY']
run, attempt = (int(os.environ[key]) for key in ('GITHUB_RUN_ID', 'GITHUB_RUN_ATTEMPT'))
@@ -67,7 +137,9 @@ def context():
require(run > 0 and attempt > 0, 'invalid run identity')
sha = git('HEAD')
require(sha == os.environ['GITHUB_SHA'], 'checkout differs from workflow source')
- return dict(repository=repository, run_id=run, run_attempt=attempt, source_sha=sha, source_tree=git('HEAD^{tree}'))
+ tree = git('HEAD^{tree}')
+ return dict(repository=repository, run_id=run, run_attempt=attempt, source_sha=sha, source_tree=tree,
+ build_inputs=build_inputs(tree, local_tree))
def check_prefix(prefix, run=None, attempt=None):
@@ -111,7 +183,7 @@ def seal(version, bucket, prefix, parts, identity):
with ThreadPoolExecutor(max_workers=3) as pool:
results = list(pool.map(lambda item: verification.verify_artifact(item, 90), pinned.items()))
require(all(item['status'] == 'passed' for item in results), 'candidate downloads failed verification')
- return dict(schema=1, status='passed', version=version, bucket=bucket, prefix=prefix,
+ return dict(schema=2, status='passed', version=version, bucket=bucket, prefix=prefix,
created_at=verification.utc_now(), **identity, artifacts=pinned)
@@ -124,7 +196,7 @@ def check_run(run, repository, version):
'candidate has no run/attempt identity')
-def read_receipt(artifact, archive, run, repository, version, tree, bucket, now):
+def read_receipt(artifact, archive, run, repository, version, tree, inputs, bucket, now):
name = f'desktop-release-candidate-{run["run_attempt"]}'
provenance = artifact.get('workflow_run', {})
require(artifact.get('name') == name and artifact.get('expired') is False
@@ -136,9 +208,9 @@ def read_receipt(artifact, archive, run, repository, version, tree, bucket, now)
require(bundle.namelist() == ['desktop-release-candidate.json']
and bundle.infolist()[0].file_size <= 65536, 'unexpected candidate archive contents')
receipt = json.loads(bundle.read('desktop-release-candidate.json'))
- require(isinstance(receipt, dict) and receipt.get('schema') == 1 and receipt.get('status') == 'passed', 'candidate did not pass')
+ require(isinstance(receipt, dict) and receipt.get('schema') == 2 and receipt.get('status') == 'passed', 'candidate did not pass')
expected = dict(repository=repository, run_id=run['id'], run_attempt=run['run_attempt'],
- source_sha=run['head_sha'], source_tree=tree, version=version, bucket=bucket)
+ source_sha=run['head_sha'], source_tree=tree, build_inputs=inputs, version=version, bucket=bucket)
require(all(receipt.get(key) == value for key, value in expected.items()), 'candidate source/version/run differs from release')
created = timestamp(receipt['created_at'])
require(timestamp(run['created_at']) <= created <= now and now - created < RETENTION, 'candidate is expired or has invalid timestamps')
@@ -167,10 +239,17 @@ def check_jobs(run, jobs):
require(all(step.get('conclusion') in {'success', 'skipped'} for step in job.get('steps', [])), 'candidate contains a failed step')
-def find_candidate(client, version, tree, bucket, current_run, wait_seconds=MAX_AGE_SECONDS):
+def find_candidate(client, version, inputs, bucket, current_run, wait_seconds=MAX_AGE_SECONDS):
"""Look at a bounded recent set. Never wait past a candidate's build budget."""
now = datetime.now(timezone.utc)
runs = client.api('actions/workflows/release-desktop.yml/runs?event=workflow_dispatch&per_page=30')['workflow_runs']
+ # Unrelated commits often share these small parent trees. Keep immutable API
+ # responses for this lookup, without downloading whole recursive file lists.
+ trees = {}
+ def read_tree(sha):
+ if sha not in trees:
+ trees[sha] = client.api(f'git/trees/{sha}')
+ return trees[sha]
# A completed candidate with identical bytes is immediately usable even if
# someone has unnecessarily started another build of the same tree/version.
runs.sort(key=lambda run: (run.get('conclusion') != 'success', -run['id']))
@@ -183,10 +262,11 @@ def find_candidate(client, version, tree, bucket, current_run, wait_seconds=MAX_
if now - timestamp(run['created_at']) >= RETENTION:
continue
commit = client.api(f'git/commits/{run["head_sha"]}')
- if commit.get('tree', {}).get('sha') != tree:
+ producer_tree = commit.get('tree', {}).get('sha')
+ if build_inputs(producer_tree, read_tree) != inputs:
continue
- # A clean squash changes the commit but preserves this complete tree,
- # including the SDK pin, actions, packaging scripts and workflow itself.
+ # Producer provenance still names its actual full tree. Only unrelated
+ # components may differ from the release; every build input must match.
end = time.monotonic() + max(0, min(wait_seconds, MAX_AGE_SECONDS - (now - timestamp(run['created_at'])).total_seconds()))
while run['status'] != 'completed' and time.monotonic() < end:
print(f'Waiting for matching candidate {run["id"]} ({run["status"]})', flush=True)
@@ -213,7 +293,8 @@ def find_candidate(client, version, tree, bucket, current_run, wait_seconds=MAX_
require(len(selected) == 1 and selected[0].get('size_in_bytes', 65537) <= 65536, 'candidate receipt is missing or too large')
artifact = selected[0]
archive = client.command('api', f'repos/{client.repository}/actions/artifacts/{artifact["id"]}/zip', binary=True)
- receipt = read_receipt(artifact, archive, run, client.repository, version, tree, bucket, datetime.now(timezone.utc))
+ receipt = read_receipt(artifact, archive, run, client.repository, version, producer_tree,
+ inputs, bucket, datetime.now(timezone.utc))
latest = client.api(f'actions/runs/{run["id"]}')
require(all(latest.get(key) == run.get(key) for key in ('id', 'head_sha', 'run_attempt', 'status', 'conclusion')),
'candidate run changed while collecting it')
@@ -325,7 +406,7 @@ def main(argv=None):
started = time.monotonic()
client = Client(identity['repository'], started + args.wait_seconds + 90)
try:
- receipt = find_candidate(client, args.version, identity['source_tree'], args.bucket, identity['run_id'], args.wait_seconds)
+ receipt = find_candidate(client, args.version, identity['build_inputs'], args.bucket, identity['run_id'], args.wait_seconds)
if receipt:
check_sources(receipt)
except CandidatePendingError:
@@ -342,7 +423,9 @@ def main(argv=None):
promote(receipt, args.destination, args.scratch)
output('reused', 'true')
output('candidate_prefix', receipt['prefix'])
- result = dict(status='passed', version=args.version, source_tree=identity['source_tree'],
+ result = dict(status='passed', version=args.version, source_sha=identity['source_sha'], source_tree=identity['source_tree'],
+ candidate_source_sha=receipt['source_sha'], candidate_source_tree=receipt['source_tree'],
+ build_inputs_sha256=input_fingerprint(identity['build_inputs']),
candidate_run=receipt['run_id'], candidate_attempt=receipt['run_attempt'],
duration_seconds=round(time.monotonic() - started, 3))
Path('desktop-candidate-reuse.json').write_text(json.dumps(result, indent=2) + '\n')
diff --git a/scripts/tests/test_desktop_release_candidate.py b/scripts/tests/test_desktop_release_candidate.py
index 6a4ec0e8e..296aab1a0 100644
--- a/scripts/tests/test_desktop_release_candidate.py
+++ b/scripts/tests/test_desktop_release_candidate.py
@@ -8,6 +8,8 @@ import json
import os
from pathlib import Path
import tempfile
+import subprocess
+import re
import unittest
from unittest.mock import patch
import zipfile
@@ -26,14 +28,28 @@ VERSION = '1.2.55'
NOW = datetime.now(timezone.utc)
+def tree_entry(path, number, kind='tree'):
+ return dict(path=path, sha=str(number) * 40, type=kind, mode='040000' if kind == 'tree' else '100644')
+
+
+TREES = {
+ TREE: dict(sha=TREE, truncated=False, tree=[tree_entry('desktop', 1), tree_entry('scripts', 2),
+ tree_entry('.github', 3), tree_entry('mobile', 4)]),
+ '3' * 40: dict(sha='3' * 40, truncated=False, tree=[tree_entry('actions', 5), tree_entry('workflows', 6)]),
+ '6' * 40: dict(sha='6' * 40, truncated=False, tree=[tree_entry('release-desktop.yml', 7, 'blob')]),
+ '4' * 40: dict(sha='4' * 40, truncated=False, tree=[tree_entry('pubspec.lock', 8, 'blob')]),
+}
+INPUTS = candidate.build_inputs(TREE, lambda sha: TREES[sha])
+
+
def make_receipt():
entries = {}
for index, (key, filename) in enumerate(candidate.FILES.items(), 1):
payload = key.encode()
entries[key] = dict(version=VERSION, size=len(payload), sha256=hashlib.sha256(payload).hexdigest(),
generation=str(index), url=f'https://storage.googleapis.com/{BUCKET}/{PREFIX}/{filename}?generation={index}')
- return dict(schema=1, status='passed', version=VERSION, bucket=BUCKET, prefix=PREFIX, repository=REPOSITORY,
- run_id=123, run_attempt=1, source_sha=SHA, source_tree=TREE,
+ return dict(schema=2, status='passed', version=VERSION, bucket=BUCKET, prefix=PREFIX, repository=REPOSITORY,
+ run_id=123, run_attempt=1, source_sha=SHA, source_tree=TREE, build_inputs=copy.deepcopy(INPUTS),
created_at=(NOW - timedelta(seconds=10)).isoformat(), artifacts=entries)
@@ -68,6 +84,7 @@ class FakeGithub:
self.receipt = make_receipt()
self.artifact, self.archive = archive_receipt(self.receipt)
self.tree = TREE
+ self.trees = copy.deepcopy(TREES)
self.jobs = make_jobs()
self.calls = []
self.run_reads = 0
@@ -78,6 +95,8 @@ class FakeGithub:
return dict(workflow_runs=[copy.deepcopy(self.run)])
if path.startswith('git/commits/'):
return dict(tree=dict(sha=self.tree))
+ if path.startswith('git/trees/'):
+ return copy.deepcopy(self.trees[path.rsplit('/', 1)[1]])
if '/artifacts?' in path:
return dict(artifacts=[copy.deepcopy(self.artifact)])
if '/jobs?' in path:
@@ -98,7 +117,7 @@ class CandidateIdentityTests(unittest.TestCase):
artifact, archive = archive_receipt(receipt)
if artifact_change:
artifact_change(artifact)
- args = dict(run=make_run(), repository=REPOSITORY, version=VERSION, tree=TREE, bucket=BUCKET, now=NOW)
+ args = dict(run=make_run(), repository=REPOSITORY, version=VERSION, tree=TREE, inputs=INPUTS, bucket=BUCKET, now=NOW)
args.update(overrides)
return candidate.read_receipt(artifact, archive, **args)
@@ -106,9 +125,23 @@ class CandidateIdentityTests(unittest.TestCase):
receipt = self.read()
self.assertEqual(receipt['source_sha'], SHA)
self.assertEqual(receipt['source_tree'], TREE)
- # The release commit may differ; its complete tree is the contract.
+ # The producer commit/full tree remain authenticated; build inputs match.
client = FakeGithub()
- self.assertEqual(candidate.find_candidate(client, VERSION, TREE, BUCKET, 456), receipt)
+ self.assertEqual(candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456), receipt)
+
+ def test_unrelated_release_tree_can_reuse_but_cannot_relabel_the_producer(self):
+ client = FakeGithub()
+ release_tree = 'd' * 40
+ client.trees[release_tree] = copy.deepcopy(client.trees[TREE])
+ client.trees[release_tree]['sha'] = release_tree
+ client.trees[release_tree]['tree'].append(tree_entry('cli', 9))
+ release_inputs = candidate.build_inputs(release_tree, lambda sha: client.trees[sha])
+ receipt = candidate.find_candidate(client, VERSION, release_inputs, BUCKET, 456)
+ self.assertEqual(receipt['source_tree'], TREE)
+ self.assertEqual(receipt['source_sha'], SHA)
+ receipt['source_tree'] = release_tree
+ with self.assertRaisesRegex(ValueError, 'source/version/run'):
+ self.read(receipt)
def test_source_version_and_repository_changes_cannot_reuse(self):
for field, value in [('tree', 'd' * 40), ('version', '1.2.56'), ('repository', 'other/project'), ('bucket', 'other-bucket')]:
@@ -117,8 +150,10 @@ class CandidateIdentityTests(unittest.TestCase):
def test_receipt_requires_all_six_verified_objects_with_exact_paths(self):
mutations = [
- lambda r: r.update(status='failed'), lambda r: r.update(schema=2),
+ lambda r: r.update(status='failed'), lambda r: r.update(schema=1),
lambda r: r.update(run_attempt=2), lambda r: r.update(source_sha='e' * 40),
+ lambda r: r.pop('build_inputs'), lambda r: r['build_inputs'].pop('desktop'),
+ lambda r: r['build_inputs']['desktop'].update(sha='e' * 40),
lambda r: r.update(prefix=PREFIX.replace('/123-', '/124-')),
lambda r: r['artifacts'].pop('desktop-linux-arm64'),
lambda r: r['artifacts'].update(extra=r['artifacts']['desktop-macos']),
@@ -136,6 +171,27 @@ class CandidateIdentityTests(unittest.TestCase):
change(receipt)
self.read(receipt)
+ def test_incomplete_or_forged_input_tree_response_cannot_reuse(self):
+ for change in ['truncated', 'missing-root', 'wrong-sha', 'duplicate', 'symlink', 'bad-child-sha']:
+ with self.subTest(change=change), self.assertRaises(ValueError):
+ client = FakeGithub()
+ data = client.trees[TREE]
+ if change == 'truncated': data['truncated'] = True
+ if change == 'missing-root': data['tree'].pop(0)
+ if change == 'wrong-sha': data['sha'] = 'e' * 40
+ if change == 'duplicate': data['tree'].append(copy.deepcopy(data['tree'][0]))
+ if change == 'symlink': data['tree'][0].update(mode='120000', type='blob')
+ if change == 'bad-child-sha': data['tree'][2]['sha'] = '../other/ref'
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
+
+ def test_tree_lookup_is_nonrecursive_and_reads_shared_parents_once(self):
+ client = FakeGithub()
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
+ paths = [path for path in client.calls if path.startswith('git/trees/')]
+ self.assertEqual(len(paths), 4)
+ self.assertEqual(len(paths), len(set(paths)))
+ self.assertTrue(all('?' not in path for path in paths))
+
def test_github_artifact_provenance_and_digest_are_required(self):
for field, value in [('name', 'unrelated'), ('expired', True), ('digest', 'sha256:' + 'f' * 64),
('workflow_run', dict(id=456, head_sha=SHA)), ('workflow_run', dict(id=123, head_sha='f' * 40))]:
@@ -143,15 +199,15 @@ class CandidateIdentityTests(unittest.TestCase):
self.read(artifact_change=lambda a: a.update({field: value}))
def test_no_candidate_for_failed_incomplete_changed_or_old_build(self):
- for change in ['failed', 'different-tree', 'different-version', 'old']:
+ for change in ['failed', 'different-inputs', 'different-version', 'old']:
with self.subTest(change=change):
client = FakeGithub()
if change == 'failed': client.run['conclusion'] = 'failure'
- if change == 'different-tree': client.tree = 'd' * 40
+ if change == 'different-inputs': client.trees[TREE]['tree'][0]['sha'] = 'd' * 40
if change == 'different-version': client.run['display_title'] = 'Desktop candidate 1.2.56'
if change == 'old': client.run['created_at'] = (NOW - timedelta(days=8)).isoformat()
with patch.object(candidate.time, 'sleep') as sleep:
- self.assertIsNone(candidate.find_candidate(client, VERSION, TREE, BUCKET, 456, wait_seconds=0))
+ self.assertIsNone(candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456, wait_seconds=0))
sleep.assert_not_called()
def test_builds_and_verification_cannot_be_missing_skipped_or_failed(self):
@@ -162,7 +218,7 @@ class CandidateIdentityTests(unittest.TestCase):
if change == 'skipped': client.jobs[2]['conclusion'] = 'skipped'
if change == 'failed-step': client.jobs[2]['steps'][0]['conclusion'] = 'failure'
if change == 'wrong-source': client.jobs[2]['head_sha'] = 'f' * 40
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
def test_wait_follows_one_live_run_without_starting_another(self):
client = FakeGithub()
@@ -170,7 +226,7 @@ class CandidateIdentityTests(unittest.TestCase):
def finish(seconds):
client.run.update(status='completed', conclusion='success')
with patch.object(candidate.time, 'sleep', side_effect=finish) as sleep:
- result = candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ result = candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
self.assertEqual(result['run_id'], 123)
self.assertEqual(sleep.call_count, 1)
self.assertTrue(all(path.startswith(('actions/', 'git/')) for path in client.calls))
@@ -181,7 +237,7 @@ class CandidateIdentityTests(unittest.TestCase):
created_at=(NOW - timedelta(seconds=candidate.MAX_AGE_SECONDS + 1)).isoformat())
with patch.object(candidate.time, 'sleep') as sleep:
with self.assertRaisesRegex(candidate.CandidatePendingError, 'actions/runs/123'):
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
sleep.assert_not_called()
def test_pending_deadline_stops_release_without_dispatching_a_replacement(self):
@@ -189,10 +245,10 @@ class CandidateIdentityTests(unittest.TestCase):
client.run.update(status='queued', conclusion=None)
with patch.object(candidate.time, 'sleep') as sleep:
with self.assertRaises(candidate.CandidatePendingError):
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456, wait_seconds=0)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456, wait_seconds=0)
sleep.assert_not_called()
receipt = make_receipt()
- identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree']}
+ identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree', 'build_inputs']}
with patch.object(candidate, 'context', return_value=identity), \
patch.object(candidate, 'find_candidate', side_effect=candidate.CandidatePendingError('still live')), \
patch.object(candidate, 'output') as output, patch.object(candidate, 'promote') as promote:
@@ -211,7 +267,7 @@ class CandidateIdentityTests(unittest.TestCase):
return data
client.api = api
with self.assertRaisesRegex(ValueError, 'changed while collecting'):
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
def test_failed_observation_of_a_live_build_never_triggers_another_build(self):
client = FakeGithub()
@@ -224,7 +280,7 @@ class CandidateIdentityTests(unittest.TestCase):
client.api = api
with patch.object(candidate.time, 'sleep'):
with self.assertRaisesRegex(candidate.CandidatePendingError, 'Cannot confirm'):
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
def test_other_workflow_or_repository_cannot_produce_a_candidate(self):
for field, value in [('path', '.github/workflows/desktop-internal-build.yml'), ('event', 'pull_request'),
@@ -232,11 +288,11 @@ class CandidateIdentityTests(unittest.TestCase):
with self.subTest(field=field), self.assertRaises(ValueError):
client = FakeGithub()
client.run[field] = value
- candidate.find_candidate(client, VERSION, TREE, BUCKET, 456)
+ candidate.find_candidate(client, VERSION, INPUTS, BUCKET, 456)
def test_copy_failure_is_fatal_and_never_becomes_a_fresh_build(self):
receipt = make_receipt()
- identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree']}
+ identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree', 'build_inputs']}
with patch.object(candidate, 'context', return_value=identity), \
patch.object(candidate, 'find_candidate', return_value=receipt), \
patch.object(candidate, 'check_sources'), \
@@ -248,7 +304,7 @@ class CandidateIdentityTests(unittest.TestCase):
def test_lookup_failure_falls_back_but_disposable_check_requires_reuse(self):
receipt = make_receipt()
- identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree']}
+ identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree', 'build_inputs']}
with patch.object(candidate, 'context', return_value=identity), \
patch.object(candidate, 'find_candidate', side_effect=RuntimeError('API unavailable')), \
patch.object(candidate, 'output') as output, patch.object(candidate, 'promote') as promote:
@@ -259,6 +315,100 @@ class CandidateIdentityTests(unittest.TestCase):
promote.assert_not_called()
+class BuildInputTests(unittest.TestCase):
+ def setUp(self):
+ self.temp = tempfile.TemporaryDirectory()
+ self.addCleanup(self.temp.cleanup)
+ self.root = Path(self.temp.name)
+ self.git('init', '-q')
+ self.files = {
+ 'desktop/lib/main.dart': 'application', 'desktop/assets/icon.png': 'asset',
+ 'desktop/pubspec.lock': 'dependencies', 'desktop/third_party/xterm/lib/terminal.dart': 'vendor',
+ 'desktop/macos/Runner/native.swift': 'native', 'desktop/scripts/upload-desktop.sh': 'package',
+ 'scripts/validate-change.py': 'shared helper',
+ '.github/actions/desktop-flutter/action.yml': 'setup',
+ candidate.WORKFLOW: 'toolchain and build flags',
+ 'mobile/pubspec.lock': 'shared cache identity', '.gitattributes': '*.png binary\n',
+ '.gitignore': 'build/\n', 'cli/main.ts': 'CLI', 'devices/firmware.c': 'firmware',
+ 'docs/guide.md': 'documentation',
+ }
+ for path, content in self.files.items():
+ file = self.root / path
+ file.parent.mkdir(parents=True, exist_ok=True)
+ file.write_text(content)
+ self.commit()
+ self.original = self.inputs()
+ self.original_tree = self.git('rev-parse', 'HEAD^{tree}')
+
+ def git(self, *args):
+ return subprocess.check_output(['git', '-c', 'user.name=Fixture', '-c', '[email protected]',
+ '-c', 'commit.gpgsign=false', '-c', 'core.hooksPath=/dev/null', *args],
+ cwd=self.root, text=True, stderr=subprocess.STDOUT, timeout=10).strip()
+
+ def commit(self):
+ self.git('add', '-A')
+ self.git('commit', '-qm', 'fixture')
+
+ def inputs(self):
+ return candidate.build_inputs(self.git('rev-parse', 'HEAD^{tree}'),
+ lambda sha: candidate.local_tree(sha, cwd=self.root))
+
+ def test_unrelated_components_change_the_tree_without_invalidating_inputs(self):
+ for path in ['cli/main.ts', 'devices/firmware.c', 'docs/guide.md']:
+ (self.root / path).write_text('unrelated new behavior')
+ self.commit()
+ self.assertNotEqual(self.git('rev-parse', 'HEAD^{tree}'), self.original_tree)
+ self.assertEqual(self.inputs(), self.original)
+
+ def test_every_source_dependency_toolchain_and_packaging_scope_invalidates(self):
+ for path in self.files:
+ if path.startswith(('cli/', 'devices/', 'docs/')):
+ continue
+ with self.subTest(path=path):
+ before = self.inputs()
+ (self.root / path).write_text('changed input')
+ self.commit()
+ self.assertNotEqual(self.inputs(), before)
+
+ def test_additions_deletions_executable_modes_and_optional_files_invalidate(self):
+ edits = [
+ lambda: (self.root / 'desktop/lib/new.dart').write_text('new source'),
+ lambda: (self.root / 'desktop/assets/icon.png').unlink(),
+ lambda: (self.root / 'scripts/validate-change.py').chmod(0o755),
+ lambda: (self.root / '.gitmodules').write_text('new submodule settings'),
+ lambda: (self.root / 'mobile/pubspec.lock').unlink(),
+ lambda: (self.root / '.gitattributes').unlink(),
+ ]
+ for index, edit in enumerate(edits):
+ with self.subTest(index=index):
+ before = self.inputs()
+ edit()
+ self.commit()
+ self.assertNotEqual(self.inputs(), before)
+
+ def test_sparse_checkout_preserves_identity_without_materializing_other_components(self):
+ self.git('sparse-checkout', 'set', '--no-cone', '/scripts/validate-change.py')
+ self.assertFalse((self.root / 'desktop').exists())
+ self.assertEqual(self.inputs(), self.original)
+ self.assertEqual(self.git('status', '--porcelain'), '')
+
+ def test_native_checkout_inputs_are_all_covered_by_the_reuse_contract(self):
+ for filename, jobs in [('release-desktop.yml', ['build-macos', 'build-linux']),
+ ('desktop-internal-build.yml', ['build-macos'])]:
+ workflow = (ROOT / '.github/workflows' / filename).read_text()
+ for job in jobs:
+ with self.subTest(workflow=filename, job=job):
+ body = re.search(rf'^ {job}:\n(.*?)(?=^ [\w-]+:|\Z)', workflow, re.M | re.S)[1]
+ checkout = re.search(r'^ sparse-checkout: \|\n((?: .+\n)+)', body, re.M)
+ self.assertIsNotNone(checkout, 'native checkout must declare its complete inputs')
+ for pattern in checkout[1].splitlines():
+ path = pattern.strip().strip('/')
+ self.assertRegex(path, r'^[A-Za-z0-9_./-]+$')
+ self.assertTrue(any(path == scope or (kind == 'tree' and path.startswith(scope + '/'))
+ for scope, kind in candidate.BUILD_INPUTS.items()),
+ f'{path} is checked out but absent from the candidate input contract')
+
+
class CandidatePromotionTests(unittest.TestCase):
def test_copies_pinned_bytes_and_preserves_canonical_manifest_contract(self):
receipt = make_receipt()
@@ -314,7 +464,7 @@ class CandidatePromotionTests(unittest.TestCase):
def test_sealing_rejects_missing_platform_or_wrong_source_url(self):
receipt = make_receipt()
- identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree']}
+ identity = {key: receipt[key] for key in ['repository', 'run_id', 'run_attempt', 'source_sha', 'source_tree', 'build_inputs']}
with tempfile.TemporaryDirectory() as folder:
path = Path(folder)
for name, keys in candidate.publisher.PARTS.items():