Skip to content

Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts.

Preflight console assets before managed installed CLI acceptance (#198)

澄潭39184cf5b23f
diff --git a/docs/code-map/testing.md b/docs/code-map/testing.md
index 619674ef..15eff29c 100644
--- a/docs/code-map/testing.md
+++ b/docs/code-map/testing.md
@@ -6,6 +6,10 @@
[Execution locations](../validation-routing.md) owns caller HOME routing,
per-origin SSH workbenches, shared compiler capacity and separate candidate/driver
revision identities. A runner setup failure is not a product test result.
+The [console artifact owner](../../scripts/console-assets.py) checks selected SSH
+installed-package prerequisites before Rust execution and verifies the same-candidate
+frontend from an execution-host staging directory under the runner's checkout lease.
+Standard file tools supply that directory; the runner does not transport frontend bytes.
Windows development execution is owned by [windows-dev.py](../../scripts/windows-dev.py)
and its [native executor](../../scripts/windows-dev-host.py), reached through
`validation.py windows`. HOME configuration owns machine/session/client locations;
diff --git a/docs/validation-routing.md b/docs/validation-routing.md
index dec980d8..7ef35b87 100644
--- a/docs/validation-routing.md
+++ b/docs/validation-routing.md
@@ -569,12 +569,56 @@ explicit rollback path and is never run automatically after a DAG failure. Other
selectors, profiles and filters use the ordinary runner. A bridge over the old
three-slot locks keeps legacy and eight-slot workers mutually exclusive.
-The installed Linux package scenario requires the same candidate's production frontend
-at `apps/desktop/dist`. Build it through the configured frontend target, then stage those
-bytes into the inactive managed backend checkout under its checkout lease before running
-`installed_standalone_cli_completes_the_headless_management_loop`. Do not replace them with
-a dummy page or install a second frontend toolchain on the backend. Hosted full CI uses
-its frontend artifact handoff for this prerequisite.
+The installed Linux management, Quickstart and model-feed scenarios require the same
+candidate's production frontend at `apps/desktop/dist`. The SSH backend inspects the
+Cargo/libtest selection and checks this prerequisite before Cargo or sccache starts.
+Missing receipts, another source SHA, missing/extra files and changed hashes block
+the run with `blocker=console_assets`, a null process exit and preparation commands
+in `result.console_assets.preparation`. Pure library checks, compilation, listing,
+other targets and filters excluding these cases do not require console assets.
+
+Use the exact clean committed candidate on the configured frontend target:
+
+```sh
+python3 scripts/validation.py frontend ci
+python3 scripts/validation.py frontend build --console-assets
+```
+
+Copy the resulting dist, including its receipt, with standard file tools to a
+separate directory on the configured backend host. For a local frontend target,
+using your configured SSH alias and a fresh staging directory:
+
+```sh
+backend_host=backend
+staging_dir=.cache/hiroute/console-assets/CANDIDATE_SHA-BLOCKED_RUN_ID
+ssh "$backend_host" "mkdir -p '$staging_dir'"
+scp -r apps/desktop/dist "$backend_host:$staging_dir/"
+python3 scripts/validation.py --frontend-dist "~/$staging_dir/dist" backend run \
+ --ref refs/heads/codex/my-change --sha CANDIDATE_SHA \
+ --reuse-checkout BLOCKED_RUN_ID --retention always \
+ --plan my-change --phase final -- \
+ cargo test --locked -p hiroute-daemon --test publication_process installed_ -- --nocapture
+```
+
+`frontend build --console-assets` records the successful build's source SHA and every
+output hash. Ordinary `frontend build` keeps its development behavior without a receipt.
+For SSH backend runs, `--frontend-dist` is an **execution-host** absolute or `~/`
+directory, including that receipt. With a remote frontend target, copy its output
+from that host to the backend staging directory. The runner reads those files,
+verifies source and hashes under the existing checkout lease, and stages a verified
+snapshot into the managed checkout. It does not transfer frontend bytes or require
+a backend frontend toolchain. Do not copy files directly into a managed checkout.
+The supplied staging directory remains caller-owned and may be removed after the
+run finishes. A valid retained dist is rechecked on subsequent installed runs
+without supplying `--frontend-dist` again.
+An early console-blocked checkout retained by `always` or `on-failure` can be reused
+at the same SHA/driver; its original blocked result remains unchanged. `never`
+removes the checkout, so omit reuse for the next run. Usual completed-run reuse and
+cleanup ownership still apply. The checkout owner records this setup association;
+`--related-run` accepts executed reports, so do not pass a setup-blocked run there.
+Do not replace production assets with a dummy page
+or weaken package assertions. Hosted CI retains its existing frontend artifact
+handoff; this SSH preflight does not change that boundary.
## Pilot from either development machine
diff --git a/issue-spec/specs/product-smoke-verification/spec.md b/issue-spec/specs/product-smoke-verification/spec.md
index c4ee0f3c..96631ad6 100644
--- a/issue-spec/specs/product-smoke-verification/spec.md
+++ b/issue-spec/specs/product-smoke-verification/spec.md
@@ -55,6 +55,19 @@ exit MUST fail the run independently of otherwise successful test summaries.
- **WHEN** the final observed child summary has no completed parent summary, including when both selected the same number of tests
- **THEN** validation remains incomplete and the child summary cannot establish parent success or independent parent counts.
+#### Scenario: Installed-package prerequisites fail before expensive execution
+
+When managed SSH backend selection includes installed-package acceptance requiring
+the console, the runner MUST verify the candidate's frontend source revision and
+output hashes before starting Rust execution. Missing or mismatched assets block
+setup with actionable preparation instructions; they are not product failures or
+successful acceptance. The caller supplies prepared assets in a separate execution-host
+staging directory; the runner verifies them and places them in the retained candidate
+only under its checkout lease. Retained assets are rechecked before use. Selections
+that exclude the installed scenarios MUST NOT require a frontend build. Real installed
+management and Quickstart assertions remain required after preparation; artifact verification alone does not
+establish their outcomes.
+
### Requirement: Evidence binds actual revision, artifact, role, and path
Report MUST include checked source revision, build/executable identity, platform,
diff --git a/scripts/console-assets.py b/scripts/console-assets.py
new file mode 100644
index 00000000..f4288a17
--- /dev/null
+++ b/scripts/console-assets.py
@@ -0,0 +1,189 @@
+#!/usr/bin/env python3
+"""Exact-candidate console handoff for managed installed-package acceptance."""
+import fnmatch
+import hashlib
+import json
+from pathlib import Path
+import shutil
+import stat
+import subprocess
+import tempfile
+
+
+RECEIPT = "hiroute-console-assets.json"
+SCHEMA = "hiroute.validation.console-assets/v1"
+CASES = (
+ "installed_standalone_cli_completes_the_headless_management_loop",
+ "installed_cli_quickstart_reuses_resources_and_exact_apply_files",
+ "installed_feed_withdrawal_preserves_published_sessions_workers_and_continue",
+)
+MAX_BYTES = 32 * 1024 * 1024
+
+
+def selected(cargo, platform):
+ """Inspect Cargo/libtest selectors without compiling or listing test binaries."""
+ if platform != "linux" or cargo[:2] != ["cargo", "test"]:
+ return []
+ boundary = cargo.index("--") if "--" in cargo else len(cargo)
+ before, after = cargo[2:boundary], cargo[boundary + 1:]
+ if "--no-run" in before or "--list" in after or "--ignored" in after:
+ return []
+ values, flags, filters = {}, set(), []
+ takes_value = {"-p", "--package", "--exclude", "--test", "--bin", "--example", "--bench",
+ "--target", "--features", "-F", "-j", "--jobs", "--profile", "--color",
+ "--message-format", "--target-dir", "--manifest-path", "--config"}
+ index = 0
+ while index < len(before):
+ arg = before[index]
+ key, equal, value = arg.partition("=")
+ if key in takes_value:
+ if not equal:
+ index += 1
+ value = before[index]
+ values.setdefault("--package" if key == "-p" else key, []).append(value)
+ elif arg.startswith("-p") and not arg.startswith("--"):
+ values.setdefault("--package", []).append(arg[2:])
+ elif arg.startswith("-"):
+ flags.add(arg)
+ else:
+ filters.append(arg)
+ index += 1
+ packages = values.get("--package", [])
+ if (packages and "--workspace" not in flags
+ and not any(fnmatch.fnmatchcase("hiroute-daemon", p.split("@", 1)[0]) for p in packages)):
+ return []
+ if any(fnmatch.fnmatchcase("hiroute-daemon", p) for p in values.get("--exclude", [])):
+ return []
+ if values.get("--target") and not any("linux" in p for p in values["--target"]):
+ return []
+ tests = values.get("--test", [])
+ if tests and not any(fnmatch.fnmatchcase("publication_process", p) for p in tests):
+ return []
+ if not tests and not flags.intersection({"--tests", "--all-targets"}) and (
+ flags.intersection({"--lib", "--bins", "--examples", "--benches", "--doc"})
+ or any(key in values for key in ("--bin", "--example", "--bench"))):
+ return []
+ skips = []
+ index = 0
+ while index < len(after):
+ arg = after[index]
+ if arg in ("--skip", "--test-threads", "--format", "--color", "--logfile"):
+ index += 1
+ if arg == "--skip":
+ skips.append(after[index])
+ elif arg.startswith("--skip="):
+ skips.append(arg.split("=", 1)[1])
+ elif not arg.startswith("-"):
+ filters.append(arg)
+ index += 1
+ exact = "--exact" in after
+ return [case for case in CASES
+ if (not filters or any(case == f if exact else f in case for f in filters))
+ and not any(case == skip if exact else skip in case for skip in skips)]
+
+
+def source_revision(repo):
+ def git(*args):
+ return subprocess.check_output(["git", "-C", str(repo), *args], text=True).strip()
+ if git("status", "--porcelain", "--untracked-files=all"):
+ raise ValueError("Console build requires a clean committed candidate")
+ return git("rev-parse", "HEAD")
+
+
+def files(dist):
+ """Inventory regular bytes only; reject links, traversal and special files."""
+ if dist.is_symlink() or not dist.is_dir():
+ raise ValueError("Console dist is missing or is a link")
+ result, total = {}, 0
+ for path in sorted(dist.rglob("*")):
+ mode = path.lstat().st_mode
+ if stat.S_ISLNK(mode) or not (stat.S_ISDIR(mode) or stat.S_ISREG(mode)):
+ raise ValueError("Console dist contains a link or special file")
+ if not stat.S_ISREG(mode) or path == dist / RECEIPT:
+ continue
+ data = path.read_bytes()
+ total += len(data)
+ if total > MAX_BYTES:
+ raise ValueError("Console dist exceeds the size limit")
+ result[path.relative_to(dist).as_posix()] = data
+ if not result.get("index.html"):
+ raise ValueError("Console dist must contain a nonempty index.html")
+ return result
+
+
+def manifest(sha, contents):
+ return dict(schema=SCHEMA, source_revision=sha,
+ files={name: hashlib.sha256(data).hexdigest() for name, data in contents.items()})
+
+
+def build(repo, env):
+ sha = source_revision(repo)
+ dist = repo / "apps/desktop/dist"
+ if dist.is_symlink():
+ raise ValueError("Console dist must not be a link")
+ # A failed build must not leave a previous build's provenance usable.
+ receipt = dist / RECEIPT
+ if receipt.is_symlink():
+ raise ValueError("Console receipt must not be a link")
+ receipt.unlink(missing_ok=True)
+ code = subprocess.call(["npm", "--prefix", "apps/desktop", "run", "build"], cwd=repo, env=env)
+ if code:
+ return code
+ if source_revision(repo) != sha:
+ raise ValueError("Console source changed during the build")
+ receipt.write_text(json.dumps(manifest(sha, files(dist)), indent=2) + "\n")
+ return 0
+
+
+def verify(dist, sha):
+ contents = files(dist)
+ receipt = dist / RECEIPT
+ if receipt.is_symlink() or not receipt.is_file():
+ raise ValueError("Console build receipt is missing; use validation.py frontend build --console-assets")
+ value = json.loads(receipt.read_text())
+ if value != manifest(sha, contents):
+ raise ValueError("Console source revision or file hashes differ from the candidate receipt")
+ return value
+
+
+def stage(checkout, sha, source):
+ if source.resolve() == checkout.resolve() or checkout.resolve() in source.resolve().parents:
+ raise ValueError("Console staging directory must be outside the managed checkout")
+ value = verify(source, sha)
+ contents = files(source)
+ if value != manifest(sha, contents):
+ raise ValueError("Console bytes changed while reading the staging directory")
+ parent = checkout / "apps/desktop"
+ if any(p.is_symlink() for p in (checkout, checkout / "apps", parent, parent / "dist")):
+ raise ValueError("Console destination must not traverse links")
+ # Caller holds the managed checkout lease. Never replace a developer checkout.
+ temporary = Path(tempfile.mkdtemp(prefix=".console-", dir=parent))
+ try:
+ for name, data in contents.items():
+ path = temporary / name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_bytes(data)
+ (temporary / RECEIPT).write_text(json.dumps(value, indent=2) + "\n")
+ verify(temporary, sha)
+ dist = parent / "dist"
+ if dist.exists():
+ shutil.rmtree(dist)
+ temporary.rename(dist)
+ finally:
+ if temporary.exists():
+ shutil.rmtree(temporary)
+
+
+def preflight(checkout, request, platform, supplied=None):
+ cases = selected(request["command"], platform)
+ if not cases:
+ return dict(state="not_required", cases=[])
+ if supplied is not None:
+ source = Path(supplied).expanduser()
+ if not source.is_absolute():
+ raise ValueError("Console staging directory must be an execution-host absolute path or ~/ path")
+ stage(checkout, request["sha"], source)
+ value = verify(checkout / "apps/desktop/dist", request["sha"])
+ return dict(state="verified", cases=cases, source_revision=value["source_revision"],
+ files=len(value["files"]), receipt_sha256=hashlib.sha256(
+ json.dumps(value, sort_keys=True).encode()).hexdigest())
diff --git a/scripts/remote-rust.py b/scripts/remote-rust.py
index 88ae7b6d..b99edbf9 100644
--- a/scripts/remote-rust.py
+++ b/scripts/remote-rust.py
@@ -62,6 +62,16 @@ def scheduling():
return module
+def console_assets():
+ path = Path(__file__).with_name("console-assets.py")
+ if not path.exists():
+ path = Path(__file__).with_name(Path(__file__).stem + "-console.py")
+ spec = importlib.util.spec_from_file_location("console_assets", path)
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+ return module
+
+
def full_test(cargo):
before = cargo[:cargo.index("--")] if "--" in cargo else cargo
return cargo[:2] == ["cargo", "test"] and "--workspace" in before
@@ -262,8 +272,13 @@ def verify_reused_checkout(root, request):
raise RunBlocked("checkout_missing", "Reusable checkout owner run is missing")
previous = json.loads(previous_path.read_text())
checkout = root / "worktrees" / owner_id
- if (previous.get("id") != owner_id or previous.get("status") != "completed"
- or previous.get("retention") != "always" or previous.get("sha") != request["sha"]
+ console_blocked = (previous.get("status") == "blocked" and previous.get("blocker") == "console_assets"
+ and previous.get("process_exit") is None and not previous.get("command_started_at"))
+ ready = previous.get("status") == "completed" or console_blocked
+ if (previous.get("id") != owner_id or not ready
+ or (previous.get("retention") != "always"
+ and not (console_blocked and previous.get("retention") == "on-failure"))
+ or previous.get("sha") != request["sha"]
or previous.get("origin") != request["origin"]
or previous.get("executor_sha") != request["executor_sha"]
or previous.get("checkout") != str(checkout)
@@ -582,6 +597,37 @@ def worker(request_path):
result.update(status="running", checkout=str(checkout), checkout_owner=owner_id or request["id"],
started_at=time.time(), checkout_finished_at=time.time())
save(result_path, result)
+ try:
+ result["console_assets"] = console_assets().preflight(
+ checkout, request, sys.platform, request.get("frontend_dist"))
+ except (ValueError, OSError, KeyError) as error:
+ result["console_assets"] = dict(state="blocked", reason=str(error))
+ staging = Path.home() / ".cache/hiroute/console-assets" / request["id"]
+ host = request["ssh_host"]
+ rerun = ["python3", "scripts/validation.py", "--frontend-dist", str(staging / "dist"),
+ "backend", "run", "--ref", request["ref"], "--sha", request["sha"],
+ "--retention", "always"]
+ if owner_id or request["retention"] != "never":
+ rerun += ["--reuse-checkout", owner_id or request["id"]]
+ if request.get("schedule"):
+ rerun += ["--schedule", request["schedule"]]
+ if request.get("exclusive"):
+ rerun += ["--exclusive"]
+ if request.get("plan"):
+ rerun += ["--plan", request["plan"], "--phase", request.get("phase", "focused")]
+ rerun += ["--", *request["command"]]
+ result["console_assets"]["preparation"] = [
+ "Use clean candidate " + request["sha"] + " on the configured frontend target:",
+ "python3 scripts/validation.py frontend ci",
+ "python3 scripts/validation.py frontend build --console-assets",
+ "From the machine with that dist, copy it to a separate backend staging directory:",
+ shlex.join(["ssh", host, shlex.join(["mkdir", "-p", str(staging)])]),
+ shlex.join(["scp", "-r", "apps/desktop/dist", host + ":" + str(staging) + "/"]),
+ "Rerun from the caller; --frontend-dist is a path on the backend host:", shlex.join(rerun)]
+ raise RunBlocked("console_assets", str(error) +
+ "; prepare with validation.py frontend build --console-assets, then pass "
+ "--frontend-dist to SSH backend run; see result.console_assets.preparation") from error
+ save(result_path, result)
server_env = cache_server_environment(root)
result["cache_server_environment"] = {
"temporary_directory": server_env["TMPDIR"], "idle_timeout": "0",
@@ -715,7 +761,7 @@ try:
except FileExistsError: pass
if driver.read_text()!=source: raise RuntimeError("immutable driver mismatch")
finally: temporary.unlink()
-for kind in ("report","schedule"):
+for kind in ("report","schedule","console"):
helper=p.pop(kind+"_source",None)
if helper is not None:
if hashlib.sha256(helper.encode()).hexdigest()!=p[kind+"_digest"]: raise RuntimeError(kind+" helper digest mismatch")
@@ -809,6 +855,7 @@ def main():
run = sub.add_parser("run", help="Queue a focused Cargo command against a pushed commit")
run.add_argument("--ref", required=True, help="Advertised branch, e.g. refs/heads/codex/my-proposal")
run.add_argument("--sha", help="Default: current HEAD")
+ run.add_argument("--frontend-dist", help="Execution-host staging directory from validation.py frontend build --console-assets")
run.add_argument("--exclusive", action="store_true", help="Reserve all 8 slots, 64 compiler jobs (bench/Gates)")
run.add_argument("--timeout", type=int, default=3600)
run.add_argument("--retention", choices=RETENTION_POLICIES, default="on-failure",
@@ -868,12 +915,15 @@ def main():
retention=args.retention, executor_sha=executor_sha,
reuse_checkout=args.reuse_checkout, continue_checkout=args.continue_checkout,
schedule=args.schedule,
+ ssh_host=args.host, frontend_dist=args.frontend_dist,
executor_digest=hashlib.sha256(source.encode()).hexdigest(), source=source)
payload.update(reporting().options(args), submitted_at=time.time())
helper = command(["git", "show", executor_sha + ":scripts/validation-report.py"]) + "\n"
payload.update(report_source=helper, report_digest=hashlib.sha256(helper.encode()).hexdigest())
scheduler = command(["git", "show", executor_sha + ":scripts/validation-schedule.py"]) + "\n"
payload.update(schedule_source=scheduler, schedule_digest=hashlib.sha256(scheduler.encode()).hexdigest())
+ console = command(["git", "show", executor_sha + ":scripts/console-assets.py"]) + "\n"
+ payload.update(console_source=console, console_digest=hashlib.sha256(console.encode()).hexdigest())
# bashrc owns the user's proxy settings. No credentials or proxy URLs are serialized.
remote = "source ~/.bashrc >/dev/null 2>&1; exec python3 -c " + shlex.quote(BOOTSTRAP)
subprocess.run(["ssh", "-o", "BatchMode=yes", args.host, "bash -lc " + shlex.quote(remote)],
diff --git a/scripts/test-console-assets.py b/scripts/test-console-assets.py
new file mode 100644
index 00000000..42b95afa
--- /dev/null
+++ b/scripts/test-console-assets.py
@@ -0,0 +1,243 @@
+#!/usr/bin/env python3
+"""Console selection, provenance and real managed-checkout admission regressions."""
+import importlib.util
+import json
+import os
+from pathlib import Path
+import shlex
+import shutil
+import subprocess
+import sys
+import tempfile
+import unittest
+from unittest.mock import patch
+
+sys.dont_write_bytecode = True
+
+
+def load(name):
+ spec = importlib.util.spec_from_file_location(name, Path(__file__).with_name(name + '.py'))
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+ return module
+
+
+assets = load('console-assets')
+runner = load('remote-rust')
+
+
+class Selection(unittest.TestCase):
+ def test_selected_installed_cases_and_broad_commands(self):
+ for args in ([], ['--workspace', '--exclude', 'hiroute-desktop', '--all-features'],
+ ['-p', 'hiroute-daemon'], ['--package=hiroute-daemon'], ['-phiroute-daemon'],
+ ['-p', 'hiroute-*', '--test', 'publication_*'],
+ ['--test', 'publication_process', '--', 'installed_'],
+ ['--test', 'publication_process', '--', '--include-ignored']):
+ with self.subTest(args=args):
+ self.assertEqual(assets.selected(['cargo', 'test', *args], 'linux'), list(assets.CASES))
+ for case in assets.CASES:
+ self.assertEqual(assets.selected(['cargo', 'test', '-p', 'hiroute-daemon', '--test',
+ 'publication_process', case, '--', '--exact'], 'linux'), [case])
+ source = Path(__file__).parent.parent / 'crates/daemon/tests/publication_process.rs'
+ import re
+ self.assertEqual(set(re.findall(r'fn (installed_\w+)\(', source.read_text())), set(assets.CASES))
+
+ def test_unrelated_commands_do_not_require_or_stage_console(self):
+ for command in (['cargo', 'check'], ['cargo', 'build'],
+ ['cargo', 'test', '-p', 'hiroute-cli'],
+ ['cargo', 'test', '--workspace', '--exclude=hiroute-daemon'],
+ ['cargo', 'test', '-p', 'hiroute-daemon', '--lib'],
+ ['cargo', 'test', '--test=control_process'],
+ ['cargo', 'test', '--test', 'publication_process', 'real_process_'],
+ ['cargo', 'test', '--no-run'], ['cargo', 'test', '--', '--list'],
+ ['cargo', 'test', '--', '--ignored'],
+ ['cargo', 'test', '--', '--skip', 'installed_'],
+ ['cargo', 'test', '--target', 'x86_64-pc-windows-msvc']):
+ with self.subTest(command=command), patch.object(assets, 'stage') as stage:
+ self.assertEqual(assets.preflight(Path('/unused'), dict(command=command),
+ 'linux', '/unused/missing-dist'), dict(state='not_required', cases=[]))
+ stage.assert_not_called()
+ self.assertEqual(assets.selected(['cargo', 'test'], 'darwin'), [])
+
+ def test_exact_filters_and_skips_keep_remaining_cases(self):
+ first = assets.CASES[0]
+ self.assertEqual(assets.selected(['cargo', 'test', first, '--', '--skip=' + first,
+ '--exact'], 'linux'), [])
+ self.assertEqual(assets.selected(['cargo', 'test', first, '--', '--skip=installed_',
+ '--exact'], 'linux'), [first])
+ self.assertEqual(assets.selected(['cargo', 'test', '--', '--skip', first], 'linux'),
+ list(assets.CASES[1:]))
+
+
+class Artifact(unittest.TestCase):
+ def setUp(self):
+ self.temp = tempfile.TemporaryDirectory(dir='/tmp')
+ self.addCleanup(self.temp.cleanup)
+ self.root = Path(self.temp.name).resolve()
+ self.repo = self.root / 'source'
+ self.repo.mkdir()
+ (self.repo / 'apps/desktop').mkdir(parents=True)
+ (self.repo / 'apps/desktop/package.json').write_text('{}')
+ (self.repo / '.gitignore').write_text('apps/desktop/dist/\ntarget/\n')
+ self.git('init', '-q')
+ self.git('add', '.')
+ self.git('-c', 'user.name=Fixture', '-c', '[email protected]',
+ '-c', 'commit.gpgsign=false', 'commit', '-qm', 'fixture')
+ self.sha = self.git('rev-parse', 'HEAD')
+ self.dist = self.repo / 'apps/desktop/dist'
+ self.dist.mkdir()
+ (self.dist / 'index.html').write_text('<script src="/assets/app.js"></script>')
+ (self.dist / 'assets').mkdir()
+ (self.dist / 'assets/app.js').write_text('console.log("fixture asset")')
+ (self.dist / assets.RECEIPT).write_text(json.dumps(assets.manifest(self.sha, assets.files(self.dist))))
+
+ def git(self, *args):
+ return subprocess.check_output(['git', '-C', str(self.repo), *args], text=True).strip()
+
+ def test_build_stamps_only_successful_unchanged_source(self):
+ with patch.object(assets.subprocess, 'call', return_value=0) as npm:
+ self.assertEqual(assets.build(self.repo, {}), 0)
+ self.assertEqual(npm.call_args.args[0], ['npm', '--prefix', 'apps/desktop', 'run', 'build'])
+ self.assertEqual(assets.verify(self.dist, self.sha)['source_revision'], self.sha)
+ with patch.object(assets.subprocess, 'call', return_value=12):
+ self.assertEqual(assets.build(self.repo, {}), 12)
+ self.assertFalse((self.dist / assets.RECEIPT).exists())
+ with patch.object(assets, 'source_revision', side_effect=[self.sha, '0' * 40]), \
+ patch.object(assets.subprocess, 'call', return_value=0), self.assertRaisesRegex(ValueError, 'changed'):
+ assets.build(self.repo, {})
+ (self.repo / 'apps/desktop/package.json').write_text('{"dirty":true}')
+ with self.assertRaisesRegex(ValueError, 'clean committed'):
+ assets.build(self.repo, {})
+
+ def test_staging_rejects_old_sha_wrong_hash_missing_extra_and_linked_files(self):
+ receipt = assets.verify(self.dist, self.sha)
+ destination = self.root / 'destination'
+ (destination / 'apps/desktop').mkdir(parents=True)
+ assets.stage(destination, self.sha, self.dist)
+ retained = destination / 'apps/desktop/dist'
+ self.assertEqual(assets.verify(retained, self.sha), receipt)
+ self.assertEqual(assets.verify(self.dist, self.sha), receipt)
+ for kind in ('old', 'hash', 'missing', 'extra', 'file-link', 'directory-link', 'receipt-link', 'special'):
+ bad = self.root / kind
+ shutil.copytree(self.dist, bad)
+ if kind == 'old':
+ (bad / assets.RECEIPT).write_text(json.dumps(dict(receipt, source_revision='0' * 40)))
+ elif kind == 'hash':
+ (bad / 'index.html').write_text('corrupted')
+ elif kind == 'missing':
+ (bad / 'index.html').unlink()
+ elif kind == 'extra':
+ (bad / 'extra').write_text('')
+ elif kind in ('file-link', 'receipt-link'):
+ name = assets.RECEIPT if kind == 'receipt-link' else 'index.html'
+ (bad / name).unlink()
+ (bad / name).symlink_to(self.dist / name)
+ elif kind == 'directory-link':
+ shutil.rmtree(bad / 'assets')
+ (bad / 'assets').symlink_to(self.dist / 'assets', target_is_directory=True)
+ elif kind == 'special':
+ os.mkfifo(bad / 'pipe')
+ with self.subTest(kind=kind), self.assertRaises(ValueError):
+ assets.stage(destination, self.sha, bad)
+ self.assertEqual(assets.verify(retained, self.sha), receipt)
+ with self.assertRaisesRegex(ValueError, 'outside'):
+ assets.stage(destination, self.sha, retained)
+ with patch.object(assets, 'MAX_BYTES', 1), self.assertRaisesRegex(ValueError, 'size limit'):
+ assets.stage(destination, self.sha, self.dist)
+ snapshots = [assets.files(self.dist), dict(assets.files(self.dist), **{'index.html': b'changed'})]
+ with patch.object(assets, 'files', side_effect=snapshots), self.assertRaisesRegex(ValueError, 'changed'):
+ assets.stage(destination, self.sha, self.dist)
+ with self.assertRaisesRegex(ValueError, 'execution-host'):
+ assets.preflight(destination, dict(command=['cargo', 'test'], sha=self.sha), 'linux', 'relative/dist')
+ (destination / 'apps/desktop/dist').rename(destination / 'apps/desktop/original')
+ retained.symlink_to(destination / 'apps/desktop/original', target_is_directory=True)
+ with self.assertRaisesRegex(ValueError, 'link'):
+ assets.stage(destination, self.sha, self.dist)
+ self.assertEqual(assets.verify(retained.parent / 'original', self.sha), receipt)
+
+ def test_clean_worker_blocks_before_rust_then_reuses_same_checkout_with_verified_assets(self):
+ root = self.root / 'workbench'
+ first = '20261021-000001-abcdef12'
+ second = '20261021-000002-abcdef12'
+ ref = self.git('symbolic-ref', 'HEAD')
+ request = dict(id=first, ref=ref, sha=self.sha, executor_sha=self.sha, origin=str(self.repo),
+ command=['cargo', 'test', '-p', 'hiroute-daemon', '--test', 'publication_process',
+ 'installed_', '--', '--nocapture'],
+ retention='on-failure', exclusive=False, timeout=10, plan='console-assets', phase='focused',
+ ssh_host='fixture-backend')
+ home = self.root / 'home'
+ binary = home / '.cargo/bin'
+ binary.mkdir(parents=True)
+ fake = '''import sys,json,pathlib
+if '--version' in sys.argv:
+ print(pathlib.Path(sys.argv[0]).name+' fixture');sys.exit(0)
+if 'metadata' in sys.argv:
+ print(json.dumps({'target_directory':str(pathlib.Path.cwd()/'target')}));sys.exit(0)
+print('Running tests/publication_process.rs (target/debug/deps/publication_process-abcdef0123456789)')
+print('running 1 test')
+print('test component_fixture ... ok')
+print('test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s')
+'''
+ for name in ('cargo', 'rustc'):
+ path = binary / name
+ path.write_text('#!' + sys.executable + '\n' + fake)
+ path.chmod(0o700)
+
+ def execute(row, source=None):
+ run = root / 'runs' / row['id']
+ run.mkdir(parents=True)
+ if source is not None:
+ row = dict(row, frontend_dist=str(source))
+ runner.save(run / 'request.json', row)
+ with patch.object(runner.Path, 'home', return_value=home), patch.dict(os.environ, {}, clear=False), \
+ patch.object(runner.sys, 'platform', 'linux'), \
+ patch.object(runner.tempfile, 'gettempdir', return_value=str(self.root)), \
+ patch.object(runner, 'MIN_FREE_BYTES', 0), \
+ patch.object(runner, 'stats', return_value={'max_cache_size': 30 * 1024**3}) as stats:
+ runner.worker(run / 'request.json')
+ return json.loads((run / 'result.json').read_text()), stats
+
+ before, stats = execute(request)
+ self.assertEqual((before['status'], before['blocker'], before['process_exit']),
+ ('blocked', 'console_assets', None), before)
+ self.assertNotIn('command_started_at', before)
+ stats.assert_not_called()
+ self.assertFalse((root / 'runs' / first / 'command.log').exists())
+ self.assertIn('--reuse-checkout ' + first, before['console_assets']['preparation'][-1])
+ staging = home / '.cache/hiroute/console-assets' / first
+ preparation = before['console_assets']['preparation']
+ self.assertIn(shlex.join(['scp', '-r', 'apps/desktop/dist', 'fixture-backend:' + str(staging) + '/']), preparation)
+ self.assertEqual(shlex.split(preparation[-1])[2:4], ['--frontend-dist', str(staging / 'dist')])
+ row = dict(request, id=second, reuse_checkout=first, retention='always')
+ lease = runner.checkout_lease(root, first)
+ try:
+ blocked, _ = execute(dict(row, id='20261021-000003-abcdef12'), self.dist)
+ self.assertEqual(blocked.get('blocker'), 'checkout_busy')
+ finally:
+ lease.close()
+ for index, kind in enumerate(('sha', 'hash'), 4):
+ bad = self.root / ('worker-' + kind)
+ shutil.copytree(self.dist, bad)
+ if kind == 'sha':
+ receipt = json.loads((bad / assets.RECEIPT).read_text())
+ (bad / assets.RECEIPT).write_text(json.dumps(dict(receipt, source_revision='0' * 40)))
+ else:
+ (bad / 'index.html').write_text('corrupted')
+ invalid, stats = execute(dict(row, id='20261021-00000' + str(index) + '-abcdef12'), bad)
+ self.assertEqual(invalid.get('blocker'), 'console_assets')
+ self.assertNotIn('command_started_at', invalid)
+ stats.assert_not_called()
+ after, _ = execute(row, self.dist)
+ self.assertEqual(after['status'], 'completed', after)
+ self.assertEqual(after['checkout'], before['checkout'])
+ self.assertEqual(after['console_assets']['state'], 'verified')
+ (Path(after['checkout']) / 'apps/desktop/dist/assets/app.js').write_text('damaged retained asset')
+ invalid, stats = execute(dict(row, id='20261021-000006-abcdef12'))
+ self.assertEqual(invalid.get('blocker'), 'console_assets')
+ stats.assert_not_called()
+ self.assertEqual(json.loads((root / 'runs' / first / 'result.json').read_text())['status'], 'blocked')
+ runner.cleanup_temp_directory(after['temporary_directory'])
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/scripts/test-plan.py b/scripts/test-plan.py
index dc314ae8..f73368ab 100644
--- a/scripts/test-plan.py
+++ b/scripts/test-plan.py
@@ -102,6 +102,7 @@ HOSTED_BACKEND_EXECUTION = {
"scripts/ci-shards.py",
}
VALIDATION_TOOLING = {"scripts/validation.py", "scripts/validation-host.py", "scripts/test-validation.py",
+ "scripts/console-assets.py", "scripts/test-console-assets.py",
"scripts/windows-dev.py", "scripts/windows-dev-host.py", "scripts/test-windows-dev.py",
"scripts/windows-desktop-session.py", "scripts/windows-dev-profile.py",
"scripts/validation-report.py", "scripts/test-validation-report.py",
@@ -557,7 +558,7 @@ def select(paths, full=False):
["python3", "scripts/release-contracts.py", "check"]])
if validation_tooling:
commands.extend([["python3", "scripts/" + name] for name in
- ("test-validation.py", "test-windows-dev.py", "test-validation-report.py", "test-local-rust.py", "test-remote-rust.py", "test-validation-schedule.py",
+ ("test-validation.py", "test-console-assets.py", "test-windows-dev.py", "test-validation-report.py", "test-local-rust.py", "test-remote-rust.py", "test-validation-schedule.py",
"test-ci-run.py", "test-ci-shards.py", "test-desktop-pilot.py", "test-pilot-builds.py")])
if full or validation_tooling or local_gc_tooling:
if ["python3", "scripts/test-local-rust.py"] not in commands:
diff --git a/scripts/test-remote-rust.py b/scripts/test-remote-rust.py
index 62344082..c800f72d 100644
--- a/scripts/test-remote-rust.py
+++ b/scripts/test-remote-rust.py
@@ -59,9 +59,11 @@ class Contract(unittest.TestCase):
self.fail('Unexpected controller command')
with patch.object(runner, 'CONFIG', config), \
patch.object(runner, 'validate'), patch.object(runner, 'command', side_effect=committed_command), \
+ patch.object(runner, 'console_assets', side_effect=AssertionError('Caller must not read frontend bytes')), \
patch.object(runner.subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)) as ssh, \
patch.object(runner.sys, 'argv', ['remote-rust.py', 'run', '--ref', 'refs/heads/fixture',
'--sha', candidate, '--plan', 'workbench-port', '--phase', 'focused',
+ '--frontend-dist', '~/.cache/console candidate/dist',
'--', 'cargo', 'check', '--locked']):
runner.main()
payload = json.loads(ssh.call_args.kwargs['input'])
@@ -71,18 +73,24 @@ class Contract(unittest.TestCase):
self.assertIn(['git', 'show', executor + ':scripts/remote-rust.py'], commands)
self.assertEqual(payload['source'], '# committed fixture source\n')
self.assertEqual(payload['executor_digest'], hashlib.sha256(payload['source'].encode()).hexdigest())
+ self.assertEqual(payload['frontend_dist'], '~/.cache/console candidate/dist')
+ self.assertEqual(payload['ssh_host'], 'fixture-host')
+ self.assertNotIn('console_assets_bundle', payload)
def test_configured_bootstrap_keeps_repository_files_separate(self):
with tempfile.TemporaryDirectory() as directory:
source = '# immutable fixture driver\n'
payload = dict(source=source, executor_sha='a' * 40, sha='b' * 40,
executor_digest=hashlib.sha256(source.encode()).hexdigest(),
- id='run-1', workbench='.local/share/github-workbench')
+ id='run-1', workbench='.local/share/github-workbench',
+ frontend_dist='~/remote-only/dist')
with patch('pathlib.Path.home', return_value=Path(directory)), \
patch('sys.stdin', io.StringIO(json.dumps(payload))), patch('sys.stdout', io.StringIO()), \
patch('os.umask'), patch('subprocess.Popen'):
exec(runner.BOOTSTRAP, {})
- self.assertTrue((Path(directory) / payload['workbench'] / 'runs/run-1/request.json').exists())
+ run = Path(directory) / payload['workbench'] / 'runs/run-1'
+ self.assertEqual(json.loads((run / 'request.json').read_text())['frontend_dist'], '~/remote-only/dist')
+ self.assertFalse((run / 'console-assets.json').exists())
self.assertFalse((Path(directory) / runner.ROOT / 'runs').exists())
def test_new_repository_worker_respects_an_existing_workbench_capacity_lease(self):
@@ -239,8 +247,10 @@ sys.exit(101 if bad else 0)
runner.save(path, request)
with patch.object(runner.Path, 'home', return_value=home), patch.dict(os.environ, {}, clear=False), \
patch.object(runner.tempfile, 'gettempdir', return_value=str(base.resolve())), \
+ patch.object(runner, 'console_assets') as console, \
patch.object(runner, 'MIN_FREE_BYTES', 0), \
patch.object(runner, 'stats', return_value={'max_cache_size': 30 * 1024**3}):
+ console.return_value.preflight.return_value = dict(state='not_required', cases=[])
runner.worker(path)
result = json.loads((run / 'result.json').read_text())
if result.get('temporary_directory'):
@@ -714,7 +724,7 @@ sys.exit(int((home/'exit-code').read_text()))
run = workbench / 'runs' / key
run.mkdir(parents=True)
request = dict(id=key, ref=ref, sha=sha, executor_sha=sha, origin=str(source),
- command=['cargo', 'test', '--no-fail-fast'], retention='always',
+ command=['cargo', 'test', '--lib', '--no-fail-fast'], retention='always',
exclusive=False, timeout=10, plan='focused-counts', phase='focused')
runner.save(run / 'request.json', request)
with patch.object(runner.Path, 'home', return_value=home), \
diff --git a/scripts/test-test-plan.py b/scripts/test-test-plan.py
index 2a558989..627342a6 100644
--- a/scripts/test-test-plan.py
+++ b/scripts/test-test-plan.py
@@ -319,10 +319,11 @@ class SelectionTests(unittest.TestCase):
def test_validation_tooling_selects_runner_consumers_without_product_builds(self):
result = plan.select(['scripts/validation.py', 'scripts/local-rust.py', 'scripts/pilot-builds.py',
- 'scripts/desktop-pilot.py', 'docs/validation-routing.md'])
+ 'scripts/desktop-pilot.py', 'scripts/console-assets.py',
+ 'scripts/test-console-assets.py', 'docs/validation-routing.md'])
self.assertFalse(result['rust'])
self.assertFalse(result['frontend'])
- for name in ('test-validation.py', 'test-local-rust.py', 'test-remote-rust.py',
+ for name in ('test-validation.py', 'test-console-assets.py', 'test-local-rust.py', 'test-remote-rust.py',
'test-ci-shards.py', 'test-desktop-pilot.py', 'test-pilot-builds.py'):
self.assertIn(['python3', 'scripts/' + name], result['commands'])
mixed = plan.select(['scripts/validation.py', 'crates/daemon/src/lib.rs'])
diff --git a/scripts/test-validation.py b/scripts/test-validation.py
index f2320225..36e0693d 100644
--- a/scripts/test-validation.py
+++ b/scripts/test-validation.py
@@ -105,6 +105,22 @@ class Routing(unittest.TestCase):
self.assertEqual(result, 7)
self.assertEqual(call.call_args.args[0][-4:], ['--host', 'linux', 'logs', 'RUN'])
+ def test_frontend_staging_path_routes_only_to_ssh_backend_run(self):
+ target = {'backend': {'transport': 'ssh', 'host': 'linux'}}
+ with patch.object(m.subprocess, 'check_output', return_value='a' * 40), \
+ patch.object(m.subprocess, 'call', return_value=0) as call:
+ m.dispatch(target, 'backend', ['run', '--ref', 'refs/heads/example', '--', 'cargo', 'test'],
+ frontend_dist='~/.cache/candidate/dist')
+ argv = call.call_args.args[0]
+ self.assertEqual(argv[argv.index('run') + 1:argv.index('run') + 3],
+ ['--frontend-dist', '~/.cache/candidate/dist'])
+ self.assertNotIn('--frontend-dist', argv[argv.index('--') + 1:])
+ for transport, action, arguments in (('local', 'backend', ['run']),
+ ('ssh', 'backend', ['status', 'RUN']),
+ ('local', 'frontend', ['build'])):
+ with self.subTest(transport=transport, action=action), self.assertRaises(ValueError):
+ m.dispatch({action: {'transport': transport}}, action, arguments, frontend_dist='/dist')
+
def test_pilot_build_routes_to_desktop_with_small_tooling_bundle(self):
target = {'transport': 'ssh', 'host': 'mac', 'repo': '~/git/HiRoute'}
with patch.object(m.subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)) as call:
@@ -308,6 +324,19 @@ pilot.record('build:' + sys.argv[1])
with patch.object(host.subprocess, 'run', return_value=subprocess.CompletedProcess([], 12)) as run:
self.assertEqual(host.execute(p), 12)
self.assertEqual(run.call_args.args[0], ['npm', '--prefix', 'apps/desktop', 'run', 'test'])
+ p['arguments'] = ['build']
+ with patch.object(host.subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)) as run:
+ self.assertEqual(host.execute(p), 0)
+ self.assertEqual(run.call_args.args[0], ['npm', '--prefix', 'apps/desktop', 'run', 'build'])
+
+ def test_console_frontend_build_explicitly_uses_the_provenance_owner(self):
+ p = self.payload('frontend', ['build', '--console-assets'])
+ p['name'] = 'frontend'
+ with patch.object(host.importlib.util, 'module_from_spec') as module:
+ module.return_value.build.return_value = 17
+ with patch.object(host.importlib.util, 'spec_from_file_location'):
+ self.assertEqual(host.execute(p), 17)
+ self.assertEqual(module.return_value.build.call_args.args[0], Path(p['repo']).resolve())
def test_executor_runs_as_json_stdin_program(self):
p = self.payload('frontend', ['not-a-command'])
diff --git a/scripts/validation-host.py b/scripts/validation-host.py
index 3842f8f8..761a768e 100644
--- a/scripts/validation-host.py
+++ b/scripts/validation-host.py
@@ -1,6 +1,7 @@
#!/usr/bin/env python3
"""Execution-host adapter. Receives a JSON request from validation.py over stdin."""
import hashlib
+import importlib.util
import json
import os
from pathlib import Path
@@ -149,8 +150,13 @@ def execute(payload):
str(Path(payload["frontend_dist"]).expanduser())], cwd=repo, env=env, text=True).strip()
argv = [sys.executable, str(script), *arguments]
elif action == "frontend":
- if arguments not in (["ci"], ["build"], ["test"]):
- raise ValueError("frontend expects ci, build or test")
+ if arguments not in (["ci"], ["build"], ["build", "--console-assets"], ["test"]):
+ raise ValueError("frontend expects ci, build, build --console-assets or test")
+ if arguments == ["build", "--console-assets"]:
+ spec = importlib.util.spec_from_file_location("console_assets", repo / "scripts/console-assets.py")
+ console = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(console)
+ return console.build(repo, env)
argv = ["npm", "--prefix", "apps/desktop", *(arguments if arguments == ["ci"] else ["run", *arguments])]
elif action == "pilot":
if not arguments:
diff --git a/scripts/validation.py b/scripts/validation.py
index e225449f..d16a0230 100644
--- a/scripts/validation.py
+++ b/scripts/validation.py
@@ -84,8 +84,9 @@ def dispatch(targets, action, arguments, frontend_dist=None, dry_run=False):
if name not in targets:
raise ValueError("Configure a " + name + " target in ~/.config/hiroute/validation.json")
target = targets[name]
- if frontend_dist and not (action == "desktop" and arguments[:1] == ["run"]):
- raise ValueError("--frontend-dist is only for desktop run")
+ if frontend_dist and not (arguments[:1] == ["run"] and (
+ action == "desktop" or (action == "backend" and target["transport"] == "ssh"))):
+ raise ValueError("--frontend-dist is only for desktop run or SSH backend run")
if action in ("backend", "desktop", "pilot-build"):
arguments = with_candidate(arguments, REPO)
if name == "backend" and target["transport"] == "ssh":
@@ -93,6 +94,8 @@ def dispatch(targets, action, arguments, frontend_dist=None, dry_run=False):
# This is read-only; no worker submission or sccache server startup.
payload = dict(target=target, action="doctor", name=name, arguments=[], repo="~/git/HiRoute")
else:
+ if frontend_dist:
+ arguments[1:1] = ["--frontend-dist", frontend_dist]
argv = [sys.executable, str(REPO / "scripts/remote-rust.py"), "--host", target["host"], *arguments]
if dry_run:
print(json.dumps({"target": target, "argv": argv}, indent=2))
@@ -121,7 +124,7 @@ def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--config", type=Path, default=CONFIG)
parser.add_argument("--dry-run", action="store_true")
- parser.add_argument("--frontend-dist", help="Frontend path on the Desktop execution host")
+ parser.add_argument("--frontend-dist", help="Execution-host frontend directory for desktop run or SSH backend run")
parser.add_argument("action", choices=("show", "doctor", *TARGETS, "pilot", "pilot-cli", "pilot-build", "windows"))
parser.add_argument("arguments", nargs=argparse.REMAINDER)
args = parser.parse_args()