Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

store: every viewer and both wrappers install and run on a machine with no node, uv or Python ≥ 3.10

deee06625f494de
diff --git a/store/agents/autonomous-circuit/.gitignore b/store/agents/autonomous-circuit/.gitignore
index 8ecd08629..12d09bc65 100644
--- a/store/agents/autonomous-circuit/.gitignore
+++ b/store/agents/autonomous-circuit/.gitignore
@@ -1,3 +1,5 @@
# The wrapped project, fetched by toolchain/setup.sh — never source here.
upstream/
upstream.partial/
+# The board pipeline's Python, made by toolchain/setup.sh.
+.venv/
diff --git a/store/agents/autonomous-circuit/README.md b/store/agents/autonomous-circuit/README.md
index d89f76fb8..34efe45b8 100644
--- a/store/agents/autonomous-circuit/README.md
+++ b/store/agents/autonomous-circuit/README.md
@@ -16,6 +16,12 @@ run against that copy.
- `VERSIONS` — the repository, the pinned commit on its main, and the sparse patterns.
- `toolchain/fetch-upstream.sh` — the read-only, sparse, blob-less fetch; `setup.sh`, `doctor.sh`,
`init-workspace.sh`, `viewer.sh` hand off to the project's scripts of the same names.
+- `toolchain/runtimes.sh` — the store's shared helper (a copy; `store/tools/sync-runtimes.mjs` writes it).
+ What the project's scripts expect on PATH comes from it, so a machine with no Node and Apple's
+ Python 3.9 installs as well as any: Node >= 22.12 (Harness's own when the machine has none) and a
+ CPython 3.12 `.venv` with numpy for the board pipeline.
+- `toolchain/python` — that `.venv`'s Python with Node on PATH, handed to the agent as `CIRCUIT_PYTHON`:
+ the project's skills start the pipeline with `python3` and re-exec it there when that is too old.
## Credit and stewardship
diff --git a/store/agents/autonomous-circuit/harness.json b/store/agents/autonomous-circuit/harness.json
index 50d4a167e..feb4d3066 100644
--- a/store/agents/autonomous-circuit/harness.json
+++ b/store/agents/autonomous-circuit/harness.json
@@ -17,6 +17,7 @@
"skills": ["upstream/skills"],
"env": {
"CIRCUIT_TOOLCHAIN": "${dsh}/upstream/toolchain",
+ "CIRCUIT_PYTHON": "${dsh}/toolchain/python",
"CIRCUIT_SKILLS_DIR": "${workspace}/.claude/skills",
"DISABLE_AUTOUPDATER": "1"
}
diff --git a/store/agents/autonomous-circuit/toolchain/doctor.sh b/store/agents/autonomous-circuit/toolchain/doctor.sh
index 157f3a112..eca129927 100755
--- a/store/agents/autonomous-circuit/toolchain/doctor.sh
+++ b/store/agents/autonomous-circuit/toolchain/doctor.sh
@@ -1,12 +1,24 @@
#!/usr/bin/env bash
-# The project's own doctor, run against the fetched copy. cwd = the install dir.
+# The project's own doctor, run against the fetched copy. cwd = the install dir. It looks for node and
+# a python >= 3.10 on PATH; both are handed to it the way setup made them: Harness's Node when the
+# machine has none, and the wrapper's .venv as CIRCUIT_PYTHON.
set -uo pipefail
cd "$(dirname "$0")/.."
. ./VERSIONS
+# shellcheck source=runtimes.sh
+. toolchain/runtimes.sh
if [ ! -f upstream/.harness-commit ]; then
echo "miss autonomous-circuit is not fetched — run toolchain/setup.sh"; exit 1
fi
[ "$(cat upstream/.harness-commit)" = "${UPSTREAM_COMMIT}" ] \
&& echo "ok autonomous-circuit @ ${UPSTREAM_COMMIT:0:12}" \
|| echo "warn autonomous-circuit @ $(cut -c1-12 upstream/.harness-commit), VERSIONS pins ${UPSTREAM_COMMIT:0:12} — run toolchain/setup.sh"
-HARNESS_DSH_DIR="$PWD/upstream" CIRCUIT_TOOLCHAIN="${CIRCUIT_TOOLCHAIN:-$PWD/upstream/toolchain}" exec upstream/harness/toolchain/doctor.sh
+# The project's own node line says it when there is none.
+harness_node 22.12 >/dev/null || true
+venv=0
+.venv/bin/python -c 'import numpy' >/dev/null 2>&1 \
+ || { echo "miss .venv with numpy (the board pipeline's Python) — run toolchain/setup.sh"; venv=1; }
+HARNESS_DSH_DIR="$PWD/upstream" CIRCUIT_TOOLCHAIN="${CIRCUIT_TOOLCHAIN:-$PWD/upstream/toolchain}" \
+ CIRCUIT_PYTHON="${CIRCUIT_PYTHON:-$PWD/toolchain/python}" upstream/harness/toolchain/doctor.sh
+code=$?
+exit $((code ? code : venv))
diff --git a/store/agents/autonomous-circuit/toolchain/python b/store/agents/autonomous-circuit/toolchain/python
new file mode 100755
index 000000000..e7eace2f0
--- /dev/null
+++ b/store/agents/autonomous-circuit/toolchain/python
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# CIRCUIT_PYTHON: the board pipeline's interpreter. The project's skills start its tools with
+# `python3`, re-exec them here when that is older than 3.10 (Apple's is 3.9), and shell out to `node`
+# from PATH — which the agent's shell may not have. So: Harness's Node on PATH, then the .venv setup made.
+here="$(cd "$(dirname "$0")/.." && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/toolchain/runtimes.sh"
+harness_node 22.12 >&2 || true
+exec "$here/.venv/bin/python" "$@"
diff --git a/store/agents/autonomous-circuit/toolchain/runtimes.sh b/store/agents/autonomous-circuit/toolchain/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/agents/autonomous-circuit/toolchain/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/agents/autonomous-circuit/toolchain/setup.sh b/store/agents/autonomous-circuit/toolchain/setup.sh
index e5d9a4b95..bd37efe01 100755
--- a/store/agents/autonomous-circuit/toolchain/setup.sh
+++ b/store/agents/autonomous-circuit/toolchain/setup.sh
@@ -1,8 +1,18 @@
#!/usr/bin/env bash
# Runs once at install, cwd = the install dir: fetch Autonomous Circuit at the pinned commit, then
# run the project's OWN setup in it (pinned tscircuit toolchain, skill runtimes, the board viewer).
+# What that setup and the board pipeline take from PATH is provided here first, since a new Mac has
+# neither: a Node >= 22.12 with npm (Vite 7's floor; Harness's own when the machine has none), and a
+# Python >= 3.10 with numpy (the pipeline's floor and its gates) in .venv, which toolchain/python runs
+# as the agent's CIRCUIT_PYTHON.
set -euo pipefail
cd "$(dirname "$0")/.."
+# shellcheck source=runtimes.sh
+. toolchain/runtimes.sh
+harness_node 22.12 || exit 1
+harness_venv .venv 3.12 3.10 || exit 1
+harness_pip .venv numpy
+echo "ok numpy $(.venv/bin/python -c 'import numpy; print(numpy.__version__)') (the board pipeline's gates)"
toolchain/fetch-upstream.sh
[ -x upstream/harness/toolchain/setup.sh ] || { echo "miss upstream/harness/toolchain/setup.sh"; exit 1; }
HARNESS_DSH_DIR="$PWD/upstream" upstream/harness/toolchain/setup.sh
diff --git a/store/agents/autonomous-circuit/toolchain/test_toolchain.py b/store/agents/autonomous-circuit/toolchain/test_toolchain.py
index 014a77587..c9291ad93 100644
--- a/store/agents/autonomous-circuit/toolchain/test_toolchain.py
+++ b/store/agents/autonomous-circuit/toolchain/test_toolchain.py
@@ -6,6 +6,11 @@ scripts that print what they were handed. Each test lays out a throwaway install
scripts linked in and a VERSIONS that keeps the real sparse patterns but pins the stand-in. Git runs with
GIT_ALLOW_PROTOCOL=file and no user or system config, so nothing here can reach the network, and the
stand-in is checked byte for byte afterwards: the wrapper only ever reads upstream.
+
+The scripts run on a PATH of their own, as on a new Mac: git and the few coreutils they use, linked in,
+and stubs for what runtimes.sh provides — `node` (on PATH, as Harness's own through current-node, or
+nowhere) and `uv`, whose `uv venv` makes a venv holding a stub python. HOME is a temp dir. What
+runtimes.sh does with no uv at all is store/tools/test_runtimes.py's business.
"""
import hashlib
import os
@@ -19,10 +24,29 @@ from pathlib import Path
TOOLCHAIN = Path(__file__).resolve().parent
PACKAGE = TOOLCHAIN.parent
NAME = "autonomous-circuit"
-SCRIPTS = ("fetch-upstream.sh", "setup.sh", "doctor.sh", "init-workspace.sh", "viewer.sh")
+SCRIPTS = ("fetch-upstream.sh", "setup.sh", "doctor.sh", "init-workspace.sh", "viewer.sh", "python", "runtimes.sh")
PROJECT_SCRIPTS = ("setup.sh", "doctor.sh", "init-workspace.sh", "viewer.sh")
# A project script that says which one ran, with what, where; exits with $STUB_EXIT.
-STUB = '#!/bin/sh\necho "{name} dsh=$HARNESS_DSH_DIR toolchain=${{CIRCUIT_TOOLCHAIN:-}} pwd=$(pwd -P)"\nexit "${{STUB_EXIT:-0}}"\n'
+STUB = ('#!/bin/sh\necho "{name} dsh=$HARNESS_DSH_DIR toolchain=${{CIRCUIT_TOOLCHAIN:-}} python=${{CIRCUIT_PYTHON:-}} '
+ 'node=$(command -v node) pwd=$(pwd -P)"\nexit "${{STUB_EXIT:-0}}"\n')
+# Real commands the scripts use, linked into the sandbox's bin/. Never stub one of these names: a stub
+# written over the link would be written through it, onto the real command.
+LINKED = ("bash", "cat", "cp", "cut", "dirname", "du", "git", "mkdir", "mv", "rm")
+# node: `-v` prints NODE_VERSION; `-e SCRIPT MIN` is runtimes.sh's at-least probe, answered in bash.
+NODE = r'''case "$1" in
+ -v) echo "v$NODE_VERSION" ;;
+ -e) IFS=. read -r a b _ <<< "$3.0"; IFS=. read -r x y _ <<< "$NODE_VERSION"
+ if [ "$x" -gt "$a" ] || { [ "$x" -eq "$a" ] && [ "$y" -ge "$b" ]; }; then exit 0; fi; exit 1 ;;
+esac'''
+# The venv's python: its version (VENV_PY_OK answers runtimes.sh's keep-or-replace probe), numpy's
+# version and import (IMPORT_NUMPY); any other run echoes its arguments and the node it would find.
+VENV_PYTHON = r'''case "$1" in
+ --version) echo "Python 3.12.11" ;;
+ -c) case "$2" in *numpy.__version__*) echo "2.3.4" ;; *version_info*) exit "${VENV_PY_OK:-0}" ;; *) exit "${IMPORT_NUMPY:-0}" ;; esac ;;
+ *) echo "python $* node=$(command -v node)" ;;
+esac'''
+# A bash line tracer (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT) rides along when set.
+TRACER = ("BASH_ENV", "SHCOV_OUT")
def versions_value(key: str) -> str:
@@ -94,13 +118,22 @@ class Wrapper(unittest.TestCase):
shutil.rmtree(cls.class_tmp)
def setUp(self):
- self.tmp = Path(tempfile.mkdtemp())
+ self.tmp = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.tmp)
self.pkg = self.tmp / NAME
(self.pkg / "toolchain").mkdir(parents=True)
for name in SCRIPTS:
+ # linked, not copied: a line tracer maps back to the source
(self.pkg / "toolchain" / name).symlink_to(TOOLCHAIN / name)
self.pin(self.upstream.first)
+ self.home = self.tmp / "home"
+ self.home.mkdir()
+ self.calls = self.tmp / "calls.log"
+ self.calls.touch()
+ self.bin = self.tmp / "bin"
+ self.bin.mkdir()
+ for tool in LINKED:
+ (self.bin / tool).symlink_to(shutil.which(tool))
def tearDown(self):
self.assertEqual(snapshot(self.upstream.bare), self.pristine, "the upstream repository was written to")
@@ -111,11 +144,48 @@ class Wrapper(unittest.TestCase):
text = re.sub(r'^UPSTREAM_COMMIT=.*$', f'UPSTREAM_COMMIT="{commit}"', text, flags=re.M)
(self.pkg / "VERSIONS").write_text(text)
- def run_script(self, name: str, cwd: Path | None = None, path: str | None = None, **env: str):
- full = dict(self.upstream.env, **env)
- if path is not None:
- full["PATH"] = path
- return subprocess.run([str(self.pkg / "toolchain" / name)], cwd=cwd or self.pkg, env=full,
+ # the sandbox's commands
+
+ def stub(self, name: str, body: str = "", where: Path | None = None) -> Path:
+ assert where is not None or name not in LINKED, name
+ path = (where or self.bin) / name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.unlink(missing_ok=True) # never write through a link to a real command
+ path.write_text(f'#!/bin/bash\necho "{name} $*" >> "$CALLS"\n{body}\n')
+ path.chmod(0o755)
+ return path
+
+ def node(self, version: str = "22.23.2") -> Path:
+ """node on PATH."""
+ return self.stub("node", f'NODE_VERSION={version}\n{NODE}')
+
+ def harness_node(self, version: str = "22.23.2") -> Path:
+ """No node on PATH; Harness's own, named by ~/.harness/runtime/current-node."""
+ node = self.stub("node", f'NODE_VERSION={version}\n{NODE}', where=self.tmp / "harness-node" / "bin")
+ runtime = self.home / ".harness" / "runtime"
+ runtime.mkdir(parents=True)
+ (runtime / "current-node").write_text(f"{node}\n")
+ return node
+
+ def uv(self) -> Path:
+ """uv on PATH: `uv venv … DIR` makes a venv of a stub python (UV_VENV_EXIT fails it); `uv pip` exits PIP_EXIT."""
+ python = self.stub("python", VENV_PYTHON, where=self.tmp / "templates")
+ return self.stub("uv", f'''case "$1" in
+ venv) [ "${{UV_VENV_EXIT:-0}}" = 0 ] || exit "$UV_VENV_EXIT"; for last in "$@"; do :; done; mkdir -p "$last/bin" && cp "{python}" "$last/bin/python" ;;
+ pip) exit "${{PIP_EXIT:-0}}" ;;
+esac''')
+
+ def venv(self) -> Path:
+ """The .venv setup leaves."""
+ return self.stub("python", VENV_PYTHON, where=self.pkg / ".venv" / "bin")
+
+ def logged(self) -> list:
+ return self.calls.read_text().splitlines()
+
+ def run_script(self, name: str, cwd: Path | None = None, path: str | None = None, args=(), **env: str):
+ full = {k: v for k, v in self.upstream.env.items() if k not in ("PATH", "HOME") and (k in TRACER or not k.startswith("BASH_"))}
+ full.update(PATH=path or str(self.bin), HOME=str(self.home), CALLS=str(self.calls), **env)
+ return subprocess.run([str(self.pkg / "toolchain" / name), *args], cwd=cwd or self.pkg, env=full,
capture_output=True, text=True, timeout=120)
def fake_fetched(self, commit: str, scripts=PROJECT_SCRIPTS):
@@ -133,6 +203,9 @@ class Wrapper(unittest.TestCase):
def real(self, path: Path) -> str:
return os.path.realpath(path)
+ def field(self, line: str, key: str) -> str:
+ return re.search(rf"{key}=(\S*)", line).group(1)
+
# VERSIONS
def test_versions_pins_a_full_commit_of_the_projects_public_repository(self):
@@ -140,6 +213,12 @@ class Wrapper(unittest.TestCase):
self.assertEqual(versions_value("UPSTREAM_REPO"), f"https://github.com/autonomous-ai/{NAME}.git")
self.assertRegex(versions_value("UPSTREAM_COMMIT"), r"^[0-9a-f]{40}$")
+ def test_the_agent_runs_the_pipeline_on_the_wrappers_python(self):
+ import json
+ env = json.loads((PACKAGE / "harness.json").read_text())["agent"]["env"]
+ self.assertEqual(env["CIRCUIT_PYTHON"], "${dsh}/toolchain/python")
+ self.assertIn(".venv/", (PACKAGE / ".gitignore").read_text().splitlines())
+
# fetch-upstream.sh
def test_fetch_checks_out_the_pinned_commit_sparse_shallow_and_blobless(self):
@@ -199,7 +278,7 @@ class Wrapper(unittest.TestCase):
self.assertFalse((self.pkg / "upstream.partial").exists())
def test_fetch_without_git_says_so(self):
- bin_ = self.tmp / "bin"
+ bin_ = self.tmp / "bin-no-git"
bin_.mkdir()
for tool in ("bash", "cat"):
(bin_ / tool).symlink_to(shutil.which(tool))
@@ -210,34 +289,87 @@ class Wrapper(unittest.TestCase):
# setup.sh
- def test_setup_fetches_then_runs_the_projects_own_setup_in_its_checkout(self):
+ def test_setup_provides_node_and_python_then_runs_the_projects_own_setup_in_its_checkout(self):
+ node = self.node()
+ self.uv()
run = self.run_script("setup.sh", cwd=self.tmp)
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
lines = run.stdout.splitlines()
- self.assertEqual(len(lines), 3, lines)
- self.assertIn(f"ok {NAME} @ {self.upstream.first[:12]} fetched", lines[1])
- self.assertTrue(lines[2].startswith("setup "), lines)
- self.assertEqual(self.real(Path(self.dsh_dir(lines[2]))), self.real(self.pkg / "upstream"))
- self.assertTrue(lines[2].endswith(f" pwd={self.real(self.pkg)}"), lines)
+ self.assertEqual(lines[:3], [
+ " python 3.12 in .venv (uv downloads it when this machine has none)",
+ "ok Python 3.12.11 in .venv",
+ "ok numpy 2.3.4 (the board pipeline's gates)",
+ ])
+ self.assertIn(f"ok {NAME} @ {self.upstream.first[:12]} fetched", lines[4])
+ self.assertEqual(len(lines), 6, lines)
+ self.assertTrue(lines[5].startswith("setup "), lines)
+ self.assertEqual(self.real(Path(self.field(lines[5], "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.field(lines[5], "node"), str(node))
+ self.assertTrue(lines[5].endswith(f" pwd={self.real(self.pkg)}"), lines)
+ log = self.logged()
+ self.assertIn("uv venv --quiet --seed --python 3.12 --python-preference only-managed .venv", log)
+ self.assertIn("uv pip install --quiet --python .venv/bin/python numpy", log)
+
+ def test_setup_on_a_machine_without_node_hands_the_project_harnesss_own(self):
+ node = self.harness_node()
+ self.uv()
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
+ self.assertEqual(self.field(run.stdout.splitlines()[-1], "node"), str(node))
+
+ def test_setup_without_a_new_enough_node_is_a_miss_before_anything_else(self):
+ self.uv()
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 1)
+ self.assertEqual(run.stdout, f"miss node >= 22.12, and Harness's own Node is not in {self.home}/.harness/runtime"
+ " — run `harness start` once to lay it down\n")
+ self.harness_node("22.11.0")
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 1)
+ self.assertEqual(run.stdout, "miss node >= 22.12 — this machine's newest is v22.11.0, Harness's own; update Harness\n")
+ self.assertEqual([line for line in self.logged() if line.startswith("uv ")], [])
+ self.assertFalse((self.pkg / "upstream").exists())
+
+ def test_setup_stops_when_no_python_can_be_made(self):
+ self.node()
+ self.uv()
+ run = self.run_script("setup.sh", UV_VENV_EXIT="1")
+ self.assertEqual(run.returncode, 1)
+ self.assertEqual(run.stdout.splitlines()[-1], "miss could not make a Python 3.12 environment in .venv")
+ self.assertFalse((self.pkg / "upstream").exists())
- def dsh_dir(self, line: str) -> str:
- return re.search(r"dsh=(\S+)", line).group(1)
+ def test_setup_stops_when_numpy_does_not_install(self):
+ self.node()
+ self.uv()
+ run = self.run_script("setup.sh", PIP_EXIT="2")
+ self.assertEqual(run.returncode, 2)
+ self.assertNotIn("numpy", run.stdout)
+ self.assertFalse((self.pkg / "upstream").exists())
def test_setup_fails_with_the_projects_setup(self):
+ self.node()
+ self.uv()
run = self.run_script("setup.sh", STUB_EXIT="3")
self.assertEqual(run.returncode, 3)
def test_setup_stops_when_the_fetch_fails(self):
+ self.node()
+ self.uv()
self.pin("0123456789abcdef0123456789abcdef01234567")
run = self.run_script("setup.sh")
self.assertNotEqual(run.returncode, 0)
self.assertNotIn("setup dsh=", run.stdout)
def test_setup_says_when_the_project_has_no_setup(self):
+ self.node()
+ self.uv()
+ self.venv()
self.fake_fetched(versions_value_for(self.pkg), scripts=("doctor.sh",))
run = self.run_script("setup.sh")
self.assertEqual(run.returncode, 1)
self.assertEqual(run.stdout.splitlines(), [
+ "ok Python 3.12.11 in .venv",
+ "ok numpy 2.3.4 (the board pipeline's gates)",
f"ok {NAME} @ {self.upstream.first[:12]} already fetched",
"miss upstream/harness/toolchain/setup.sh",
])
@@ -249,25 +381,44 @@ class Wrapper(unittest.TestCase):
self.assertEqual(run.returncode, 1)
self.assertEqual(run.stdout, f"miss {NAME} is not fetched — run toolchain/setup.sh\n")
- def test_doctor_at_the_pin_hands_over_to_the_projects_doctor(self):
+ def test_doctor_at_the_pin_hands_over_to_the_projects_doctor_with_node_and_the_pipelines_python(self):
self.fake_fetched(self.upstream.first)
+ self.venv()
+ node = self.harness_node()
run = self.run_script("doctor.sh", cwd=self.tmp)
self.assertEqual(run.returncode, 0, run.stderr)
ok, project = run.stdout.splitlines()
self.assertEqual(ok, f"ok {NAME} @ {self.upstream.first[:12]}")
self.assertTrue(project.startswith("doctor "), project)
- self.assertEqual(self.real(Path(self.dsh_dir(project))), self.real(self.pkg / "upstream"))
- self.assertEqual(self.real(Path(re.search(r"toolchain=(\S+)", project).group(1))), self.real(self.pkg / "upstream" / "toolchain"))
+ self.assertEqual(self.real(Path(self.field(project, "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.real(Path(self.field(project, "toolchain"))), self.real(self.pkg / "upstream" / "toolchain"))
+ self.assertEqual(self.field(project, "python"), str(self.pkg / "toolchain" / "python"))
+ self.assertEqual(self.field(project, "node"), str(node))
self.assertTrue(project.endswith(f"pwd={self.real(self.pkg)}"), project)
self.assertEqual(self.run_script("doctor.sh", STUB_EXIT="4").returncode, 4)
- def test_doctor_keeps_a_toolchain_the_environment_names(self):
+ def test_doctor_keeps_a_toolchain_and_a_python_the_environment_names(self):
+ self.fake_fetched(self.upstream.first)
+ self.venv()
+ run = self.run_script("doctor.sh", CIRCUIT_TOOLCHAIN="/Users/example/circuit/toolchain",
+ CIRCUIT_PYTHON="/Users/example/python3.12")
+ self.assertIn("toolchain=/Users/example/circuit/toolchain python=/Users/example/python3.12 node= ", run.stdout)
+
+ def test_doctor_without_the_pipelines_python_is_a_miss_and_still_asks_the_project(self):
self.fake_fetched(self.upstream.first)
- run = self.run_script("doctor.sh", CIRCUIT_TOOLCHAIN="/Users/example/circuit/toolchain")
- self.assertIn("toolchain=/Users/example/circuit/toolchain ", run.stdout)
+ run = self.run_script("doctor.sh")
+ self.assertEqual(run.returncode, 1)
+ lines = run.stdout.splitlines()
+ self.assertEqual(lines[1], "miss .venv with numpy (the board pipeline's Python) — run toolchain/setup.sh")
+ self.assertTrue(lines[2].startswith("doctor "), lines)
+ self.venv()
+ self.assertEqual(self.run_script("doctor.sh", IMPORT_NUMPY="1").returncode, 1)
+ # the project's own failure wins: its code is the one Harness reports
+ self.assertEqual(self.run_script("doctor.sh", IMPORT_NUMPY="1", STUB_EXIT="4").returncode, 4)
def test_doctor_warns_when_the_copy_is_behind_the_pin(self):
self.fake_fetched(self.upstream.first)
+ self.venv()
self.pin(self.upstream.second)
run = self.run_script("doctor.sh")
self.assertEqual(run.returncode, 0)
@@ -277,23 +428,64 @@ class Wrapper(unittest.TestCase):
def test_doctor_fails_when_the_project_has_no_doctor(self):
self.fake_fetched(self.upstream.first, scripts=())
+ self.venv()
run = self.run_script("doctor.sh")
self.assertNotEqual(run.returncode, 0)
self.assertEqual(run.stdout, f"ok {NAME} @ {self.upstream.first[:12]}\n")
# init-workspace.sh, viewer.sh
- def test_init_and_viewer_run_the_projects_own_in_the_workspace(self):
+ def test_init_runs_the_projects_own_in_the_workspace(self):
self.fake_fetched(self.upstream.first)
ws = self.tmp / "workspace"
ws.mkdir()
- for script, word in (("init-workspace.sh", "init-workspace"), ("viewer.sh", "viewer")):
- run = self.run_script(script, cwd=ws)
- self.assertEqual(run.returncode, 0, run.stderr)
- self.assertTrue(run.stdout.startswith(f"{word} "), run.stdout)
- self.assertEqual(self.real(Path(self.dsh_dir(run.stdout))), self.real(self.pkg / "upstream"))
- self.assertTrue(run.stdout.rstrip().endswith(f"pwd={self.real(ws)}"), run.stdout)
- self.assertEqual(self.run_script(script, cwd=ws, STUB_EXIT="5").returncode, 5)
+ run = self.run_script("init-workspace.sh", cwd=ws)
+ self.assertEqual(run.returncode, 0, run.stderr)
+ self.assertTrue(run.stdout.startswith("init-workspace "), run.stdout)
+ self.assertEqual(self.real(Path(self.field(run.stdout, "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertTrue(run.stdout.rstrip().endswith(f"pwd={self.real(ws)}"), run.stdout)
+ self.assertEqual(self.run_script("init-workspace.sh", cwd=ws, STUB_EXIT="5").returncode, 5)
+
+ def test_viewer_runs_the_projects_own_in_the_workspace_on_harnesss_node(self):
+ self.fake_fetched(self.upstream.first)
+ node = self.harness_node()
+ ws = self.tmp / "workspace"
+ ws.mkdir()
+ run = self.run_script("viewer.sh", cwd=ws)
+ self.assertEqual(run.returncode, 0, run.stderr)
+ self.assertTrue(run.stdout.startswith("viewer "), run.stdout)
+ self.assertEqual(self.real(Path(self.field(run.stdout, "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.field(run.stdout, "node"), str(node))
+ self.assertEqual(self.field(run.stdout, "python"), str(self.pkg / "toolchain" / "python"))
+ self.assertTrue(run.stdout.rstrip().endswith(f"pwd={self.real(ws)}"), run.stdout)
+ self.assertEqual(self.run_script("viewer.sh", cwd=ws, STUB_EXIT="5").returncode, 5)
+ run = self.run_script("viewer.sh", cwd=ws, CIRCUIT_PYTHON="/Users/example/python3.12")
+ self.assertEqual(self.field(run.stdout, "python"), "/Users/example/python3.12")
+
+ def test_viewer_without_any_node_is_a_miss(self):
+ self.fake_fetched(self.upstream.first)
+ run = self.run_script("viewer.sh", cwd=self.tmp)
+ self.assertEqual(run.returncode, 1)
+ self.assertRegex(run.stdout, r"^miss node >= 22\.12, and Harness's own Node is not in ")
+ self.assertNotIn("viewer ", run.stdout)
+
+ # python (CIRCUIT_PYTHON)
+
+ def test_python_is_the_venv_with_node_on_path(self):
+ self.venv()
+ node = self.harness_node()
+ run = self.run_script("python", cwd=self.tmp, args=("scripts/circuit", "boards/main.tsx"))
+ self.assertEqual((run.returncode, run.stderr), (0, ""))
+ self.assertEqual(run.stdout, f"python scripts/circuit boards/main.tsx node={node}\n")
+ version = self.run_script("python", args=("-c", "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)"))
+ self.assertEqual(version.returncode, 0)
+
+ def test_python_without_node_still_runs_and_says_why_on_stderr(self):
+ self.venv()
+ run = self.run_script("python", args=("-m", "circuit"))
+ self.assertEqual(run.returncode, 0)
+ self.assertEqual(run.stdout, "python -m circuit node=\n") # stdout stays the tool's own
+ self.assertRegex(run.stderr, r"^miss node >= 22\.12")
def versions_value_for(pkg: Path) -> str:
diff --git a/store/agents/autonomous-circuit/toolchain/viewer.sh b/store/agents/autonomous-circuit/toolchain/viewer.sh
index 3190bdd30..2eb40e6cf 100755
--- a/store/agents/autonomous-circuit/toolchain/viewer.sh
+++ b/store/agents/autonomous-circuit/toolchain/viewer.sh
@@ -1,5 +1,11 @@
#!/usr/bin/env bash
-# The project's own board viewer, over the agent's workspace, on the port Harness hands it.
+# The project's own board viewer, over the agent's workspace, on the port Harness hands it. Harness
+# starts it through a login shell that may have no node: Harness's own stands in. Its board check
+# runs the pipeline, so it gets the pipeline's Python too.
set -euo pipefail
here="$(cd "$(dirname "$0")/.." && pwd)"
-HARNESS_DSH_DIR="$here/upstream" exec "$here/upstream/harness/toolchain/viewer.sh"
+# shellcheck source=runtimes.sh
+. "$here/toolchain/runtimes.sh"
+harness_node 22.12 || exit 1
+HARNESS_DSH_DIR="$here/upstream" CIRCUIT_PYTHON="${CIRCUIT_PYTHON:-$here/toolchain/python}" \
+ exec "$here/upstream/harness/toolchain/viewer.sh"
diff --git a/store/agents/autonomous-workshop/README.md b/store/agents/autonomous-workshop/README.md
index f03e71965..98f99ba71 100644
--- a/store/agents/autonomous-workshop/README.md
+++ b/store/agents/autonomous-workshop/README.md
@@ -16,6 +16,9 @@ the pane is the store's shared [CAD Viewer](../../viewers/cad-viewer).
- `VERSIONS` — the repository, the pinned commit on its main, and the sparse folders.
- `toolchain/fetch-upstream.sh` — the read-only, sparse, blob-less fetch; `setup.sh`, `doctor.sh` and
`init-workspace.sh` hand off to the project's scripts of the same names.
+- `toolchain/runtimes.sh` — the store's shared helper (a copy; `store/tools/sync-runtimes.mjs` writes it).
+ The project's setup needs `uv` on PATH, which a new Mac lacks: the wrapper puts a pinned one there
+ first (uv then brings the Python). The doctor finds it, and Node, the same way.
## Credit and stewardship
diff --git a/store/agents/autonomous-workshop/toolchain/doctor.sh b/store/agents/autonomous-workshop/toolchain/doctor.sh
index 4e87664fb..3655f1d59 100755
--- a/store/agents/autonomous-workshop/toolchain/doctor.sh
+++ b/store/agents/autonomous-workshop/toolchain/doctor.sh
@@ -1,12 +1,19 @@
#!/usr/bin/env bash
-# The project's own doctor, run against the fetched copy. cwd = the install dir.
+# The project's own doctor, run against the fetched copy. cwd = the install dir. It looks for uv and
+# node on PATH; both are put there the way setup found them — the machine's own, else the pinned uv
+# setup fetched and Harness's own Node.
set -uo pipefail
cd "$(dirname "$0")/.."
. ./VERSIONS
+# shellcheck source=runtimes.sh
+. toolchain/runtimes.sh
if [ ! -f upstream/.harness-commit ]; then
echo "miss autonomous-workshop is not fetched — run toolchain/setup.sh"; exit 1
fi
[ "$(cat upstream/.harness-commit)" = "${UPSTREAM_COMMIT}" ] \
&& echo "ok autonomous-workshop @ ${UPSTREAM_COMMIT:0:12}" \
|| echo "warn autonomous-workshop @ $(cut -c1-12 upstream/.harness-commit), VERSIONS pins ${UPSTREAM_COMMIT:0:12} — run toolchain/setup.sh"
+# The project's own uv and node lines say it when there is none.
+harness_uv >/dev/null || true
+harness_node 20 >/dev/null || true
HARNESS_DSH_DIR="$PWD/upstream" exec upstream/harness/toolchain/doctor.sh
diff --git a/store/agents/autonomous-workshop/toolchain/runtimes.sh b/store/agents/autonomous-workshop/toolchain/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/agents/autonomous-workshop/toolchain/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/agents/autonomous-workshop/toolchain/setup.sh b/store/agents/autonomous-workshop/toolchain/setup.sh
index 1ad2cf75e..4ce3d9d61 100755
--- a/store/agents/autonomous-workshop/toolchain/setup.sh
+++ b/store/agents/autonomous-workshop/toolchain/setup.sh
@@ -1,8 +1,16 @@
#!/usr/bin/env bash
# Runs once at install, cwd = the install dir: fetch Autonomous Workshop at the pinned commit, then
-# run the project's OWN setup in it (a .venv from its pinned uv.lock, Python >= 3.11).
+# run the project's OWN setup in it (a .venv from its pinned uv.lock, Python >= 3.11). That setup
+# needs uv on PATH and a new Mac has none: the pinned one from runtimes.sh stands in.
set -euo pipefail
cd "$(dirname "$0")/.."
+# shellcheck source=runtimes.sh
+. toolchain/runtimes.sh
+harness_uv || exit 1
toolchain/fetch-upstream.sh
[ -x upstream/harness/toolchain/setup.sh ] || { echo "miss upstream/harness/toolchain/setup.sh"; exit 1; }
HARNESS_DSH_DIR="$PWD/upstream" upstream/harness/toolchain/setup.sh
+# The first import of OCP and vtk from a new .venv takes minutes (macOS verifies each new library on
+# first load) and the project's doctor makes it: paid here, in setup's time, not in the doctor's.
+echo " loading cadgen and OpenCascade once (the first load is slow, the rest are not)"
+upstream/.venv/bin/python -c 'import cadgen, build123d' >/dev/null 2>&1 || true
diff --git a/store/agents/autonomous-workshop/toolchain/test_toolchain.py b/store/agents/autonomous-workshop/toolchain/test_toolchain.py
index 5cf170102..fe8c48c5b 100644
--- a/store/agents/autonomous-workshop/toolchain/test_toolchain.py
+++ b/store/agents/autonomous-workshop/toolchain/test_toolchain.py
@@ -6,6 +6,12 @@ scripts that print what they were handed. Each test lays out a throwaway install
scripts linked in and a VERSIONS that keeps the real sparse patterns but pins the stand-in. Git runs with
GIT_ALLOW_PROTOCOL=file and no user or system config, so nothing here can reach the network, and the
stand-in is checked byte for byte afterwards: the wrapper only ever reads upstream.
+
+The scripts run on a PATH of their own, as on a new Mac: git and the few coreutils they use, linked in,
+and stubs for what runtimes.sh provides — `uv` (on PATH, as the pinned one a setup fetched into
+~/.harness/runtime, or nowhere, with no curl to fetch it) and `node` (on PATH, as Harness's own through
+current-node, or nowhere). HOME is a temp dir. What runtimes.sh does when it does download uv is
+store/tools/test_runtimes.py's business.
"""
import hashlib
import os
@@ -19,10 +25,23 @@ from pathlib import Path
TOOLCHAIN = Path(__file__).resolve().parent
PACKAGE = TOOLCHAIN.parent
NAME = "autonomous-workshop"
-SCRIPTS = ("fetch-upstream.sh", "setup.sh", "doctor.sh", "init-workspace.sh")
+SCRIPTS = ("fetch-upstream.sh", "setup.sh", "doctor.sh", "init-workspace.sh", "runtimes.sh")
PROJECT_SCRIPTS = ("setup.sh", "doctor.sh", "init-workspace.sh")
# A project script that says which one ran, with what, where; exits with $STUB_EXIT.
-STUB = '#!/bin/sh\necho "{name} dsh=$HARNESS_DSH_DIR pwd=$(pwd -P)"\nexit "${{STUB_EXIT:-0}}"\n'
+STUB = '#!/bin/sh\necho "{name} dsh=$HARNESS_DSH_DIR uv=$(command -v uv) node=$(command -v node) pwd=$(pwd -P)"\nexit "${{STUB_EXIT:-0}}"\n'
+# Real commands the scripts use, linked into the sandbox's bin/ — curl is not one, so nothing here can
+# download. Never stub one of these names: a stub written over the link would be written through it,
+# onto the real command.
+LINKED = ("bash", "cat", "cut", "dirname", "du", "git", "mkdir", "mktemp", "mv", "rm", "uname")
+# node: `-v` prints NODE_VERSION; `-e SCRIPT MIN` is runtimes.sh's at-least probe, answered in bash.
+NODE = r'''case "$1" in
+ -v) echo "v$NODE_VERSION" ;;
+ -e) IFS=. read -r a b _ <<< "$3.0"; IFS=. read -r x y _ <<< "$NODE_VERSION"
+ if [ "$x" -gt "$a" ] || { [ "$x" -eq "$a" ] && [ "$y" -ge "$b" ]; }; then exit 0; fi; exit 1 ;;
+esac'''
+UV_VERSION = re.search(r"^HARNESS_UV_VERSION=(\S+)$", (TOOLCHAIN / "runtimes.sh").read_text(), re.M).group(1)
+# A bash line tracer (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT) rides along when set.
+TRACER = ("BASH_ENV", "SHCOV_OUT")
def versions_value(key: str) -> str:
@@ -95,13 +114,21 @@ class Wrapper(unittest.TestCase):
shutil.rmtree(cls.class_tmp)
def setUp(self):
- self.tmp = Path(tempfile.mkdtemp())
+ self.tmp = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.tmp)
self.pkg = self.tmp / NAME
(self.pkg / "toolchain").mkdir(parents=True)
for name in SCRIPTS:
+ # linked, not copied: a line tracer maps back to the source
(self.pkg / "toolchain" / name).symlink_to(TOOLCHAIN / name)
self.pin(self.upstream.first)
+ self.home = self.tmp / "home"
+ self.home.mkdir()
+ self.runtime = self.home / ".harness" / "runtime"
+ self.bin = self.tmp / "bin"
+ self.bin.mkdir()
+ for tool in LINKED:
+ (self.bin / tool).symlink_to(shutil.which(tool))
def tearDown(self):
self.assertEqual(snapshot(self.upstream.bare), self.pristine, "the upstream repository was written to")
@@ -112,10 +139,35 @@ class Wrapper(unittest.TestCase):
text = re.sub(r'^UPSTREAM_COMMIT=.*$', f'UPSTREAM_COMMIT="{commit}"', text, flags=re.M)
(self.pkg / "VERSIONS").write_text(text)
+ # the sandbox's commands
+
+ def stub(self, name: str, body: str = "", where: Path | None = None) -> Path:
+ assert where is not None or name not in LINKED, name
+ path = (where or self.bin) / name
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.unlink(missing_ok=True) # never write through a link to a real command
+ path.write_text(f"#!/bin/bash\n{body}\n")
+ path.chmod(0o755)
+ return path
+
+ def uv(self) -> Path:
+ """uv on PATH."""
+ return self.stub("uv", 'echo "uv $UV_VERSION"')
+
+ def fetched_uv(self) -> Path:
+ """No uv on PATH; the pinned one a setup fetched into ~/.harness/runtime."""
+ return self.stub("uv", 'echo "uv $UV_VERSION"', where=self.runtime / f"uv-{UV_VERSION}")
+
+ def harness_node(self, version: str = "22.23.2") -> Path:
+ """No node on PATH; Harness's own, named by ~/.harness/runtime/current-node."""
+ node = self.stub("node", f"NODE_VERSION={version}\n{NODE}", where=self.tmp / "harness-node" / "bin")
+ self.runtime.mkdir(parents=True, exist_ok=True)
+ (self.runtime / "current-node").write_text(f"{node}\n")
+ return node
+
def run_script(self, name: str, cwd: Path | None = None, path: str | None = None, **env: str):
- full = dict(self.upstream.env, **env)
- if path is not None:
- full["PATH"] = path
+ full = {k: v for k, v in self.upstream.env.items() if k not in ("PATH", "HOME") and (k in TRACER or not k.startswith("BASH_"))}
+ full.update(PATH=path or str(self.bin), HOME=str(self.home), UV_VERSION=UV_VERSION, **env)
return subprocess.run([str(self.pkg / "toolchain" / name)], cwd=cwd or self.pkg, env=full,
capture_output=True, text=True, timeout=120)
@@ -134,6 +186,9 @@ class Wrapper(unittest.TestCase):
def real(self, path: Path) -> str:
return os.path.realpath(path)
+ def field(self, line: str, key: str) -> str:
+ return re.search(rf"{key}=(\S*)", line).group(1)
+
# VERSIONS
def test_versions_pins_a_full_commit_of_the_projects_public_repository(self):
@@ -202,7 +257,7 @@ class Wrapper(unittest.TestCase):
self.assertFalse((self.pkg / "upstream.partial").exists())
def test_fetch_without_git_says_so(self):
- bin_ = self.tmp / "bin"
+ bin_ = self.tmp / "bin-no-git"
bin_.mkdir()
for tool in ("bash", "cat"):
(bin_ / tool).symlink_to(shutil.which(tool))
@@ -213,30 +268,56 @@ class Wrapper(unittest.TestCase):
# setup.sh
- def test_setup_fetches_then_runs_the_projects_own_setup_in_its_checkout(self):
+ def test_setup_fetches_then_runs_the_projects_own_setup_in_its_checkout_with_uv_on_path(self):
+ uv = self.uv()
run = self.run_script("setup.sh", cwd=self.tmp)
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
lines = run.stdout.splitlines()
- self.assertEqual(len(lines), 3, lines)
+ self.assertEqual(len(lines), 4, lines)
self.assertIn(f"ok {NAME} @ {self.upstream.first[:12]} fetched", lines[1])
self.assertTrue(lines[2].startswith("setup "), lines)
- self.assertEqual(self.real(Path(self.dsh_dir(lines[2]))), self.real(self.pkg / "upstream"))
+ self.assertEqual(lines[3], " loading cadgen and OpenCascade once (the first load is slow, the rest are not)")
+ self.assertEqual(self.real(Path(self.field(lines[2], "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.field(lines[2], "uv"), str(uv))
self.assertTrue(lines[2].endswith(f" pwd={self.real(self.pkg)}"), lines)
- def dsh_dir(self, line: str) -> str:
- return re.search(r"dsh=(\S+)", line).group(1)
+ def test_setup_on_a_machine_without_uv_hands_the_project_the_pinned_one(self):
+ uv = self.fetched_uv()
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
+ self.assertEqual(self.field(run.stdout.splitlines()[-2], "uv"), str(uv))
+
+ def test_setup_loads_the_projects_cad_stack_once_so_the_doctor_does_not_pay_for_it(self):
+ self.uv()
+ self.fake_fetched(versions_value_for(self.pkg))
+ calls = self.tmp / "python.log"
+ self.stub("python", f'echo "$*" >> "{calls}"; exit "${{IMPORT_EXIT:-0}}"', where=self.pkg / "upstream" / ".venv" / "bin")
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 0, run.stderr)
+ self.assertEqual(calls.read_text(), "-c import cadgen, build123d\n")
+ # an import that fails is the doctor's to report, not a failed install
+ self.assertEqual(self.run_script("setup.sh", IMPORT_EXIT="1").returncode, 0)
+
+ def test_setup_without_uv_and_no_way_to_fetch_it_is_a_miss_before_the_fetch(self):
+ run = self.run_script("setup.sh")
+ self.assertEqual(run.returncode, 1)
+ self.assertRegex(run.stdout.splitlines()[-1], rf"^miss could not download https://github\.com/astral-sh/uv/releases/download/{re.escape(UV_VERSION)}/")
+ self.assertFalse((self.pkg / "upstream").exists())
def test_setup_fails_with_the_projects_setup(self):
+ self.uv()
run = self.run_script("setup.sh", STUB_EXIT="3")
self.assertEqual(run.returncode, 3)
def test_setup_stops_when_the_fetch_fails(self):
+ self.uv()
self.pin("0123456789abcdef0123456789abcdef01234567")
run = self.run_script("setup.sh")
self.assertNotEqual(run.returncode, 0)
self.assertNotIn("setup dsh=", run.stdout)
def test_setup_says_when_the_project_has_no_setup(self):
+ self.uv()
self.fake_fetched(versions_value_for(self.pkg), scripts=("doctor.sh",))
run = self.run_script("setup.sh")
self.assertEqual(run.returncode, 1)
@@ -252,17 +333,27 @@ class Wrapper(unittest.TestCase):
self.assertEqual(run.returncode, 1)
self.assertEqual(run.stdout, f"miss {NAME} is not fetched — run toolchain/setup.sh\n")
- def test_doctor_at_the_pin_hands_over_to_the_projects_doctor(self):
+ def test_doctor_at_the_pin_hands_over_to_the_projects_doctor_with_the_uv_and_node_setup_used(self):
self.fake_fetched(self.upstream.first)
+ uv = self.fetched_uv()
+ node = self.harness_node()
run = self.run_script("doctor.sh", cwd=self.tmp)
self.assertEqual(run.returncode, 0, run.stderr)
ok, project = run.stdout.splitlines()
self.assertEqual(ok, f"ok {NAME} @ {self.upstream.first[:12]}")
self.assertTrue(project.startswith("doctor "), project)
- self.assertEqual(self.real(Path(self.dsh_dir(project))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.real(Path(self.field(project, "dsh"))), self.real(self.pkg / "upstream"))
+ self.assertEqual((self.field(project, "uv"), self.field(project, "node")), (str(uv), str(node)))
self.assertTrue(project.endswith(f"pwd={self.real(self.pkg)}"), project)
self.assertEqual(self.run_script("doctor.sh", STUB_EXIT="4").returncode, 4)
+ def test_doctor_on_a_machine_with_neither_leaves_saying_so_to_the_project(self):
+ self.fake_fetched(self.upstream.first)
+ run = self.run_script("doctor.sh")
+ self.assertEqual(run.returncode, 0, run.stderr)
+ ok, project = run.stdout.splitlines() # nothing from runtimes.sh: the project's own lines say it
+ self.assertEqual((self.field(project, "uv"), self.field(project, "node")), ("", ""))
+
def test_doctor_warns_when_the_copy_is_behind_the_pin(self):
self.fake_fetched(self.upstream.first)
self.pin(self.upstream.second)
@@ -287,7 +378,7 @@ class Wrapper(unittest.TestCase):
run = self.run_script("init-workspace.sh", cwd=ws)
self.assertEqual(run.returncode, 0, run.stderr)
self.assertTrue(run.stdout.startswith("init-workspace "), run.stdout)
- self.assertEqual(self.real(Path(self.dsh_dir(run.stdout))), self.real(self.pkg / "upstream"))
+ self.assertEqual(self.real(Path(self.field(run.stdout, "dsh"))), self.real(self.pkg / "upstream"))
self.assertTrue(run.stdout.rstrip().endswith(f"pwd={self.real(ws)}"), run.stdout)
self.assertEqual(self.run_script("init-workspace.sh", cwd=ws, STUB_EXIT="5").returncode, 5)
diff --git a/store/agents/blender/toolchain/runtimes.sh b/store/agents/blender/toolchain/runtimes.sh
index 6c8556043..4e21942d0 100644
--- a/store/agents/blender/toolchain/runtimes.sh
+++ b/store/agents/blender/toolchain/runtimes.sh
@@ -6,8 +6,8 @@
# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
# has failed its install. So:
#
-# harness_node 18 node >= 18 on PATH: the machine's own, else the Node Harness itself
-# runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
# fetched into ~/.harness/runtime
# harness_venv DIR 3.12 [MIN [BELOW]]
@@ -27,6 +27,10 @@
# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
HARNESS_UV_VERSION=0.12.15
HARNESS_MICROMAMBA_VERSION=2.9.0-0
@@ -67,15 +71,20 @@ _harness_node_at_least() {
}
harness_node() {
- local min="${1:-18}" recorded
- _harness_node_at_least "$min" && return 0
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
_harness_node_at_least "$min" && return 0
- echo "miss node >= $min — this machine's newest is $(node -v), Harness's own; update Harness"
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
return 1
fi
+ [ "$own" = 1 ] && return 0
echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
return 1
}
diff --git a/store/tools/runtimes.sh b/store/tools/runtimes.sh
index 6c8556043..4e21942d0 100644
--- a/store/tools/runtimes.sh
+++ b/store/tools/runtimes.sh
@@ -6,8 +6,8 @@
# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
# has failed its install. So:
#
-# harness_node 18 node >= 18 on PATH: the machine's own, else the Node Harness itself
-# runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
# fetched into ~/.harness/runtime
# harness_venv DIR 3.12 [MIN [BELOW]]
@@ -27,6 +27,10 @@
# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
HARNESS_UV_VERSION=0.12.15
HARNESS_MICROMAMBA_VERSION=2.9.0-0
@@ -67,15 +71,20 @@ _harness_node_at_least() {
}
harness_node() {
- local min="${1:-18}" recorded
- _harness_node_at_least "$min" && return 0
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
_harness_node_at_least "$min" && return 0
- echo "miss node >= $min — this machine's newest is $(node -v), Harness's own; update Harness"
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
return 1
fi
+ [ "$own" = 1 ] && return 0
echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
return 1
}
diff --git a/store/tools/test_runtimes.py b/store/tools/test_runtimes.py
index e3bc888cb..89f578df1 100644
--- a/store/tools/test_runtimes.py
+++ b/store/tools/test_runtimes.py
@@ -166,10 +166,32 @@ class Node(unittest.TestCase):
def test_the_machines_own_node_when_new_enough(self):
box = Box(self)
box.stub("node", NODE)
+ box.stub("npm")
self.runtime_node(box, "22.23.2")
r = box.run('harness_node 18 && command -v node', NODE_VERSION="20.1.0")
self.assertEqual((r.returncode, r.stdout.strip()), (0, str(box.bin / "node")), r.stderr)
+ def test_a_node_without_npm_borrows_harness_node_and_its_npm(self):
+ box = Box(self)
+ box.stub("node", NODE)
+ node = self.runtime_node(box, "22.23.2")
+ box.stub("npm", where=node.parent)
+ r = box.run('harness_node 18 && command -v node && command -v npm', NODE_VERSION="20.1.0")
+ self.assertEqual(lines(r), [str(node), str(node.parent / "npm")], r.stderr)
+
+ def test_a_node_without_npm_is_kept_when_harness_node_is_older(self):
+ box = Box(self)
+ box.stub("node", NODE)
+ self.runtime_node(box, "18.0.0")
+ r = box.run('harness_node 20 && command -v node && echo "$PATH"', NODE_VERSION="22.1.0")
+ self.assertEqual(lines(r), [str(box.bin / "node"), str(box.bin)], r.stderr)
+
+ def test_a_node_without_npm_and_no_harness_node_still_serves(self):
+ box = Box(self)
+ box.stub("node", NODE)
+ r = box.run('harness_node 18 && command -v node', NODE_VERSION="20.1.0")
+ self.assertEqual((r.returncode, r.stdout.strip()), (0, str(box.bin / "node")), r.stderr)
+
def test_minor_versions_count(self):
box = Box(self)
box.stub("node", NODE)
@@ -221,6 +243,18 @@ class Node(unittest.TestCase):
self.assertEqual(box.run(f"_harness_node_at_least {major - 1}.999").returncode, 0)
+class PythonInstallDir(unittest.TestCase):
+ def test_uv_pythons_live_in_the_harness_runtime(self):
+ box = Box(self)
+ (box.bin / "sh").symlink_to("/bin/sh")
+ self.assertEqual(box.run('echo "$UV_PYTHON_INSTALL_DIR"; sh -c \'echo "$UV_PYTHON_INSTALL_DIR"\'').stdout.splitlines(),
+ [f"{box.runtime}/python"] * 2, "exported, so uv and upstream setups see it")
+
+ def test_a_dir_the_person_chose_is_kept(self):
+ box = Box(self)
+ self.assertEqual(box.run('echo "$UV_PYTHON_INSTALL_DIR"', UV_PYTHON_INSTALL_DIR="/opt/pythons").stdout.strip(), "/opt/pythons")
+
+
class Uv(unittest.TestCase):
def test_uv_on_path_is_used_as_it_is(self):
box = Box(self)
diff --git a/store/viewers/cad-viewer/README.md b/store/viewers/cad-viewer/README.md
index f1ad8031e..10744a159 100644
--- a/store/viewers/cad-viewer/README.md
+++ b/store/viewers/cad-viewer/README.md
@@ -15,7 +15,8 @@ This package is a viewer, not an agent: it has no engine, no workspace and no ve
never a tile. It is installed once per machine and shared by every harness that uses it.
- `harness.json` — the manifest (spec 1.1, `kind: viewer`).
-- `setup.sh` — one venv with the pinned `cadgen` (see `CADGEN_VERSION`); pulls OpenCascade.
+- `setup.sh` — one venv with the pinned `cadgen` (see `CADGEN_VERSION`); pulls OpenCascade. The venv is on
+ Python 3.12, which uv downloads when the machine has none (`runtimes.sh`, the store's shared copy).
- `doctor.sh` — can this machine run it.
- `viewer.sh` — `cadgen viewer --host 127.0.0.1 --port $HARNESS_VIEWER_PORT` in the workspace. The
viewer already follows the agent: a re-export of the open file redraws in place and keeps the
diff --git a/store/viewers/cad-viewer/runtimes.sh b/store/viewers/cad-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/cad-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/cad-viewer/setup.sh b/store/viewers/cad-viewer/setup.sh
index 914305671..e81739bb9 100755
--- a/store/viewers/cad-viewer/setup.sh
+++ b/store/viewers/cad-viewer/setup.sh
@@ -1,22 +1,22 @@
#!/usr/bin/env bash
# Runs once at install, cwd = this directory. Makes the one venv the viewer runs from: the pinned
# `cadgen` release, which carries the CAD Viewer's server and built client (no Node at run time).
-# `cadquery-ocp` comes with it — that is the OpenCascade the viewer tessellates STEP with.
+# `cadquery-ocp` comes with it — that is the OpenCascade the viewer tessellates STEP with. cadgen wants
+# Python 3.11+ and cadquery-ocp has wheels for 3.11–3.14, so the venv is on 3.12 whatever this machine
+# has (uv downloads it when it is not here); a venv already on 3.11–3.14 is kept.
set -euo pipefail
cd "$(dirname "$0")"
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
CADGEN_VERSION="$(cat CADGEN_VERSION)"
-PY=""
-for candidate in python3.13 python3.12 python3.11 python3; do
- if command -v "$candidate" >/dev/null 2>&1 && "$candidate" -c 'import sys; raise SystemExit(0 if sys.version_info >= (3, 11) else 1)' 2>/dev/null; then
- PY="$candidate"; break
- fi
-done
-[ -n "$PY" ] || { echo "miss python 3.11+ (brew install [email protected])"; exit 1; }
-echo "ok $($PY --version)"
-if [ ! -x .venv/bin/python ]; then "$PY" -m venv .venv; fi
-.venv/bin/python -m pip install --quiet --upgrade pip >/dev/null 2>&1 || true
+harness_venv .venv 3.12 3.11 3.15 || exit 1
echo " installing cadgen $CADGEN_VERSION (this pulls OpenCascade; a few minutes the first time)"
-.venv/bin/python -m pip install --quiet "cadgen==${CADGEN_VERSION}"
+harness_pip .venv "cadgen==${CADGEN_VERSION}"
echo "ok $(.venv/bin/cadgen --version 2>/dev/null || echo "cadgen $CADGEN_VERSION")"
+# The doctor's check, once here: the first load of a fresh OpenCascade is slow on macOS (every new
+# library is scanned once, minutes on a busy machine), and setup has the time a doctor or a pane has not.
+echo " loading OpenCascade once"
+.venv/bin/python -c 'import build123d' || { echo "miss build123d (OpenCascade) does not load in .venv"; exit 1; }
+echo "ok build123d + OpenCascade"
# The client the pane serves (pane_client.py): the bundled one with its fetches kept on loopback.
if [ -n "$(.venv/bin/python pane_client.py 2>/dev/null)" ]; then echo "ok pane client"; else echo "warn pane client not made — the pane serves the bundled client"; fi
diff --git a/store/viewers/cad-viewer/test/test_scripts.py b/store/viewers/cad-viewer/test/test_scripts.py
index c1873cd4a..91163bd69 100644
--- a/store/viewers/cad-viewer/test/test_scripts.py
+++ b/store/viewers/cad-viewer/test/test_scripts.py
@@ -1,8 +1,9 @@
-"""setup.sh, doctor.sh and viewer.sh, run against a copy of this package in a temp directory (its name
-has a space, as a home directory may) with every command they call stubbed: the Python candidates,
-the venv's python (which runs the real pane_client.py against test/fake_cadgen.py), and cadgen.
-PATH holds only the stubs and the few system tools the scripts use, so nothing on this machine —
-no real Python, no network, no pip — is touched.
+"""setup.sh, doctor.sh and viewer.sh, run against a scratch copy of this package in a temp directory
+(its name has a space, as a home directory may) with every command they call stubbed: uv (which makes
+a venv of stubs and "installs" cadgen), the venv's python (which runs the real pane_client.py against
+test/fake_cadgen.py), and cadgen. PATH holds only the stubs and the few system tools the scripts use, so
+nothing on this machine — no real Python, no network, no uv — is touched. What runtimes.sh does beyond
+that is store/tools' business.
python3 -m unittest discover -s test
"""
@@ -10,7 +11,6 @@ from __future__ import annotations
import os
import shutil
-import stat
import subprocess
import sys
import tempfile
@@ -23,16 +23,24 @@ sys.path.insert(0, str(HERE))
import fake_cadgen # noqa: E402
+# A bash line tracer (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT) is passed through when set.
+TRACER = {k: os.environ[k] for k in ("BASH_ENV", "SHCOV_OUT") if k in os.environ}
+# Real commands linked into bin/: a stub is never written over one of these (it would go through the link).
+LINKED = ("bash", "dirname", "basename", "cat", "head", "mkdir", "mktemp", "rm")
+
+# The venv's python: its version and runtimes.sh's keep-or-replace probe (STUB_VENV_OUT_OF_RANGE), the
+# build123d import (STUB_NO_BUILD123D); anything else is the real Python, for pane_client.py.
VENV_PYTHON = """#!/bin/sh
+case "$1" in
+ --version) echo "Python ${STUB_VENV_PY:-3.12.11}"; exit 0 ;;
+ -c)
+ case "$2" in
+ *version_info*) printf 'python version probe %s %s\\n' "$3" "$4" >> "$STUB_LOG"; [ -z "$STUB_VENV_OUT_OF_RANGE" ] || exit 1; exit 0 ;;
+ esac
+ printf 'python %s\\n' "$*" >> "$STUB_LOG"
+ [ -z "$STUB_NO_BUILD123D" ] || exit 1; exit 0 ;;
+esac
printf 'python %s\\n' "$*" >> "$STUB_LOG"
-if [ "$1" = "-m" ] && [ "$2" = "pip" ]; then
- case " $* " in
- *" --upgrade pip "*) [ -z "$STUB_PIP_UPGRADE_FAIL" ] || exit 1 ;;
- *) [ -z "$STUB_PIP_INSTALL_FAIL" ] || { echo "ERROR: No matching distribution found for cadgen" >&2; exit 1; } ;;
- esac
- exit 0
-fi
-if [ "$1" = "-c" ]; then [ -z "$STUB_NO_BUILD123D" ] || exit 1; exit 0; fi
exec "$REAL_PYTHON" "$@"
"""
@@ -50,44 +58,41 @@ case "$1" in
esac
"""
-# A python3.x on PATH: old enough to be skipped, or new enough to make the venv.
-CANDIDATE = """#!/bin/sh
+# uv on PATH: `uv venv … DIR` makes a venv of the stubs above; `uv pip install` succeeds unless told not to.
+UV = """#!/bin/sh
+printf 'uv %s\\n' "$*" >> "$STUB_LOG"
case "$1" in
- -c) exit {check} ;;
- --version) echo "Python {version}" ;;
- -m)
- printf '%s %s\\n' "$(basename "$0")" "$*" >> "$STUB_LOG"
- /bin/mkdir -p .venv/bin
- /bin/cp "$STUBS/venv-python" .venv/bin/python
- /bin/cp "$STUBS/venv-cadgen" .venv/bin/cadgen ;;
+ venv)
+ for last in "$@"; do :; done
+ /bin/mkdir -p "$last/bin"
+ /bin/cp "$STUBS/venv-python" "$last/bin/python"
+ /bin/cp "$STUBS/venv-cadgen" "$last/bin/cadgen" ;;
+ pip) [ -z "$STUB_PIP_INSTALL_FAIL" ] || { echo "error: No solution found when resolving dependencies: cadgen" >&2; exit 1; } ;;
esac
"""
-def executable(path: Path, text: str) -> Path:
- path.write_text(text, encoding="utf-8")
- path.chmod(path.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
- return path
-
-
class ScriptTest(unittest.TestCase):
def setUp(self) -> None:
self.root = Path(tempfile.mkdtemp(prefix="cad-viewer scripts "))
self.addCleanup(shutil.rmtree, self.root, True)
self.pkg = self.root / "cad viewer"
self.pkg.mkdir()
- for name in ("setup.sh", "doctor.sh", "viewer.sh", "pane_client.py", "CADGEN_VERSION"):
+ # The scripts are linked, not copied, so a line tracer maps back to the source; pane_client.py is
+ # copied, since it writes its client next to itself.
+ for name in ("setup.sh", "doctor.sh", "viewer.sh", "runtimes.sh"):
+ (self.pkg / name).symlink_to(PACKAGE / name)
+ for name in ("pane_client.py", "CADGEN_VERSION"):
shutil.copy2(PACKAGE / name, self.pkg / name)
- self.stubs = self.root / "stubs"
- self.stubs.mkdir()
- executable(self.stubs / "venv-python", VENV_PYTHON)
- executable(self.stubs / "venv-cadgen", CADGEN)
self.bin = self.root / "bin"
self.bin.mkdir()
- for tool in ("bash", "dirname", "basename", "cat", "head"):
+ for tool in LINKED:
found = shutil.which(tool)
assert found, f"{tool} is needed to run the scripts"
(self.bin / tool).symlink_to(found)
+ self.stubs = self.root / "stubs"
+ self.stub("venv-python", VENV_PYTHON, self.stubs)
+ self.stub("venv-cadgen", CADGEN, self.stubs)
self.log = self.root / "calls.log"
self.log.touch()
self.site = fake_cadgen.make_site()
@@ -95,19 +100,28 @@ class ScriptTest(unittest.TestCase):
self.workspace = self.root / "my workspace"
self.workspace.mkdir()
- def candidate(self, name: str, *, ok: bool, version: str) -> None:
- executable(self.bin / name, CANDIDATE.format(check=0 if ok else 1, version=version))
+ def stub(self, name: str, text: str, where: Path | None = None) -> Path:
+ where = where or self.bin
+ assert not (where == self.bin and name in LINKED), f"{name} is a link to the real command"
+ where.mkdir(parents=True, exist_ok=True)
+ path = where / name
+ path.unlink(missing_ok=True) # never write through a link to a real command
+ path.write_text(text, encoding="utf-8")
+ path.chmod(0o755)
+ return path
+
+ def uv(self) -> None:
+ self.stub("uv", UV)
def venv(self) -> None:
"""A venv as setup.sh leaves it."""
- (self.pkg / ".venv" / "bin").mkdir(parents=True)
- shutil.copy2(self.stubs / "venv-python", self.pkg / ".venv" / "bin" / "python")
- shutil.copy2(self.stubs / "venv-cadgen", self.pkg / ".venv" / "bin" / "cadgen")
+ self.stub("python", VENV_PYTHON, self.pkg / ".venv" / "bin")
+ self.stub("cadgen", CADGEN, self.pkg / ".venv" / "bin")
def run_script(self, script: str, *, cadgen_importable: bool = True, **env: str) -> subprocess.CompletedProcess[str]:
base = {
"PATH": str(self.bin), "HOME": str(self.root), "STUB_LOG": str(self.log), "STUBS": str(self.stubs),
- "REAL_PYTHON": sys.executable, "PYTHONDONTWRITEBYTECODE": "1",
+ "REAL_PYTHON": sys.executable, "PYTHONDONTWRITEBYTECODE": "1", **TRACER,
}
if cadgen_importable:
base["PYTHONPATH"] = str(self.site)
@@ -118,53 +132,73 @@ class ScriptTest(unittest.TestCase):
class SetupTest(ScriptTest):
- def test_without_python_311_or_newer_it_says_what_to_install_and_fails(self) -> None:
- self.candidate("python3", ok=False, version="3.9.6")
- result = self.run_script("setup.sh")
- self.assertEqual(result.returncode, 1)
- self.assertEqual(result.stdout, "miss python 3.11+ (brew install [email protected])\n")
- self.assertFalse((self.pkg / ".venv").exists())
-
- def test_a_fresh_install_makes_the_venv_with_the_first_new_enough_python_pins_cadgen_and_makes_the_client(self) -> None:
- self.candidate("python3.12", ok=False, version="3.12.0") # present, but says it is too old: skipped
- self.candidate("python3.11", ok=True, version="3.11.9")
- self.candidate("python3", ok=True, version="3.13.1")
+ def test_a_fresh_install_makes_a_3_12_venv_through_uv_pins_cadgen_and_makes_the_client(self) -> None:
+ self.uv()
result = self.run_script("setup.sh")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.splitlines(), [
- "ok Python 3.11.9",
+ " python 3.12 in .venv (uv downloads it when this machine has none)",
+ "ok Python 3.12.11 in .venv",
" installing cadgen 0.5.1 (this pulls OpenCascade; a few minutes the first time)",
"ok cadgen 0.5.1",
+ " loading OpenCascade once",
+ "ok build123d + OpenCascade",
"ok pane client",
])
- self.assertEqual(self.calls()[:3], [
- "python3.11 -m venv .venv",
- "python -m pip install --quiet --upgrade pip",
- "python -m pip install --quiet cadgen==0.5.1",
+ self.assertEqual([c for c in self.calls() if c.startswith("uv ")], [
+ "uv venv --quiet --seed --python 3.12 --python-preference only-managed .venv",
+ "uv pip install --quiet --python .venv/bin/python cadgen==0.5.1",
])
self.assertTrue((self.pkg / ".pane-client-0.5.1" / ".harness-pane").is_file())
- def test_an_existing_venv_is_kept_a_failed_pip_upgrade_is_tolerated_and_a_missing_client_is_a_warning(self) -> None:
- self.candidate("python3", ok=True, version="3.12.4")
+ def test_an_existing_venv_on_3_11_to_3_14_is_kept_and_a_missing_client_is_a_warning(self) -> None:
+ self.uv()
self.venv()
- result = self.run_script("setup.sh", cadgen_importable=False, STUB_PIP_UPGRADE_FAIL="1", STUB_CADGEN_BROKEN="1")
+ result = self.run_script("setup.sh", cadgen_importable=False, STUB_VENV_PY="3.13.7", STUB_CADGEN_BROKEN="1")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.splitlines(), [
- "ok Python 3.12.4",
+ "ok Python 3.13.7 in .venv",
" installing cadgen 0.5.1 (this pulls OpenCascade; a few minutes the first time)",
"ok cadgen 0.5.1",
+ " loading OpenCascade once",
+ "ok build123d + OpenCascade",
"warn pane client not made — the pane serves the bundled client",
])
- self.assertNotIn("python3 -m venv .venv", self.calls())
+ self.assertIn("python version probe 3.11 3.15", self.calls())
+ self.assertFalse(any(c.startswith("uv venv") for c in self.calls()))
self.assertFalse(any(p.name.startswith(".pane-client") for p in self.pkg.iterdir()))
+ def test_a_venv_on_a_python_cadquery_ocp_has_no_wheel_for_is_replaced(self) -> None:
+ self.uv()
+ self.venv()
+ result = self.run_script("setup.sh", STUB_VENV_OUT_OF_RANGE="1")
+ self.assertEqual(result.returncode, 0, result.stderr)
+ self.assertIn("uv venv --quiet --seed --python 3.12 --python-preference only-managed .venv", self.calls())
+
def test_a_failed_cadgen_install_fails_setup(self) -> None:
- self.candidate("python3.13", ok=True, version="3.13.1")
+ self.uv()
result = self.run_script("setup.sh", STUB_PIP_INSTALL_FAIL="1")
self.assertNotEqual(result.returncode, 0)
- self.assertIn("No matching distribution found for cadgen", result.stderr)
+ self.assertIn("No solution found when resolving dependencies: cadgen", result.stderr)
self.assertNotIn("ok cadgen", result.stdout)
+ def test_an_opencascade_that_does_not_load_fails_setup(self) -> None:
+ self.uv()
+ result = self.run_script("setup.sh", STUB_NO_BUILD123D="1")
+ self.assertEqual(result.returncode, 1)
+ self.assertEqual(result.stdout.splitlines()[-2:], [" loading OpenCascade once", "miss build123d (OpenCascade) does not load in .venv"])
+ self.assertIn("python -c import build123d", self.calls())
+
+ def test_no_uv_and_no_network_is_a_miss_and_no_venv(self) -> None:
+ self.stub("uname", '#!/bin/sh\ncase "$1" in -s) echo Darwin ;; -m) echo arm64 ;; esac\n')
+ self.stub("curl", "#!/bin/sh\nexit 6\n")
+ result = self.run_script("setup.sh", ADAPTER_RUNTIME_DIR=str(self.root / "runtime"))
+ self.assertEqual(result.returncode, 1)
+ self.assertEqual(result.stdout.splitlines()[-1], "miss could not download https://github.com/astral-sh/uv/releases/download/"
+ "0.12.15/uv-aarch64-apple-darwin.tar.gz — check this machine's internet connection")
+ self.assertFalse((self.pkg / ".venv").exists())
+ self.assertNotIn("installing cadgen", result.stdout)
+
class DoctorTest(ScriptTest):
def test_before_setup_both_checks_miss(self) -> None:
diff --git a/store/viewers/doc-viewer/doctor.sh b/store/viewers/doc-viewer/doctor.sh
index bef86869c..7e358dd31 100755
--- a/store/viewers/doc-viewer/doctor.sh
+++ b/store/viewers/doc-viewer/doctor.sh
@@ -1,6 +1,9 @@
#!/usr/bin/env bash
+# The Node setup.sh uses, found the same way (runtimes.sh), then what it installed.
set -u; cd "$(dirname "$0")"
-command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 20 ? 0 : 1)' || { echo "miss node >= 20 on PATH"; exit 1; }
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+harness_node 20 || exit 1
for f in build/pdf.min.mjs build/pdf.worker.min.mjs web/pdf_viewer.mjs web/pdf_viewer.css legacy/build/pdf.min.mjs legacy/web/pdf_viewer.mjs; do
[ -f "node_modules/pdfjs-dist/$f" ] || { echo "miss node_modules/pdfjs-dist/$f — run ./setup.sh"; exit 1; }
done
diff --git a/store/viewers/doc-viewer/runtimes.sh b/store/viewers/doc-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/doc-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/doc-viewer/setup.sh b/store/viewers/doc-viewer/setup.sh
index c624dd5d4..db54011b4 100755
--- a/store/viewers/doc-viewer/setup.sh
+++ b/store/viewers/doc-viewer/setup.sh
@@ -1,10 +1,13 @@
#!/usr/bin/env bash
# Runs once at install, cwd = the install dir: pdf.js into node_modules, from the lockfile. The reader
-# itself (app/, lib/, viewer.mjs) has no build step.
+# itself (app/, lib/, viewer.mjs) has no build step. Node is this machine's, or Harness's own when the
+# machine has none or too old a one (runtimes.sh); npm is the one beside it.
set -euo pipefail
cd "$(dirname "$0")"
-command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 20 ? 0 : 1)' || { echo "miss node >= 20 on PATH"; exit 1; }
-command -v npm >/dev/null 2>&1 || { echo "miss npm on PATH"; exit 1; }
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+harness_node 20 || exit 1
+command -v npm >/dev/null 2>&1 || { echo "miss npm beside node $(node --version)"; exit 1; }
npm ci --silent --no-audit --no-fund
for f in build/pdf.min.mjs build/pdf.worker.min.mjs web/pdf_viewer.mjs web/pdf_viewer.css legacy/build/pdf.min.mjs legacy/web/pdf_viewer.mjs; do
[ -f "node_modules/pdfjs-dist/$f" ] || { echo "miss pdfjs-dist/$f after npm ci"; exit 1; }
diff --git a/store/viewers/doc-viewer/test/scripts.test.mjs b/store/viewers/doc-viewer/test/scripts.test.mjs
index c5a38b8cd..77919b7de 100644
--- a/store/viewers/doc-viewer/test/scripts.test.mjs
+++ b/store/viewers/doc-viewer/test/scripts.test.mjs
@@ -1,9 +1,11 @@
// setup.sh, doctor.sh and viewer.sh, run for real against a scratch copy of the package with a PATH
-// that holds only what they may use: bash and dirname, plus a `node` and an `npm` the test controls
-// (the real node, one too old, or none; an npm that records what it was asked and exits as told).
+// that holds only what they may use: bash, dirname and cat, plus a `node` and an `npm` the test controls
+// (the real node, one too old, or none; an npm that records what it was asked and exits as told). The
+// scripts find Node through runtimes.sh, so each case also says whether Harness's own Node is laid down
+// under HOME (~/.harness/runtime/current-node) — the Node a fresh Mac, with none on PATH, gets.
import assert from 'node:assert/strict'
import { spawnSync } from 'node:child_process'
-import { accessSync, chmodSync, constants, copyFileSync, existsSync, mkdirSync, readFileSync, realpathSync, symlinkSync } from 'node:fs'
+import { accessSync, chmodSync, constants, existsSync, mkdirSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { delimiter, join } from 'node:path'
import { after, test } from 'node:test'
import { cleanup, pkg, put, scratch } from './helpers.mjs'
@@ -12,6 +14,10 @@ after(cleanup)
const PDFJS_FILES = ['build/pdf.min.mjs', 'build/pdf.worker.min.mjs', 'web/pdf_viewer.mjs', 'web/pdf_viewer.css', 'legacy/build/pdf.min.mjs', 'legacy/web/pdf_viewer.mjs']
const APP_FILES = ['app/index.html', 'app/app.js', 'app/app.css', 'lib/workspace.mjs']
+// A bash line tracer's hooks (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT), passed through when set.
+const TRACER = Object.fromEntries(['BASH_ENV', 'SHCOV_OUT'].filter((k) => process.env[k]).map((k) => [k, process.env[k]]))
+// A node whose `-e` version check fails, as Node 18 fails `>= 20`.
+const OLD_NODE = '#!/bin/sh\n[ "$1" = "-e" ] && exit 1\necho "v18.0.0"\n'
/** Where `name` lives on this machine's PATH. */
function tool(name) {
@@ -22,55 +28,95 @@ function tool(name) {
throw new Error(`${name} is not on PATH`)
}
+/** An executable script at dir/name; whatever was there goes first, so nothing is written through a link. */
+function stub(dir, name, body) {
+ const full = join(dir, name)
+ mkdirSync(dir, { recursive: true })
+ rmSync(full, { force: true })
+ writeFileSync(full, body)
+ chmodSync(full, 0o755)
+ return full
+}
+
/**
- * A scratch package holding `script` (copied from this package) and a bin directory for PATH.
+ * A scratch package holding `script` and runtimes.sh (linked from this package, so a line tracer maps
+ * back to them) and a bin directory for PATH.
* node: 'real' | 'old' | 'none'; npm: an exit code, or null for none.
+ * harness: Harness's own Node under HOME — 'real' | 'old' | null — with harnessNpm (an exit code, or null) beside it.
*/
-function sandbox(script, { node = 'real', npm = null } = {}) {
+function sandbox(script, { node = 'real', npm = null, harness = null, harnessNpm = null } = {}) {
const root = scratch({}, 'doc-viewer-sh-')
const dir = join(root, 'pkg')
mkdirSync(dir)
- copyFileSync(join(pkg, script), join(dir, script))
- chmodSync(join(dir, script), 0o755)
+ for (const name of [script, 'runtimes.sh']) symlinkSync(join(pkg, name), join(dir, name))
const bin = join(root, 'bin')
mkdirSync(bin)
- for (const name of ['bash', 'dirname']) symlinkSync(tool(name), join(bin, name))
+ for (const name of ['bash', 'dirname', 'cat']) symlinkSync(tool(name), join(bin, name))
+ const npmStub = (where, code) => stub(where, 'npm', `#!/bin/sh\necho "$*" >> '${join(root, 'npm.log')}'\nexit ${code}\n`)
if (node === 'real') symlinkSync(process.execPath, join(bin, 'node'))
- // A node whose `-e` version check fails, as Node 18 fails `>= 20`.
- if (node === 'old') chmodSync(put(bin, 'node', '#!/bin/sh\n[ "$1" = "-e" ] && exit 1\necho "v18.0.0"\n'), 0o755)
- if (npm !== null) chmodSync(put(bin, 'npm', `#!/bin/sh\necho "$*" >> '${join(root, 'npm.log')}'\nexit ${npm}\n`), 0o755)
+ if (node === 'old') stub(bin, 'node', OLD_NODE)
+ if (npm !== null) npmStub(bin, npm)
+ const home = join(root, 'home')
+ if (harness) {
+ const harnessBin = join(root, 'harness-node', 'bin')
+ mkdirSync(harnessBin, { recursive: true })
+ if (harness === 'real') symlinkSync(process.execPath, join(harnessBin, 'node'))
+ else stub(harnessBin, 'node', OLD_NODE)
+ if (harnessNpm !== null) npmStub(harnessBin, harnessNpm)
+ put(home, '.harness/runtime/current-node', join(harnessBin, 'node'))
+ }
const npmCalls = () => (existsSync(join(root, 'npm.log')) ? readFileSync(join(root, 'npm.log'), 'utf8').split('\n').filter(Boolean) : [])
const run = (env = {}) => {
// From another directory: each script finds its own.
- const r = spawnSync(join(dir, script), [], { cwd: root, env: { PATH: bin, HOME: root, ...env }, encoding: 'utf8' })
+ const r = spawnSync(join(dir, script), [], { cwd: root, env: { PATH: bin, HOME: home, ...TRACER, ...env }, encoding: 'utf8' })
return { code: r.status, out: r.stdout, err: r.stderr }
}
- return { root, dir, bin, run, npmCalls }
+ return { root, dir, bin, home, run, npmCalls }
}
+const noHarnessNode = (box) => `miss node >= 20, and Harness's own Node is not in ${box.home}/.harness/runtime — run \`harness start\` once to lay it down\n`
+
function installPdfjs(dir, { skip = null, version = '9.9.9' } = {}) {
for (const f of PDFJS_FILES) if (f !== skip) put(dir, `node_modules/pdfjs-dist/${f}`, '')
put(dir, 'node_modules/pdfjs-dist/package.json', JSON.stringify({ name: 'pdfjs-dist', version }))
}
-test('the scripts are executable in the package', () => {
+test('the scripts are executable in the package, and runtimes.sh is beside them', () => {
for (const script of ['setup.sh', 'doctor.sh', 'viewer.sh']) accessSync(join(pkg, script), constants.X_OK)
+ accessSync(join(pkg, 'runtimes.sh'), constants.R_OK)
})
-test('setup: no node, or one older than 20, is a miss before npm runs', () => {
+test('setup: no node, or one older than 20, and no Node of Harness\'s own is a miss before npm runs', () => {
for (const node of ['none', 'old']) {
const box = sandbox('setup.sh', { node, npm: 0 })
const r = box.run()
assert.equal(r.code, 1, node)
- assert.equal(r.out, 'miss node >= 20 on PATH\n', node)
+ assert.equal(r.out, noHarnessNode(box), node)
assert.deepEqual(box.npmCalls(), [], node)
}
})
-test('setup: no npm is a miss', () => {
+test('setup: Harness\'s own Node too old as well says to update Harness', () => {
+ const box = sandbox('setup.sh', { node: 'old', npm: 0, harness: 'old' })
+ const r = box.run()
+ assert.equal(r.code, 1)
+ assert.equal(r.out, "miss node >= 20 — this machine's newest is v18.0.0, Harness's own; update Harness\n")
+ assert.deepEqual(box.npmCalls(), [])
+})
+
+test('setup: no node on PATH is no miss — Harness\'s own Node and the npm beside it install pdf.js', () => {
+ const box = sandbox('setup.sh', { node: 'none', harness: 'real', harnessNpm: 0 })
+ installPdfjs(box.dir)
+ const r = box.run()
+ assert.equal(r.code, 0, r.err)
+ assert.equal(r.out, 'ok pdf.js 9.9.9 (viewer components, modern and legacy builds)\n')
+ assert.deepEqual(box.npmCalls(), ['ci --silent --no-audit --no-fund'])
+})
+
+test('setup: no npm beside the node is a miss', () => {
const r = sandbox('setup.sh', { npm: null }).run()
assert.equal(r.code, 1)
- assert.equal(r.out, 'miss npm on PATH\n')
+ assert.equal(r.out, `miss npm beside node ${process.version}\n`)
})
test('setup: a failing npm ci stops setup with its exit code', () => {
@@ -98,11 +144,12 @@ test('setup: npm ci from the lockfile, in the package, then the version installe
assert.deepEqual(box.npmCalls(), ['ci --silent --no-audit --no-fund'])
})
-test('doctor: no node, or one older than 20, is a miss', () => {
+test('doctor: no node, or one older than 20, and no Node of Harness\'s own is a miss', () => {
for (const node of ['none', 'old']) {
- const r = sandbox('doctor.sh', { node }).run()
+ const box = sandbox('doctor.sh', { node })
+ const r = box.run()
assert.equal(r.code, 1, node)
- assert.equal(r.out, 'miss node >= 20 on PATH\n', node)
+ assert.equal(r.out, noHarnessNode(box), node)
}
})
@@ -124,13 +171,15 @@ test('doctor: a reader file missing means the package is incomplete', () => {
assert.equal(r.out, 'miss app/app.css — the package is incomplete\n')
})
-test('doctor: everything there is one ok line', () => {
- const box = sandbox('doctor.sh')
- installPdfjs(box.dir, { version: '6.3.289' })
- for (const f of APP_FILES) put(box.dir, f, '')
- const r = box.run()
- assert.equal(r.code, 0, r.err)
- assert.equal(r.out, 'ok pdf.js 6.3.289 and the reader\n')
+test('doctor: everything there is one ok line, on this machine\'s Node or on Harness\'s own', () => {
+ for (const [node, harness] of [['real', null], ['none', 'real']]) {
+ const box = sandbox('doctor.sh', { node, harness })
+ installPdfjs(box.dir, { version: '6.3.289' })
+ for (const f of APP_FILES) put(box.dir, f, '')
+ const r = box.run()
+ assert.equal(r.code, 0, r.err)
+ assert.equal(r.out, 'ok pdf.js 6.3.289 and the reader\n', node)
+ }
})
test('viewer.sh: the port and the workspace are required', () => {
@@ -145,7 +194,7 @@ test('viewer.sh: the port and the workspace are required', () => {
test('viewer.sh: runs node on the viewer.mjs beside it, from anywhere', () => {
const box = sandbox('viewer.sh', { node: 'none' })
- chmodSync(put(box.bin, 'node', '#!/bin/sh\necho "node $*"\n'), 0o755)
+ stub(box.bin, 'node', '#!/bin/sh\n[ "$1" = "-e" ] && exit 0\necho "node $*"\n')
const r = box.run({ HARNESS_VIEWER_PORT: '4000', HARNESS_WORKSPACE: box.root })
assert.equal(r.code, 0, r.err)
const [word, file] = r.out.trim().split(' ')
@@ -153,3 +202,16 @@ test('viewer.sh: runs node on the viewer.mjs beside it, from anywhere', () => {
assert.equal(realpathSync(join(file, '..')), realpathSync(box.dir))
assert.match(file, /\/viewer\.mjs$/)
})
+
+test('viewer.sh: a login shell with no node on PATH runs Harness\'s own; with none at all, a miss and no server', () => {
+ const box = sandbox('viewer.sh', { node: 'none', harness: 'real' })
+ writeFileSync(join(box.dir, 'viewer.mjs'), 'console.log(`viewer on ${process.execPath}`)\n')
+ const env = { HARNESS_VIEWER_PORT: '4000', HARNESS_WORKSPACE: box.root }
+ const r = box.run(env)
+ assert.equal(r.code, 0, r.err)
+ assert.equal(r.out, `viewer on ${process.execPath}\n`)
+ const bare = sandbox('viewer.sh', { node: 'none' })
+ const none = bare.run(env)
+ assert.equal(none.code, 1)
+ assert.equal(none.out, noHarnessNode(bare))
+})
diff --git a/store/viewers/doc-viewer/viewer.sh b/store/viewers/doc-viewer/viewer.sh
index 9a40a19cc..eef982344 100755
--- a/store/viewers/doc-viewer/viewer.sh
+++ b/store/viewers/doc-viewer/viewer.sh
@@ -1,5 +1,10 @@
#!/usr/bin/env bash
-# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE.
+# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE, through a login shell
+# whose PATH may hold no node: runtimes.sh finds this machine's or Harness's own.
set -euo pipefail
: "${HARNESS_VIEWER_PORT:?}"; : "${HARNESS_WORKSPACE:?}"
-exec node "$(cd "$(dirname "$0")" && pwd)/viewer.mjs"
+here="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/runtimes.sh"
+harness_node 20 || exit 1
+exec node "$here/viewer.mjs"
diff --git a/store/viewers/model-viewer/doctor.sh b/store/viewers/model-viewer/doctor.sh
index b4d9afe25..f3a0e4269 100755
--- a/store/viewers/model-viewer/doctor.sh
+++ b/store/viewers/model-viewer/doctor.sh
@@ -1,8 +1,10 @@
#!/usr/bin/env bash
set -u; cd "$(dirname "$0")"
fail=0
-# The same Node setup.sh requires: a doctor that passes where setup fails helps no one.
-if command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 18 ? 0 : 1)'; then echo "ok node $(node --version)"; else echo "miss node >= 18 on PATH"; fail=1; fi
+# The same Node setup.sh uses, found the same way: a doctor that passes where setup fails helps no one.
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+if harness_node 18; then echo "ok node $(node --version)"; else fail=1; fi
if [ -f node_modules/three/build/three.module.js ] && [ -f node_modules/three/examples/jsm/loaders/GLTFLoader.js ]; then
echo "ok three $(node -p "require('./node_modules/three/package.json').version")"
else
diff --git a/store/viewers/model-viewer/runtimes.sh b/store/viewers/model-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/model-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/model-viewer/setup.sh b/store/viewers/model-viewer/setup.sh
index b2784e5cc..ef8cfe00e 100755
--- a/store/viewers/model-viewer/setup.sh
+++ b/store/viewers/model-viewer/setup.sh
@@ -1,10 +1,14 @@
#!/usr/bin/env bash
# Runs once at install, cwd = the install dir: three.js into node_modules, from the lockfile, then the
# server's smoke test (npm test is the whole suite; install needs only the smoke test). Nothing here touches the machine outside this directory.
+# Node is this machine's, or Harness's own when the machine has none or too old a one (runtimes.sh);
+# npm is the one beside it.
set -euo pipefail
cd "$(dirname "$0")"
-command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 18 ? 0 : 1)' || { echo "miss node >= 18 on PATH"; exit 1; }
-command -v npm >/dev/null 2>&1 || { echo "miss npm on PATH"; exit 1; }
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+harness_node 18 || exit 1
+command -v npm >/dev/null 2>&1 || { echo "miss npm beside node $(node --version)"; exit 1; }
npm ci --silent --no-audit --no-fund
[ -f node_modules/three/build/three.module.js ] || { echo "miss three after npm ci"; exit 1; }
[ -f node_modules/three/examples/jsm/loaders/GLTFLoader.js ] || { echo "miss three's glTF loader after npm ci"; exit 1; }
diff --git a/store/viewers/model-viewer/test/scripts.test.mjs b/store/viewers/model-viewer/test/scripts.test.mjs
index 8dabf9070..b9231c2bd 100644
--- a/store/viewers/model-viewer/test/scripts.test.mjs
+++ b/store/viewers/model-viewer/test/scripts.test.mjs
@@ -1,11 +1,13 @@
// setup.sh, doctor.sh and viewer.sh, run for real against a scratch copy of the package with only the
// tools each case allows on PATH: Node (the real one, one that is too old, or none), and an npm stub
-// that installs as much as the case says and logs what it was asked.
+// that installs as much as the case says and logs what it was asked. The scripts find Node through
+// runtimes.sh, so a case can also lay down Harness's own Node under HOME (~/.harness/runtime/current-node)
+// — the Node a fresh Mac, with none on PATH, gets — with npm beside it or not.
//
// npm test
import assert from 'node:assert/strict'
import { spawn, spawnSync } from 'node:child_process'
-import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { after, describe, test } from 'node:test'
@@ -13,6 +15,8 @@ import { freePort, packageDir, sleep } from './helpers.mjs'
const roots = []
after(() => { for (const root of roots) rmSync(root, { recursive: true, force: true }) })
+// A bash line tracer's hooks (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT), passed through when set.
+const TRACER = Object.fromEntries(['BASH_ENV', 'SHCOV_OUT'].filter((k) => process.env[k]).map((k) => [k, process.env[k]]))
function systemTool(name) {
const found = ['/bin', '/usr/bin'].map((dir) => join(dir, name)).find((path) => existsSync(path))
@@ -20,6 +24,16 @@ function systemTool(name) {
return found
}
+/** An executable script at dir/name; whatever was there goes first, so nothing is written through a link. */
+function stub(dir, name, body) {
+ const full = join(dir, name)
+ mkdirSync(dir, { recursive: true })
+ rmSync(full, { force: true })
+ writeFileSync(full, body)
+ chmodSync(full, 0o755)
+ return full
+}
+
const NPM = `#!/bin/sh
echo "$*" >> "$NPM_LOG"
case "$1" in
@@ -37,32 +51,46 @@ case "$1" in
esac
exit 0
`
+const OLD_NODE = '#!/bin/sh\n[ "$1" = "-e" ] && exit 1\necho v16.20.2\n'
/**
- * `script` copied into an empty package folder, with a bin folder holding bash and dirname plus
- * `node` ('real', 'old' or none) and the npm stub (or none). `files` are created in the package.
+ * `script` and runtimes.sh linked into an empty package folder (linked, so a line tracer maps back to
+ * them), with a bin folder holding bash, dirname and cat plus `node` ('real', 'old' or none) and the npm
+ * stub (or none). harness: Harness's own Node under HOME ('real', 'old' or none), with the npm stub
+ * beside it when harnessNpm. `files` are created in the package.
*/
-function sandbox(script, { node = 'real', npm = true, files = [] } = {}) {
+function sandbox(script, { node = 'real', npm = true, harness = null, harnessNpm = true, files = [] } = {}) {
const root = mkdtempSync(join(tmpdir(), 'model-viewer-scripts-'))
roots.push(root)
- const pkg = join(root, 'pkg'), bin = join(root, 'bin')
- mkdirSync(pkg); mkdirSync(bin)
- copyFileSync(join(packageDir, script), join(pkg, script))
- chmodSync(join(pkg, script), 0o755)
- for (const tool of ['bash', 'dirname']) symlinkSync(systemTool(tool), join(bin, tool))
+ const pkg = join(root, 'pkg'), bin = join(root, 'bin'), home = join(root, 'home')
+ mkdirSync(pkg); mkdirSync(bin); mkdirSync(home)
+ for (const name of [script, 'runtimes.sh']) symlinkSync(join(packageDir, name), join(pkg, name))
+ for (const tool of ['bash', 'dirname', 'cat']) symlinkSync(systemTool(tool), join(bin, tool))
if (node === 'real') symlinkSync(process.execPath, join(bin, 'node'))
- if (node === 'old') { writeFileSync(join(bin, 'node'), '#!/bin/sh\nexit 1\n'); chmodSync(join(bin, 'node'), 0o755) }
- if (npm) { writeFileSync(join(bin, 'npm'), NPM); chmodSync(join(bin, 'npm'), 0o755) }
+ if (node === 'old') stub(bin, 'node', OLD_NODE)
+ if (npm) stub(bin, 'npm', NPM)
+ if (harness) {
+ const harnessBin = join(root, 'harness-node', 'bin')
+ mkdirSync(harnessBin, { recursive: true })
+ if (harness === 'real') symlinkSync(process.execPath, join(harnessBin, 'node'))
+ else stub(harnessBin, 'node', OLD_NODE)
+ if (harnessNpm) stub(harnessBin, 'npm', NPM)
+ mkdirSync(join(home, '.harness', 'runtime'), { recursive: true })
+ writeFileSync(join(home, '.harness', 'runtime', 'current-node'), join(harnessBin, 'node'))
+ }
for (const [rel, body] of files) { mkdirSync(dirname(join(pkg, rel)), { recursive: true }); writeFileSync(join(pkg, rel), body) }
const log = join(root, 'npm.log')
const run = (env = {}) => {
- const r = spawnSync(join(pkg, script), [], { env: { PATH: bin, NPM_LOG: log, ...env }, encoding: 'utf8' })
+ const r = spawnSync(join(pkg, script), [], { env: { PATH: bin, HOME: home, NPM_LOG: log, ...TRACER, ...env }, encoding: 'utf8' })
const npmCalls = existsSync(log) ? readFileSync(log, 'utf8').trim().split('\n') : []
return { status: r.status, lines: r.stdout.trim().split('\n').filter(Boolean), stderr: r.stderr, npmCalls }
}
- return { pkg, run }
+ return { root, pkg, home, run }
}
+const noHarnessNode = (box) => `miss node >= 18, and Harness's own Node is not in ${box.home}/.harness/runtime — run \`harness start\` once to lay it down`
+const harnessTooOld = "miss node >= 18 — this machine's newest is v16.20.2, Harness's own; update Harness"
+
const THREE = [
['node_modules/three/package.json', '{ "version": "0.186.0" }'],
['node_modules/three/build/three.module.js', ''],
@@ -78,18 +106,28 @@ describe('setup.sh', () => {
assert.equal(r.status, 0)
})
- test('refuses without Node 18 or newer, and without npm', () => {
+ test("refuses without Node 18 or newer — this machine's or Harness's own — and without npm beside it", () => {
for (const node of [null, 'old']) {
- const r = sandbox('setup.sh', { node }).run()
- assert.deepEqual(r.lines, ['miss node >= 18 on PATH'], `node: ${node}`)
+ const box = sandbox('setup.sh', { node })
+ const r = box.run()
+ assert.deepEqual(r.lines, [noHarnessNode(box)], `node: ${node}`)
assert.equal(r.status, 1)
assert.deepEqual(r.npmCalls, [], 'nothing is installed')
}
- const r = sandbox('setup.sh', { npm: false }).run()
- assert.deepEqual(r.lines, ['miss npm on PATH'])
+ let r = sandbox('setup.sh', { node: 'old', harness: 'old' }).run()
+ assert.deepEqual([r.status, r.lines, r.npmCalls], [1, [harnessTooOld], []])
+ r = sandbox('setup.sh', { npm: false }).run()
+ assert.deepEqual(r.lines, [`miss npm beside node ${process.version}`])
assert.equal(r.status, 1)
})
+ test("a machine with no Node on PATH installs with Harness's own Node and the npm beside it", () => {
+ const r = sandbox('setup.sh', { node: null, npm: false, harness: 'real' }).run()
+ assert.deepEqual(r.npmCalls, ['ci --silent --no-audit --no-fund', 'run smoke --silent'])
+ assert.deepEqual(r.lines, ['smoke ok', 'ok three 0.186.0 (3D Viewer)'])
+ assert.equal(r.status, 0)
+ })
+
test('stops when npm ci fails, when it leaves three or its glTF loader out, and when the smoke test fails', () => {
let r = sandbox('setup.sh').run({ NPM_CI: 'fail' })
assert.equal(r.status, 1)
@@ -107,19 +145,24 @@ describe('setup.sh', () => {
})
describe('doctor.sh', () => {
- test('ready: node, three and the page, one ok line each for the tools', () => {
- const r = sandbox('doctor.sh', { files: [...THREE, ...WEB] }).run()
- assert.deepEqual(r.lines, [`ok node ${process.version}`, 'ok three 0.186.0'])
- assert.equal(r.status, 0)
+ test("ready: node, three and the page, one ok line each for the tools — on Harness's own Node too", () => {
+ for (const [node, harness] of [['real', null], [null, 'real']]) {
+ const r = sandbox('doctor.sh', { node, harness, files: [...THREE, ...WEB] }).run()
+ assert.deepEqual(r.lines, [`ok node ${process.version}`, 'ok three 0.186.0'], `node: ${node}`)
+ assert.equal(r.status, 0)
+ }
})
test('says what is missing: Node (or too old a Node), three or its loader, the page', () => {
for (const node of [null, 'old']) {
- const r = sandbox('doctor.sh', { node, files: [...THREE, ...WEB] }).run()
- assert.equal(r.lines[0], 'miss node >= 18 on PATH', `node: ${node}`)
+ const box = sandbox('doctor.sh', { node, files: [...THREE, ...WEB] })
+ const r = box.run()
+ assert.equal(r.lines[0], noHarnessNode(box), `node: ${node}`)
assert.equal(r.status, 1)
}
- let r = sandbox('doctor.sh', { files: WEB }).run()
+ let r = sandbox('doctor.sh', { node: 'old', harness: 'old', files: [...THREE, ...WEB] }).run()
+ assert.deepEqual([r.status, r.lines[0]], [1, harnessTooOld])
+ r = sandbox('doctor.sh', { files: WEB }).run()
assert.deepEqual([r.status, r.lines.slice(1)], [1, ['miss node_modules/three — run ./setup.sh']])
r = sandbox('doctor.sh', { files: [THREE[0], THREE[1], ...WEB] }).run()
assert.deepEqual([r.status, r.lines.slice(1)], [1, ['miss node_modules/three — run ./setup.sh']])
@@ -161,4 +204,15 @@ describe('viewer.sh', () => {
}
assert.equal(await exited, 0, 'exec: the signal reaches node, which exits cleanly')
})
+
+ test("a login shell with no node on PATH runs Harness's own; with none at all, a miss and no server", () => {
+ const env = { HARNESS_VIEWER_PORT: '4000', HARNESS_WORKSPACE: tmpdir() }
+ const box = sandbox('viewer.sh', { node: null, npm: false, harness: 'real', harnessNpm: false })
+ writeFileSync(join(box.pkg, 'viewer.mjs'), 'console.log(`viewer on ${process.execPath} from ${process.argv[1]}`)\n')
+ let r = box.run(env)
+ assert.deepEqual([r.status, r.lines, r.stderr], [0, [`viewer on ${process.execPath} from ${join(box.pkg, 'viewer.mjs')}`], ''])
+ const bare = sandbox('viewer.sh', { node: null, npm: false })
+ r = bare.run(env)
+ assert.deepEqual([r.status, r.lines], [1, [noHarnessNode(bare)]])
+ })
})
diff --git a/store/viewers/model-viewer/viewer.sh b/store/viewers/model-viewer/viewer.sh
index 9a40a19cc..be9182cbb 100755
--- a/store/viewers/model-viewer/viewer.sh
+++ b/store/viewers/model-viewer/viewer.sh
@@ -1,5 +1,10 @@
#!/usr/bin/env bash
-# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE.
+# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE, through a login shell
+# whose PATH may hold no node: runtimes.sh finds this machine's or Harness's own.
set -euo pipefail
: "${HARNESS_VIEWER_PORT:?}"; : "${HARNESS_WORKSPACE:?}"
-exec node "$(cd "$(dirname "$0")" && pwd)/viewer.mjs"
+here="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/runtimes.sh"
+harness_node 18 || exit 1
+exec node "$here/viewer.mjs"
diff --git a/store/viewers/mujoco-viewer/doctor.sh b/store/viewers/mujoco-viewer/doctor.sh
index 587fe6c7f..e18144895 100755
--- a/store/viewers/mujoco-viewer/doctor.sh
+++ b/store/viewers/mujoco-viewer/doctor.sh
@@ -1,7 +1,10 @@
#!/usr/bin/env bash
set -u; cd "$(dirname "$0")"
fail=0
-if command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 18 ? 0 : 1)'; then echo "ok node $(node --version)"; else echo "miss node >= 18 on PATH"; fail=1; fi
+# The same Node setup.sh uses, found the same way (runtimes.sh).
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+if harness_node 18; then echo "ok node $(node --version)"; else fail=1; fi
if [ -f node_modules/@mujoco/mujoco/mujoco.wasm ]; then echo "ok mujoco $(node -p "require('./node_modules/@mujoco/mujoco/package.json').version") (wasm)"; else echo "miss node_modules/@mujoco/mujoco — run ./setup.sh"; fail=1; fi
if [ -f node_modules/three/build/three.module.js ]; then echo "ok three $(node -p "require('./node_modules/three/package.json').version")"; else echo "miss node_modules/three — run ./setup.sh"; fail=1; fi
if [ -f public/main.js ] && [ -f public/index.html ]; then echo "ok pane (public/)"; else echo "miss public/ — this package is incomplete"; fail=1; fi
diff --git a/store/viewers/mujoco-viewer/runtimes.sh b/store/viewers/mujoco-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/mujoco-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/mujoco-viewer/setup.sh b/store/viewers/mujoco-viewer/setup.sh
index a5c10cb31..8447c73b5 100755
--- a/store/viewers/mujoco-viewer/setup.sh
+++ b/store/viewers/mujoco-viewer/setup.sh
@@ -1,10 +1,13 @@
#!/usr/bin/env bash
# Runs once at install, cwd = the install dir: MuJoCo's WASM build and three.js into node_modules,
-# from the lockfile. Nothing here touches the machine outside this directory.
+# from the lockfile. Nothing here touches the machine outside this directory. Node is this machine's,
+# or Harness's own when the machine has none or too old a one (runtimes.sh); npm is the one beside it.
set -euo pipefail
cd "$(dirname "$0")"
-command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 18 ? 0 : 1)' || { echo "miss node >= 18 on PATH"; exit 1; }
-command -v npm >/dev/null 2>&1 || { echo "miss npm on PATH"; exit 1; }
+# shellcheck source=runtimes.sh
+. ./runtimes.sh
+harness_node 18 || exit 1
+command -v npm >/dev/null 2>&1 || { echo "miss npm beside node $(node --version)"; exit 1; }
npm ci --silent --no-audit --no-fund
[ -f node_modules/@mujoco/mujoco/mujoco.wasm ] || { echo "miss mujoco.wasm after npm ci"; exit 1; }
[ -f node_modules/three/build/three.module.js ] || { echo "miss three after npm ci"; exit 1; }
diff --git a/store/viewers/mujoco-viewer/test/scripts.test.mjs b/store/viewers/mujoco-viewer/test/scripts.test.mjs
index ffce018fc..84fbc0598 100644
--- a/store/viewers/mujoco-viewer/test/scripts.test.mjs
+++ b/store/viewers/mujoco-viewer/test/scripts.test.mjs
@@ -1,11 +1,13 @@
// setup.sh, doctor.sh and viewer.sh, each run from a copy of the package in a scratch folder with a
-// PATH that holds only what the case gives it: bash, dirname, and a real, old or missing node and npm.
-// Every line they can print, and the exit code that goes with it.
+// PATH that holds only what the case gives it: bash, dirname, cat, and a real, old or missing node and
+// npm. The scripts find Node through runtimes.sh, so a case can also lay down Harness's own Node under
+// HOME (~/.harness/runtime/current-node) — the Node a fresh Mac, with none on PATH, gets — with an npm
+// beside it. Every line they can print, and the exit code that goes with it.
//
// npm test
import assert from 'node:assert/strict'
import { spawnSync } from 'node:child_process'
-import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, symlinkSync, writeFileSync } from 'node:fs'
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { isAbsolute, join } from 'node:path'
import { after, test } from 'node:test'
@@ -14,6 +16,9 @@ import { PACKAGE, removeTree, write } from './helpers.mjs'
const root = mkdtempSync(join(tmpdir(), 'mujoco-viewer-scripts-'))
after(() => removeTree(root))
+// A bash line tracer's hooks (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT), passed through when set.
+const TRACER = Object.fromEntries(['BASH_ENV', 'SHCOV_OUT'].filter((k) => process.env[k]).map((k) => [k, process.env[k]]))
+const OLD_NODE = '#!/bin/sh\n[ "$1" = "-e" ] && exit 1\necho v16.20.2\n'
const which = (tool) => ['/bin', '/usr/bin'].map((dir) => join(dir, tool)).find((path) => existsSync(path))
const INSTALLED = {
'node_modules/@mujoco/mujoco/mujoco.wasm': 'wasm',
@@ -24,41 +29,72 @@ const INSTALLED = {
const PANE = { 'public/main.js': '', 'public/index.html': '' }
let cases = 0
+
+/** An executable script at dir/name; whatever was there goes first, so nothing is written through a link. */
+function stub(dir, name, body) {
+ const full = join(dir, name)
+ mkdirSync(dir, { recursive: true })
+ rmSync(full, { force: true })
+ writeFileSync(full, body)
+ chmodSync(full, 0o755)
+ return full
+}
+
/**
* A package copy and its PATH. node: 'real' | 'old' | null. npm: null, or the exit codes of `npm ci`
- * and `npm run smoke`; the stub logs what it was asked to npm.log in the package.
+ * and `npm run smoke`; the stub logs what it was asked to npm.log in the package. harness: Harness's own
+ * Node under HOME, 'real' | 'old' | null, with that npm stub beside it when there is one. The scripts and
+ * runtimes.sh are linked, not copied, so a line tracer maps back to them.
*/
-function sandbox({ node = 'real', npm = null, files = {} } = {}) {
+function sandbox({ node = 'real', npm = null, harness = null, files = {} } = {}) {
const dir = join(root, `case-${++cases}`)
const pkg = join(dir, 'pkg')
const bin = join(dir, 'bin')
+ const home = join(dir, 'home')
mkdirSync(pkg, { recursive: true })
mkdirSync(bin)
- for (const script of ['setup.sh', 'doctor.sh', 'viewer.sh']) {
- copyFileSync(join(PACKAGE, script), join(pkg, script))
- chmodSync(join(pkg, script), 0o755)
- }
- for (const tool of ['bash', 'dirname']) symlinkSync(which(tool), join(bin, tool))
+ mkdirSync(home)
+ for (const script of ['setup.sh', 'doctor.sh', 'viewer.sh', 'runtimes.sh']) symlinkSync(join(PACKAGE, script), join(pkg, script))
+ for (const tool of ['bash', 'dirname', 'cat']) symlinkSync(which(tool), join(bin, tool))
+ const npmStub = (where) => stub(where, 'npm', `#!/bin/sh\necho "$*" >> npm.log\ncase "$1" in\n ci) exit ${npm.ci} ;;\n run) exit ${npm.smoke} ;;\nesac\nexit 99\n`)
if (node === 'real') symlinkSync(process.execPath, join(bin, 'node'))
- if (node === 'old') writeFileSync(join(bin, 'node'), '#!/bin/sh\nexit 1\n', { mode: 0o755 })
- if (npm) {
- writeFileSync(join(bin, 'npm'), `#!/bin/sh\necho "$*" >> npm.log\ncase "$1" in\n ci) exit ${npm.ci} ;;\n run) exit ${npm.smoke} ;;\nesac\nexit 99\n`, { mode: 0o755 })
+ if (node === 'old') stub(bin, 'node', OLD_NODE)
+ if (harness) {
+ const harnessBin = join(dir, 'harness-node', 'bin')
+ mkdirSync(harnessBin, { recursive: true })
+ if (harness === 'real') symlinkSync(process.execPath, join(harnessBin, 'node'))
+ else stub(harnessBin, 'node', OLD_NODE)
+ if (npm) npmStub(harnessBin)
+ write(join(home, '.harness', 'runtime', 'current-node'), join(harnessBin, 'node'))
+ } else if (npm) {
+ npmStub(bin)
}
for (const [rel, body] of Object.entries(files)) write(join(pkg, rel), body)
const run = (script, env = {}, cwd = pkg) => {
- const r = spawnSync(script, [], { cwd, env: { PATH: bin, ...env }, encoding: 'utf8' })
+ const r = spawnSync(script, [], { cwd, env: { PATH: bin, HOME: home, ...TRACER, ...env }, encoding: 'utf8' })
return { code: r.status, stdout: r.stdout, stderr: r.stderr, npm: existsSync(join(pkg, 'npm.log')) ? readFileSync(join(pkg, 'npm.log'), 'utf8') : null }
}
- return { dir, pkg, bin, run }
+ return { dir, pkg, bin, home, run }
}
-test('setup.sh needs node 18 or newer and npm, before it installs anything', () => {
+const noHarnessNode = (box) => `miss node >= 18, and Harness's own Node is not in ${box.home}/.harness/runtime — run \`harness start\` once to lay it down\n`
+const harnessTooOld = "miss node >= 18 — this machine's newest is v16.20.2, Harness's own; update Harness\n"
+
+test('setup.sh needs node 18 or newer — this machine\'s or Harness\'s own — and npm beside it, before it installs anything', () => {
for (const node of [null, 'old']) {
- const r = sandbox({ node, npm: { ci: 0, smoke: 0 } }).run('./setup.sh')
- assert.deepEqual([r.code, r.stdout, r.npm], [1, 'miss node >= 18 on PATH\n', null], `node: ${node}`)
+ const box = sandbox({ node, npm: { ci: 0, smoke: 0 } })
+ const r = box.run('./setup.sh')
+ assert.deepEqual([r.code, r.stdout, r.npm], [1, noHarnessNode(box), null], `node: ${node}`)
}
- const r = sandbox({ npm: null }).run('./setup.sh')
- assert.deepEqual([r.code, r.stdout], [1, 'miss npm on PATH\n'])
+ let r = sandbox({ node: 'old', harness: 'old', npm: { ci: 0, smoke: 0 } }).run('./setup.sh')
+ assert.deepEqual([r.code, r.stdout, r.npm], [1, harnessTooOld, null])
+ r = sandbox({ npm: null }).run('./setup.sh')
+ assert.deepEqual([r.code, r.stdout], [1, `miss npm beside node ${process.version}\n`])
+})
+
+test('setup.sh on a machine with no node on PATH installs with Harness\'s own Node and the npm beside it', () => {
+ const r = sandbox({ node: null, harness: 'real', npm: { ci: 0, smoke: 0 }, files: INSTALLED }).run('./setup.sh')
+ assert.deepEqual([r.code, r.stdout, r.npm], [0, 'ok mujoco 3.13.0 (wasm) · three 0.186.0\n', 'ci --silent --no-audit --no-fund\nrun smoke --silent\n'])
})
test('setup.sh stops when npm ci fails, and says which dependency npm ci did not deliver', () => {
@@ -95,10 +131,19 @@ test('doctor.sh: one line per check, exit 0 only when everything is there', () =
// Before the fix the doctor never asked for node, and said ready on a machine the viewer cannot run on.
for (const node of [null, 'old']) {
- r = sandbox({ node, files: { ...INSTALLED, ...PANE } }).run('./doctor.sh')
+ const box = sandbox({ node, files: { ...INSTALLED, ...PANE } })
+ r = box.run('./doctor.sh')
assert.equal(r.code, 1, `node: ${node}`)
- assert.match(r.stdout, /^miss node >= 18 on PATH\n/, `node: ${node}`)
+ assert.ok(r.stdout.startsWith(noHarnessNode(box)), `node: ${node}`)
}
+ r = sandbox({ node: 'old', harness: 'old', files: { ...INSTALLED, ...PANE } }).run('./doctor.sh')
+ assert.equal(r.code, 1)
+ assert.ok(r.stdout.startsWith(harnessTooOld))
+
+ // A fresh Mac: no node on PATH, Harness's own laid down — the Node setup.sh installed with.
+ r = sandbox({ node: null, harness: 'real', files: { ...INSTALLED, ...PANE } }).run('./doctor.sh')
+ assert.equal(r.code, 0)
+ assert.equal(r.stdout, `ok node ${process.version}\nok mujoco 3.13.0 (wasm)\nok three 0.186.0\nok pane (public/)\n`)
})
test('viewer.sh insists on the port and the workspace, then execs node on the viewer beside it', () => {
@@ -118,3 +163,15 @@ test('viewer.sh insists on the port and the workspace, then execs node on the vi
assert.equal(realpathSync(seen.script), realpathSync(join(box.pkg, 'viewer.mjs')))
assert.deepEqual([seen.port, seen.workspace], ['18997', '/tmp/ws'])
})
+
+test('viewer.sh from a login shell with no node on PATH runs Harness\'s own; with none at all, a miss and no server', () => {
+ const env = { HARNESS_VIEWER_PORT: '18997', HARNESS_WORKSPACE: '/tmp/ws' }
+ const box = sandbox({ node: null, harness: 'real' })
+ writeFileSync(join(box.pkg, 'viewer.mjs'), 'console.log(process.execPath)\n')
+ let r = box.run('./viewer.sh', env)
+ assert.deepEqual([r.code, r.stdout, r.stderr], [0, `${process.execPath}\n`, ''])
+ const bare = sandbox({ node: null })
+ writeFileSync(join(bare.pkg, 'viewer.mjs'), 'console.log(process.execPath)\n')
+ r = bare.run('./viewer.sh', env)
+ assert.deepEqual([r.code, r.stdout], [1, noHarnessNode(bare)])
+})
diff --git a/store/viewers/mujoco-viewer/viewer.sh b/store/viewers/mujoco-viewer/viewer.sh
index 7b9241958..47de184d1 100755
--- a/store/viewers/mujoco-viewer/viewer.sh
+++ b/store/viewers/mujoco-viewer/viewer.sh
@@ -1,6 +1,11 @@
#!/usr/bin/env bash
# The pane. Harness runs this here with HARNESS_VIEWER_PORT, HARNESS_WORKSPACE and HARNESS_DSH_DIR
-# (the harness that uses this viewer — where its Menagerie robots are).
+# (the harness that uses this viewer — where its Menagerie robots are), through a login shell whose
+# PATH may hold no node: runtimes.sh finds this machine's or Harness's own.
set -euo pipefail
: "${HARNESS_VIEWER_PORT:?}"; : "${HARNESS_WORKSPACE:?}"
-exec node "$(cd "$(dirname "$0")" && pwd)/viewer.mjs"
+here="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/runtimes.sh"
+harness_node 18 || exit 1
+exec node "$here/viewer.mjs"
diff --git a/store/viewers/video-viewer/doctor.sh b/store/viewers/video-viewer/doctor.sh
index 18af484b3..06f6611f2 100755
--- a/store/viewers/video-viewer/doctor.sh
+++ b/store/viewers/video-viewer/doctor.sh
@@ -1,5 +1,8 @@
#!/usr/bin/env bash
-# Exit 0 when this machine can run the viewer: only Node is needed.
+# Exit 0 when this machine can run the viewer: only Node is needed — this machine's, or Harness's own
+# when it has none or too old a one (runtimes.sh, as viewer.sh finds it).
set -u
-if command -v node >/dev/null 2>&1 && node -e 'process.exit(Number(process.versions.node.split(".")[0]) >= 18 ? 0 : 1)'; then echo "ok node $(node --version)"; exit 0; fi
-echo "miss node >= 18 on PATH"; exit 1
+# shellcheck source=runtimes.sh
+. "$(dirname "$0")/runtimes.sh"
+harness_node 18 || exit 1
+echo "ok node $(node --version)"
diff --git a/store/viewers/video-viewer/runtimes.sh b/store/viewers/video-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/video-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/video-viewer/test/scripts.test.mjs b/store/viewers/video-viewer/test/scripts.test.mjs
index 527fe9647..bd6994ab8 100644
--- a/store/viewers/video-viewer/test/scripts.test.mjs
+++ b/store/viewers/video-viewer/test/scripts.test.mjs
@@ -1,11 +1,13 @@
// The shell scripts Harness runs, on a PATH that holds only what each one needs: doctor.sh says whether
// this machine has a Node that can run the viewer; viewer.sh insists on the port and the workspace, then
-// runs the viewer.mjs that sits beside it, whatever the working directory.
+// runs the viewer.mjs that sits beside it, whatever the working directory. Both find Node through
+// runtimes.sh: this machine's, or Harness's own under HOME (~/.harness/runtime/current-node) — the Node a
+// fresh Mac, with none on PATH, gets.
//
// node --test test/*.test.mjs
import assert from 'node:assert/strict'
import { spawnSync } from 'node:child_process'
-import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
+import { chmodSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { after, test } from 'node:test'
@@ -13,6 +15,9 @@ import { PACKAGE } from './media.mjs'
const roots = []
after(() => { for (const root of roots) rmSync(root, { recursive: true, force: true }) })
+// A bash line tracer's hooks (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT), passed through when set.
+const TRACER = Object.fromEntries(['BASH_ENV', 'SHCOV_OUT'].filter((k) => process.env[k]).map((k) => [k, process.env[k]]))
+const OLD_NODE = '#!/bin/sh\n# Node 16: the version check fails\n[ "$1" = "-e" ] && exit 1\necho v16.20.2\n'
/** A bin folder with the named system tools linked in, and `extra` scripts written as executables. */
function bin(tools, extra = {}) {
@@ -24,6 +29,7 @@ function bin(tools, extra = {}) {
symlinkSync(found, join(root, tool))
}
for (const [name, body] of Object.entries(extra)) {
+ rmSync(join(root, name), { force: true }) // never write through a link to a real command
if (body === 'real-node') { symlinkSync(process.execPath, join(root, name)); continue }
writeFileSync(join(root, name), body)
chmodSync(join(root, name), 0o755)
@@ -31,28 +37,47 @@ function bin(tools, extra = {}) {
return root
}
-const run = (script, { path, env = {}, cwd = tmpdir() }) => {
- const r = spawnSync(script, { cwd, env: { PATH: path, ...env }, encoding: 'utf8' })
+/** A HOME, with Harness's own Node laid down in it when `node` is given ('real-node' or a script). */
+function home(node) {
+ const root = mkdtempSync(join(tmpdir(), 'video-viewer-home-'))
+ roots.push(root)
+ if (node) {
+ const harnessBin = bin([], { node })
+ mkdirSync(join(root, '.harness', 'runtime'), { recursive: true })
+ writeFileSync(join(root, '.harness', 'runtime', 'current-node'), join(harnessBin, 'node'))
+ }
+ return root
+}
+
+const run = (script, { path, env = {}, cwd = tmpdir(), homeDir = home() }) => {
+ const r = spawnSync(script, { cwd, env: { PATH: path, HOME: homeDir, ...TRACER, ...env }, encoding: 'utf8' })
return { code: r.status, out: r.stdout, err: r.stderr }
}
+const TOOLS = ['bash', 'dirname', 'cat']
+const noHarnessNode = (homeDir) => `miss node >= 18, and Harness's own Node is not in ${homeDir}/.harness/runtime — run \`harness start\` once to lay it down\n`
-test('doctor.sh: ok with Node 18 or newer, a miss without Node or with an older one', () => {
+test('doctor.sh: ok with Node 18 or newer, this machine\'s or Harness\'s own; a miss without one', () => {
const doctor = join(PACKAGE, 'doctor.sh')
- const ok = run(doctor, { path: bin(['bash'], { node: 'real-node' }) })
+ const ok = run(doctor, { path: bin(TOOLS, { node: 'real-node' }) })
assert.deepEqual(ok, { code: 0, out: `ok node ${process.version}\n`, err: '' })
- const old = run(doctor, { path: bin(['bash'], { node: '#!/bin/sh\n# Node 16: the version check fails\n[ "$1" = "-e" ] && exit 1\necho v16.20.2\n' }) })
- assert.deepEqual(old, { code: 1, out: 'miss node >= 18 on PATH\n', err: '' })
- const none = run(doctor, { path: bin(['bash']) })
- assert.deepEqual(none, { code: 1, out: 'miss node >= 18 on PATH\n', err: '' })
+ const harnessOwn = run(doctor, { path: bin(TOOLS), homeDir: home('real-node') })
+ assert.deepEqual(harnessOwn, { code: 0, out: `ok node ${process.version}\n`, err: '' })
+ const bareHome = home()
+ const old = run(doctor, { path: bin(TOOLS, { node: OLD_NODE }), homeDir: bareHome })
+ assert.deepEqual(old, { code: 1, out: noHarnessNode(bareHome), err: '' })
+ const none = run(doctor, { path: bin(TOOLS), homeDir: bareHome })
+ assert.deepEqual(none, { code: 1, out: noHarnessNode(bareHome), err: '' })
+ const bothOld = run(doctor, { path: bin(TOOLS, { node: OLD_NODE }), homeDir: home(OLD_NODE) })
+ assert.deepEqual(bothOld, { code: 1, out: "miss node >= 18 — this machine's newest is v16.20.2, Harness's own; update Harness\n", err: '' })
})
test('viewer.sh: the port and the workspace are required; then the viewer beside the script runs', () => {
const pkg = mkdtempSync(join(tmpdir(), 'video-viewer-pkg-'))
roots.push(pkg)
- copyFileSync(join(PACKAGE, 'viewer.sh'), join(pkg, 'viewer.sh'))
- chmodSync(join(pkg, 'viewer.sh'), 0o755)
+ // Linked, not copied: a line tracer maps back to them.
+ for (const name of ['viewer.sh', 'runtimes.sh']) symlinkSync(join(PACKAGE, name), join(pkg, name))
writeFileSync(join(pkg, 'viewer.mjs'), "console.log(`viewer ${process.env.HARNESS_VIEWER_PORT} ${process.env.HARNESS_WORKSPACE}`)\n")
- const path = bin(['bash', 'dirname'], { node: 'real-node' })
+ const path = bin(TOOLS, { node: 'real-node' })
const script = join(pkg, 'viewer.sh')
const noPort = run(script, { path, env: { HARNESS_WORKSPACE: '/Users/example/ws' } })
@@ -68,4 +93,12 @@ test('viewer.sh: the port and the workspace are required; then the viewer beside
mkdirSync(join(elsewhere, 'sub'))
const started = run(script, { path, cwd: join(elsewhere, 'sub'), env: { HARNESS_VIEWER_PORT: '4100', HARNESS_WORKSPACE: '/Users/example/ws' } })
assert.deepEqual(started, { code: 0, out: 'viewer 4100 /Users/example/ws\n', err: '' })
+
+ // The daemon's login shell may have no node on PATH: Harness's own runs the viewer; with none at all, a miss.
+ const env = { HARNESS_VIEWER_PORT: '4100', HARNESS_WORKSPACE: '/Users/example/ws' }
+ const onHarnessNode = run(script, { path: bin(TOOLS), homeDir: home('real-node'), env })
+ assert.deepEqual(onHarnessNode, { code: 0, out: 'viewer 4100 /Users/example/ws\n', err: '' })
+ const bareHome = home()
+ const noNode = run(script, { path: bin(TOOLS), homeDir: bareHome, env })
+ assert.deepEqual(noNode, { code: 1, out: noHarnessNode(bareHome), err: '' })
})
diff --git a/store/viewers/video-viewer/viewer.sh b/store/viewers/video-viewer/viewer.sh
index 9a40a19cc..be9182cbb 100755
--- a/store/viewers/video-viewer/viewer.sh
+++ b/store/viewers/video-viewer/viewer.sh
@@ -1,5 +1,10 @@
#!/usr/bin/env bash
-# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE.
+# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE, through a login shell
+# whose PATH may hold no node: runtimes.sh finds this machine's or Harness's own.
set -euo pipefail
: "${HARNESS_VIEWER_PORT:?}"; : "${HARNESS_WORKSPACE:?}"
-exec node "$(cd "$(dirname "$0")" && pwd)/viewer.mjs"
+here="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/runtimes.sh"
+harness_node 18 || exit 1
+exec node "$here/viewer.mjs"
diff --git a/store/viewers/web-viewer/doctor.sh b/store/viewers/web-viewer/doctor.sh
new file mode 100755
index 000000000..5c46a5fdb
--- /dev/null
+++ b/store/viewers/web-viewer/doctor.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Exit 0 when this machine can run the viewer: only Node 20 or newer is needed — this machine's, or
+# Harness's own when it has none or too old a one (runtimes.sh, as viewer.sh finds it).
+set -u
+# shellcheck source=runtimes.sh
+. "$(dirname "$0")/runtimes.sh"
+harness_node 20 || exit 1
+echo "ok node $(node --version)"
diff --git a/store/viewers/web-viewer/harness.json b/store/viewers/web-viewer/harness.json
index 236d4399f..35d9b2e19 100644
--- a/store/viewers/web-viewer/harness.json
+++ b/store/viewers/web-viewer/harness.json
@@ -7,10 +7,10 @@
"author": "OpenHarness contributors",
"description": "A live preview for HTML, CSS, and JavaScript in a harness workspace. Reloads when files change.",
"toolchain": {
- "doctor": "node -e \"process.exit(Number(process.versions.node.split('.')[0]) >= 20 ? 0 : 1)\""
+ "doctor": "./doctor.sh"
},
"viewer": {
- "command": "node viewer.mjs",
+ "command": "./viewer.sh",
"url": "http://127.0.0.1:${port}/?file=index.html",
"artifactExtensions": [".html"]
}
diff --git a/store/viewers/web-viewer/runtimes.sh b/store/viewers/web-viewer/runtimes.sh
new file mode 100644
index 000000000..4e21942d0
--- /dev/null
+++ b/store/viewers/web-viewer/runtimes.sh
@@ -0,0 +1,187 @@
+# shellcheck shell=bash
+# runtimes.sh — the interpreters a store package runs on, found on this machine or fetched, never
+# asked of the person who pressed Get.
+#
+# Sourced (not run) by a package's setup, doctor and viewer scripts. A new Mac has Apple's Python 3.9,
+# no Homebrew and, for most people, no Node on PATH; a package that answers "brew install [email protected]"
+# has failed its install. So:
+#
+# harness_node 18 node >= 18 on PATH, with npm: the machine's own, else the Node Harness
+# itself runs on (~/.harness/runtime/current-node, laid down with the CLI)
+# harness_uv uv on PATH: the machine's own, else a pinned, checksummed release
+# fetched into ~/.harness/runtime
+# harness_venv DIR 3.12 [MIN [BELOW]]
+# DIR is a venv on CPython 3.12, which uv downloads when the machine
+# has none. A venv already there on a Python in [MIN, BELOW) is kept.
+# harness_pip DIR PKG… install into that venv, through uv
+# harness_micromamba micromamba, pinned and checksummed, for native libraries PyPI has
+# no wheel for (pycairo on macOS)
+# harness_conda_env DIR SPEC… DIR is a conda-forge environment with those packages
+#
+# Every function prints `miss …` and returns 1 when it cannot deliver, so `harness_node 18 || exit 1`
+# reads like the rest of a setup script. Nothing here touches the user's shell profile or anything
+# outside ~/.harness/runtime and the package's own directory.
+#
+# ONE copy is written by hand: store/tools/runtimes.sh in the OpenHarness repository. Each package
+# carries an identical copy (a package installs alone, so it cannot reach this one);
+# `node store/tools/sync-runtimes.mjs` rewrites them and the CLI's store spec fails when one drifts.
+
+HARNESS_RUNTIME="${ADAPTER_RUNTIME_DIR:-${HOME:-}/.harness/runtime}"
+# The Pythons uv downloads live with Harness's other runtimes, not in uv's own ~/.local/share/uv:
+# a package's venv points at its interpreter, and `uv python uninstall` there must not break it.
+UV_PYTHON_INSTALL_DIR="${UV_PYTHON_INSTALL_DIR:-$HARNESS_RUNTIME/python}"
+export UV_PYTHON_INSTALL_DIR
+HARNESS_UV_VERSION=0.12.15
+HARNESS_MICROMAMBA_VERSION=2.9.0-0
+
+_harness_platform() {
+ case "$(uname -s)-$(uname -m)" in
+ Darwin-arm64) echo darwin-arm64 ;;
+ Darwin-x86_64) echo darwin-x64 ;;
+ Linux-x86_64) echo linux-x64 ;;
+ Linux-aarch64 | Linux-arm64) echo linux-arm64 ;;
+ *) return 1 ;;
+ esac
+}
+
+_harness_sha256() {
+ if command -v shasum >/dev/null 2>&1; then shasum -a 256 "$1" | cut -d' ' -f1
+ else sha256sum "$1" | cut -d' ' -f1; fi
+}
+
+# _harness_fetch URL DEST SHA256 — DEST exists only if what arrived matches the pinned checksum.
+_harness_fetch() {
+ local part="$2.part.$$"
+ if ! curl -fsSL --retry 3 --connect-timeout 20 --max-time 900 -o "$part" "$1"; then
+ rm -f "$part"; echo "miss could not download $1 — check this machine's internet connection"; return 1
+ fi
+ if [ "$(_harness_sha256 "$part")" != "$3" ]; then
+ rm -f "$part"; echo "miss $1 did not match its pinned checksum; nothing was installed"; return 1
+ fi
+ mv -f "$part" "$2"
+}
+
+_harness_node_at_least() {
+ command -v node >/dev/null 2>&1 || return 1
+ node -e '
+ const [a, b = 0] = process.argv[1].split(".").map(Number)
+ const [x, y] = process.versions.node.split(".").map(Number)
+ process.exit(x > a || (x === a && y >= b) ? 0 : 1)
+ ' "$1" >/dev/null 2>&1
+}
+
+harness_node() {
+ local min="${1:-18}" recorded saved="$PATH" own=0
+ _harness_node_at_least "$min" && own=1
+ # A node without npm beside it (a distro's nodejs package) cannot run a setup's `npm ci`.
+ if [ "$own" = 1 ] && command -v npm >/dev/null 2>&1; then return 0; fi
+ recorded="$(cat "$HARNESS_RUNTIME/current-node" 2>/dev/null || true)"
+ if [ -n "$recorded" ] && [ -x "${recorded%/*}/node" ]; then
+ PATH="${recorded%/*}:$PATH"; export PATH; hash -r 2>/dev/null || true
+ _harness_node_at_least "$min" && return 0
+ PATH="$saved"; export PATH; hash -r 2>/dev/null || true
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min — this machine's newest is $("${recorded%/*}/node" -v), Harness's own; update Harness"
+ return 1
+ fi
+ [ "$own" = 1 ] && return 0
+ echo "miss node >= $min, and Harness's own Node is not in $HARNESS_RUNTIME — run \`harness start\` once to lay it down"
+ return 1
+}
+
+harness_uv() {
+ local dir asset sum tmp
+ command -v uv >/dev/null 2>&1 && return 0
+ for dir in "${HOME:-}/.local/bin" "${HOME:-}/.cargo/bin"; do
+ [ -n "${HOME:-}" ] || break
+ if [ -x "$dir/uv" ]; then PATH="$dir:$PATH"; export PATH; return 0; fi
+ done
+ dir="$HARNESS_RUNTIME/uv-$HARNESS_UV_VERSION"
+ if [ ! -x "$dir/uv" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=uv-aarch64-apple-darwin sum=dc304b9ed1b24174572290fba60ac3f6fe63c73a671f0439e62a91375841964d ;;
+ darwin-x64) asset=uv-x86_64-apple-darwin sum=e9ca61775532368fe518ab03e7a354c7ecab8ccb3c7d941c775fcc4a362b801b ;;
+ linux-x64) asset=uv-x86_64-unknown-linux-gnu sum=f97935763c04be3e692460a7aaeaaab8fc3b78fcf8b389da820b38ae7423a638 ;;
+ linux-arm64) asset=uv-aarch64-unknown-linux-gnu sum=0e9a3499b0587d449c9ff684c0160da607826e4af1cee220bc87f378702d3e08 ;;
+ *) echo "miss uv — there is no uv release for $(uname -s) $(uname -m); install it from https://docs.astral.sh/uv/"; return 1 ;;
+ esac
+ echo " fetching uv $HARNESS_UV_VERSION into $HARNESS_RUNTIME (it brings Python and the packages)"
+ mkdir -p "$HARNESS_RUNTIME" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ tmp="$(mktemp -d "$HARNESS_RUNTIME/.uv.XXXXXX")" || { echo "miss cannot write $HARNESS_RUNTIME"; return 1; }
+ if ! _harness_fetch "https://github.com/astral-sh/uv/releases/download/$HARNESS_UV_VERSION/$asset.tar.gz" "$tmp/uv.tar.gz" "$sum"; then
+ rm -rf "$tmp"; return 1
+ fi
+ tar -xzf "$tmp/uv.tar.gz" -C "$tmp" && chmod +x "$tmp/$asset/uv" "$tmp/$asset/uvx" \
+ || { rm -rf "$tmp"; echo "miss the uv archive would not unpack"; return 1; }
+ # Two installs at once both get here; whichever renames first wins and the other's copy goes.
+ # (mv onto a directory that exists would nest the copy inside it, hence the test.)
+ [ -e "$dir" ] || mv "$tmp/$asset" "$dir" 2>/dev/null || true
+ rm -rf "$tmp"
+ [ -x "$dir/uv" ] || { echo "miss uv did not land in $dir"; return 1; }
+ fi
+ PATH="$dir:$PATH"; export PATH
+}
+
+# harness_venv DIR VERSION [MIN [BELOW]] — MIN defaults to VERSION; BELOW is exclusive and optional.
+harness_venv() {
+ local dir="${1:?harness_venv needs a directory}" want="${2:?harness_venv needs a Python version}"
+ local min="${3:-$2}" below="${4:-}"
+ if [ -x "$dir/bin/python" ] && "$dir/bin/python" -c '
+import sys
+at = lambda s: tuple(int(p) for p in s.split("."))
+v = sys.version_info[:2]
+sys.exit(0 if v >= at(sys.argv[1]) and (not sys.argv[2] or v < at(sys.argv[2])) else 1)
+' "$min" "$below" 2>/dev/null; then
+ echo "ok $("$dir/bin/python" --version) in $dir"
+ return 0
+ fi
+ harness_uv || return 1
+ echo " python $want in $dir (uv downloads it when this machine has none)"
+ rm -rf "$dir"
+ # only-managed: the same python-build-standalone interpreter on every machine, so a package tested
+ # once is tested everywhere — not whichever Homebrew or pyenv Python happened to be first on PATH.
+ uv venv --quiet --seed --python "$want" --python-preference only-managed "$dir" \
+ || { echo "miss could not make a Python $want environment in $dir"; return 1; }
+ echo "ok $("$dir/bin/python" --version) in $dir"
+}
+
+harness_pip() {
+ local dir="${1:?harness_pip needs a venv}"
+ shift
+ harness_uv || return 1
+ uv pip install --quiet --python "$dir/bin/python" "$@"
+}
+
+harness_micromamba() {
+ local dir asset sum
+ dir="$HARNESS_RUNTIME/micromamba-$HARNESS_MICROMAMBA_VERSION"
+ if [ ! -x "$dir/micromamba" ]; then
+ case "$(_harness_platform)" in
+ darwin-arm64) asset=micromamba-osx-arm64 sum=ec2a072f028e1a7cf20f3e2e74d5a8127cf5a5f27636375b5359811565f4e5be ;;
+ darwin-x64) asset=micromamba-osx-64 sum=1e71054bb3ac9a076e21f7ec48acfef536f9b3f1408f371a942784bf5ef83d8a ;;
+ linux-x64) asset=micromamba-linux-64 sum=366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 ;;
+ linux-arm64) asset=micromamba-linux-aarch64 sum=9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7 ;;
+ *) echo "miss micromamba — there is no release for $(uname -s) $(uname -m)"; return 1 ;;
+ esac
+ echo " fetching micromamba $HARNESS_MICROMAMBA_VERSION into $HARNESS_RUNTIME (native libraries PyPI has no wheel for)"
+ mkdir -p "$dir" || { echo "miss cannot write $dir"; return 1; }
+ _harness_fetch "https://github.com/mamba-org/micromamba-releases/releases/download/$HARNESS_MICROMAMBA_VERSION/$asset" "$dir/micromamba.$$" "$sum" || return 1
+ chmod +x "$dir/micromamba.$$" && mv -f "$dir/micromamba.$$" "$dir/micromamba" \
+ || { rm -f "$dir/micromamba.$$"; echo "miss micromamba would not install into $dir"; return 1; }
+ fi
+ HARNESS_MICROMAMBA="$dir/micromamba"
+ export HARNESS_MICROMAMBA
+}
+
+# harness_conda_env DIR SPEC… — conda-forge only, packages cached under ~/.harness/runtime/mamba.
+harness_conda_env() {
+ local dir="${1:?harness_conda_env needs a directory}"
+ shift
+ # micromamba takes a relative --prefix as a name under its root prefix and "succeeds" there.
+ case "$dir" in /*) ;; *) dir="$PWD/$dir" ;; esac
+ harness_micromamba || return 1
+ MAMBA_ROOT_PREFIX="$HARNESS_RUNTIME/mamba" "$HARNESS_MICROMAMBA" create --yes --quiet \
+ --prefix "$dir" --override-channels --channel conda-forge "$@" >/dev/null \
+ && [ -d "$dir/conda-meta" ] \
+ || { echo "miss could not make the conda-forge environment in $dir ($*)"; return 1; }
+}
diff --git a/store/viewers/web-viewer/test/process.test.mjs b/store/viewers/web-viewer/test/process.test.mjs
index 35dba12f4..a72f1f9e6 100644
--- a/store/viewers/web-viewer/test/process.test.mjs
+++ b/store/viewers/web-viewer/test/process.test.mjs
@@ -1,15 +1,22 @@
// viewer.mjs as Harness runs it: `node viewer.mjs` with HARNESS_WORKSPACE and HARNESS_VIEWER_PORT,
-// refusing to start without them, and closing cleanly (open streams included) on SIGTERM or SIGINT.
+// refusing to start without them, and closing cleanly (open streams included) on SIGTERM or SIGINT. And
+// the manifest's commands, viewer.sh and doctor.sh, on a PATH of only what they use: they find Node
+// through runtimes.sh — this machine's, or Harness's own under HOME (~/.harness/runtime/current-node),
+// the Node a fresh Mac with none on PATH gets.
import assert from 'node:assert/strict';
import { spawn, spawnSync } from 'node:child_process';
-import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { chmod, mkdir, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs/promises';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
+import { existsSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { after, before, test } from 'node:test';
const viewer = fileURLToPath(new URL('../viewer.mjs', import.meta.url));
+const pkg = dirname(viewer);
+// A bash line tracer's hooks (BASH_ENV sourcing a DEBUG trap that appends to SHCOV_OUT), passed through when set.
+const TRACER = Object.fromEntries(['BASH_ENV', 'SHCOV_OUT'].filter((k) => process.env[k]).map((k) => [k, process.env[k]]));
let workspace;
before(async () => {
workspace = await mkdtemp(join(tmpdir(), 'web-viewer-process-'));
@@ -69,15 +76,87 @@ for (const signal of ['SIGTERM', 'SIGINT']) {
});
}
-test('the manifest\'s doctor passes on Node 20 or newer and fails on an older one, run as Harness runs it', async () => {
- const { toolchain } = JSON.parse(await readFile(new URL('../harness.json', import.meta.url), 'utf8'));
- const doctor = (bin) => spawnSync('/bin/sh', ['-c', toolchain.doctor], { env: { PATH: bin }, encoding: 'utf8' }).status;
- assert.equal(doctor(dirname(process.execPath)), 0, `node ${process.version}`);
- const old = join(workspace, 'old-node');
- await mkdir(old);
+const scratches = [];
+after(() => Promise.all(scratches.map((dir) => rm(dir, { recursive: true, force: true }))));
+/** A fresh temp directory, removed after the tests. */
+async function scratch(name) {
+ const dir = await mkdtemp(join(tmpdir(), `web-viewer-${name}-`));
+ scratches.push(dir);
+ return dir;
+}
+
+/** A PATH of bash, dirname and cat, plus `node` when given (a path to link, or a script body). */
+async function binWith(node) {
+ const bin = await scratch('bin');
+ for (const tool of ['bash', 'dirname', 'cat']) await symlink(['/bin', '/usr/bin'].map((d) => join(d, tool)).find((p) => existsSync(p)), join(bin, tool));
+ if (node?.startsWith('#!')) { await writeFile(join(bin, 'node'), node); await chmod(join(bin, 'node'), 0o755); }
+ else if (node) await symlink(node, join(bin, 'node'));
+ return bin;
+}
+
+/** A HOME, with Harness's own Node recorded in it when `node` is given (a path). */
+async function homeWith(node) {
+ const home = await scratch('home');
+ if (node) {
+ await mkdir(join(home, '.harness', 'runtime'), { recursive: true });
+ await writeFile(join(home, '.harness', 'runtime', 'current-node'), node);
+ }
+ return home;
+}
+
+/** A node that is the real one but says it is 18.20.4. */
+async function oldNode() {
+ const old = await scratch('old-node');
const fake = 'Object.defineProperty(process, "versions", { value: { ...process.versions, node: "18.20.4" } })'; // no single quote: it goes inside one
- await writeFile(join(old, 'node'), `#!/bin/sh\nexec '${process.execPath}' --import 'data:text/javascript,${encodeURIComponent(fake)}' "$@"\n`);
+ // `node -v` is answered by the binary before any --import, so the stand-in answers it itself.
+ await writeFile(join(old, 'node'), `#!/bin/sh\n[ "$1" = -v ] && { echo v18.20.4; exit 0; }\nexec '${process.execPath}' --import 'data:text/javascript,${encodeURIComponent(fake)}' "$@"\n`);
await chmod(join(old, 'node'), 0o755);
assert.equal(spawnSync(join(old, 'node'), ['-p', 'process.versions.node'], { encoding: 'utf8' }).stdout.trim(), '18.20.4', 'the stand-in reports an old Node');
- assert.equal(doctor(old), 1);
+ return join(old, 'node');
+}
+
+const noHarnessNode = (home) => `miss node >= 20, and Harness's own Node is not in ${home}/.harness/runtime — run \`harness start\` once to lay it down\n`;
+
+test('the manifest runs doctor.sh and viewer.sh, from the package', async () => {
+ const { toolchain, viewer: own } = JSON.parse(await readFile(new URL('../harness.json', import.meta.url), 'utf8'));
+ assert.deepEqual([toolchain.doctor, own.command], ['./doctor.sh', './viewer.sh']);
});
+
+test('doctor.sh passes on Node 20 or newer — this machine\'s or Harness\'s own — and fails on an older one or none, run as Harness runs it', async () => {
+ const { toolchain } = JSON.parse(await readFile(new URL('../harness.json', import.meta.url), 'utf8'));
+ const doctor = async (bin, home) => {
+ const r = spawnSync('/bin/sh', ['-c', toolchain.doctor], { cwd: pkg, env: { PATH: bin, HOME: home, ...TRACER }, encoding: 'utf8' });
+ return [r.status, r.stdout];
+ };
+ const ok = [0, `ok node ${process.version}\n`];
+ assert.deepEqual(await doctor(await binWith(process.execPath), await homeWith()), ok, `node ${process.version}`);
+ assert.deepEqual(await doctor(await binWith(), await homeWith(process.execPath)), ok, 'no node on PATH, Harness\'s own');
+ const bare = await homeWith();
+ assert.deepEqual(await doctor(await binWith(await oldNode()), bare), [1, noHarnessNode(bare)]);
+ assert.deepEqual(await doctor(await binWith(), bare), [1, noHarnessNode(bare)]);
+ const old = await oldNode();
+ assert.deepEqual(await doctor(await binWith(old), await homeWith(old)), [1, "miss node >= 20 — this machine's newest is v18.20.4, Harness's own; update Harness\n"]);
+});
+
+test('viewer.sh serves from a login shell with no node on PATH on Harness\'s own Node, and exits cleanly on SIGTERM; with no Node at all, a miss', async () => {
+ const bin = await binWith();
+ const port = await freePort();
+ const child = spawn('/bin/sh', ['-c', 'exec ./viewer.sh'], { cwd: pkg, env: { PATH: bin, HOME: await homeWith(process.execPath), HARNESS_WORKSPACE: workspace, HARNESS_VIEWER_PORT: String(port), ...TRACER }, stdio: ['ignore', 'pipe', 'pipe'] });
+ let stdout = '', stderr = '';
+ child.stdout.on('data', (chunk) => { stdout += chunk; });
+ child.stderr.on('data', (chunk) => { stderr += chunk; });
+ const exited = new Promise((resolve) => child.on('exit', (code) => resolve(code)));
+ const deadline = Date.now() + 10_000;
+ while (!stdout.includes(`[web-viewer] http://127.0.0.1:${port}/`)) {
+ assert.ok(Date.now() < deadline, `the viewer did not start: ${stderr}`);
+ await new Promise((resolve) => setTimeout(resolve, 25));
+ }
+ assert.match(await (await fetch(`http://127.0.0.1:${port}/files/index.html`)).text(), /Hello, process/);
+ child.kill('SIGTERM');
+ assert.equal(await exited, 0, 'exec: the signal reaches node, which exits cleanly');
+
+ const home = await homeWith();
+ const none = spawnSync('/bin/sh', ['-c', './viewer.sh'], { cwd: pkg, env: { PATH: bin, HOME: home, HARNESS_WORKSPACE: workspace, HARNESS_VIEWER_PORT: String(port), ...TRACER }, encoding: 'utf8' });
+ assert.deepEqual([none.status, none.stdout], [1, noHarnessNode(home)]);
+});
+
diff --git a/store/viewers/web-viewer/viewer.sh b/store/viewers/web-viewer/viewer.sh
new file mode 100755
index 000000000..6877d9b01
--- /dev/null
+++ b/store/viewers/web-viewer/viewer.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# The pane. Harness runs this here with HARNESS_VIEWER_PORT and HARNESS_WORKSPACE, through a login shell
+# whose PATH may hold no node: runtimes.sh finds this machine's or Harness's own. viewer.mjs itself
+# checks both variables.
+set -euo pipefail
+here="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=runtimes.sh
+. "$here/runtimes.sh"
+harness_node 20 || exit 1
+exec node "$here/viewer.mjs"