Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Measure OS payload compression and verify extracted filesystem fidelity (#704)

Dee06e0724e285c
diff --git a/.github/workflows/os-compression.yml b/.github/workflows/os-compression.yml
new file mode 100644
index 000000000..cd0e2237f
--- /dev/null
+++ b/.github/workflows/os-compression.yml
@@ -0,0 +1,78 @@
+name: Harness OS compression assessment
+
+on:
+ workflow_dispatch:
+ inputs:
+ image_run_id:
+ description: 'Exact existing image artifact to assess'
+ type: string
+ default: '37175252094'
+ required: true
+ iso_sha256:
+ description: 'Trusted SHA-256 of that image'
+ type: string
+ default: 'd7fe30c65310904decb67a03dc14a85afc48ed76e3b2f62de5431f9354beed6f'
+ required: true
+ native_comparison:
+ description: 'Also compare real 1 GiB USB installs and boots (levels 6/19)'
+ type: boolean
+ default: false
+
+permissions:
+ contents: read
+ actions: read
+
+jobs:
+ compression:
+ runs-on: ubuntu-24.04
+ timeout-minutes: 65
+ defaults:
+ run:
+ shell: bash
+ steps:
+ - uses: actions/checkout@v7
+ - name: Validate the assessment and install filesystem tools
+ run: |
+ python3 -m py_compile os/tests/payload_compression.py os/tests/compression_vm.py os/tests/test_compression.py
+ go run github.com/rhysd/actionlint/cmd/[email protected] .github/workflows/os-compression.yml
+ sudo apt-get update -qq
+ sudo apt-get install -y --no-install-recommends squashfs-tools xorriso time libcap2-bin
+ sudo python3 -m unittest discover -s os/tests -p test_compression.py -v |& tee /tmp/harness-compression-tests.log
+ - name: Prepare native VM comparison
+ if: ${{ inputs.native_comparison }}
+ run: |
+ sudo apt-get install -y --no-install-recommends qemu-system-x86 qemu-utils ovmf python3-pil tesseract-ocr
+ test -e /dev/kvm
+ sudo chmod a+rw /dev/kvm
+ - uses: actions/download-artifact@v6
+ with:
+ name: programmer-os-x86_64
+ path: os/dist
+ run-id: ${{ inputs.image_run_id }}
+ github-token: ${{ github.token }}
+ - name: Measure unchanged filesystem at three compression levels
+ env:
+ ISO_SHA256: ${{ inputs.iso_sha256 }}
+ NATIVE_COMPARISON: ${{ inputs.native_comparison }}
+ run: |
+ options=()
+ if [[ "$NATIVE_COMPARISON" == true ]]; then options+=(--vm); fi
+ sudo python3 os/tests/payload_compression.py --iso os/dist/*.iso --sha256 "$ISO_SHA256" "${options[@]}"
+ - name: Retain evidence without images or extracted files
+ if: always()
+ uses: actions/upload-artifact@v6
+ with:
+ name: harness-os-compression
+ path: |
+ /tmp/harness-compression-tests.log
+ os/test-results/compression/*.json
+ os/test-results/compression/*.json.gz
+ os/test-results/compression/*.log
+ os/test-results/compression/*.time
+ os/test-results/compression/*/*.json
+ os/test-results/compression/*/*.jsonl
+ os/test-results/compression/*/*.png
+ os/test-results/compression/*/*.txt
+ os/test-results/compression/*/*.log
+ retention-days: 14
+ compression-level: 0
diff --git a/os/DEVELOPMENT.md b/os/DEVELOPMENT.md
index 18036af81..b4ae0baf0 100644
--- a/os/DEVELOPMENT.md
+++ b/os/DEVELOPMENT.md
@@ -105,6 +105,41 @@ establish a precise causal difference. It provides no evidence for a saving from
these flags; the shipped defaults remain unchanged. Retain raw per-round data
and use representative long-running agent work before adopting a memory limit.
+The **Harness OS compression assessment** workflow compares Zstandard levels 6,
+15, and 19 on the same checksum-verified ISO filesystem. It rebuilds level 6 with
+the same runner tools as the alternatives, then measures three extractions per
+level using the installer's 64 MiB SquashFS cache. Receipts include compression
+time and peak process memory, payload bytes, extraction timings, tool versions,
+and a filesystem manifest. Every extraction must preserve file hashes, ownership,
+permissions, timestamps, symlink targets, hardlink groups, device numbers and
+extended attributes, including capabilities. A small native roundtrip checks the
+verifier before the full image download.
+
+Ubuntu's 4.6.1 extractor uses explicit 32 MiB data and fragment queues; these
+match the queue allocation of `-mem 64M` in newer SquashFS tools. The installed
+OS keeps its existing extractor and command.
+
+The default assessment uses an unconstrained Linux host with a warm/uncontrolled
+page cache. It does not measure a complete installation or low-memory boot.
+Enable `native_comparison` to repack the same source ISO at levels 6 and 19,
+preserving its boot layout and regenerating its payload checksum. It runs three
+alternating BIOS/plain installations per level and one UEFI/encrypted installation
+per level. Every trial uses a fresh 1 GiB KVM guest, Nehalem CPU profile, private
+disk and USB overlay, and disconnected network. It verifies the actual media
+checksum, saves a trial project, installs offline, reboots, checks that project
+and sends real keyboard input through the graphical terminal. Guest caches are
+dropped after the integrity read and before installation. Guest memory/swap is
+sampled every 250 ms; install, live boot and installed boot measurements remain
+separate. Retain every trial, including failures.
+
+The native comparison still uses an unthrottled virtual USB and uncontrolled host
+cache; it cannot establish physical USB throughput or laptop boot time. It calls
+no models and does not replace the full release journey. Run either assessment
+through Actions with the existing image's `image_run_id` and independently trusted
+`iso_sha256`. It retains receipts/screenshots, discards candidate images and changes
+no release defaults. Adopting a candidate requires a production image build and
+the normal plain/encrypted installation, agent trial and recovery checks.
+
## The feedback loop
| Work being tested | Best environment | What it proves |
diff --git a/os/tests/compression_vm.py b/os/tests/compression_vm.py
new file mode 100644
index 000000000..69942b1d2
--- /dev/null
+++ b/os/tests/compression_vm.py
@@ -0,0 +1,146 @@
+#!/usr/bin/env python3
+"""Measure unchanged payloads in fresh, offline 1 GiB USB installations."""
+import base64
+import hashlib
+import json
+from pathlib import Path
+import shlex
+import time
+
+from vm import VM, check_graphical_keyboard
+
+
+# Run inside the disposable guest. Sample the whole guest, including decompressor
+# caches, rather than presenting the host's extraction RSS as machine memory use.
+INSTALL_PROBE = '''import json, os, resource, signal, subprocess, time
+from pathlib import Path
+samples = []
+failure = None
+started = time.monotonic()
+before = resource.getrusage(resource.RUSAGE_CHILDREN)
+with open('/tmp/compression-install.log', 'wb') as log:
+ child = subprocess.Popen(['harness', 'install', '--config', '/tmp/compression-config.json',
+ '--yes-erase-disk'], stdout=log, stderr=subprocess.STDOUT,
+ start_new_session=True)
+ try:
+ while child.poll() is None:
+ fields = {k: int(v.split()[0]) for k, v in (line.split(':', 1)
+ for line in Path('/proc/meminfo').read_text().splitlines())}
+ samples.append(dict(seconds=round(time.monotonic() - started, 3),
+ used_kib=fields['MemTotal']-fields['MemAvailable'],
+ available_kib=fields['MemAvailable'],
+ swap_used_kib=fields['SwapTotal']-fields['SwapFree']))
+ if time.monotonic() - started > 600:
+ raise TimeoutError('Offline installation exceeded ten minutes')
+ time.sleep(.25)
+ except BaseException as error:
+ failure = repr(error)
+ finally:
+ if child.poll() is None:
+ os.killpg(child.pid, signal.SIGTERM)
+ try:
+ child.wait(timeout=5)
+ except subprocess.TimeoutExpired:
+ os.killpg(child.pid, signal.SIGKILL)
+ child.wait()
+after = resource.getrusage(resource.RUSAGE_CHILDREN)
+Path('/tmp/compression-install.json').write_text(json.dumps(dict(
+ exit_status=child.returncode, error=failure, wall_seconds=time.monotonic()-started,
+ user_seconds=after.ru_utime-before.ru_utime, system_seconds=after.ru_stime-before.ru_stime,
+ samples=samples)))
+raise SystemExit(child.returncode)
+'''
+
+
+def run_trial(iso, folder, firmware):
+ folder.mkdir(parents=True, exist_ok=False)
+ result = dict(status='running', started_at=time.time(), firmware=firmware,
+ memory_mib=1024, cpu='Nehalem', live_transport='usb', network='disconnected')
+ vm = VM(folder, iso, firmware, 1024, 'usb', 'Nehalem')
+ config = dict(disk='/dev/vda', expected_serial='HN_OS_TEST', confirm_erase='/dev/vda',
+ username='me', hostname='harness', password='test-password-123',
+ encrypt=firmware == 'uefi', serial_console=True)
+ user = lambda cmd: ('runuser -u me -- env XDG_RUNTIME_DIR=/run/user/1000 '
+ 'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus ' + cmd)
+ project = b'Compression must preserve the trial project.\n'
+ try:
+ vm.start(live=True)
+ if vm.acceleration != 'kvm':
+ raise RuntimeError('Native KVM is required for the comparison')
+ vm.monitor('set_link', name='hnnet', up=False)
+ vm.wait(r'root@[^\r\n]*[#] ')
+ vm.shell_ready = True
+ vm.command('stty -echo')
+ vm.command(user('/usr/lib/harness-os/wait-runtime'), timeout=160)
+ vm.command(user('sh -c ' + shlex.quote(
+ 'for n in $(seq 1 60); do hn capture-pane -p | '
+ 'grep -q "Connect to start with an agent" && exit 0; sleep .25; done; exit 1')))
+ result['live_harness_ready_seconds'] = round(time.monotonic() - vm.started, 3)
+ vm.command('! pgrep -x chromium && ! pgrep -x opencode')
+ vm.screenshot('live-network')
+ vm.command('nmcli networking off')
+ # Check the recompressed media's actual payload checksum in the guest.
+ vm.command('cd /run/archiso/bootmnt/arch/x86_64 && sha512sum -c airootfs.sha512', timeout=90)
+ vm.command(user('sh -c ' + shlex.quote('mkdir -p "$HOME/Projects/compression"; printf %s ' +
+ shlex.quote(base64.b64encode(project).decode()) +
+ ' | base64 -d > "$HOME/Projects/compression/retained.txt"')))
+ for name, data in [('config.json', json.dumps(config).encode()),
+ ('probe.py', INSTALL_PROBE.encode())]:
+ encoded = base64.b64encode(data).decode()
+ vm.command(': > /tmp/compression-' + name + '.b64')
+ for offset in range(0, len(encoded), 2000):
+ vm.command('printf %s ' + encoded[offset:offset + 2000] +
+ ' >> /tmp/compression-' + name + '.b64')
+ vm.command('base64 -d /tmp/compression-' + name + '.b64 > /tmp/compression-' + name)
+ # The integrity read above must not give one compression level a larger
+ # guest page-cache head start. Host cache state is recorded as uncontrolled.
+ vm.command('sync; echo 3 > /proc/sys/vm/drop_caches')
+ _, status = vm.command('python3 /tmp/compression-probe.py', timeout=650, check=False)
+ (folder / 'install.log').write_bytes(vm.read_file('/tmp/compression-install.log'))
+ measured = vm.read_file('/tmp/compression-install.json')
+ (folder / 'install-measurements.json').write_bytes(measured)
+ result['installation'] = json.loads(measured)
+ samples = result['installation'].pop('samples')
+ assert status == result['installation']['exit_status'] == 0, 'Offline installation failed'
+ assert samples, 'Installation produced no memory samples'
+ result['installation'].update(peak_used_mib=max(r['used_kib'] for r in samples) / 1024,
+ minimum_available_mib=min(r['available_kib'] for r in samples) / 1024,
+ peak_swap_used_mib=max(r['swap_used_kib'] for r in samples) / 1024,
+ sample_count=len(samples))
+ vm.command('sync')
+ vm.stop()
+ vm.start(live=False)
+ vm.monitor('set_link', name='hnnet', up=False)
+ vm.login_installed(config)
+ result['installed_harness_ready_seconds_including_test_login'] = round(time.monotonic()-vm.started, 3)
+ if config['encrypt']:
+ result['unlock_prompt_seconds'] = vm.unlock_prompt_seconds
+ result['keyboard'] = check_graphical_keyboard(vm, 'compression')
+ assert vm.read_file('/home/me/Projects/compression/retained.txt') == project
+ vm.command('test ! -e /etc/harness-live && ! pgrep -x chromium && ! pgrep -x opencode')
+ # A new disk/boot per trial prevents page-cache reuse inside the guest.
+ vm.command('sleep 20; hn-os measure > /tmp/compression-idle.json', timeout=35)
+ result['idle'] = json.loads(vm.read_file('/tmp/compression-idle.json'))
+ result['install_receipt'] = json.loads(vm.read_file('/var/lib/harness-os/install.json'))
+ vm.screenshot('installed-home')
+ vm.boot_diagnostics(config, 'installed')
+ result.update(status='passed', trial_project_sha256=hashlib.sha256(project).hexdigest())
+ except BaseException as error:
+ result.update(status='failed', error=repr(error))
+ try:
+ vm.screenshot('failure')
+ except Exception:
+ pass
+ raise
+ finally:
+ vm.stop()
+ vm.log.close()
+ vm.stderr.close()
+ vm.control.cleanup()
+ # Only large files owned by this disposable VM are discarded.
+ for path in [vm.disk, getattr(vm, 'usb', None)]:
+ if path is not None:
+ path.unlink(missing_ok=True)
+ result['finished_at'] = time.time()
+ (folder / 'receipt.json').write_text(json.dumps(result, indent=2) + '\n')
+ return result
diff --git a/os/tests/payload_compression.py b/os/tests/payload_compression.py
new file mode 100644
index 000000000..1954f48c7
--- /dev/null
+++ b/os/tests/payload_compression.py
@@ -0,0 +1,305 @@
+#!/usr/bin/env python3
+"""Compare compression of one verified OS payload, without changing its contents."""
+import argparse
+from collections import Counter
+import gzip
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import re
+import shutil
+import signal
+import stat
+import statistics
+import subprocess
+import tempfile
+import time
+
+
+def digest(path):
+ with path.open('rb') as handle:
+ return hashlib.file_digest(handle, 'sha256').hexdigest()
+
+
+def canonical(value):
+ return json.dumps(value, sort_keys=True, separators=(',', ':')).encode()
+
+
+def inventory(root):
+ """Never follow symlinks; compare data and all persistent SquashFS metadata."""
+ paths = [root]
+ for folder, directories, files in os.walk(root, followlinks=False):
+ paths.extend(Path(folder) / name for name in directories + files)
+ entries, groups = {}, {}
+ for path in sorted(paths):
+ info = path.lstat()
+ name = path.relative_to(root).as_posix()
+ row = dict(mode=info.st_mode, uid=info.st_uid, gid=info.st_gid,
+ mtime=int(info.st_mtime), xattrs={})
+ for attribute in sorted(os.listxattr(path, follow_symlinks=False)):
+ value = os.getxattr(path, attribute, follow_symlinks=False)
+ row['xattrs'][attribute] = hashlib.sha256(value).hexdigest()
+ if stat.S_ISREG(info.st_mode):
+ row.update(bytes=info.st_size, sha256=digest(path))
+ groups.setdefault((info.st_dev, info.st_ino), []).append(name)
+ elif stat.S_ISLNK(info.st_mode):
+ row['target'] = os.readlink(path)
+ elif stat.S_ISCHR(info.st_mode) or stat.S_ISBLK(info.st_mode):
+ row['device'] = [os.major(info.st_rdev), os.minor(info.st_rdev)]
+ entries[name] = row
+ hardlinks = sorted(sorted(group) for group in groups.values() if len(group) > 1)
+ return dict(entries=entries, hardlinks=hardlinks)
+
+
+def compare(expected, actual):
+ if expected == actual:
+ return None
+ names = expected['entries'].keys() | actual['entries'].keys()
+ changed = [name for name in sorted(names)
+ if expected['entries'].get(name) != actual['entries'].get(name)]
+ return dict(changed_paths=changed[:100], changed_path_count=len(changed),
+ hardlinks_changed=expected['hardlinks'] != actual['hardlinks'])
+
+
+def measured(command, folder, label, timeout=900):
+ """GNU time covers the command; timeouts stop only this owned process group."""
+ metrics = folder / (label + '.time')
+ started = time.monotonic()
+ with (folder / (label + '.log')).open('wb') as output:
+ child = subprocess.Popen(['/usr/bin/time', '-f', '%e %U %S %M', '-o', str(metrics),
+ *map(str, command)], stdout=output, stderr=subprocess.STDOUT,
+ start_new_session=True)
+ try:
+ status = child.wait(timeout=timeout)
+ except BaseException:
+ try:
+ os.killpg(child.pid, signal.SIGTERM)
+ except ProcessLookupError:
+ pass
+ try:
+ child.wait(timeout=5)
+ except subprocess.TimeoutExpired:
+ os.killpg(child.pid, signal.SIGKILL)
+ child.wait()
+ raise
+ if status:
+ detail = (folder / (label + '.log')).read_text(errors='replace')[-4000:]
+ raise RuntimeError(f'{label} exited {status}; see {label}.log\n{detail}')
+ elapsed, user, system, rss = map(float, metrics.read_text().splitlines()[-1].split())
+ return dict(command=list(map(str, command)), wall_seconds=time.monotonic() - started,
+ command_seconds=elapsed, user_seconds=user, system_seconds=system,
+ peak_rss_kib=int(rss))
+
+
+def extract(image, target, folder, label):
+ # Upstream 4.7's -mem 64M sets each of these queues to 32 MiB. The explicit
+ # form also works with Ubuntu 24.04's 4.6.1 tools; compare like with like.
+ return measured(['unsquashfs', '-processors', '2', '-data-queue', '32',
+ '-frag-queue', '32', '-no-progress',
+ '-d', target, image], folder, label)
+
+
+def package_sizes(root):
+ result = []
+ for desc in sorted((root / 'var/lib/pacman/local').glob('*/desc')):
+ fields = {}
+ for block in desc.read_text().strip().split('\n\n'):
+ lines = block.splitlines()
+ if len(lines) >= 2:
+ fields[lines[0]] = lines[1:]
+ # Installed sizes are ancillary evidence. A package without that field
+ # must be reported as unknown, not as zero or a failed filesystem test.
+ size = fields.get('%SIZE%') or fields.get('%ISIZE%')
+ row = dict(name=fields['%NAME%'][0], bytes=int(size[0]) if size else None)
+ if size is None:
+ row.update(size_status='not recorded', database_fields=sorted(fields))
+ result.append(row)
+ return sorted(result, key=lambda row: (row['bytes'] is None, -(row['bytes'] or 0), row['name']))
+
+
+def tool_versions():
+ versions = {}
+ for tool in ['mksquashfs', 'unsquashfs', 'xorriso']:
+ # unsquashfs 4.6.1 prints its version and exits 1 without an input image.
+ probe = subprocess.run([tool, '-version'], stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT, text=True, timeout=10)
+ if probe.returncode not in (0, 1) or not re.search(
+ rf'(?im)^{re.escape(tool)} (?:version )?\d+\.\d+', probe.stdout):
+ raise RuntimeError(f'Cannot identify {tool}: {probe.returncode}\n{probe.stdout}')
+ versions[tool] = dict(output=probe.stdout.strip(), returncode=probe.returncode)
+ return versions
+
+
+def repack(iso, payload, destination, folder, label):
+ """Keep the original boot layout while replacing only payload and checksum."""
+ # xorriso's documented replay operation must follow file modifications:
+ # https://www.gnu.org/software/xorriso/man_1_xorriso.html
+ checksum = payload.with_suffix('.sha512')
+ with payload.open('rb') as handle:
+ checksum.write_text(hashlib.file_digest(handle, 'sha512').hexdigest() + ' airootfs.sfs\n')
+ measured(['xorriso', '-no_rc', '-indev', iso, '-outdev', destination,
+ '-map', payload, '/arch/x86_64/airootfs.sfs',
+ '-map', checksum, '/arch/x86_64/airootfs.sha512',
+ '-boot_image', 'any', 'replay'], folder, label)
+ measured(['xorriso', '-no_rc', '-indev', destination, '-report_el_torito', 'plain',
+ '-report_system_area', 'plain'], folder, label + '-boot-layout')
+ return dict(bytes=destination.stat().st_size, sha256=digest(destination))
+
+
+def compare_native(variants, folder):
+ from compression_vm import run_trial
+ # Alternate ordering to expose host drift; every row gets a new guest, disk,
+ # USB overlay and guest page cache. Host caches remain uncontrolled.
+ sequence = [('bios', n) for n in [6, 19, 19, 6, 6, 19]] + [('uefi', 6), ('uefi', 19)]
+ result = dict(status='running', sequence=sequence, trials=[], summaries=[],
+ limits=['Fresh 1 GiB guests on one shared native CI host; not physical laptops.',
+ 'Guest file caches are dropped after integrity checks, before installation.',
+ 'Virtual USB is unthrottled; host page cache is uncontrolled.',
+ 'Boot timings include fixture login and readiness probes.',
+ 'No models are called; installation and shell input are offline.',
+ 'No automatic adoption or publication.'])
+ path = folder / 'native-comparison.json'
+ try:
+ for index, (firmware, level) in enumerate(sequence, 1):
+ name = f'{index:02}-{firmware}-level-{level}'
+ print('Measuring fresh 1 GiB USB installation: ' + name, flush=True)
+ trial = run_trial(variants[level], folder / name, firmware)
+ result['trials'].append(dict(level=level, **trial))
+ path.write_text(json.dumps(result, indent=2) + '\n')
+ for firmware in ['bios', 'uefi']:
+ for level in [6, 19]:
+ rows = [r for r in result['trials'] if r['firmware'] == firmware and r['level'] == level]
+ result['summaries'].append(dict(firmware=firmware, level=level, repetitions=len(rows),
+ median_install_seconds=statistics.median(r['install_receipt']['duration_seconds'] for r in rows),
+ median_live_ready_seconds=statistics.median(r['live_harness_ready_seconds'] for r in rows),
+ median_installed_ready_seconds=statistics.median(r['installed_harness_ready_seconds_including_test_login'] for r in rows),
+ max_install_used_mib=max(r['installation']['peak_used_mib'] for r in rows),
+ max_install_swap_mib=max(r['installation']['peak_swap_used_mib'] for r in rows)))
+ result['status'] = 'passed'
+ except BaseException as error:
+ result.update(status='failed', error=repr(error))
+ raise
+ finally:
+ path.write_text(json.dumps(result, indent=2) + '\n')
+ return result
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--iso', type=Path, required=True)
+ parser.add_argument('--sha256', required=True)
+ parser.add_argument('--output', type=Path, default=Path('os/test-results/compression'))
+ parser.add_argument('--vm', action='store_true', help='Compare rebuilt 6/19 with fresh 1 GiB USB installs')
+ args = parser.parse_args()
+ if platform.system() != 'Linux' or os.geteuid() != 0:
+ parser.error('Use root on a disposable native Linux runner to preserve filesystem metadata')
+ iso = args.iso.resolve()
+ manifest = json.loads(iso.with_name('manifest.json').read_text())
+ if (iso.name != manifest['iso']['name'] or iso.stat().st_size != manifest['iso']['bytes']
+ or digest(iso) != args.sha256 or args.sha256 != manifest['iso']['sha256']):
+ parser.error('The ISO, manifest, and supplied trusted SHA-256 must all match')
+ for tool in ['mksquashfs', 'unsquashfs', 'xorriso', '/usr/bin/time']:
+ if not shutil.which(tool):
+ parser.error('Missing required tool: ' + tool)
+ if args.vm:
+ if not os.access('/dev/kvm', os.R_OK | os.W_OK):
+ parser.error('The native comparison requires accessible KVM')
+ for tool in ['qemu-system-x86_64', 'qemu-img', 'tesseract']:
+ if not shutil.which(tool):
+ parser.error('Missing native VM tool: ' + tool)
+ folder = args.output.resolve()
+ folder.parent.mkdir(parents=True, exist_ok=True)
+ if shutil.disk_usage(folder.parent).free < 18 * 1024**3:
+ parser.error('This assessment requires 18 GiB free space after the ISO download')
+ folder.mkdir(parents=True, exist_ok=False)
+ result = dict(status='running', started_at=time.time(), iso=manifest['iso'],
+ image_source_commit=manifest['source_commit'], profiles=[],
+ runner=dict(platform=platform.platform(), cpu_count=os.cpu_count(),
+ memory=Path('/proc/meminfo').read_text(),
+ cpu=subprocess.check_output(['lscpu'], text=True)),
+ conditions=dict(compression_processors=2, compression_memory='1G',
+ block_size='1M', extraction_processors=2,
+ extraction_memory='64M', data_queue_mib=32,
+ fragment_queue_mib=32, repetitions=3,
+ cache='Warm/uncontrolled host page cache; no cache flushing',
+ source_date_epoch=os.environ.get('SOURCE_DATE_EPOCH')),
+ limits=['Host extraction is not a complete installation or boot test.',
+ 'The 64 MiB SquashFS cache does not constrain total host memory.',
+ 'Published level 6 uses a different toolchain; use rebuilt level 6 as control.',
+ 'No candidate is promoted by this assessment.'])
+ try:
+ result['test_source_commit'] = subprocess.check_output(
+ ['git', '-c', f'safe.directory={Path.cwd()}', 'rev-parse', 'HEAD'], text=True).strip()
+ result['tools'] = tool_versions()
+ if os.environ.get('SOURCE_DATE_EPOCH'):
+ raise RuntimeError('Unset SOURCE_DATE_EPOCH: file timestamps must be preserved for this assessment')
+ with tempfile.TemporaryDirectory(prefix='payload-', dir=folder) as temp:
+ work = Path(temp)
+ original = work / 'original.sfs'
+ measured(['xorriso', '-no_rc', '-osirrox', 'on', '-indev', iso,
+ '-extract', '/arch/x86_64/airootfs.sfs', original], folder, 'payload')
+ result['published_payload'] = dict(bytes=original.stat().st_size, sha256=digest(original))
+ root = work / 'root'
+ result['original_extract'] = extract(original, root, folder, 'source-extract')
+ expected = inventory(root)
+ encoded = canonical(expected)
+ with gzip.open(folder / 'filesystem.json.gz', 'wb') as handle:
+ handle.write(encoded)
+ result['filesystem'] = dict(sha256=hashlib.sha256(encoded).hexdigest(),
+ paths=len(expected['entries']),
+ hardlink_groups=len(expected['hardlinks']),
+ types=dict(Counter(str(stat.S_IFMT(row['mode']))
+ for row in expected['entries'].values())))
+ result['package_installed_sizes'] = package_sizes(root)
+ original.unlink()
+ variants = {}
+ for level in [6, 15, 19]:
+ print(f'Compressing unchanged filesystem at zstd level {level}', flush=True)
+ image = work / f'level-{level}.sfs'
+ profile = dict(level=level, extractions=[])
+ result['profiles'].append(profile)
+ profile['compression'] = measured(
+ ['mksquashfs', root, image, '-noappend', '-no-progress', '-comp', 'zstd',
+ '-Xcompression-level', str(level), '-b', '1M', '-processors', '2', '-mem', '1G'],
+ folder, f'compress-{level}')
+ profile.update(bytes=image.stat().st_size, sha256=digest(image))
+ for repeat in range(1, 4):
+ target = work / f'extracted-{level}-{repeat}'
+ reading = extract(image, target, folder, f'extract-{level}-{repeat}')
+ profile['extractions'].append(reading)
+ difference = compare(expected, inventory(target))
+ reading['filesystem_matches'] = difference is None
+ if difference:
+ reading['difference'] = difference
+ raise RuntimeError(f'Filesystem changed at level {level}, repetition {repeat}')
+ shutil.rmtree(target)
+ profile['median_extract_seconds'] = statistics.median(
+ row['command_seconds'] for row in profile['extractions'])
+ profile['max_extract_rss_kib'] = max(row['peak_rss_kib'] for row in profile['extractions'])
+ if args.vm and level in (6, 19):
+ variants[level] = work / f'level-{level}.iso'
+ profile['repacked_iso'] = repack(iso, image, variants[level], folder, f'iso-{level}')
+ image.unlink()
+ print(json.dumps({key: value for key, value in profile.items()
+ if key not in ['compression', 'extractions']}), flush=True)
+ (folder / 'receipt.json').write_text(json.dumps(result, indent=2) + '\n')
+ control = result['profiles'][0]
+ for row in result['profiles']:
+ row['saved_bytes_vs_control'] = control['bytes'] - row['bytes']
+ row['saved_percent_vs_control'] = 100 * (1 - row['bytes'] / control['bytes'])
+ row['extraction_ratio_vs_control'] = row['median_extract_seconds'] / control['median_extract_seconds']
+ if args.vm:
+ result['native_comparison'] = compare_native(variants, folder)
+ result['status'] = 'passed'
+ except BaseException as error:
+ result.update(status='failed', error=repr(error))
+ raise
+ finally:
+ result['finished_at'] = time.time()
+ (folder / 'receipt.json').write_text(json.dumps(result, indent=2) + '\n')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/os/tests/test_compression.py b/os/tests/test_compression.py
new file mode 100644
index 000000000..9277e477c
--- /dev/null
+++ b/os/tests/test_compression.py
@@ -0,0 +1,139 @@
+import os
+from pathlib import Path
+import platform
+import shutil
+import stat
+import subprocess
+import sys
+import tempfile
+import time
+import unittest
+
+from payload_compression import compare, extract, inventory, measured, package_sizes, tool_versions
+
+
+class PackageSizes(unittest.TestCase):
+ def test_installed_database_sizes_are_ranked_without_sync_database_fields(self):
+ with tempfile.TemporaryDirectory() as temp:
+ root = Path(temp)
+ for name, size in [('small', 31), ('largest', 901), ('middle', 210)]:
+ desc = root / 'var/lib/pacman/local' / (name + '-1.0-1') / 'desc'
+ desc.parent.mkdir(parents=True)
+ desc.write_text(f'%NAME%\n{name}\n\n%VERSION%\n1.0-1\n\n%SIZE%\n{size}\n\n%REASON%\n0\n')
+ self.assertEqual(package_sizes(root), [
+ {'name': 'largest', 'bytes': 901}, {'name': 'middle', 'bytes': 210}, {'name': 'small', 'bytes': 31}])
+
+ def test_missing_size_is_explicitly_unknown_and_does_not_stop_the_assessment(self):
+ with tempfile.TemporaryDirectory() as temp:
+ root = Path(temp)
+ desc = root / 'var/lib/pacman/local/metadata-only-1/desc'
+ desc.parent.mkdir(parents=True)
+ desc.write_text('%NAME%\nmetadata-only\n\n%VERSION%\n1\n')
+ row, = package_sizes(root)
+ self.assertIsNone(row['bytes'])
+ self.assertEqual(row['size_status'], 'not recorded')
+ self.assertEqual(row['database_fields'], ['%NAME%', '%VERSION%'])
+
+
+class Fixture:
+ def setUp(self):
+ self.temp = tempfile.TemporaryDirectory()
+ self.addCleanup(self.temp.cleanup)
+ self.base = Path(self.temp.name)
+ self.root = self.base / 'source'
+ self.root.mkdir()
+ self.file = self.root / 'executable'
+ self.file.write_bytes(b'original')
+ self.file.chmod(0o755)
+ os.link(self.file, self.root / 'hardlink')
+ (self.root / 'symlink').symlink_to('executable')
+
+
[email protected](platform.system() == 'Linux', 'Linux filesystem extended attribute API')
+class InventoryTest(Fixture, unittest.TestCase):
+ def test_inventory_is_stable_and_does_not_follow_symlinks(self):
+ (self.base / 'outside').mkdir()
+ (self.base / 'outside' / 'secret').write_text('outside the image')
+ (self.root / 'outside').symlink_to(self.base / 'outside', target_is_directory=True)
+ first = inventory(self.root)
+ self.assertIsNone(compare(first, inventory(self.root)))
+ self.assertNotIn('outside/secret', first['entries'])
+ self.assertEqual([['executable', 'hardlink']], first['hardlinks'])
+
+ def test_same_length_corruption_is_detected(self):
+ first = inventory(self.root)
+ info = self.file.stat()
+ self.file.write_bytes(b'corrupt!')
+ os.utime(self.file, ns=(info.st_atime_ns, info.st_mtime_ns))
+ self.assertIn('executable', compare(first, inventory(self.root))['changed_paths'])
+
+ def test_permissions_and_symlink_targets_are_detected(self):
+ first = inventory(self.root)
+ self.file.chmod(0o644)
+ link = self.root / 'symlink'
+ link.unlink()
+ link.symlink_to('elsewhere')
+ changed = compare(first, inventory(self.root))['changed_paths']
+ self.assertIn('executable', changed)
+ self.assertIn('symlink', changed)
+
+ def test_broken_hardlink_is_detected_with_identical_file_contents(self):
+ first = inventory(self.root)
+ link = self.root / 'hardlink'
+ link.unlink()
+ shutil.copy2(self.file, link)
+ self.assertTrue(compare(first, inventory(self.root))['hardlinks_changed'])
+
+ @unittest.skipUnless(platform.system() == 'Linux', 'Linux extended attribute semantics')
+ def test_xattr_change_is_detected(self):
+ os.setxattr(self.file, 'user.harness-test', b'original')
+ first = inventory(self.root)
+ os.setxattr(self.file, 'user.harness-test', b'changed')
+ self.assertIn('executable', compare(first, inventory(self.root))['changed_paths'])
+
+
[email protected](platform.system() == 'Linux', 'Needs Linux SquashFS and GNU time')
+class NativeTest(Fixture, unittest.TestCase):
+ @unittest.skipUnless(all(shutil.which(tool) for tool in ['mksquashfs', 'unsquashfs', 'xorriso']),
+ 'Needs the assessment toolchain')
+ def test_version_reporting_accepts_tools_without_an_input_image(self):
+ versions = tool_versions()
+ self.assertEqual(set(versions), {'mksquashfs', 'unsquashfs', 'xorriso'})
+ self.assertTrue(all(row['output'] for row in versions.values()))
+
+ @unittest.skipUnless(os.geteuid() == 0 and all(shutil.which(tool) for tool in ['mksquashfs', 'unsquashfs', 'setcap']),
+ 'SquashFS metadata roundtrip needs root, squashfs-tools and libcap2-bin')
+ def test_roundtrip_preserves_privileged_metadata(self):
+ self.assertEqual(os.geteuid(), 0, 'Run the native fixture tests as root')
+ self.assertTrue(shutil.which('setcap'), 'Install libcap2-bin for capability preservation checks')
+ os.chown(self.root / 'hardlink', 1000, 1000)
+ self.file.chmod(0o4755)
+ subprocess.run(['setcap', 'cap_net_bind_service=ep', str(self.file)], check=True)
+ os.setxattr(self.file, 'user.harness-test', b'preserve me')
+ os.mkfifo(self.root / 'fifo', 0o640)
+ os.mknod(self.root / 'device', stat.S_IFCHR | 0o600, os.makedev(1, 3))
+ expected = inventory(self.root)
+ image = self.base / 'fixture.sfs'
+ measured(['mksquashfs', self.root, image, '-noappend', '-no-progress', '-comp', 'zstd',
+ '-Xcompression-level', '15', '-b', '1M', '-processors', '2', '-mem', '64M'],
+ self.base, 'compress-fixture', timeout=30)
+ extract(image, self.base / 'extracted', self.base, 'extract-fixture')
+ self.assertIsNone(compare(expected, inventory(self.base / 'extracted')))
+
+ def test_failed_command_is_not_a_measurement(self):
+ with self.assertRaisesRegex(RuntimeError, 'exited 7'):
+ measured([sys.executable, '-c', 'raise SystemExit(7)'], self.base, 'failed', timeout=5)
+
+ def test_timeout_stops_descendants(self):
+ marker = self.base / 'should-not-exist'
+ child = 'import time,pathlib; time.sleep(.6); pathlib.Path(' + repr(str(marker)) + ').touch()'
+ parent = ('import subprocess,sys,time; subprocess.Popen([sys.executable,"-c",'
+ + repr(child) + ']); time.sleep(20)')
+ with self.assertRaises(subprocess.TimeoutExpired):
+ measured([sys.executable, '-c', parent], self.base, 'timeout', timeout=.2)
+ time.sleep(.7)
+ self.assertFalse(marker.exists(), 'Timed-out command left a descendant running')
+
+
+if __name__ == '__main__':
+ unittest.main()