Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Assess optional local AI and NVIDIA packages on installed Harness (#695)

Deea77876a6a47c
diff --git a/.github/workflows/os-local-ai.yml b/.github/workflows/os-local-ai.yml
new file mode 100644
index 000000000..fe65f9279
--- /dev/null
+++ b/.github/workflows/os-local-ai.yml
@@ -0,0 +1,62 @@
+name: Harness OS optional local AI assessment
+
+on:
+ workflow_dispatch:
+ inputs:
+ image_run_id:
+ description: 'Exact Harness OS image artifact to assess'
+ type: string
+ default: '37149350400'
+ required: true
+ probe:
+ description: 'Assessment to run'
+ type: choice
+ options: [both, local-ai, nvidia]
+ default: both
+
+permissions:
+ contents: read
+ actions: read
+
+concurrency:
+ group: harness-local-ai-${{ github.ref }}
+ cancel-in-progress: false
+
+jobs:
+ optional:
+ runs-on: ubuntu-24.04
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ probe: ${{ inputs.probe == 'nvidia' && fromJSON('["nvidia"]') || inputs.probe == 'local-ai' && fromJSON('["local-ai"]') || fromJSON('["local-ai", "nvidia"]') }}
+ steps:
+ - uses: actions/checkout@v7
+ - uses: actions/download-artifact@v6
+ with:
+ name: programmer-os-x86_64
+ path: os/dist
+ run-id: ${{ inputs.image_run_id || '37149350400' }}
+ github-token: ${{ github.token }}
+ - name: Check the experiment and install VM tools
+ run: |
+ python3 -m py_compile os/tests/local_ai_guest.py os/tests/local_ai_vm.py
+ go run github.com/rhysd/actionlint/cmd/[email protected] .github/workflows/os-local-ai.yml
+ sudo apt-get update -qq
+ sudo apt-get install -y --no-install-recommends qemu-system-x86 qemu-utils ovmf python3-pil tesseract-ocr
+ sudo chmod a+rw /dev/kvm
+ - name: Assess optional packages on the installed encrypted image
+ env:
+ PROBE: ${{ matrix.probe }}
+ run: python3 os/tests/local_ai_vm.py --iso os/dist/*.iso --probe "$PROBE"
+ - name: Retain evidence without models or virtual disks
+ if: always()
+ uses: actions/upload-artifact@v6
+ with:
+ name: harness-os-${{ matrix.probe }}
+ path: |
+ os/test-results/${{ matrix.probe }}/
+ !os/test-results/${{ matrix.probe }}/*.qcow2
+ !os/test-results/${{ matrix.probe }}/OVMF_VARS.fd
+ retention-days: 14
+ compression-level: 0
diff --git a/os/DEVELOPMENT.md b/os/DEVELOPMENT.md
index 19d1810e1..f85ba4c89 100644
--- a/os/DEVELOPMENT.md
+++ b/os/DEVELOPMENT.md
@@ -5,8 +5,8 @@ boot changes. Reflashing is a release/install test, not the intended way to try
every interface fix. Keep the development tools on the build/test host; the
installed OS keeps the same minimal interface.
-This plan was recorded on October 3, 2026. Preview 5 and its small bootstrap bundle
-and update channel are published. Native update/rollback acceptance passed; remote-launcher
+This plan was recorded on October 3, 2026. Preview 6 and its small bootstrap bundle
+and update channel are published. Native upgrades from previews 4 and 5 passed; remote-launcher
argument tests passed but remote display/SSH interaction is still unverified.
## Next release priorities
@@ -56,6 +56,40 @@ keys, and checks DHCP, DNS, HTTP, reconnection and non-silent audio output. Acce
point tools are installed only inside the disposable guest, never in the ISO.
Physical radio, backlight, speaker and microphone tests remain separate.
+The **Harness OS optional local AI assessment** workflow reuses an exact image
+artifact and installs it to an encrypted disposable disk. Its two independent
+checks add packages only inside their guests:
+
+- `local-ai` installs the snapshot's CPU Ollama package and downloads a small
+ Qwen model. A project-local OpenCode configuration selects that local endpoint.
+ After one connected setup pass, networking is disabled; a direct API request
+ and an OpenCode command typed through hn must return the independently checked
+ arithmetic answer. The receipt records the model digest, actual CPU use and
+ loopback-only listener. This is a transport/compatibility check, not a coding
+ quality benchmark or a recommendation to use a tiny model for daily work.
+- `nvidia` installs matching `nvidia-open-lts` and `nvidia-utils` packages without
+ changing base packages. It checks all four module versions against the running
+ kernel, the driver's shipped support table, initramfs generation, encrypted
+ reboot, hn keyboard input and Chromium on the virtual display. No GPU is passed
+ through: binding, accelerated rendering, CUDA, physical suspend and inference
+ on a real card remain unverified. A failing `nvidia-smi` on this fixture is
+ recorded as unavailable hardware, never GPU success.
+
+Model weights, inference servers and NVIDIA packages do not enter the default
+image. Use an agent to set up the runtime and model that the actual machine and
+project need. [OpenCode's local-provider guide](https://opencode.ai/docs/providers/#ollama)
+and [Ollama's integration guide](https://docs.ollama.com/integrations/opencode)
+cover configuration; full agent workloads need substantially more context and
+memory than this small conversation test.
+
+Run the assessment with `image_run_id` and `probe` in that workflow, or on a native
+x86 KVM host with its image and matching `manifest.json`:
+
+```sh
+python3 os/tests/local_ai_vm.py --iso os/dist/IMAGE.iso --probe local-ai
+python3 os/tests/local_ai_vm.py --iso os/dist/IMAGE.iso --probe nvidia
+```
+
## The feedback loop
| Work being tested | Best environment | What it proves |
@@ -66,8 +100,8 @@ Physical radio, backlight, speaker and microphone tests remain separate.
| Future arm64 userspace and package compatibility | Accelerated ARM Linux VM on Apple Silicon | ARM application/session behavior; not Apple boot or drivers |
| Apple boot, internal storage, GPU, input and power management | Physical Mac with the matching hardware stack | Mac OS support for that exact model |
-For the current user setup, install preview 4 on the ThinkPad once and use it for
-normal programming. Run installer experiments on the server VM. Record an issue
+For the current user setup, keep the ThinkPad installation and update it in place
+for normal programming. Run installer experiments on the server VM. Record an issue
with the exact keys/action, expected result, actual result and a screenshot or
error text. Keep the image version and hardware model with the report.
@@ -238,9 +272,9 @@ the small updater; install its separately validated development bundle to add it
Intel Macs and Apple Silicon are both intended OS targets. They share the Harness
interface and behavior, but need separate platform work. None is claimed as a
-validated Harness OS hardware target by preview 5.
+validated Harness OS hardware target by preview 6.
-The next image prepares selected older Broadcom radios by PCI ID, not Mac model.
+Preview 6 prepares selected older Broadcom radios by PCI ID, not Mac model.
BCM4331 (`14e4:4331`) and BCM4360 (`14e4:43a0`) may load the optional wl driver;
an already working native interface is preserved. BCM43602 and other native
brcmfmac/brcmsmac devices are outside that selection. The vendor package's broad
diff --git a/os/tests/local_ai_guest.py b/os/tests/local_ai_guest.py
new file mode 100644
index 000000000..4d75a8551
--- /dev/null
+++ b/os/tests/local_ai_guest.py
@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Optional local inference and driver assessment in a disposable installed guest."""
+import argparse
+import hashlib
+import html
+import json
+from pathlib import Path
+import re
+import subprocess
+import time
+from urllib.request import Request, urlopen
+
+
+ROOT = Path('/home/me/local-ai-check')
+
+
+def run(*args, timeout=240, check=True):
+ result = subprocess.run(args, capture_output=True, text=True, timeout=timeout)
+ with (ROOT / 'commands.log').open('a') as log:
+ log.write(json.dumps(list(args)) + '\n' + result.stdout + result.stderr + '\n')
+ if check and result.returncode:
+ raise RuntimeError(f'{args[0]} failed ({result.returncode}): ' + result.stderr[-1500:])
+ return result
+
+
+def packages():
+ return dict(line.split(' ', 1) for line in run('pacman', '-Q').stdout.splitlines())
+
+
+def api(path, body=None, timeout=180):
+ data = json.dumps(body).encode() if body is not None else None
+ request = Request('http://127.0.0.1:11434' + path, data=data,
+ headers={'Content-Type': 'application/json'})
+ with urlopen(request, timeout=timeout) as response:
+ return json.load(response)
+
+
+def install(names):
+ before = packages()
+ started = time.monotonic()
+ run('systemctl', 'start', 'harness-keyring.service')
+ run('pacman', '-S', '--needed', '--noconfirm', *names, timeout=420)
+ after = packages()
+ assert all(after.get(name) == version for name, version in before.items()), 'Base package changed'
+ return {'added': {name: version for name, version in after.items() if name not in before},
+ 'seconds': round(time.monotonic() - started, 3), 'base_packages_unchanged': True}
+
+
+def nvidia():
+ result = install(['nvidia-open-lts', 'nvidia-utils'])
+ kernel = run('uname', '-r').stdout.strip()
+ result.update(kernel=kernel, modules={})
+ for name in ['nvidia', 'nvidia_modeset', 'nvidia_uvm', 'nvidia_drm']:
+ magic = run('modinfo', '-F', 'vermagic', name).stdout.strip()
+ version = run('modinfo', '-F', 'version', name).stdout.strip()
+ assert magic.split()[0] == kernel
+ result['modules'][name] = {'vermagic': magic, 'version': version}
+ run('modprobe', '--show-depends', name)
+ assert len({row['version'] for row in result['modules'].values()}) == 1
+ files = run('pacman', '-Qlq', 'nvidia-utils').stdout.splitlines()
+ support = next(Path(p) for p in files if p.endswith('/html/supportedchips.html'))
+ (ROOT / 'supportedchips.html').write_bytes(support.read_bytes())
+ encoded = ' '.join(html.unescape(re.sub(r'<[^>]*>', ' ', support.read_text())).split())
+ targets = ['GeForce RTX 4090', 'GeForce RTX 5090', 'RTX 6000 Ada Generation', 'RTX PRO 6000 Blackwell']
+ assert all(target in encoded for target in targets), 'Target absent from the packaged support list'
+ result['packaged_support_list_targets'] = targets
+ result['support_file_sha256'] = hashlib.sha256(support.read_bytes()).hexdigest()
+ # A VM cannot demonstrate binding, GPU compute, or display on a real card.
+ probe = run('nvidia-smi', check=False)
+ result['nvidia_smi'] = {'exit': probe.returncode, 'output': probe.stdout + probe.stderr}
+ result['physical_gpu_validation'] = 'unavailable: no NVIDIA GPU is passed through'
+ run('mkinitcpio', '-P', timeout=240)
+ result['initramfs_regenerated'] = True
+ return result
+
+
+def local_ai():
+ result = install(['ollama'])
+ # This is a test-only runtime choice. No model/server enters the OS image.
+ override = Path('/etc/systemd/system/ollama.service.d/harness-test.conf')
+ override.parent.mkdir(parents=True, exist_ok=True)
+ override.write_text('[Service]\nEnvironment=OLLAMA_HOST=127.0.0.1:11434\n'
+ 'Environment=OLLAMA_CONTEXT_LENGTH=16384\n'
+ 'Environment=OLLAMA_NUM_PARALLEL=1\n')
+ run('systemctl', 'daemon-reload')
+ run('systemctl', 'start', 'ollama')
+ deadline = time.monotonic() + 30
+ while True:
+ try:
+ result['server'] = api('/api/version', timeout=3)
+ break
+ except OSError:
+ if time.monotonic() > deadline:
+ raise
+ time.sleep(.5)
+ started = time.monotonic()
+ pulled = api('/api/pull', {'model': 'qwen3:0.6b', 'stream': False}, timeout=420)
+ assert pulled.get('status') == 'success', pulled
+ result['model_download_seconds'] = round(time.monotonic() - started, 3)
+ result['models'] = api('/api/tags')['models']
+ model = next(row for row in result['models'] if row['name'] == 'qwen3:0.6b')
+ assert model.get('digest') and model.get('size', 0) > 0
+ result['model'] = model
+ config = {'$schema': 'https://opencode.ai/config.json', 'enabled_providers': ['ollama'],
+ 'model': 'ollama/qwen3:0.6b', 'small_model': 'ollama/qwen3:0.6b',
+ 'provider': {'ollama': {'npm': '@ai-sdk/openai-compatible',
+ 'name': 'Ollama (local)', 'options': {'baseURL': 'http://127.0.0.1:11434/v1'},
+ 'models': {'qwen3:0.6b': {'name': 'Qwen 3 0.6B (CPU test)',
+ 'limit': {'context': 16384, 'output': 512}}}}}}
+ (ROOT / 'opencode.json').write_text(json.dumps(config, indent=2) + '\n')
+ # The graphical check runs this ordinary user command with networking off.
+ prompt = 'What is six times seven? Reply with only the decimal number. Do not use tools. /no_think'
+ (ROOT / 'chat.sh').write_text('#!/bin/bash\nset -euo pipefail\ncd /home/me/local-ai-check\n'
+ 'opencode run --format json ' + __import__('shlex').quote(prompt) +
+ ' > chat.jsonl 2> chat.err\npython3 verify-chat.py\n')
+ (ROOT / 'verify-chat.py').write_text('''import json,re
+from pathlib import Path
+events=[json.loads(line) for line in Path('chat.jsonl').read_text().splitlines() if line.startswith('{')]
+reply=''.join(row.get('part',{}).get('text','') for row in events if row.get('type')=='text').strip()
+assert re.fullmatch(r'(?:6\\s*[×*x]\\s*7\\s*=\\s*)?42[.!]?',reply), repr(reply)
+Path('chat-passed').write_text(reply+'\\n')
+print('Local model answered: '+reply)
+''')
+ command = Path('/home/me/.local/bin/local-chat')
+ command.parent.mkdir(parents=True, exist_ok=True)
+ command.write_text('#!/bin/sh\nexec bash /home/me/local-ai-check/chat.sh\n')
+ command.chmod(0o755)
+ run('chown', '-R', 'me:me', str(ROOT), str(command))
+ # Warm the provider while connected so any dependency fetch is completed.
+ run('runuser', '-u', 'me', '--', 'bash', str(ROOT / 'chat.sh'), timeout=300)
+ (ROOT / 'chat-passed').unlink()
+ (ROOT / 'online-chat.jsonl').write_bytes((ROOT / 'chat.jsonl').read_bytes())
+ run('nmcli', 'networking', 'off')
+ started = time.monotonic()
+ response = api('/api/chat', {'model': 'qwen3:0.6b', 'stream': False, 'think': False,
+ 'messages': [{'role': 'user', 'content': 'What is six times seven? Reply with only the number.'}],
+ 'options': {'temperature': 0, 'num_predict': 32}})
+ (ROOT / 'offline-api.json').write_text(json.dumps(response, indent=2) + '\n')
+ # Check the independently known answer, allowing the small model to show
+ # its equation. This assesses inference transport, not formatting skill.
+ assert response.get('done') and re.fullmatch(
+ r'(?:6\s*[×*x]\s*7\s*=\s*)?42[.!]?', response['message']['content'].strip()), response
+ result['offline_api_seconds'] = round(time.monotonic() - started, 3)
+ result['offline_reply'] = response['message']['content']
+ result['inference'] = {k: response[k] for k in ['eval_count', 'eval_duration', 'prompt_eval_count', 'prompt_eval_duration'] if k in response}
+ result['loaded_models'] = api('/api/ps')['models']
+ assert all(row.get('size_vram', 0) == 0 for row in result['loaded_models']), 'Expected CPU inference'
+ result['networking'] = run('nmcli', 'networking').stdout.strip()
+ assert result['networking'] == 'disabled'
+ result['listening'] = run('ss', '-ltnp').stdout
+ assert re.search(r'127\.0\.0\.1:11434\b', result['listening'])
+ assert not re.search(r'(?:0\.0\.0\.0|\*|\[::\]):11434\b', result['listening'])
+ result['scope'] = 'CPU inference and agent transport; this tiny model is not a coding-quality benchmark'
+ return result
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('probe', choices=['local-ai', 'nvidia'])
+ args = parser.parse_args()
+ assert not Path('/etc/harness-live').exists(), 'Installed disposable guest required'
+ assert Path('/var/lib/harness-os/install.json').is_file()
+ ROOT.mkdir(exist_ok=True)
+ result = {'status': 'running', 'probe': args.probe, 'started_at': time.time()}
+ try:
+ result.update(nvidia() if args.probe == 'nvidia' else local_ai())
+ result['status'] = 'passed'
+ except BaseException as error:
+ result.update(status='failed', error=repr(error))
+ raise
+ finally:
+ result['finished_at'] = time.time()
+ (ROOT / 'probe.json').write_text(json.dumps(result, indent=2) + '\n')
+ print('HN_LOCAL_AI_PROBE=' + json.dumps(result))
+
+
+if __name__ == '__main__':
+ main()
diff --git a/os/tests/local_ai_vm.py b/os/tests/local_ai_vm.py
new file mode 100644
index 000000000..c99881049
--- /dev/null
+++ b/os/tests/local_ai_vm.py
@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+"""Install the real ISO; assess optional CPU inference or NVIDIA packages."""
+import argparse
+import base64
+from functools import partial
+import hashlib
+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
+import json
+import os
+from pathlib import Path
+import shlex
+import shutil
+import subprocess
+import threading
+import time
+from vm import VM, check_graphical_keyboard
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--iso', type=Path, required=True)
+ parser.add_argument('--probe', choices=['local-ai', 'nvidia'], required=True)
+ parser.add_argument('--output', type=Path)
+ args = parser.parse_args()
+ if not os.access('/dev/kvm', os.R_OK | os.W_OK):
+ parser.error('Native x86 KVM is required')
+ iso = args.iso.resolve()
+ manifest = json.loads(iso.with_name('manifest.json').read_text())
+ with iso.open('rb') as handle:
+ assert hashlib.file_digest(handle, 'sha256').hexdigest() == manifest['iso']['sha256']
+ folder = (args.output or Path('os/test-results') / args.probe).resolve()
+ folder.mkdir(parents=True, exist_ok=False)
+ served = folder / 'served'
+ served.mkdir()
+ script = Path(__file__).with_name('local_ai_guest.py')
+ shutil.copyfile(script, served / script.name)
+ (served / 'browser.html').write_text('<!doctype html><html lang="en"><title>Harness check</title>'
+ '<style>body{font:32px monospace;margin:3rem}</style>'
+ '<h1>Optional driver browser check</h1><p>Harness preview is visible.</p></html>')
+ server = ThreadingHTTPServer(('127.0.0.1', 0), partial(SimpleHTTPRequestHandler, directory=str(served)))
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ vm = VM(folder, iso, 'uefi', 4096)
+ result = {'status': 'running', 'started_at': time.time(), 'image_source_commit': manifest['source_commit'],
+ 'test_source_commit': subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip(),
+ 'iso_sha256': manifest['iso']['sha256'], 'probe': args.probe, 'memory_mib': 4096,
+ 'physical_gpu_validation': 'unavailable: no GPU passthrough'}
+ config = dict(disk='/dev/vda', expected_serial='HN_OS_TEST', confirm_erase='/dev/vda',
+ username='me', hostname='harness', password='test-password-123', encrypt=True, serial_console=True)
+ try:
+ vm.start(live=True)
+ vm.wait(r'root@[^\r\n]*[#] ', timeout=180)
+ vm.shell_ready = True
+ vm.command('stty -echo')
+ encoded = base64.b64encode(json.dumps(config).encode()).decode()
+ vm.command('printf %s ' + encoded + ' | base64 -d > /tmp/install-config.json')
+ vm.command('nmcli networking off')
+ output, _ = vm.command('harness install --config /tmp/install-config.json --yes-erase-disk', timeout=360)
+ (folder / 'install.log').write_text(output)
+ vm.command('sync')
+ vm.stop()
+ vm.start(live=False)
+ vm.login_installed(config)
+ vm.command('printf %s ' + shlex.quote(config['password'] + '\n') + ' | sudo -S -v')
+ vm.command('nm-online -q --timeout=60 && curl --fail --silent --show-error ' +
+ shlex.quote(f'http://10.0.2.2:{server.server_port}/local_ai_guest.py') + ' -o /home/me/local_ai_guest.py')
+ output, status = vm.command('sudo -n python3 /home/me/local_ai_guest.py ' + args.probe, timeout=1100, check=False)
+ (folder / 'probe.log').write_text(output)
+ assert status == 0, 'Optional package/inference probe failed; see probe.log'
+ marker = 'HN_LOCAL_AI_PROBE='
+ # sudo may prepend a shell-integration OSC sequence on the same line.
+ result['guest'] = json.loads(next(line.split(marker, 1)[1] for line in output.splitlines() if marker in line))
+ assert result['guest']['status'] == 'passed'
+ if args.probe == 'local-ai':
+ vm.keys('ctrl', 'b')
+ vm.keys('shift', 't')
+ vm.type_probe('local-chat')
+ vm.keys('ret')
+ output, status = vm.command('for n in $(seq 1 120); do '
+ 'test -f /home/me/local-ai-check/chat-passed && exit 0; sleep 1; done; exit 1', timeout=135, check=False)
+ assert status == 0, 'Physical-keyboard local model conversation did not finish'
+ output, _ = vm.command('hn capture-pane -p')
+ assert 'Local model answered:' in output
+ vm.command('test -s /home/me/local-ai-check/chat-passed && test "$(nmcli networking)" = disabled')
+ (folder / 'local-model-pane.txt').write_text(output)
+ vm.screenshot('offline-local-model')
+ result['offline_keyboard_conversation'] = 'passed'
+ vm.keys('ctrl', 'd')
+ else:
+ vm.command('sync')
+ vm.stop()
+ vm.start(live=False)
+ vm.login_installed(config)
+ result['post_driver_reboot_keyboard'] = check_graphical_keyboard(vm, 'nvidia-package-reboot')
+ # One launch owns the test window. A second launch before Chromium
+ # finishes startup can leave the first New Tab in front of it.
+ vm.command('hn-browser ' + shlex.quote(f'http://10.0.2.2:{server.server_port}/browser.html'))
+ deadline = time.monotonic() + 45
+ while time.monotonic() < deadline:
+ vm.screenshot('browser-after-driver-packages')
+ visible = subprocess.check_output(['tesseract', str(folder / 'browser-after-driver-packages.png'),
+ 'stdout', '--psm', '11'], text=True, stderr=subprocess.DEVNULL, timeout=10)
+ (folder / 'browser-after-driver-packages.txt').write_text(visible)
+ if 'Harness preview is visible.' in visible:
+ break
+ time.sleep(1)
+ else:
+ raise AssertionError('Chromium did not render the expected page after driver installation')
+ vm.keys('meta_l', 'ret')
+ result['return_to_terminal_keyboard'] = check_graphical_keyboard(vm, 'after-browser')
+ result['generic_browser_after_driver_reboot'] = 'passed: rendered page and return to hn; virtual GPU only'
+ result['status'] = 'passed'
+ except BaseException as error:
+ result.update(status='failed', error=repr(error))
+ try:
+ vm.screenshot('failure')
+ except Exception:
+ pass
+ raise
+ finally:
+ if vm.shell_ready:
+ for name in ['probe.json', 'commands.log', 'chat.jsonl', 'chat.err', 'offline-api.json', 'supportedchips.html']:
+ try:
+ output, _ = vm.command('cat /home/me/local-ai-check/' + name, timeout=15, check=False)
+ (folder / ('guest-' + name)).write_text(output)
+ except Exception:
+ pass
+ result['finished_at'] = time.time()
+ (folder / 'receipt.json').write_text(json.dumps(result, indent=2) + '\n')
+ vm.stop()
+ server.shutdown()
+ server.server_close()
+
+
+if __name__ == '__main__':
+ main()