Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(lume): add pluggable registry support, auto-updater, and unattended improvements (#754)
Francesco Bonacci3fe1f1e6f1f5
diff --git a/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdx b/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdxindex 7d97f5fd6c..26839ddb71 100644--- a/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdx+++ b/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdx@@ -27,14 +27,26 @@ lume create my-vm --os macOS --ipsw latest --unattended tahoe# Using a custom config filelume create my-vm --os macOS --ipsw latest --unattended path/to/config.yml++# With debug mode enabled+lume create my-vm --os macOS --ipsw latest --unattended tahoe --debug```This will:1. Create the VM and install macOS from the IPSW-2. Boot the VM+2. Boot the VM (without opening VNC client by default)3. Automatically run through the Setup Assistant using your configuration+**Additional options for `lume create --unattended`:**++| Option | Description |+| ------ | ----------- |+| `--debug` | Save screenshots with OCR annotations for debugging |+| `--debug-dir` | Custom directory for debug screenshots |+| `--no-display` | Don't open VNC client (default: true for unattended) |+| `--vnc-port` | VNC server port (default: 0 for auto-assign) |+### Option 2: On an Existing VMUse the `lume setup` command on a freshly created VM that hasn't completed the Setup Assistant:@@ -248,16 +260,22 @@ health_check:## Debugging-Enable debug mode to save screenshots with OCR annotations:+Enable debug mode to save screenshots with OCR annotations. Debug flags work with both `lume setup` and `lume create --unattended`:```bash-# Save screenshots to default location (unique folder in system temp, e.g. /tmp/unattended-<uuid>)+# Using lume setuplume setup my-vm --unattended tahoe --debug-# Save screenshots to custom directory+# Using lume create with unattended setup+lume create my-vm --os macOS --ipsw latest --unattended tahoe --debug++# Save screenshots to custom directory (both commands support this)lume setup my-vm --unattended tahoe --debug --debug-dir /path/to/debug+lume create my-vm --os macOS --ipsw latest --unattended tahoe --debug --debug-dir /path/to/debug```+By default, debug screenshots are saved to a unique folder in the system temp directory (e.g., `/tmp/unattended-<uuid>`).+Debug screenshots show:- OCR-detected text with bounding boxesdiff --git a/docs/content/docs/lume/guide/getting-started/installation.mdx b/docs/content/docs/lume/guide/getting-started/installation.mdxindex 54e627a231..d1f5656127 100644--- a/docs/content/docs/lume/guide/getting-started/installation.mdx+++ b/docs/content/docs/lume/guide/getting-started/installation.mdx@@ -20,7 +20,9 @@ Install Lume with a single command:/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"```-By default, Lume is installed as a background service that starts automatically on login.+By default, Lume is installed with:+- A **background service** that starts automatically on login+- An **auto-updater** that checks for updates at login and every 24 hours### Manual Start (No Background Service)@@ -36,6 +38,20 @@ If you prefer to start the Lume API service manually when needed, use the `--no-Computer Use Agent).</Callout>+### Disable Auto-Updates++If you prefer to manage updates manually, use the `--no-auto-updater` option:++```bash+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)" -- --no-auto-updater+```++You can combine options to disable both the background service and auto-updates:++```bash+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)" -- --no-background-service --no-auto-updater+```+## Manual Download and InstallationYou can also download the `lume.pkg.tar.gz` archive from the [latest release](https://github.com/trycua/cua/releases?q=lume&expanded=true), extract it, and install the package manually.@@ -67,6 +83,31 @@ tail -f /tmp/lume_daemon.loglume serve```+## Managing Auto-Updates++If you installed with auto-updates enabled (default), Lume will automatically check for and install updates at login and every 24 hours.++```bash+# Check if the auto-updater is running+launchctl list | grep com.trycua.lume_updater++# View auto-updater logs+tail -f /tmp/lume_updater.log++# Manually run an update check+~/.local/bin/lume-update+```++### Disable Auto-Updates After Installation++To disable auto-updates after installation:++```bash+launchctl unload ~/Library/LaunchAgents/com.trycua.lume_updater.plist+rm ~/Library/LaunchAgents/com.trycua.lume_updater.plist+rm ~/.local/bin/lume-update+```+## Uninstall Lume### Remove the Background Service@@ -74,8 +115,18 @@ lume serveIf you installed Lume with the background service, first remove the daemon:```bash-launchctl unload "/Users/$(whoami)/Library/LaunchAgents/com.trycua.lume_daemon.plist"-rm "/Users/$(whoami)/Library/LaunchAgents/com.trycua.lume_daemon.plist"+launchctl unload ~/Library/LaunchAgents/com.trycua.lume_daemon.plist+rm ~/Library/LaunchAgents/com.trycua.lume_daemon.plist+```++### Remove the Auto-Updater++If you installed with auto-updates enabled, remove the updater:++```bash+launchctl unload ~/Library/LaunchAgents/com.trycua.lume_updater.plist+rm ~/Library/LaunchAgents/com.trycua.lume_updater.plist+rm ~/.local/bin/lume-update```### Remove Lume Binarydiff --git a/docs/content/docs/lume/reference/cli-reference.mdx b/docs/content/docs/lume/reference/cli-reference.mdxindex 0320d182f0..966d58d7b2 100644--- a/docs/content/docs/lume/reference/cli-reference.mdx+++ b/docs/content/docs/lume/reference/cli-reference.mdx@@ -43,6 +43,13 @@ Create a new virtual machine- `--ipsw` - Path to IPSW file or 'latest' for macOS VMs- `--storage` - VM storage location to use- `--unattended` - [Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: tahoe. Only supported for macOS VMs.+- `--debug-dir` - Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp)+- `--vnc-port` - Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign) (default: 0)++**Flags:**++- `--debug` - Enable debug mode for unattended setup - saves screenshots with click coordinates+- `--no-display` - Do not open the VNC client during unattended setup (default: true for unattended)### lume clonediff --git a/libs/lume/resources/unattended-tahoe.yml b/libs/lume/resources/unattended-tahoe.ymlindex 4e5fd80966..262b8f2dbb 100644--- a/libs/lume/resources/unattended-tahoe.yml+++ b/libs/lume/resources/unattended-tahoe.yml@@ -43,17 +43,10 @@ boot_commands:# Transfer Your Data to This Mac (Migration Assistant)- "<wait 'Transfer Your Data'>"- - "<delay 20>" # wait for refresh+ - "<delay 30>" # wait for refresh# Use tab navigation to select "Set up as new" (4th option)- # Tab order: From a Mac -> From Windows PC -> Set up with iPhone -> Set up as new- - "<tab>"- - "<delay 2>"- - "<tab>"+ - "<click 'Set up as new'>"- "<delay 2>"- - "<tab>"- - "<delay 2>"- - "<space>"- - "<delay 1>"- "<click 'Continue'>"- "<delay 2>"diff --git a/libs/lume/scripts/install.sh b/libs/lume/scripts/install.shindex a2db7a2fdc..31fc085f24 100755--- a/libs/lume/scripts/install.sh+++ b/libs/lume/scripts/install.sh@@ -33,6 +33,9 @@ LATEST_RELEASE_URL="https://api.github.com/repos/$GITHUB_REPO/releases/latest"# Option to skip background service setup (default: install it)INSTALL_BACKGROUND_SERVICE=true+# Option to skip auto-updater setup (default: install it)+INSTALL_AUTO_UPDATER=true+# Default port for lume serve (default: 7777)LUME_PORT=7777@@ -50,14 +53,18 @@ while [ "$#" -gt 0 ]; do--no-background-service)INSTALL_BACKGROUND_SERVICE=false;;+ --no-auto-updater)+ INSTALL_AUTO_UPDATER=false+ ;;--help)echo "${BOLD}${BLUE}Lume Installer${NORMAL}"echo "Usage: $0 [OPTIONS]"echo ""echo "Options:"echo " --install-dir DIR Install to the specified directory (default: $DEFAULT_INSTALL_DIR)"- echo " --port PORT Specify the port for lume serve (default: 7777)"+ echo " --port PORT Specify the port for lume serve (default: 7777)"echo " --no-background-service Do not setup the Lume background service (LaunchAgent)"+ echo " --no-auto-updater Do not setup automatic updates at login"echo " --help Display this help message"echo ""echo "Examples:"@@ -272,6 +279,214 @@ install_binary() {fi}+# Install the auto-updater script+install_updater_script() {+ UPDATER_SCRIPT="$INSTALL_DIR/lume-update"++ echo "Installing auto-updater script to $UPDATER_SCRIPT..."++ cat <<'UPDATER_EOF' > "$UPDATER_SCRIPT"+#!/bin/bash+# Lume Auto-Updater+# This script checks for updates and installs them if available++set -e++LOG_FILE="/tmp/lume_updater.log"+GITHUB_REPO="trycua/cua"++log() {+ echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" >> "$LOG_FILE"+}++log "Starting Lume update check..."++# Find lume binary location+LUME_BIN=$(command -v lume 2>/dev/null || echo "$HOME/.local/bin/lume")+INSTALL_DIR=$(dirname "$LUME_BIN")++if [ ! -x "$LUME_BIN" ]; then+ log "ERROR: lume binary not found at $LUME_BIN"+ exit 1+fi++# Get current version+CURRENT_VERSION=$("$LUME_BIN" --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || echo "0.0.0")+log "Current version: $CURRENT_VERSION"++# Get latest release tag from GitHub+get_latest_tag() {+ local page=1+ local per_page=100+ local max_pages=5++ while [ $page -le $max_pages ]; do+ local response=$(curl -s "https://api.github.com/repos/$GITHUB_REPO/tags?per_page=$per_page&page=$page")++ if [ -z "$response" ]; then+ return 1+ fi++ local tag=$(echo "$response" | grep '"name": "lume-' | head -n 1 | cut -d '"' -f 4)++ if [ -n "$tag" ]; then+ echo "$tag"+ return 0+ fi++ page=$((page + 1))+ done++ return 1+}++LATEST_TAG=$(get_latest_tag)+if [ -z "$LATEST_TAG" ]; then+ log "ERROR: Could not fetch latest release tag"+ exit 1+fi++# Extract version from tag (lume-vX.Y.Z -> X.Y.Z)+LATEST_VERSION=$(echo "$LATEST_TAG" | sed 's/lume-v//' | sed 's/lume-//')+log "Latest version: $LATEST_VERSION"++# Compare versions (simple string comparison works for semver)+version_gt() {+ [ "$(printf '%s\n' "$1" "$2" | sort -V | tail -n1)" = "$1" ] && [ "$1" != "$2" ]+}++if version_gt "$LATEST_VERSION" "$CURRENT_VERSION"; then+ log "New version available: $LATEST_VERSION (current: $CURRENT_VERSION)"+ log "Downloading update..."++ TEMP_DIR=$(mktemp -d)+ trap 'rm -rf "$TEMP_DIR"' EXIT++ DOWNLOAD_URL="https://github.com/$GITHUB_REPO/releases/download/$LATEST_TAG/lume.tar.gz"++ if curl -sL "$DOWNLOAD_URL" -o "$TEMP_DIR/lume.tar.gz"; then+ if tar -tzf "$TEMP_DIR/lume.tar.gz" > /dev/null 2>&1; then+ tar -xzf "$TEMP_DIR/lume.tar.gz" -C "$TEMP_DIR"++ # Stop the daemon before updating+ launchctl unload "$HOME/Library/LaunchAgents/com.trycua.lume_daemon.plist" 2>/dev/null || true++ # Install new binary+ mv "$TEMP_DIR/lume" "$INSTALL_DIR/"+ chmod +x "$INSTALL_DIR/lume"++ # Restart the daemon+ launchctl load "$HOME/Library/LaunchAgents/com.trycua.lume_daemon.plist" 2>/dev/null || true++ log "Successfully updated lume to version $LATEST_VERSION"+ else+ log "ERROR: Downloaded file is not a valid archive"+ exit 1+ fi+ else+ log "ERROR: Failed to download update"+ exit 1+ fi+else+ log "Already up to date (version $CURRENT_VERSION)"+fi++log "Update check complete"+UPDATER_EOF++ chmod +x "$UPDATER_SCRIPT"+ echo "Auto-updater script installed."+}++# Setup the auto-updater LaunchAgent+setup_auto_updater() {+ UPDATER_SERVICE_NAME="com.trycua.lume_updater"+ UPDATER_PLIST_PATH="$HOME/Library/LaunchAgents/$UPDATER_SERVICE_NAME.plist"+ UPDATER_SCRIPT="$INSTALL_DIR/lume-update"++ echo ""+ echo "Setting up auto-updater LaunchAgent..."++ # Create LaunchAgents directory if it doesn't exist+ mkdir -p "$HOME/Library/LaunchAgents"++ # Unload existing service if present+ if [ -f "$UPDATER_PLIST_PATH" ]; then+ echo "Existing updater LaunchAgent found. Unloading..."+ launchctl unload "$UPDATER_PLIST_PATH" 2>/dev/null || true+ fi++ # Create the plist file+ # RunAtLoad: run when loaded (at login)+ # StartInterval: also check every 24 hours (86400 seconds)+ cat <<EOF > "$UPDATER_PLIST_PATH"+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>Label</key>+ <string>$UPDATER_SERVICE_NAME</string>+ <key>ProgramArguments</key>+ <array>+ <string>$UPDATER_SCRIPT</string>+ </array>+ <key>RunAtLoad</key>+ <true/>+ <key>StartInterval</key>+ <integer>86400</integer>+ <key>WorkingDirectory</key>+ <string>$HOME</string>+ <key>EnvironmentVariables</key>+ <dict>+ <key>PATH</key>+ <string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$HOME/.local/bin</string>+ <key>HOME</key>+ <string>$HOME</string>+ </dict>+ <key>StandardOutPath</key>+ <string>/tmp/lume_updater.log</string>+ <key>StandardErrorPath</key>+ <string>/tmp/lume_updater.error.log</string>+</dict>+</plist>+EOF++ # Set permissions+ chmod 644 "$UPDATER_PLIST_PATH"+ touch /tmp/lume_updater.log /tmp/lume_updater.error.log+ chmod 644 /tmp/lume_updater.log /tmp/lume_updater.error.log++ # Load the LaunchAgent+ echo "Loading auto-updater LaunchAgent..."+ launchctl load "$UPDATER_PLIST_PATH"++ echo "${GREEN}Auto-updater installed! Lume will check for updates at login and every 24 hours.${NORMAL}"+ echo "To view update logs: tail -f /tmp/lume_updater.log"+ echo ""+ echo "To disable auto-updates, run:"+ echo " launchctl unload \"$UPDATER_PLIST_PATH\""+ echo " rm \"$UPDATER_PLIST_PATH\""+}++# Remove auto-updater if it exists+remove_auto_updater() {+ UPDATER_SERVICE_NAME="com.trycua.lume_updater"+ UPDATER_PLIST_PATH="$HOME/Library/LaunchAgents/$UPDATER_SERVICE_NAME.plist"+ UPDATER_SCRIPT="$INSTALL_DIR/lume-update"++ if [ -f "$UPDATER_PLIST_PATH" ]; then+ echo "Removing existing auto-updater LaunchAgent..."+ launchctl unload "$UPDATER_PLIST_PATH" 2>/dev/null || true+ rm "$UPDATER_PLIST_PATH"+ fi++ if [ -f "$UPDATER_SCRIPT" ]; then+ rm "$UPDATER_SCRIPT"+ fi++ echo "Auto-updater removed."+}+# Main installation flowmain() {check_permissions@@ -371,6 +586,14 @@ EOFecho "Skipping Lume background service (LaunchAgent) setup as requested (use --no-background-service)."fifi++ # Setup auto-updater+ if [ "$INSTALL_AUTO_UPDATER" = true ]; then+ install_updater_script+ setup_auto_updater+ else+ remove_auto_updater+ fi}# Run the installationdiff --git a/libs/lume/src/Commands/Create.swift b/libs/lume/src/Commands/Create.swiftindex 9abaad065f..509f159837 100644--- a/libs/lume/src/Commands/Create.swift+++ b/libs/lume/src/Commands/Create.swift@@ -50,6 +50,27 @@ struct Create: AsyncParsableCommand {)var unattended: String?+ @Flag(+ name: .customLong("debug"),+ help: "Enable debug mode for unattended setup - saves screenshots with click coordinates")+ var debug: Bool = false++ @Option(+ name: .customLong("debug-dir"),+ help: "Custom directory for debug screenshots (defaults to unique folder in system temp)",+ completion: .directory)+ var debugDir: String?++ @Flag(+ name: .customLong("no-display"),+ help: "Do not open the VNC client during unattended setup (default: true for unattended)")+ var noDisplay: Bool = false++ @Option(+ name: .customLong("vnc-port"),+ help: "Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign)")+ var vncPort: Int = 0+init() {}@@ -82,7 +103,11 @@ struct Create: AsyncParsableCommand {display: display.string,ipsw: ipsw,storage: storage,- unattendedConfig: unattendedConfig+ unattendedConfig: unattendedConfig,+ debug: debug,+ debugDir: debugDir,+ noDisplay: unattendedConfig != nil ? true : noDisplay, // Default to no-display for unattended+ vncPort: vncPort)}}diff --git a/libs/lume/src/ContainerRegistry/GCSImageRegistry.swift b/libs/lume/src/ContainerRegistry/GCSImageRegistry.swiftnew file mode 100644index 0000000000..95bd97656d--- /dev/null+++ b/libs/lume/src/ContainerRegistry/GCSImageRegistry.swift@@ -0,0 +1,618 @@+import CommonCrypto+import Foundation++/// GCS-backed image registry that uses Richmond API for signed URLs+class GCSImageRegistry: ImageRegistry, @unchecked Sendable {+ private let config: GCSConfig+ private let urlSession: URLSession++ /// Chunk size for multipart uploads (100MB as per Richmond API)+ private static let chunkSize: Int = 100 * 1024 * 1024++ init(config: GCSConfig) throws {+ guard !config.apiUrl.isEmpty else {+ throw RegistryConfigError.invalidApiUrl(config.apiUrl)+ }+ guard !config.apiKey.isEmpty else {+ throw RegistryConfigError.missingApiKey+ }+ self.config = config++ // Configure URLSession with reasonable timeouts+ let sessionConfig = URLSessionConfiguration.default+ sessionConfig.timeoutIntervalForRequest = 300 // 5 minutes+ sessionConfig.timeoutIntervalForResource = 3600 // 1 hour for large downloads+ self.urlSession = URLSession(configuration: sessionConfig)+ }++ // MARK: - ImageRegistry Protocol++ func pull(+ image: String,+ name: String?,+ locationName: String?+ ) async throws -> VMDirectory {+ // Parse image name and tag+ let components = image.split(separator: ":")+ guard components.count == 2 else {+ throw GCSRegistryError.invalidImageFormat(image)+ }++ let imageName = String(components[0])+ let imageTag = String(components[1])+ let vmName = name ?? imageName++ Logger.info(+ "Pulling image from GCS",+ metadata: [+ "image": image,+ "name": vmName,+ "api_url": config.apiUrl+ ])++ // Get the VM directory+ let home = Home()+ let vmDir: VMDirectory+ if let locationName = locationName,+ locationName.contains("/") || locationName.contains("\\") {+ vmDir = try home.getVMDirectoryFromPath(vmName, storagePath: locationName)+ } else {+ vmDir = try home.getVMDirectory(vmName, storage: locationName)+ }++ // Get signed download URL from Richmond API+ Logger.info("Getting download URL from API")+ let downloadInfo = try await getDownloadURL(imageName: imageName, tag: imageTag)++ Logger.info(+ "Got download URL",+ metadata: [+ "size_bytes": "\(downloadInfo.sizeBytes)",+ "expires_in": "\(downloadInfo.expiresIn)s"+ ])++ // Create temporary directory for download+ let tempDir = FileManager.default.temporaryDirectory.appendingPathComponent("lume_gcs_\(UUID().uuidString)")+ try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true)+ defer {+ try? FileManager.default.removeItem(at: tempDir)+ }++ let archivePath = tempDir.appendingPathComponent("image.tar.gz")++ // Download the image+ Logger.info("Downloading image from GCS")+ try await downloadFile(from: downloadInfo.url, to: archivePath, expectedSize: downloadInfo.sizeBytes)++ // Verify checksum if provided+ if let expectedChecksum = downloadInfo.checksum {+ Logger.info("Verifying checksum")+ let actualChecksum = try calculateSHA256(fileAt: archivePath)+ guard actualChecksum == expectedChecksum else {+ throw GCSRegistryError.checksumMismatch(expected: expectedChecksum, actual: actualChecksum)+ }+ Logger.info("Checksum verified")+ }++ // Ensure VM directory exists+ try FileManager.default.createDirectory(at: vmDir.dir.url, withIntermediateDirectories: true)++ // Extract the archive to VM directory+ Logger.info("Extracting image to VM directory")+ try await extractArchive(archivePath, to: vmDir.dir.url)++ Logger.info("Pull complete", metadata: ["vm_path": vmDir.dir.path])++ return vmDir+ }++ func push(+ vmDirPath: String,+ imageName: String,+ tags: [String],+ chunkSizeMb: Int,+ verbose: Bool,+ dryRun: Bool,+ reassemble: Bool+ ) async throws {+ guard !tags.isEmpty else {+ throw GCSRegistryError.noTagsProvided+ }++ let vmDir = URL(fileURLWithPath: vmDirPath)+ guard FileManager.default.fileExists(atPath: vmDirPath) else {+ throw GCSRegistryError.vmDirectoryNotFound(vmDirPath)+ }++ Logger.info(+ "Pushing VM to GCS",+ metadata: [+ "vm_path": vmDirPath,+ "image_name": imageName,+ "tags": tags.joined(separator: ", "),+ "api_url": config.apiUrl+ ])++ // Create temporary directory for the archive+ let tempDir = FileManager.default.temporaryDirectory.appendingPathComponent("lume_gcs_push_\(UUID().uuidString)")+ try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true)+ defer {+ try? FileManager.default.removeItem(at: tempDir)+ }++ let archivePath = tempDir.appendingPathComponent("image.tar.gz")++ // Create tar.gz archive of the VM directory+ Logger.info("Creating archive")+ try await createArchive(from: vmDir, to: archivePath)++ // Calculate size and checksum+ let fileAttributes = try FileManager.default.attributesOfItem(atPath: archivePath.path)+ let fileSize = fileAttributes[.size] as! Int64+ let checksum = try calculateSHA256(fileAt: archivePath)++ Logger.info(+ "Archive created",+ metadata: [+ "size_bytes": "\(fileSize)",+ "checksum": checksum+ ])++ if dryRun {+ Logger.info("Dry run mode - skipping upload")+ return+ }++ // Push to each tag+ for tag in tags {+ Logger.info("Uploading image", metadata: ["tag": tag])++ // Initiate upload+ let uploadSession = try await initiateUpload(+ imageName: imageName,+ tag: tag,+ sizeBytes: fileSize,+ checksum: checksum+ )++ Logger.info("Upload initiated", metadata: ["upload_id": uploadSession.uploadId])++ do {+ // Upload in chunks+ let parts = try await uploadInChunks(+ imageName: imageName,+ uploadId: uploadSession.uploadId,+ filePath: archivePath,+ fileSize: fileSize+ )++ // Complete upload+ try await completeUpload(+ imageName: imageName,+ uploadId: uploadSession.uploadId,+ parts: parts+ )++ Logger.info("Upload complete", metadata: ["tag": tag])+ } catch {+ // Abort upload on failure+ Logger.error("Upload failed, aborting", metadata: ["error": "\(error)"])+ try? await abortUpload(imageName: imageName, uploadId: uploadSession.uploadId)+ throw error+ }+ }++ Logger.info("Push complete")+ }++ func getImages() async throws -> [CachedImage] {+ // Call Richmond API to list images+ let url = URL(string: "\(config.apiUrl)/v1/images")!+ var request = URLRequest(url: url)+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")++ let (data, response) = try await urlSession.data(for: request)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ throw GCSRegistryError.apiError(statusCode: httpResponse.statusCode, message: String(data: data, encoding: .utf8) ?? "Unknown error")+ }++ struct ListResponse: Codable {+ let images: [ImageInfo]+ }++ struct ImageInfo: Codable {+ let name: String+ let versions: [VersionInfo]?+ }++ struct VersionInfo: Codable {+ let tag: String+ let sizeBytes: Int64?+ let createdAt: String?++ enum CodingKeys: String, CodingKey {+ case tag+ case sizeBytes = "size_bytes"+ case createdAt = "created_at"+ }+ }++ let listResponse = try JSONDecoder().decode(ListResponse.self, from: data)++ // Convert to CachedImage format+ var images: [CachedImage] = []+ for imageInfo in listResponse.images {+ for version in imageInfo.versions ?? [] {+ images.append(CachedImage(+ repository: imageInfo.name,+ imageId: "\(imageInfo.name):\(version.tag)",+ manifestId: "\(imageInfo.name):\(version.tag)"+ ))+ }+ }++ return images+ }++ // MARK: - Private API Methods++ private struct DownloadInfo {+ let url: URL+ let sizeBytes: Int64+ let checksum: String?+ let expiresIn: Int+ }++ private func getDownloadURL(imageName: String, tag: String) async throws -> DownloadInfo {+ var urlComponents = URLComponents(string: "\(config.apiUrl)/v1/images/\(imageName)/download")!+ urlComponents.queryItems = [URLQueryItem(name: "tag", value: tag)]++ var request = URLRequest(url: urlComponents.url!)+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")++ let (data, response) = try await urlSession.data(for: request)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ if httpResponse.statusCode == 404 {+ throw GCSRegistryError.imageNotFound(imageName, tag)+ }+ throw GCSRegistryError.apiError(statusCode: httpResponse.statusCode, message: String(data: data, encoding: .utf8) ?? "Unknown error")+ }++ struct Response: Codable {+ let downloadUrl: String+ let expiresIn: Int+ let sizeBytes: Int64+ let checksumSha256: String?++ enum CodingKeys: String, CodingKey {+ case downloadUrl = "download_url"+ case expiresIn = "expires_in"+ case sizeBytes = "size_bytes"+ case checksumSha256 = "checksum_sha256"+ }+ }++ let downloadResponse = try JSONDecoder().decode(Response.self, from: data)++ guard let downloadURL = URL(string: downloadResponse.downloadUrl) else {+ throw GCSRegistryError.invalidDownloadURL(downloadResponse.downloadUrl)+ }++ return DownloadInfo(+ url: downloadURL,+ sizeBytes: downloadResponse.sizeBytes,+ checksum: downloadResponse.checksumSha256,+ expiresIn: downloadResponse.expiresIn+ )+ }++ private struct UploadSession {+ let uploadId: String+ let partSize: Int+ }++ private func initiateUpload(imageName: String, tag: String, sizeBytes: Int64, checksum: String) async throws -> UploadSession {+ let url = URL(string: "\(config.apiUrl)/v1/images/\(imageName)/upload")!+ var request = URLRequest(url: url)+ request.httpMethod = "POST"+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")+ request.setValue("application/json", forHTTPHeaderField: "Content-Type")++ let body: [String: Any] = [+ "tag": tag,+ "image_type": "lume",+ "size_bytes": sizeBytes,+ "checksum_sha256": checksum+ ]+ request.httpBody = try JSONSerialization.data(withJSONObject: body)++ let (data, response) = try await urlSession.data(for: request)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ throw GCSRegistryError.apiError(statusCode: httpResponse.statusCode, message: String(data: data, encoding: .utf8) ?? "Unknown error")+ }++ struct Response: Codable {+ let uploadId: String+ let partSize: Int?++ enum CodingKeys: String, CodingKey {+ case uploadId = "upload_id"+ case partSize = "part_size"+ }+ }++ let uploadResponse = try JSONDecoder().decode(Response.self, from: data)++ return UploadSession(+ uploadId: uploadResponse.uploadId,+ partSize: uploadResponse.partSize ?? GCSImageRegistry.chunkSize+ )+ }++ private struct PartInfo: Codable {+ let partNumber: Int+ let etag: String++ enum CodingKeys: String, CodingKey {+ case partNumber = "part_number"+ case etag+ }+ }++ private func uploadInChunks(imageName: String, uploadId: String, filePath: URL, fileSize: Int64) async throws -> [PartInfo] {+ let fileHandle = try FileHandle(forReadingFrom: filePath)+ defer { try? fileHandle.close() }++ var parts: [PartInfo] = []+ var partNumber = 1+ var bytesUploaded: Int64 = 0++ while bytesUploaded < fileSize {+ // Get signed URL for this part+ let partURL = try await getPartUploadURL(imageName: imageName, uploadId: uploadId, partNumber: partNumber)++ // Read chunk+ let chunkSize = min(GCSImageRegistry.chunkSize, Int(fileSize - bytesUploaded))+ guard let chunkData = try fileHandle.read(upToCount: chunkSize) else {+ break+ }++ // Upload chunk+ let etag = try await uploadPart(url: partURL, data: chunkData)++ parts.append(PartInfo(partNumber: partNumber, etag: etag))+ bytesUploaded += Int64(chunkData.count)+ partNumber += 1++ let progress = Double(bytesUploaded) / Double(fileSize) * 100+ Logger.info("Upload progress: \(String(format: "%.1f", progress))%")+ }++ return parts+ }++ private func getPartUploadURL(imageName: String, uploadId: String, partNumber: Int) async throws -> URL {+ let url = URL(string: "\(config.apiUrl)/v1/images/\(imageName)/upload/\(uploadId)/part/\(partNumber)")!+ var request = URLRequest(url: url)+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")++ let (data, response) = try await urlSession.data(for: request)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ throw GCSRegistryError.apiError(statusCode: httpResponse.statusCode, message: String(data: data, encoding: .utf8) ?? "Unknown error")+ }++ struct Response: Codable {+ let uploadUrl: String++ enum CodingKeys: String, CodingKey {+ case uploadUrl = "upload_url"+ }+ }++ let partResponse = try JSONDecoder().decode(Response.self, from: data)++ guard let partURL = URL(string: partResponse.uploadUrl) else {+ throw GCSRegistryError.invalidDownloadURL(partResponse.uploadUrl)+ }++ return partURL+ }++ private func uploadPart(url: URL, data: Data) async throws -> String {+ var request = URLRequest(url: url)+ request.httpMethod = "PUT"+ request.setValue("application/octet-stream", forHTTPHeaderField: "Content-Type")+ request.setValue("\(data.count)", forHTTPHeaderField: "Content-Length")++ let (_, response) = try await urlSession.upload(for: request, from: data)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 || httpResponse.statusCode == 201 else {+ throw GCSRegistryError.uploadFailed(partURL: url.absoluteString, statusCode: httpResponse.statusCode)+ }++ // GCS returns ETag in response header+ return httpResponse.value(forHTTPHeaderField: "ETag") ?? ""+ }++ private func completeUpload(imageName: String, uploadId: String, parts: [PartInfo]) async throws {+ let url = URL(string: "\(config.apiUrl)/v1/images/\(imageName)/upload/\(uploadId)/complete")!+ var request = URLRequest(url: url)+ request.httpMethod = "POST"+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")+ request.setValue("application/json", forHTTPHeaderField: "Content-Type")++ let body = ["parts": parts.map { ["part_number": $0.partNumber, "etag": $0.etag] }]+ request.httpBody = try JSONSerialization.data(withJSONObject: body)++ let (data, response) = try await urlSession.data(for: request)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ throw GCSRegistryError.apiError(statusCode: httpResponse.statusCode, message: String(data: data, encoding: .utf8) ?? "Unknown error")+ }+ }++ private func abortUpload(imageName: String, uploadId: String) async throws {+ let url = URL(string: "\(config.apiUrl)/v1/images/\(imageName)/upload/\(uploadId)")!+ var request = URLRequest(url: url)+ request.httpMethod = "DELETE"+ request.setValue(config.apiKey, forHTTPHeaderField: "X-API-Key")++ let (_, _) = try await urlSession.data(for: request)+ }++ // MARK: - Private Helper Methods++ private func downloadFile(from url: URL, to destination: URL, expectedSize: Int64) async throws {+ let (tempURL, response) = try await urlSession.download(from: url)++ guard let httpResponse = response as? HTTPURLResponse else {+ throw GCSRegistryError.invalidResponse+ }++ guard httpResponse.statusCode == 200 else {+ throw GCSRegistryError.downloadFailed(url: url.absoluteString, statusCode: httpResponse.statusCode)+ }++ // Move to destination+ if FileManager.default.fileExists(atPath: destination.path) {+ try FileManager.default.removeItem(at: destination)+ }+ try FileManager.default.moveItem(at: tempURL, to: destination)++ // Verify size+ let attributes = try FileManager.default.attributesOfItem(atPath: destination.path)+ let actualSize = attributes[.size] as! Int64+ if actualSize != expectedSize {+ Logger.info("Downloaded file size mismatch", metadata: [+ "expected": "\(expectedSize)",+ "actual": "\(actualSize)"+ ])+ }+ }++ private func calculateSHA256(fileAt path: URL) throws -> String {+ let fileHandle = try FileHandle(forReadingFrom: path)+ defer { try? fileHandle.close() }++ var context = CC_SHA256_CTX()+ CC_SHA256_Init(&context)++ let bufferSize = 1024 * 1024 // 1MB chunks+ while autoreleasepool(invoking: {+ guard let data = try? fileHandle.read(upToCount: bufferSize), !data.isEmpty else {+ return false+ }+ data.withUnsafeBytes { bytes in+ _ = CC_SHA256_Update(&context, bytes.baseAddress, CC_LONG(data.count))+ }+ return true+ }) {}++ var digest = [UInt8](repeating: 0, count: Int(CC_SHA256_DIGEST_LENGTH))+ CC_SHA256_Final(&digest, &context)++ return digest.map { String(format: "%02x", $0) }.joined()+ }++ private func extractArchive(_ archive: URL, to destination: URL) async throws {+ let process = Process()+ process.executableURL = URL(fileURLWithPath: "/usr/bin/tar")+ process.arguments = ["-xzf", archive.path, "-C", destination.path]++ try process.run()+ process.waitUntilExit()++ guard process.terminationStatus == 0 else {+ throw GCSRegistryError.extractionFailed(archive.path)+ }+ }++ private func createArchive(from source: URL, to destination: URL) async throws {+ let process = Process()+ process.executableURL = URL(fileURLWithPath: "/usr/bin/tar")+ process.currentDirectoryURL = source.deletingLastPathComponent()+ process.arguments = ["-czf", destination.path, source.lastPathComponent]++ try process.run()+ process.waitUntilExit()++ guard process.terminationStatus == 0 else {+ throw GCSRegistryError.archiveCreationFailed(source.path)+ }+ }+}++// MARK: - Errors++public enum GCSRegistryError: Error, LocalizedError {+ case invalidImageFormat(String)+ case imageNotFound(String, String)+ case invalidResponse+ case invalidDownloadURL(String)+ case apiError(statusCode: Int, message: String)+ case downloadFailed(url: String, statusCode: Int)+ case uploadFailed(partURL: String, statusCode: Int)+ case checksumMismatch(expected: String, actual: String)+ case extractionFailed(String)+ case archiveCreationFailed(String)+ case vmDirectoryNotFound(String)+ case noTagsProvided++ public var errorDescription: String? {+ switch self {+ case .invalidImageFormat(let image):+ return "Invalid image format '\(image)'. Expected format: name:tag"+ case .imageNotFound(let name, let tag):+ return "Image '\(name):\(tag)' not found in GCS registry"+ case .invalidResponse:+ return "Invalid response from API"+ case .invalidDownloadURL(let url):+ return "Invalid download URL: \(url)"+ case .apiError(let statusCode, let message):+ return "API error (HTTP \(statusCode)): \(message)"+ case .downloadFailed(let url, let statusCode):+ return "Download failed (HTTP \(statusCode)) from: \(url)"+ case .uploadFailed(let partURL, let statusCode):+ return "Upload failed (HTTP \(statusCode)) to: \(partURL)"+ case .checksumMismatch(let expected, let actual):+ return "Checksum mismatch. Expected: \(expected), Got: \(actual)"+ case .extractionFailed(let path):+ return "Failed to extract archive: \(path)"+ case .archiveCreationFailed(let path):+ return "Failed to create archive from: \(path)"+ case .vmDirectoryNotFound(let path):+ return "VM directory not found: \(path)"+ case .noTagsProvided:+ return "At least one tag must be provided"+ }+ }+}+diff --git a/libs/lume/src/ContainerRegistry/ImageContainerRegistry.swift b/libs/lume/src/ContainerRegistry/ImageContainerRegistry.swiftindex 8646370d35..647b6d4010 100644--- a/libs/lume/src/ContainerRegistry/ImageContainerRegistry.swift+++ b/libs/lume/src/ContainerRegistry/ImageContainerRegistry.swift@@ -413,7 +413,7 @@ actor TaskCounter {func current() -> Int { count }}-class ImageContainerRegistry: @unchecked Sendable {+class ImageContainerRegistry: ImageRegistry, @unchecked Sendable {private let registry: Stringprivate let organization: Stringprivate let downloadProgress = ProgressTracker() // Renamed for claritydiff --git a/libs/lume/src/ContainerRegistry/ImageRegistry.swift b/libs/lume/src/ContainerRegistry/ImageRegistry.swiftnew file mode 100644index 0000000000..65afcd27c3--- /dev/null+++ b/libs/lume/src/ContainerRegistry/ImageRegistry.swift@@ -0,0 +1,65 @@+import Foundation++/// Protocol defining the interface for image registries (ghcr, gcs, etc.)+protocol ImageRegistry: Sendable {+ /// Pull an image from the registry+ /// - Parameters:+ /// - image: Image reference (format: name:tag)+ /// - name: Optional VM name (defaults to image name without tag)+ /// - locationName: Optional storage location name or direct path+ /// - Returns: The VM directory where the image was pulled+ func pull(+ image: String,+ name: String?,+ locationName: String?+ ) async throws -> VMDirectory++ /// Push a VM to the registry+ /// - Parameters:+ /// - vmDirPath: Path to the VM directory to push+ /// - imageName: Name for the image in the registry+ /// - tags: Tags to apply to the image+ /// - chunkSizeMb: Chunk size for large files in MB+ /// - verbose: Enable verbose logging+ /// - dryRun: Prepare files without uploading+ /// - reassemble: In dry-run mode, verify integrity by reassembling+ func push(+ vmDirPath: String,+ imageName: String,+ tags: [String],+ chunkSizeMb: Int,+ verbose: Bool,+ dryRun: Bool,+ reassemble: Bool+ ) async throws++ /// Get list of cached/available images+ func getImages() async throws -> [CachedImage]+}++// Default implementations for optional parameters+extension ImageRegistry {+ func pull(image: String, name: String? = nil, locationName: String? = nil) async throws -> VMDirectory {+ try await pull(image: image, name: name, locationName: locationName)+ }++ func push(+ vmDirPath: String,+ imageName: String,+ tags: [String],+ chunkSizeMb: Int = 512,+ verbose: Bool = false,+ dryRun: Bool = false,+ reassemble: Bool = false+ ) async throws {+ try await push(+ vmDirPath: vmDirPath,+ imageName: imageName,+ tags: tags,+ chunkSizeMb: chunkSizeMb,+ verbose: verbose,+ dryRun: dryRun,+ reassemble: reassemble+ )+ }+}diff --git a/libs/lume/src/ContainerRegistry/RegistryConfig.swift b/libs/lume/src/ContainerRegistry/RegistryConfig.swiftnew file mode 100644index 0000000000..fec8d2533d--- /dev/null+++ b/libs/lume/src/ContainerRegistry/RegistryConfig.swift@@ -0,0 +1,81 @@+import Foundation++/// Type of container registry to use for pulling/pushing images+public enum RegistryType: String, Codable, CaseIterable, Sendable {+ case ghcr = "ghcr"+ case gcs = "gcs"++ public static let defaultType: RegistryType = .ghcr+}++/// Configuration for GitHub Container Registry (ghcr.io)+public struct GHCRConfig: Codable, Equatable, Sendable {+ /// The registry URL (default: ghcr.io)+ public var registry: String+ /// The organization/user namespace (default: trycua)+ public var organization: String++ public static let defaultConfig = GHCRConfig(+ registry: "ghcr.io",+ organization: "trycua"+ )++ public init(registry: String = "ghcr.io", organization: String = "trycua") {+ self.registry = registry+ self.organization = organization+ }+}++/// Configuration for GCS-backed registry (via Richmond API)+public struct GCSConfig: Codable, Equatable, Sendable {+ /// The API URL for getting signed URLs (e.g., https://api.cua.ai)+ public var apiUrl: String+ /// API key for authentication+ public var apiKey: String++ public init(apiUrl: String, apiKey: String) {+ self.apiUrl = apiUrl+ self.apiKey = apiKey+ }+}++/// Overall registry configuration+public struct RegistryConfig: Codable, Equatable, Sendable {+ /// The type of registry to use+ public var type: RegistryType+ /// GHCR configuration (used when type == .ghcr)+ public var ghcr: GHCRConfig+ /// GCS configuration (used when type == .gcs)+ public var gcs: GCSConfig?++ public static let defaultConfig = RegistryConfig(+ type: .ghcr,+ ghcr: .defaultConfig,+ gcs: nil+ )++ public init(type: RegistryType = .ghcr, ghcr: GHCRConfig = .defaultConfig, gcs: GCSConfig? = nil) {+ self.type = type+ self.ghcr = ghcr+ self.gcs = gcs+ }+}++// MARK: - Errors++public enum RegistryConfigError: Error, LocalizedError {+ case gcsConfigMissing+ case invalidApiUrl(String)+ case missingApiKey++ public var errorDescription: String? {+ switch self {+ case .gcsConfigMissing:+ return "GCS registry type selected but GCS configuration is missing. Run 'lume config set registry.gcs.api_url <url>' and 'lume config set registry.gcs.api_key <key>'"+ case .invalidApiUrl(let url):+ return "Invalid API URL: \(url)"+ case .missingApiKey:+ return "API key is required for GCS registry. Run 'lume config set registry.gcs.api_key <key>'"+ }+ }+}diff --git a/libs/lume/src/ContainerRegistry/RegistryFactory.swift b/libs/lume/src/ContainerRegistry/RegistryFactory.swiftnew file mode 100644index 0000000000..0aa874f436--- /dev/null+++ b/libs/lume/src/ContainerRegistry/RegistryFactory.swift@@ -0,0 +1,70 @@+import Foundation++/// Factory for creating image registries based on configuration+struct RegistryFactory {+ private init() {}++ // Default values for CLI options (must match Pull.swift and Push.swift defaults)+ private static let defaultGHCRRegistry = "ghcr.io"+ private static let defaultGHCROrganization = "trycua"++ /// Create an image registry based on the current settings+ /// - Parameters:+ /// - registry: CLI override for the ghcr registry URL (from --registry flag)+ /// - organization: CLI override for the ghcr organization (from --organization flag)+ /// - Returns: An ImageRegistry instance appropriate for the configured registry type+ ///+ /// Logic:+ /// - If CLI provides non-default values for registry/organization, always use ghcr with those values+ /// - Otherwise, use the registry type from settings (ghcr or gcs)+ static func createRegistry(+ registry: String? = nil,+ organization: String? = nil+ ) throws -> any ImageRegistry {+ let settings = SettingsManager.shared.getSettings()+ let config = settings.registry ?? .defaultConfig++ // Check if user explicitly provided CLI overrides that differ from defaults+ let hasCliOverride = (registry != nil && registry != defaultGHCRRegistry) ||+ (organization != nil && organization != defaultGHCROrganization)++ // If CLI override provided, force ghcr with those values+ if hasCliOverride {+ let registryURL = registry ?? defaultGHCRRegistry+ let org = organization ?? defaultGHCROrganization+ return ImageContainerRegistry(registry: registryURL, organization: org)+ }++ // Otherwise use configured registry type+ switch config.type {+ case .ghcr:+ return ImageContainerRegistry(+ registry: config.ghcr.registry,+ organization: config.ghcr.organization+ )++ case .gcs:+ guard let gcsConfig = config.gcs else {+ throw RegistryConfigError.gcsConfigMissing+ }+ return try GCSImageRegistry(config: gcsConfig)+ }+ }++ /// Create a registry with explicit configuration (useful for testing or direct API calls)+ static func createRegistry(config: RegistryConfig) throws -> any ImageRegistry {+ switch config.type {+ case .ghcr:+ return ImageContainerRegistry(+ registry: config.ghcr.registry,+ organization: config.ghcr.organization+ )++ case .gcs:+ guard let gcsConfig = config.gcs else {+ throw RegistryConfigError.gcsConfigMissing+ }+ return try GCSImageRegistry(config: gcsConfig)+ }+ }+}diff --git a/libs/lume/src/FileSystem/Home.swift b/libs/lume/src/FileSystem/Home.swiftindex 5a91b49030..aa32ef0224 100644--- a/libs/lume/src/FileSystem/Home.swift+++ b/libs/lume/src/FileSystem/Home.swift@@ -146,55 +146,12 @@ final class Home {// Loop through all locationslet settings = settingsManager.getSettings()-- // Also check ephemeral directory (macOS temporary directory)- let tmpDir = ProcessInfo.processInfo.environment["TMPDIR"] ?? "/tmp"- let cleanPath = tmpDir.hasSuffix("/") ? String(tmpDir.dropLast()) : tmpDir-- // If tmp directory exists, check for VMs there- if fileExists(at: cleanPath) {- let tmpDirPath = Path(cleanPath)- do {- let directoryURL = URL(fileURLWithPath: cleanPath)- let contents = try FileManager.default.contentsOfDirectory(- at: directoryURL,- includingPropertiesForKeys: [.isDirectoryKey],- options: .skipsHiddenFiles- )-- for subdir in contents {- do {- guard let isDirectory = try subdir.resourceValues(forKeys: [.isDirectoryKey]).isDirectory,- isDirectory else {- continue- }-- let vmName = subdir.lastPathComponent- let vmDir = VMDirectory(tmpDirPath.directory(vmName))-- // Only include if it's a valid VM directory- if vmDir.initialized() {- results.append(VMDirectoryWithLocation(- directory: vmDir,- locationName: "ephemeral"- ))- }- } catch {- // Skip any directories we can't access- continue- }- }- } catch {- Logger.error(- "Failed to access ephemeral directory",- metadata: [- "path": cleanPath,- "error": error.localizedDescription,- ]- )- // Continue to regular locations rather than failing completely- }- }++ // Note: We intentionally do NOT scan TMPDIR for ephemeral VMs.+ // Ephemeral VMs are short-lived and should be explicitly managed by the code+ // that creates them, not automatically discovered. Scanning TMPDIR would pick up+ // test leftovers and other temporary directories that happen to look like VMs.+for location in settings.vmLocations {let locationPath = Path(location.expandedPath)diff --git a/libs/lume/src/FileSystem/Settings.swift b/libs/lume/src/FileSystem/Settings.swiftindex 2025774bc8..1db10ad263 100644--- a/libs/lume/src/FileSystem/Settings.swift+++ b/libs/lume/src/FileSystem/Settings.swift@@ -6,6 +6,7 @@ struct LumeSettings: Codable, Sendable {var defaultLocationName: Stringvar cacheDirectory: Stringvar cachingEnabled: Bool+ var registry: RegistryConfig?var defaultLocation: VMLocation? {vmLocations.first { $0.name == defaultLocationName }@@ -22,7 +23,8 @@ struct LumeSettings: Codable, Sendable {],defaultLocationName: "default",cacheDirectory: "~/.lume/cache",- cachingEnabled: true+ cachingEnabled: true,+ registry: .defaultConfig)/// Gets all locations sorted by name@@ -81,7 +83,8 @@ final class SettingsManager: @unchecked Sendable {],defaultLocationName: "default",cacheDirectory: "~/.lume/cache",- cachingEnabled: true+ cachingEnabled: true,+ registry: .defaultConfig)// Try to save default settings@@ -112,6 +115,20 @@ final class SettingsManager: @unchecked Sendable {yamlContent += " path: \"\(location.path)\"\n"}+ // Registry configuration+ if let registry = settings.registry {+ yamlContent += "\n# Registry Configuration\nregistry:\n"+ yamlContent += " type: \"\(registry.type.rawValue)\"\n"+ yamlContent += " ghcr:\n"+ yamlContent += " registry: \"\(registry.ghcr.registry)\"\n"+ yamlContent += " organization: \"\(registry.ghcr.organization)\"\n"+ if let gcs = registry.gcs {+ yamlContent += " gcs:\n"+ yamlContent += " apiUrl: \"\(gcs.apiUrl)\"\n"+ yamlContent += " apiKey: \"\(gcs.apiKey)\"\n"+ }+ }+// Write YAML content to filetry yamlContent.write(to: URL(fileURLWithPath: configFilePath), atomically: true, encoding: .utf8)@@ -283,7 +300,22 @@ final class SettingsManager: @unchecked Sendable {var cachingEnabled = true // default to true for backward compatibilityvar vmLocations: [VMLocation] = []- var inLocationsSection = false+ // Registry config parsing state+ var registryType: RegistryType = .ghcr+ var ghcrRegistry = "ghcr.io"+ var ghcrOrganization = "trycua"+ var gcsApiUrl: String?+ var gcsApiKey: String?++ enum ParserSection {+ case top+ case vmLocations+ case registry+ case registryGhcr+ case registryGcs+ }++ var currentSection: ParserSection = .topvar currentLocation: (name: String?, path: String?) = (nil, nil)let lines = yamlString.split(separator: "\n")@@ -296,14 +328,37 @@ final class SettingsManager: @unchecked Sendable {continue}- // Check for section marker+ // Check for section markersif trimmedLine == "vmLocations:" {- inLocationsSection = true+ // Save any pending location before switching sections+ if let name = currentLocation.name, let path = currentLocation.path {+ vmLocations.append(VMLocation(name: name, path: path))+ }+ currentLocation = (nil, nil)+ currentSection = .vmLocations+ continue+ } else if trimmedLine == "registry:" {+ // Save any pending location before switching sections+ if let name = currentLocation.name, let path = currentLocation.path {+ vmLocations.append(VMLocation(name: name, path: path))+ }+ currentLocation = (nil, nil)+ currentSection = .registry+ continue+ } else if trimmedLine == "ghcr:" && currentSection == .registry {+ currentSection = .registryGhcr+ continue+ } else if trimmedLine == "gcs:" && currentSection == .registry {+ currentSection = .registryGcscontinue}- // In the locations section, handle line indentation more carefully- if inLocationsSection {+ // Handle indentation-based section detection+ let lineIndent = line.prefix(while: { $0 == " " }).count++ // Process based on current section+ switch currentSection {+ case .vmLocations:if trimmedLine.hasPrefix("-") || trimmedLine.contains("- name:") {// Process the previous location before starting a new oneif let name = currentLocation.name, let path = currentLocation.path {@@ -325,8 +380,63 @@ final class SettingsManager: @unchecked Sendable {currentLocation.path = value}}- } else {- // Process top-level keys outside the locations section++ case .registry:+ if let colonIndex = trimmedLine.firstIndex(of: ":") {+ let key = trimmedLine[..<colonIndex].trimmingCharacters(in: .whitespaces)+ let rawValue = trimmedLine[trimmedLine.index(after: colonIndex)...]+ .trimmingCharacters(in: .whitespaces)+ let value = extractValueFromYaml(rawValue)++ if key == "type" {+ registryType = RegistryType(rawValue: value) ?? .ghcr+ }+ }++ case .registryGhcr:+ if lineIndent < 4 && !trimmedLine.isEmpty {+ // Back to registry section or new section+ if trimmedLine == "gcs:" {+ currentSection = .registryGcs+ continue+ } else if !trimmedLine.hasPrefix("registry") && !trimmedLine.hasPrefix("organization") {+ currentSection = .top+ }+ }++ if let colonIndex = trimmedLine.firstIndex(of: ":") {+ let key = trimmedLine[..<colonIndex].trimmingCharacters(in: .whitespaces)+ let rawValue = trimmedLine[trimmedLine.index(after: colonIndex)...]+ .trimmingCharacters(in: .whitespaces)+ let value = extractValueFromYaml(rawValue)++ if key == "registry" {+ ghcrRegistry = value+ } else if key == "organization" {+ ghcrOrganization = value+ }+ }++ case .registryGcs:+ if lineIndent < 4 && !trimmedLine.isEmpty && !trimmedLine.hasPrefix("apiUrl") && !trimmedLine.hasPrefix("apiKey") {+ currentSection = .top+ }++ if let colonIndex = trimmedLine.firstIndex(of: ":") {+ let key = trimmedLine[..<colonIndex].trimmingCharacters(in: .whitespaces)+ let rawValue = trimmedLine[trimmedLine.index(after: colonIndex)...]+ .trimmingCharacters(in: .whitespaces)+ let value = extractValueFromYaml(rawValue)++ if key == "apiUrl" {+ gcsApiUrl = value+ } else if key == "apiKey" {+ gcsApiKey = value+ }+ }++ case .top:+ // Process top-level keysif let colonIndex = trimmedLine.firstIndex(of: ":") {let key = trimmedLine[..<colonIndex].trimmingCharacters(in: .whitespaces)let rawValue = trimmedLine[trimmedLine.index(after: colonIndex)...]@@ -354,11 +464,20 @@ final class SettingsManager: @unchecked Sendable {vmLocations.append(VMLocation(name: "default", path: "~/.lume"))}+ // Build registry config+ let ghcrConfig = GHCRConfig(registry: ghcrRegistry, organization: ghcrOrganization)+ var gcsConfig: GCSConfig?+ if let apiUrl = gcsApiUrl, let apiKey = gcsApiKey {+ gcsConfig = GCSConfig(apiUrl: apiUrl, apiKey: apiKey)+ }+ let registryConfig = RegistryConfig(type: registryType, ghcr: ghcrConfig, gcs: gcsConfig)+return LumeSettings(vmLocations: vmLocations,defaultLocationName: defaultLocationName,cacheDirectory: cacheDirectory,- cachingEnabled: cachingEnabled+ cachingEnabled: cachingEnabled,+ registry: registryConfig)}diff --git a/libs/lume/src/LumeController.swift b/libs/lume/src/LumeController.swiftindex aade532917..017ce42655 100644--- a/libs/lume/src/LumeController.swift+++ b/libs/lume/src/LumeController.swift@@ -271,7 +271,11 @@ final class LumeController {display: String,ipsw: String?,storage: String? = nil,- unattendedConfig: UnattendedConfig? = nil+ unattendedConfig: UnattendedConfig? = nil,+ debug: Bool = false,+ debugDir: String? = nil,+ noDisplay: Bool = true,+ vncPort: Int = 0) async throws {Logger.info("Creating VM",@@ -285,6 +289,8 @@ final class LumeController {"display": display,"ipsw": ipsw ?? "none","unattended": unattendedConfig != nil ? "yes" : "no",+ "debug": "\(debug)",+ "noDisplay": "\(noDisplay)",])do {@@ -312,6 +318,11 @@ final class LumeController {// Run unattended setup if config is providedif let config = unattendedConfig, os.lowercased() == "macos" {+ // Wait for the installation VZVirtualMachine to fully release auxiliary storage locks.+ // The VM created during IPSW installation holds a lock on the auxiliary storage file.+ // Swift's ARC may not immediately deallocate it, so we wait to ensure the lock is released.+ Logger.info("Waiting for installation resources to be released before unattended setup")+ try await Task.sleep(nanoseconds: 3_000_000_000) // 3 secondsLogger.info("Starting unattended Setup Assistant automation", metadata: ["name": name])// Load the finalized VM@@ -319,7 +330,14 @@ final class LumeController {// Run the unattended installerlet installer = UnattendedInstaller()- try await installer.install(vm: finalVM, config: config)+ try await installer.install(+ vm: finalVM,+ config: config,+ vncPort: vncPort,+ noDisplay: noDisplay,+ debug: debug,+ debugDir: debugDir+ )Logger.info("Unattended setup completed", metadata: ["name": name])}@@ -700,9 +718,9 @@ final class LumeController {storage: storage)- let imageContainerRegistry = ImageContainerRegistry(+ let imageRegistry = try RegistryFactory.createRegistry(registry: registry, organization: organization)- let _ = try await imageContainerRegistry.pull(+ let _ = try await imageRegistry.pull(image: actualImage,name: vmName,locationName: storage)@@ -775,11 +793,11 @@ final class LumeController {// Get the VM directorylet vmDir = try home.getVMDirectory(name, storage: actualLocation)- // Use ImageContainerRegistry to push the VM- let imageContainerRegistry = ImageContainerRegistry(+ // Use configured registry to push the VM+ let imageRegistry = try RegistryFactory.createRegistry(registry: registry, organization: organization)- try await imageContainerRegistry.push(+ try await imageRegistry.push(vmDirPath: vmDir.dir.path,imageName: imageName,tags: tags,@@ -842,9 +860,8 @@ final class LumeController {public func getImages(organization: String = "trycua") async throws -> ImageList {Logger.info("Listing local images", metadata: ["organization": organization])- let imageContainerRegistry = ImageContainerRegistry(- registry: "ghcr.io", organization: organization)- let cachedImages = try await imageContainerRegistry.getImages()+ let imageRegistry = try RegistryFactory.createRegistry(organization: organization)+ let cachedImages = try await imageRegistry.getImages()let imageInfos = cachedImages.map { image inImageInfo(diff --git a/libs/lume/src/Server/Handlers.swift b/libs/lume/src/Server/Handlers.swiftindex 88e26c782d..94ee0ac2ab 100644--- a/libs/lume/src/Server/Handlers.swift+++ b/libs/lume/src/Server/Handlers.swift@@ -744,6 +744,52 @@ extension Server {}}+ // MARK: - Host Status Handler++ /// Response structure for host status endpoint+ struct HostStatusResponse: Codable {+ let status: String+ let vmCount: Int+ let maxVMs: Int+ let availableSlots: Int+ let version: String++ enum CodingKeys: String, CodingKey {+ case status+ case vmCount = "vm_count"+ case maxVMs = "max_vms"+ case availableSlots = "available_slots"+ case version+ }+ }++ /// Handle GET /lume/host/status - Report host capacity and health for orchestrator+ func handleGetHostStatus() async throws -> HTTPResponse {+ do {+ let vmController = LumeController()++ // Get all VMs across all storage locations+ let vms = try vmController.list(storage: nil)++ // Count running VMs (Apple policy: max 2 VMs per host)+ let runningVMs = vms.filter { $0.status == "running" }+ let maxVMs = 2 // Apple Virtualization Framework limit++ let response = HostStatusResponse(+ status: "healthy",+ vmCount: runningVMs.count,+ maxVMs: maxVMs,+ availableSlots: max(0, maxVMs - runningVMs.count),+ version: "1.0.0" // Could be derived from build info+ )++ return try .json(response)+ } catch {+ Logger.error("Failed to get host status", metadata: ["error": error.localizedDescription])+ return .badRequest(message: error.localizedDescription)+ }+ }+// MARK: - Private Helper Methodsnonisolated private func startVM(diff --git a/libs/lume/src/Server/Server.swift b/libs/lume/src/Server/Server.swiftindex 617dfd435c..0ebd92c8bd 100644--- a/libs/lume/src/Server/Server.swift+++ b/libs/lume/src/Server/Server.swift@@ -314,6 +314,13 @@ final class Server {guard let self else { throw HTTPError.internalError }return try await self.handlePush(request.body)}),+ // Host status endpoint for orchestrator health checks+ Route(+ method: "GET", path: "/lume/host/status",+ handler: { [weak self] _ in+ guard let self else { throw HTTPError.internalError }+ return try await self.handleGetHostStatus()+ }),]}diff --git a/libs/lume/src/Unattended/UnattendedInstaller.swift b/libs/lume/src/Unattended/UnattendedInstaller.swiftindex 36a97057e3..51d456a650 100644--- a/libs/lume/src/Unattended/UnattendedInstaller.swift+++ b/libs/lume/src/Unattended/UnattendedInstaller.swift@@ -37,25 +37,54 @@ final class UnattendedInstaller {Logger.info("Parsed boot commands", metadata: ["count": "\(commands.count)"])// Start the VM in a background task (vm.run() blocks forever on success)+ // We use retry logic to handle cases where auxiliary storage is still locked+ // from a previous IPSW installation (race condition with VZVirtualMachine deallocation)Logger.info("Starting VM for unattended setup (background task)")+ var vmStartError: Error?let vmTask = Task {- try await vm.run(- noDisplay: noDisplay,- sharedDirectories: [],- mount: nil,- vncPort: vncPort,- recoveryMode: false,- usbMassStoragePaths: []- )+ var attempts = 0+ let maxAttempts = 3+ while attempts < maxAttempts {+ do {+ try await vm.run(+ noDisplay: noDisplay,+ sharedDirectories: [],+ mount: nil,+ vncPort: vncPort,+ recoveryMode: false,+ usbMassStoragePaths: []+ )+ return // VM started successfully+ } catch {+ let errorDescription = error.localizedDescription+ // Check if this is the auxiliary storage lock error+ if errorDescription.contains("auxiliary storage") || errorDescription.contains("Failed to lock") {+ attempts += 1+ if attempts < maxAttempts {+ Logger.info("VM start failed due to auxiliary storage lock, retrying", metadata: [+ "attempt": "\(attempts)/\(maxAttempts)",+ "waitTime": "5s"+ ])+ try await Task.sleep(nanoseconds: 5_000_000_000) // Wait 5 seconds before retry+ continue+ }+ }+ // For other errors or after max retries, store and rethrow+ vmStartError = error+ throw error+ }+ }}- // Give the VM a moment to start up- // Note: Startup errors will be caught when attempting to connect to the VNC input- // client below, as that requires the VM to be successfully initialized with a- // responding VNC server. Task.isCancelled only returns true for explicitly cancelled- // tasks, not for tasks that threw errors, so checking it here would be ineffective.+ // Give the VM a moment to start up and check for early failurestry await Task.sleep(nanoseconds: 2_000_000_000) // 2 seconds+ // Check if VM start failed early (e.g., auxiliary storage lock error)+ if let error = vmStartError {+ Logger.error("VM failed to start", metadata: ["error": error.localizedDescription])+ throw error+ }+// Wait for initial bootLogger.info("Waiting for VM to boot", metadata: ["bootWait": "\(config.bootWait)s"])try await Task.sleep(nanoseconds: UInt64(config.bootWait) * 1_000_000_000)diff --git a/libs/lume/src/Utils/CommandDocExtractor.swift b/libs/lume/src/Utils/CommandDocExtractor.swiftindex e29415f16f..d2780750f8 100644--- a/libs/lume/src/Utils/CommandDocExtractor.swift+++ b/libs/lume/src/Utils/CommandDocExtractor.swift@@ -109,8 +109,13 @@ enum CommandDocExtractor {OptionDoc(name: "ipsw", shortName: nil, help: "Path to IPSW file or 'latest' for macOS VMs", type: "String", defaultValue: nil, isOptional: true),OptionDoc(name: "storage", shortName: nil, help: "VM storage location to use", type: "String", defaultValue: nil, isOptional: true),OptionDoc(name: "unattended", shortName: nil, help: "[Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: tahoe. Only supported for macOS VMs.", type: "String", defaultValue: nil, isOptional: true),+ OptionDoc(name: "debug-dir", shortName: nil, help: "Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp)", type: "String", defaultValue: nil, isOptional: true),+ OptionDoc(name: "vnc-port", shortName: nil, help: "Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign)", type: "Int", defaultValue: "0", isOptional: true),+ ],+ flags: [+ FlagDoc(name: "debug", shortName: nil, help: "Enable debug mode for unattended setup - saves screenshots with click coordinates", defaultValue: false),+ FlagDoc(name: "no-display", shortName: nil, help: "Do not open the VNC client during unattended setup (default: true for unattended)", defaultValue: false),],- flags: [],subcommands: [])}diff --git a/libs/lume/src/VNC/VNCClient.swift b/libs/lume/src/VNC/VNCClient.swiftindex 0628434f3a..891d08a808 100644--- a/libs/lume/src/VNC/VNCClient.swift+++ b/libs/lume/src/VNC/VNCClient.swift@@ -33,7 +33,7 @@ actor VNCClient {connection = NWConnection(to: endpoint, using: .tcp)return try await withCheckedThrowingContinuation { continuation in- connection?.stateUpdateHandler = { [weak self] state in+ connection?.stateUpdateHandler = { state inTask { @MainActor inswitch state {case .ready:@@ -55,7 +55,7 @@ actor VNCClient {/// Perform the VNC handshake (RFB protocol)func handshake() async throws {- guard let conn = connection else {+ guard connection != nil else {throw VNCClientError.notConnected}diff --git a/libs/lume/tests/VMTests.swift b/libs/lume/tests/VMTests.swiftindex e7e312878a..fca6f11532 100644--- a/libs/lume/tests/VMTests.swift+++ b/libs/lume/tests/VMTests.swift@@ -46,6 +46,7 @@ private func setupVMDirectory(_ tempDir: URL) throws -> VMDirectory {@Test("VM initialization and configuration")func testVMInitialization() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = try setupVMDirectory(tempDir)var config = try VMConfig(os: "mock-os",@@ -76,6 +77,7 @@ func testVMInitialization() async throws {@Test("VM run and stop operations")func testVMRunAndStop() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = try setupVMDirectory(tempDir)var config = try VMConfig(os: "mock-os",@@ -113,6 +115,7 @@ func testVMRunAndStop() async throws {@Test("VM configuration updates")func testVMConfigurationUpdates() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = try setupVMDirectory(tempDir)var config = try VMConfig(os: "mock-os",@@ -148,6 +151,7 @@ func testVMConfigurationUpdates() async throws {@Test("VM setup process")func testVMSetup() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = try setupVMDirectory(tempDir)var config = try VMConfig(os: "mock-os",diff --git a/libs/lume/tests/VNCServiceTests.swift b/libs/lume/tests/VNCServiceTests.swiftindex 9ea6f2c84f..44dd8989d2 100644--- a/libs/lume/tests/VNCServiceTests.swift+++ b/libs/lume/tests/VNCServiceTests.swift@@ -5,6 +5,7 @@ import Testing@Test("VNCService starts correctly")func testVNCServiceStart() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = VMDirectory(Path(tempDir.path))let service = await MockVNCService(vmDirectory: vmDir)@@ -23,6 +24,7 @@ func testVNCServiceStart() async throws {@Test("VNCService stops correctly")func testVNCServiceStop() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = VMDirectory(Path(tempDir.path))let service = await MockVNCService(vmDirectory: vmDir)try await service.start(port: 5900, virtualMachine: nil)@@ -37,6 +39,7 @@ func testVNCServiceStop() async throws {@Test("VNCService handles client operations")func testVNCServiceClient() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = VMDirectory(Path(tempDir.path))let service = await MockVNCService(vmDirectory: vmDir)@@ -70,6 +73,7 @@ func testVNCServiceClient() async throws {@Test("VNCService handles virtual machine attachment")func testVNCServiceVMAttachment() async throws {let tempDir = try createTempDirectory()+ defer { try? FileManager.default.removeItem(at: tempDir) }let vmDir = VMDirectory(Path(tempDir.path))let service = await MockVNCService(vmDirectory: vmDir)let mockVM = "mock_vm"