Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

feat(lume): use offline unattended macOS setup

Francesco Bonaccib4cefbc3312d
diff --git a/README.md b/README.md
index 2db03ceb4b..9c216efaba 100644
--- a/README.md
+++ b/README.md
@@ -143,8 +143,10 @@ Create and manage macOS/Linux VMs with near-native performance on Apple Silicon
# Install Lume
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
-# Pull & start a macOS VM
-lume run macos-sequoia-vanilla:latest
+# Create and start a vanilla macOS VM from an Apple restore image
+curl -L "$(lume ipsw | tail -n 1)" -o ~/Downloads/macos.ipsw
+lume create macos-sequoia --ipsw ~/Downloads/macos.ipsw --unattended sequoia
+lume run macos-sequoia
```
**[Get Started](https://cua.ai/docs/lume)** | **[FAQ](https://cua.ai/docs/lume/guide/getting-started/faq)** | **[CLI Reference](https://cua.ai/docs/lume/reference/cli-reference)**
diff --git a/docs/content/docs/how-to-guides/lume/install-lume.mdx b/docs/content/docs/how-to-guides/lume/install-lume.mdx
index ae0aa8756e..2fb4e14bce 100644
--- a/docs/content/docs/how-to-guides/lume/install-lume.mdx
+++ b/docs/content/docs/how-to-guides/lume/install-lume.mdx
@@ -45,19 +45,35 @@ lume update --apply
The installer no longer creates scheduled auto-updaters. Older `lume-update` cron jobs and
LaunchAgents are removed the next time the installer runs.
-## Run a prebuilt VM
+## Create a vanilla macOS VM
-Pull and start a prebuilt macOS image:
+Download the latest supported Apple restore image, then let Lume prepare the VM
+offline. This skips Setup Assistant, creates the `lume` user, enables SSH and
+autologin, and disables screen-lock sleep settings:
```bash
-lume run macos-sequoia-vanilla:latest
+IPSW_URL="$(lume ipsw | tail -n 1)"
+curl -L "$IPSW_URL" -o ~/Downloads/macos.ipsw
+lume create macos-sequoia --ipsw ~/Downloads/macos.ipsw --unattended sequoia
+lume run macos-sequoia
```
<Callout title="Security Note">
- Prebuilt images use the default password `lume`. Change this immediately after your first login
- using the `passwd` command.
+ The unattended VM uses `lume`/`lume` for initial SSH and login access. Change
+ the password immediately with the `passwd` command.
</Callout>
+## Use a registry image
+
+Lume can also pull and run prebuilt or custom VM snapshots from an OCI-compatible
+registry. Registry images are optional; they are useful when you need a prepared
+snapshot or want to share a customized VM.
+
+```bash
+lume pull <image>:<tag>
+lume run <image>:<tag>
+```
+
## Start the API service manually
By default, Lume is installed as a background service that starts automatically on login. If you prefer to start the Lume API service manually when needed, you can use the `--no-background-service` option:
diff --git a/docs/content/docs/reference/lume/cli-reference.mdx b/docs/content/docs/reference/lume/cli-reference.mdx
index fdaad4c50b..d1485983ef 100644
--- a/docs/content/docs/reference/lume/cli-reference.mdx
+++ b/docs/content/docs/reference/lume/cli-reference.mdx
@@ -55,16 +55,16 @@ Create a new virtual machine
| `--display` | String | 1024x768 | Display resolution (e.g., 1024x768) |
| `--ipsw` | String | - | Path to IPSW file or 'latest' for macOS VMs |
| `--storage` | String | - | VM storage location to use |
-| `--unattended` | String | - | [Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. Only supported for macOS VMs. |
-| `--debug-dir` | String | - | Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp) |
-| `--vnc-port` | Int | 0 | Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign) |
+| `--unattended` | String | - | Prepare macOS unattended setup offline after install. Preset name or YAML path is accepted for compatibility. Built-in presets: sequoia, tahoe. Only supported for macOS VMs. |
+| `--debug-dir` | String | - | Compatibility option; ignored by offline setup. |
+| `--vnc-port` | Int | 0 | Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign). |
**Flags:**
| Name | Default | Description |
| ---- | ------- | ----------- |
-| `--debug` | false | Enable debug mode for unattended setup - saves screenshots with click coordinates |
-| `--no-display` | false | Do not open the VNC client during unattended setup (default: true for unattended) |
+| `--debug` | false | Compatibility flag; ignored by offline setup. |
+| `--no-display` | false | Compatibility flag; offline setup verifies headlessly. |
### lume clone
@@ -205,7 +205,7 @@ List virtual machines
### lume pull
-Pull a macOS image from GitHub Container Registry
+Pull a prebuilt or custom macOS image from an OCI-compatible registry
**Arguments:**
@@ -224,7 +224,7 @@ Pull a macOS image from GitHub Container Registry
### lume push
-Push a macOS VM to GitHub Container Registry
+Push a macOS VM to an OCI-compatible registry
**Arguments:**
@@ -293,7 +293,7 @@ Connect to a VM via SSH or execute commands remotely
### lume setup
-[Preview] Run unattended Setup Assistant automation on a macOS VM
+Prepare unattended macOS setup
**Arguments:**
@@ -305,17 +305,17 @@ Connect to a VM via SSH or execute commands remotely
| Name | Type | Default | Description |
| ---- | ---- | ------- | ----------- |
-| `--unattended` | String | - | Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. |
+| `--unattended` | String | tahoe | Defaults to tahoe. Preset name or YAML path for compatibility and optional post-SSH commands. Built-in presets: sequoia, tahoe. |
| `--storage` | String | - | VM storage location to use or direct path to VM location |
-| `--vnc-port` | Int | 0 | Port to use for the VNC server. Defaults to 0 (auto-assign) |
-| `--debug-dir` | String | - | Custom directory for debug screenshots (defaults to unique folder in system temp) |
+| `--vnc-port` | Int | 0 | Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign) |
+| `--debug-dir` | String | - | Compatibility option; ignored by offline setup |
**Flags:**
| Name | Default | Description |
| ---- | ------- | ----------- |
-| `--no-display` | false | Do not open the VNC client automatically |
-| `--debug` | false | Enable debug mode - saves screenshots with click coordinates |
+| `--no-display` | false | Compatibility flag; offline setup verifies headlessly |
+| `--debug` | false | Compatibility flag; ignored by offline setup |
## Configuration and Server
diff --git a/docs/content/docs/reference/lume/http-api.mdx b/docs/content/docs/reference/lume/http-api.mdx
index bec78d3ea6..51afe52900 100644
--- a/docs/content/docs/reference/lume/http-api.mdx
+++ b/docs/content/docs/reference/lume/http-api.mdx
@@ -568,7 +568,7 @@ const data = await response.json();
---
-### Pull a VM image from a container registry
+### Pull a prebuilt or custom VM image from a container registry
Pull a VM image from a container registry
@@ -592,7 +592,7 @@ Pull a VM image from a container registry
curl -X POST "http://localhost:7777/lume/pull" \
-H "Content-Type: application/json" \
-d '{
- "image": "macos-tahoe-vanilla:latest"
+ "image": "my-macos-snapshot:latest"
}'
```
</Tab>
@@ -601,7 +601,7 @@ curl -X POST "http://localhost:7777/lume/pull" \
import requests
data = {
- "image": "macos-tahoe-vanilla:latest",
+ "image": "my-macos-snapshot:latest",
}
response = requests.post("http://localhost:7777/lume/pull", json=data)
@@ -614,7 +614,7 @@ const response = await fetch(`http://localhost:7777/lume/pull`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
- image: "macos-tahoe-vanilla:latest",
+ image: "my-macos-snapshot:latest",
}),
});
const data = await response.json();
diff --git a/libs/lume/src/Commands/Create.swift b/libs/lume/src/Commands/Create.swift
index 05ff21fb1b..e62d228664 100644
--- a/libs/lume/src/Commands/Create.swift
+++ b/libs/lume/src/Commands/Create.swift
@@ -46,30 +46,30 @@ struct Create: AsyncParsableCommand {
@Option(
name: .customLong("unattended"),
- help: "[Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.",
+ help: "Prepare macOS unattended setup offline after install. Preset name or YAML path is accepted for compatibility. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.",
completion: .file(extensions: ["yml", "yaml"])
)
var unattended: String?
@Flag(
name: .customLong("debug"),
- help: "Enable debug mode for unattended setup - saves screenshots with click coordinates")
+ help: "Compatibility flag; ignored by offline setup.")
var debug: Bool = false
@Option(
name: .customLong("debug-dir"),
- help: "Custom directory for debug screenshots (defaults to unique folder in system temp)",
+ help: "Compatibility option; ignored by offline setup.",
completion: .directory)
var debugDir: String?
@Flag(
name: .customLong("no-display"),
- help: "Do not open the VNC client during unattended setup (default: true for unattended)")
+ help: "Compatibility flag; offline setup verifies headlessly.")
var noDisplay: Bool = false
@Option(
name: .customLong("vnc-port"),
- help: "Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign)")
+ help: "Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign).")
var vncPort: Int = 0
@Option(
@@ -108,12 +108,11 @@ struct Create: AsyncParsableCommand {
// Load unattended config if provided
var unattendedConfig: UnattendedConfig?
if let unattendedArg = unattended {
- Logger.info("[Preview] Unattended setup is an experimental feature")
let isPreset = UnattendedConfig.isPreset(name: unattendedArg)
unattendedConfig = try UnattendedConfig.load(from: unattendedArg)
Logger.info("Loaded unattended config", metadata: [
"source": isPreset ? "preset:\(unattendedArg)" : unattendedArg,
- "commands": "\(unattendedConfig?.bootCommands.count ?? 0)"
+ "config_boot_commands": "\(unattendedConfig?.bootCommands.count ?? 0)"
])
}
diff --git a/libs/lume/src/Commands/Pull.swift b/libs/lume/src/Commands/Pull.swift
index 1b77e0ee85..a02f7ba3f9 100644
--- a/libs/lume/src/Commands/Pull.swift
+++ b/libs/lume/src/Commands/Pull.swift
@@ -3,7 +3,7 @@ import Foundation
struct Pull: AsyncParsableCommand {
static let configuration = CommandConfiguration(
- abstract: "Pull a macOS image from GitHub Container Registry"
+ abstract: "Pull a prebuilt or custom macOS image from an OCI-compatible registry"
)
@Argument(help: "Image to pull (format: name:tag)")
@@ -13,7 +13,7 @@ struct Pull: AsyncParsableCommand {
help: "Name for the VM (defaults to image name without tag)", transform: { Optional($0) })
var name: String?
- @Option(help: "Github Container Registry to pull from. Defaults to ghcr.io")
+ @Option(help: "Container registry to pull from. Defaults to ghcr.io")
var registry: String = "ghcr.io"
@Option(help: "Organization to pull from. Defaults to trycua")
diff --git a/libs/lume/src/Commands/Push.swift b/libs/lume/src/Commands/Push.swift
index 62ffac2e7e..ba27160922 100644
--- a/libs/lume/src/Commands/Push.swift
+++ b/libs/lume/src/Commands/Push.swift
@@ -3,7 +3,7 @@ import Foundation
struct Push: AsyncParsableCommand {
static let configuration = CommandConfiguration(
- abstract: "Push a macOS VM to GitHub Container Registry"
+ abstract: "Push a macOS VM to an OCI-compatible registry"
)
@Argument(help: "Name of the VM to push")
@@ -15,7 +15,7 @@ struct Push: AsyncParsableCommand {
@Option(parsing: .upToNextOption, help: "Additional tags to push the same image to")
var additionalTags: [String] = []
- @Option(help: "Github Container Registry to push to. Defaults to ghcr.io")
+ @Option(help: "Container registry to push to. Defaults to ghcr.io")
var registry: String = "ghcr.io"
@Option(help: "Organization to push to. Defaults to trycua")
@@ -88,4 +88,4 @@ struct Push: AsyncParsableCommand {
legacy: legacy
)
}
-}
\ No newline at end of file
+}
diff --git a/libs/lume/src/Commands/Setup.swift b/libs/lume/src/Commands/Setup.swift
index d40823cfec..da058edd56 100644
--- a/libs/lume/src/Commands/Setup.swift
+++ b/libs/lume/src/Commands/Setup.swift
@@ -3,17 +3,13 @@ import Foundation
struct Setup: AsyncParsableCommand {
static let configuration = CommandConfiguration(
- abstract: "[Preview] Run unattended Setup Assistant automation on a macOS VM",
+ abstract: "Prepare unattended macOS setup",
discussion: """
- This is an experimental feature. Two modes are available:
+ Lume prepares the macOS disk offline, skips Setup Assistant, enables
+ autologin and SSH, disables screensaver lock, verifies SSH, then stops
+ the VM.
- - preset: Uses a YAML config file with scripted click/type commands (default).
- Presets are specific to macOS versions and may break across releases.
-
- - agent: Uses Claude's computer-use API to intelligently navigate the Setup Assistant.
- Adapts to any macOS version. Requires an Anthropic API key.
-
- The VM must be freshly created and not have completed the Setup Assistant yet.
+ The VM must be stopped before setup can patch its disk.
"""
)
@@ -22,37 +18,12 @@ struct Setup: AsyncParsableCommand {
completion: .custom(completeVMName))
var name: String
- @Option(
- name: .customLong("mode"),
- help: "Setup mode: 'preset' for YAML-based automation, 'agent' for Claude computer-use agent (default: preset)")
- var mode: String = "preset"
-
@Option(
name: .customLong("unattended"),
- help: "Preset name or path to YAML config file (required for preset mode). Built-in presets: sequoia, tahoe.",
+ help: "Defaults to tahoe. Preset name or YAML path for compatibility and optional post-SSH commands. Built-in presets: sequoia, tahoe.",
completion: .file(extensions: ["yml", "yaml"]))
var unattended: String?
- @Option(
- name: .customLong("anthropic-key"),
- help: "Anthropic API key for agent mode. Can also be set via ANTHROPIC_API_KEY env var.")
- var anthropicKey: String?
-
- @Option(
- name: .customLong("model"),
- help: "Claude model to use for agent mode (default: claude-sonnet-4-6)")
- var model: String = "claude-sonnet-4-6"
-
- @Option(
- name: .customLong("max-iterations"),
- help: "Maximum agent iterations before giving up (default: 200)")
- var maxIterations: Int = 200
-
- @Option(
- name: .customLong("system-prompt"),
- help: "Custom system prompt for the agent (overrides the default)")
- var systemPrompt: String?
-
@Option(
name: .customLong("storage"),
help: "VM storage location to use or direct path to VM location")
@@ -60,22 +31,22 @@ struct Setup: AsyncParsableCommand {
@Option(
name: .customLong("vnc-port"),
- help: "Port to use for the VNC server. Defaults to 0 (auto-assign)")
+ help: "Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign)")
var vncPort: Int = 0
@Flag(
name: .customLong("no-display"),
- help: "Do not open the VNC client automatically")
+ help: "Compatibility flag; offline setup verifies headlessly")
var noDisplay: Bool = false
@Flag(
name: .customLong("debug"),
- help: "Enable debug mode - saves screenshots with click coordinates")
+ help: "Compatibility flag; ignored by offline setup")
var debug: Bool = false
@Option(
name: .customLong("debug-dir"),
- help: "Custom directory for debug screenshots (defaults to unique folder in system temp)",
+ help: "Compatibility option; ignored by offline setup",
completion: .directory)
var debugDir: String?
@@ -83,73 +54,34 @@ struct Setup: AsyncParsableCommand {
@MainActor
func run() async throws {
- Logger.info("[Preview] Unattended setup is an experimental feature")
-
- switch mode {
- case "agent":
- // Resolve API key from flag or environment
- let apiKey = anthropicKey ?? ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]
- guard let apiKey, !apiKey.isEmpty else {
- throw UnattendedError.configLoadFailed(
- "Agent mode requires an Anthropic API key. Provide --anthropic-key or set ANTHROPIC_API_KEY env var."
- )
- }
-
- TelemetryClient.shared.record(event: TelemetryEvent.setup, properties: [
- "mode": "agent",
- "model": model
+ let unattended = unattended ?? "tahoe"
+
+ let config: UnattendedConfig
+ let isPreset = UnattendedConfig.isPreset(name: unattended)
+ do {
+ config = try UnattendedConfig.load(from: unattended)
+ Logger.info("Loaded unattended config", metadata: [
+ "source": isPreset ? "preset:\(unattended)" : unattended,
+ "bootWait": "\(config.bootWait)s",
+ "config_boot_commands": "\(config.bootCommands.count)"
])
-
- try await LumeController().setupWithAgent(
- name: name,
- apiKey: apiKey,
- model: model,
- maxIterations: maxIterations,
- systemPrompt: systemPrompt,
- storage: storage,
- vncPort: vncPort,
- noDisplay: noDisplay,
- debug: debug,
- debugDir: debugDir
- )
-
- case "preset":
- guard let unattended else {
- throw UnattendedError.configLoadFailed(
- "Preset mode requires --unattended <preset|path>. Built-in presets: sequoia, tahoe."
- )
- }
-
- let config: UnattendedConfig
- let isPreset = UnattendedConfig.isPreset(name: unattended)
- do {
- config = try UnattendedConfig.load(from: unattended)
- Logger.info("Loaded unattended config", metadata: [
- "source": isPreset ? "preset:\(unattended)" : unattended,
- "bootWait": "\(config.bootWait)s",
- "commands": "\(config.bootCommands.count)"
- ])
- } catch {
- throw UnattendedError.configLoadFailed("Failed to load config from \(unattended): \(error.localizedDescription)")
- }
-
- TelemetryClient.shared.record(event: TelemetryEvent.setup, properties: [
- "mode": "preset",
- "preset_name": isPreset ? unattended : "custom"
- ])
-
- try await LumeController().setup(
- name: name,
- config: config,
- storage: storage,
- vncPort: vncPort,
- noDisplay: noDisplay,
- debug: debug,
- debugDir: debugDir
- )
-
- default:
- throw UnattendedError.configLoadFailed("Unknown setup mode '\(mode)'. Use 'preset' or 'agent'.")
+ } catch {
+ throw UnattendedError.configLoadFailed("Failed to load config from \(unattended): \(error.localizedDescription)")
}
+
+ TelemetryClient.shared.record(event: TelemetryEvent.setup, properties: [
+ "mode": "offline",
+ "preset_name": isPreset ? unattended : "custom"
+ ])
+
+ try await LumeController().setup(
+ name: name,
+ config: config,
+ storage: storage,
+ vncPort: vncPort,
+ noDisplay: noDisplay,
+ debug: debug,
+ debugDir: debugDir
+ )
}
}
diff --git a/libs/lume/src/LumeController.swift b/libs/lume/src/LumeController.swift
index 9940888d9c..f2bffced9a 100644
--- a/libs/lume/src/LumeController.swift
+++ b/libs/lume/src/LumeController.swift
@@ -718,13 +718,12 @@ final class LumeController {
// Run unattended setup if config is provided
if let config = unattendedConfig, os.lowercased() == "macos" {
// Note: We don't write a provisioning marker for unattended setup.
- // The VM has disk + nvram at this point, so it's "running" during
- // the setup automation, not "provisioning".
+ // The finalized VM is patched offline, then booted briefly for SSH verification.
// Wait for the installation VZVirtualMachine to fully release auxiliary storage locks.
Logger.info("Waiting for installation resources to be released before unattended setup")
try await Task.sleep(nanoseconds: 3_000_000_000) // 3 seconds
- Logger.info("Starting unattended Setup Assistant automation", metadata: ["name": name])
+ Logger.info("Starting offline unattended setup", metadata: ["name": name])
// Load the finalized VM
let finalVM = try get(name: name, storage: storage)
@@ -754,12 +753,12 @@ final class LumeController {
throw error
}
- Logger.info("Unattended setup completed", metadata: ["name": name])
+ Logger.info("Offline unattended setup completed", metadata: ["name": name])
}
}
- /// Run unattended Setup Assistant automation on an existing macOS VM
+ /// Run offline unattended setup on an existing macOS VM
@MainActor
public func setup(
name: String,
@@ -776,8 +775,8 @@ final class LumeController {
metadata: [
"name": normalizedName,
"storage": storage ?? "home",
- "bootWait": "\(config.bootWait)s",
- "commands": "\(config.bootCommands.count)",
+ "config_boot_wait": "\(config.bootWait)s",
+ "config_boot_commands": "\(config.bootCommands.count)",
"debug": "\(debug)",
"debugDir": debugDir ?? "default"
])
@@ -802,57 +801,6 @@ final class LumeController {
}
}
- /// Run agent-based Setup Assistant automation using Claude computer-use API
- @MainActor
- public func setupWithAgent(
- name: String,
- apiKey: String,
- model: String = "claude-sonnet-4-6",
- maxIterations: Int = 100,
- systemPrompt: String? = nil,
- storage: String? = nil,
- vncPort: Int = 0,
- noDisplay: Bool = false,
- debug: Bool = false,
- debugDir: String? = nil
- ) async throws {
- let normalizedName = normalizeVMName(name: name)
- Logger.info(
- "Running agent-based setup",
- metadata: [
- "name": normalizedName,
- "model": model,
- "maxIterations": "\(maxIterations)",
- "debug": "\(debug)"
- ])
-
- do {
- let vm = try get(name: normalizedName, storage: storage)
-
- guard vm.config.os.lowercased() == "macos" else {
- throw VMError.unsupportedOS("Unattended setup is only supported for macOS VMs, got: \(vm.config.os)")
- }
-
- let installer = UnattendedInstaller()
- try await installer.installWithAgent(
- vm: vm,
- apiKey: apiKey,
- model: model,
- maxIterations: maxIterations,
- systemPrompt: systemPrompt,
- vncPort: vncPort,
- noDisplay: noDisplay,
- debug: debug,
- debugDir: debugDir
- )
-
- Logger.info("Agent-based setup completed", metadata: ["name": normalizedName])
- } catch {
- Logger.error("Failed to run agent-based setup", metadata: ["error": error.localizedDescription])
- throw error
- }
- }
-
@MainActor
public func delete(name: String, storage: String? = nil) async throws {
let normalizedName = normalizeVMName(name: name)
diff --git a/libs/lume/src/Resources/unattended-presets/sequoia.yml b/libs/lume/src/Resources/unattended-presets/sequoia.yml
index 6eddfe2d9c..f09d9221c4 100644
--- a/libs/lume/src/Resources/unattended-presets/sequoia.yml
+++ b/libs/lume/src/Resources/unattended-presets/sequoia.yml
@@ -1,291 +1,16 @@
-# Unattended macOS Sequoia Setup Assistant Configuration
-# Use with: lume create my-vm --os macOS --ipsw latest --unattended sequoia
+# Offline macOS unattended setup preset.
+#
+# Lume patches the installed disk offline, creates the `lume` user, enables
+# autologin and SSH, disables sleep/screen locking, boots once to verify SSH,
+# then stops.
-# Seconds to wait after VM boots before starting automation
-boot_wait: 30
+boot_wait: 0
+boot_commands: []
-boot_commands:
- # Wait for greeting screen to appear (shows "Hello" animation cycling through languages)
- # The boot_wait should be enough for the VM to reach this screen
- # Note: VM framebuffer is 1920x1440 (Retina-scaled)
- - "<delay 5>"
-
- # Press spacebar to dismiss the greeting screen ("Hello" animation)
- - "<space>"
- - "<delay 2>"
-
- # Language selection screen
- # Wait for "English" to be visible in the list (may take a moment as languages cycle)
- - "<wait 'English', timeout=120>"
- - "<delay 1>"
-
- # Type "English" to filter the list, click on "English" to select it, then press Enter
- - "<type 'English'>"
- - "<delay 1>"
- - "<click 'English'>"
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Country/Region selection
- - "<wait 'Country or Region'>"
- - "<delay 1>"
- # Click in the center of the list area to give it focus before typing
- - "<click_at 960,900>"
- - "<delay 1>"
- # Type to filter the country list
- - "<type 'United States'>"
- - "<delay 1>"
- # Press Enter to select the filtered item in the list
- - "<enter>"
- - "<delay 1>"
- # Click Continue button to proceed
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Transfer Your Data to This Mac (Migration Assistant)
- - "<wait 'Transfer Your Data'>"
- - "<delay 30>" # wait for refresh
- # Use tab navigation to select "Set up as new" (4th option)
- - "<click 'Set up as new'>"
- - "<delay 2>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Written and Spoken Languages screen (appears AFTER "Set up as new")
- - "<wait 'Written and Spoken Languages'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Accessibility screen - skip it
- - "<wait 'Accessibility'>"
- - "<delay 1>"
- - "<click 'Not Now'>"
- - "<delay 2>"
-
- # Data & Privacy screen
- - "<wait 'Data & Privacy'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Create a Mac Account
- # Field order: Full Name -> Account Name (auto-filled) -> Password -> Verify Password -> Hint -> Checkbox -> Continue
- - "<wait 'Create a Mac Account'>"
- - "<delay 10>"
- - "<click 'Full Name'>"
- - "<delay 1>"
- # Type Full Name
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Skip Account Name (auto-filled from Full Name)
- - "<tab>"
- - "<delay 1>"
- # Type Password
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Type Verify Password
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Skip Hint field
- - "<tab>"
- - "<delay 1>"
- # Untoggle "Allow computer account password to be reset with Apple Account" checkbox
- - "<space>"
- - "<delay 1>"
- # Click Continue button
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Sign In to Your Apple Account - skip it
- # Wait for "Email or Phone Number" to appear (indicates page is fully loaded and buttons are clickable)
- - "<wait 'Email or Phone Number'>"
- - "<delay 1>"
- - "<click 'Set Up Later'>"
- - "<delay 1>"
- # Confirmation popup: "Are you sure you want to skip signing in with an Apple Account?"
- # Tab to move focus to "Skip" button, then press Enter
- - '<wait "Don''t Skip">'
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Terms and Conditions
- # "Agree" appears in license text, so use index=-1 to click the last occurrence (the button)
- - "<wait 'Terms and Conditions'>"
- - "<delay 1>"
- - "<click 'Agree', index=-1>"
- - "<delay 1>"
- # Confirmation popup: "I have read and agree to the macOS Software License Agreement"
- # Modal "Agree" button is above the grayed-out background button, so use index=0 (first/topmost)
- - "<wait 'I have read and agree'>"
- - "<delay 1>"
- - "<click 'Agree', index=0>"
- - "<delay 2>"
-
- # Enable Location Services (skip it)
- - "<wait 'Enable Location Services'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 1>"
- # Confirmation popup: "Are you sure you don't want to use Location Services?"
- - '<wait "Don''t Use">'
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Select Your Time Zone
- - "<wait 'Select Your Time Zone'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Analytics
- - "<wait 'Analytics'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Screen Time
- - "<wait 'Screen Time'>"
- - "<delay 1>"
- - "<click 'Set Up Later'>"
- - "<delay 2>"
-
- # Siri (longer timeout as Screen Time page can be slow)
- - "<wait 'Siri', timeout=300>"
- - "<delay 2>"
- - "<click 'Enable Ask Siri'>"
- - "<delay 2>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Choose Your Look (Light/Dark mode)
- - "<wait 'Choose Your Look'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Update Mac Automatically
- - "<wait 'Update Mac Automatically'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Welcome screen - final step
- - "<wait 'Welcome to Mac'>"
- - "<delay 2>"
- - "<space>"
- - "<delay 5>"
-
- # Setup complete - wait for desktop to fully load
- - "<delay 30>"
-
- # Click on desktop to give it keyboard focus
- # Without this, hotkeys won't work (you'll hear the "trrr" error sound)
- - "<click_at 960,540>"
- - "<delay 1>"
-
- # ============================================
- # Enable full keyboard navigation (required for Tab in System Settings)
- # ============================================
- - "<cmd+space>"
- - "<delay 5>"
- - "<type 'Termina', delay=350>"
- - "<delay 5>"
- - "<enter>"
- - "<delay 5>"
- - "<type 'defaults write NSGlobalDomain AppleKeyboardUIMode -int 3', delay=350>"
- - "<enter>"
- - "<delay 3>"
- - "<cmd+q>"
- - "<delay 4>"
-
- # Click on desktop to refocus
- - "<click_at 960,540>"
- - "<delay 1>"
-
- # ============================================
- # Enable SSH and Full Disk Access via System Settings
- # ============================================
- # Open Spotlight and search for System Settings
- - "<cmd+space>"
- - "<delay 2>"
- - "<type 'System Settings'>"
- - "<delay 2>"
- - "<enter>"
- - "<delay 5>"
-
- # Wait for System Settings to fully load (sidebar shows General)
- - "<wait 'General', timeout=30>"
- - "<delay 2>"
-
- # Search for Remote Login settings
- - "<cmd+f>"
- - "<delay 1>"
- - "<type 'login'>"
- - "<delay 2>"
- - "<click 'Remote Login'>"
- - "<delay 30>"
-
- # Enable Remote Login toggle (first toggle on the page)
- - "<space>"
- - "<delay 5>"
- - "<tab>"
- - "<delay 5>"
- - "<space>"
- - "<delay 5>"
- - "<click 'Done'>"
- - "<delay 5>"
-
- # Close System Settings
- - "<cmd+q>"
- - "<delay 1>"
-
- # ============================================
- # Setup Complete!
- # ============================================
- # SSH is now enabled - Connect with: ssh lume@<ip-address> (password: lume)
- # Full disk access for remote users is enabled
-
-# Health check to verify setup was successful
-# This will SSH into the VM to confirm SSH is working
health_check:
type: ssh
user: lume
password: lume
- timeout: 30
- retries: 5
- retry_delay: 10
-
-# Post-setup commands to run via SSH after health check passes
-# These are more reliable than typing in Terminal via VNC
-post_ssh_commands:
- # Enable auto-login for the lume user
- - "sudo sysadminctl -autologin set -userName lume -password lume"
- # Generate kcpassword for auto-login (sysadminctl alone doesn't always create it)
- - "perl -e 'my @k=(125,137,82,35,210,188,221,234,163,185,31);my @p=unpack(q{C*},q{lume});my @e;for my $i(0..$#p){$e[$i]=$p[$i]^$k[$i%scalar@k]}my $r=scalar@e%12;push@e,(0)x(12-$r) if $r;open my $f,q{>},q{/tmp/kcp};binmode $f;print $f pack(q{C*},@e);close $f'"
- - "sudo cp /tmp/kcp /etc/kcpassword"
- - "sudo chmod 600 /etc/kcpassword"
- - "rm /tmp/kcp"
- # Disable screen saver
- - "defaults -currentHost write com.apple.screensaver idleTime -int 0"
- # Disable require password after sleep/screen saver
- - "defaults write com.apple.screensaver askForPassword -int 0"
- # Set password delay to 0 (no delay before requiring password)
- - "defaults write com.apple.screensaver askForPasswordDelay -int 0"
- # Prevent display from sleeping (never)
- - "sudo pmset -a displaysleep 0"
- # Prevent system sleep (never)
- - "sudo pmset -a sleep 0"
- # Disable auto-logout
- - "sudo defaults write /Library/Preferences/.GlobalPreferences com.apple.autologout.AutoLogOutDelay -int 0"
+ timeout: 5
+ retries: 60
+ retry_delay: 5
diff --git a/libs/lume/src/Resources/unattended-presets/tahoe.yml b/libs/lume/src/Resources/unattended-presets/tahoe.yml
index 43dd58ce34..f09d9221c4 100644
--- a/libs/lume/src/Resources/unattended-presets/tahoe.yml
+++ b/libs/lume/src/Resources/unattended-presets/tahoe.yml
@@ -1,302 +1,16 @@
-# Unattended macOS Tahoe Setup Assistant Configuration
-# Use with: lume create my-vm --os macOS --ipsw latest --unattended tahoe
+# Offline macOS unattended setup preset.
+#
+# Lume patches the installed disk offline, creates the `lume` user, enables
+# autologin and SSH, disables sleep/screen locking, boots once to verify SSH,
+# then stops.
-# Seconds to wait after VM boots before starting automation
-boot_wait: 30
+boot_wait: 0
+boot_commands: []
-boot_commands:
- # Wait for greeting screen to appear (shows "Hello" animation cycling through languages)
- # The boot_wait should be enough for the VM to reach this screen
- # Note: VM framebuffer is 1920x1440 (Retina-scaled)
- - "<delay 5>"
-
- # Press spacebar to dismiss the greeting screen ("Hello" animation)
- - "<space>"
- - "<delay 2>"
-
- # Language selection screen
- # Wait for "English" to be visible in the list (may take a moment as languages cycle)
- - "<wait 'English', timeout=120>"
- - "<delay 1>"
-
- # Type "English" to filter the list, click on "English" to select it, then press Enter
- - "<type 'English'>"
- - "<delay 1>"
- - "<click 'English'>"
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Country/Region selection
- - "<wait 'Country or Region'>"
- - "<delay 1>"
- # Click in the center of the list area to give it focus before typing
- - "<click_at 960,900>"
- - "<delay 1>"
- # Type to filter the country list
- - "<type 'United States'>"
- - "<delay 1>"
- # Press Enter to select the filtered item in the list
- - "<enter>"
- - "<delay 1>"
- # Click Continue button to proceed
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Transfer Your Data to This Mac (Migration Assistant)
- - "<wait 'Transfer Your Data'>"
- - "<delay 30>" # wait for refresh
- # Use tab navigation to select "Set up as new" (4th option)
- - "<click 'Set up as new'>"
- - "<delay 2>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Written and Spoken Languages screen (appears AFTER "Set up as new")
- - "<wait 'Written and Spoken Languages'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Accessibility screen - skip it
- - "<wait 'Accessibility'>"
- - "<delay 1>"
- - "<click 'Not Now'>"
- - "<delay 2>"
-
- # Data & Privacy screen
- - "<wait 'Data & Privacy'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Create a Mac Account
- # Field order: Full Name -> Account Name (auto-filled) -> Password -> Verify Password -> Hint -> Checkbox -> Continue
- - "<wait 'Create a Mac Account'>"
- - "<delay 10>"
- - "<click 'Full Name'>"
- - "<delay 1>"
- # Type Full Name
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Skip Account Name (auto-filled from Full Name)
- - "<tab>"
- - "<delay 1>"
- # Type Password
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Type Verify Password
- - "<type 'lume'>"
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- # Skip Hint field
- - "<tab>"
- - "<delay 1>"
- # Untoggle "Allow computer account password to be reset with Apple Account" checkbox
- - "<space>"
- - "<delay 1>"
- # Click Continue button
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Sign In to Your Apple Account - skip it
- # Wait for "Email or Phone Number" to appear (indicates page is fully loaded and buttons are clickable)
- - "<wait 'Email or Phone Number'>"
- - "<delay 1>"
- - "<click 'Set Up Later'>"
- - "<delay 1>"
- # Confirmation popup: "Are you sure you want to skip signing in with an Apple Account?"
- # Tab to move focus to "Skip" button, then press Enter
- - '<wait "Don''t Skip">'
- - "<delay 1>"
- - "<tab>"
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Terms and Conditions
- # "Agree" appears in license text, so use index=-1 to click the last occurrence (the button)
- - "<wait 'Terms and Conditions'>"
- - "<delay 1>"
- - "<click 'Agree', index=-1>"
- - "<delay 1>"
- # Confirmation popup: "I have read and agree to the macOS Software License Agreement"
- # Modal "Agree" button is above the grayed-out background button, so use index=0 (first/topmost)
- - "<wait 'I have read and agree'>"
- - "<delay 1>"
- - "<click 'Agree', index=0>"
- - "<delay 2>"
-
- # Enable Location Services (skip it)
- - "<wait 'Enable Location Services'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 1>"
- # Confirmation popup: "Are you sure you don't want to use Location Services?"
- - '<wait "Don''t Use">'
- - "<delay 1>"
- - "<enter>"
- - "<delay 2>"
-
- # Select Your Time Zone
- - "<wait 'Select Your Time Zone'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Analytics
- - "<wait 'Analytics'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Screen Time
- - "<wait 'Screen Time'>"
- - "<delay 1>"
- - "<click 'Set Up Later'>"
- - "<delay 2>"
-
- # Siri (longer timeout as Screen Time page can be slow)
- - "<wait 'Siri', timeout=300>"
- - "<delay 2>"
- - "<click 'Enable Ask Siri'>"
- - "<delay 2>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # FileVault - skip it
- - "<wait 'FileVault'>"
- - "<delay 1>"
- - "<click 'Not Now'>"
- - "<delay 1>"
- # Confirmation popup: "Mac Data Will Not Be Securely Encrypted"
- - "<wait 'Securely Encrypted'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Choose Your Look (Light/Dark mode)
- - "<wait 'Choose Your Look'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Update Mac Automatically
- - "<wait 'Update Mac Automatically'>"
- - "<delay 1>"
- - "<click 'Continue'>"
- - "<delay 2>"
-
- # Welcome screen - final step
- - "<wait 'welcome'>"
- - "<delay 1>"
- - "<click 'Get Started'>"
- - "<delay 5>"
-
- # Setup complete - wait for desktop to fully load
- - "<delay 30>"
-
- # Click on desktop to give it keyboard focus
- # Without this, hotkeys won't work (you'll hear the "trrr" error sound)
- - "<click_at 960,540>"
- - "<delay 1>"
-
- # ============================================
- # Enable full keyboard navigation (required for Tab in System Settings)
- # ============================================
- - "<cmd+space>"
- - "<delay 5>"
- - "<type 'Termina', delay=350>"
- - "<delay 5>"
- - "<enter>"
- - "<delay 5>"
- - "<type 'defaults write NSGlobalDomain AppleKeyboardUIMode -int 3', delay=350>"
- - "<enter>"
- - "<delay 3>"
- - "<cmd+q>"
- - "<delay 4>"
-
- # Click on desktop to refocus
- - "<click_at 960,540>"
- - "<delay 1>"
-
- # ============================================
- # Enable SSH and Full Disk Access via System Settings
- # ============================================
- # Open Spotlight and search for System Settings
- - "<cmd+space>"
- - "<delay 2>"
- - "<type 'System Settings'>"
- - "<delay 2>"
- - "<enter>"
- - "<delay 5>"
-
- # Wait for System Settings to fully load (sidebar shows General)
- - "<wait 'General', timeout=30>"
- - "<delay 2>"
-
- # Search for Remote Login settings
- - "<cmd+f>"
- - "<delay 1>"
- - "<type 'login'>"
- - "<delay 2>"
- - "<click 'Remote Login'>"
- - "<delay 30>"
-
- # Enable Remote Login toggle (first toggle on the page)
- - "<space>"
- - "<delay 5>"
- - "<tab>"
- - "<delay 5>"
- - "<space>"
- - "<delay 5>"
- - "<click 'Done'>"
- - "<delay 5>"
-
- # Close System Settings
- - "<cmd+q>"
- - "<delay 2>"
-
- # ============================================
- # Setup Complete!
- # ============================================
- # SSH is now enabled - Connect with: ssh lume@<ip-address> (password: lume)
- # Full disk access for remote users is enabled
-
-# Health check to verify setup was successful
-# This will SSH into the VM to confirm SSH is working
health_check:
type: ssh
user: lume
password: lume
- timeout: 30
- retries: 5
- retry_delay: 10
-
-# Post-setup commands to run via SSH after health check passes
-# These are more reliable than typing in Terminal via VNC
-post_ssh_commands:
- # Enable auto-login for the lume user
- - "sudo sysadminctl -autologin set -userName lume -password lume"
- # Generate kcpassword for auto-login (sysadminctl alone doesn't always create it)
- - "perl -e 'my @k=(125,137,82,35,210,188,221,234,163,185,31);my @p=unpack(q{C*},q{lume});my @e;for my $i(0..$#p){$e[$i]=$p[$i]^$k[$i%scalar@k]}my $r=scalar@e%12;push@e,(0)x(12-$r) if $r;open my $f,q{>},q{/tmp/kcp};binmode $f;print $f pack(q{C*},@e);close $f'"
- - "sudo cp /tmp/kcp /etc/kcpassword"
- - "sudo chmod 600 /etc/kcpassword"
- - "rm /tmp/kcp"
- # Disable screen saver
- - "defaults -currentHost write com.apple.screensaver idleTime -int 0"
- # Disable require password after sleep/screen saver
- - "defaults write com.apple.screensaver askForPassword -int 0"
- # Set password delay to 0 (no delay before requiring password)
- - "defaults write com.apple.screensaver askForPasswordDelay -int 0"
- # Prevent display from sleeping (never)
- - "sudo pmset -a displaysleep 0"
- # Prevent system sleep (never)
- - "sudo pmset -a sleep 0"
- # Disable auto-logout
- - "sudo defaults write /Library/Preferences/.GlobalPreferences com.apple.autologout.AutoLogOutDelay -int 0"
+ timeout: 5
+ retries: 60
+ retry_delay: 5
diff --git a/libs/lume/src/Server/MCPServer.swift b/libs/lume/src/Server/MCPServer.swift
index 5864e0b14b..f167fe4b1a 100644
--- a/libs/lume/src/Server/MCPServer.swift
+++ b/libs/lume/src/Server/MCPServer.swift
@@ -2,7 +2,7 @@ import Foundation
import MCP
/// MCP (Model Context Protocol) server for Lume VM management
-/// Allows AI agents like Claude to manage VMs through MCP tools
+/// Allows MCP clients to manage VMs through tools
@MainActor
final class LumeMCPServer {
private let controller: LumeController
@@ -140,11 +140,11 @@ final class LumeMCPServer {
```
### 2. Create a New VM (if needed)
- Creating a VM takes 15-30 minutes. Use `unattended: "tahoe"` for automatic setup with SSH enabled:
+ Creating a VM takes 15-30 minutes. Use `unattended: "tahoe"` for offline setup with autologin and SSH enabled:
```
lume_create_vm(name: "sandbox", unattended: "tahoe")
```
- The tool returns immediately. Poll `lume_list_vms` to monitor progress—status changes from `provisioning (ipsw_install)` → `running` (during unattended setup) → `stopped`.
+ The tool returns immediately. Poll `lume_list_vms` to monitor progress—status changes from `provisioning (ipsw_install)` → `running` (during SSH verification) → `stopped`.
### 3. Start the VM
Start with optional shared directory for file access:
@@ -197,7 +197,7 @@ final class LumeMCPServer {
| `stopped` | Ready to start |
| `running` | VM is active |
| `provisioning (ipsw_install)` | Installing macOS |
- | `running` | VM is running (including during unattended setup) |
+ | `running` | VM is running (including during unattended SSH verification) |
## Limitations
- Max 2 macOS VMs running simultaneously (Apple licensing)
@@ -259,7 +259,7 @@ final class LumeMCPServer {
messages: [
.user("""
Create a new macOS sandbox VM named '\(vmName)' with these requirements:
- 1. Use unattended setup (tahoe preset) for automatic configuration
+ 1. Use unattended setup (tahoe preset) for offline configuration
2. The VM should have SSH enabled with credentials lume/lume
3. Monitor the provisioning status until complete
4. Once ready, start the VM in headless mode
diff --git a/libs/lume/src/Server/Requests.swift b/libs/lume/src/Server/Requests.swift
index 86b4c30e19..df4e6a6908 100644
--- a/libs/lume/src/Server/Requests.swift
+++ b/libs/lume/src/Server/Requests.swift
@@ -85,7 +85,7 @@ struct CreateVMRequest: Codable {
let display: String
let ipsw: String?
let storage: String?
- /// Preset name or path to YAML config file for unattended macOS Setup Assistant automation
+ /// Preset name or YAML path for offline unattended macOS setup
let unattended: String?
/// Network mode: "nat" (default), "bridged", or "bridged:<interface>"
let network: String?
diff --git a/libs/lume/src/Unattended/AgentSetupRunner.swift b/libs/lume/src/Unattended/AgentSetupRunner.swift
deleted file mode 100644
index e9acbac354..0000000000
--- a/libs/lume/src/Unattended/AgentSetupRunner.swift
+++ /dev/null
@@ -1,566 +0,0 @@
-import Foundation
-import CoreGraphics
-import ImageIO
-
-/// Runs macOS Setup Assistant automation using Claude's computer-use API
-/// instead of brittle YAML presets. Adapts to any macOS version.
-@MainActor
-final class AgentSetupRunner {
- private let vncService: VNCService
- private let client: AnthropicClient
- /// Display dimensions reported to Claude (capped for API efficiency)
- private let displayWidth: Int
- private let displayHeight: Int
- /// Actual VNC framebuffer dimensions (for coordinate mapping)
- private let vncWidth: Int
- private let vncHeight: Int
- private let maxIterations: Int
- private let debug: Bool
- private let debugDirectory: URL
- private var stepIndex: Int = 0
- /// Optional SSH reachability check — returns true if SSH is reachable
- private let sshCheck: (() async -> Bool)?
-
- /// Max dimensions to report to Claude API (keeps screenshot payloads small)
- private static let maxAPIWidth = 1024
- private static let maxAPIHeight = 768
-
- static let defaultSystemPrompt = """
- You are automating the macOS Setup Assistant on a freshly installed virtual machine.
- Your goal is to complete the setup as quickly as possible with these settings:
-
- 1. Language: English
- 2. Country: United States
- 3. Skip all Apple ID sign-in (look for "Set Up Later", "Skip", "Other Sign-In Options", or similar)
- 4. Create a local user account with:
- - Full Name: lume
- - Account Name: lume
- - Password: lume
- 5. Skip or decline all optional features (Siri, Analytics, Screen Time, Location Services, etc.)
- 6. Accept any privacy/terms screens by clicking Continue/Agree
- 7. Reach the macOS desktop and enable SSH
-
- ## Expected screen sequence (adapt as needed — screens may vary by macOS version):
- 1. "Hello" greeting animation → press Space or click to dismiss
- 2. Language selection → type "English", select it, press Enter
- 3. Country/Region → type "United States", press Enter, click Continue
- 4. Transfer Your Data → click "Set up as new", then Continue
- 5. Written and Spoken Languages → click Continue
- 6. Accessibility → click "Not Now"
- 7. Data & Privacy → click Continue
- 8. Create a Mac Account → fill Full Name "lume", Tab to skip Account Name (auto-fills), type password "lume", Tab, verify password "lume", Tab to skip Hint, Tab to checkbox, press Space to untoggle "Allow reset with Apple Account", click Continue
- 9. Apple Account sign-in → look for "Set Up Later" or "Other Sign-In Options" or "Skip". If a "Don't Skip" confirmation appears, Tab to "Skip" and press Enter
- 10. Terms and Conditions → click "Agree" button (at bottom, not in the text), then click "Agree" again in the confirmation popup
- 11. Enable Location Services → click Continue, then confirm "Don't Use" in popup
- 12. Select Your Time Zone → click Continue
- 13. Analytics → click Continue
- 14. Screen Time → click "Set Up Later"
- 15. Siri → click to enable/disable, then click Continue
- 16. FileVault → click "Not Now", then confirm in popup
- 17. Choose Your Look → click Continue
- 18. Update Mac Automatically → click Continue
- 19. Welcome/Get Started → click "Get Started"
-
- ## After reaching the desktop, enable SSH:
- 1. Open Spotlight (Cmd+Space), type "System Settings", press Enter
- 2. Search for "Remote Login" using Cmd+F, click "Remote Login"
- 3. Enable the Remote Login toggle, allow all users, click Done
- 4. Close System Settings (Cmd+Q)
- 5. The task is now complete.
-
- ## Important tips:
- - The screenshot colors may appear tinted/shifted — this is a VNC artifact. Focus on shapes and layout.
- - After each action, take a screenshot to verify the result before proceeding.
- - If a button or text is not visible, try waiting 2-3 seconds, then take another screenshot.
- - Use keyboard shortcuts when buttons are hard to click (Tab to navigate, Enter to confirm, Space to toggle).
- - If you see a password confirmation field, enter "lume" again.
- - When you see the macOS desktop with the Dock and menu bar, proceed to enable SSH.
- - When you see the lock screen with the user "lume", the setup is complete — just stop.
- """
-
- init(
- vncService: VNCService,
- apiKey: String,
- model: String = "claude-sonnet-4-6",
- displayWidth: Int,
- displayHeight: Int,
- maxIterations: Int = 200,
- debug: Bool = false,
- debugDirectory: URL? = nil,
- sshCheck: (() async -> Bool)? = nil
- ) {
- self.vncService = vncService
- self.client = AnthropicClient(apiKey: apiKey, model: model)
- self.vncWidth = displayWidth
- self.vncHeight = displayHeight
- // Cap display dimensions for API efficiency
- self.displayWidth = min(displayWidth, Self.maxAPIWidth)
- self.displayHeight = min(displayHeight, Self.maxAPIHeight)
- self.maxIterations = maxIterations
- self.debug = debug
- self.debugDirectory = debugDirectory ?? FileManager.default.temporaryDirectory
- .appendingPathComponent("agent-setup-\(UUID().uuidString)")
- self.sshCheck = sshCheck
- }
-
- /// Run the agent loop to complete Setup Assistant
- func run(systemPrompt: String? = nil) async throws {
- let prompt = systemPrompt ?? Self.defaultSystemPrompt
-
- if debug {
- try? FileManager.default.createDirectory(at: debugDirectory, withIntermediateDirectories: true)
- Logger.info("Agent debug screenshots will be saved to \(debugDirectory.path)")
- }
-
- // Take initial screenshot
- Logger.info("Agent taking initial screenshot")
- let initialScreenshot = try await captureScreenshot()
-
- // Build initial messages with screenshot
- var messages: [[String: Any]] = [
- [
- "role": "user",
- "content": [
- [
- "type": "text",
- "text": "Here is the current screen. Please complete the macOS Setup Assistant."
- ] as [String: Any],
- [
- "type": "image",
- "source": [
- "type": "base64",
- "media_type": "image/jpeg",
- "data": initialScreenshot
- ]
- ] as [String: Any]
- ]
- ]
- ]
-
- var iteration = 0
- while iteration < maxIterations {
- iteration += 1
- Logger.info("Agent iteration \(iteration)/\(maxIterations)")
-
- // Call Claude API
- let response = try await client.sendMessage(
- systemPrompt: prompt,
- messages: messages,
- displayWidth: displayWidth,
- displayHeight: displayHeight
- )
-
- // Add assistant response to conversation
- let assistantContent = try responseToRawContent(response)
- messages.append(["role": "assistant", "content": assistantContent])
-
- // Check if Claude is done (no tool use)
- let toolUseBlocks = response.content.filter {
- if case .toolUse = $0 { return true }
- return false
- }
-
- if toolUseBlocks.isEmpty {
- // Claude thinks the task is complete
- for block in response.content {
- if case .text(let text) = block {
- Logger.info("Agent completed: \(text)")
- }
- }
- Logger.info("Agent setup finished after \(iteration) iterations")
- return
- }
-
- // Process each tool use
- var toolResults: [[String: Any]] = []
- for block in response.content {
- guard case .toolUse(let id, let name, let input) = block else { continue }
- guard name == "computer" else {
- toolResults.append([
- "type": "tool_result",
- "tool_use_id": id,
- "content": "Unknown tool: \(name)",
- "is_error": true
- ])
- continue
- }
-
- let result = try await executeComputerAction(id: id, input: input)
- toolResults.append(result)
- }
-
- // Add tool results to messages
- messages.append(["role": "user", "content": toolResults])
-
- // Check SSH reachability — if SSH is up, setup is complete regardless of agent state.
- // Only start checking after step 50 (setup assistant takes at least that many) and
- // check every 10 iterations to avoid slowing down the loop.
- if iteration >= 50 && iteration % 10 == 0, let sshCheck = sshCheck {
- if await sshCheck() {
- Logger.info("SSH is reachable — setup complete after \(iteration) iterations")
- return
- }
- }
-
- // Keep conversation history bounded to prevent request_too_large errors.
- // Keep the first message (initial screenshot) plus the last 20 messages (10 turns).
- let maxHistory = 20
- if messages.count > maxHistory + 1 {
- let first = messages[0]
- let recent = Array(messages.suffix(maxHistory))
- messages = [first] + recent
- }
- }
-
- throw AnthropicError.maxIterationsReached
- }
-
- // MARK: - Action Execution
-
- private func executeComputerAction(id: String, input: [String: AnyCodable]) async throws -> [String: Any] {
- guard let action = input["action"]?.stringValue else {
- return errorResult(id: id, message: "Missing 'action' field")
- }
-
- Logger.info("Agent action: \(action)", metadata: inputMetadata(input))
- stepIndex += 1
-
- do {
- switch action {
- case "screenshot":
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "left_click":
- guard let coord = coordinateFromInput(input) else {
- return errorResult(id: id, message: "Missing coordinate for left_click")
- }
- try await vncService.sendMouseClick(at: coord, button: .left)
- try await Task.sleep(nanoseconds: 500_000_000) // 500ms for UI to update
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "right_click":
- guard let coord = coordinateFromInput(input) else {
- return errorResult(id: id, message: "Missing coordinate for right_click")
- }
- try await vncService.sendMouseClick(at: coord, button: .right)
- try await Task.sleep(nanoseconds: 500_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "double_click":
- guard let coord = coordinateFromInput(input) else {
- return errorResult(id: id, message: "Missing coordinate for double_click")
- }
- try await vncService.sendMouseClick(at: coord, button: .left)
- try await Task.sleep(nanoseconds: 100_000_000)
- try await vncService.sendMouseClick(at: coord, button: .left)
- try await Task.sleep(nanoseconds: 500_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "type":
- guard let text = input["text"]?.stringValue else {
- return errorResult(id: id, message: "Missing text for type action")
- }
- try await vncService.sendText(text, delayMs: 50)
- try await Task.sleep(nanoseconds: 300_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "key":
- guard let keyText = input["text"]?.stringValue else {
- return errorResult(id: id, message: "Missing text for key action")
- }
- try await sendKey(keyText)
- try await Task.sleep(nanoseconds: 300_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "scroll":
- // For scroll, we just take a screenshot after — VNC scroll support varies
- guard let coord = coordinateFromInput(input) else {
- return errorResult(id: id, message: "Missing coordinate for scroll")
- }
- let direction = input["scroll_direction"]?.stringValue ?? "down"
- let amount = input["scroll_amount"]?.intValue ?? 3
- try await sendScroll(at: coord, direction: direction, amount: amount)
- try await Task.sleep(nanoseconds: 500_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- case "mouse_move":
- // Just acknowledge — VNC doesn't need explicit moves before clicks
- return [
- "type": "tool_result",
- "tool_use_id": id,
- "content": "Mouse moved"
- ]
-
- case "wait":
- let duration = input["duration"]?.intValue ?? 2
- try await Task.sleep(nanoseconds: UInt64(duration) * 1_000_000_000)
- let base64 = try await captureScreenshot()
- return screenshotResult(id: id, base64: base64)
-
- default:
- return errorResult(id: id, message: "Unsupported action: \(action)")
- }
- } catch {
- Logger.error("Agent action failed", metadata: ["action": action, "error": error.localizedDescription])
- return errorResult(id: id, message: "Action '\(action)' failed: \(error.localizedDescription)")
- }
- }
-
- // MARK: - Screenshot
-
- private func captureScreenshot() async throws -> String {
- let image = try await vncService.captureFramebuffer()
-
- // Normalize and resize: re-render through a standard RGBA CGContext.
- // This fixes VNC color channel issues (BGRA→RGBA) and scales to display size.
- let normalized = normalizeImage(image, maxWidth: displayWidth, maxHeight: displayHeight)
-
- if debug {
- saveDebugScreenshot(normalized)
- }
-
- // Encode as JPEG for much smaller payloads (PNG screenshots can be 3MB+)
- guard let base64 = normalized.jpegBase64(quality: 0.7) else {
- throw UnattendedError.framebufferCaptureFailed("Failed to encode screenshot as JPEG")
- }
-
- return base64
- }
-
- /// Resize the image to fit within the given bounds. Let CoreGraphics handle pixel format
- /// conversion naturally — no manual channel swapping needed.
- private func normalizeImage(_ image: CGImage, maxWidth: Int, maxHeight: Int) -> CGImage {
- let srcWidth = image.width
- let srcHeight = image.height
-
- // Scale to fit within bounds
- let scale: Double
- if srcWidth > maxWidth || srcHeight > maxHeight {
- scale = min(Double(maxWidth) / Double(srcWidth), Double(maxHeight) / Double(srcHeight))
- } else {
- scale = 1.0
- }
- let newWidth = Int(Double(srcWidth) * scale)
- let newHeight = Int(Double(srcHeight) * scale)
-
- let colorSpace = CGColorSpaceCreateDeviceRGB()
- guard let context = CGContext(
- data: nil,
- width: newWidth,
- height: newHeight,
- bitsPerComponent: 8,
- bytesPerRow: 0,
- space: colorSpace,
- bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue
- ) else {
- return image
- }
-
- context.interpolationQuality = .high
- context.draw(image, in: CGRect(x: 0, y: 0, width: newWidth, height: newHeight))
-
- return context.makeImage() ?? image
- }
-
- private func saveDebugScreenshot(_ image: CGImage) {
- let filename = String(format: "%03d-agent-step.png", stepIndex)
- let url = debugDirectory.appendingPathComponent(filename)
- guard let destination = CGImageDestinationCreateWithURL(url as CFURL, "public.png" as CFString, 1, nil) else {
- return
- }
- CGImageDestinationAddImage(destination, image, nil)
- CGImageDestinationFinalize(destination)
- Logger.info("Saved debug screenshot", metadata: ["path": url.path])
- }
-
- // MARK: - Input Helpers
-
- /// Parse coordinate from Claude's response and scale to VNC framebuffer space.
- /// Claude operates in displayWidth x displayHeight space, VNC uses vncWidth x vncHeight.
- private func coordinateFromInput(_ input: [String: AnyCodable]) -> CGPoint? {
- if let coordArray = input["coordinate"]?.value as? [Any], coordArray.count == 2 {
- let x: Double
- let y: Double
- if let xi = coordArray[0] as? Int, let yi = coordArray[1] as? Int {
- x = Double(xi)
- y = Double(yi)
- } else if let xd = coordArray[0] as? Double, let yd = coordArray[1] as? Double {
- x = xd
- y = yd
- } else {
- return nil
- }
- // Scale from Claude's coordinate space to VNC framebuffer space
- let scaleX = Double(vncWidth) / Double(displayWidth)
- let scaleY = Double(vncHeight) / Double(displayHeight)
- return CGPoint(x: x * scaleX, y: y * scaleY)
- }
- return nil
- }
-
- private func sendKey(_ keyText: String) async throws {
- // Parse key combinations like "ctrl+s", "Return", "space", etc.
- let parts = keyText.lowercased().split(separator: "+").map(String.init)
-
- var modifiers: VNCKeyModifiers = .none
- var mainKey: String = ""
-
- for part in parts {
- switch part {
- case "ctrl", "control": modifiers.insert(.control)
- case "shift": modifiers.insert(.shift)
- case "alt", "option": modifiers.insert(.option)
- case "cmd", "command", "super", "meta": modifiers.insert(.command)
- default: mainKey = part
- }
- }
-
- let keyCode = keyNameToKeyCode(mainKey)
- try await vncService.sendKeyPress(keyCode, modifiers: modifiers)
- }
-
- private func sendScroll(at point: CGPoint, direction: String, amount: Int) async throws {
- // VNC scroll is button 4 (up) or button 5 (down)
- // Use key-based scrolling as fallback (VNC scroll button support varies)
- for _ in 0..<amount {
- switch direction {
- case "up":
- try await vncService.sendKeyPress(keyNameToKeyCode("up"), modifiers: .none)
- case "down":
- try await vncService.sendKeyPress(keyNameToKeyCode("down"), modifiers: .none)
- case "left":
- try await vncService.sendKeyPress(keyNameToKeyCode("left"), modifiers: .none)
- case "right":
- try await vncService.sendKeyPress(keyNameToKeyCode("right"), modifiers: .none)
- default:
- break
- }
- try await Task.sleep(nanoseconds: 100_000_000)
- }
- }
-
- private func keyNameToKeyCode(_ name: String) -> UInt16 {
- // Map common key names to macOS key codes
- switch name {
- case "return", "enter": return 36
- case "tab": return 48
- case "space": return 49
- case "escape", "esc": return 53
- case "backspace", "delete": return 51
- case "up": return 126
- case "down": return 125
- case "left": return 123
- case "right": return 124
- case "a": return 0
- case "b": return 11
- case "c": return 8
- case "d": return 2
- case "e": return 14
- case "f": return 3
- case "g": return 5
- case "h": return 4
- case "i": return 34
- case "j": return 38
- case "k": return 40
- case "l": return 37
- case "m": return 46
- case "n": return 45
- case "o": return 31
- case "p": return 35
- case "q": return 12
- case "r": return 15
- case "s": return 1
- case "t": return 17
- case "u": return 32
- case "v": return 9
- case "w": return 13
- case "x": return 7
- case "y": return 16
- case "z": return 6
- case "f1": return 122
- case "f2": return 120
- case "f3": return 99
- case "f4": return 118
- case "f5": return 96
- case "f6": return 97
- case "f7": return 98
- case "f8": return 100
- case "f9": return 101
- case "f10": return 109
- case "f11": return 103
- case "f12": return 111
- default: return 49 // space as fallback
- }
- }
-
- // MARK: - Result Builders
-
- private func screenshotResult(id: String, base64: String) -> [String: Any] {
- return [
- "type": "tool_result",
- "tool_use_id": id,
- "content": [
- [
- "type": "image",
- "source": [
- "type": "base64",
- "media_type": "image/jpeg",
- "data": base64
- ]
- ] as [String: Any]
- ]
- ]
- }
-
- private func errorResult(id: String, message: String) -> [String: Any] {
- return [
- "type": "tool_result",
- "tool_use_id": id,
- "content": message,
- "is_error": true
- ]
- }
-
- private func inputMetadata(_ input: [String: AnyCodable]) -> [String: String] {
- var meta: [String: String] = [:]
- if let action = input["action"]?.stringValue { meta["action"] = action }
- if let text = input["text"]?.stringValue { meta["text"] = text }
- if let coord = input["coordinate"]?.value as? [Any] {
- meta["coordinate"] = "\(coord)"
- }
- return meta
- }
-
- // MARK: - Response Serialization
-
- private func responseToRawContent(_ response: AnthropicClient.APIResponse) throws -> [[String: Any]] {
- var content: [[String: Any]] = []
- for block in response.content {
- switch block {
- case .text(let text):
- content.append(["type": "text", "text": text])
- case .toolUse(let id, let name, let input):
- var inputDict: [String: Any] = [:]
- for (key, val) in input {
- inputDict[key] = val.value
- }
- content.append([
- "type": "tool_use",
- "id": id,
- "name": name,
- "input": inputDict
- ])
- default:
- break
- }
- }
- return content
- }
-}
diff --git a/libs/lume/src/Unattended/AnthropicClient.swift b/libs/lume/src/Unattended/AnthropicClient.swift
deleted file mode 100644
index 7296b110a7..0000000000
--- a/libs/lume/src/Unattended/AnthropicClient.swift
+++ /dev/null
@@ -1,240 +0,0 @@
-import Foundation
-import CoreGraphics
-import ImageIO
-
-/// Minimal Anthropic API client for computer-use agent loop
-struct AnthropicClient {
- let apiKey: String
- let model: String
- let betaFlag: String = "computer-use-2025-11-24"
-
- private let baseURL = URL(string: "https://api.anthropic.com/v1/messages")!
-
- struct Message: Codable {
- let role: String
- let content: [ContentBlock]
- }
-
- enum ContentBlock: Codable {
- case text(String)
- case image(base64: String, mediaType: String)
- case toolUse(id: String, name: String, input: [String: AnyCodable])
- case toolResult(toolUseId: String, content: [ContentBlock], isError: Bool)
-
- enum CodingKeys: String, CodingKey {
- case type, text, source, id, name, input
- case toolUseId = "tool_use_id"
- case content, isError = "is_error"
- }
-
- func encode(to encoder: Encoder) throws {
- var container = encoder.container(keyedBy: CodingKeys.self)
- switch self {
- case .text(let text):
- try container.encode("text", forKey: .type)
- try container.encode(text, forKey: .text)
- case .image(let base64, let mediaType):
- try container.encode("image", forKey: .type)
- let source: [String: String] = [
- "type": "base64",
- "media_type": mediaType,
- "data": base64
- ]
- try container.encode(source, forKey: .source)
- case .toolUse(let id, let name, let input):
- try container.encode("tool_use", forKey: .type)
- try container.encode(id, forKey: .id)
- try container.encode(name, forKey: .name)
- try container.encode(input, forKey: .input)
- case .toolResult(let toolUseId, let content, let isError):
- try container.encode("tool_result", forKey: .type)
- try container.encode(toolUseId, forKey: .toolUseId)
- try container.encode(content, forKey: .content)
- if isError {
- try container.encode(true, forKey: .isError)
- }
- }
- }
-
- init(from decoder: Decoder) throws {
- let container = try decoder.container(keyedBy: CodingKeys.self)
- let type = try container.decode(String.self, forKey: .type)
- switch type {
- case "text":
- let text = try container.decode(String.self, forKey: .text)
- self = .text(text)
- case "tool_use":
- let id = try container.decode(String.self, forKey: .id)
- let name = try container.decode(String.self, forKey: .name)
- let input = try container.decode([String: AnyCodable].self, forKey: .input)
- self = .toolUse(id: id, name: name, input: input)
- default:
- self = .text("[unsupported block type: \(type)]")
- }
- }
- }
-
- struct APIResponse: Decodable {
- let id: String
- let content: [ContentBlock]
- let stopReason: String?
-
- enum CodingKeys: String, CodingKey {
- case id, content
- case stopReason = "stop_reason"
- }
- }
-
- struct APIError: Decodable {
- let type: String
- let error: ErrorDetail
-
- struct ErrorDetail: Decodable {
- let type: String
- let message: String
- }
- }
-
- @MainActor
- func sendMessage(
- systemPrompt: String,
- messages: [[String: Any]],
- displayWidth: Int,
- displayHeight: Int
- ) async throws -> APIResponse {
- var request = URLRequest(url: baseURL)
- request.httpMethod = "POST"
- request.setValue("application/json", forHTTPHeaderField: "content-type")
- request.setValue(apiKey, forHTTPHeaderField: "x-api-key")
- request.setValue("2023-06-01", forHTTPHeaderField: "anthropic-version")
- request.setValue(betaFlag, forHTTPHeaderField: "anthropic-beta")
- request.timeoutInterval = 120
-
- let body: [String: Any] = [
- "model": model,
- "max_tokens": 4096,
- "system": systemPrompt,
- "tools": [
- [
- "type": "computer_20251124",
- "name": "computer",
- "display_width_px": displayWidth,
- "display_height_px": displayHeight
- ] as [String: Any]
- ],
- "messages": messages
- ]
-
- request.httpBody = try JSONSerialization.data(withJSONObject: body)
-
- let (data, response) = try await URLSession.shared.data(for: request)
-
- guard let httpResponse = response as? HTTPURLResponse else {
- throw AnthropicError.invalidResponse
- }
-
- if httpResponse.statusCode != 200 {
- if let errorResponse = try? JSONDecoder().decode(APIError.self, from: data) {
- throw AnthropicError.apiError(
- status: httpResponse.statusCode,
- message: errorResponse.error.message
- )
- }
- let body = String(data: data, encoding: .utf8) ?? "unknown"
- throw AnthropicError.apiError(status: httpResponse.statusCode, message: body)
- }
-
- return try JSONDecoder().decode(APIResponse.self, from: data)
- }
-}
-
-enum AnthropicError: Error, LocalizedError {
- case invalidResponse
- case apiError(status: Int, message: String)
- case maxIterationsReached
-
- var errorDescription: String? {
- switch self {
- case .invalidResponse:
- return "Invalid response from Anthropic API"
- case .apiError(let status, let message):
- return "Anthropic API error (\(status)): \(message)"
- case .maxIterationsReached:
- return "Agent reached maximum iteration limit"
- }
- }
-}
-
-/// Type-erased Codable wrapper for JSON values
-struct AnyCodable: Codable {
- let value: Any
-
- init(_ value: Any) {
- self.value = value
- }
-
- init(from decoder: Decoder) throws {
- let container = try decoder.singleValueContainer()
- if let intVal = try? container.decode(Int.self) {
- value = intVal
- } else if let doubleVal = try? container.decode(Double.self) {
- value = doubleVal
- } else if let stringVal = try? container.decode(String.self) {
- value = stringVal
- } else if let boolVal = try? container.decode(Bool.self) {
- value = boolVal
- } else if let arrayVal = try? container.decode([AnyCodable].self) {
- value = arrayVal.map { $0.value }
- } else if let dictVal = try? container.decode([String: AnyCodable].self) {
- value = dictVal.mapValues { $0.value }
- } else {
- value = NSNull()
- }
- }
-
- func encode(to encoder: Encoder) throws {
- var container = encoder.singleValueContainer()
- switch value {
- case let val as Int: try container.encode(val)
- case let val as Double: try container.encode(val)
- case let val as String: try container.encode(val)
- case let val as Bool: try container.encode(val)
- case is NSNull: try container.encodeNil()
- default: try container.encodeNil()
- }
- }
-
- var stringValue: String? { value as? String }
- var intValue: Int? { value as? Int }
- var doubleValue: Double? { value as? Double }
- var arrayValue: [Any]? { value as? [Any] }
-}
-
-// MARK: - CGImage to base64
-
-extension CGImage {
- func pngBase64() -> String? {
- let mutableData = NSMutableData()
- guard let destination = CGImageDestinationCreateWithData(mutableData, "public.png" as CFString, 1, nil) else {
- return nil
- }
- CGImageDestinationAddImage(destination, self, nil)
- guard CGImageDestinationFinalize(destination) else {
- return nil
- }
- return (mutableData as Data).base64EncodedString()
- }
-
- func jpegBase64(quality: Double = 0.8) -> String? {
- let mutableData = NSMutableData()
- guard let destination = CGImageDestinationCreateWithData(mutableData, "public.jpeg" as CFString, 1, nil) else {
- return nil
- }
- let options: [CFString: Any] = [kCGImageDestinationLossyCompressionQuality: quality]
- CGImageDestinationAddImage(destination, self, options as CFDictionary)
- guard CGImageDestinationFinalize(destination) else {
- return nil
- }
- return (mutableData as Data).base64EncodedString()
- }
-}
diff --git a/libs/lume/src/Unattended/BootCommandParser.swift b/libs/lume/src/Unattended/BootCommandParser.swift
deleted file mode 100644
index 64d087b0c4..0000000000
--- a/libs/lume/src/Unattended/BootCommandParser.swift
+++ /dev/null
@@ -1,430 +0,0 @@
-import Foundation
-
-/// Represents a parsed boot command for VNC automation
-enum BootCommand: Sendable {
- /// Wait for text to appear on screen (OCR-based)
- case waitForText(String, timeout: TimeInterval)
-
- /// Click on text detected on screen
- case clickText(String)
-
- /// Click on text with X and/or Y offset (positive = right/down, negative = left/up)
- case clickTextWithOffset(String, xOffset: Int, yOffset: Int)
-
- /// Click on text with index to select specific match (0 = first, -1 = last)
- case clickTextWithIndex(String, index: Int, xOffset: Int, yOffset: Int)
-
- /// Click at specific screen coordinates
- case clickAt(x: Int, y: Int)
-
- /// Type text string with optional delay between characters (in milliseconds)
- case typeText(String, delay: Int?)
-
- /// Press a special key
- case keyPress(SpecialKey)
-
- /// Press a hotkey combination with a special key (e.g., cmd+space, ctrl+alt+delete)
- case hotkey(modifiers: [SpecialKey], key: SpecialKey)
-
- /// Press a hotkey combination with a character key (e.g., cmd+q, cmd+c, cmd+v)
- case hotkeyChar(modifiers: [SpecialKey], char: Character)
-
- /// Wait for a specified duration
- case delay(TimeInterval)
-}
-
-/// Special keys for keyboard automation
-enum SpecialKey: String, Sendable {
- case enter
- case tab
- case escape = "esc"
- case leftSuper // Command key
- case leftShift
- case leftAlt // Option key
- case leftCtrl
- case backspace
- case delete
- case up
- case down
- case left
- case right
- case space
- case f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12
-}
-
-/// Parser for boot command DSL
-struct BootCommandParser {
- /// Default timeout for wait commands in seconds
- static let defaultWaitTimeout: TimeInterval = 120
-
- /// Parse a single command string into a BootCommand
- func parse(_ commandString: String) throws -> BootCommand {
- let trimmed = commandString.trimmingCharacters(in: .whitespaces)
-
- // Check for <wait 'text'> pattern
- if let match = try parseWaitCommand(trimmed) {
- return match
- }
-
- // Check for <click 'text', offset=N> pattern (with Y offset)
- if let match = try parseClickTextWithOffsetCommand(trimmed) {
- return match
- }
-
- // Check for <click 'text'> pattern
- if let match = try parseClickTextCommand(trimmed) {
- return match
- }
-
- // Check for <click_at x,y> pattern
- if let match = try parseClickAtCommand(trimmed) {
- return match
- }
-
- // Check for <type 'text'> pattern
- if let match = try parseTypeCommand(trimmed) {
- return match
- }
-
- // Check for <delay N> pattern
- if let match = try parseDelayCommand(trimmed) {
- return match
- }
-
- // Check for hotkey combination (e.g., <cmd+space>)
- if let match = parseHotkeyCommand(trimmed) {
- return match
- }
-
- // Check for special key commands
- if let match = parseSpecialKeyCommand(trimmed) {
- return match
- }
-
- throw BootCommandParserError.unknownCommand(trimmed)
- }
-
- /// Parse multiple command strings
- func parseAll(_ commands: [String]) throws -> [BootCommand] {
- try commands.map { try parse($0) }
- }
-
- // MARK: - Private parsing methods
-
- private func parseWaitCommand(_ input: String) throws -> BootCommand? {
- // Match <wait 'text'> or <wait "text"> with optional timeout
- let pattern = #"^<wait\s+['"](.+?)['"]\s*(?:,\s*timeout\s*=\s*(\d+))?\s*>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input))
- else {
- return nil
- }
-
- guard let textRange = Range(match.range(at: 1), in: input) else {
- return nil
- }
-
- let text = String(input[textRange])
- var timeout = Self.defaultWaitTimeout
-
- if match.numberOfRanges > 2,
- let timeoutRange = Range(match.range(at: 2), in: input),
- let timeoutValue = TimeInterval(input[timeoutRange]) {
- timeout = timeoutValue
- }
-
- return .waitForText(text, timeout: timeout)
- }
-
- private func parseClickTextWithOffsetCommand(_ input: String) throws -> BootCommand? {
- // Match <click 'text', xoffset=N, yoffset=N, index=N> or combinations
- // Also supports just xoffset, yoffset, or index individually
- // index: 0 = first match, 1 = second match, -1 = last match, -2 = second to last, etc.
- var xOffset = 0
- var yOffset = 0
- var index: Int?
- var text: String?
-
- // Check if this looks like a click command with parameters
- let hasParams = input.contains(",")
- guard hasParams else { return nil }
-
- // Extract text and parameters using a more flexible approach
- // Pattern: <click 'text', param=value, param=value, ...>
- let textPattern = #"^<click\s+['"](.+?)['"]"#
- guard let textRegex = try? NSRegularExpression(pattern: textPattern, options: []),
- let textMatch = textRegex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let textRange = Range(textMatch.range(at: 1), in: input)
- else {
- return nil
- }
- text = String(input[textRange])
-
- // Extract xoffset if present
- let xoffsetPattern = #"xoffset\s*=\s*(-?\d+)"#
- if let xRegex = try? NSRegularExpression(pattern: xoffsetPattern, options: []),
- let xMatch = xRegex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let xRange = Range(xMatch.range(at: 1), in: input),
- let x = Int(input[xRange]) {
- xOffset = x
- }
-
- // Extract yoffset if present
- let yoffsetPattern = #"yoffset\s*=\s*(-?\d+)"#
- if let yRegex = try? NSRegularExpression(pattern: yoffsetPattern, options: []),
- let yMatch = yRegex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let yRange = Range(yMatch.range(at: 1), in: input),
- let y = Int(input[yRange]) {
- yOffset = y
- }
-
- // Extract index if present
- let indexPattern = #"index\s*=\s*(-?\d+)"#
- if let indexRegex = try? NSRegularExpression(pattern: indexPattern, options: []),
- let indexMatch = indexRegex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let indexRange = Range(indexMatch.range(at: 1), in: input),
- let idx = Int(input[indexRange]) {
- index = idx
- }
-
- // Legacy format: <click 'text', offset=N> (Y offset only, no index)
- if index == nil && xOffset == 0 && yOffset == 0 {
- let legacyPattern = #"offset\s*=\s*(-?\d+)"#
- if let legacyRegex = try? NSRegularExpression(pattern: legacyPattern, options: []),
- let legacyMatch = legacyRegex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let offsetRange = Range(legacyMatch.range(at: 1), in: input),
- let offset = Int(input[offsetRange]) {
- yOffset = offset
- }
- }
-
- // Return appropriate command type
- if let idx = index {
- return .clickTextWithIndex(text!, index: idx, xOffset: xOffset, yOffset: yOffset)
- } else if xOffset != 0 || yOffset != 0 {
- return .clickTextWithOffset(text!, xOffset: xOffset, yOffset: yOffset)
- }
-
- return nil
- }
-
- private func parseClickTextCommand(_ input: String) throws -> BootCommand? {
- // Match <click 'text'> or <click "text">
- let pattern = #"^<click\s+['"](.+?)['"]\s*>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let textRange = Range(match.range(at: 1), in: input)
- else {
- return nil
- }
-
- let text = String(input[textRange])
- return .clickText(text)
- }
-
- private func parseClickAtCommand(_ input: String) throws -> BootCommand? {
- // Match <click_at x,y>
- let pattern = #"^<click_at\s+(\d+)\s*,\s*(\d+)\s*>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let xRange = Range(match.range(at: 1), in: input),
- let yRange = Range(match.range(at: 2), in: input),
- let x = Int(input[xRange]),
- let y = Int(input[yRange])
- else {
- return nil
- }
-
- return .clickAt(x: x, y: y)
- }
-
- private func parseTypeCommand(_ input: String) throws -> BootCommand? {
- // Match <type 'text'> or <type "text"> with optional delay parameter
- // Examples: <type 'hello'>, <type 'hello', delay=100>
- let pattern = #"^<type\s+['"](.*)['"](?:\s*,\s*delay\s*=\s*(\d+))?\s*>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let textRange = Range(match.range(at: 1), in: input)
- else {
- return nil
- }
-
- let text = String(input[textRange])
- var delay: Int? = nil
-
- if match.numberOfRanges > 2,
- let delayRange = Range(match.range(at: 2), in: input),
- let delayValue = Int(input[delayRange]) {
- delay = delayValue
- }
-
- return .typeText(text, delay: delay)
- }
-
- private func parseDelayCommand(_ input: String) throws -> BootCommand? {
- // Match <delay N> where N is seconds (can be decimal)
- let pattern = #"^<delay\s+([\d.]+)\s*>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let durationRange = Range(match.range(at: 1), in: input),
- let duration = TimeInterval(input[durationRange])
- else {
- return nil
- }
-
- return .delay(duration)
- }
-
- private func parseHotkeyCommand(_ input: String) -> BootCommand? {
- // Match <key1+key2+key3> for hotkey combinations
- // Examples: <cmd+space>, <ctrl+alt+delete>, <shift+cmd+3>
- let pattern = #"^<([\w+]+)>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let contentRange = Range(match.range(at: 1), in: input)
- else {
- return nil
- }
-
- let content = String(input[contentRange])
-
- // Check if it contains a + (indicating a hotkey combo)
- guard content.contains("+") else {
- return nil
- }
-
- let parts = content.split(separator: "+").map { String($0).lowercased() }
- guard parts.count >= 2 else {
- return nil
- }
-
- // Map key names to SpecialKey enum
- let keyMap: [String: SpecialKey] = [
- "enter": .enter,
- "return": .enter,
- "tab": .tab,
- "esc": .escape,
- "escape": .escape,
- "command": .leftSuper,
- "cmd": .leftSuper,
- "super": .leftSuper,
- "shift": .leftShift,
- "option": .leftAlt,
- "alt": .leftAlt,
- "ctrl": .leftCtrl,
- "control": .leftCtrl,
- "backspace": .backspace,
- "delete": .delete,
- "up": .up,
- "down": .down,
- "left": .left,
- "right": .right,
- "space": .space,
- "f1": .f1, "f2": .f2, "f3": .f3, "f4": .f4,
- "f5": .f5, "f6": .f6, "f7": .f7, "f8": .f8,
- "f9": .f9, "f10": .f10, "f11": .f11, "f12": .f12
- ]
-
- // Modifier keys
- let modifierKeys: Swift.Set<String> = ["command", "cmd", "super", "shift", "option", "alt", "ctrl", "control"]
-
- // Parse all parts - last non-modifier is the main key, rest are modifiers
- var modifiers: [SpecialKey] = []
- var mainSpecialKey: SpecialKey?
- var mainChar: Character?
-
- for (index, part) in parts.enumerated() {
- // If it's the last part, it's the main key
- if index == parts.count - 1 {
- // Check if it's a special key
- if let key = keyMap[part] {
- mainSpecialKey = key
- } else if part.count == 1, let char = part.first {
- // Single character key (a-z, 0-9, etc.)
- mainChar = char
- } else {
- return nil // Unknown key
- }
- } else {
- // Must be a modifier
- guard let key = keyMap[part], modifierKeys.contains(part) else {
- return nil // Unknown or non-modifier key in modifier position
- }
- modifiers.append(key)
- }
- }
-
- // Return appropriate hotkey type
- if let specialKey = mainSpecialKey {
- return .hotkey(modifiers: modifiers, key: specialKey)
- } else if let char = mainChar {
- return .hotkeyChar(modifiers: modifiers, char: char)
- }
-
- return nil
- }
-
- private func parseSpecialKeyCommand(_ input: String) -> BootCommand? {
- // Match <keyname> for special keys
- let pattern = #"^<(\w+)>$"#
- guard let regex = try? NSRegularExpression(pattern: pattern, options: []),
- let match = regex.firstMatch(in: input, range: NSRange(input.startIndex..., in: input)),
- let keyRange = Range(match.range(at: 1), in: input)
- else {
- return nil
- }
-
- let keyName = String(input[keyRange]).lowercased()
-
- // Map key names to SpecialKey enum
- let keyMap: [String: SpecialKey] = [
- "enter": .enter,
- "return": .enter,
- "tab": .tab,
- "esc": .escape,
- "escape": .escape,
- "leftsuper": .leftSuper,
- "command": .leftSuper,
- "cmd": .leftSuper,
- "leftshift": .leftShift,
- "shift": .leftShift,
- "leftalt": .leftAlt,
- "option": .leftAlt,
- "alt": .leftAlt,
- "leftctrl": .leftCtrl,
- "ctrl": .leftCtrl,
- "control": .leftCtrl,
- "backspace": .backspace,
- "delete": .delete,
- "up": .up,
- "down": .down,
- "left": .left,
- "right": .right,
- "space": .space,
- "f1": .f1, "f2": .f2, "f3": .f3, "f4": .f4,
- "f5": .f5, "f6": .f6, "f7": .f7, "f8": .f8,
- "f9": .f9, "f10": .f10, "f11": .f11, "f12": .f12
- ]
-
- guard let key = keyMap[keyName] else {
- return nil
- }
-
- return .keyPress(key)
- }
-}
-
-/// Errors from boot command parsing
-enum BootCommandParserError: Error, LocalizedError {
- case unknownCommand(String)
- case invalidSyntax(String)
-
- var errorDescription: String? {
- switch self {
- case .unknownCommand(let cmd):
- return "Unknown boot command: \(cmd)"
- case .invalidSyntax(let msg):
- return "Invalid boot command syntax: \(msg)"
- }
- }
-}
diff --git a/libs/lume/src/Unattended/MacOSOfflineSetupPatcher.swift b/libs/lume/src/Unattended/MacOSOfflineSetupPatcher.swift
new file mode 100644
index 0000000000..cb7c2c27cc
--- /dev/null
+++ b/libs/lume/src/Unattended/MacOSOfflineSetupPatcher.swift
@@ -0,0 +1,597 @@
+import CryptoKit
+import Darwin
+import Foundation
+
+/// Prepares a freshly-installed macOS VM without driving Setup Assistant over VNC.
+///
+/// The patcher edits the guest Data volume offline:
+/// - creates/updates the local `lume` admin user
+/// - marks Setup Assistant complete
+/// - enables autologin and SSH
+/// - disables screensaver lock and sleep timers
+///
+/// Some macOS system plists are edited in place because launchd/loginwindow may ignore
+/// atomically-replaced files on the next boot even when their contents look correct.
+@MainActor
+final class MacOSOfflineSetupPatcher {
+ private let fileManager: FileManager
+
+ init(fileManager: FileManager = .default) {
+ self.fileManager = fileManager
+ }
+
+ func patch(vm: VM) throws {
+ let diskPath = vm.vmDirContext.diskPath.path
+ try ensureStopped(vm: vm)
+
+ Logger.info("Starting offline macOS unattended setup", metadata: [
+ "name": vm.name,
+ "diskPath": diskPath
+ ])
+
+ let mount = try attachDataVolume(diskPath: diskPath)
+ defer {
+ cleanup(mount: mount)
+ }
+
+ try patchMountedDataVolume(at: mount.mountPoint)
+
+ Logger.info("Offline macOS unattended setup completed", metadata: [
+ "name": vm.name,
+ "mountPoint": mount.mountPoint.path
+ ])
+ }
+
+ private func ensureStopped(vm: VM) throws {
+ let handle = try FileHandle(forReadingFrom: vm.vmDirContext.dir.configPath.url)
+ defer { try? handle.close() }
+
+ guard flock(handle.fileDescriptor, LOCK_EX | LOCK_NB) == 0 else {
+ throw VMError.alreadyRunning(vm.name)
+ }
+ flock(handle.fileDescriptor, LOCK_UN)
+ }
+
+ // MARK: - Disk Attach/Mount
+
+ private struct MountedDataVolume {
+ let wholeDisk: String
+ let mountPoint: URL
+ }
+
+ private func attachDataVolume(diskPath: String) throws -> MountedDataVolume {
+ let attachOutput = try run("/usr/bin/hdiutil", ["attach", "-readwrite", "-nomount", diskPath])
+ guard let wholeDisk = parseWholeDisk(from: attachOutput) else {
+ throw UnattendedError.commandExecutionFailed("Could not determine attached disk from hdiutil output")
+ }
+
+ do {
+ let container = try apfsContainer(forMainPartitionOf: wholeDisk)
+ let dataDevice = try dataVolumeDevice(inContainer: container)
+ _ = try run("/usr/sbin/diskutil", ["mount", dataDevice])
+ let mountPoint = try mountPoint(forDevice: dataDevice)
+ return MountedDataVolume(wholeDisk: wholeDisk, mountPoint: mountPoint)
+ } catch {
+ cleanup(mount: MountedDataVolume(wholeDisk: wholeDisk, mountPoint: URL(fileURLWithPath: "/Volumes/Data")))
+ throw error
+ }
+ }
+
+ private func parseWholeDisk(from hdiutilOutput: String) -> String? {
+ for line in hdiutilOutput.split(separator: "\n") {
+ let fields = line.split(whereSeparator: { $0 == " " || $0 == "\t" })
+ guard fields.count >= 2 else { continue }
+ if fields[1] == "GUID_partition_scheme" {
+ return fields[0].replacingOccurrences(of: "/dev/", with: "")
+ }
+ }
+ return nil
+ }
+
+ private func apfsContainer(forMainPartitionOf wholeDisk: String) throws -> String {
+ let plist = try runPlist("/usr/sbin/diskutil", ["info", "-plist", "\(wholeDisk)s2"])
+ guard let container = plist["APFSContainerReference"] as? String else {
+ throw UnattendedError.commandExecutionFailed("Could not find APFS container for \(wholeDisk)s2")
+ }
+ return container
+ }
+
+ private func dataVolumeDevice(inContainer container: String) throws -> String {
+ let plist = try runPlist("/usr/sbin/diskutil", ["apfs", "list", "-plist", container])
+ guard let containers = plist["Containers"] as? [[String: Any]] else {
+ throw UnattendedError.commandExecutionFailed("Could not parse APFS container list for \(container)")
+ }
+
+ for container in containers {
+ guard let volumes = container["Volumes"] as? [[String: Any]] else { continue }
+ for volume in volumes {
+ let roles = volume["Roles"] as? [String] ?? []
+ if roles.contains("Data"), let device = volume["DeviceIdentifier"] as? String {
+ return device
+ }
+ }
+ }
+
+ throw UnattendedError.commandExecutionFailed("Could not find APFS Data volume in \(container)")
+ }
+
+ private func mountPoint(forDevice device: String) throws -> URL {
+ let plist = try runPlist("/usr/sbin/diskutil", ["info", "-plist", device])
+ guard let mountPoint = plist["MountPoint"] as? String, !mountPoint.isEmpty else {
+ throw UnattendedError.commandExecutionFailed("Could not determine mount point for \(device)")
+ }
+ return URL(fileURLWithPath: mountPoint, isDirectory: true)
+ }
+
+ private func cleanup(mount: MountedDataVolume) {
+ _ = try? run("/usr/sbin/diskutil", ["unmount", mount.mountPoint.path])
+ _ = try? run("/usr/bin/hdiutil", ["detach", "/dev/\(mount.wholeDisk)"])
+ }
+
+ // MARK: - Guest Patching
+
+ private func patchMountedDataVolume(at mountPoint: URL) throws {
+ let user = GuestUser(username: "lume", realName: "lume", password: "lume", uid: "501", gid: "20")
+
+ try ensureDirectoryExecutable(mountPoint.appendingPath("private/var/db/dslocal/nodes/Default"))
+ try ensureDirectoryExecutable(mountPoint.appendingPath("private/var/db/dslocal/nodes/Default/users"))
+
+ let userUUID = try createOrUpdateUser(user, mountPoint: mountPoint)
+ try addUserToGroups(user, uuid: userUUID, mountPoint: mountPoint)
+ try createHomeAndUserPreferences(user, mountPoint: mountPoint)
+ try markSetupAssistantComplete(user, mountPoint: mountPoint)
+ try configureAutologin(user, mountPoint: mountPoint)
+ try enableSSH(mountPoint: mountPoint)
+ try configurePowerAndLockSettings(mountPoint: mountPoint)
+ }
+
+ private struct GuestUser {
+ let username: String
+ let realName: String
+ let password: String
+ let uid: String
+ let gid: String
+
+ var homeDirectory: String { "/Users/\(username)" }
+ }
+
+ private func createOrUpdateUser(_ user: GuestUser, mountPoint: URL) throws -> String {
+ let usersDir = mountPoint.appendingPath("private/var/db/dslocal/nodes/Default/users")
+ let userPlist = usersDir.appendingPathComponent("\(user.username).plist")
+
+ var record: [String: Any]
+ let userUUID: String
+
+ if fileManager.fileExists(atPath: userPlist.path) {
+ record = try readPlist(userPlist)
+ userUUID = (record["generateduid"] as? [String])?.first ?? UUID().uuidString.uppercased()
+ } else {
+ try ensureUIDAvailable(user.uid, usersDir: usersDir)
+ userUUID = UUID().uuidString.uppercased()
+ record = [
+ "_writers_UserCertificate": [user.username],
+ "_writers_hint": [user.username],
+ "_writers_jpegphoto": [user.username],
+ "_writers_passwd": [user.username],
+ "_writers_picture": [user.username],
+ "_writers_realname": [user.username],
+ "record_daemon_version": ["9040000"],
+ "unlockOptions": ["0"]
+ ]
+ }
+
+ record["ShadowHashData"] = [try shadowHashData(for: user.password)]
+ record["authentication_authority"] = [";ShadowHash;HASHLIST:<SALTED-SHA512-PBKDF2>"]
+ record["generateduid"] = [userUUID]
+ record["gid"] = [user.gid]
+ record["home"] = [user.homeDirectory]
+ record["name"] = [user.username]
+ record["passwd"] = ["********"]
+ record["realname"] = [user.realName]
+ record["shell"] = ["/bin/zsh"]
+ record["uid"] = [user.uid]
+
+ try writePlist(record, to: userPlist, mode: 0o600, preserveExisting: fileManager.fileExists(atPath: userPlist.path))
+ return userUUID
+ }
+
+ private func ensureUIDAvailable(_ uid: String, usersDir: URL) throws {
+ guard let enumerator = fileManager.enumerator(at: usersDir, includingPropertiesForKeys: nil) else {
+ throw UnattendedError.commandExecutionFailed("Could not enumerate \(usersDir.path)")
+ }
+
+ for case let url as URL in enumerator where url.pathExtension == "plist" {
+ guard let record = try? readPlist(url),
+ let existingUID = (record["uid"] as? [String])?.first,
+ existingUID == uid else {
+ continue
+ }
+ throw UnattendedError.commandExecutionFailed("UID \(uid) already exists in \(url.lastPathComponent)")
+ }
+ }
+
+ private func addUserToGroups(_ user: GuestUser, uuid: String, mountPoint: URL) throws {
+ let groupsDir = mountPoint.appendingPath("private/var/db/dslocal/nodes/Default/groups")
+ for groupName in ["admin", "staff"] {
+ let groupPlist = groupsDir.appendingPathComponent("\(groupName).plist")
+ guard fileManager.fileExists(atPath: groupPlist.path) else { continue }
+
+ var group = try readPlist(groupPlist)
+ var users = group["users"] as? [String] ?? []
+ if !users.contains(user.username) {
+ users.append(user.username)
+ }
+ group["users"] = users
+
+ var members = group["groupmembers"] as? [String] ?? []
+ if !members.contains(uuid) {
+ members.append(uuid)
+ }
+ group["groupmembers"] = members
+
+ try writePlist(group, to: groupPlist, mode: permissions(of: groupPlist) ?? 0o644, preserveExisting: true)
+ }
+ }
+
+ private func createHomeAndUserPreferences(_ user: GuestUser, mountPoint: URL) throws {
+ let home = mountPoint.appendingPath(user.homeDirectory.trimmingCharacters(in: CharacterSet(charactersIn: "/")))
+ for relativePath in [
+ "Library/Preferences",
+ "Library/Preferences/ByHost",
+ "Desktop",
+ "Documents",
+ "Downloads"
+ ] {
+ try createDirectory(home.appendingPath(relativePath), mode: 0o755)
+ }
+ try setPermissions(home, 0o755)
+
+ let textEncoding = String(format: "0x%X:0x0:0x0\n", Int(user.uid) ?? 501)
+ try writeData(Data(textEncoding.utf8), to: home.appendingPath(".CFUserTextEncoding"), mode: 0o644)
+
+ var globalPreferences = try readPlistIfPresent(home.appendingPath("Library/Preferences/.GlobalPreferences.plist"))
+ globalPreferences["AppleKeyboardUIMode"] = 3
+ try writePlist(globalPreferences, to: home.appendingPath("Library/Preferences/.GlobalPreferences.plist"))
+
+ var screensaver = try readPlistIfPresent(home.appendingPath("Library/Preferences/com.apple.screensaver.plist"))
+ screensaver["askForPassword"] = 0
+ screensaver["askForPasswordDelay"] = 0
+ screensaver["idleTime"] = 0
+ try writePlist(screensaver, to: home.appendingPath("Library/Preferences/com.apple.screensaver.plist"))
+
+ for uuid in byHostUUIDs(in: home.appendingPath("Library/Preferences/ByHost")) {
+ try writePlist(
+ ["idleTime": 0],
+ to: home.appendingPath("Library/Preferences/ByHost/com.apple.screensaver.\(uuid).plist")
+ )
+ }
+ }
+
+ private func markSetupAssistantComplete(_ user: GuestUser, mountPoint: URL) throws {
+ let marker = mountPoint.appendingPath("private/var/db/.AppleSetupDone")
+ try createDirectory(marker.deletingLastPathComponent(), mode: 0o755)
+ if !fileManager.fileExists(atPath: marker.path) {
+ fileManager.createFile(atPath: marker.path, contents: Data())
+ }
+ try setPermissions(marker, 0o644)
+
+ let setupPreferences: [String: Any] = [
+ "DidSeeCloudSetup": true,
+ "DidSeePrivacy": true,
+ "DidSeeSiriSetup": true,
+ "DidSeeTouchIDSetup": true,
+ "DidSeeTrueToneSetup": true,
+ "GestureMovieSeen": "none",
+ "LastSeenBuddyBuildVersion": "25F84",
+ "LastSeenCloudProductVersion": "26.5.2"
+ ]
+
+ let home = mountPoint.appendingPath(user.homeDirectory.trimmingCharacters(in: CharacterSet(charactersIn: "/")))
+ try writePlist(setupPreferences, to: home.appendingPath("Library/Preferences/com.apple.SetupAssistant.plist"))
+ let systemSetupAssistant = mountPoint.appendingPath("Library/Preferences/com.apple.SetupAssistant.plist")
+ try writePlist(
+ setupPreferences,
+ to: systemSetupAssistant,
+ preserveExisting: fileManager.fileExists(atPath: systemSetupAssistant.path)
+ )
+ }
+
+ private func configureAutologin(_ user: GuestUser, mountPoint: URL) throws {
+ let loginwindowPath = mountPoint.appendingPath("Library/Preferences/com.apple.loginwindow.plist")
+ var loginwindow = try readPlistIfPresent(loginwindowPath)
+
+ var accountInfo = loginwindow["AccountInfo"] as? [String: Any] ?? [:]
+ var firstLogins = accountInfo["FirstLogins"] as? [String: Any] ?? [:]
+ firstLogins[user.username] = 1
+ accountInfo["FirstLogins"] = firstLogins
+ accountInfo["MaximumUsers"] = max(accountInfo["MaximumUsers"] as? Int ?? 1, 1)
+ accountInfo["OnConsole"] = accountInfo["OnConsole"] as? [String: Any] ?? [:]
+
+ loginwindow["AccountInfo"] = accountInfo
+ loginwindow["autoLoginUser"] = user.username
+ loginwindow["GuestEnabled"] = false
+ loginwindow["lastUser"] = "loggedIn"
+ loginwindow["lastUserName"] = user.username
+ loginwindow["RecentUsers"] = [user.username]
+ loginwindow["SHOWFULLNAME"] = false
+
+ try writePlist(
+ loginwindow,
+ to: loginwindowPath,
+ preserveExisting: fileManager.fileExists(atPath: loginwindowPath.path)
+ )
+
+ let kcpassword = mountPoint.appendingPath("private/etc/kcpassword")
+ try createDirectory(kcpassword.deletingLastPathComponent(), mode: 0o755)
+ try writeData(kcpasswordData(for: user.password), to: kcpassword, mode: 0o600, preserveExisting: fileManager.fileExists(atPath: kcpassword.path))
+ }
+
+ private func enableSSH(mountPoint: URL) throws {
+ let disabledPlist = mountPoint.appendingPath("private/var/db/com.apple.xpc.launchd/disabled.plist")
+ let launchdStateDir = disabledPlist.deletingLastPathComponent()
+ try createDirectory(launchdStateDir, mode: 0o755)
+
+ let migratedMarker = launchdStateDir.appendingPathComponent("disabled.migrated")
+ if !fileManager.fileExists(atPath: migratedMarker.path) {
+ fileManager.createFile(atPath: migratedMarker.path, contents: Data())
+ }
+ try setPermissions(migratedMarker, 0o644)
+
+ var disabled = try readPlistIfPresent(disabledPlist)
+ disabled["com.openssh.sshd"] = false
+ try writePlist(
+ disabled,
+ to: disabledPlist,
+ preserveExisting: fileManager.fileExists(atPath: disabledPlist.path),
+ format: .xml
+ )
+ }
+
+ private func configurePowerAndLockSettings(mountPoint: URL) throws {
+ let systemGlobal = mountPoint.appendingPath("Library/Preferences/.GlobalPreferences.plist")
+ var global = try readPlistIfPresent(systemGlobal)
+ global["com.apple.autologout.AutoLogOutDelay"] = 0
+ try writePlist(global, to: systemGlobal, preserveExisting: fileManager.fileExists(atPath: systemGlobal.path))
+
+ let powerManagement: [String: Any] = [
+ "AC Power": [
+ "Automatic Restart On Power Loss": true,
+ "DarkWakeBackgroundTasks": 0,
+ "Disk Sleep Timer": 0,
+ "Display Sleep Timer": 0,
+ "System Sleep Timer": 0,
+ "Wake On LAN": 1
+ ],
+ "SystemPowerSettings": [
+ "Update DarkWakeBG Setting": true
+ ]
+ ]
+ let powerPath = mountPoint.appendingPath("Library/Preferences/com.apple.PowerManagement.plist")
+ try writePlist(powerManagement, to: powerPath, preserveExisting: fileManager.fileExists(atPath: powerPath.path))
+
+ let byHostDir = mountPoint.appendingPath("Users/lume/Library/Preferences/ByHost")
+ for uuid in byHostUUIDs(in: byHostDir) {
+ let perHostPower: [String: Any] = [
+ "AC Power": [
+ "PrioritizeNetworkReachabilityOverSleep": 0,
+ "Sleep On Power Button": false,
+ "SleepServices": 0,
+ "Standby Delay": 0,
+ "Standby Enabled": 0,
+ "TCPKeepAlivePref": 1,
+ "TTYSPreventSleep": 1
+ ]
+ ]
+ let perHostPath = mountPoint.appendingPath("Library/Preferences/com.apple.PowerManagement.\(uuid).plist")
+ try writePlist(perHostPower, to: perHostPath, preserveExisting: fileManager.fileExists(atPath: perHostPath.path))
+ }
+ }
+
+ // MARK: - Hashing
+
+ private func shadowHashData(for password: String) throws -> Data {
+ var salt = Data()
+ salt.reserveCapacity(32)
+ var generator = SystemRandomNumberGenerator()
+ for _ in 0..<32 {
+ salt.append(UInt8.random(in: UInt8.min...UInt8.max, using: &generator))
+ }
+
+ let entropy = pbkdf2SHA512(password: Data(password.utf8), salt: salt, iterations: 50_000, keyLength: 128)
+ let shadow: [String: Any] = [
+ "SALTED-SHA512-PBKDF2": [
+ "entropy": entropy,
+ "iterations": 50_000,
+ "salt": salt
+ ]
+ ]
+ return try PropertyListSerialization.data(fromPropertyList: shadow, format: .binary, options: 0)
+ }
+
+ private func pbkdf2SHA512(password: Data, salt: Data, iterations: Int, keyLength: Int) -> Data {
+ let hmacKey = SymmetricKey(data: password)
+ let hashLength = 64
+ let blockCount = Int(ceil(Double(keyLength) / Double(hashLength)))
+ var derivedKey = Data()
+
+ for blockIndex in 1...blockCount {
+ var blockSalt = salt
+ blockSalt.append(UInt8((blockIndex >> 24) & 0xff))
+ blockSalt.append(UInt8((blockIndex >> 16) & 0xff))
+ blockSalt.append(UInt8((blockIndex >> 8) & 0xff))
+ blockSalt.append(UInt8(blockIndex & 0xff))
+
+ var u = Data(HMAC<SHA512>.authenticationCode(for: blockSalt, using: hmacKey))
+ var t = u
+
+ if iterations > 1 {
+ for _ in 2...iterations {
+ u = Data(HMAC<SHA512>.authenticationCode(for: u, using: hmacKey))
+ for index in 0..<t.count {
+ t[index] ^= u[index]
+ }
+ }
+ }
+
+ derivedKey.append(t)
+ }
+
+ return derivedKey.prefix(keyLength)
+ }
+
+ private func kcpasswordData(for password: String) -> Data {
+ let key: [UInt8] = [0x7d, 0x89, 0x52, 0x23, 0xd2, 0xbc, 0xdd, 0xea, 0xa3, 0xb9, 0x1f]
+ var bytes = [UInt8]()
+ for (index, byte) in password.utf8.enumerated() {
+ bytes.append(byte ^ key[index % key.count])
+ }
+
+ let remainder = bytes.count % 12
+ if remainder != 0 {
+ bytes.append(contentsOf: Array(repeating: 0, count: 12 - remainder))
+ }
+
+ return Data(bytes)
+ }
+
+ // MARK: - Plist and File Helpers
+
+ private func readPlistIfPresent(_ url: URL) throws -> [String: Any] {
+ guard fileManager.fileExists(atPath: url.path) else { return [:] }
+ return try readPlist(url)
+ }
+
+ private func readPlist(_ url: URL) throws -> [String: Any] {
+ let data = try Data(contentsOf: url)
+ guard let plist = try PropertyListSerialization.propertyList(from: data, options: [], format: nil) as? [String: Any] else {
+ throw UnattendedError.commandExecutionFailed("Invalid plist at \(url.path)")
+ }
+ return plist
+ }
+
+ private func writePlist(
+ _ plist: [String: Any],
+ to url: URL,
+ mode: Int = 0o644,
+ preserveExisting: Bool = false,
+ format: PropertyListSerialization.PropertyListFormat = .binary
+ ) throws {
+ let data = try PropertyListSerialization.data(fromPropertyList: plist, format: format, options: 0)
+ try writeData(data, to: url, mode: mode, preserveExisting: preserveExisting)
+ }
+
+ private func writeData(
+ _ data: Data,
+ to url: URL,
+ mode: Int,
+ preserveExisting: Bool = false
+ ) throws {
+ try createDirectory(url.deletingLastPathComponent(), mode: 0o755)
+ let exists = fileManager.fileExists(atPath: url.path)
+ if exists {
+ try setPermissions(url, mode | 0o200)
+ }
+
+ if preserveExisting && exists {
+ let handle = try FileHandle(forWritingTo: url)
+ defer { try? handle.close() }
+ try handle.truncate(atOffset: 0)
+ try handle.write(contentsOf: data)
+ } else {
+ try data.write(to: url)
+ }
+
+ try setPermissions(url, mode)
+ }
+
+ private func createDirectory(_ url: URL, mode: Int) throws {
+ try fileManager.createDirectory(at: url, withIntermediateDirectories: true)
+ try setPermissions(url, mode)
+ }
+
+ private func ensureDirectoryExecutable(_ url: URL) throws {
+ guard fileManager.fileExists(atPath: url.path) else { return }
+ let current = permissions(of: url) ?? 0o700
+ try setPermissions(url, current | 0o100)
+ }
+
+ private func setPermissions(_ url: URL, _ mode: Int) throws {
+ try fileManager.setAttributes([.posixPermissions: NSNumber(value: mode)], ofItemAtPath: url.path)
+ }
+
+ private func permissions(of url: URL) -> Int? {
+ guard let attrs = try? fileManager.attributesOfItem(atPath: url.path),
+ let permissions = attrs[.posixPermissions] as? NSNumber else {
+ return nil
+ }
+ return permissions.intValue
+ }
+
+ private func byHostUUIDs(in directory: URL) -> [String] {
+ guard let urls = try? fileManager.contentsOfDirectory(at: directory, includingPropertiesForKeys: nil) else {
+ return []
+ }
+
+ let pattern = #"([0-9A-Fa-f]{8}-(?:[0-9A-Fa-f]{4}-){3}[0-9A-Fa-f]{12})\.plist$"#
+ guard let regex = try? NSRegularExpression(pattern: pattern) else { return [] }
+
+ return Array(Swift.Set(urls.compactMap { url in
+ let name = url.lastPathComponent
+ let range = NSRange(name.startIndex..<name.endIndex, in: name)
+ guard let match = regex.firstMatch(in: name, range: range),
+ let uuidRange = Range(match.range(at: 1), in: name) else {
+ return nil
+ }
+ return String(name[uuidRange]).uppercased()
+ })).sorted()
+ }
+
+ // MARK: - Process Helpers
+
+ private func runPlist(_ executable: String, _ arguments: [String]) throws -> [String: Any] {
+ let output = try run(executable, arguments)
+ guard let data = output.data(using: .utf8),
+ let plist = try PropertyListSerialization.propertyList(from: data, options: [], format: nil) as? [String: Any] else {
+ throw UnattendedError.commandExecutionFailed("Command did not return a plist: \(executable) \(arguments.joined(separator: " "))")
+ }
+ return plist
+ }
+
+ @discardableResult
+ private func run(_ executable: String, _ arguments: [String]) throws -> String {
+ let process = Process()
+ process.executableURL = URL(fileURLWithPath: executable)
+ process.arguments = arguments
+
+ let output = Pipe()
+ let error = Pipe()
+ process.standardOutput = output
+ process.standardError = error
+
+ try process.run()
+ process.waitUntilExit()
+
+ let stdout = try output.fileHandleForReading.readToEnd() ?? Data()
+ let stderr = try error.fileHandleForReading.readToEnd() ?? Data()
+ let stdoutString = String(data: stdout, encoding: .utf8) ?? ""
+ let stderrString = String(data: stderr, encoding: .utf8) ?? ""
+
+ guard process.terminationStatus == 0 else {
+ throw UnattendedError.commandExecutionFailed(
+ "\(executable) \(arguments.joined(separator: " ")) failed: \(stderrString.isEmpty ? stdoutString : stderrString)"
+ )
+ }
+
+ return stdoutString
+ }
+}
+
+private extension URL {
+ func appendingPath(_ path: String) -> URL {
+ var result = self
+ for component in path.split(separator: "/") where !component.isEmpty {
+ result.appendPathComponent(String(component))
+ }
+ return result
+ }
+}
diff --git a/libs/lume/src/Unattended/OCRService.swift b/libs/lume/src/Unattended/OCRService.swift
deleted file mode 100644
index b13c9bf0dc..0000000000
--- a/libs/lume/src/Unattended/OCRService.swift
+++ /dev/null
@@ -1,192 +0,0 @@
-import Foundation
-import Vision
-import CoreGraphics
-
-/// Represents a text observation from OCR
-struct TextObservation: Sendable {
- /// The recognized text string
- let text: String
-
- /// Bounding box in normalized coordinates (0-1), origin at bottom-left (Vision convention)
- let boundingBox: CGRect
-
- /// Confidence score (0-1)
- let confidence: Float
-
- /// Convert bounding box to screen coordinates
- /// - Parameters:
- /// - screenWidth: The actual screen width in pixels
- /// - screenHeight: The actual screen height in pixels
- /// - Returns: CGRect in screen coordinates with origin at top-left
- func screenRect(screenWidth: CGFloat, screenHeight: CGFloat) -> CGRect {
- // Vision uses bottom-left origin, screen uses top-left
- // Also need to flip Y axis
- let x = boundingBox.origin.x * screenWidth
- let y = (1 - boundingBox.origin.y - boundingBox.height) * screenHeight
- let width = boundingBox.width * screenWidth
- let height = boundingBox.height * screenHeight
-
- return CGRect(x: x, y: y, width: width, height: height)
- }
-
- /// Get the center point in screen coordinates
- func centerPoint(screenWidth: CGFloat, screenHeight: CGFloat) -> CGPoint {
- let rect = screenRect(screenWidth: screenWidth, screenHeight: screenHeight)
- return CGPoint(x: rect.midX, y: rect.midY)
- }
-}
-
-/// Protocol for OCR services
-protocol OCRServiceProtocol: Sendable {
- /// Recognize text in an image
- /// - Parameter image: The CGImage to analyze
- /// - Returns: Array of text observations found in the image
- func recognizeText(in image: CGImage) async throws -> [TextObservation]
-
- /// Find text matching a pattern
- /// - Parameters:
- /// - pattern: The text to search for (supports regex)
- /// - image: The image to search in
- /// - Returns: The first matching text observation, or nil if not found
- func findText(_ pattern: String, in image: CGImage) async throws -> TextObservation?
-
- /// Find all text occurrences matching a pattern
- /// - Parameters:
- /// - pattern: The text to search for (supports regex)
- /// - image: The image to search in
- /// - Returns: All matching text observations, sorted by vertical position (top to bottom)
- func findAllText(_ pattern: String, in image: CGImage) async throws -> [TextObservation]
-}
-
-/// Default OCR service using Apple Vision framework
-final class VisionOCRService: OCRServiceProtocol, @unchecked Sendable {
- /// Recognition level for text detection
- enum RecognitionLevel {
- case fast
- case accurate
- }
-
- private let recognitionLevel: RecognitionLevel
- private let minimumConfidence: Float
-
- init(recognitionLevel: RecognitionLevel = .accurate, minimumConfidence: Float = 0.3) {
- self.recognitionLevel = recognitionLevel
- self.minimumConfidence = minimumConfidence
- }
-
- func recognizeText(in image: CGImage) async throws -> [TextObservation] {
- try await withCheckedThrowingContinuation { continuation in
- let request = VNRecognizeTextRequest { request, error in
- if let error = error {
- continuation.resume(throwing: UnattendedError.ocrFailed(error.localizedDescription))
- return
- }
-
- guard let observations = request.results as? [VNRecognizedTextObservation] else {
- continuation.resume(returning: [])
- return
- }
-
- let textObservations = observations.compactMap { observation -> TextObservation? in
- guard let topCandidate = observation.topCandidates(1).first else {
- return nil
- }
-
- // Filter by confidence
- guard topCandidate.confidence >= self.minimumConfidence else {
- return nil
- }
-
- return TextObservation(
- text: topCandidate.string,
- boundingBox: observation.boundingBox,
- confidence: topCandidate.confidence
- )
- }
-
- continuation.resume(returning: textObservations)
- }
-
- // Configure recognition level
- switch recognitionLevel {
- case .fast:
- request.recognitionLevel = .fast
- case .accurate:
- request.recognitionLevel = .accurate
- }
-
- // Use revision that supports latest features
- request.revision = VNRecognizeTextRequestRevision3
-
- // Enable automatic language detection
- request.automaticallyDetectsLanguage = true
-
- // Perform the request
- let handler = VNImageRequestHandler(cgImage: image, options: [:])
- do {
- try handler.perform([request])
- } catch {
- continuation.resume(throwing: UnattendedError.ocrFailed(error.localizedDescription))
- }
- }
- }
-
- func findText(_ pattern: String, in image: CGImage) async throws -> TextObservation? {
- let matches = try await findAllText(pattern, in: image)
- return matches.first
- }
-
- func findAllText(_ pattern: String, in image: CGImage) async throws -> [TextObservation] {
- let observations = try await recognizeText(in: image)
- var matches: [TextObservation] = []
-
- // Check if pattern looks like a regex (contains special characters)
- let regexSpecialChars = CharacterSet(charactersIn: "^$.*+?[]{}()|\\")
- let isRegexPattern = pattern.unicodeScalars.contains { regexSpecialChars.contains($0) }
-
- if isRegexPattern {
- // Try regex match for patterns with special characters
- if let regex = try? NSRegularExpression(pattern: pattern, options: [.caseInsensitive]) {
- for observation in observations {
- let range = NSRange(observation.text.startIndex..., in: observation.text)
- if regex.firstMatch(in: observation.text, range: range) != nil {
- matches.append(observation)
- }
- }
- }
- } else {
- // First, try exact match (case-insensitive) - this prevents "Agree" matching "Disagree"
- let exactMatches = observations.filter {
- $0.text.localizedCaseInsensitiveCompare(pattern) == .orderedSame
- }
-
- if !exactMatches.isEmpty {
- matches = exactMatches
- } else {
- // Fallback to substring match (case-insensitive)
- matches = observations.filter {
- $0.text.localizedCaseInsensitiveContains(pattern)
- }
-
- // Final fallback: try regex match if no substring matches
- if matches.isEmpty {
- if let regex = try? NSRegularExpression(pattern: pattern, options: [.caseInsensitive]) {
- for observation in observations {
- let range = NSRange(observation.text.startIndex..., in: observation.text)
- if regex.firstMatch(in: observation.text, range: range) != nil {
- matches.append(observation)
- }
- }
- }
- }
- }
- }
-
- // Sort by vertical position (top to bottom)
- // Note: Vision uses bottom-left origin, so higher Y = lower on screen
- // We want top-to-bottom, so sort by descending Y
- matches.sort { $0.boundingBox.origin.y > $1.boundingBox.origin.y }
-
- return matches
- }
-}
diff --git a/libs/lume/src/Unattended/UnattendedConfig.swift b/libs/lume/src/Unattended/UnattendedConfig.swift
index fcd98c088c..84bf330f00 100644
--- a/libs/lume/src/Unattended/UnattendedConfig.swift
+++ b/libs/lume/src/Unattended/UnattendedConfig.swift
@@ -45,7 +45,7 @@ struct UnattendedConfig: Codable, Sendable {
/// Seconds to wait before starting automation after VM boots
let bootWait: Int
- /// Boot commands to execute for Setup Assistant automation
+ /// Compatibility boot commands. Offline setup ignores these.
let bootCommands: [String]
/// Optional health check to verify setup success
diff --git a/libs/lume/src/Unattended/UnattendedInstaller.swift b/libs/lume/src/Unattended/UnattendedInstaller.swift
index 95e842570e..a26e7a8853 100644
--- a/libs/lume/src/Unattended/UnattendedInstaller.swift
+++ b/libs/lume/src/Unattended/UnattendedInstaller.swift
@@ -1,167 +1,92 @@
import Foundation
-import Virtualization
/// Orchestrates unattended macOS installation
@MainActor
final class UnattendedInstaller {
- /// Run agent-based setup using Claude computer-use API
- func installWithAgent(
+ /// Run unattended setup on a VM by patching the installed macOS disk offline.
+ func install(
vm: VM,
- apiKey: String,
- model: String = "claude-sonnet-4-6",
- maxIterations: Int = 100,
- systemPrompt: String? = nil,
+ config: UnattendedConfig,
vncPort: Int = 0,
noDisplay: Bool = false,
debug: Bool = false,
debugDir: String? = nil
) async throws {
- Logger.info("Starting agent-based setup", metadata: [
+ Logger.info("Starting offline unattended installation", metadata: [
"vm": vm.name,
- "model": model,
- "maxIterations": "\(maxIterations)",
- "debug": "\(debug)"
+ "configBootCommandCount": "\(config.bootCommands.count)",
+ "postSshCommandCount": "\(config.postSshCommands?.count ?? 0)",
+ "verifyVncPort": "\(vncPort)",
+ "compatNoDisplayFlag": "\(noDisplay)",
+ "compatDebugFlag": "\(debug)",
+ "compatDebugDir": debugDir ?? "default"
+ ])
+
+ try await materializeFirstBootState(vm: vm, vncPort: vncPort)
+
+ let patcher = MacOSOfflineSetupPatcher()
+ try patcher.patch(vm: vm)
+
+ try await verifySetup(vm: vm, config: config, vncPort: vncPort)
+ }
+
+ private func materializeFirstBootState(vm: VM, vncPort: Int) async throws {
+ Logger.info("Starting VM once to materialize first-boot state before offline patch", metadata: [
+ "vm": vm.name
])
- // Start the VM (same retry logic as preset mode)
- Logger.info("Starting VM for agent setup (background task)")
var vmStartError: Error?
let vmTask = Task {
- var attempts = 0
- let maxAttempts = 3
- while attempts < maxAttempts {
- do {
- try await vm.run(
- noDisplay: noDisplay,
- sharedDirectories: [],
- mount: nil,
- vncPort: vncPort,
- recoveryMode: false,
- usbMassStoragePaths: []
- )
- return
- } catch {
- let errorDescription = error.localizedDescription
- if errorDescription.contains("auxiliary storage") || errorDescription.contains("Failed to lock") {
- attempts += 1
- if attempts < maxAttempts {
- Logger.info("VM start failed due to auxiliary storage lock, retrying", metadata: [
- "attempt": "\(attempts)/\(maxAttempts)",
- "waitTime": "5s"
- ])
- try await Task.sleep(nanoseconds: 5_000_000_000)
- continue
- }
- }
- vmStartError = error
- throw error
- }
+ do {
+ try await vm.run(
+ noDisplay: true,
+ sharedDirectories: [],
+ mount: nil,
+ vncPort: vncPort,
+ recoveryMode: false,
+ usbMassStoragePaths: []
+ )
+ } catch {
+ vmStartError = error
+ throw error
}
}
- // Give the VM a moment to start
try await Task.sleep(nanoseconds: 2_000_000_000)
if let error = vmStartError {
- Logger.error("VM failed to start", metadata: ["error": error.localizedDescription])
+ Logger.error("VM failed to start during first-boot materialization", metadata: [
+ "error": error.localizedDescription
+ ])
throw error
}
- // Wait for boot (agent mode uses a fixed 60s boot wait)
- let bootWait: UInt64 = 60
- Logger.info("Waiting for VM to boot", metadata: ["bootWait": "\(bootWait)s"])
- try await Task.sleep(nanoseconds: bootWait * 1_000_000_000)
-
- // Connect VNC input client
- Logger.info("Connecting VNC input client for agent automation")
- try await vm.vncService.connectInputClient()
-
- // Get display dimensions from VNC framebuffer
- let framebufferSize = await vm.vncService.getFrameBufferSize() ?? (width: 1920, height: 1440)
- let displayWidth = Int(framebufferSize.width)
- let displayHeight = Int(framebufferSize.height)
-
- Logger.info("VNC framebuffer size", metadata: [
- "width": "\(displayWidth)",
- "height": "\(displayHeight)"
- ])
-
- // Create SSH reachability check closure
- let macAddress = vm.vmDirContext.config.macAddress
- let sshCheck: (() async -> Bool)? = macAddress != nil ? {
- guard let ip = DHCPLeaseParser.getIPAddress(forMAC: macAddress!) else {
- return false
- }
- Logger.info("Checking SSH reachability", metadata: ["ip": ip])
- let runner = HealthCheckRunner()
- let check = HealthCheck(type: "ssh", user: "lume", password: "lume", timeout: 5, retries: 1, retryDelay: 0)
- return (try? await runner.run(check: check, vmIP: ip)) ?? false
- } : nil
-
- // Create and run the agent
- let debugDirectory: URL? = debugDir.map { URL(fileURLWithPath: $0) }
- let agent = AgentSetupRunner(
- vncService: vm.vncService,
- apiKey: apiKey,
- model: model,
- displayWidth: displayWidth,
- displayHeight: displayHeight,
- maxIterations: maxIterations,
- debug: debug,
- debugDirectory: debugDirectory,
- sshCheck: sshCheck
- )
-
do {
- try await agent.run(systemPrompt: systemPrompt)
- Logger.info("Agent setup completed successfully")
-
- vm.vncService.disconnectInputClient()
-
- // Stop the VM
- Logger.info("Agent setup finished - stopping VM")
- vmTask.cancel()
- try? await vm.stop()
- Logger.info("VM stopped", metadata: ["name": vm.name])
+ let vmIP = try await waitForIPAddress(vm: vm, timeoutSeconds: 120)
+ Logger.info("First-boot state materialized enough for offline patch", metadata: [
+ "vm": vm.name,
+ "ip": vmIP
+ ])
} catch {
- Logger.error("Agent setup failed", metadata: ["error": error.localizedDescription])
- vm.vncService.disconnectInputClient()
vmTask.cancel()
try? await vm.stop()
- Logger.info("VM stopped after failure", metadata: ["name": vm.name])
throw error
}
- }
- /// Run unattended setup on a VM using preset commands
- func install(
- vm: VM,
- config: UnattendedConfig,
- vncPort: Int = 0,
- noDisplay: Bool = false,
- debug: Bool = false,
- debugDir: String? = nil
- ) async throws {
- Logger.info("Starting unattended installation", metadata: [
- "vm": vm.name,
- "bootWait": "\(config.bootWait)s",
- "commandCount": "\(config.bootCommands.count)",
- "noDisplay": "\(noDisplay)",
- "debug": "\(debug)",
- "debugDir": debugDir ?? "default"
- ])
+ try await Task.sleep(nanoseconds: 10_000_000_000)
- // Parse all boot commands first to catch any syntax errors
- let parser = BootCommandParser()
- let commands = try parser.parseAll(config.bootCommands)
+ Logger.info("Stopping VM before offline patch", metadata: ["vm": vm.name])
+ vmTask.cancel()
+ try? await vm.stop()
+ try await Task.sleep(nanoseconds: 3_000_000_000)
+ }
- Logger.info("Parsed boot commands", metadata: ["count": "\(commands.count)"])
+ private func verifySetup(vm: VM, config: UnattendedConfig, vncPort: Int) async throws {
+ Logger.info("Starting VM to verify offline unattended setup", metadata: [
+ "vm": vm.name
+ ])
- // Start the VM in a background task (vm.run() blocks forever on success)
- // We use retry logic to handle cases where auxiliary storage is still locked
- // from a previous IPSW installation (race condition with VZVirtualMachine deallocation)
- Logger.info("Starting VM for unattended setup (background task)")
var vmStartError: Error?
let vmTask = Task {
var attempts = 0
@@ -169,17 +94,16 @@ final class UnattendedInstaller {
while attempts < maxAttempts {
do {
try await vm.run(
- noDisplay: noDisplay,
+ noDisplay: true,
sharedDirectories: [],
mount: nil,
vncPort: vncPort,
recoveryMode: false,
usbMassStoragePaths: []
)
- return // VM started successfully
+ return
} catch {
let errorDescription = error.localizedDescription
- // Check if this is the auxiliary storage lock error
if errorDescription.contains("auxiliary storage") || errorDescription.contains("Failed to lock") {
attempts += 1
if attempts < maxAttempts {
@@ -191,111 +115,130 @@ final class UnattendedInstaller {
continue
}
}
- // For other errors or after max retries, store and rethrow
vmStartError = error
throw error
}
}
}
- // Give the VM a moment to start up and check for early failures
- try await Task.sleep(nanoseconds: 2_000_000_000) // 2 seconds
+ try await Task.sleep(nanoseconds: 2_000_000_000)
- // Check if VM start failed early (e.g., auxiliary storage lock error)
if let error = vmStartError {
Logger.error("VM failed to start", metadata: ["error": error.localizedDescription])
throw error
}
- // Wait for initial boot
- Logger.info("Waiting for VM to boot", metadata: ["bootWait": "\(config.bootWait)s"])
- try await Task.sleep(nanoseconds: UInt64(config.bootWait) * 1_000_000_000)
-
- // Connect VNC input client for sending mouse/keyboard events
- Logger.info("Connecting VNC input client for automation")
- try await vm.vncService.connectInputClient()
-
- // Get display dimensions from VNC framebuffer (actual resolution)
- // This is the resolution the VNC server expects for input coordinates
- let (vncWidth, vncHeight) = await vm.vncService.getFrameBufferSize() ?? (1920, 1440)
- let displayWidth = CGFloat(vncWidth)
- let displayHeight = CGFloat(vncHeight)
-
- Logger.info("VNC framebuffer size", metadata: [
- "width": "\(Int(displayWidth))",
- "height": "\(Int(displayHeight))"
- ])
-
- // Create VNC automation engine
- let debugDirectory: URL? = debugDir.map { URL(fileURLWithPath: $0) }
- let automation = VNCAutomation(
- vncService: vm.vncService,
- displayWidth: displayWidth,
- displayHeight: displayHeight,
- debug: debug,
- debugDirectory: debugDirectory
- )
-
- // Execute boot commands
- Logger.info("Executing boot commands for Setup Assistant automation")
-
do {
- try await automation.executeAll(commands)
- Logger.info("Unattended setup completed successfully")
-
- // Disconnect VNC input client (no longer needed for automation)
- vm.vncService.disconnectInputClient()
-
- // Run health check if configured
- if let healthCheck = config.healthCheck {
- Logger.info("Running health check", metadata: ["type": healthCheck.type])
-
- // Get VM IP address from MAC address
- guard let macAddress = vm.vmDirContext.config.macAddress,
- let vmIP = DHCPLeaseParser.getIPAddress(forMAC: macAddress) else {
- Logger.error("Cannot run health check - VM IP address not available")
- throw UnattendedError.healthCheckFailed("VM IP address not available")
- }
+ let vmIP = try await waitForIPAddress(vm: vm, timeoutSeconds: 300)
+ let healthCheck = config.healthCheck ?? HealthCheck(
+ type: "ssh",
+ user: "lume",
+ password: "lume",
+ timeout: 5,
+ retries: 60,
+ retryDelay: 5
+ )
+
+ Logger.info("Running offline setup health check", metadata: [
+ "type": healthCheck.type,
+ "ip": vmIP
+ ])
- let runner = HealthCheckRunner()
- let passed = try await runner.run(check: healthCheck, vmIP: vmIP)
+ let runner = HealthCheckRunner()
+ let passed = try await runner.run(check: healthCheck, vmIP: vmIP)
- if passed {
- Logger.info("Health check passed ✓", metadata: ["type": healthCheck.type])
+ guard passed else {
+ throw UnattendedError.healthCheckFailed("Health check '\(healthCheck.type)' failed")
+ }
- // Run post-SSH commands if configured (more reliable than VNC typing)
- if let postCommands = config.postSshCommands, !postCommands.isEmpty {
- Logger.info("Running post-SSH commands", metadata: ["count": "\(postCommands.count)"])
- try await runner.runPostSshCommands(
- commands: postCommands,
- vmIP: vmIP,
- user: healthCheck.user ?? "lume",
- password: healthCheck.password ?? "lume"
- )
- Logger.info("Post-SSH commands completed ✓")
- }
- } else {
- Logger.error("Health check failed ✗", metadata: ["type": healthCheck.type])
- throw UnattendedError.healthCheckFailed("Health check '\(healthCheck.type)' failed")
- }
+ try await finalizeGuestSystemSetup(
+ vmIP: vmIP,
+ user: healthCheck.user ?? "lume",
+ password: healthCheck.password ?? "lume"
+ )
+
+ if let postCommands = config.postSshCommands, !postCommands.isEmpty {
+ Logger.info("Running post-SSH commands", metadata: ["count": "\(postCommands.count)"])
+ try await runner.runPostSshCommands(
+ commands: postCommands,
+ vmIP: vmIP,
+ user: healthCheck.user ?? "lume",
+ password: healthCheck.password ?? "lume"
+ )
+ Logger.info("Post-SSH commands completed")
}
- // Success - stop the VM
- Logger.info("Unattended setup finished successfully - stopping VM")
+ Logger.info("Offline unattended setup verified - stopping VM")
vmTask.cancel()
try? await vm.stop()
Logger.info("VM stopped", metadata: ["name": vm.name])
} catch {
- Logger.error("Unattended setup failed", metadata: [
+ Logger.error("Offline unattended setup verification failed", metadata: [
"error": error.localizedDescription
])
- // Disconnect VNC input client
- vm.vncService.disconnectInputClient()
- // Cancel VM task and stop
vmTask.cancel()
try? await vm.stop()
Logger.info("VM stopped after failure", metadata: ["name": vm.name])
throw error
}
}
+
+ private func finalizeGuestSystemSetup(vmIP: String, user: String, password: String) async throws {
+ Logger.info("Finalizing guest system setup via SSH", metadata: [
+ "host": vmIP,
+ "user": user
+ ])
+
+ let sshClient = SSHClient(host: vmIP, port: 22, user: user, password: password)
+ let escapedPassword = password.replacingOccurrences(of: "'", with: "'\\''")
+ let script = """
+ set -e
+ /usr/bin/touch /var/db/.AppleSetupDone
+ /usr/sbin/chown root:wheel /var/db/.AppleSetupDone
+ /bin/chmod 644 /var/db/.AppleSetupDone
+ /usr/bin/defaults write /Library/Preferences/com.apple.loginwindow autoLoginUser -string lume
+ /usr/bin/defaults write /Library/Preferences/com.apple.loginwindow GuestEnabled -bool false
+ /usr/bin/defaults write /Library/Preferences/com.apple.loginwindow lastUser -string loggedIn
+ /usr/bin/defaults write /Library/Preferences/com.apple.loginwindow lastUserName -string lume
+ /usr/bin/defaults write /Library/Preferences/com.apple.SetupAssistant DidSeeCloudSetup -bool true
+ /usr/bin/defaults write /Library/Preferences/com.apple.SetupAssistant DidSeePrivacy -bool true
+ /usr/bin/defaults write /Library/Preferences/com.apple.SetupAssistant DidSeeSiriSetup -bool true
+ /usr/bin/defaults write /Library/Preferences/com.apple.SetupAssistant DidSeeTouchIDSetup -bool true
+ /usr/bin/defaults write /Library/Preferences/com.apple.SetupAssistant DidSeeTrueToneSetup -bool true
+ /bin/launchctl enable system/com.openssh.sshd
+ /bin/ls -lan /var/db/.AppleSetupDone
+ """
+ let encodedScript = Data(script.utf8).base64EncodedString()
+ let command = "printf '%s\\n' '\(escapedPassword)' | /usr/bin/sudo -S /bin/sh -c '/bin/echo \(encodedScript) | /usr/bin/base64 -D | /bin/sh'"
+ let result = try await sshClient.execute(command: command, timeout: 60)
+
+ guard result.exitCode == 0 else {
+ throw UnattendedError.commandExecutionFailed(
+ "Guest system finalization failed with exit code \(result.exitCode): \(result.output)"
+ )
+ }
+
+ Logger.info("Guest system setup finalized via SSH", metadata: ["host": vmIP])
+ }
+
+ private func waitForIPAddress(vm: VM, timeoutSeconds: Int) async throws -> String {
+ guard let macAddress = vm.vmDirContext.config.macAddress else {
+ throw UnattendedError.healthCheckFailed("VM MAC address not available")
+ }
+
+ let deadline = Date().addingTimeInterval(TimeInterval(timeoutSeconds))
+ while Date() < deadline {
+ if let ip = DHCPLeaseParser.getIPAddress(forMAC: macAddress) {
+ Logger.info("Found VM IP for offline setup verification", metadata: [
+ "vm": vm.name,
+ "ip": ip
+ ])
+ return ip
+ }
+
+ try await Task.sleep(nanoseconds: 5_000_000_000)
+ }
+
+ throw UnattendedError.healthCheckFailed("VM IP address not available after \(timeoutSeconds)s")
+ }
}
diff --git a/libs/lume/src/Unattended/VNCAutomation.swift b/libs/lume/src/Unattended/VNCAutomation.swift
deleted file mode 100644
index 8fe07c78e5..0000000000
--- a/libs/lume/src/Unattended/VNCAutomation.swift
+++ /dev/null
@@ -1,501 +0,0 @@
-import Foundation
-import CoreGraphics
-import AppKit
-
-/// VNC automation engine for executing boot commands
-@MainActor
-final class VNCAutomation {
- private let vncService: VNCService
- private let ocrService: OCRServiceProtocol
- private let displayWidth: CGFloat
- private let displayHeight: CGFloat
- private let debug: Bool
- private let debugDirectory: URL
-
- /// Polling interval for OCR-based waits (in seconds)
- private let pollInterval: TimeInterval = 1.0
-
- /// Default timeout for wait commands
- private let defaultWaitTimeout: TimeInterval = 120.0
-
- /// Command index for debug screenshot naming
- private var commandIndex: Int = 0
-
- init(
- vncService: VNCService,
- ocrService: OCRServiceProtocol = VisionOCRService(),
- displayWidth: CGFloat,
- displayHeight: CGFloat,
- debug: Bool = false,
- debugDirectory: URL? = nil
- ) {
- self.vncService = vncService
- self.ocrService = ocrService
- self.displayWidth = displayWidth
- self.displayHeight = displayHeight
- self.debug = debug
- // Use custom directory or default to unique folder in system temp directory
- if let debugDirectory = debugDirectory {
- self.debugDirectory = debugDirectory
- } else {
- self.debugDirectory = FileManager.default.temporaryDirectory
- .appendingPathComponent("unattended-\(UUID().uuidString)")
- }
- }
-
- /// Execute a single boot command
- func execute(_ command: BootCommand) async throws {
- switch command {
- case .waitForText(let text, let timeout):
- try await waitForText(text, timeout: timeout)
-
- case .clickText(let text):
- try await clickOnText(text, xOffset: 0, yOffset: 0, index: nil)
-
- case .clickTextWithOffset(let text, let xOffset, let yOffset):
- try await clickOnText(text, xOffset: xOffset, yOffset: yOffset, index: nil)
-
- case .clickTextWithIndex(let text, let index, let xOffset, let yOffset):
- try await clickOnText(text, xOffset: xOffset, yOffset: yOffset, index: index)
-
- case .clickAt(let x, let y):
- try await vncService.sendMouseClick(
- at: CGPoint(x: x, y: y),
- button: .left
- )
-
- case .typeText(let text, let delay):
- try await vncService.sendText(text, delayMs: delay)
-
- case .keyPress(let key):
- let keyCode = specialKeyToKeyCode(key)
- try await vncService.sendKeyPress(keyCode, modifiers: .none)
-
- case .hotkey(let modifiers, let key):
- try await sendHotkey(modifiers: modifiers, key: key)
-
- case .hotkeyChar(let modifiers, let char):
- try await sendHotkeyChar(modifiers: modifiers, char: char)
-
- case .delay(let duration):
- try await Task.sleep(nanoseconds: UInt64(duration * 1_000_000_000))
- }
- }
-
- /// Convert SpecialKey modifiers to VNCKeyModifiers
- private func specialKeysToVNCModifiers(_ modifiers: [SpecialKey]) -> VNCKeyModifiers {
- var vncModifiers: VNCKeyModifiers = .none
- for modifier in modifiers {
- switch modifier {
- case .leftShift:
- vncModifiers.insert(.shift)
- case .leftCtrl:
- vncModifiers.insert(.control)
- case .leftAlt:
- vncModifiers.insert(.option)
- case .leftSuper:
- vncModifiers.insert(.command)
- default:
- break
- }
- }
- return vncModifiers
- }
-
- /// Send a hotkey combination (modifiers + special key)
- private func sendHotkey(modifiers: [SpecialKey], key: SpecialKey) async throws {
- let vncModifiers = specialKeysToVNCModifiers(modifiers)
- let keyCode = specialKeyToKeyCode(key)
- try await vncService.sendKeyPress(keyCode, modifiers: vncModifiers)
-
- Logger.debug("Sent hotkey", metadata: [
- "modifiers": modifiers.map { "\($0)" }.joined(separator: "+"),
- "key": "\(key)"
- ])
- }
-
- /// Send a hotkey combination (modifiers + character key like 'q', 'c', 'v')
- private func sendHotkeyChar(modifiers: [SpecialKey], char: Character) async throws {
- let vncModifiers = specialKeysToVNCModifiers(modifiers)
- try await vncService.sendCharWithModifiers(char, modifiers: vncModifiers)
-
- Logger.debug("Sent hotkey char", metadata: [
- "modifiers": modifiers.map { "\($0)" }.joined(separator: "+"),
- "char": "\(char)"
- ])
- }
-
- /// Execute a sequence of boot commands
- func executeAll(_ commands: [BootCommand]) async throws {
- // Create debug directory if needed
- if debug {
- try? FileManager.default.createDirectory(at: debugDirectory, withIntermediateDirectories: true)
- Logger.info("Debug mode enabled - saving screenshots to \(debugDirectory.path)")
- }
-
- for (index, command) in commands.enumerated() {
- commandIndex = index + 1
- Logger.info("Executing boot command", metadata: [
- "index": "\(commandIndex)/\(commands.count)",
- "command": describeCommand(command)
- ])
- try await execute(command)
-
- // Small delay between commands for stability
- try await Task.sleep(nanoseconds: 200_000_000)
- }
- }
-
- /// Wait for text to appear on screen
- private func waitForText(_ text: String, timeout: TimeInterval) async throws {
- let deadline = Date().addingTimeInterval(timeout)
- var pollCount = 0
-
- while Date() < deadline {
- pollCount += 1
- do {
- let framebuffer = try await vncService.captureFramebuffer()
- Logger.debug("Captured framebuffer", metadata: [
- "poll": "\(pollCount)",
- "width": "\(framebuffer.width)",
- "height": "\(framebuffer.height)"
- ])
-
- // Get all text from screen for debugging
- let allObservations = try await ocrService.recognizeText(in: framebuffer)
- if !allObservations.isEmpty {
- let foundTexts = allObservations.map { "\($0.text)(\(String(format: "%.2f", $0.confidence)))" }.joined(separator: " | ")
- Logger.debug("OCR found texts", metadata: [
- "count": "\(allObservations.count)",
- "texts": String(foundTexts.prefix(500))
- ])
- } else {
- Logger.debug("OCR found no text", metadata: ["poll": "\(pollCount)"])
- }
-
- if let _ = try await ocrService.findText(text, in: framebuffer) {
- Logger.info("Text found on screen", metadata: ["text": text])
- return
- }
- } catch {
- // Log but continue polling - framebuffer capture might fail temporarily
- Logger.debug("OCR poll failed", metadata: ["error": error.localizedDescription])
- }
-
- try await Task.sleep(nanoseconds: UInt64(pollInterval * 1_000_000_000))
- }
-
- throw UnattendedError.textNotFound(text: text, timeout: timeout)
- }
-
- /// Find text on screen and click on it
- /// - Parameters:
- /// - text: Text to find via OCR
- /// - xOffset: Horizontal offset in VNC pixels (negative = left, positive = right)
- /// - yOffset: Vertical offset in VNC pixels (negative = up, positive = down)
- /// - index: Which match to click (0 = first, 1 = second, -1 = last, -2 = second to last, etc.)
- private func clickOnText(_ text: String, xOffset: Int, yOffset: Int, index: Int?) async throws {
- let framebuffer = try await vncService.captureFramebuffer()
-
- // Always get all OCR observations for debug logging
- let allObservations = debug ? try await ocrService.recognizeText(in: framebuffer) : []
-
- let observation: TextObservation?
- if let idx = index {
- // Find all matches and select by index
- let allMatches = try await ocrService.findAllText(text, in: framebuffer)
- if allMatches.isEmpty {
- observation = nil
- } else if idx >= 0 {
- // Positive index: 0 = first, 1 = second, etc.
- observation = idx < allMatches.count ? allMatches[idx] : nil
- } else {
- // Negative index: -1 = last, -2 = second to last, etc.
- let positiveIdx = allMatches.count + idx
- observation = positiveIdx >= 0 ? allMatches[positiveIdx] : nil
- }
-
- Logger.info("Text search with index", metadata: [
- "text": text,
- "index": "\(idx)",
- "totalMatches": "\(allMatches.count)",
- "selected": observation != nil ? "yes" : "no"
- ])
- } else {
- observation = try await ocrService.findText(text, in: framebuffer)
- }
-
- guard let observation = observation else {
- // Save debug screenshot even on failure with full OCR tree
- if debug {
- saveDebugScreenshot(framebuffer: framebuffer, clickPoint: nil, text: text, failed: true, ocrObservations: allObservations)
- }
- throw UnattendedError.textNotFound(text: text, timeout: 0)
- }
-
- // Use captured framebuffer dimensions for coordinate mapping
- let capturedWidth = CGFloat(framebuffer.width)
- let capturedHeight = CGFloat(framebuffer.height)
-
- // Get center point in screen coordinates (using captured image dimensions)
- let centerPoint = observation.centerPoint(
- screenWidth: capturedWidth,
- screenHeight: capturedHeight
- )
-
- // Use captured framebuffer coordinates directly for VNC input
- // The Screen Sharing window capture represents the actual VM display,
- // and VNC coordinates should match what we see on screen
- let vncPoint = CGPoint(
- x: centerPoint.x + CGFloat(xOffset),
- y: centerPoint.y + CGFloat(yOffset)
- )
-
- // The click point in captured framebuffer coordinates (for debug screenshot)
- let clickPointInCapture = CGPoint(
- x: centerPoint.x + CGFloat(xOffset),
- y: centerPoint.y + CGFloat(yOffset)
- )
-
- Logger.info("Clicking on text", metadata: [
- "text": text,
- "captured": "\(Int(capturedWidth))x\(Int(capturedHeight))",
- "display": "\(Int(displayWidth))x\(Int(displayHeight))",
- "capturedPoint": "(\(Int(centerPoint.x)), \(Int(centerPoint.y)))",
- "vncPoint": "(\(Int(vncPoint.x)), \(Int(vncPoint.y)))",
- "xOffset": "\(xOffset)",
- "yOffset": "\(yOffset)"
- ])
-
- // Save debug screenshot before clicking with full OCR tree
- if debug {
- saveDebugScreenshot(framebuffer: framebuffer, clickPoint: clickPointInCapture, text: text, failed: false, ocrObservations: allObservations)
- }
-
- try await vncService.sendMouseClick(at: vncPoint, button: .left)
- }
-
- /// Convert special key enum to macOS key code
- private func specialKeyToKeyCode(_ key: SpecialKey) -> UInt16 {
- switch key {
- case .enter: return 36
- case .tab: return 48
- case .escape: return 53
- case .leftSuper: return 55 // Command
- case .leftShift: return 56
- case .leftAlt: return 58 // Option
- case .leftCtrl: return 59
- case .backspace: return 51
- case .delete: return 117
- case .up: return 126
- case .down: return 125
- case .left: return 123
- case .right: return 124
- case .space: return 49
- case .f1: return 122
- case .f2: return 120
- case .f3: return 99
- case .f4: return 118
- case .f5: return 96
- case .f6: return 97
- case .f7: return 98
- case .f8: return 100
- case .f9: return 101
- case .f10: return 109
- case .f11: return 103
- case .f12: return 111
- }
- }
-
- /// Generate a human-readable description of a command
- private func describeCommand(_ command: BootCommand) -> String {
- switch command {
- case .waitForText(let text, let timeout):
- return "wait for '\(text)' (timeout: \(Int(timeout))s)"
- case .clickText(let text):
- return "click '\(text)'"
- case .clickTextWithOffset(let text, let xOffset, let yOffset):
- return "click '\(text)' (xoffset: \(xOffset), yoffset: \(yOffset))"
- case .clickTextWithIndex(let text, let index, let xOffset, let yOffset):
- return "click '\(text)' (index: \(index), xoffset: \(xOffset), yoffset: \(yOffset))"
- case .clickAt(let x, let y):
- return "click at (\(x), \(y))"
- case .typeText(let text, let delay):
- let displayText = text.count > 20 ? String(text.prefix(20)) + "..." : text
- if let delay = delay {
- return "type '\(displayText)' (delay: \(delay)ms)"
- }
- return "type '\(displayText)'"
- case .keyPress(let key):
- return "press <\(key.rawValue)>"
- case .hotkey(let modifiers, let key):
- let modifierStr = modifiers.map { "\($0)" }.joined(separator: "+")
- return "hotkey <\(modifierStr)+\(key)>"
- case .hotkeyChar(let modifiers, let char):
- let modifierStr = modifiers.map { "\($0)" }.joined(separator: "+")
- return "hotkey <\(modifierStr)+\(char)>"
- case .delay(let duration):
- return "delay \(duration)s"
- }
- }
-
- /// Save a debug screenshot with click point marked and OCR tree log
- /// - Parameters:
- /// - framebuffer: The captured framebuffer
- /// - clickPoint: The point where click will occur (in framebuffer coordinates), or nil if text not found
- /// - text: The text being searched for
- /// - failed: Whether the text search failed
- /// - ocrObservations: All OCR observations for logging (optional)
- private func saveDebugScreenshot(
- framebuffer: CGImage,
- clickPoint: CGPoint?,
- text: String,
- failed: Bool,
- ocrObservations: [TextObservation]? = nil
- ) {
- let width = framebuffer.width
- let height = framebuffer.height
-
- // Create a new image with the click point marked
- let colorSpace = CGColorSpaceCreateDeviceRGB()
- guard let context = CGContext(
- data: nil,
- width: width,
- height: height,
- bitsPerComponent: 8,
- bytesPerRow: width * 4,
- space: colorSpace,
- bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue
- ) else {
- Logger.error("Failed to create graphics context for debug screenshot")
- return
- }
-
- // Draw the original framebuffer
- context.draw(framebuffer, in: CGRect(x: 0, y: 0, width: width, height: height))
-
- if let clickPoint = clickPoint {
- // Draw a red crosshair at the click point
- // Note: CGContext has origin at bottom-left, so flip Y coordinate
- let flippedY = CGFloat(height) - clickPoint.y
-
- // Set red color
- context.setStrokeColor(CGColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 1.0))
- context.setLineWidth(3.0)
-
- // Draw crosshair
- let crosshairSize: CGFloat = 20
-
- // Horizontal line
- context.move(to: CGPoint(x: clickPoint.x - crosshairSize, y: flippedY))
- context.addLine(to: CGPoint(x: clickPoint.x + crosshairSize, y: flippedY))
- context.strokePath()
-
- // Vertical line
- context.move(to: CGPoint(x: clickPoint.x, y: flippedY - crosshairSize))
- context.addLine(to: CGPoint(x: clickPoint.x, y: flippedY + crosshairSize))
- context.strokePath()
-
- // Draw a circle around the click point
- context.setLineWidth(2.0)
- let circleRect = CGRect(
- x: clickPoint.x - crosshairSize,
- y: flippedY - crosshairSize,
- width: crosshairSize * 2,
- height: crosshairSize * 2
- )
- context.strokeEllipse(in: circleRect)
- }
-
- guard let annotatedImage = context.makeImage() else {
- Logger.error("Failed to create annotated image for debug screenshot")
- return
- }
-
- // Generate filename
- let sanitizedText = text
- .replacingOccurrences(of: "/", with: "_")
- .replacingOccurrences(of: "\\", with: "_")
- .replacingOccurrences(of: "^", with: "")
- .replacingOccurrences(of: "$", with: "")
- .prefix(30)
- let status = failed ? "FAILED" : "click"
- let filename = String(format: "%03d-%@-%@.png", commandIndex, status, String(sanitizedText))
- let fileURL = debugDirectory.appendingPathComponent(filename)
-
- // Save as PNG
- let bitmapRep = NSBitmapImageRep(cgImage: annotatedImage)
- guard let pngData = bitmapRep.representation(using: .png, properties: [:]) else {
- Logger.error("Failed to create PNG data for debug screenshot")
- return
- }
-
- do {
- try pngData.write(to: fileURL)
- Logger.info("Saved debug screenshot", metadata: [
- "path": fileURL.path,
- "clickPoint": clickPoint.map { "(\(Int($0.x)), \(Int($0.y)))" } ?? "none"
- ])
- } catch {
- Logger.error("Failed to save debug screenshot", metadata: ["error": error.localizedDescription])
- }
-
- // Save OCR tree log as JSON alongside the screenshot
- if let observations = ocrObservations, !observations.isEmpty {
- let ocrLogFilename = String(format: "%03d-%@-%@-ocr.json", commandIndex, status, String(sanitizedText))
- let ocrLogURL = debugDirectory.appendingPathComponent(ocrLogFilename)
-
- // Build OCR tree with all observations, sorted top-to-bottom
- let sortedObservations = observations.sorted { $0.boundingBox.origin.y > $1.boundingBox.origin.y }
-
- var ocrEntries: [[String: Any]] = []
- for (idx, obs) in sortedObservations.enumerated() {
- let screenRect = obs.screenRect(screenWidth: CGFloat(width), screenHeight: CGFloat(height))
- let center = obs.centerPoint(screenWidth: CGFloat(width), screenHeight: CGFloat(height))
-
- // Check if this observation matches the search text
- let isMatch = obs.text.localizedCaseInsensitiveContains(text)
-
- let entry: [String: Any] = [
- "index": idx,
- "text": obs.text,
- "confidence": String(format: "%.3f", obs.confidence),
- "boundingBox": [
- "x": Int(screenRect.origin.x),
- "y": Int(screenRect.origin.y),
- "width": Int(screenRect.width),
- "height": Int(screenRect.height)
- ],
- "center": [
- "x": Int(center.x),
- "y": Int(center.y)
- ],
- "matchesSearch": isMatch
- ]
- ocrEntries.append(entry)
- }
-
- let ocrLog: [String: Any] = [
- "timestamp": ISO8601DateFormatter().string(from: Date()),
- "commandIndex": commandIndex,
- "searchText": text,
- "failed": failed,
- "framebufferSize": ["width": width, "height": height],
- "clickPoint": clickPoint.map { ["x": Int($0.x), "y": Int($0.y)] } as Any,
- "observationCount": observations.count,
- "observations": ocrEntries
- ]
-
- do {
- let jsonData = try JSONSerialization.data(withJSONObject: ocrLog, options: [.prettyPrinted, .sortedKeys])
- try jsonData.write(to: ocrLogURL)
- Logger.info("Saved OCR tree log", metadata: [
- "path": ocrLogURL.path,
- "observationCount": "\(observations.count)"
- ])
- } catch {
- Logger.error("Failed to save OCR tree log", metadata: ["error": error.localizedDescription])
- }
- }
- }
-}
diff --git a/libs/lume/src/Utils/CommandDocExtractor.swift b/libs/lume/src/Utils/CommandDocExtractor.swift
index 0a57eb1439..164446fe8c 100644
--- a/libs/lume/src/Utils/CommandDocExtractor.swift
+++ b/libs/lume/src/Utils/CommandDocExtractor.swift
@@ -111,13 +111,13 @@ enum CommandDocExtractor {
OptionDoc(name: "display", shortName: nil, help: "Display resolution (e.g., 1024x768)", type: "String", defaultValue: "1024x768", isOptional: false),
OptionDoc(name: "ipsw", shortName: nil, help: "Path to IPSW file or 'latest' for macOS VMs", type: "String", defaultValue: nil, isOptional: true),
OptionDoc(name: "storage", shortName: nil, help: "VM storage location to use", type: "String", defaultValue: nil, isOptional: true),
- OptionDoc(name: "unattended", shortName: nil, help: "[Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.", type: "String", defaultValue: nil, isOptional: true),
- OptionDoc(name: "debug-dir", shortName: nil, help: "Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp)", type: "String", defaultValue: nil, isOptional: true),
- OptionDoc(name: "vnc-port", shortName: nil, help: "Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign)", type: "Int", defaultValue: "0", isOptional: true),
+ OptionDoc(name: "unattended", shortName: nil, help: "Prepare macOS unattended setup offline after install. Preset name or YAML path is accepted for compatibility. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.", type: "String", defaultValue: nil, isOptional: true),
+ OptionDoc(name: "debug-dir", shortName: nil, help: "Compatibility option; ignored by offline setup.", type: "String", defaultValue: nil, isOptional: true),
+ OptionDoc(name: "vnc-port", shortName: nil, help: "Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign).", type: "Int", defaultValue: "0", isOptional: true),
],
flags: [
- FlagDoc(name: "debug", shortName: nil, help: "Enable debug mode for unattended setup - saves screenshots with click coordinates", defaultValue: false),
- FlagDoc(name: "no-display", shortName: nil, help: "Do not open the VNC client during unattended setup (default: true for unattended)", defaultValue: false),
+ FlagDoc(name: "debug", shortName: nil, help: "Compatibility flag; ignored by offline setup.", defaultValue: false),
+ FlagDoc(name: "no-display", shortName: nil, help: "Compatibility flag; offline setup verifies headlessly.", defaultValue: false),
],
subcommands: []
)
@@ -128,7 +128,7 @@ enum CommandDocExtractor {
private static var pullDoc: CommandDoc {
CommandDoc(
name: "pull",
- abstract: "Pull a macOS image from GitHub Container Registry",
+ abstract: "Pull a prebuilt or custom macOS image from an OCI-compatible registry",
discussion: nil,
arguments: [
ArgumentDoc(name: "image", help: "Image to pull (format: name:tag)", type: "String", isOptional: false),
@@ -149,7 +149,7 @@ enum CommandDocExtractor {
private static var pushDoc: CommandDoc {
CommandDoc(
name: "push",
- abstract: "Push a macOS VM to GitHub Container Registry",
+ abstract: "Push a macOS VM to an OCI-compatible registry",
discussion: nil,
arguments: [
ArgumentDoc(name: "name", help: "Name of VM to push", type: "String", isOptional: false),
@@ -548,20 +548,20 @@ enum CommandDocExtractor {
private static var setupDoc: CommandDoc {
CommandDoc(
name: "setup",
- abstract: "[Preview] Run unattended Setup Assistant automation on a macOS VM",
- discussion: "This is an experimental feature. Unattended configurations are specific to macOS versions and may not work across different releases.",
+ abstract: "Prepare unattended macOS setup",
+ discussion: "Lume prepares the macOS disk offline, skips Setup Assistant, enables autologin and SSH, disables screensaver lock, verifies SSH, then stops the VM.",
arguments: [
ArgumentDoc(name: "name", help: "Name of the virtual machine", type: "String", isOptional: false),
],
options: [
- OptionDoc(name: "unattended", shortName: nil, help: "Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe.", type: "String", defaultValue: nil, isOptional: false),
+ OptionDoc(name: "unattended", shortName: nil, help: "Defaults to tahoe. Preset name or YAML path for compatibility and optional post-SSH commands. Built-in presets: sequoia, tahoe.", type: "String", defaultValue: "tahoe", isOptional: true),
OptionDoc(name: "storage", shortName: nil, help: "VM storage location to use or direct path to VM location", type: "String", defaultValue: nil, isOptional: true),
- OptionDoc(name: "vnc-port", shortName: nil, help: "Port to use for the VNC server. Defaults to 0 (auto-assign)", type: "Int", defaultValue: "0", isOptional: true),
- OptionDoc(name: "debug-dir", shortName: nil, help: "Custom directory for debug screenshots (defaults to unique folder in system temp)", type: "String", defaultValue: nil, isOptional: true),
+ OptionDoc(name: "vnc-port", shortName: nil, help: "Port to use for the temporary verification VNC server. Defaults to 0 (auto-assign)", type: "Int", defaultValue: "0", isOptional: true),
+ OptionDoc(name: "debug-dir", shortName: nil, help: "Compatibility option; ignored by offline setup", type: "String", defaultValue: nil, isOptional: true),
],
flags: [
- FlagDoc(name: "no-display", shortName: nil, help: "Do not open the VNC client automatically", defaultValue: false),
- FlagDoc(name: "debug", shortName: nil, help: "Enable debug mode - saves screenshots with click coordinates", defaultValue: false),
+ FlagDoc(name: "no-display", shortName: nil, help: "Compatibility flag; offline setup verifies headlessly", defaultValue: false),
+ FlagDoc(name: "debug", shortName: nil, help: "Compatibility flag; ignored by offline setup", defaultValue: false),
],
subcommands: []
)