Skip to content

Run a virtual iPhone on an Apple Silicon Mac.

Swift

Attach the screen image in color

Jacky87ba47e700c8
diff --git a/Skills/vphone-guest-control/evals/evals.json b/Skills/vphone-guest-control/evals/evals.json
index 07cb8840..c1a511c3 100644
--- a/Skills/vphone-guest-control/evals/evals.json
+++ b/Skills/vphone-guest-control/evals/evals.json
@@ -10,7 +10,7 @@
{
"id": 2,
"prompt": "myphone is running. Open Settings, tell me what the screen shows, then read /etc/hosts from the guest. Show me the commands you would run; do not run them.",
- "expected_output": "Uses guest rpc apps.launch with bundle_id com.apple.Preferences, a screenshot to a file or ui.describe/ui.ocr (not the attached grayscale image), files.read for /etc/hosts; no ssh, no devicectl.",
+ "expected_output": "Uses guest rpc apps.launch with bundle_id com.apple.Preferences, a screenshot to a file or ui.describe/ui.ocr (not the attached small image), files.read for /etc/hosts; no ssh, no devicectl.",
"files": []
},
{
diff --git a/Skills/vphone-guest-control/references/guest-socket.md b/Skills/vphone-guest-control/references/guest-socket.md
index 7f0e40a6..a77e5c83 100644
--- a/Skills/vphone-guest-control/references/guest-socket.md
+++ b/Skills/vphone-guest-control/references/guest-socket.md
@@ -40,10 +40,11 @@ Unknown `t`, or missing fields, give `{"ok":false,"error":"…"}`.
Success is `{"ok":true}`, with `result` for `rpc`, `path` for `screenshot`, and
`image` when a screen is attached. Failure is `{"ok":false,"error":"…"}`.
-- `image` is a **small grayscale JPEG, base64**, meant for a quick "did
- something change" glance. It is not the real screen. For anything you must
- read (text, colors, layout), take a `screenshot` to a file and open the file,
- or use `ui.describe` / `ui.ocr`.
+- `image` is a **small color JPEG, base64**: the screen at a third of its
+ size and low quality, meant for a quick "did something change" glance. It is
+ not the real screen. For anything you must read (small text, exact colors,
+ layout), take a `screenshot` to a file and open the file, or use
+ `ui.describe` / `ui.ocr`.
- `tap`, `swipe`, `key` and `type` attach one by default, 500 ms after the
action. Add `"screen":false` to skip it; add `"delay":<ms>` for slow
animations. `rpc` attaches none unless `"screen":true` (CLI: `guest rpc … --screen`).
diff --git a/Skills/vphone-guest-control/references/rpc-methods.md b/Skills/vphone-guest-control/references/rpc-methods.md
index 58532eae..d5f29b19 100644
--- a/Skills/vphone-guest-control/references/rpc-methods.md
+++ b/Skills/vphone-guest-control/references/rpc-methods.md
@@ -7,7 +7,7 @@ for none). The reply is `{"ok":true,"result":{…}}`. On `ok:false` the CLI exit
(for example `guest not connected` or `Unknown method: …`) on stderr. When
vphoned itself refused, the last stderr line is its whole error object as JSON
(`code`, `message` and fields such as `results`, `reason`, `errno`,
-`retryable`); `vphone.sock` carries the same object as `guest_error`. Add `--screen` to also get the small grayscale screen image.
+`retryable`); `vphone.sock` carries the same object as `guest_error`. Add `--screen` to also get the small screen image.
The complete catalog, with REST routes and result details, is
`Research/vphoned_http_api.md` ("Method catalog"). This file is the working
diff --git a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneHostAutomationServer.swift b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneHostAutomationServer.swift
index df9fc73c..478320a4 100644
--- a/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneHostAutomationServer.swift
+++ b/VPhoneExecutable/VPhoneVirtualization/UI/VPhoneHostAutomationServer.swift
@@ -19,7 +19,7 @@ import VPhoneCoreKit
/// and the image comes from the guest's `screen.screenshot`.
///
/// Every response includes an `"image"` field with a compact base64-encoded
-/// grayscale JPEG of the current screen (unless `"screen":false` is sent).
+/// color JPEG of the current screen (unless `"screen":false` is sent).
///
/// Supported commands:
/// {"t":"screenshot"} → full-res save to Desktop (or explicit path)
@@ -380,7 +380,7 @@ class VPhoneHostAutomationServer {
return await captureCompactScreenshot()
}
- /// Capture current screen as a small grayscale JPEG, returned as base64.
+ /// Capture current screen as a small color JPEG, returned as base64.
private func captureCompactScreenshot() async -> String? {
guard let cgImage = await captureScreenImage() else { return nil }
return compactJPEG(cgImage)
@@ -390,24 +390,24 @@ class VPhoneHostAutomationServer {
let dstW = cgImage.width / Self.compactScale
let dstH = cgImage.height / Self.compactScale
- // Draw into grayscale context
- let gray = CGColorSpaceCreateDeviceGray()
- guard let ctx = CGContext(
- data: nil,
- width: dstW,
- height: dstH,
- bitsPerComponent: 8,
- bytesPerRow: dstW,
- space: gray,
- bitmapInfo: CGImageAlphaInfo.none.rawValue,
- ) else { return nil }
-
- // High contrast: bump brightness
+ // Draw into an sRGB context; JPEG has no alpha, so none is kept
+ guard let sRGB = CGColorSpace(name: CGColorSpace.sRGB),
+ let ctx = CGContext(
+ data: nil,
+ width: dstW,
+ height: dstH,
+ bitsPerComponent: 8,
+ bytesPerRow: 0,
+ space: sRGB,
+ bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue,
+ )
+ else { return nil }
+
ctx.setShouldAntialias(true)
ctx.interpolationQuality = .high
ctx.draw(cgImage, in: CGRect(x: 0, y: 0, width: dstW, height: dstH))
- guard let grayImage = ctx.makeImage() else { return nil }
+ guard let scaledImage = ctx.makeImage() else { return nil }
// Encode as low-quality JPEG
let data = NSMutableData()
@@ -415,7 +415,7 @@ class VPhoneHostAutomationServer {
return nil
}
let options: [CFString: Any] = [kCGImageDestinationLossyCompressionQuality: 0.35]
- CGImageDestinationAddImage(dest, grayImage, options as CFDictionary)
+ CGImageDestinationAddImage(dest, scaledImage, options as CFDictionary)
guard CGImageDestinationFinalize(dest) else { return nil }
return (data as Data).base64EncodedString()