Skip to content

Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.

Fix board setup recovery and simplify model provider configuration

Henry346feea2d5be
diff --git a/docs/connection-setup-debug-2026-10-16.md b/docs/connection-setup-debug-2026-10-16.md
new file mode 100644
index 0000000..9b070a0
--- /dev/null
+++ b/docs/connection-setup-debug-2026-10-16.md
@@ -0,0 +1,32 @@
+# Connection and model setup verification — 2026-10-16
+
+Base: local main, e035e424f492225e1a84a75dc172da75bfdb73bc.
+Branch: codex/diagnose-board-pairing.
+
+## Findings
+
+- Both the old and new iOS apps were installed and running. Their logs showed matching connection timeouts and encryption failures. Stopping the old app did not eliminate failures, so app contention was not established as the root cause.
+- The board advertised the expected GATT service. Initial observed advertising RSSI was -95 to -99 dBm. USB enumeration alone did not prove BLE connectivity.
+- The app remembered a device before connection succeeded, had no connection deadline, hid nearby alternatives when a device was remembered, and marked setup ready before reading the encrypted network characteristic.
+- A board already connected through another app can be absent from scanning results; setup now also retrieves connected peripherals by service.
+
+## Changes
+
+- Bound scanning to 12 seconds and connection/pairing to 30 seconds; expose errors and a Scan Again action. Retain discovered peripheral references and ignore obsolete callbacks.
+- Read encrypted configuration before marking the board ready and persisting its identity. Reconnect after disconnection with a delay.
+- Firmware uses 9 dBm and 1M PHY instead of 0 dBm and forced 2M, favoring link reliability. Video throughput under this setting has not been re-benchmarked.
+- Sample RSSI for diagnostics. Only weak signal displays a short proximity/antenna hint; no permanent antenna instructions or links in setup.
+- Compact provider/key/model rows; Others exposes Base URL inline. Models remain editable when a list is available. OpenRouter Free presets openrouter/free; OpenAI, Gemini, Qwen China/International, DeepSeek, and OpenRouter provide endpoints and key links.
+- Do not automatically choose the first model in an API response. Connection testing sends a generated color image; configuration changes invalidate previous results. Keys do not carry across hosts.
+
+## Validation and limits
+
+- Firmware compiled, flashed over USB, and verified by the uploader's hash check.
+- Debug iOS build installed on the connected phone; encrypted GATT setup and reconnect after app relaunch observed.
+- With phone and Mac connected simultaneously: 60 ping commands, zero failures, median 24.2 ms, maximum 68.9 ms; board reported zero notification/reply drops.
+- One observation window lasted over four minutes without a disconnect, but subsequent timeouts recurred. This is not proof that the RF problem is permanently resolved.
+- After the phone was moved nearer, observed RSSI was approximately -78 to -83 dBm and connection/relaunch succeeded. Antenna installation was not verified.
+- 71 iOS tests passed, including provider selection/custom-model coverage. Final small presentation changes compiled successfully. Onboarding layouts inspected on an iPhone 17 Pro simulator.
+- No paid provider calls were made and no provider credentials were provisioned. Live provider model-list and inference compatibility require the user's key.
+
+SwiftUI layout review used https://github.com/twostraws/SwiftUI-Agent-Skill.
diff --git a/hardware/hid_bridge/ble_link.cpp b/hardware/hid_bridge/ble_link.cpp
index 3bb35ca..4da7abc 100644
--- a/hardware/hid_bridge/ble_link.cpp
+++ b/hardware/hid_bridge/ble_link.cpp
@@ -100,9 +100,9 @@ static void logConn(const char *what, NimBLEConnInfo &ci) {
ci.isEncrypted() ? (ci.isBonded() ? "encrypted+bonded" : "encrypted") : "plain");
}
-// 15 ms interval (min = max, the fastest iOS accepts), no latency, 4 s supervision; 2M PHY; DLE 251.
+// 15 ms interval, no latency, 4 s supervision; reliable 1M PHY; DLE 251.
static void tuneLink(uint16_t h) {
- server->updatePhy(h, BLE_GAP_LE_PHY_2M_MASK, BLE_GAP_LE_PHY_2M_MASK, BLE_GAP_LE_PHY_CODED_ANY);
+ // Keep the connection on 1M: forcing 2M makes weak links repeatedly time out.
server->setDataLen(h, 251);
server->updateConnParams(h, 12, 12, 0, 400);
}
@@ -253,8 +253,8 @@ void ble_begin(const String &name) {
statusLock = xSemaphoreCreateMutex();
NimBLEDevice::init(name.c_str());
NimBLEDevice::setMTU(517);
- NimBLEDevice::setPower(0); // dBm; the phone is centimetres away (and less current, see netcfg.cpp)
- NimBLEDevice::setDefaultPhy(BLE_GAP_LE_PHY_2M_MASK, BLE_GAP_LE_PHY_2M_MASK);
+ NimBLEDevice::setPower(9); // dBm: retain link margin during setup, including a board powered by a Mac.
+ NimBLEDevice::setDefaultPhy(BLE_GAP_LE_PHY_1M_MASK, BLE_GAP_LE_PHY_1M_MASK);
NimBLEDevice::setSecurityAuth(true, false, true); // bonding, Just Works, LE Secure Connections
NimBLEDevice::setSecurityIOCap(BLE_HS_IO_NO_INPUT_OUTPUT);
NimBLEDevice::setSecurityInitKey(BLE_SM_PAIR_KEY_DIST_ENC | BLE_SM_PAIR_KEY_DIST_ID);
diff --git a/ios/IPhoneAgent.xcodeproj/project.pbxproj b/ios/IPhoneAgent.xcodeproj/project.pbxproj
index f087f8c..a2da39a 100644
--- a/ios/IPhoneAgent.xcodeproj/project.pbxproj
+++ b/ios/IPhoneAgent.xcodeproj/project.pbxproj
@@ -70,12 +70,14 @@
96C08C1CFEECA69484811137 /* StatusChips.swift in Sources */ = {isa = PBXBuildFile; fileRef = 96BDA430E0C54DC956DE9366 /* StatusChips.swift */; };
99D64CF9B896616382FC8A0E /* AgentActivity.appex in Embed Foundation Extensions */ = {isa = PBXBuildFile; fileRef = A11E0FF01881451EBB3D18FF /* AgentActivity.appex */; settings = {ATTRIBUTES = (RemoveHeadersOnCopy, ); }; };
9BA8ABE4398546618D6AE675 /* DiagnosticsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2F3122E7A1DB53F3BFCCFD06 /* DiagnosticsView.swift */; };
+ 9C68E2C1C9A3DDC1026186EB /* ProviderFields.swift in Sources */ = {isa = PBXBuildFile; fileRef = 56CC3C0A640902A2CD3F8387 /* ProviderFields.swift */; };
9CB0E6A2FE92EF5422B95377 /* AgentSnapshot.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3EC8ECB7FF0B63991E8AEFFB /* AgentSnapshot.swift */; };
A0DECAC4E4DBADB432942A83 /* IntentBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5538B879586D04FB7F15C7E5 /* IntentBridge.swift */; };
A177729443FE818CD754FD64 /* StepFrame.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8E64CF4B18DBAB014C7E9314 /* StepFrame.swift */; };
A271DC80F0EB0BA42FD7D022 /* PokeableFace.swift in Sources */ = {isa = PBXBuildFile; fileRef = A409B3696F547E712B4D4D14 /* PokeableFace.swift */; };
A42DB537742861FF668DB34A /* JSON.swift in Sources */ = {isa = PBXBuildFile; fileRef = FCC7FAA4989F36E7F9B4C021 /* JSON.swift */; };
A5A1640A615256B576F9A055 /* RunSession.swift in Sources */ = {isa = PBXBuildFile; fileRef = 38FB6FEBD55DCBE8EED6834C /* RunSession.swift */; };
+ A5A78F1B4D15D843B8932932 /* ModelProvider.swift in Sources */ = {isa = PBXBuildFile; fileRef = D8A02583BE955659071F3172 /* ModelProvider.swift */; };
ABEF22E60573E04FACD0E4A9 /* AgentFace.swift in Sources */ = {isa = PBXBuildFile; fileRef = A7314ABD64EC1106422C685E /* AgentFace.swift */; };
AD702230192ADBF827044490 /* LiveActivity.swift in Sources */ = {isa = PBXBuildFile; fileRef = 27A03E1A122C50B8ED88AC91 /* LiveActivity.swift */; };
B356EECAFE045249704E63FD /* MockMode.swift in Sources */ = {isa = PBXBuildFile; fileRef = 13664BFD54E7827E0E53B7F6 /* MockMode.swift */; };
@@ -174,6 +176,7 @@
4D9FABC284798F9949C1A8EF /* LogUploader.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LogUploader.swift; sourceTree = "<group>"; };
541D1A2837E5F270700EAF55 /* Redactor.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Redactor.swift; sourceTree = "<group>"; };
5538B879586D04FB7F15C7E5 /* IntentBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = IntentBridge.swift; sourceTree = "<group>"; };
+ 56CC3C0A640902A2CD3F8387 /* ProviderFields.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ProviderFields.swift; sourceTree = "<group>"; };
5754C9233BF0BA12F8EB78D7 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist; path = Info.plist; sourceTree = "<group>"; };
5962900FEB08728899160DD1 /* DemoRecorder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DemoRecorder.swift; sourceTree = "<group>"; };
607F9EBBC39E74507B73F751 /* H264Decoder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = H264Decoder.swift; sourceTree = "<group>"; };
@@ -214,6 +217,7 @@
D17A722B1C48BF8FBF8B4222 /* OnboardingDemos.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OnboardingDemos.swift; sourceTree = "<group>"; };
D222722DF7856F9253B9784E /* IntentDetailView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = IntentDetailView.swift; sourceTree = "<group>"; };
D317C60AA727B6A20A589DAC /* AssistiveTouchSpots.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AssistiveTouchSpots.swift; sourceTree = "<group>"; };
+ D8A02583BE955659071F3172 /* ModelProvider.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ModelProvider.swift; sourceTree = "<group>"; };
D9615E8664A5DC600B1B97E0 /* VideoFeed.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VideoFeed.swift; sourceTree = "<group>"; };
DA2DCFC32B990B166DF9C1C6 /* IPhoneAgent.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = IPhoneAgent.entitlements; sourceTree = "<group>"; };
DCDA925D0FFD42810895573C /* LLMClient.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LLMClient.swift; sourceTree = "<group>"; };
@@ -326,6 +330,7 @@
99381FBF82EA7EB76556866E /* OnboardingView.swift */,
A409B3696F547E712B4D4D14 /* PokeableFace.swift */,
8C498727CAF51C5A955A8A10 /* PrivacyViews.swift */,
+ 56CC3C0A640902A2CD3F8387 /* ProviderFields.swift */,
915FC0D291F09BE0A0439D9C /* RunsView.swift */,
3E74E2758C4170812147466F /* SettingsView.swift */,
96BDA430E0C54DC956DE9366 /* StatusChips.swift */,
@@ -396,6 +401,7 @@
DCDA925D0FFD42810895573C /* LLMClient.swift */,
4D9FABC284798F9949C1A8EF /* LogUploader.swift */,
B9719A5EC1DD7C5033CC8DF5 /* Metrics.swift */,
+ D8A02583BE955659071F3172 /* ModelProvider.swift */,
05DA6345A40715C8BD2AED6C /* Notifier.swift */,
541D1A2837E5F270700EAF55 /* Redactor.swift */,
B1B84B4F1C81E697ACB30630 /* RunShare.swift */,
@@ -643,6 +649,7 @@
F2FFDB2986493F65EB6A9738 /* LogUploader.swift in Sources */,
7B684FAA8B218BB9798F9EC3 /* Metrics.swift in Sources */,
B356EECAFE045249704E63FD /* MockMode.swift in Sources */,
+ A5A78F1B4D15D843B8932932 /* ModelProvider.swift in Sources */,
7F412478410DC75BC26A0D5C /* ModelSettingsView.swift in Sources */,
4CA70811574C401EB0012355 /* Notifier.swift in Sources */,
CFDC07BA9E09D3446164B1B9 /* OnboardingDemos.swift in Sources */,
@@ -653,6 +660,7 @@
72E72D656EFE51EB73F94A5C /* PrivacyViews.swift in Sources */,
58405C052C5CE287759E2AA4 /* ProbeFit.swift in Sources */,
CED032EC090264C33AD7D1D1 /* Prompts.swift in Sources */,
+ 9C68E2C1C9A3DDC1026186EB /* ProviderFields.swift in Sources */,
079927568C3047DA5C2F105E /* Recovery.swift in Sources */,
8F45E1F46FB046780AC0838F /* Redactor.swift in Sources */,
2A48924EF6EF7A39D5A05E21 /* ReviewMedia.swift in Sources */,
diff --git a/ios/IPhoneAgent/App/AppModel.swift b/ios/IPhoneAgent/App/AppModel.swift
index e81861f..244b4b0 100644
--- a/ios/IPhoneAgent/App/AppModel.swift
+++ b/ios/IPhoneAgent/App/AppModel.swift
@@ -25,6 +25,7 @@ import UIKit
let pointer: Pointer
let agent: AgentLoop
@Published private(set) var log: [LogLine] = []
+ @Published var modelError: String?
@Published var modelOK: Bool? // last LLM ping result
@Published var hotspotSSID: String?
@Published var otherWiFi: String? // the phone is on this network, not the board's
@@ -54,6 +55,7 @@ import UIKit
var currentIntent: SavedTask?
var lastTask = ""
var runStarted = Date()
+ private var settingsSink: AnyCancellable?
var videoSink: AnyCancellable?
var videoDir: URL?
private(set) lazy var calibrator = AutoCalibrator(app: self)
@@ -87,6 +89,10 @@ import UIKit
board.commands.onAppCommand = { [weak self] line in self?.handleAppCommand(line) }
DebugLog.write("app launched \(Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") ?? "")"
+ " (\(Bundle.main.object(forInfoDictionaryKey: "CFBundleVersion") ?? ""))")
+ settingsSink = settings.$configurationRevision.dropFirst().sink { [weak self] _ in
+ self?.modelOK = nil
+ self?.modelError = nil
+ }
board.onVideo = { [weak self] in self?.feed.receive($0) }
board.onStatus = { [weak self] s in
self?.feed.mirroring = s.ap == "streaming"
@@ -188,7 +194,10 @@ import UIKit
var mood: Mood { runMood ?? (board.connected ? .idle : .sleeping) }
/// The model endpoint is filled in (demo data simulates one).
- var modelReady: Bool { MockMode.on || settings.isConfigured }
+ var modelReady: Bool {
+ MockMode.on || (settings.isConfigured && settings.currentModel != nil
+ && (settings.provider == .others || !settings.apiKey.isEmpty))
+ }
func say(_ text: String) {
log = Array((log + [LogLine(text: text)]).suffix(200))
@@ -228,15 +237,28 @@ import UIKit
return say("LLM ok (demo data)")
}
let started = now()
+ let revision = settings.configurationRevision
+ modelError = nil
do {
let client = try await settings.client("actor")
- let r = try await client.chat(system: "Reply with OK.", text: "ping", image: nil, maxTokens: 600, temperature: 0)
+ let image = UIGraphicsImageRenderer(size: CGSize(width: 64, height: 64)).image { context in
+ UIColor.red.setFill()
+ context.fill(CGRect(x: 0, y: 0, width: 64, height: 64))
+ }.jpegData(compressionQuality: 0.8)
+ let r = try await client.chat(system: "Name the dominant color in the image. Reply with one English word.",
+ text: "What color is this image?", image: image, maxTokens: 600, temperature: 0)
+ guard revision == settings.configurationRevision else { return }
+ guard r.text.lowercased().contains("red") else {
+ throw LLMError(message: "The model did not identify the sample image. Choose a model that supports images.")
+ }
metrics.llmPingMs = (now() - started) * 1000
metrics.llm(first: r.firstToken, total: r.total)
modelOK = true
say(String(format: "LLM ok in %.0f ms: %@", metrics.llmPingMs ?? 0, String(r.text.prefix(40))))
} catch {
+ guard revision == settings.configurationRevision else { return }
modelOK = false
+ modelError = error.localizedDescription
say("LLM: \(error.localizedDescription)")
}
}
diff --git a/ios/IPhoneAgent/App/DebugEntry.swift b/ios/IPhoneAgent/App/DebugEntry.swift
index 52c8ac3..a69bc53 100644
--- a/ios/IPhoneAgent/App/DebugEntry.swift
+++ b/ios/IPhoneAgent/App/DebugEntry.swift
@@ -82,6 +82,10 @@ extension AppModel {
guard let i = args.firstIndex(of: flag), i + 1 < args.count else { return nil }
return args[i + 1]
}
+ #if DEBUG
+ if let requested = value(after: "-setup"),
+ let page = ["connect": SetupPage.connect, "brain": .brain][requested] { setupPage = page }
+ #endif
if let task = value(after: "-run") { debugCommand("run", task: task) }
if args.contains("-calibrate") { debugCommand("calibrate") }
if args.contains("-bench") { debugCommand("bench", kb: Int(value(after: "-bench") ?? "") ?? 256) }
diff --git a/ios/IPhoneAgent/Board/BoardLink.swift b/ios/IPhoneAgent/Board/BoardLink.swift
index 8861717..306b6d0 100644
--- a/ios/IPhoneAgent/Board/BoardLink.swift
+++ b/ios/IPhoneAgent/Board/BoardLink.swift
@@ -14,12 +14,14 @@ import Foundation
@Published private(set) var found: [Found] = []
@Published private(set) var scanning = false
@Published private(set) var connected = false // subscribed and ready for commands
+ @Published private(set) var connecting = false
@Published private(set) var linking = false // connected over the air, pairing / subscribing
@Published private(set) var pairedID: UUID?
@Published private(set) var name = ""
@Published private(set) var status: BoardStatus?
@Published private(set) var net: NetConfig?
@Published private(set) var lastError = ""
+ @Published private(set) var signalStrength: Int?
let commands = CommandChannel()
var onVideo: ((Data) -> Void)?
@@ -37,6 +39,10 @@ import Foundation
private var netWaiters: [CheckedContinuation<NetConfig, Error>] = []
private static let pairedKey = "pairedBoard"
private var statusReads: Task<Void, Never>?
+ private var scanDeadline: Task<Void, Never>?
+ private var connectionDeadline: Task<Void, Never>?
+ private var retryTask: Task<Void, Never>?
+ private var candidates: [UUID: CBPeripheral] = [:]
let mock: Bool // demo data: no Bluetooth, every command answers ok (MockMode)
init(mock: Bool = false) {
@@ -57,13 +63,28 @@ import Foundation
// MARK: - scan / pair
func scan() {
- guard state == .poweredOn, !mock else { return }
+ guard state == .poweredOn, !mock, !connected, !connecting, !linking else { return }
found = []
+ lastError = ""
scanning = true
central.scanForPeripherals(withServices: [BoardUUID.service], options: nil)
+ log("BLE scanning for board service")
+ scanDeadline?.cancel()
+ scanDeadline = Task { [weak self] in
+ guard (try? await Task.sleep(nanoseconds: 12_000_000_000)) != nil, let self, self.scanning else { return }
+ self.stopScan()
+ if self.found.isEmpty {
+ self.lastError = String(localized: "No board found. Check its power, keep it nearby, and scan again.")
+ }
+ }
+ // A board already connected by another app may no longer advertise.
+ for p in central.retrieveConnectedPeripherals(withServices: [BoardUUID.service]) {
+ discovered(p, name: p.name ?? "ESP32 Agent", rssi: 0)
+ }
}
func stopScan() {
+ scanDeadline?.cancel()
central?.stopScan()
scanning = false
}
@@ -73,7 +94,13 @@ import Foundation
if mock { return mockPair(id) }
stopScan()
pairedID = id
- UserDefaults.standard.set(id.uuidString, forKey: Self.pairedKey)
+ lastError = ""
+ if let previous = peripheral, previous.identifier != id {
+ peripheral = nil
+ central.cancelPeripheralConnection(previous)
+ setDisconnected()
+ }
+ peripheral = candidates[id]
reconnect()
}
@@ -86,15 +113,49 @@ import Foundation
/// Forget the board in the app only.
func forget() {
- if let peripheral { central?.cancelPeripheralConnection(peripheral) }
+ retryTask?.cancel()
+ connectionDeadline?.cancel()
+ let previous = peripheral
peripheral = nil
+ if let previous { central?.cancelPeripheralConnection(previous) }
pairedID = nil
+ signalStrength = nil
UserDefaults.standard.removeObject(forKey: Self.pairedKey)
setDisconnected()
}
+ func retryDiscovery() {
+ forget()
+ lastError = ""
+ scan()
+ }
+
+ private func discovered(_ p: CBPeripheral, name: String, rssi: Int) {
+ candidates[p.identifier] = p
+ if rssi < 0 { signalStrength = rssi }
+ let item = Found(id: p.identifier, name: name, rssi: rssi)
+ if let i = found.firstIndex(where: { $0.id == item.id }) { found[i] = item } else { found.append(item) }
+ log("BLE found \(name), RSSI \(rssi)")
+ if p.identifier == pairedID && !connected && !linking && !connecting {
+ stopScan()
+ peripheral = p
+ reconnect()
+ }
+ }
+
+ private func failConnection(_ message: String) {
+ retryTask?.cancel()
+ lastError = message
+ log("BLE setup failed: \(message)")
+ let previous = peripheral
+ peripheral = nil
+ setDisconnected()
+ stopScan()
+ if let previous { central.cancelPeripheralConnection(previous) }
+ }
+
private func reconnect() {
- guard state == .poweredOn, !mock, let id = pairedID else { return }
+ guard state == .poweredOn, !mock, !connected, !connecting, !linking, let id = pairedID else { return }
if peripheral?.identifier != id {
peripheral = central.retrievePeripherals(withIdentifiers: [id]).first
}
@@ -105,13 +166,25 @@ import Foundation
}
peripheral.delegate = self
name = peripheral.name ?? name
- central.connect(peripheral, options: nil) // no timeout: completes whenever the board shows up
+ connecting = true
+ log("BLE connecting to \(name)")
+ connectionDeadline?.cancel()
+ connectionDeadline = Task { [weak self] in
+ guard (try? await Task.sleep(nanoseconds: 30_000_000_000)) != nil, let self, !self.connected else { return }
+ self.failConnection(String(localized: "Connection timed out. Keep the board nearby and try again."))
+ }
+ if peripheral.state == .connected {
+ connecting = false
+ linking = true
+ peripheral.discoverServices([BoardUUID.service])
+ } else { central.connect(peripheral, options: nil) }
}
// MARK: - net
/// Read the network config (the first read pairs the phone with the board).
func readNet() async throws -> NetConfig {
+ if connected, let net { return net }
guard let peripheral, let c = chars[BoardUUID.net] else { throw DeviceError(message: "board not connected") }
return try await withCheckedThrowingContinuation { cont in
netWaiters.append(cont)
@@ -207,7 +280,7 @@ import Foundation
var tick = 0, lastTick = now()
while !Task.isCancelled {
guard let self, self.connected, let p = self.peripheral, let c = self.chars[BoardUUID.status] else { return }
- if tick % 5 == 0 { p.readValue(for: c) }
+ if tick % 5 == 0 { p.readValue(for: c); p.readRSSI() }
if now() - lastTick < 2 && now() - self.lastNotify > 5 {
DebugLog.write(String(format: "BLE: no notification for %.0f s; dropping the link", now() - self.lastNotify))
return self.dropLink()
@@ -221,6 +294,9 @@ import Foundation
private func setDisconnected() {
statusReads?.cancel()
+ connectionDeadline?.cancel()
+ connecting = false
+ net = nil
let was = connected
connected = false
linking = false
@@ -240,7 +316,8 @@ extension BoardLink: CBCentralManagerDelegate {
nonisolated func centralManagerDidUpdateState(_ central: CBCentralManager) {
MainActor.assumeIsolated {
state = central.state
- if central.state == .poweredOn { reconnect() } else { setDisconnected() }
+ log("BLE state \(central.state.rawValue)")
+ if central.state == .poweredOn { reconnect() } else { stopScan(); setDisconnected() }
}
}
@@ -258,18 +335,14 @@ extension BoardLink: CBCentralManagerDelegate {
advertisementData: [String: Any], rssi: NSNumber) {
MainActor.assumeIsolated {
let name = advertisementData[CBAdvertisementDataLocalNameKey] as? String ?? p.name ?? "ESP32 Agent"
- let item = Found(id: p.identifier, name: name, rssi: rssi.intValue)
- if let i = found.firstIndex(where: { $0.id == item.id }) { found[i] = item } else { found.append(item) }
- if p.identifier == pairedID && !connected && !linking { // the paired board is back: connect at once
- stopScan()
- peripheral = p
- reconnect()
- }
+ discovered(p, name: name, rssi: rssi.intValue)
}
}
nonisolated func centralManager(_ central: CBCentralManager, didConnect p: CBPeripheral) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
+ connecting = false
name = p.name ?? name
linking = true
log("board connected, discovering")
@@ -279,32 +352,48 @@ extension BoardLink: CBCentralManagerDelegate {
nonisolated func centralManager(_ central: CBCentralManager, didFailToConnect p: CBPeripheral, error: Error?) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
lastError = error?.localizedDescription ?? "connect failed"
let e = error as NSError?
DebugLog.write("BLE connect failed: \(e?.domain ?? "-") \(e?.code ?? 0): \(lastError)")
- reconnect()
+ failConnection(lastError)
}
}
nonisolated func centralManager(_ central: CBCentralManager, didDisconnectPeripheral p: CBPeripheral,
error: Error?) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
let e = error as NSError?
DebugLog.write(String(format: "BLE disconnect: %@ %d: %@; last notification %.1f s ago; link up %.0f s",
e?.domain ?? "no error", e?.code ?? 0, e?.localizedDescription ?? "-",
lastNotify > 0 ? now() - lastNotify : -1, readySince > 0 ? now() - readySince : -1))
setDisconnected()
if let error { lastError = error.localizedDescription }
- reconnect() // pending connect: iOS completes it when the board is back, even in background
+ retryTask?.cancel()
+ retryTask = Task { [weak self] in
+ guard (try? await Task.sleep(nanoseconds: 2_000_000_000)) != nil else { return }
+ self?.reconnect()
+ }
}
}
}
extension BoardLink: CBPeripheralDelegate {
+ nonisolated func peripheral(_ p: CBPeripheral, didReadRSSI RSSI: NSNumber, error: Error?) {
+ MainActor.assumeIsolated {
+ guard p === peripheral, error == nil else { return }
+ signalStrength = RSSI.intValue
+ log("BLE signal \(RSSI.intValue) dBm")
+ }
+ }
+
nonisolated func peripheral(_ p: CBPeripheral, didDiscoverServices error: Error?) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
+ if let error { failConnection(error.localizedDescription); return }
guard let service = p.services?.first(where: { $0.uuid == BoardUUID.service }) else {
- lastError = "board service missing"
+ failConnection(String(localized: "Board service missing. Check the board firmware."))
return
}
p.discoverCharacteristics(BoardUUID.all, for: service)
@@ -313,6 +402,12 @@ extension BoardLink: CBPeripheralDelegate {
nonisolated func peripheral(_ p: CBPeripheral, didDiscoverCharacteristicsFor service: CBService, error: Error?) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
+ if let error { failConnection(error.localizedDescription); return }
+ let available = Set((service.characteristics ?? []).map(\.uuid))
+ guard available.isSuperset(of: BoardUUID.all) else {
+ failConnection(String(localized: "Board firmware is missing required features. Update the firmware.")); return
+ }
for c in service.characteristics ?? [] {
chars[c.uuid] = c
if [BoardUUID.reply, BoardUUID.status, BoardUUID.video].contains(c.uuid) { p.setNotifyValue(true, for: c) }
@@ -322,31 +417,43 @@ extension BoardLink: CBPeripheralDelegate {
nonisolated func peripheral(_ p: CBPeripheral, didUpdateNotificationStateFor c: CBCharacteristic, error: Error?) {
MainActor.assumeIsolated {
- if let error { lastError = "subscribe \(c.uuid): \(error.localizedDescription)"; return }
+ guard p === peripheral else { return }
+ if let error { failConnection("subscribe \(c.uuid): \(error.localizedDescription)"); return }
+ guard c.isNotifying else { return }
subscribed.insert(c.uuid)
if !connected && subscribed.isSuperset(of: [BoardUUID.reply, BoardUUID.status, BoardUUID.video]) {
- connected = true
- linking = false
- startFullStatusReads()
- commands.linkUp()
- readySince = now()
- lastNotify = now()
- onLinkUp?()
- log("board ready (MTU \(mtu))")
+ // Reading the encrypted characteristic triggers pairing before setup is marked ready.
+ if let netChar = chars[BoardUUID.net] { p.readValue(for: netChar) }
}
}
}
nonisolated func peripheral(_ p: CBPeripheral, didUpdateValueFor c: CBCharacteristic, error: Error?) {
MainActor.assumeIsolated {
+ guard p === peripheral else { return }
lastNotify = now()
if c.uuid == BoardUUID.net {
let waiters = netWaiters
netWaiters = []
if let error {
waiters.forEach { $0.resume(throwing: error) }
+ if linking { failConnection(error.localizedDescription) }
} else if let v = c.value, let config = try? JSONDecoder().decode(NetConfig.self, from: v) {
net = config
+ if linking && subscribed.isSuperset(of: [BoardUUID.reply, BoardUUID.status, BoardUUID.video]) {
+ connectionDeadline?.cancel()
+ stopScan()
+ connected = true
+ linking = false
+ lastError = ""
+ pairedID = p.identifier
+ UserDefaults.standard.set(p.identifier.uuidString, forKey: Self.pairedKey)
+ readySince = now()
+ commands.linkUp()
+ startFullStatusReads()
+ onLinkUp?()
+ log("board ready, encrypted configuration read (MTU \(mtu))")
+ }
waiters.forEach { $0.resume(returning: config) }
} else {
waiters.forEach { $0.resume(throwing: DeviceError(message: "bad net config")) }
diff --git a/ios/IPhoneAgent/Services/ModelProvider.swift b/ios/IPhoneAgent/Services/ModelProvider.swift
new file mode 100644
index 0000000..430aba9
--- /dev/null
+++ b/ios/IPhoneAgent/Services/ModelProvider.swift
@@ -0,0 +1,49 @@
+import Foundation
+
+/// Presets only configure public endpoints. Users always supply their own credentials.
+enum ModelProvider: String, CaseIterable, Identifiable {
+ case openRouterFree, openAI, gemini, qwen, qwenInternational, deepSeek, openRouter, others
+ var id: String { rawValue }
+ var title: String {
+ switch self {
+ case .openRouterFree: return "OpenRouter Free"
+ case .openAI: return "OpenAI"
+ case .gemini: return "Google Gemini"
+ case .qwen: return "Qwen · China"
+ case .qwenInternational: return "Qwen · International"
+ case .deepSeek: return "DeepSeek"
+ case .openRouter: return "OpenRouter"
+ case .others: return "Others"
+ }
+ }
+ var baseURL: String {
+ switch self {
+ case .openRouterFree, .openRouter: return "https://openrouter.ai/api/v1"
+ case .openAI: return "https://api.openai.com/v1"
+ case .gemini: return "https://generativelanguage.googleapis.com/v1beta/openai"
+ case .qwen: return "https://dashscope.aliyuncs.com/compatible-mode/v1"
+ case .qwenInternational: return "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
+ case .deepSeek: return "https://api.deepseek.com"
+ case .others: return ""
+ }
+ }
+ var keyURL: URL? {
+ let address: String
+ switch self {
+ case .openRouterFree, .openRouter: address = "https://openrouter.ai/settings/keys"
+ case .openAI: address = "https://platform.openai.com/api-keys"
+ case .gemini: address = "https://aistudio.google.com/apikey"
+ case .qwen: address = "https://bailian.console.aliyun.com/"
+ case .qwenInternational: address = "https://modelstudio.console.alibabacloud.com/"
+ case .deepSeek: address = "https://platform.deepseek.com/api_keys"
+ case .others: return nil
+ }
+ return URL(string: address)
+ }
+ static func infer(baseURL: String, model: String) -> Self {
+ let base = baseURL.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
+ if base == openRouter.baseURL { return model == "openrouter/free" ? .openRouterFree : .openRouter }
+ return allCases.first { $0 != .others && $0.baseURL == base } ?? .others
+ }
+}
+
diff --git a/ios/IPhoneAgent/Services/Settings.swift b/ios/IPhoneAgent/Services/Settings.swift
index dfa5691..fcfb7f1 100644
--- a/ios/IPhoneAgent/Services/Settings.swift
+++ b/ios/IPhoneAgent/Services/Settings.swift
@@ -8,15 +8,24 @@ import UIKit
static let defaultBaseURL = "" // user enters their own OpenAI-compatible endpoint
static let defaultModel = ""
+ @Published private(set) var provider: ModelProvider = .others
+ @Published private(set) var configurationRevision = 0
private let defaults: UserDefaults
- @Published var baseURL: String { didSet { defaults.set(baseURL, forKey: "baseURL"); if baseURL != oldValue { modelsChanged() } } }
- @Published var model: String { didSet { defaults.set(model, forKey: "model") } } // "" = first from GET /models
+ @Published var baseURL: String { didSet {
+ defaults.set(baseURL, forKey: "baseURL")
+ if baseURL != oldValue {
+ // Never carry an old provider's credential to a newly typed endpoint.
+ if URL(string: oldValue)?.host != URL(string: baseURL)?.host { apiKey = "" }
+ modelsChanged()
+ }
+ } }
+ @Published var model: String { didSet { defaults.set(model, forKey: "model"); configurationRevision += 1 } } // empty preserves a legacy saved selection
/// The model picked from the endpoint's list when `model` is empty (forgotten when the endpoint or key changes).
@Published private(set) var autoModel: String { didSet { defaults.set(autoModel, forKey: "autoModel") } }
- @Published var plannerModel: String { didSet { defaults.set(plannerModel, forKey: "plannerModel") } } // "" = model
- @Published var actorModel: String { didSet { defaults.set(actorModel, forKey: "actorModel") } } // "" = model
- @Published var tokenParam: String { didSet { defaults.set(tokenParam, forKey: "tokenParam") } } // auto | ...
- @Published var extraBody: String { didSet { defaults.set(extraBody, forKey: "extraBody") } } // JSON object
+ @Published var plannerModel: String { didSet { defaults.set(plannerModel, forKey: "plannerModel"); configurationRevision += 1 } } // "" = model
+ @Published var actorModel: String { didSet { defaults.set(actorModel, forKey: "actorModel"); configurationRevision += 1 } } // "" = model
+ @Published var tokenParam: String { didSet { defaults.set(tokenParam, forKey: "tokenParam"); configurationRevision += 1 } } // auto | ...
+ @Published var extraBody: String { didSet { defaults.set(extraBody, forKey: "extraBody"); configurationRevision += 1 } } // JSON object
@Published var apiKey: String { didSet { Keychain.set(apiKey, for: "llm-api-key"); DebugLog.redact([apiKey])
if apiKey != oldValue { modelsChanged() } } }
/// What the agent is called (Agent tab title, greeting, Live Activity); empty = the default.
@@ -55,14 +64,17 @@ import UIKit
agentName = defaults.string(forKey: Self.agentNameKey) ?? Self.defaultAgentName
onboarded = defaults.bool(forKey: "onboarded")
acceptedDisclaimer = defaults.bool(forKey: "acceptedDisclaimer")
+ provider = defaults.string(forKey: "modelProvider").flatMap(ModelProvider.init(rawValue:))
+ ?? ModelProvider.infer(baseURL: baseURL, model: model)
DebugLog.redact([apiKey]) // never in debug.log
}
private static func stored(_ v: String?) -> String? { v?.isEmpty == false ? v : nil }
- /// The endpoint is filled in (the key is optional: some endpoints need none; the model is picked automatically).
+ /// The endpoint is valid (local endpoints may not require a key).
var isConfigured: Bool {
- URL(string: baseURL)?.scheme?.hasPrefix("http") == true
+ guard let url = URL(string: baseURL), let host = url.host, !host.isEmpty else { return false }
+ return ["http", "https"].contains(url.scheme?.lowercased() ?? "")
}
/// The manual override, else the model already picked from the endpoint, else nil.
@@ -72,21 +84,45 @@ import UIKit
return autoModel.isEmpty ? nil : autoModel
}
- /// The manual override, else the first model from GET {base}/models (fetched once per endpoint and key).
+ /// Use an explicit model or a previously saved selection; never guess from API list order.
func resolvedModel() async throws -> String {
if let m = currentModel { return m }
- let picked = try await LLMClient.firstModel(baseURL: baseURL, apiKey: apiKey)
- autoModel = picked
- return picked
+ throw LLMError(message: "Choose a vision model or enter its model ID before testing.")
}
private func modelsChanged() {
+ configurationRevision += 1
+ modelsRequest += 1
+ loadingModels = false
autoModel = ""
models = []
modelsError = nil
}
- /// Fetch the model list; the latest request wins. "Automatic" becomes the first id.
+ var selectedProvider: ModelProvider {
+ get { provider }
+ set { selectProvider(newValue) }
+ }
+
+ func selectProvider(_ choice: ModelProvider) {
+ guard choice != provider else { return }
+ // Keep credentials scoped to an endpoint when switching providers.
+ if !apiKey.isEmpty { Keychain.set(apiKey, for: "llm-endpoint-" + baseURL) }
+ provider = choice
+ defaults.set(choice.rawValue, forKey: "modelProvider")
+ if choice == .others { configurationRevision += 1; return }
+ let changedEndpoint = baseURL != choice.baseURL
+ baseURL = choice.baseURL
+ if changedEndpoint { apiKey = Keychain.get("llm-endpoint-" + baseURL) ?? "" }
+ model = choice == .openRouterFree ? "openrouter/free" : ""
+ plannerModel = ""
+ actorModel = ""
+ tokenParam = "auto"
+ extraBody = "{}"
+ modelsChanged()
+ }
+
+ /// Fetch the model list; only a result for the current configuration is accepted.
func loadModels() async {
guard isConfigured else { return }
modelsRequest += 1
@@ -98,7 +134,7 @@ import UIKit
guard request == modelsRequest else { return }
models = list
modelsError = list.isEmpty ? String(localized: "The endpoint lists no models.") : nil
- if let first = list.first { autoModel = first }
+ // Model order does not indicate image support; never silently select the first entry.
DebugLog.write("models: \(list.count) listed, first \(list.first ?? "-")")
} catch {
guard request == modelsRequest else { return }
diff --git a/ios/IPhoneAgent/UI/ModelSettingsView.swift b/ios/IPhoneAgent/UI/ModelSettingsView.swift
index 2a8bd5e..13840df 100644
--- a/ios/IPhoneAgent/UI/ModelSettingsView.swift
+++ b/ios/IPhoneAgent/UI/ModelSettingsView.swift
@@ -10,26 +10,11 @@ struct ModelSettingsView: View {
var body: some View {
Form {
Section {
- TextField("Base URL", text: $settings.baseURL, prompt: Text(verbatim: "https://example.com/v1"))
- .keyboardType(.URL)
- SecureField("API key", text: $settings.apiKey, prompt: Text("API key (optional)"))
- ModelPicker(settings: settings, title: "Model", selection: $settings.model, automatic: ModelPicker.automatic(settings))
- Button { Task { await settings.loadModels() } } label: {
- HStack {
- Text("Refresh Model List")
- Spacer()
- if settings.loadingModels { ProgressView() }
- }
- }
- .disabled(settings.loadingModels || !settings.isConfigured)
+ ProviderFields(settings: settings)
} header: {
- Text("OpenAI-compatible API")
+ Text("Model Provider")
} footer: {
- if let error = settings.modelsError {
- Text("Could not load the model list: \(error) Type a model id instead.")
- } else {
- Text("Your key stays in this iPhone's Keychain and is only sent to this endpoint.")
- }
+ if let url = settings.provider.keyURL { Link("Get an API key", destination: url) }
}
Section {
Button {
@@ -46,6 +31,9 @@ struct ModelSettingsView: View {
}
}
.disabled(testing || !app.modelReady)
+ if let error = app.modelError {
+ Text(verbatim: error).font(.footnote).foregroundStyle(Palette.stop)
+ }
}
Section {
DisclosureGroup("Advanced", isExpanded: $advanced) {
@@ -64,68 +52,32 @@ struct ModelSettingsView: View {
.autocorrectionDisabled()
.navigationTitle("Model")
.navigationBarTitleDisplayMode(.inline)
- .task(id: settings.baseURL + "\n" + settings.apiKey) {
- guard (try? await Task.sleep(nanoseconds: 600_000_000)) != nil else { return } // typing: wait for a pause
- await settings.loadModels()
- }
+
}
}
-/// A model id: a menu of the endpoint's models when the list loaded, else a text field to type one.
-/// "" is the automatic choice (`automatic` labels it).
+/// A model ID is always editable, even when the endpoint supplies a model list.
struct ModelPicker: View {
@ObservedObject var settings: Settings
let title: LocalizedStringKey
@Binding var selection: String
let automatic: String
- static func automatic(_ settings: Settings) -> String {
- settings.models.first.map { String(localized: "Automatic (first: \($0))") } ?? String(localized: "Automatic")
- }
-
- var body: some View {
- if settings.models.isEmpty {
- LabeledContent {
- if settings.loadingModels && selection.isEmpty {
- ProgressView()
- } else {
- TextField(title, text: $selection, prompt: Text(verbatim: automatic))
- .multilineTextAlignment(.trailing)
- }
- } label: { Text(title) }
- } else {
- Picker(title, selection: $selection) {
- Text(verbatim: automatic).tag("")
- if !selection.isEmpty && !settings.models.contains(selection) { Text(verbatim: selection).tag(selection) }
- ForEach(settings.models, id: \.self) { Text(verbatim: $0).tag($0) }
- }
- .pickerStyle(.menu)
- }
- }
-}
-
-/// Onboarding, after Test Connection: the model in use and a menu to change it.
-struct PickedModelRow: View {
- @ObservedObject var settings: Settings
-
var body: some View {
HStack {
- VStack(alignment: .leading, spacing: 2) {
- Text("Model").font(.subheadline).foregroundStyle(.secondary)
- Text(verbatim: settings.currentModel ?? ModelPicker.automatic(settings)).font(.body.monospaced()).lineLimit(1)
- }
- Spacer()
- if settings.loadingModels {
- ProgressView()
- } else if !settings.models.isEmpty {
- Menu("Change") {
- Picker("Model", selection: $settings.model) {
- Text(verbatim: ModelPicker.automatic(settings)).tag("")
- ForEach(settings.models, id: \.self) { Text(verbatim: $0).tag($0) }
+ TextField(title, text: $selection, prompt: Text(verbatim: automatic))
+ .textInputAutocapitalization(.never).autocorrectionDisabled()
+ .accessibilityLabel(title)
+ if settings.loadingModels { ProgressView() }
+ if !settings.models.isEmpty {
+ Menu {
+ if title != "Model ID" { Button("Use Default") { selection = "" } }
+ ForEach(settings.models, id: \.self) { id in
+ Button { selection = id } label: { Text(verbatim: id) }
}
- }
+ } label: { Image(systemName: "list.bullet") }
+ .accessibilityLabel("Choose Model")
}
}
- .task { if settings.models.isEmpty { await settings.loadModels() } }
}
}
diff --git a/ios/IPhoneAgent/UI/OnboardingPages.swift b/ios/IPhoneAgent/UI/OnboardingPages.swift
index e46c714..115eea2 100644
--- a/ios/IPhoneAgent/UI/OnboardingPages.swift
+++ b/ios/IPhoneAgent/UI/OnboardingPages.swift
@@ -71,12 +71,14 @@ struct ConnectPage: View {
@ViewBuilder private var boards: some View {
if board.connected {
CheckRow(title: "Connected to \(board.name.isEmpty ? "the board" : board.name)", done: true)
+ } else if board.connecting {
+ CheckRow(title: "Connecting to the board…", done: false, active: true, waiting: true)
} else if board.linking {
CheckRow(title: "Pairing… tap Pair when iOS asks", done: false, active: true, waiting: true)
} else if board.pairedID == nil, board.found.count == 1, let f = board.found.first {
CheckRow(title: "Found \(f.name)", done: false, active: true, waiting: true)
- } else if board.pairedID == nil, board.found.count > 1 {
- Text("Several boards nearby. Choose yours:").font(.subheadline).foregroundStyle(.secondary)
+ } else if !board.found.isEmpty {
+ Text("Choose your board:").font(.subheadline).foregroundStyle(.secondary)
.padding(.bottom, Space.s)
ForEach(board.found) { f in
Button { board.pair(f.id) } label: {
@@ -89,8 +91,17 @@ struct ConnectPage: View {
}
}
} else {
- CheckRow(title: "Not connected — plug the board into USB-C", done: false, active: true, waiting: true,
- detail: "I'll connect as soon as it's plugged in.")
+ CheckRow(title: "Searching for the board over Bluetooth…", done: false, active: true, waiting: board.scanning,
+ detail: "Power the board and keep it nearby. USB connection alone does not confirm Bluetooth is ready.")
+ }
+ if let signal = board.signalStrength, signal < -80 {
+ Text("Weak Bluetooth signal. Move closer to the board or attach its antenna.")
+ .font(.footnote).foregroundStyle(.secondary).padding(.top, Space.s)
+ }
+ if !board.connected {
+ if !board.lastError.isEmpty {
+ Text(verbatim: board.lastError).font(.footnote).foregroundStyle(Palette.warning)
+ }
}
}
}
@@ -241,11 +252,8 @@ struct BrainPage: View {
Text(verbatim: IntentStore.examples[0].prompt).font(.subheadline).foregroundStyle(.secondary)
.padding(.top, Space.s)
}
- if !MockMode.on {
- Panel { PickedModelRow(settings: settings) }
- }
case false?:
- Label("Could not reach the model. Check the URL and key.", systemImage: "xmark.octagon.fill")
+ Label(app.modelError ?? String(localized: "Could not reach the model. Check the URL and key."), systemImage: "xmark.octagon.fill")
.foregroundStyle(Palette.stop).font(.subheadline)
case nil:
EmptyView()
@@ -259,11 +267,13 @@ struct ModelFieldsPanel: View {
@ObservedObject var settings: Settings
var body: some View {
- Panel {
- TextField("Base URL", text: $settings.baseURL, prompt: Text(verbatim: "https://example.com/v1"))
- .keyboardType(.URL).textContentType(.URL)
- Divider().padding(.vertical, Space.m)
- SecureField("API key", text: $settings.apiKey, prompt: Text("API key (optional)"))
+ VStack(spacing: Space.xs) {
+ Panel { ProviderFields(settings: settings) }
+ if let url = settings.provider.keyURL {
+ Link("Get an API key", destination: url)
+ .font(.footnote).tint(.secondary).underline()
+ .frame(maxWidth: .infinity, minHeight: 44, alignment: .trailing)
+ }
}
.textInputAutocapitalization(.never)
.autocorrectionDisabled()
diff --git a/ios/IPhoneAgent/UI/OnboardingView.swift b/ios/IPhoneAgent/UI/OnboardingView.swift
index e6dc4d3..21c2c60 100644
--- a/ios/IPhoneAgent/UI/OnboardingView.swift
+++ b/ios/IPhoneAgent/UI/OnboardingView.swift
@@ -127,7 +127,7 @@ struct OnboardingView: View {
/// Waiting on something the app detects by itself.
private var waiting: Bool {
switch page {
- case .connect: return !board.connected && board.state == .poweredOn
+ case .connect: return !board.connected && (board.scanning || board.connecting || board.linking)
case .see: return !streaming && (onHotspot || app.join.busy)
case .act: return calibrator.busy
case .brain: return testing
@@ -146,7 +146,7 @@ struct OnboardingView: View {
private var primaryTitle: LocalizedStringKey {
switch page {
case .welcome: return "Get Started"
- case .connect: return board.connected ? "Continue" : "Looking for the board…"
+ case .connect: return board.connected ? "Continue" : waiting ? "Looking for the board…" : "Scan Again"
case .see:
if streaming { return "Continue" }
if onHotspot { return "Waiting for mirroring…" }
@@ -169,7 +169,7 @@ struct OnboardingView: View {
private var primaryEnabled: Bool {
switch page {
case .welcome: return accepted
- case .connect: return board.connected
+ case .connect: return board.connected || (board.state == .poweredOn && !waiting)
case .see: return streaming || (!onHotspot && !app.join.busy)
case .act: return !calibrator.busy && (pointer.calibrated || actReady)
case .brain: return !testing && (app.modelOK == true || app.modelReady)
@@ -178,6 +178,8 @@ struct OnboardingView: View {
private func primary() {
switch page {
+ case .connect where !board.connected:
+ board.retryDiscovery()
case .see where !streaming:
app.startJoin()
case .act where !pointer.calibrated:
diff --git a/ios/IPhoneAgent/UI/ProviderFields.swift b/ios/IPhoneAgent/UI/ProviderFields.swift
new file mode 100644
index 0000000..01fbe01
--- /dev/null
+++ b/ios/IPhoneAgent/UI/ProviderFields.swift
@@ -0,0 +1,67 @@
+import SwiftUI
+
+/// One compact configuration form shared by onboarding and Settings.
+struct ProviderFields: View {
+ @ObservedObject var settings: Settings
+
+ var body: some View {
+ VStack(spacing: 0) {
+ LabeledContent("Provider") {
+ Picker("Provider", selection: $settings.selectedProvider) {
+ ForEach(ModelProvider.allCases) { provider in Text(verbatim: provider.title).tag(provider) }
+ }
+ .labelsHidden().pickerStyle(.menu).tint(.primary).lineLimit(1)
+ }
+ .frame(minHeight: 44)
+ Divider()
+ if settings.provider == .others {
+ LabeledContent("Base URL") {
+ TextField("Base URL", text: $settings.baseURL, prompt: Text(verbatim: "https://…/v1"))
+ .keyboardType(.URL).multilineTextAlignment(.trailing).labelsHidden()
+ }
+ .frame(minHeight: 44).padding(.vertical, Space.xs)
+ Divider()
+ }
+ LabeledContent("API key") {
+ SecureField("API key", text: $settings.apiKey,
+ prompt: Text(settings.provider == .others ? "Optional" : "Required"))
+ .multilineTextAlignment(.trailing).labelsHidden()
+ }
+ .frame(minHeight: 44).padding(.vertical, Space.xs)
+ Divider()
+ LabeledContent("Model") {
+ HStack(spacing: Space.xs) {
+ TextField("Model ID", text: $settings.model, prompt: Text("Choose or enter"))
+ .multilineTextAlignment(.trailing).labelsHidden()
+ if settings.loadingModels {
+ ProgressView().frame(minWidth: 44, minHeight: 44)
+ } else {
+ Menu("Choose Model", systemImage: "chevron.up.chevron.down") {
+ ForEach(settings.models, id: \.self) { id in
+ Button { settings.model = id } label: { Text(verbatim: id) }
+ }
+ Divider()
+ Button("Refresh Models", systemImage: "arrow.clockwise") {
+ Task { await settings.loadModels() }
+ }
+ .disabled(!settings.isConfigured)
+ }
+ .labelStyle(.iconOnly).tint(.secondary)
+ .frame(minWidth: 44, minHeight: 44)
+ }
+ }
+ }
+ .frame(minHeight: 44).padding(.vertical, Space.xs)
+ if let error = settings.modelsError {
+ Text(verbatim: error).font(.footnote).foregroundStyle(.secondary)
+ .frame(maxWidth: .infinity, alignment: .leading)
+ }
+ }
+ .textInputAutocapitalization(.never).autocorrectionDisabled()
+ .task(id: settings.baseURL + "\n" + settings.apiKey) {
+ guard settings.provider != .others, !settings.apiKey.isEmpty,
+ (try? await Task.sleep(for: .milliseconds(700))) != nil else { return }
+ await settings.loadModels()
+ }
+ }
+}
diff --git a/ios/IPhoneAgentTests/LLMTests.swift b/ios/IPhoneAgentTests/LLMTests.swift
index dd86312..766fa7f 100644
--- a/ios/IPhoneAgentTests/LLMTests.swift
+++ b/ios/IPhoneAgentTests/LLMTests.swift
@@ -36,6 +36,40 @@ final class StubProtocol: URLProtocol {
/// Port of server/tests/test_llm.py, plus streaming.
@MainActor final class LLMTests: XCTestCase {
+ func testProviderInferenceAndCustomEndpoints() {
+ XCTAssertEqual(ModelProvider.infer(baseURL: "https://openrouter.ai/api/v1/", model: "openrouter/free"), .openRouterFree)
+ XCTAssertEqual(ModelProvider.infer(baseURL: "https://api.openai.com/v1", model: "custom"), .openAI)
+ XCTAssertEqual(ModelProvider.infer(baseURL: "http://192.168.1.4:8080/v1", model: "local"), .others)
+ XCTAssertNotEqual(ModelProvider.qwen.baseURL, ModelProvider.qwenInternational.baseURL)
+ }
+
+ func testProviderSwitchResetsModelOverridesAndOthersKeepsEditableConfig() async throws {
+ let name = "provider-test-" + UUID().uuidString
+ let defaults = UserDefaults(suiteName: name)!
+ defer { defaults.removePersistentDomain(forName: name) }
+ let settings = Settings(defaults: defaults, info: [:])
+ settings.selectProvider(.openRouterFree)
+ XCTAssertEqual(settings.baseURL, "https://openrouter.ai/api/v1")
+ let model = try await settings.resolvedModel()
+ XCTAssertEqual(model, "openrouter/free")
+ settings.plannerModel = "old-planner"
+ settings.actorModel = "old-actor"
+ settings.selectProvider(.openAI)
+ XCTAssertEqual(settings.model, "")
+ XCTAssertEqual(settings.actorModel, "")
+ XCTAssertEqual(settings.plannerModel, "")
+ do { _ = try await settings.resolvedModel(); XCTFail("Must explicitly choose a model") } catch {}
+ settings.selectProvider(.others)
+ settings.baseURL = "http://192.168.1.4:8080/v1"
+ settings.model = "my-custom-vision-model"
+ XCTAssertTrue(settings.isConfigured)
+ let custom = try await settings.resolvedModel()
+ XCTAssertEqual(custom, "my-custom-vision-model")
+ XCTAssertEqual(settings.provider, .others)
+ settings.baseURL = "httpjunk://endpoint"
+ XCTAssertFalse(settings.isConfigured)
+ }
+
func client(_ model: String, extra: [String: Any] = [:], tokenParam: String = "auto") -> LLMClient {
let config = URLSessionConfiguration.ephemeral
config.protocolClasses = [StubProtocol.self]