Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.
Keep active run status consistent across list and review
Henry58992042e095
diff --git a/docs/run-state-consistency-2026-10-16.md b/docs/run-state-consistency-2026-10-16.mdnew file mode 100644index 0000000..4f26286--- /dev/null+++ b/docs/run-state-consistency-2026-10-16.md@@ -0,0 +1,19 @@+# Consistent active-run status++Source: `851a8c6`; only untracked `ios/fastlane/` existed in the source checkout. Changes used the isolated `codex/run-state-consistency` branch.++## Evidence++The connected phone's latest trace was `20261016-113321`. Its Live Activity was successfully requested at 11:33:21 China time, and the agent started at 11:33:30. The user had returned to the app at 11:33:27, so the foreground safety gate held the agent before its first model request. The board disconnected at 11:34:00.++The list correctly used the active agent state. Review instead used only the stored trace: no terminal step meant Failed, even while the run was active. Retry was disabled because that same agent was still running. Foreground holding also prevented connection recovery from being checked.++## Fix++List and review now share the same active-run outcome resolver. Review observes agent and run-store changes, shows the foreground/recovery reason and Stop for an active run, and hides inactive-run editing controls while active. Failed and stopped runs offer Retry. Intent-associated runs also display their actual outcome.++The foreground gate now checks screen recovery when the feed is lost, allowing existing bounded recovery and failure handling to run. It publishes an explicit Paused Live Activity state while waiting for the user to leave the app.++## Validation++All 81 simulator tests passed, including a regression for an empty active trace versus an abandoned trace. The final device build succeeded. Activity creation logs do not establish on-screen visibility when another app has a competing Live Activity; physical display behavior remains unverified.diff --git a/ios/IPhoneAgent/App/AppPresence.swift b/ios/IPhoneAgent/App/AppPresence.swiftindex f7b5c6c..b61bdc0 100644--- a/ios/IPhoneAgent/App/AppPresence.swift+++ b/ios/IPhoneAgent/App/AppPresence.swift@@ -47,9 +47,14 @@ extension AppModel {if ours { away = await leaveApp() }if !away {homeHint = String(localized: "Paused while you're here. Go to the Home Screen and I'll go on.")+ live.show(.pausedInApp, mood: .thinking)defer { homeHint = nil }while !inBackground {try Task.checkCancellation()+ if !feed.live {+ _ = try await agent.waitForScreen()+ live.show(.pausedInApp, mood: .thinking)+ }try await Task.sleep(nanoseconds: 300_000_000)}}diff --git a/ios/IPhoneAgent/App/IslandStatus.swift b/ios/IPhoneAgent/App/IslandStatus.swiftindex de0153a..fa5ef8e 100644--- a/ios/IPhoneAgent/App/IslandStatus.swift+++ b/ios/IPhoneAgent/App/IslandStatus.swift@@ -14,6 +14,12 @@ struct IslandStatus: Equatable {static let compactLimit = 16+ static var pausedInApp: Self {+ Self(short: String(localized: "Paused"),+ full: String(localized: "Paused while you're here. Go to the Home Screen and I'll go on."),+ look: nil, minimal: String(localized: "Wait"))+ }+static var looking: Self {Self(short: String(localized: "Looking…"), full: String(localized: "Looking at the screen…"), look: nil, minimal: String(localized: "Look"))}diff --git a/ios/IPhoneAgent/App/RunSession.swift b/ios/IPhoneAgent/App/RunSession.swiftindex 19c61e3..f5f37c7 100644--- a/ios/IPhoneAgent/App/RunSession.swift+++ b/ios/IPhoneAgent/App/RunSession.swift@@ -204,6 +204,10 @@ extension AppModel {// MARK: - review+ func outcome(for run: Run) -> Outcome {+ run.outcome(isActive: run.id == runID && (startingAgent || agent.running))+ }+/// Opens the review editor for a run (with its intent, if it ran one).func openReview(run id: String) {runs.reload()diff --git a/ios/IPhoneAgent/Localizable.xcstrings b/ios/IPhoneAgent/Localizable.xcstringsindex 0bdeeb5..920fca7 100644--- a/ios/IPhoneAgent/Localizable.xcstrings+++ b/ios/IPhoneAgent/Localizable.xcstrings@@ -1,6 +1,26 @@{"sourceLanguage" : "en","strings" : {+ "The run is still active. Return to the Home Screen to continue.": {+ "localizations": {+ "zh-Hans": {+ "stringUnit": {+ "state": "translated",+ "value": "任务仍在进行中。请返回手机主屏幕以继续。"+ }+ }+ }+ },+ "Paused": {+ "localizations": {+ "zh-Hans": {+ "stringUnit": {+ "state": "translated",+ "value": "已暂停"+ }+ }+ }+ },"The board connection was lost. Move the board closer and keep it powered on, then tap Retry.": {"localizations": {"zh-Hans": {diff --git a/ios/IPhoneAgent/Review/ReviewView.swift b/ios/IPhoneAgent/Review/ReviewView.swiftindex a3a853d..0b8efa2 100644--- a/ios/IPhoneAgent/Review/ReviewView.swift+++ b/ios/IPhoneAgent/Review/ReviewView.swift@@ -6,6 +6,20 @@ import SwiftUIstruct ReviewView: View {@ObservedObject var draft: ReviewDraft@ObservedObject var app: AppModel+ @ObservedObject var agent: AgentLoop+ @ObservedObject var runs: RunStore++ init(draft: ReviewDraft, app: AppModel) {+ self.draft = draft+ self.app = app+ self.agent = app.agent+ self.runs = app.runs+ }++ private var outcome: Outcome? {+ guard let id = draft.runID, let run = runs.run(id) else { return draft.outcome }+ return app.outcome(for: run)+ }@Environment(\.dismiss) private var dismiss@Environment(\.accessibilityReduceMotion) private var reduceMotion@State private var index = 0@@ -30,7 +44,20 @@ struct ReviewView: View {var body: some View {VStack(spacing: 0) {topBar- if draft.outcome == .failed {+ if outcome == .running {+ VStack(spacing: Space.s) {+ ProgressView()+ Text(verbatim: app.recovering?.status.full ?? app.homeHint+ ?? String(localized: "The run is still active. Return to the Home Screen to continue."))+ .font(.subheadline).multilineTextAlignment(.center)+ if !app.startingAgent {+ Button("Stop", systemImage: "stop.fill") { Task { await app.stopAgent() } }+ .buttonStyle(.bordered)+ }+ }+ .padding(Space.l)+ }+ if outcome == .failed {VStack(alignment: .leading, spacing: Space.s) {Label("Could not finish", systemImage: "exclamationmark.triangle").font(.headline)@@ -48,7 +75,8 @@ struct ReviewView: View {.padding(.horizontal, Space.xl).padding(.vertical, Space.s)}if steps.isEmpty {- IntentWordsView(draft: draft)+ if outcome == .running { Spacer() }+ else { IntentWordsView(draft: draft) }} else {if !video.isEmpty {Picker("View", selection: $showVideo) {@@ -73,7 +101,7 @@ struct ReviewView: View {}}.background(Color(uiColor: .systemBackground).ignoresSafeArea())- .safeAreaInset(edge: .bottom) { if !draft.readOnly { bottomBar } }+ .safeAreaInset(edge: .bottom) { if !draft.readOnly && outcome != .running { bottomBar } }.task(id: [playing, showVideo]) { await autoplay() }.onChange(of: showVideo) { playing = false }.onChange(of: videoTime) { _, t in // the step follows the video@@ -139,10 +167,12 @@ struct ReviewView: View {@ViewBuilder private var subtitle: some View {switch draft.source {case .recording: Label("Demonstration · check the yellow steps", systemImage: "record.circle").foregroundStyle(.secondary)- case .intent: Label("Intent", systemImage: draft.icon).foregroundStyle(.secondary)+ case .intent:+ if let o = outcome { OutcomeLabel(outcome: o) }+ else { Label("Intent", systemImage: draft.icon).foregroundStyle(.secondary) }case .run:if draft.readOnly { Label("Imported", systemImage: "square.and.arrow.down").foregroundStyle(.secondary) }- else if let o = draft.outcome { OutcomeLabel(outcome: o) }+ else if let o = outcome { OutcomeLabel(outcome: o) }}}@@ -318,7 +348,7 @@ struct ReviewView: View {private var bottomBar: some View {BottomBar {VStack(spacing: Space.s) {- if draft.runID != nil, draft.outcome == .failed {+ if draft.runID != nil, outcome == .failed || outcome == .stopped {Button("Retry", systemImage: "arrow.clockwise", action: retryRun).buttonStyle(BigButton()).disabled(retrying || app.startingAgent || app.agent.running || app.recorder.recording)diff --git a/ios/IPhoneAgent/Services/Stores.swift b/ios/IPhoneAgent/Services/Stores.swiftindex 500f719..589750e 100644--- a/ios/IPhoneAgent/Services/Stores.swift+++ b/ios/IPhoneAgent/Services/Stores.swift@@ -113,6 +113,9 @@ struct Run: Identifiable, Equatable {/// From someone else's shared archive: replay only.var imported: Bool { RunShare.isImported(folder) }var outcome: Outcome { Outcome(last: steps.last?.name, running: false) }+ func outcome(isActive: Bool) -> Outcome {+ Outcome(last: steps.last?.name, running: isActive)+ }var cover: URL? { steps.last(where: { $0.frame != nil })?.frame }var firstFrame: URL? { steps.first(where: { $0.frame != nil })?.frame }var duration: Double? {diff --git a/ios/IPhoneAgent/UI/RunsView.swift b/ios/IPhoneAgent/UI/RunsView.swiftindex fa13eba..6d55eb9 100644--- a/ios/IPhoneAgent/UI/RunsView.swift+++ b/ios/IPhoneAgent/UI/RunsView.swift@@ -45,7 +45,7 @@ struct RunsView: View {}private func isRunning(_ run: Run) -> Bool {- agent.running && run.id == app.traces.dir?.lastPathComponent+ app.outcome(for: run) == .running}}diff --git a/ios/IPhoneAgentTests/StoreTests.swift b/ios/IPhoneAgentTests/StoreTests.swiftindex e8d3b15..f7bf67b 100644--- a/ios/IPhoneAgentTests/StoreTests.swift+++ b/ios/IPhoneAgentTests/StoreTests.swift@@ -3,6 +3,13 @@ import XCTest/// Runs read back from trace folders (what the Runs list and the replay player show), and saved intents.@MainActor final class StoreTests: XCTestCase {+ func testEmptyActiveRunIsNotFailedButAbandonedRunIs() {+ let run = Run(id: "pending", folder: URL(fileURLWithPath: "/tmp/pending"),+ task: "Open Notes", date: Date(), steps: [])+ XCTAssertEqual(run.outcome(isActive: true), .running)+ XCTAssertEqual(run.outcome(isActive: false), .failed)+ }+func testRunIsReadBackWithFramesOutcomeAndDuration() throws {let root = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)let traces = TraceStore(root: root)