Skip to content

Liquid Glass and Apple Music touches for the Spotify iOS app. A GPL-3.0 fork of spoti.pw 0.21.1.

C

chore: the phone driver takes Mac commands over USB or an authenticated tailnet

My-Name-Is-Jeff06aa13883ee8
diff --git a/docs/tweaks.md b/docs/tweaks.md
index bef3b894..aff7278c 100644
--- a/docs/tweaks.md
+++ b/docs/tweaks.md
@@ -17,7 +17,7 @@
tweak/Sources/Redesigned/ the redesign, running only while Redesigned UI is on
tweak/Sources/App/ what brings the layers together: Mod Settings' root and composed pages, the Mod page,
backup and signing, the welcome tour
- tweak/Sources/Diagnostics/ screen dumps, the tree server and the main thread hang sampler of FLEX builds
+ tweak/Sources/Diagnostics/ screen dumps, the tree server, the phone driver and the main thread hang sampler of FLEX builds
extension/LiveActivity/ the Live Activity widget, a WidgetKit extension of its own
extension/AppGroups/ a dylib loaded by Spotify and its home screen widget that moves Spotify's App Groups
into a group the re-signed IPA has; without it the widget stays a placeholder
@@ -514,6 +514,46 @@ and Live Activity. The root page in `App/ModSettings.x` links the Appearance pag
make log # stream [spotifyglass] log lines from the phone
make flags # regenerate the flag table from the IPA
+## Diagnostics
+
+A build with FLEX in it (`make build`, `make install FLEX=1`) runs `Diagnostics/`: the tree server, the hang
+sampler, and the phone driver, which lets an agent on the Mac work the app without anyone touching the phone.
+The server listens on the phone's loopback only, 127.0.0.1:8085, which the Mac reaches through iproxy over
+USB and nothing on Wi-Fi reaches. `GET /tree` answers with the visible screen's view tree, as `make trees`
+records it. The driver adds `GET /<command>?<params>`, each answering JSON with `ok` and either what happened
+or an `error`. A command needs an `X-Phone-Driver` header, which a web page on the phone cannot send to the
+loopback without a CORS preflight the server never grants (`curl -H 'X-Phone-Driver: 1'`). It is compiled only with `SG_DRIVER=1`, which `scripts/pipeline.sh` sets for the builds that
+carry FLEX; `make release` and the release workflow's `.deb` leave it out, and it starts only when FLEX is
+there.
+
+`scripts/phone.py` is the client. It uses whatever already answers on 127.0.0.1:8085 or starts iproxy and
+leaves it running:
+
+ scripts/phone.py tree | state | log [--since N]
+ scripts/phone.py find --class UILabel
+ scripts/phone.py tap --id X | --label X | --text X | --class C [--index N] | --at X,Y
+ scripts/phone.py longpress --id X [--duration 1]
+ scripts/phone.py swipe --from X,Y --to X,Y [--duration 0.2] [--drag 1]
+ scripts/phone.py scroll [--id X] --by 0,400
+ scripts/phone.py type "text"
+ scripts/phone.py player.open | player.close | player.more | play | pause | next | seek 42
+ scripts/phone.py menu.pick "Sleep timer"
+ scripts/phone.py tab 0 | tab Search
+ scripts/phone.py settings.open | settings.page "Appearance"
+ scripts/phone.py wait --id X [--gone 1] | --menu 1 | --log "system menu: the player's menu is up" [--timeout 3]
+ scripts/phone.py screenshot out.png
+
+Views are addressed as the tree prints them: `id=` is the accessibility identifier, `a11y=` the accessibility
+label, the quoted text a label's, and a class with an index counts that class's views in the tree's order.
+Points are screen points, the frames `find` prints. Touches are made as a finger's (`harness/tabbar/touch.m`'s
+way: a UITouch and the IOHIDEvent gesture recognizers read, through `-[UIApplication sendEvent:]`), so
+controls get their control events, recognizers recognize, and a button whose menu is its primary action opens
+it on the touch down. `menu.pick` taps a row of the system's menu, not of Spotify's own sheet (tap that by
+`--text`). `log` reads the app's last 2000 `SGLog` lines from memory, so no device log capture is needed;
+a log wait looks from the start of the last command that did something. Everything runs on the main thread
+through `dispatch_async` and a semaphore with a timeout, waited on from the server's thread only. Checked in
+the simulator by harness/driver/.
+
## Mod Settings
Mod Settings, opened by holding Home on the tab bar or from the first row of the side drawer and the
diff --git a/harness/driver/README.md b/harness/driver/README.md
new file mode 100644
index 00000000..c8a5f953
--- /dev/null
+++ b/harness/driver/README.md
@@ -0,0 +1,21 @@
+# Phone driver harness
+
+`Diagnostics/Diagnostics.x`'s tree server and `Diagnostics/Driver.m`, compiled as a FLEX build compiles them
+(`SG_DRIVER=1`), run in the simulator on a mock app, and `proof.py` drives it through `scripts/phone.py` the
+way an agent drives the phone. Simulator apps share the Mac's loopback, so no iproxy; the harness serves on
+127.0.0.1:8095 (`PORT=` at build time) so an iproxy to the phone keeps 8085.
+
+ THEOS=$HOME/theos ./build.sh
+ xcrun simctl install <udid> build/DriverHarness.app
+ xcrun simctl launch <udid> com.vitrine.driverharness
+ ./proof.py
+
+The mock (`main.m`): a tab bar (Home, Search); on Home a button with a touch-up-inside action, a view with a
+tap recognizer, a button whose menu is its primary action, a scroll view and a text field; a now playing bar
+under Spotify's bar controller's class name that presents a player under `NPVScrollViewController`'s, with the
+down arrow and the ⋯ by Spotify's identifiers and a system menu on the ⋯; a player behind
+`SGDiagnosticsPlayer`; and `SGOpenModSettings` pushing a 40-row list. Each reacts with an `SGLog` line, read
+back through the driver's `log` and `wait --log`, and `proof.py` prints a `PASS` or `FAIL` line per check:
+the button's action, the recognizer, the menu up before the finger lifts, a pick, a scroll, a fling, typing,
+a screenshot, the player's open, ⋯, pick and close, seek/pause/play/next, tabs by title and index, Mod
+Settings and a row below its fold, and the tree served while a wait runs.
diff --git a/harness/driver/build.sh b/harness/driver/build.sh
new file mode 100755
index 00000000..ccce2d22
--- /dev/null
+++ b/harness/driver/build.sh
@@ -0,0 +1,41 @@
+#!/bin/sh
+# Builds the phone driver harness for the simulator: Diagnostics.x's tree server and Driver.m, as a FLEX build
+# compiles them (SG_DRIVER=1), on a mock app (main.m). It serves on 127.0.0.1:8095 (PORT=), not the phone's
+# 8085, so an iproxy to the phone keeps its port.
+set -e
+SRC=$(cd "$(dirname "$0")/../../tweak/Sources" && pwd)
+OUT=$(dirname "$0")/build
+rm -rf "$OUT"; mkdir -p "$OUT/gen" "$OUT/DriverHarness.app"
+"$THEOS/bin/logos.pl" -c generator=internal "$SRC/Diagnostics/Diagnostics.x" > "$OUT/gen/Diagnostics.m"
+
+SDK=$(xcrun --sdk iphonesimulator --show-sdk-path)
+xcrun -sdk iphonesimulator clang -target arm64-apple-ios17.4-simulator -fobjc-arc -g -O0 -DSG_DRIVER=1 -DSG_TREE_PORT=${PORT:-8095} \
+ -I"$SRC" -I"$SRC/Diagnostics" -isysroot "$SDK" -Wno-deprecated-declarations \
+ "$(dirname "$0")/main.m" "$(dirname "$0")/../scene.m" "$OUT"/gen/*.m "$SRC"/Diagnostics/Driver.m \
+ "$SRC"/Core/SGLog.m "$SRC"/Core/SGViewTree.m \
+ -framework UIKit -framework QuartzCore -framework CoreGraphics -framework Foundation \
+ -o "$OUT/DriverHarness.app/DriverHarness"
+
+cat > "$OUT/DriverHarness.app/Info.plist" <<'PLIST'
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0"><dict>
+<key>CFBundleExecutable</key><string>DriverHarness</string>
+<key>CFBundleIdentifier</key><string>com.vitrine.driverharness</string>
+<key>CFBundleName</key><string>DriverHarness</string>
+<key>CFBundleVersion</key><string>1</string>
+<key>CFBundleShortVersionString</key><string>1.0</string>
+<key>UIUserInterfaceStyle</key><string>Dark</string>
+<key>UILaunchScreen</key><dict/>
+<key>UIApplicationSceneManifest</key><dict>
+ <key>UIApplicationSupportsMultipleScenes</key><false/>
+ <key>UISceneConfigurations</key><dict>
+ <key>UIWindowSceneSessionRoleApplication</key><array><dict>
+ <key>UISceneConfigurationName</key><string>Default</string>
+ <key>UISceneDelegateClassName</key><string>SGRHarnessScene</string>
+ </dict></array>
+ </dict>
+</dict>
+</dict></plist>
+PLIST
+echo "built $OUT/DriverHarness.app"
diff --git a/harness/driver/main.m b/harness/driver/main.m
new file mode 100644
index 00000000..6eb9e772
--- /dev/null
+++ b/harness/driver/main.m
@@ -0,0 +1,270 @@
+// The phone driver (Diagnostics/Driver.m) on a mock app in the simulator: Diagnostics.x's tree server starts
+// as in a FLEX build (a FLEXManager class is all it looks for) and proof.py drives it through scripts/phone.py
+// over the Mac's loopback, which simulator apps share, so no iproxy.
+//
+// The mock: a tab bar (Home, Search); on Home a button, a view with a tap recognizer, a button whose menu is
+// its primary action, a scroll view and a text field; a now playing bar under Spotify's bar controller's class
+// name that opens a player under NPVScrollViewController's, with the down arrow and the ⋯ by Spotify's
+// identifiers and a menu on the ⋯; a player behind SGDiagnosticsPlayer; and SGOpenModSettings pushing a long
+// list. Everything that reacts says so with an SGLog line, which the driver's log command reads back.
+#import <UIKit/UIKit.h>
+#import "Core/SGCore.h"
+#import "Diagnostics/Diagnostics.h"
+
+// Diagnostics.x's SGIsDebugBuild looks for FLEX by this class.
+@interface FLEXManager : NSObject
+@end
+@implementation FLEXManager
+@end
+
+#pragma mark - the player
+
+@interface HarnessTrack : NSObject
+@property (nonatomic, readonly) NSString *trackTitle, *artistName;
+@property (nonatomic, readonly) id URI;
+@end
+@implementation HarnessTrack
+- (NSString *)trackTitle { return @"Harness Song"; }
+- (NSString *)artistName { return @"Harness Artist"; }
+- (id)URI { return @"spotify:track:harness"; }
+@end
+
+@interface HarnessState : NSObject
+@property (nonatomic) BOOL paused;
+@property (nonatomic) double at;
+@end
+@implementation HarnessState
+- (HarnessTrack *)track { return [HarnessTrack new]; }
+- (BOOL)isPaused { return self.paused; }
+- (BOOL)isPlaying { return YES; }
+- (double)position { return self.at; }
+- (double)positionAsOfTimestamp { return self.at; }
+- (double)duration { return 200; }
+@end
+
+@interface HarnessPlayer : NSObject
+@property (nonatomic, strong) HarnessState *state;
+@end
+@implementation HarnessPlayer
+- (instancetype)init {
+ if ((self = [super init])) _state = [HarnessState new];
+ return self;
+}
+- (void)seekTo:(double)seconds { self.state.at = seconds; SGLog(@"harness: player seekTo %.0f", seconds); }
+- (id)pause:(id)options { self.state.paused = YES; SGLog(@"harness: player pause"); return nil; }
+- (id)resume:(id)options { self.state.paused = NO; SGLog(@"harness: player resume"); return nil; }
+- (id)skipToNextTrackWithOptions:(id)options { SGLog(@"harness: player next"); return nil; }
+@end
+
+static HarnessPlayer *sg_player;
+id SGDiagnosticsPlayer(void) { return sg_player; }
+
+#pragma mark - Mod Settings
+
+@interface HarnessSettings : UITableViewController
+@end
+@implementation HarnessSettings
+- (void)viewDidLoad {
+ [super viewDidLoad];
+ self.title = @"Mod Settings";
+ [self.tableView registerClass:UITableViewCell.class forCellReuseIdentifier:@"row"];
+}
+- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section { return 40; }
+- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
+ UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:@"row" forIndexPath:indexPath];
+ cell.textLabel.text = [NSString stringWithFormat:@"Row %ld", (long)indexPath.row];
+ return cell;
+}
+- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {
+ SGLog(@"harness: settings row Row %ld", (long)indexPath.row);
+}
+@end
+
+static UINavigationController *sg_home;
+void SGOpenModSettings(UIView *source) {
+ SGLog(@"harness: SGOpenModSettings from a %@", source.class);
+ [sg_home pushViewController:[HarnessSettings new] animated:NO];
+}
+
+#pragma mark - the player screen and the bar, under Spotify's class names
+
+static UIButton *menuButton(NSString *title, NSString *identifier, NSArray<NSString *> *rows, NSString *what);
+
+@interface _TtC21NowPlaying_ScrollImpl23NPVScrollViewController : UIViewController
+@end
+@implementation _TtC21NowPlaying_ScrollImpl23NPVScrollViewController
+- (void)viewDidLoad {
+ [super viewDidLoad];
+ self.view.backgroundColor = UIColor.darkGrayColor;
+ UIButton *close = [UIButton buttonWithType:UIButtonTypeSystem];
+ [close setTitle:@"Down" forState:UIControlStateNormal];
+ close.accessibilityIdentifier = @"now-playing-minimize-button";
+ close.frame = CGRectMake(12, 70, 48, 48);
+ [close addTarget:self action:@selector(close) forControlEvents:UIControlEventTouchUpInside];
+ [self.view addSubview:close];
+ UIButton *more = menuButton(@"⋯", @"Context menu", @[@"Share", @"Go to album", @"Sleep timer"], @"player menu");
+ more.frame = CGRectMake(340, 70, 48, 48);
+ [self.view addSubview:more];
+}
+- (void)close {
+ SGLog(@"harness: player down arrow");
+ [self dismissViewControllerAnimated:NO completion:nil];
+}
+@end
+
+@interface _TtC18NowPlaying_BarImpl27NowPlayingBarViewController : UIViewController
+@end
+@implementation _TtC18NowPlaying_BarImpl27NowPlayingBarViewController
+- (void)viewDidLoad {
+ [super viewDidLoad];
+ self.view.backgroundColor = UIColor.systemIndigoColor;
+ [self.view addGestureRecognizer:[[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(open)]];
+}
+- (void)open {
+ SGLog(@"harness: bar tapped");
+ UIViewController *player = [_TtC21NowPlaying_ScrollImpl23NPVScrollViewController new];
+ player.modalPresentationStyle = UIModalPresentationFullScreen;
+ [self presentViewController:player animated:NO completion:nil];
+}
+@end
+
+#pragma mark - Home
+
+// A button whose menu is its primary action; says when the menu is about to show.
+@interface HarnessMenuButton : UIButton
+@property (nonatomic, copy) NSString *what;
+@end
+@implementation HarnessMenuButton
+- (void)contextMenuInteraction:(UIContextMenuInteraction *)interaction
+ willDisplayMenuForConfiguration:(UIContextMenuConfiguration *)configuration
+ animator:(id<UIContextMenuInteractionAnimating>)animator {
+ SGLog(@"harness: %@ will display", self.what);
+ if ([UIButton instancesRespondToSelector:_cmd]) [super contextMenuInteraction:interaction willDisplayMenuForConfiguration:configuration animator:animator];
+}
+@end
+
+static UIButton *menuButton(NSString *title, NSString *identifier, NSArray<NSString *> *rows, NSString *what) {
+ HarnessMenuButton *button = [HarnessMenuButton buttonWithType:UIButtonTypeSystem];
+ button.what = what;
+ [button setTitle:title forState:UIControlStateNormal];
+ button.accessibilityIdentifier = identifier;
+ NSMutableArray<UIAction *> *actions = [NSMutableArray array];
+ for (NSString *row in rows) {
+ [actions addObject:[UIAction actionWithTitle:row image:nil identifier:nil handler:^(UIAction *action) {
+ SGLog(@"harness: %@ picked %@", what, row);
+ }]];
+ }
+ button.menu = [UIMenu menuWithChildren:actions];
+ button.showsMenuAsPrimaryAction = YES;
+ return button;
+}
+
+@interface HarnessHome : UIViewController
+@property (nonatomic) NSInteger taps;
+@end
+@implementation HarnessHome
+- (void)viewDidLoad {
+ [super viewDidLoad];
+ self.title = @"Home";
+ self.view.backgroundColor = UIColor.blackColor;
+
+ UIButton *counter = [UIButton buttonWithType:UIButtonTypeSystem];
+ counter.frame = CGRectMake(20, 120, 160, 44);
+ counter.accessibilityIdentifier = @"counter";
+ [counter setTitle:@"Counter" forState:UIControlStateNormal];
+ [counter addTarget:self action:@selector(count:) forControlEvents:UIControlEventTouchUpInside];
+ [self.view addSubview:counter];
+
+ UIView *tapView = [[UIView alloc] initWithFrame:CGRectMake(220, 120, 160, 44)];
+ tapView.backgroundColor = UIColor.systemTealColor;
+ tapView.accessibilityIdentifier = @"tapview";
+ [tapView addGestureRecognizer:[[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(recognized)]];
+ [self.view addSubview:tapView];
+
+ UIButton *menu = menuButton(@"Menu", @"menubutton", @[@"Alpha", @"Sleep timer"], @"menu");
+ menu.frame = CGRectMake(20, 180, 160, 44);
+ [self.view addSubview:menu];
+
+ UIScrollView *scroller = [[UIScrollView alloc] initWithFrame:CGRectMake(20, 240, 362, 220)];
+ scroller.accessibilityIdentifier = @"scroller";
+ scroller.backgroundColor = [UIColor colorWithWhite:0.15 alpha:1];
+ scroller.contentSize = CGSizeMake(362, 3000);
+ for (NSInteger i = 0; i < 60; i++) {
+ UILabel *line = [[UILabel alloc] initWithFrame:CGRectMake(12, i * 50, 300, 44)];
+ line.text = [NSString stringWithFormat:@"Line %ld", (long)i];
+ line.textColor = UIColor.whiteColor;
+ [scroller addSubview:line];
+ }
+ [self.view addSubview:scroller];
+
+ UITextField *field = [[UITextField alloc] initWithFrame:CGRectMake(20, 480, 362, 40)];
+ field.accessibilityIdentifier = @"field";
+ field.borderStyle = UITextBorderStyleRoundedRect;
+ field.placeholder = @"Type here";
+ [field addTarget:self action:@selector(edited:) forControlEvents:UIControlEventEditingChanged];
+ [self.view addSubview:field];
+
+ UIViewController *bar = [_TtC18NowPlaying_BarImpl27NowPlayingBarViewController new];
+ [self addChildViewController:bar];
+ bar.view.frame = CGRectMake(8, 640, 386, 56);
+ [self.view addSubview:bar.view];
+ [bar didMoveToParentViewController:self];
+}
+- (void)count:(UIButton *)button {
+ self.taps++;
+ [button setTitle:[NSString stringWithFormat:@"Tapped %ld", (long)self.taps] forState:UIControlStateNormal];
+ SGLog(@"harness: button tapped %ld", (long)self.taps);
+}
+- (void)recognized {
+ SGLog(@"harness: recognizer tap");
+}
+- (void)edited:(UITextField *)field {
+ SGLog(@"harness: field text %@", field.text);
+}
+@end
+
+@interface HarnessSearch : UIViewController
+@end
+@implementation HarnessSearch
+- (void)viewDidLoad {
+ [super viewDidLoad];
+ self.title = @"Search";
+ self.view.backgroundColor = UIColor.systemGray6Color;
+}
+@end
+
+// Says when a finger lifts, so the proof can tell a menu opened while it was still down.
+@interface HarnessWindow : UIWindow
+@end
+@implementation HarnessWindow
+- (void)sendEvent:(UIEvent *)event {
+ for (UITouch *touch in event.allTouches) {
+ if (touch.phase == UITouchPhaseEnded) SGLog(@"harness: finger up");
+ }
+ [super sendEvent:event];
+}
+@end
+
+@interface SGHarnessApp : UIResponder <UIApplicationDelegate>
+@property (nonatomic, strong) UIWindow *window;
+@end
+@implementation SGHarnessApp
+- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)options {
+ sg_player = [HarnessPlayer new];
+ sg_home = [[UINavigationController alloc] initWithRootViewController:[HarnessHome new]];
+ sg_home.tabBarItem = [[UITabBarItem alloc] initWithTitle:@"Home" image:[UIImage systemImageNamed:@"house"] tag:0];
+ UIViewController *search = [HarnessSearch new];
+ search.tabBarItem = [[UITabBarItem alloc] initWithTitle:@"Search" image:[UIImage systemImageNamed:@"magnifyingglass"] tag:1];
+ UITabBarController *tabs = [UITabBarController new];
+ tabs.viewControllers = @[sg_home, search];
+ self.window = [[HarnessWindow alloc] initWithFrame:UIScreen.mainScreen.bounds];
+ self.window.rootViewController = tabs;
+ return YES;
+}
+@end
+
+int main(int argc, char *argv[]) {
+ @autoreleasepool {
+ return UIApplicationMain(argc, argv, nil, NSStringFromClass(SGHarnessApp.class));
+ }
+}
diff --git a/harness/driver/proof.py b/harness/driver/proof.py
new file mode 100755
index 00000000..47a020d9
--- /dev/null
+++ b/harness/driver/proof.py
@@ -0,0 +1,127 @@
+#!/usr/bin/env python3
+"""Drives the driver harness (main.m) through scripts/phone.py and prints PASS or FAIL per check.
+
+ THEOS=$HOME/theos ./build.sh
+ xcrun simctl install <udid> build/DriverHarness.app && xcrun simctl launch <udid> com.vitrine.driverharness
+ ./proof.py # PHONE_PORT=8095 unless set, the port build.sh serves on
+"""
+import json
+import os
+import struct
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+PHONE = os.path.join(HERE, "..", "..", "scripts", "phone.py")
+os.environ.setdefault("PHONE_PORT", "8095")
+failures = 0
+
+
+def phone(*args):
+ out = subprocess.run([sys.executable, PHONE, *args], capture_output=True, text=True).stdout
+ return out if args[0] == "tree" else json.loads(out)
+
+
+def check(ok, what):
+ global failures
+ failures += not ok
+ print(f"{'PASS' if ok else 'FAIL'} {what}")
+
+
+def lines(since):
+ return [line["text"] for line in phone("log", "--since", str(since))["lines"]]
+
+
+def last():
+ return phone("state")["logSeq"]
+
+
+phone("tab", "Home")
+check("== window" in phone("tree"), "tree: GET /tree still answers with the tree")
+check(phone("state")["top"] == "HarnessHome", "state: the top controller is Home")
+
+start = last()
+answer = phone("tap", "--id", "counter")
+check(answer["ok"] and answer["hit"] == "UIButton", "tap --id: the finger lands on the button")
+check(any(t.startswith("harness: button tapped") for t in lines(start)), "tap --id: the button's touch-up-inside action ran")
+
+start = last()
+phone("tap", "--id", "tapview")
+check("harness: recognizer tap" in lines(start), "tap: a UITapGestureRecognizer recognized the touch")
+
+start = last()
+phone("longpress", "--id", "menubutton", "--duration", "1")
+log = lines(start)
+check("harness: menu will display" in log and "harness: finger up" in log
+ and log.index("harness: menu will display") < log.index("harness: finger up"),
+ "longpress: a primary-action menu opens on the touch down, before the finger lifts")
+check(phone("state")["menu"] is True, "state: menu is up")
+start = last()
+check(phone("menu.pick", "Alpha")["ok"], "menu.pick: the row is found and tapped")
+check(phone("wait", "--log", "harness: menu picked Alpha", "--timeout", "2")["ok"], "menu.pick: the row's action ran")
+check(phone("wait", "--menu", "1", "--gone", "1", "--timeout", "2")["ok"], "wait --menu --gone: the menu closed")
+
+answer = phone("scroll", "--id", "scroller", "--by", "0,500")
+check(answer["ok"] and answer["after"] == [0, 500], "scroll: the scroll view moved by the offset")
+phone("swipe", "--id", "scroller", "--by", "0,-300")
+time.sleep(1.5)
+offset = phone("find", "--id", "scroller")["views"][0]["contentOffset"][1]
+check(offset > 850, f"swipe: a real fling moved the list past the 300 pt of the swipe ({offset:.0f})")
+
+start = last()
+phone("tap", "--id", "field")
+phone("type", "hello driver")
+check("harness: field text hello driver" in lines(start), "type: the text went into the tapped field")
+
+shot = os.path.join(tempfile.gettempdir(), f"driver-proof-{os.getpid()}.png")
+answer = phone("screenshot", shot)
+with open(shot, "rb") as f:
+ head = f.read(24)
+width, height = struct.unpack(">II", head[16:24])
+check(answer["ok"] and head[:8] == b"\x89PNG\r\n\x1a\n" and width > 1000, f"screenshot: a {width}x{height} PNG came back")
+os.remove(shot)
+
+check(phone("player.open")["ok"] and phone("wait", "--id", "Context menu")["ok"], "player.open: the bar opened the player")
+check(phone("player.more")["ok"], "player.more: the ⋯ was tapped")
+check(phone("wait", "--log", "harness: player menu will display", "--timeout", "3")["ok"], "wait --log: the ⋯'s menu came up")
+phone("menu.pick", "Sleep timer")
+check(phone("wait", "--log", "harness: player menu picked Sleep timer", "--timeout", "3")["ok"], "menu.pick: Sleep timer picked in the player's menu")
+phone("player.close")
+check(phone("state")["player"] is False, "player.close: the down arrow closed the player")
+
+start = last()
+phone("seek", "42")
+phone("pause")
+state = phone("state")["nowPlaying"]
+phone("play")
+phone("next")
+log = lines(start)
+check(state["position"] == 42 and state["paused"] and {"harness: player seekTo 42", "harness: player pause", "harness: player resume", "harness: player next"} <= set(log),
+ "seek, pause, play, next: the player was told, and state reads it back")
+
+phone("tab", "Search")
+check(phone("state")["top"] == "HarnessSearch", "tab by title: Search is shown")
+phone("tab", "0")
+check(phone("state")["top"] == "HarnessHome", "tab by index: Home is shown again")
+
+phone("settings.open")
+check(phone("state")["top"] == "HarnessSettings", "settings.open: Mod Settings is pushed")
+check(phone("settings.page", "Row 33")["ok"] and phone("wait", "--log", "harness: settings row Row 33")["ok"],
+ "settings.page: a row below the fold is scrolled to and tapped")
+
+check(phone("tap", "--id", "nope")["error"] == "no view matches id=nope", "tap: a missing view is an error, not a crash")
+
+# The tree still answers while a command waits.
+waiting = threading.Thread(target=phone, args=("wait", "--log", "never", "--timeout", "2"))
+waiting.start()
+time.sleep(0.3)
+began = time.time()
+phone("tree")
+check(time.time() - began < 1 and waiting.is_alive(), "the tree is served while a wait runs")
+waiting.join()
+
+print("done," , "all passed" if not failures else f"{failures} failed")
+sys.exit(1 if failures else 0)
diff --git a/scripts/phone.py b/scripts/phone.py
new file mode 100755
index 00000000..793a53ce
--- /dev/null
+++ b/scripts/phone.py
@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Drive the app on the iPhone from the Mac: the phone driver of FLEX builds (Diagnostics/Driver.m).
+
+ scripts/phone.py tree # the visible screen's view tree, as make trees records it
+ scripts/phone.py state # now playing, the top controller, what is presented, menu up
+ scripts/phone.py find --class UILabel # every match with its index, frame (screen points), id, label, text
+ scripts/phone.py tap --id X | --label X | --text X | --class C [--index N] | --at X,Y
+ scripts/phone.py longpress --id X [--duration 1]
+ scripts/phone.py swipe --from X,Y --to X,Y [--duration 0.2] # or a view and --by DX,DY; a quick one flings
+ scripts/phone.py swipe --id X --by 0,-300 --drag 1 # a slow drag held still before letting go
+ scripts/phone.py scroll [--id X | --at X,Y] --by 0,400 [--animated 1] # the page's list when no view is named
+ scripts/phone.py type "some text" # into the first responder (tap a field first)
+ scripts/phone.py player.open | player.close | player.more
+ scripts/phone.py menu.pick "Sleep timer"
+ scripts/phone.py play | pause | next
+ scripts/phone.py seek 42
+ scripts/phone.py tab 0 | tab Search
+ scripts/phone.py settings.open
+ scripts/phone.py settings.page "Appearance"
+ scripts/phone.py wait --id X [--gone 1] [--timeout 5]
+ scripts/phone.py wait --menu 1 [--gone 1]
+ scripts/phone.py wait --log "system menu: the player's menu is up" --timeout 3
+ scripts/phone.py log [--since N] # the app's last SGLog lines; each has a seq
+ scripts/phone.py screenshot out.png [--scale 1]
+
+Every command prints the app's JSON answer, {"ok": true, ...} or {"ok": false, "error": ...}, and exits 1
+when it is not ok. --label and --text match whole and ignore case; --contains 1 matches a part. Points are
+screen points, the frame `find` prints. A log wait looks at the lines since the last command that did
+something began, so `player.more` then `wait --log ...` sees a line that came before the wait.
+
+The server listens on the phone's loopback only, port 8085, so this goes through iproxy over USB: it uses
+whatever already answers on 127.0.0.1:8085 (an iproxy left running, or the simulator, whose apps share the
+Mac's loopback) and otherwise starts iproxy and leaves it running for the next call. PHONE_PORT changes the
+port. Over Tailscale: PHONE_HOST=100.71.6.75 scripts/phone.py state (PHONE_TOKEN, or the token from the
+device log; FLEX builds listen on the phone's Tailscale address too, token required). Needs a FLEX build (make install FLEX=1) open in the foreground: iOS suspends it in the background.
+"""
+import base64
+import json
+import os
+import re
+import shutil
+import socket
+import subprocess
+import sys
+import time
+import urllib.error
+import urllib.parse
+import urllib.request
+
+PORT = int(os.environ.get("PHONE_PORT", "8085"))
+# Over Tailscale instead of the cable: the phone's tailnet address and the token the app logs at launch
+# ("driver: tailnet listener on 100.x.y.z:8085, token ..."), read from the device log when not given.
+HOST = os.environ.get("PHONE_HOST")
+LOG = "/tmp/claude-501/device.log"
+
+
+def token():
+ if os.environ.get("PHONE_TOKEN"):
+ return os.environ["PHONE_TOKEN"]
+ try:
+ with open(LOG, errors="replace") as f:
+ found = re.findall(r"driver: tailnet listener on \S+, token ([0-9a-f]{32})", f.read())
+ except OSError:
+ found = []
+ if not found:
+ sys.exit(f"no PHONE_TOKEN and no token in {LOG}: run once over the cable, or read it from the app's log")
+ return found[-1]
+# The argument a command takes without a flag.
+POSITIONAL = {"type": "text", "menu.pick": "title", "seek": "seconds", "tab": "arg", "settings.page": "title"}
+
+
+def port_open():
+ try:
+ with socket.create_connection(("127.0.0.1", PORT), timeout=0.5):
+ return True
+ except OSError:
+ return False
+
+
+def ensure_tunnel():
+ if port_open():
+ return
+ if not shutil.which("iproxy"):
+ sys.exit(f"nothing answers on 127.0.0.1:{PORT} and there is no iproxy (brew install libimobiledevice)")
+ subprocess.Popen(["iproxy", f"{PORT}:{PORT}"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
+ for _ in range(30):
+ if port_open():
+ return
+ time.sleep(0.1)
+ sys.exit(f"iproxy did not open 127.0.0.1:{PORT}: is the iPhone plugged in and trusted?")
+
+
+def fetch(path, timeout):
+ # The header the app asks of a driver command, which a web page on the phone cannot send.
+ headers = {"X-Phone-Driver": "1"}
+ if HOST:
+ headers["X-Phone-Token"] = token()
+ url = f"http://{HOST}:{PORT}/{path}"
+ else:
+ ensure_tunnel()
+ url = f"http://127.0.0.1:{PORT}/{path}"
+ request = urllib.request.Request(url, headers=headers)
+ try:
+ with urllib.request.urlopen(request, timeout=timeout) as response:
+ return response.read().decode("utf-8")
+ except urllib.error.HTTPError as e:
+ return e.read().decode("utf-8", "replace")
+ except (urllib.error.URLError, ConnectionError, socket.timeout) as e:
+ sys.exit(f"the app did not answer ({e}): is a FLEX build open in the foreground?")
+
+
+def parse(args):
+ """`cmd --key value ... [positional]` into the command and its query items."""
+ params, rest = {}, list(args)
+ while rest:
+ arg = rest.pop(0)
+ if arg.startswith("--") and len(arg) > 2:
+ key = arg[2:]
+ params[key] = rest.pop(0) if rest and not rest[0].startswith("--") else "1"
+ else:
+ params.setdefault("arg", arg)
+ return params
+
+
+def run(argv):
+ if not argv or argv[0] in ("-h", "--help", "help"):
+ print(__doc__)
+ return 0
+ command, params = argv[0], parse(argv[1:])
+ if command == "tree":
+ print(fetch("tree", 15), end="")
+ return 0
+ out = None
+ if command == "screenshot":
+ out = params.pop("arg", None)
+ if not out:
+ sys.exit("screenshot takes the path to save the PNG to")
+ if "arg" in params and command in POSITIONAL:
+ params[POSITIONAL[command]] = params.pop("arg")
+ timeout = float(params.get("timeout", 5)) + float(params.get("duration", 0)) + 15
+ text = fetch(f"{urllib.parse.quote(command)}?{urllib.parse.urlencode(params, quote_via=urllib.parse.quote)}", timeout)
+ try:
+ answer = json.loads(text)
+ except ValueError:
+ sys.exit(f"not a driver answer (a build without the driver serves only the tree):\n{text[:300]}")
+ if out and answer.get("ok"):
+ with open(out, "wb") as f:
+ f.write(base64.b64decode(answer.pop("png")))
+ answer["saved"] = os.path.abspath(out)
+ print(json.dumps(answer, indent=1, ensure_ascii=False, sort_keys=True))
+ return 0 if answer.get("ok") else 1
+
+
+if __name__ == "__main__":
+ sys.exit(run(sys.argv[1:]))
diff --git a/scripts/pipeline.sh b/scripts/pipeline.sh
index 3d6f24ae..fee132f6 100755
--- a/scripts/pipeline.sh
+++ b/scripts/pipeline.sh
@@ -83,7 +83,8 @@ if ! xcrun -sdk iphoneos --find clang >/dev/null 2>&1; then
TARGET_STRIP=strip TARGET_LIPO=lipo TARGET_CODESIGN_ALLOCATE=codesign_allocate TARGET_LIBTOOL=libtool
fi
# Theos builds its Swift support tools only at MAKELEVEL 0, and `make release` hands this script MAKELEVEL 1.
-env -u MAKELEVEL gmake -C "$ROOT/tweak" clean package >/dev/null
+# A build with FLEX carries the phone driver too (tweak/Makefile, SG_DRIVER); --no-flex leaves it out.
+env -u MAKELEVEL gmake -C "$ROOT/tweak" clean package SG_DRIVER="$WITH_FLEX" >/dev/null
TWEAK_DEB="$(ls -t "$ROOT"/tweak/packages/*.deb | head -1)"
echo " $TWEAK_DEB"
diff --git a/tweak/Makefile b/tweak/Makefile
index 5a0acea2..fcf49958 100644
--- a/tweak/Makefile
+++ b/tweak/Makefile
@@ -23,6 +23,10 @@ TWEAK_NAME := spotifyglass
# is generated from the IPA by scripts/extract-flags.py.
spotifyglass_FILES := $(shell find Sources -name '*.m' -o -name '*.x' -o -name '*.swift' | sort)
spotifyglass_CFLAGS := -fobjc-arc -ISources -DSG_VERSION=\"$(SG_VERSION)\"
+# The phone driver (Diagnostics/Driver.m) goes only into builds that carry FLEX: scripts/pipeline.sh sets
+# SG_DRIVER=1 for those, and every other build (make release, the release workflow's .deb) leaves it out.
+SG_DRIVER ?= 0
+spotifyglass_CFLAGS += -DSG_DRIVER=$(SG_DRIVER)
spotifyglass_FRAMEWORKS := UIKit QuartzCore UniformTypeIdentifiers MediaPlayer CoreImage AudioToolbox AVFoundation CoreHaptics Accelerate CoreMotion CoreML CoreText
# zlib unpacks KuGou's KRC lyrics (Shared/LyricsSources/KuGou.m).
spotifyglass_LIBRARIES := z
diff --git a/tweak/Sources/Core/SGLog.h b/tweak/Sources/Core/SGLog.h
index 2d8c6b3a..68db729e 100644
--- a/tweak/Sources/Core/SGLog.h
+++ b/tweak/Sources/Core/SGLog.h
@@ -2,7 +2,21 @@
#import <os/log.h>
// %{public}s so idevicesyslog on the Mac sees the text instead of <private>.
+#if SG_DRIVER
+// A build with the phone driver (FLEX builds, SG_DRIVER in tweak/Makefile) also keeps the last lines in
+// memory, for the driver's log and wait commands (Diagnostics/Driver.m).
+#define SGLog(fmt, ...) do { \
+ NSString *sg_log_line_ = [NSString stringWithFormat:(fmt), ##__VA_ARGS__]; \
+ os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_DEFAULT, "[spotifyglass] %{public}s", sg_log_line_.UTF8String); \
+ SGLogRemember(sg_log_line_); \
+} while (0)
+void SGLogRemember(NSString *line);
+// The remembered lines numbered after `after`, oldest first, each {seq, t (seconds since 1970), text}.
+NSArray<NSDictionary *> *SGLogRecent(uint64_t after);
+uint64_t SGLogLastSeq(void);
+#else
#define SGLog(fmt, ...) os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_DEFAULT, "[spotifyglass] %{public}s", [NSString stringWithFormat:(fmt), ##__VA_ARGS__].UTF8String)
+#endif
// Long dumps, split into numbered parts under the unified log's size cap.
void SGLogLong(NSString *tag, NSString *text);
diff --git a/tweak/Sources/Core/SGLog.m b/tweak/Sources/Core/SGLog.m
index a0144e8d..601b7177 100644
--- a/tweak/Sources/Core/SGLog.m
+++ b/tweak/Sources/Core/SGLog.m
@@ -1,4 +1,41 @@
#import "SGLog.h"
+#import <os/lock.h>
+
+#if SG_DRIVER
+// The last lines, for the phone driver. They come from any thread, so a lock, never the main queue.
+static const NSUInteger kRemembered = 2000;
+static os_unfair_lock sg_logLock = OS_UNFAIR_LOCK_INIT;
+static NSMutableArray<NSDictionary *> *sg_logLines;
+static uint64_t sg_logSeq;
+
+void SGLogRemember(NSString *line) {
+ // The player's state dumps run to hundreds of parts each and would push every other line out.
+ if ([line hasPrefix:@"player state "] || [line hasPrefix:@"player track metadata"]) return;
+ NSTimeInterval now = NSDate.date.timeIntervalSince1970;
+ os_unfair_lock_lock(&sg_logLock);
+ if (!sg_logLines) sg_logLines = [NSMutableArray array];
+ [sg_logLines addObject:@{@"seq" : @(++sg_logSeq), @"t" : @(now), @"text" : line ?: @""}];
+ if (sg_logLines.count > kRemembered) [sg_logLines removeObjectAtIndex:0];
+ os_unfair_lock_unlock(&sg_logLock);
+}
+
+NSArray<NSDictionary *> *SGLogRecent(uint64_t after) {
+ os_unfair_lock_lock(&sg_logLock);
+ NSIndexSet *newer = [sg_logLines indexesOfObjectsPassingTest:^BOOL(NSDictionary *entry, NSUInteger i, BOOL *stop) {
+ return [entry[@"seq"] unsignedLongLongValue] > after;
+ }];
+ NSArray<NSDictionary *> *lines = [sg_logLines objectsAtIndexes:newer] ?: @[];
+ os_unfair_lock_unlock(&sg_logLock);
+ return lines;
+}
+
+uint64_t SGLogLastSeq(void) {
+ os_unfair_lock_lock(&sg_logLock);
+ uint64_t seq = sg_logSeq;
+ os_unfair_lock_unlock(&sg_logLock);
+ return seq;
+}
+#endif
// The unified log cuts a message at about 1 KB, so long dumps go out as numbered parts.
void SGLogLong(NSString *tag, NSString *text) {
diff --git a/tweak/Sources/Diagnostics/Diagnostics.h b/tweak/Sources/Diagnostics/Diagnostics.h
index 73e50f27..c8b61db6 100644
--- a/tweak/Sources/Diagnostics/Diagnostics.h
+++ b/tweak/Sources/Diagnostics/Diagnostics.h
@@ -1,7 +1,21 @@
// Screen dumps for FLEX builds: the visible screen's view tree, served over the phone's loopback
// and logged when the app goes to the background.
-#import <Foundation/Foundation.h>
+#import <UIKit/UIKit.h>
BOOL SGIsDebugBuild(void);
NSString *SGScreenTree(void);
void SGDumpScreen(NSString *reason);
+// Runs `block` on the main thread and waits for it at most `timeout` seconds; for any thread but main,
+// where it just runs the block. NO when main did not get to it in time (the block still runs later).
+BOOL SGRunOnMain(NSTimeInterval timeout, dispatch_block_t block);
+// The app's SPTEsperantoPlayer, nil until something asked it for its state (PlayerState.x).
+id SGDiagnosticsPlayer(void);
+
+#if SG_DRIVER
+// The phone driver (Driver.m): whether `command`, a request path without its slash, is one of its
+// commands, and running one with the request's query items, answering JSON. Called off the main thread.
+BOOL SGDriverHandles(NSString *command);
+NSData *SGDriverRun(NSString *command, NSDictionary<NSString *, NSString *> *params);
+// Defined in App/ModSettings.x: declared low, defined high.
+void SGOpenModSettings(UIView *source);
+#endif
diff --git a/tweak/Sources/Diagnostics/Diagnostics.x b/tweak/Sources/Diagnostics/Diagnostics.x
index d244d32e..eca64516 100644
--- a/tweak/Sources/Diagnostics/Diagnostics.x
+++ b/tweak/Sources/Diagnostics/Diagnostics.x
@@ -4,10 +4,16 @@
#import "Core/SGCore.h"
#import "Diagnostics.h"
#import <sys/socket.h>
+#import <ifaddrs.h>
+#import <arpa/inet.h>
#import <netinet/in.h>
#import <unistd.h>
-static const uint16_t kTreePort = 8085;
+// harness/driver builds the server on another port, so it leaves a tunnel to the phone on 8085 alone.
+#ifndef SG_TREE_PORT
+#define SG_TREE_PORT 8085
+#endif
+static const uint16_t kTreePort = SG_TREE_PORT;
static NSString *hexColor(CGColorRef color) {
CGFloat r = 0, g = 0, b = 0, a = 0;
@@ -25,7 +31,10 @@ static void appendTree(UIView *view, NSUInteger depth, NSMutableString *out) {
if (view.layer.mask) [line appendString:@" masked"];
if (view.clipsToBounds) [line appendString:@" clips"];
if (view.accessibilityIdentifier.length) [line appendFormat:@" id=%@", view.accessibilityIdentifier];
- if ([view isKindOfClass:UIControl.class] && view.accessibilityLabel.length) [line appendFormat:@" a11y=\"%@\"", view.accessibilityLabel];
+ // Controls and accessibility elements, the views the phone driver's `tap --label` finds.
+ if (([view isKindOfClass:UIControl.class] || (view.isAccessibilityElement && ![view isKindOfClass:UILabel.class])) && view.accessibilityLabel.length) {
+ [line appendFormat:@" a11y=\"%@\"", view.accessibilityLabel];
+ }
if ([view isKindOfClass:UILabel.class]) {
UILabel *label = (UILabel *)view;
[line appendFormat:@" \"%@\" %.0fpt %@", label.text, label.font.pointSize, hexColor(label.textColor.CGColor)];
@@ -96,7 +105,23 @@ void SGDumpScreen(NSString *reason) {
SGLogLong([@"screen dump " stringByAppendingString:reason], SGScreenTree());
}
-// GET anything on 127.0.0.1:kTreePort answers with the current screen's tree as text/plain.
+BOOL SGRunOnMain(NSTimeInterval timeout, dispatch_block_t block) {
+ if (NSThread.isMainThread) {
+ block();
+ return YES;
+ }
+ dispatch_semaphore_t done = dispatch_semaphore_create(0);
+ dispatch_async(dispatch_get_main_queue(), ^{
+ block();
+ dispatch_semaphore_signal(done);
+ });
+ return dispatch_semaphore_wait(done, dispatch_time(DISPATCH_TIME_NOW, (int64_t)(timeout * NSEC_PER_SEC))) == 0;
+}
+
+// The server: GET /tree (or any path that is not a driver command) answers with the current screen's
+// tree as text/plain; in a build with the phone driver, GET /<command>?<params> runs that command and
+// answers JSON (Driver.m). It listens on 127.0.0.1 only, the phone's loopback, which the Mac reaches
+// through usbmux (iproxy) over the cable and nothing on Wi-Fi reaches, and it starts only in FLEX builds.
static void sendAll(int client, NSData *data) {
const uint8_t *p = data.bytes;
size_t left = data.length;
@@ -108,38 +133,185 @@ static void sendAll(int client, NSData *data) {
}
}
-static void startTreeServer(void) {
+static void respond(int client, NSString *status, NSString *type, NSData *data) {
+ NSString *head = [NSString stringWithFormat:@"HTTP/1.0 %@\r\nContent-Type: %@; charset=utf-8\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n",
+ status, type, (unsigned long)data.length];
+ sendAll(client, [head dataUsingEncoding:NSUTF8StringEncoding]);
+ sendAll(client, data);
+}
+
+// The request line and headers, up to the blank line that ends them; a GET has no body.
+static NSString *readRequest(int client) {
+ NSMutableData *request = [NSMutableData data];
+ char buffer[4096];
+ while (request.length < 64 * 1024) {
+ ssize_t n = recv(client, buffer, sizeof(buffer), 0);
+ if (n <= 0) break;
+ [request appendBytes:buffer length:(NSUInteger)n];
+ if ([request rangeOfData:[@"\r\n\r\n" dataUsingEncoding:NSUTF8StringEncoding] options:0 range:NSMakeRange(0, request.length)].location != NSNotFound) break;
+ }
+ return [[NSString alloc] initWithData:request encoding:NSUTF8StringEncoding] ?: @"";
+}
+
+#if SG_DRIVER
+static NSString *driverToken(void);
+#endif
+
+// `tailnet`: the client came in on the Tailscale listener, which answers nothing without the token.
+static void serve(int client, BOOL tailnet) {
+ struct timeval timeout = {2, 0};
+ setsockopt(client, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
+ int no = 1;
+ setsockopt(client, SOL_SOCKET, SO_NOSIGPIPE, &no, sizeof(no));
+#if !SG_DRIVER
+ readRequest(client);
+#else
+ NSString *request = readRequest(client);
+ if (tailnet) {
+ NSString *token = [NSString stringWithFormat:@"\r\nX-Phone-Token: %@\r\n", driverToken()];
+ if ([request rangeOfString:token options:NSCaseInsensitiveSearch].location == NSNotFound) {
+ respond(client, @"403 Forbidden", @"application/json", [@"{\"ok\":false,\"error\":\"the tailnet listener needs the X-Phone-Token header\"}" dataUsingEncoding:NSUTF8StringEncoding]);
+ close(client);
+ return;
+ }
+ }
+ NSArray<NSString *> *first = [[request componentsSeparatedByString:@"\r\n"].firstObject componentsSeparatedByString:@" "];
+ NSURLComponents *url = first.count > 1 ? [NSURLComponents componentsWithString:[@"http://phone" stringByAppendingString:first[1]]] : nil;
+ NSString *command = [url.path stringByTrimmingCharactersInSet:[NSCharacterSet characterSetWithCharactersInString:@"/"]] ?: @"";
+ if (SGDriverHandles(command)) {
+ // The phone's loopback is every app's on the phone, Safari's pages included. A page can send a GET
+ // anywhere but cannot add a header of its own without a CORS preflight, which this server never
+ // allows, so a command needs the header scripts/phone.py sends.
+ if ([request rangeOfString:@"\r\nX-Phone-Driver:" options:NSCaseInsensitiveSearch].location == NSNotFound) {
+ respond(client, @"403 Forbidden", @"application/json", [@"{\"ok\":false,\"error\":\"a driver command needs the X-Phone-Driver header\"}" dataUsingEncoding:NSUTF8StringEncoding]);
+ close(client);
+ return;
+ }
+ NSMutableDictionary<NSString *, NSString *> *params = [NSMutableDictionary dictionary];
+ for (NSURLQueryItem *item in url.queryItems) params[item.name] = item.value ?: @"1";
+ // One command at a time, so two clients' touches never interleave; the tree is served alongside.
+ static dispatch_queue_t commands;
+ static dispatch_once_t once;
+ dispatch_once(&once, ^{ commands = dispatch_queue_create("spotifyglass.driver", DISPATCH_QUEUE_SERIAL); });
+ __block NSData *answer = nil;
+ dispatch_sync(commands, ^{ answer = SGDriverRun(command, params); });
+ respond(client, @"200 OK", @"application/json", answer);
+ close(client);
+ return;
+ }
+#endif
+ __block NSString *body = nil;
+ if (!SGRunOnMain(5, ^{ body = SGScreenTree(); })) {
+ respond(client, @"503 Service Unavailable", @"text/plain", [@"the main thread did not answer in 5 s\n" dataUsingEncoding:NSUTF8StringEncoding]);
+ } else {
+ respond(client, @"200 OK", @"text/plain", [body dataUsingEncoding:NSUTF8StringEncoding]);
+ }
+ close(client);
+}
+
+// Listens on `address` (network order) and serves each client on a thread of its own: a command may wait
+// seconds (wait, longpress). Answers the listening socket, -1 when it could not listen.
+static int listenOn(in_addr_t address, BOOL tailnet, dispatch_source_t __strong *source) {
int fd = socket(AF_INET, SOCK_STREAM, 0);
int yes = 1;
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
struct sockaddr_in addr = {0};
addr.sin_family = AF_INET;
addr.sin_port = htons(kTreePort);
- addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+ addr.sin_addr.s_addr = address;
if (bind(fd, (struct sockaddr *)&addr, sizeof(addr)) != 0 || listen(fd, 4) != 0) {
- SGLog(@"tree server: could not listen on %u", kTreePort);
close(fd);
- return;
+ return -1;
}
- static dispatch_source_t source;
- source = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, (uintptr_t)fd, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));
- dispatch_source_set_event_handler(source, ^{
+ *source = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, (uintptr_t)fd, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));
+ dispatch_source_set_event_handler(*source, ^{
int client = accept(fd, NULL, NULL);
if (client < 0) return;
- struct timeval timeout = {2, 0};
- setsockopt(client, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
- char request[1024];
- recv(client, request, sizeof(request), 0);
- __block NSString *body = nil;
- dispatch_sync(dispatch_get_main_queue(), ^{ body = SGScreenTree(); });
- NSData *data = [body dataUsingEncoding:NSUTF8StringEncoding];
- NSString *head = [NSString stringWithFormat:@"HTTP/1.0 200 OK\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n", (unsigned long)data.length];
- sendAll(client, [head dataUsingEncoding:NSUTF8StringEncoding]);
- sendAll(client, data);
- close(client);
+ dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0), ^{ serve(client, tailnet); });
+ });
+ dispatch_source_set_cancel_handler(*source, ^{ close(fd); });
+ dispatch_resume(*source);
+ return fd;
+}
+
+#if SG_DRIVER
+// The tailnet listener: the user asked to drive the app from their Mac over Tailscale. It binds the phone's
+// exact Tailscale address (a utun interface in 100.64.0.0/10), never every interface and never Wi-Fi or
+// cellular, so only the user's own tailnet reaches it, and every request on it needs the token, made once and
+// kept in the app's defaults, which the log prints at launch for scripts/phone.py.
+static NSString *driverToken(void) {
+ static NSString *token;
+ static dispatch_once_t once;
+ dispatch_once(&once, ^{
+ NSString *const key = @"spotifyglass.driver.token";
+ token = [NSUserDefaults.standardUserDefaults stringForKey:key];
+ if (token.length < 32) {
+ uint8_t bytes[16];
+ arc4random_buf(bytes, sizeof(bytes)); // cryptographically random on Darwin
+ NSMutableString *hex = [NSMutableString string];
+ for (size_t i = 0; i < sizeof(bytes); i++) [hex appendFormat:@"%02x", bytes[i]];
+ token = hex;
+ [NSUserDefaults.standardUserDefaults setObject:token forKey:key];
+ }
});
- dispatch_resume(source);
- SGLog(@"tree server on 127.0.0.1:%u; on the Mac: iproxy %u:%u, then GET http://127.0.0.1:%u/tree", kTreePort, kTreePort, kTreePort, kTreePort);
+ return token;
+}
+
+static in_addr_t tailnetAddress(void) {
+ struct ifaddrs *list = NULL;
+ in_addr_t found = 0;
+ if (getifaddrs(&list) != 0) return 0;
+ for (struct ifaddrs *i = list; i && !found; i = i->ifa_next) {
+ if (!i->ifa_addr || i->ifa_addr->sa_family != AF_INET || strncmp(i->ifa_name, "utun", 4) != 0) continue;
+ in_addr_t address = ((struct sockaddr_in *)i->ifa_addr)->sin_addr.s_addr;
+ if ((ntohl(address) & 0xFFC00000) == 0x64400000) found = address; // 100.64.0.0/10
+ }
+ freeifaddrs(list);
+ return found;
+}
+
+// The VPN can come up, go away or change address after launch: looked at again every 10 s.
+static void watchTailnet(void) {
+ static dispatch_source_t timer, source;
+ static in_addr_t bound;
+ timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));
+ dispatch_source_set_timer(timer, DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC, NSEC_PER_SEC);
+ dispatch_source_set_event_handler(timer, ^{
+ in_addr_t address = tailnetAddress();
+ if (address == bound) return;
+ if (source) {
+ dispatch_source_cancel(source);
+ source = nil;
+ }
+ bound = 0;
+ char text[INET_ADDRSTRLEN] = "";
+ inet_ntop(AF_INET, &address, text, sizeof(text));
+ if (!address) {
+ SGLog(@"driver: no Tailscale address, the tailnet listener is off");
+ return;
+ }
+ if (listenOn(address, YES, &source) < 0) {
+ SGLog(@"driver: could not listen on %s:%u", text, kTreePort);
+ return;
+ }
+ bound = address;
+ SGLog(@"driver: tailnet listener on %s:%u, token %@", text, kTreePort, driverToken());
+ });
+ dispatch_resume(timer);
+}
+#endif
+
+static void startTreeServer(void) {
+ static dispatch_source_t source;
+ if (listenOn(htonl(INADDR_LOOPBACK), NO, &source) < 0) { // loopback, which only the cable's iproxy reaches
+ SGLog(@"tree server: could not listen on %u", kTreePort);
+ return;
+ }
+ SGLog(@"tree server on 127.0.0.1:%u; on the Mac: iproxy %u:%u, then GET http://127.0.0.1:%u/tree%@", kTreePort, kTreePort, kTreePort, kTreePort,
+ SG_DRIVER ? @", or drive the app with scripts/phone.py" : @"");
+#if SG_DRIVER
+ watchTailnet();
+#endif
}
%hook _TtC21NowPlaying_ScrollImpl23NPVScrollViewController
diff --git a/tweak/Sources/Diagnostics/Driver.m b/tweak/Sources/Diagnostics/Driver.m
new file mode 100644
index 00000000..ad839e9a
--- /dev/null
+++ b/tweak/Sources/Diagnostics/Driver.m
@@ -0,0 +1,861 @@
+// The phone driver: commands on the tree server (Diagnostics.x) that work the app the way a finger
+// would, so an agent on the Mac can test a FLEX build without anyone touching the phone.
+// scripts/phone.py is its client; docs/tweaks.md (Diagnostics) lists the commands.
+//
+// Debug only: compiled with SG_DRIVER alone (tweak/Makefile; scripts/pipeline.sh sets it for builds that
+// carry FLEX, never for make release or the release workflow's .deb), and reached only through the tree
+// server, which starts only when FLEX is in the app and listens on 127.0.0.1, never on Wi-Fi.
+//
+// Threading: a command runs on the server's thread. What reads or touches UIKit goes to main through
+// SGRunOnMain (dispatch_async and a semaphore with a timeout, never waited on from main), and the pauses
+// between the phases of a touch are slept on the server's thread, so main runs freely between them.
+//
+// Touches are made the way harness/tabbar/touch.m makes them: a UITouch in the app's touches event,
+// carrying the IOHIDEvent UIKit's gesture recognizers read, sent through -[UIApplication sendEvent:].
+// Controls get their control events, recognizers recognize, and a button whose menu is its primary action
+// opens it on the touch down, as with a finger. The IOHIDEvent functions are looked up at run time, so the
+// dylib links nothing new.
+//
+// Views are addressed the way the tree (GET /tree) prints them: id= (accessibilityIdentifier), a11y=
+// (accessibilityLabel), the quoted text of a label, or the class name with an index counting that
+// class's views in the tree's order, windows included.
+#if SG_DRIVER
+#import "Core/SGCore.h"
+#import "Diagnostics.h"
+#import "Headers/SPTPlayer.h"
+#import <dlfcn.h>
+#import <mach/mach_time.h>
+#import <objc/message.h>
+
+static const NSTimeInterval kMainTimeout = 5;
+
+// A command that cannot go on throws, and SGDriverRun answers with the reason.
+static void fail(NSString *format, ...) NS_FORMAT_FUNCTION(1, 2) __attribute__((noreturn));
+static void fail(NSString *format, ...) {
+ va_list args;
+ va_start(args, format);
+ NSString *reason = [[NSString alloc] initWithFormat:format arguments:args];
+ va_end(args);
+ @throw [NSException exceptionWithName:@"SGDriver" reason:reason userInfo:nil];
+}
+
+// Runs `block` on main and hands back what it returned; an exception inside comes back out here.
+static id onMain(id (^block)(void)) {
+ __block id result = nil;
+ __block NSException *raised = nil;
+ BOOL answered = SGRunOnMain(kMainTimeout, ^{
+ @try {
+ result = block();
+ } @catch (NSException *exception) {
+ raised = exception;
+ }
+ });
+ if (!answered) fail(@"the main thread did not answer in %.0f s", kMainTimeout);
+ if (raised) @throw raised;
+ return result;
+}
+
+static void pause_(NSTimeInterval seconds) {
+ if (seconds > 0) usleep((useconds_t)(seconds * 1e6));
+}
+
+#pragma mark - touches
+
+@interface UITouch (SGDriver)
+- (void)setWindow:(UIWindow *)window;
+- (void)setView:(UIView *)view;
+- (void)setPhase:(UITouchPhase)phase;
+- (void)setTimestamp:(NSTimeInterval)timestamp;
+- (void)setTapCount:(NSUInteger)count;
+- (void)_setLocationInWindow:(CGPoint)location resetPrevious:(BOOL)reset;
+- (void)_setIsFirstTouchForView:(BOOL)first;
+- (void)_setHidEvent:(void *)event;
+- (void)setGestureView:(UIView *)view;
+@end
+
+@interface UIEvent (SGDriver)
+- (void)_clearTouches;
+- (void)_addTouch:(UITouch *)touch forDelayedDelivery:(BOOL)delayed;
+- (void)_setHIDEvent:(void *)event;
+@end
+
+@interface UIApplication (SGDriver)
+- (UIEvent *)_touchesEvent;
+@end
+
+typedef void *IOHIDEventRef;
+static IOHIDEventRef (*sg_createDigitizer)(CFAllocatorRef, uint64_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, double, double, double,
+ double, double, Boolean, Boolean, uint32_t);
+static IOHIDEventRef (*sg_createFinger)(CFAllocatorRef, uint64_t, uint32_t, uint32_t, uint32_t, double, double, double, double, double, double,
+ double, double, double, double, Boolean, Boolean, uint32_t);
+static void (*sg_appendEvent)(IOHIDEventRef, IOHIDEventRef, uint32_t);
+static void (*sg_setInteger)(IOHIDEventRef, uint32_t, long);
+
+static void requireTouches(void) {
+ static dispatch_once_t once;
+ dispatch_once(&once, ^{
+ sg_createDigitizer = dlsym(RTLD_DEFAULT, "IOHIDEventCreateDigitizerEvent");
+ sg_createFinger = dlsym(RTLD_DEFAULT, "IOHIDEventCreateDigitizerFingerEventWithQuality");
+ sg_appendEvent = dlsym(RTLD_DEFAULT, "IOHIDEventAppendEvent");
+ sg_setInteger = dlsym(RTLD_DEFAULT, "IOHIDEventSetIntegerValue");
+ });
+ if (!sg_createDigitizer || !sg_createFinger || !sg_appendEvent || !sg_setInteger) fail(@"IOHIDEvent functions not found, touches cannot be made");
+ if (![UIApplication.sharedApplication respondsToSelector:@selector(_touchesEvent)] || ![UITouch instancesRespondToSelector:@selector(_setHidEvent:)]) {
+ fail(@"UIKit's private touch setters are missing on this iOS");
+ }
+}
+
+static const uint32_t kDigitizer = 11, kHand = 3, kRange = 1 << 0, kTouch = 1 << 1, kPosition = 1 << 2;
+static const uint32_t kDisplayIntegrated = (kDigitizer << 16) + 25;
+
+static IOHIDEventRef hidFor(UITouch *touch) {
+ uint64_t now = mach_absolute_time();
+ IOHIDEventRef hand = sg_createDigitizer(kCFAllocatorDefault, now, kHand, 0, 0, kTouch, 0, 0, 0, 0, 0, 0, 0, 1, 0);
+ sg_setInteger(hand, kDisplayIntegrated, 1);
+ BOOL moved = touch.phase == UITouchPhaseMoved, down = touch.phase != UITouchPhaseEnded;
+ CGPoint at = [touch locationInView:touch.window];
+ IOHIDEventRef finger = sg_createFinger(kCFAllocatorDefault, now, 1, 2, moved ? kPosition : kRange | kTouch, at.x, at.y, 0, 0, 0, 5, 5, 1, 1, 1,
+ down, down, 0);
+ sg_setInteger(finger, kDisplayIntegrated, 1);
+ sg_appendEvent(hand, finger, 0);
+ CFRelease(finger);
+ return hand;
+}
+
+static void deliver(UITouch *touch) {
+ IOHIDEventRef hid = hidFor(touch);
+ [touch _setHidEvent:hid];
+ UIEvent *event = [UIApplication.sharedApplication _touchesEvent];
+ [event _clearTouches];
+ [event _setHIDEvent:hid];
+ [event _addTouch:touch forDelayedDelivery:NO];
+ CFRelease(hid);
+ [UIApplication.sharedApplication sendEvent:event];
+}
+
+static NSArray<UIWindow *> *windows(void);
+
+// The window a finger at `screen` lands in: the highest one that takes a touch there.
+static UIWindow *windowAt(CGPoint screen) {
+ // Highest level first, and of one level the later window, which is the one in front.
+ NSArray<UIWindow *> *all = [windows().reverseObjectEnumerator.allObjects sortedArrayWithOptions:NSSortStable
+ usingComparator:^NSComparisonResult(UIWindow *a, UIWindow *b) {
+ return a.windowLevel > b.windowLevel ? NSOrderedAscending : a.windowLevel < b.windowLevel ? NSOrderedDescending : NSOrderedSame;
+ }];
+ for (UIWindow *window in all) {
+ if ([window hitTest:[window convertPoint:screen fromWindow:nil] withEvent:nil]) return window;
+ }
+ return nil;
+}
+
+static NSArray *pair(CGPoint p) {
+ return @[@(round(p.x * 10) / 10), @(round(p.y * 10) / 10)];
+}
+
+// A finger down at `from`, moved to `to` over `duration` s at 60 Hz, held still `hold` s, lifted at `to`.
+// With no hold the last move is the lift, so a quick one flings.
+static NSDictionary *finger(CGPoint from, CGPoint to, NSTimeInterval duration, NSTimeInterval hold) {
+ requireTouches();
+ __block UITouch *touch = nil;
+ NSDictionary *down = onMain(^id {
+ UIWindow *window = windowAt(from);
+ if (!window) fail(@"no window takes a touch at {%.0f, %.0f}", from.x, from.y);
+ CGPoint at = [window convertPoint:from fromWindow:nil];
+ UIView *view = [window hitTest:at withEvent:nil];
+ touch = [UITouch new];
+ [touch setWindow:window];
+ [touch _setLocationInWindow:at resetPrevious:YES];
+ [touch setView:view];
+ if ([touch respondsToSelector:@selector(setGestureView:)]) [touch setGestureView:view];
+ [touch setPhase:UITouchPhaseBegan];
+ [touch _setIsFirstTouchForView:YES];
+ [touch setTapCount:1];
+ [touch setTimestamp:NSProcessInfo.processInfo.systemUptime];
+ deliver(touch);
+ return @{@"window" : NSStringFromClass(window.class), @"hit" : NSStringFromClass(view.class) ?: @"nil"};
+ });
+ void (^move)(CGPoint, UITouchPhase) = ^(CGPoint point, UITouchPhase phase) {
+ onMain(^id {
+ [touch setTimestamp:NSProcessInfo.processInfo.systemUptime];
+ [touch _setLocationInWindow:[touch.window convertPoint:point fromWindow:nil] resetPrevious:NO];
+ [touch setPhase:phase];
+ deliver(touch);
+ return nil;
+ });
+ };
+ @try {
+ NSUInteger steps = duration > 0 ? MAX(2, (NSUInteger)(duration * 60)) : 0;
+ for (NSUInteger i = 1; i <= steps; i++) {
+ pause_(1.0 / 60);
+ CGFloat t = (CGFloat)i / steps;
+ CGPoint at = CGPointMake(from.x + (to.x - from.x) * t, from.y + (to.y - from.y) * t);
+ if (i == steps && hold <= 0) {
+ move(at, UITouchPhaseEnded);
+ return down;
+ }
+ move(at, UITouchPhaseMoved);
+ }
+ pause_(hold);
+ move(to, UITouchPhaseEnded);
+ } @catch (NSException *exception) {
+ // A finger left down would hold every later touch; lift it whatever happened.
+ dispatch_async(dispatch_get_main_queue(), ^{
+ [touch setPhase:UITouchPhaseCancelled];
+ deliver(touch);
+ });
+ @throw exception;
+ }
+ return down;
+}
+
+#pragma mark - finding views
+
+static NSArray<UIWindow *> *windows(void) {
+ NSMutableArray<UIWindow *> *all = [NSMutableArray array];
+ for (UIScene *scene in UIApplication.sharedApplication.connectedScenes) {
+ if (![scene isKindOfClass:UIWindowScene.class]) continue;
+ for (UIWindow *window in ((UIWindowScene *)scene).windows) {
+ if (window.hidden || [NSStringFromClass(window.class) containsString:@"FLEX"]) continue;
+ [all addObject:window];
+ }
+ }
+ return all;
+}
+
+static UIWindow *keyWindow(void) {
+ for (UIWindow *window in windows()) {
+ if (window.isKeyWindow) return window;
+ }
+ return windows().firstObject;
+}
+
+static NSString *textOf(UIView *view) {
+ if ([view isKindOfClass:UIButton.class]) return ((UIButton *)view).currentTitle;
+ if (![view respondsToSelector:@selector(text)] || [view methodSignatureForSelector:@selector(text)].methodReturnType[0] != '@') return nil;
+ id text = ((id (*)(id, SEL))objc_msgSend)(view, @selector(text));
+ return [text isKindOfClass:NSString.class] ? text : nil;
+}
+
+static BOOL isShown(UIView *view) {
+ if (!view.window) return NO;
+ for (UIView *v = view; v; v = v.superview) {
+ if (v.hidden || v.alpha < 0.01) return NO;
+ }
+ CGRect frame = [view convertRect:view.bounds toView:nil];
+ return frame.size.width > 0 && frame.size.height > 0 && CGRectIntersectsRect(frame, view.window.bounds);
+}
+
+static CGRect screenFrame(UIView *view) {
+ return [view.window convertRect:[view convertRect:view.bounds toView:nil] toWindow:nil];
+}
+
+static CGPoint screenCenter(UIView *view) {
+ CGRect frame = screenFrame(view);
+ return CGPointMake(CGRectGetMidX(frame), CGRectGetMidY(frame));
+}
+
+static BOOL same(NSString *a, NSString *b, BOOL contains) {
+ if (!a || !b) return NO;
+ a = [a stringByTrimmingCharactersInSet:NSCharacterSet.whitespaceAndNewlineCharacterSet];
+ if (contains) return [a rangeOfString:b options:NSCaseInsensitiveSearch].location != NSNotFound;
+ return [a caseInsensitiveCompare:b] == NSOrderedSame;
+}
+
+static BOOL hasSelector(NSDictionary *p) {
+ return p[@"id"] || p[@"label"] || p[@"text"] || p[@"class"];
+}
+
+static BOOL matches(UIView *view, NSDictionary *p) {
+ BOOL contains = p[@"contains"] != nil;
+ if (p[@"class"] && ![NSStringFromClass(view.class) isEqualToString:p[@"class"]]) return NO;
+ if (p[@"id"] && ![view.accessibilityIdentifier isEqualToString:p[@"id"]]) return NO;
+ if (p[@"label"] && !same(view.accessibilityLabel, p[@"label"], contains)) return NO;
+ if (p[@"text"] && !same(textOf(view), p[@"text"], contains)) return NO;
+ return YES;
+}
+
+// Every view the selector matches, in the tree's order.
+static NSArray<UIView *> *findAll(NSDictionary *p, UIView *root) {
+ NSMutableArray<UIView *> *found = [NSMutableArray array];
+ for (UIView *top in root ? @[root] : windows()) {
+ SGForEachView(top, ^(UIView *view) {
+ if (matches(view, p)) [found addObject:view];
+ });
+ }
+ return found;
+}
+
+// The view a selector means: with an index, that one of the matches in the tree's order; without, the
+// first one shown.
+static UIView *pick(NSDictionary *p, UIView *root) {
+ NSArray<UIView *> *found = findAll(p, root);
+ if (p[@"index"]) {
+ NSInteger index = [p[@"index"] integerValue];
+ return index >= 0 && index < (NSInteger)found.count ? found[(NSUInteger)index] : nil;
+ }
+ for (UIView *view in found) {
+ if (isShown(view)) return view;
+ }
+ return nil;
+}
+
+static NSString *selectorText(NSDictionary *p) {
+ NSMutableArray<NSString *> *parts = [NSMutableArray array];
+ for (NSString *key in @[@"id", @"label", @"text", @"class", @"index"]) {
+ if (p[key]) [parts addObject:[NSString stringWithFormat:@"%@=%@", key, p[key]]];
+ }
+ return [parts componentsJoinedByString:@" "];
+}
+
+static UIView *require(NSDictionary *p, UIView *root) {
+ UIView *view = pick(p, root);
+ if (view) return view;
+ NSUInteger count = findAll(p, root).count;
+ if (count) fail(@"%@ matches %lu views, none of them shown (index= takes a hidden one)", selectorText(p), (unsigned long)count);
+ fail(@"no view matches %@", selectorText(p));
+}
+
+static NSDictionary *describe(UIView *view) {
+ NSMutableDictionary *d = [NSMutableDictionary dictionary];
+ d[@"class"] = NSStringFromClass(view.class);
+ CGRect frame = screenFrame(view);
+ d[@"frame"] = @[@(round(frame.origin.x)), @(round(frame.origin.y)), @(round(frame.size.width)), @(round(frame.size.height))];
+ d[@"shown"] = @(isShown(view));
+ if (view.accessibilityIdentifier.length) d[@"id"] = view.accessibilityIdentifier;
+ if (view.accessibilityLabel.length) d[@"label"] = view.accessibilityLabel;
+ NSString *text = textOf(view);
+ if (text.length) d[@"text"] = text;
+ if ([view isKindOfClass:UIControl.class]) {
+ d[@"enabled"] = @(((UIControl *)view).enabled);
+ d[@"selected"] = @(((UIControl *)view).selected);
+ }
+ if ([view isKindOfClass:UISwitch.class]) d[@"on"] = @(((UISwitch *)view).on);
+ if ([view isKindOfClass:UIScrollView.class]) {
+ UIScrollView *scroll = (UIScrollView *)view;
+ d[@"contentOffset"] = pair(scroll.contentOffset);
+ d[@"contentSize"] = pair(CGPointMake(scroll.contentSize.width, scroll.contentSize.height));
+ }
+ return d;
+}
+
+static CGPoint parsePoint(NSString *text, NSString *name) {
+ NSArray<NSString *> *parts = [text componentsSeparatedByString:@","];
+ if (parts.count != 2) fail(@"%@ takes x,y, not %@", name, text);
+ return CGPointMake(parts[0].doubleValue, parts[1].doubleValue);
+}
+
+static double number(NSDictionary *p, NSString *key, double fallback) {
+ return p[key] ? [p[key] doubleValue] : fallback;
+}
+
+// Where a command's finger goes: at=x,y, or the middle of the view the selector picks.
+static NSDictionary *target(NSDictionary *p) {
+ return onMain(^id {
+ if (p[@"at"]) return @{@"point" : [NSValue valueWithCGPoint:parsePoint(p[@"at"], @"at")]};
+ if (!hasSelector(p)) fail(@"say where: at=x,y, or a view by id, label, text or class (with index)");
+ UIView *view = require(p, nil);
+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(view)], @"target" : describe(view)};
+ });
+}
+
+static NSDictionary *touchAt(NSDictionary *where, NSTimeInterval hold) {
+ CGPoint point = [where[@"point"] CGPointValue];
+ NSMutableDictionary *result = [finger(point, point, 0, hold) mutableCopy];
+ result[@"point"] = pair(point);
+ if (where[@"target"]) result[@"target"] = where[@"target"];
+ return result;
+}
+
+#pragma mark - the app's own parts
+
+static NSString *const kPlayerClass = @"_TtC21NowPlaying_ScrollImpl23NPVScrollViewController";
+static NSString *const kBarClass = @"_TtC18NowPlaying_BarImpl27NowPlayingBarViewController";
+
+static BOOL insideController(UIView *view, Class cls) {
+ for (UIResponder *r = view; r; r = r.nextResponder) {
+ if (cls && [r isKindOfClass:cls]) return YES;
+ }
+ return NO;
+}
+
+// A shown view that is some controller's own view, of class `name`.
+static UIView *controllerView(NSString *name) {
+ Class cls = NSClassFromString(name);
+ if (!cls) return nil;
+ for (UIWindow *window in windows()) {
+ __block UIView *found = nil;
+ SGForEachView(window, ^(UIView *view) {
+ if (!found && [view.nextResponder isKindOfClass:cls] && isShown(view)) found = view;
+ });
+ if (found) return found;
+ }
+ return nil;
+}
+
+static BOOL playerShown(void) {
+ return controllerView(kPlayerClass) != nil;
+}
+
+// The system's menu, for the menus of UIButton and UIContextMenuInteraction.
+static BOOL isMenuPart(UIView *view) {
+ NSString *name = NSStringFromClass(view.class);
+ return [name hasPrefix:@"_UIContextMenu"] || [name hasPrefix:@"_UIMenu"];
+}
+
+static UIView *shownMenu(void) {
+ for (UIWindow *window in windows()) {
+ __block UIView *found = nil;
+ SGForEachView(window, ^(UIView *view) {
+ if (!found && [NSStringFromClass(view.class) hasPrefix:@"_UIContextMenuView"] && isShown(view)) found = view;
+ });
+ if (found) return found;
+ }
+ return nil;
+}
+
+static BOOL insideMenu(UIView *view) {
+ for (UIView *v = view; v; v = v.superview) {
+ if (isMenuPart(v)) return YES;
+ }
+ return NO;
+}
+
+// The view showing `title`, by its text or its accessibility label, shown and passing `filter`.
+static UIView *titled(NSString *title, BOOL (^filter)(UIView *)) {
+ for (NSString *key in @[@"text", @"label"]) {
+ for (UIView *view in findAll(@{key : title}, nil)) {
+ if (isShown(view) && (!filter || filter(view))) return view;
+ }
+ }
+ return nil;
+}
+
+static UIViewController *topController(NSMutableArray<NSString *> *path, NSMutableArray<NSString *> *presented) {
+ UIViewController *top = keyWindow().rootViewController;
+ while (top) {
+ [path addObject:NSStringFromClass(top.class)];
+ UIViewController *next = nil;
+ if (top.presentedViewController && !top.presentedViewController.isBeingDismissed) {
+ next = top.presentedViewController;
+ [presented addObject:NSStringFromClass(next.class)];
+ } else if ([top isKindOfClass:UINavigationController.class]) {
+ next = ((UINavigationController *)top).topViewController;
+ } else if ([top isKindOfClass:UITabBarController.class]) {
+ next = ((UITabBarController *)top).selectedViewController;
+ } else {
+ // A container of Spotify's own: the child that covers most of it, not a bar inside it.
+ CGRect own = CGRectIntersection(screenFrame(top.viewIfLoaded), keyWindow().bounds);
+ CGFloat best = own.size.width * own.size.height / 2;
+ for (UIViewController *child in top.childViewControllers) {
+ if (!child.isViewLoaded || !isShown(child.view)) continue;
+ CGRect frame = CGRectIntersection(screenFrame(child.view), keyWindow().bounds);
+ CGFloat area = frame.size.width * frame.size.height;
+ if (area > best) {
+ best = area;
+ next = child;
+ }
+ }
+ }
+ if (!next) break;
+ top = next;
+ }
+ return top;
+}
+
+static NSDictionary *nowPlaying(void) {
+ id player = SGDiagnosticsPlayer();
+ SPTPlayerState *state = [player respondsToSelector:@selector(state)] ? [(id<SPTPlayer>)player state] : nil;
+ if (!state) return nil;
+ NSMutableDictionary *d = [NSMutableDictionary dictionary];
+ SPTPlayerTrack *track = state.track;
+ if (track.trackTitle) d[@"title"] = track.trackTitle;
+ if (track.artistName) d[@"artist"] = track.artistName;
+ if (track.URI) d[@"uri"] = [track.URI description];
+ d[@"position"] = @(round((state.isPaused ? state.positionAsOfTimestamp : state.position) * 10) / 10);
+ d[@"duration"] = @(round(state.duration * 10) / 10);
+ d[@"playing"] = @((BOOL)(state.isPlaying && !state.isPaused));
+ d[@"paused"] = @(state.isPaused);
+ return d;
+}
+
+static id<SPTPlayer> requirePlayer(void) {
+ id player = SGDiagnosticsPlayer();
+ if (!player) fail(@"no player yet: play something first, or open the app's player once");
+ return player;
+}
+
+static UIView *firstResponder(void) {
+ for (UIWindow *window in windows()) {
+ id responder = nil;
+ @try {
+ responder = [window valueForKey:@"firstResponder"];
+ } @catch (NSException *exception) {
+ }
+ if ([responder isKindOfClass:UIView.class]) return responder;
+ }
+ return nil;
+}
+
+// The tab bar's items, leading first: the redesign's UITabBar, or Spotify's own bar's row.
+static NSArray<UIView *> *tabItems(NSMutableArray<NSString *> *titles) {
+ for (UIWindow *window in windows()) {
+ __block UIView *bar = nil;
+ SGForEachView(window, ^(UIView *view) {
+ if (bar || !isShown(view)) return;
+ if ([view isKindOfClass:UITabBar.class] || [NSStringFromClass(view.class) isEqualToString:@"_TtC23NavigationUI_TabBarImpl10TabBarView"]) bar = view;
+ });
+ if (!bar) continue;
+ NSMutableArray<UIView *> *items = [NSMutableArray array];
+ if ([bar isKindOfClass:UITabBar.class]) {
+ for (UITabBarItem *item in ((UITabBar *)bar).items) {
+ UIView *view = item.title ? titled(item.title, ^BOOL(UIView *v) { return [v isDescendantOfView:bar]; }) : nil;
+ if (!view) continue;
+ [items addObject:view];
+ [titles addObject:item.title];
+ }
+ } else {
+ for (UIView *item in SGRowIn(bar).arrangedSubviews) {
+ if (!isShown(item)) continue;
+ __block NSString *title = item.accessibilityLabel;
+ SGForEachView(item, ^(UIView *v) {
+ if (!title.length && textOf(v).length) title = textOf(v);
+ });
+ [items addObject:item];
+ [titles addObject:title ?: @""];
+ }
+ }
+ return items;
+ }
+ return @[];
+}
+
+// The largest scroll view shown in the key window, the page's own list.
+static UIScrollView *mainScrollView(void) {
+ __block UIScrollView *best = nil;
+ __block CGFloat bestArea = 0;
+ SGForEachView(keyWindow(), ^(UIView *view) {
+ if (![view isKindOfClass:UIScrollView.class] || !isShown(view)) return;
+ CGRect frame = screenFrame(view);
+ if (frame.size.width * frame.size.height > bestArea) {
+ bestArea = frame.size.width * frame.size.height;
+ best = (UIScrollView *)view;
+ }
+ });
+ return best;
+}
+
+static CGPoint clampedOffset(UIScrollView *scroll, CGPoint offset) {
+ UIEdgeInsets inset = scroll.adjustedContentInset;
+ CGFloat maxX = MAX(-inset.left, scroll.contentSize.width + inset.right - scroll.bounds.size.width);
+ CGFloat maxY = MAX(-inset.top, scroll.contentSize.height + inset.bottom - scroll.bounds.size.height);
+ return CGPointMake(MIN(MAX(offset.x, -inset.left), maxX), MIN(MAX(offset.y, -inset.top), maxY));
+}
+
+#pragma mark - commands
+
+typedef NSDictionary *(^SGCommand)(NSDictionary *p);
+
+// The log number when the last command that does something began: a log wait looks from there, so
+// `player.more` and then `wait log=...` sees a line that came before the wait did.
+static uint64_t sg_actionSeq;
+
+static NSDictionary<NSString *, SGCommand> *commands(void) {
+ static NSDictionary<NSString *, SGCommand> *all;
+ static dispatch_once_t once;
+ dispatch_once(&once, ^{
+ all = @{
+ @"tap" : ^NSDictionary *(NSDictionary *p) {
+ return touchAt(target(p), 0.08);
+ },
+ @"longpress" : ^NSDictionary *(NSDictionary *p) {
+ return touchAt(target(p), number(p, @"duration", 1.0));
+ },
+ @"swipe" : ^NSDictionary *(NSDictionary *p) {
+ NSDictionary *start = p[@"from"] ? @{@"point" : [NSValue valueWithCGPoint:parsePoint(p[@"from"], @"from")]} : target(p);
+ CGPoint from = [start[@"point"] CGPointValue];
+ CGPoint to;
+ if (p[@"to"]) {
+ to = parsePoint(p[@"to"], @"to");
+ } else if (p[@"by"]) {
+ CGPoint by = parsePoint(p[@"by"], @"by");
+ to = CGPointMake(from.x + by.x, from.y + by.y);
+ } else {
+ fail(@"swipe takes to=x,y or by=dx,dy");
+ }
+ BOOL drag = [p[@"drag"] boolValue];
+ NSMutableDictionary *result = [finger(from, to, number(p, @"duration", drag ? 0.6 : 0.2), number(p, @"hold", drag ? 0.3 : 0)) mutableCopy];
+ result[@"from"] = pair(from);
+ result[@"to"] = pair(to);
+ if (start[@"target"]) result[@"target"] = start[@"target"];
+ return result;
+ },
+ @"scroll" : ^NSDictionary *(NSDictionary *p) {
+ if (!p[@"by"]) fail(@"scroll takes by=dx,dy");
+ CGPoint by = parsePoint(p[@"by"], @"by");
+ return onMain(^id {
+ UIView *view = nil;
+ if (hasSelector(p)) {
+ view = require(p, nil);
+ } else if (p[@"at"]) {
+ CGPoint at = parsePoint(p[@"at"], @"at");
+ UIWindow *window = windowAt(at);
+ view = [window hitTest:[window convertPoint:at fromWindow:nil] withEvent:nil];
+ } else {
+ view = mainScrollView();
+ }
+ while (view && ![view isKindOfClass:UIScrollView.class]) view = view.superview;
+ if (!view) fail(@"no scroll view there");
+ UIScrollView *scroll = (UIScrollView *)view;
+ CGPoint before = scroll.contentOffset;
+ CGPoint after = clampedOffset(scroll, CGPointMake(before.x + by.x, before.y + by.y));
+ [scroll setContentOffset:after animated:[p[@"animated"] boolValue]];
+ return @{@"scrollView" : describe(scroll), @"before" : pair(before), @"after" : pair(after)};
+ });
+ },
+ @"type" : ^NSDictionary *(NSDictionary *p) {
+ NSString *text = p[@"text"] ?: p[@"arg"];
+ if (!text) fail(@"type takes the text");
+ return onMain(^id {
+ UIView *responder = firstResponder();
+ if (![responder conformsToProtocol:@protocol(UIKeyInput)]) fail(@"nothing takes typing: tap a text field first");
+ [(id<UIKeyInput>)responder insertText:text];
+ return @{@"into" : describe(responder)};
+ });
+ },
+ @"find" : ^NSDictionary *(NSDictionary *p) {
+ if (!hasSelector(p)) fail(@"find takes id, label, text or class");
+ return onMain(^id {
+ NSMutableArray *views = [NSMutableArray array];
+ NSArray<UIView *> *found = findAll(p, nil);
+ for (NSUInteger i = 0; i < found.count && i < 100; i++) {
+ NSMutableDictionary *d = [describe(found[i]) mutableCopy];
+ d[@"index"] = @(i);
+ [views addObject:d];
+ }
+ return @{@"count" : @(found.count), @"views" : views};
+ });
+ },
+ @"state" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ NSMutableArray<NSString *> *path = [NSMutableArray array], *presented = [NSMutableArray array];
+ UIViewController *top = topController(path, presented);
+ NSMutableDictionary *d = [NSMutableDictionary dictionary];
+ d[@"top"] = top ? NSStringFromClass(top.class) : [NSNull null];
+ if (top.title.length) d[@"topTitle"] = top.title;
+ d[@"controllers"] = path;
+ d[@"presented"] = presented;
+ d[@"menu"] = @((BOOL)(shownMenu() != nil));
+ d[@"player"] = @(playerShown());
+ d[@"nowPlaying"] = nowPlaying() ?: [NSNull null];
+ UIView *responder = firstResponder();
+ if (responder) d[@"firstResponder"] = NSStringFromClass(responder.class);
+ d[@"keyWindow"] = NSStringFromClass(keyWindow().class) ?: [NSNull null];
+ d[@"logSeq"] = @(SGLogLastSeq());
+ return d;
+ });
+ },
+ @"screenshot" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ UIWindow *key = keyWindow();
+ if (!key) fail(@"no window");
+ UIGraphicsImageRendererFormat *format = [UIGraphicsImageRendererFormat preferredFormat];
+ if (p[@"scale"]) format.scale = [p[@"scale"] doubleValue];
+ // Every window, lowest level first, so a menu in a window of its own is in the picture.
+ NSArray<UIWindow *> *all = [windows() sortedArrayWithOptions:NSSortStable usingComparator:^NSComparisonResult(UIWindow *a, UIWindow *b) {
+ return a.windowLevel < b.windowLevel ? NSOrderedAscending : a.windowLevel > b.windowLevel ? NSOrderedDescending : NSOrderedSame;
+ }];
+ UIGraphicsImageRenderer *renderer = [[UIGraphicsImageRenderer alloc] initWithBounds:key.bounds format:format];
+ NSData *png = [renderer PNGDataWithActions:^(UIGraphicsImageRendererContext *context) {
+ for (UIWindow *window in all) [window drawViewHierarchyInRect:window.frame afterScreenUpdates:NO];
+ }];
+ return @{@"png" : [png base64EncodedStringWithOptions:0], @"size" : pair(CGPointMake(key.bounds.size.width * format.scale, key.bounds.size.height * format.scale))};
+ });
+ },
+ @"log" : ^NSDictionary *(NSDictionary *p) {
+ uint64_t last = SGLogLastSeq();
+ uint64_t since = p[@"since"] ? (uint64_t)[p[@"since"] longLongValue] : (last > 100 ? last - 100 : 0);
+ return @{@"lines" : SGLogRecent(since), @"last" : @(last)};
+ },
+ @"wait" : ^NSDictionary *(NSDictionary *p) {
+ NSTimeInterval timeout = number(p, @"timeout", 5);
+ NSDate *start = NSDate.date;
+ BOOL gone = [p[@"gone"] boolValue];
+ if (!p[@"log"] && !hasSelector(p) && !p[@"menu"]) fail(@"wait takes log=text, menu=1, or a view by id, label, text or class");
+ uint64_t since = p[@"since"] ? (uint64_t)[p[@"since"] longLongValue] : sg_actionSeq;
+ while (YES) {
+ if (p[@"log"]) {
+ for (NSDictionary *line in SGLogRecent(since)) {
+ if ([line[@"text"] containsString:p[@"log"]]) return @{@"line" : line, @"after" : @(-start.timeIntervalSinceNow)};
+ }
+ } else {
+ NSDictionary *seen = onMain(^id {
+ UIView *view = p[@"menu"] ? shownMenu() : pick(p, nil);
+ if (view && !isShown(view)) view = nil;
+ if (gone) return view ? nil : @{};
+ return view ? describe(view) : nil;
+ });
+ if (seen) return @{@"view" : seen.count ? seen : [NSNull null], @"after" : @(-start.timeIntervalSinceNow)};
+ }
+ if (-start.timeIntervalSinceNow > timeout) {
+ if (p[@"log"]) fail(@"no log line with \"%@\" within %.1f s", p[@"log"], timeout);
+ NSString *what = p[@"menu"] ? @"the menu" : selectorText(p);
+ fail(@"%@ %@ after %.1f s", what, gone ? @"still shown" : @"not shown", timeout);
+ }
+ pause_(0.1);
+ }
+ },
+ @"player.open" : ^NSDictionary *(NSDictionary *p) {
+ NSDictionary *where = onMain(^id {
+ if (playerShown()) return @{};
+ UIView *bar = controllerView(kBarClass);
+ if (!bar) fail(@"no now playing bar shown");
+ // Left of the middle: the artwork and the title, away from the bar's buttons.
+ CGRect frame = screenFrame(bar);
+ return @{@"point" : [NSValue valueWithCGPoint:CGPointMake(CGRectGetMinX(frame) + frame.size.width * 0.3, CGRectGetMidY(frame))],
+ @"target" : describe(bar)};
+ });
+ if (!where.count) return @{@"note" : @"the player was already open"};
+ return touchAt(where, 0.08);
+ },
+ @"player.close" : ^NSDictionary *(NSDictionary *p) {
+ NSDictionary *where = onMain(^id {
+ for (NSString *identifier in @[@"now-playing-minimize-button", @"mobile-nowplaying-close-button"]) {
+ UIView *button = pick(@{@"id" : identifier}, nil);
+ if (button) return @{@"point" : [NSValue valueWithCGPoint:screenCenter(button)], @"target" : describe(button)};
+ }
+ if (!playerShown()) return @{};
+ fail(@"the player is open but its down arrow is not found");
+ });
+ if (!where.count) return @{@"note" : @"the player was not open"};
+ return touchAt(where, 0.08);
+ },
+ @"player.more" : ^NSDictionary *(NSDictionary *p) {
+ NSDictionary *where = onMain(^id {
+ Class player = NSClassFromString(kPlayerClass);
+ UIView *more = nil;
+ for (UIView *view in findAll(@{@"id" : @"Context menu"}, nil)) {
+ if (isShown(view) && insideController(view, player)) more = view;
+ }
+ if (!more) fail(@"%@", playerShown() ? @"the player's ⋯ (id=Context menu) is not shown" : @"the player is not open (player.open)");
+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(more)], @"target" : describe(more)};
+ });
+ return touchAt(where, 0.08);
+ },
+ @"menu.pick" : ^NSDictionary *(NSDictionary *p) {
+ NSString *title = p[@"title"] ?: p[@"arg"];
+ if (!title) fail(@"menu.pick takes the row's title");
+ NSDictionary *where = onMain(^id {
+ if (!shownMenu()) fail(@"no menu is up");
+ UIView *row = titled(title, ^BOOL(UIView *v) { return insideMenu(v); });
+ if (!row) fail(@"the menu has no row \"%@\"", title);
+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(row)], @"target" : describe(row)};
+ });
+ return touchAt(where, 0.08);
+ },
+ @"seek" : ^NSDictionary *(NSDictionary *p) {
+ double seconds = [(p[@"seconds"] ?: p[@"arg"]) doubleValue];
+ return onMain(^id {
+ [requirePlayer() seekTo:seconds];
+ return @{@"seconds" : @(seconds)};
+ });
+ },
+ @"play" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ [requirePlayer() resume:nil];
+ return @{};
+ });
+ },
+ @"pause" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ [requirePlayer() pause:nil];
+ return @{};
+ });
+ },
+ @"next" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ [requirePlayer() skipToNextTrackWithOptions:nil];
+ return @{};
+ });
+ },
+ @"tab" : ^NSDictionary *(NSDictionary *p) {
+ NSString *which = p[@"title"] ?: p[@"index"] ?: p[@"arg"];
+ if (!which) fail(@"tab takes an index (0 is the leading tab) or a title");
+ NSDictionary *where = onMain(^id {
+ NSMutableArray<NSString *> *titles = [NSMutableArray array];
+ NSArray<UIView *> *items = tabItems(titles);
+ if (!items.count) fail(@"no tab bar shown");
+ NSUInteger index = [titles indexOfObjectPassingTest:^BOOL(NSString *t, NSUInteger i, BOOL *stop) { return same(t, which, NO); }];
+ if (index == NSNotFound && !p[@"title"] && [which rangeOfCharacterFromSet:NSCharacterSet.decimalDigitCharacterSet.invertedSet].location == NSNotFound) {
+ index = (NSUInteger)which.integerValue;
+ }
+ if (index == NSNotFound || index >= items.count) fail(@"no tab %@; the tabs are %@", which, [titles componentsJoinedByString:@", "]);
+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(items[index])], @"target" : describe(items[index]), @"tabs" : titles};
+ });
+ NSMutableDictionary *result = [touchAt(where, 0.08) mutableCopy];
+ result[@"tabs"] = where[@"tabs"];
+ return result;
+ },
+ @"settings.open" : ^NSDictionary *(NSDictionary *p) {
+ return onMain(^id {
+ UIViewController *top = topController([NSMutableArray array], [NSMutableArray array]);
+ SGOpenModSettings(top.viewIfLoaded ?: keyWindow());
+ return @{};
+ });
+ },
+ @"settings.page" : ^NSDictionary *(NSDictionary *p) {
+ NSString *title = p[@"title"] ?: p[@"arg"];
+ if (!title) fail(@"settings.page takes the row's title");
+ NSDictionary *where = onMain(^id {
+ UIView *row = titled(title, nil);
+ UIScrollView *list = mainScrollView();
+ // Down the page a screen at a time until the row is laid out.
+ if (!row && list) {
+ CGPoint offset = clampedOffset(list, CGPointMake(list.contentOffset.x, -CGFLOAT_MAX));
+ for (NSUInteger step = 0; step < 50 && !row; step++) {
+ [list setContentOffset:offset animated:NO];
+ [list layoutIfNeeded];
+ row = titled(title, nil);
+ CGPoint next = clampedOffset(list, CGPointMake(offset.x, offset.y + list.bounds.size.height * 0.6));
+ if (CGPointEqualToPoint(next, offset)) break;
+ offset = next;
+ }
+ }
+ if (!row) fail(@"no row \"%@\" on this page", title);
+ if (list && [row isDescendantOfView:list]) {
+ CGRect frame = [row convertRect:row.bounds toView:list];
+ CGPoint offset = clampedOffset(list, CGPointMake(list.contentOffset.x, CGRectGetMidY(frame) - list.bounds.size.height / 2));
+ [list setContentOffset:offset animated:NO];
+ [list layoutIfNeeded];
+ }
+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(row)], @"target" : describe(row)};
+ });
+ return touchAt(where, 0.08);
+ },
+ };
+ });
+ return all;
+}
+
+static NSSet<NSString *> *readOnly(void) {
+ return [NSSet setWithArray:@[@"find", @"state", @"screenshot", @"log", @"wait"]];
+}
+
+BOOL SGDriverHandles(NSString *command) {
+ return commands()[command] != nil;
+}
+
+NSData *SGDriverRun(NSString *command, NSDictionary<NSString *, NSString *> *params) {
+ NSMutableDictionary *answer = [NSMutableDictionary dictionaryWithObject:command forKey:@"cmd"];
+ if (![readOnly() containsObject:command]) sg_actionSeq = SGLogLastSeq();
+ @try {
+ [answer addEntriesFromDictionary:commands()[command](params) ?: @{}];
+ answer[@"ok"] = @YES;
+ } @catch (NSException *exception) {
+ answer[@"ok"] = @NO;
+ answer[@"error"] = exception.reason ?: exception.name;
+ }
+ NSData *json = [NSJSONSerialization isValidJSONObject:answer] ? [NSJSONSerialization dataWithJSONObject:answer options:0 error:nil] : nil;
+ if (!json) json = [NSJSONSerialization dataWithJSONObject:@{@"cmd" : command, @"ok" : @NO, @"error" : @"the answer was not JSON"} options:0 error:nil];
+ return json;
+}
+#endif
diff --git a/tweak/Sources/Diagnostics/PlayerState.x b/tweak/Sources/Diagnostics/PlayerState.x
index 0b068576..7c2b497f 100644
--- a/tweak/Sources/Diagnostics/PlayerState.x
+++ b/tweak/Sources/Diagnostics/PlayerState.x
@@ -87,9 +87,16 @@ static void report(SPTPlayerState *state) {
if (track) SGLogLong(@"player track metadata", track.metadata.description);
}
+static __weak id sg_player;
+
+id SGDiagnosticsPlayer(void) {
+ return sg_player;
+}
+
%hook SPTEsperantoPlayer
- (id)state {
id state = %orig;
+ sg_player = self;
if (SGIsDebugBuild() && [state isKindOfClass:NSClassFromString(@"SPTPlayerState")]) report(state);
return state;
}