Liquid Glass and Apple Music touches for the Spotify iOS app. A GPL-3.0 fork of spoti.pw 0.21.1.
C
chore: the phone driver takes Mac commands over USB or an authenticated tailnet
My-Name-Is-Jeff06aa13883ee8
diff --git a/docs/tweaks.md b/docs/tweaks.mdindex bef3b894..aff7278c 100644--- a/docs/tweaks.md+++ b/docs/tweaks.md@@ -17,7 +17,7 @@tweak/Sources/Redesigned/ the redesign, running only while Redesigned UI is ontweak/Sources/App/ what brings the layers together: Mod Settings' root and composed pages, the Mod page,backup and signing, the welcome tour- tweak/Sources/Diagnostics/ screen dumps, the tree server and the main thread hang sampler of FLEX builds+ tweak/Sources/Diagnostics/ screen dumps, the tree server, the phone driver and the main thread hang sampler of FLEX buildsextension/LiveActivity/ the Live Activity widget, a WidgetKit extension of its ownextension/AppGroups/ a dylib loaded by Spotify and its home screen widget that moves Spotify's App Groupsinto a group the re-signed IPA has; without it the widget stays a placeholder@@ -514,6 +514,46 @@ and Live Activity. The root page in `App/ModSettings.x` links the Appearance pagmake log # stream [spotifyglass] log lines from the phonemake flags # regenerate the flag table from the IPA+## Diagnostics++A build with FLEX in it (`make build`, `make install FLEX=1`) runs `Diagnostics/`: the tree server, the hang+sampler, and the phone driver, which lets an agent on the Mac work the app without anyone touching the phone.+The server listens on the phone's loopback only, 127.0.0.1:8085, which the Mac reaches through iproxy over+USB and nothing on Wi-Fi reaches. `GET /tree` answers with the visible screen's view tree, as `make trees`+records it. The driver adds `GET /<command>?<params>`, each answering JSON with `ok` and either what happened+or an `error`. A command needs an `X-Phone-Driver` header, which a web page on the phone cannot send to the+loopback without a CORS preflight the server never grants (`curl -H 'X-Phone-Driver: 1'`). It is compiled only with `SG_DRIVER=1`, which `scripts/pipeline.sh` sets for the builds that+carry FLEX; `make release` and the release workflow's `.deb` leave it out, and it starts only when FLEX is+there.++`scripts/phone.py` is the client. It uses whatever already answers on 127.0.0.1:8085 or starts iproxy and+leaves it running:++ scripts/phone.py tree | state | log [--since N]+ scripts/phone.py find --class UILabel+ scripts/phone.py tap --id X | --label X | --text X | --class C [--index N] | --at X,Y+ scripts/phone.py longpress --id X [--duration 1]+ scripts/phone.py swipe --from X,Y --to X,Y [--duration 0.2] [--drag 1]+ scripts/phone.py scroll [--id X] --by 0,400+ scripts/phone.py type "text"+ scripts/phone.py player.open | player.close | player.more | play | pause | next | seek 42+ scripts/phone.py menu.pick "Sleep timer"+ scripts/phone.py tab 0 | tab Search+ scripts/phone.py settings.open | settings.page "Appearance"+ scripts/phone.py wait --id X [--gone 1] | --menu 1 | --log "system menu: the player's menu is up" [--timeout 3]+ scripts/phone.py screenshot out.png++Views are addressed as the tree prints them: `id=` is the accessibility identifier, `a11y=` the accessibility+label, the quoted text a label's, and a class with an index counts that class's views in the tree's order.+Points are screen points, the frames `find` prints. Touches are made as a finger's (`harness/tabbar/touch.m`'s+way: a UITouch and the IOHIDEvent gesture recognizers read, through `-[UIApplication sendEvent:]`), so+controls get their control events, recognizers recognize, and a button whose menu is its primary action opens+it on the touch down. `menu.pick` taps a row of the system's menu, not of Spotify's own sheet (tap that by+`--text`). `log` reads the app's last 2000 `SGLog` lines from memory, so no device log capture is needed;+a log wait looks from the start of the last command that did something. Everything runs on the main thread+through `dispatch_async` and a semaphore with a timeout, waited on from the server's thread only. Checked in+the simulator by harness/driver/.+## Mod SettingsMod Settings, opened by holding Home on the tab bar or from the first row of the side drawer and thediff --git a/harness/driver/README.md b/harness/driver/README.mdnew file mode 100644index 00000000..c8a5f953--- /dev/null+++ b/harness/driver/README.md@@ -0,0 +1,21 @@+# Phone driver harness++`Diagnostics/Diagnostics.x`'s tree server and `Diagnostics/Driver.m`, compiled as a FLEX build compiles them+(`SG_DRIVER=1`), run in the simulator on a mock app, and `proof.py` drives it through `scripts/phone.py` the+way an agent drives the phone. Simulator apps share the Mac's loopback, so no iproxy; the harness serves on+127.0.0.1:8095 (`PORT=` at build time) so an iproxy to the phone keeps 8085.++ THEOS=$HOME/theos ./build.sh+ xcrun simctl install <udid> build/DriverHarness.app+ xcrun simctl launch <udid> com.vitrine.driverharness+ ./proof.py++The mock (`main.m`): a tab bar (Home, Search); on Home a button with a touch-up-inside action, a view with a+tap recognizer, a button whose menu is its primary action, a scroll view and a text field; a now playing bar+under Spotify's bar controller's class name that presents a player under `NPVScrollViewController`'s, with the+down arrow and the ⋯ by Spotify's identifiers and a system menu on the ⋯; a player behind+`SGDiagnosticsPlayer`; and `SGOpenModSettings` pushing a 40-row list. Each reacts with an `SGLog` line, read+back through the driver's `log` and `wait --log`, and `proof.py` prints a `PASS` or `FAIL` line per check:+the button's action, the recognizer, the menu up before the finger lifts, a pick, a scroll, a fling, typing,+a screenshot, the player's open, ⋯, pick and close, seek/pause/play/next, tabs by title and index, Mod+Settings and a row below its fold, and the tree served while a wait runs.diff --git a/harness/driver/build.sh b/harness/driver/build.shnew file mode 100755index 00000000..ccce2d22--- /dev/null+++ b/harness/driver/build.sh@@ -0,0 +1,41 @@+#!/bin/sh+# Builds the phone driver harness for the simulator: Diagnostics.x's tree server and Driver.m, as a FLEX build+# compiles them (SG_DRIVER=1), on a mock app (main.m). It serves on 127.0.0.1:8095 (PORT=), not the phone's+# 8085, so an iproxy to the phone keeps its port.+set -e+SRC=$(cd "$(dirname "$0")/../../tweak/Sources" && pwd)+OUT=$(dirname "$0")/build+rm -rf "$OUT"; mkdir -p "$OUT/gen" "$OUT/DriverHarness.app"+"$THEOS/bin/logos.pl" -c generator=internal "$SRC/Diagnostics/Diagnostics.x" > "$OUT/gen/Diagnostics.m"++SDK=$(xcrun --sdk iphonesimulator --show-sdk-path)+xcrun -sdk iphonesimulator clang -target arm64-apple-ios17.4-simulator -fobjc-arc -g -O0 -DSG_DRIVER=1 -DSG_TREE_PORT=${PORT:-8095} \+ -I"$SRC" -I"$SRC/Diagnostics" -isysroot "$SDK" -Wno-deprecated-declarations \+ "$(dirname "$0")/main.m" "$(dirname "$0")/../scene.m" "$OUT"/gen/*.m "$SRC"/Diagnostics/Driver.m \+ "$SRC"/Core/SGLog.m "$SRC"/Core/SGViewTree.m \+ -framework UIKit -framework QuartzCore -framework CoreGraphics -framework Foundation \+ -o "$OUT/DriverHarness.app/DriverHarness"++cat > "$OUT/DriverHarness.app/Info.plist" <<'PLIST'+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0"><dict>+<key>CFBundleExecutable</key><string>DriverHarness</string>+<key>CFBundleIdentifier</key><string>com.vitrine.driverharness</string>+<key>CFBundleName</key><string>DriverHarness</string>+<key>CFBundleVersion</key><string>1</string>+<key>CFBundleShortVersionString</key><string>1.0</string>+<key>UIUserInterfaceStyle</key><string>Dark</string>+<key>UILaunchScreen</key><dict/>+<key>UIApplicationSceneManifest</key><dict>+ <key>UIApplicationSupportsMultipleScenes</key><false/>+ <key>UISceneConfigurations</key><dict>+ <key>UIWindowSceneSessionRoleApplication</key><array><dict>+ <key>UISceneConfigurationName</key><string>Default</string>+ <key>UISceneDelegateClassName</key><string>SGRHarnessScene</string>+ </dict></array>+ </dict>+</dict>+</dict></plist>+PLIST+echo "built $OUT/DriverHarness.app"diff --git a/harness/driver/main.m b/harness/driver/main.mnew file mode 100644index 00000000..6eb9e772--- /dev/null+++ b/harness/driver/main.m@@ -0,0 +1,270 @@+// The phone driver (Diagnostics/Driver.m) on a mock app in the simulator: Diagnostics.x's tree server starts+// as in a FLEX build (a FLEXManager class is all it looks for) and proof.py drives it through scripts/phone.py+// over the Mac's loopback, which simulator apps share, so no iproxy.+//+// The mock: a tab bar (Home, Search); on Home a button, a view with a tap recognizer, a button whose menu is+// its primary action, a scroll view and a text field; a now playing bar under Spotify's bar controller's class+// name that opens a player under NPVScrollViewController's, with the down arrow and the ⋯ by Spotify's+// identifiers and a menu on the ⋯; a player behind SGDiagnosticsPlayer; and SGOpenModSettings pushing a long+// list. Everything that reacts says so with an SGLog line, which the driver's log command reads back.+#import <UIKit/UIKit.h>+#import "Core/SGCore.h"+#import "Diagnostics/Diagnostics.h"++// Diagnostics.x's SGIsDebugBuild looks for FLEX by this class.+@interface FLEXManager : NSObject+@end+@implementation FLEXManager+@end++#pragma mark - the player++@interface HarnessTrack : NSObject+@property (nonatomic, readonly) NSString *trackTitle, *artistName;+@property (nonatomic, readonly) id URI;+@end+@implementation HarnessTrack+- (NSString *)trackTitle { return @"Harness Song"; }+- (NSString *)artistName { return @"Harness Artist"; }+- (id)URI { return @"spotify:track:harness"; }+@end++@interface HarnessState : NSObject+@property (nonatomic) BOOL paused;+@property (nonatomic) double at;+@end+@implementation HarnessState+- (HarnessTrack *)track { return [HarnessTrack new]; }+- (BOOL)isPaused { return self.paused; }+- (BOOL)isPlaying { return YES; }+- (double)position { return self.at; }+- (double)positionAsOfTimestamp { return self.at; }+- (double)duration { return 200; }+@end++@interface HarnessPlayer : NSObject+@property (nonatomic, strong) HarnessState *state;+@end+@implementation HarnessPlayer+- (instancetype)init {+ if ((self = [super init])) _state = [HarnessState new];+ return self;+}+- (void)seekTo:(double)seconds { self.state.at = seconds; SGLog(@"harness: player seekTo %.0f", seconds); }+- (id)pause:(id)options { self.state.paused = YES; SGLog(@"harness: player pause"); return nil; }+- (id)resume:(id)options { self.state.paused = NO; SGLog(@"harness: player resume"); return nil; }+- (id)skipToNextTrackWithOptions:(id)options { SGLog(@"harness: player next"); return nil; }+@end++static HarnessPlayer *sg_player;+id SGDiagnosticsPlayer(void) { return sg_player; }++#pragma mark - Mod Settings++@interface HarnessSettings : UITableViewController+@end+@implementation HarnessSettings+- (void)viewDidLoad {+ [super viewDidLoad];+ self.title = @"Mod Settings";+ [self.tableView registerClass:UITableViewCell.class forCellReuseIdentifier:@"row"];+}+- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section { return 40; }+- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {+ UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:@"row" forIndexPath:indexPath];+ cell.textLabel.text = [NSString stringWithFormat:@"Row %ld", (long)indexPath.row];+ return cell;+}+- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {+ SGLog(@"harness: settings row Row %ld", (long)indexPath.row);+}+@end++static UINavigationController *sg_home;+void SGOpenModSettings(UIView *source) {+ SGLog(@"harness: SGOpenModSettings from a %@", source.class);+ [sg_home pushViewController:[HarnessSettings new] animated:NO];+}++#pragma mark - the player screen and the bar, under Spotify's class names++static UIButton *menuButton(NSString *title, NSString *identifier, NSArray<NSString *> *rows, NSString *what);++@interface _TtC21NowPlaying_ScrollImpl23NPVScrollViewController : UIViewController+@end+@implementation _TtC21NowPlaying_ScrollImpl23NPVScrollViewController+- (void)viewDidLoad {+ [super viewDidLoad];+ self.view.backgroundColor = UIColor.darkGrayColor;+ UIButton *close = [UIButton buttonWithType:UIButtonTypeSystem];+ [close setTitle:@"Down" forState:UIControlStateNormal];+ close.accessibilityIdentifier = @"now-playing-minimize-button";+ close.frame = CGRectMake(12, 70, 48, 48);+ [close addTarget:self action:@selector(close) forControlEvents:UIControlEventTouchUpInside];+ [self.view addSubview:close];+ UIButton *more = menuButton(@"⋯", @"Context menu", @[@"Share", @"Go to album", @"Sleep timer"], @"player menu");+ more.frame = CGRectMake(340, 70, 48, 48);+ [self.view addSubview:more];+}+- (void)close {+ SGLog(@"harness: player down arrow");+ [self dismissViewControllerAnimated:NO completion:nil];+}+@end++@interface _TtC18NowPlaying_BarImpl27NowPlayingBarViewController : UIViewController+@end+@implementation _TtC18NowPlaying_BarImpl27NowPlayingBarViewController+- (void)viewDidLoad {+ [super viewDidLoad];+ self.view.backgroundColor = UIColor.systemIndigoColor;+ [self.view addGestureRecognizer:[[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(open)]];+}+- (void)open {+ SGLog(@"harness: bar tapped");+ UIViewController *player = [_TtC21NowPlaying_ScrollImpl23NPVScrollViewController new];+ player.modalPresentationStyle = UIModalPresentationFullScreen;+ [self presentViewController:player animated:NO completion:nil];+}+@end++#pragma mark - Home++// A button whose menu is its primary action; says when the menu is about to show.+@interface HarnessMenuButton : UIButton+@property (nonatomic, copy) NSString *what;+@end+@implementation HarnessMenuButton+- (void)contextMenuInteraction:(UIContextMenuInteraction *)interaction+ willDisplayMenuForConfiguration:(UIContextMenuConfiguration *)configuration+ animator:(id<UIContextMenuInteractionAnimating>)animator {+ SGLog(@"harness: %@ will display", self.what);+ if ([UIButton instancesRespondToSelector:_cmd]) [super contextMenuInteraction:interaction willDisplayMenuForConfiguration:configuration animator:animator];+}+@end++static UIButton *menuButton(NSString *title, NSString *identifier, NSArray<NSString *> *rows, NSString *what) {+ HarnessMenuButton *button = [HarnessMenuButton buttonWithType:UIButtonTypeSystem];+ button.what = what;+ [button setTitle:title forState:UIControlStateNormal];+ button.accessibilityIdentifier = identifier;+ NSMutableArray<UIAction *> *actions = [NSMutableArray array];+ for (NSString *row in rows) {+ [actions addObject:[UIAction actionWithTitle:row image:nil identifier:nil handler:^(UIAction *action) {+ SGLog(@"harness: %@ picked %@", what, row);+ }]];+ }+ button.menu = [UIMenu menuWithChildren:actions];+ button.showsMenuAsPrimaryAction = YES;+ return button;+}++@interface HarnessHome : UIViewController+@property (nonatomic) NSInteger taps;+@end+@implementation HarnessHome+- (void)viewDidLoad {+ [super viewDidLoad];+ self.title = @"Home";+ self.view.backgroundColor = UIColor.blackColor;++ UIButton *counter = [UIButton buttonWithType:UIButtonTypeSystem];+ counter.frame = CGRectMake(20, 120, 160, 44);+ counter.accessibilityIdentifier = @"counter";+ [counter setTitle:@"Counter" forState:UIControlStateNormal];+ [counter addTarget:self action:@selector(count:) forControlEvents:UIControlEventTouchUpInside];+ [self.view addSubview:counter];++ UIView *tapView = [[UIView alloc] initWithFrame:CGRectMake(220, 120, 160, 44)];+ tapView.backgroundColor = UIColor.systemTealColor;+ tapView.accessibilityIdentifier = @"tapview";+ [tapView addGestureRecognizer:[[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(recognized)]];+ [self.view addSubview:tapView];++ UIButton *menu = menuButton(@"Menu", @"menubutton", @[@"Alpha", @"Sleep timer"], @"menu");+ menu.frame = CGRectMake(20, 180, 160, 44);+ [self.view addSubview:menu];++ UIScrollView *scroller = [[UIScrollView alloc] initWithFrame:CGRectMake(20, 240, 362, 220)];+ scroller.accessibilityIdentifier = @"scroller";+ scroller.backgroundColor = [UIColor colorWithWhite:0.15 alpha:1];+ scroller.contentSize = CGSizeMake(362, 3000);+ for (NSInteger i = 0; i < 60; i++) {+ UILabel *line = [[UILabel alloc] initWithFrame:CGRectMake(12, i * 50, 300, 44)];+ line.text = [NSString stringWithFormat:@"Line %ld", (long)i];+ line.textColor = UIColor.whiteColor;+ [scroller addSubview:line];+ }+ [self.view addSubview:scroller];++ UITextField *field = [[UITextField alloc] initWithFrame:CGRectMake(20, 480, 362, 40)];+ field.accessibilityIdentifier = @"field";+ field.borderStyle = UITextBorderStyleRoundedRect;+ field.placeholder = @"Type here";+ [field addTarget:self action:@selector(edited:) forControlEvents:UIControlEventEditingChanged];+ [self.view addSubview:field];++ UIViewController *bar = [_TtC18NowPlaying_BarImpl27NowPlayingBarViewController new];+ [self addChildViewController:bar];+ bar.view.frame = CGRectMake(8, 640, 386, 56);+ [self.view addSubview:bar.view];+ [bar didMoveToParentViewController:self];+}+- (void)count:(UIButton *)button {+ self.taps++;+ [button setTitle:[NSString stringWithFormat:@"Tapped %ld", (long)self.taps] forState:UIControlStateNormal];+ SGLog(@"harness: button tapped %ld", (long)self.taps);+}+- (void)recognized {+ SGLog(@"harness: recognizer tap");+}+- (void)edited:(UITextField *)field {+ SGLog(@"harness: field text %@", field.text);+}+@end++@interface HarnessSearch : UIViewController+@end+@implementation HarnessSearch+- (void)viewDidLoad {+ [super viewDidLoad];+ self.title = @"Search";+ self.view.backgroundColor = UIColor.systemGray6Color;+}+@end++// Says when a finger lifts, so the proof can tell a menu opened while it was still down.+@interface HarnessWindow : UIWindow+@end+@implementation HarnessWindow+- (void)sendEvent:(UIEvent *)event {+ for (UITouch *touch in event.allTouches) {+ if (touch.phase == UITouchPhaseEnded) SGLog(@"harness: finger up");+ }+ [super sendEvent:event];+}+@end++@interface SGHarnessApp : UIResponder <UIApplicationDelegate>+@property (nonatomic, strong) UIWindow *window;+@end+@implementation SGHarnessApp+- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)options {+ sg_player = [HarnessPlayer new];+ sg_home = [[UINavigationController alloc] initWithRootViewController:[HarnessHome new]];+ sg_home.tabBarItem = [[UITabBarItem alloc] initWithTitle:@"Home" image:[UIImage systemImageNamed:@"house"] tag:0];+ UIViewController *search = [HarnessSearch new];+ search.tabBarItem = [[UITabBarItem alloc] initWithTitle:@"Search" image:[UIImage systemImageNamed:@"magnifyingglass"] tag:1];+ UITabBarController *tabs = [UITabBarController new];+ tabs.viewControllers = @[sg_home, search];+ self.window = [[HarnessWindow alloc] initWithFrame:UIScreen.mainScreen.bounds];+ self.window.rootViewController = tabs;+ return YES;+}+@end++int main(int argc, char *argv[]) {+ @autoreleasepool {+ return UIApplicationMain(argc, argv, nil, NSStringFromClass(SGHarnessApp.class));+ }+}diff --git a/harness/driver/proof.py b/harness/driver/proof.pynew file mode 100755index 00000000..47a020d9--- /dev/null+++ b/harness/driver/proof.py@@ -0,0 +1,127 @@+#!/usr/bin/env python3+"""Drives the driver harness (main.m) through scripts/phone.py and prints PASS or FAIL per check.++ THEOS=$HOME/theos ./build.sh+ xcrun simctl install <udid> build/DriverHarness.app && xcrun simctl launch <udid> com.vitrine.driverharness+ ./proof.py # PHONE_PORT=8095 unless set, the port build.sh serves on+"""+import json+import os+import struct+import subprocess+import sys+import tempfile+import threading+import time++HERE = os.path.dirname(os.path.abspath(__file__))+PHONE = os.path.join(HERE, "..", "..", "scripts", "phone.py")+os.environ.setdefault("PHONE_PORT", "8095")+failures = 0+++def phone(*args):+ out = subprocess.run([sys.executable, PHONE, *args], capture_output=True, text=True).stdout+ return out if args[0] == "tree" else json.loads(out)+++def check(ok, what):+ global failures+ failures += not ok+ print(f"{'PASS' if ok else 'FAIL'} {what}")+++def lines(since):+ return [line["text"] for line in phone("log", "--since", str(since))["lines"]]+++def last():+ return phone("state")["logSeq"]+++phone("tab", "Home")+check("== window" in phone("tree"), "tree: GET /tree still answers with the tree")+check(phone("state")["top"] == "HarnessHome", "state: the top controller is Home")++start = last()+answer = phone("tap", "--id", "counter")+check(answer["ok"] and answer["hit"] == "UIButton", "tap --id: the finger lands on the button")+check(any(t.startswith("harness: button tapped") for t in lines(start)), "tap --id: the button's touch-up-inside action ran")++start = last()+phone("tap", "--id", "tapview")+check("harness: recognizer tap" in lines(start), "tap: a UITapGestureRecognizer recognized the touch")++start = last()+phone("longpress", "--id", "menubutton", "--duration", "1")+log = lines(start)+check("harness: menu will display" in log and "harness: finger up" in log+ and log.index("harness: menu will display") < log.index("harness: finger up"),+ "longpress: a primary-action menu opens on the touch down, before the finger lifts")+check(phone("state")["menu"] is True, "state: menu is up")+start = last()+check(phone("menu.pick", "Alpha")["ok"], "menu.pick: the row is found and tapped")+check(phone("wait", "--log", "harness: menu picked Alpha", "--timeout", "2")["ok"], "menu.pick: the row's action ran")+check(phone("wait", "--menu", "1", "--gone", "1", "--timeout", "2")["ok"], "wait --menu --gone: the menu closed")++answer = phone("scroll", "--id", "scroller", "--by", "0,500")+check(answer["ok"] and answer["after"] == [0, 500], "scroll: the scroll view moved by the offset")+phone("swipe", "--id", "scroller", "--by", "0,-300")+time.sleep(1.5)+offset = phone("find", "--id", "scroller")["views"][0]["contentOffset"][1]+check(offset > 850, f"swipe: a real fling moved the list past the 300 pt of the swipe ({offset:.0f})")++start = last()+phone("tap", "--id", "field")+phone("type", "hello driver")+check("harness: field text hello driver" in lines(start), "type: the text went into the tapped field")++shot = os.path.join(tempfile.gettempdir(), f"driver-proof-{os.getpid()}.png")+answer = phone("screenshot", shot)+with open(shot, "rb") as f:+ head = f.read(24)+width, height = struct.unpack(">II", head[16:24])+check(answer["ok"] and head[:8] == b"\x89PNG\r\n\x1a\n" and width > 1000, f"screenshot: a {width}x{height} PNG came back")+os.remove(shot)++check(phone("player.open")["ok"] and phone("wait", "--id", "Context menu")["ok"], "player.open: the bar opened the player")+check(phone("player.more")["ok"], "player.more: the ⋯ was tapped")+check(phone("wait", "--log", "harness: player menu will display", "--timeout", "3")["ok"], "wait --log: the ⋯'s menu came up")+phone("menu.pick", "Sleep timer")+check(phone("wait", "--log", "harness: player menu picked Sleep timer", "--timeout", "3")["ok"], "menu.pick: Sleep timer picked in the player's menu")+phone("player.close")+check(phone("state")["player"] is False, "player.close: the down arrow closed the player")++start = last()+phone("seek", "42")+phone("pause")+state = phone("state")["nowPlaying"]+phone("play")+phone("next")+log = lines(start)+check(state["position"] == 42 and state["paused"] and {"harness: player seekTo 42", "harness: player pause", "harness: player resume", "harness: player next"} <= set(log),+ "seek, pause, play, next: the player was told, and state reads it back")++phone("tab", "Search")+check(phone("state")["top"] == "HarnessSearch", "tab by title: Search is shown")+phone("tab", "0")+check(phone("state")["top"] == "HarnessHome", "tab by index: Home is shown again")++phone("settings.open")+check(phone("state")["top"] == "HarnessSettings", "settings.open: Mod Settings is pushed")+check(phone("settings.page", "Row 33")["ok"] and phone("wait", "--log", "harness: settings row Row 33")["ok"],+ "settings.page: a row below the fold is scrolled to and tapped")++check(phone("tap", "--id", "nope")["error"] == "no view matches id=nope", "tap: a missing view is an error, not a crash")++# The tree still answers while a command waits.+waiting = threading.Thread(target=phone, args=("wait", "--log", "never", "--timeout", "2"))+waiting.start()+time.sleep(0.3)+began = time.time()+phone("tree")+check(time.time() - began < 1 and waiting.is_alive(), "the tree is served while a wait runs")+waiting.join()++print("done," , "all passed" if not failures else f"{failures} failed")+sys.exit(1 if failures else 0)diff --git a/scripts/phone.py b/scripts/phone.pynew file mode 100755index 00000000..793a53ce--- /dev/null+++ b/scripts/phone.py@@ -0,0 +1,155 @@+#!/usr/bin/env python3+"""Drive the app on the iPhone from the Mac: the phone driver of FLEX builds (Diagnostics/Driver.m).++ scripts/phone.py tree # the visible screen's view tree, as make trees records it+ scripts/phone.py state # now playing, the top controller, what is presented, menu up+ scripts/phone.py find --class UILabel # every match with its index, frame (screen points), id, label, text+ scripts/phone.py tap --id X | --label X | --text X | --class C [--index N] | --at X,Y+ scripts/phone.py longpress --id X [--duration 1]+ scripts/phone.py swipe --from X,Y --to X,Y [--duration 0.2] # or a view and --by DX,DY; a quick one flings+ scripts/phone.py swipe --id X --by 0,-300 --drag 1 # a slow drag held still before letting go+ scripts/phone.py scroll [--id X | --at X,Y] --by 0,400 [--animated 1] # the page's list when no view is named+ scripts/phone.py type "some text" # into the first responder (tap a field first)+ scripts/phone.py player.open | player.close | player.more+ scripts/phone.py menu.pick "Sleep timer"+ scripts/phone.py play | pause | next+ scripts/phone.py seek 42+ scripts/phone.py tab 0 | tab Search+ scripts/phone.py settings.open+ scripts/phone.py settings.page "Appearance"+ scripts/phone.py wait --id X [--gone 1] [--timeout 5]+ scripts/phone.py wait --menu 1 [--gone 1]+ scripts/phone.py wait --log "system menu: the player's menu is up" --timeout 3+ scripts/phone.py log [--since N] # the app's last SGLog lines; each has a seq+ scripts/phone.py screenshot out.png [--scale 1]++Every command prints the app's JSON answer, {"ok": true, ...} or {"ok": false, "error": ...}, and exits 1+when it is not ok. --label and --text match whole and ignore case; --contains 1 matches a part. Points are+screen points, the frame `find` prints. A log wait looks at the lines since the last command that did+something began, so `player.more` then `wait --log ...` sees a line that came before the wait.++The server listens on the phone's loopback only, port 8085, so this goes through iproxy over USB: it uses+whatever already answers on 127.0.0.1:8085 (an iproxy left running, or the simulator, whose apps share the+Mac's loopback) and otherwise starts iproxy and leaves it running for the next call. PHONE_PORT changes the+port. Over Tailscale: PHONE_HOST=100.71.6.75 scripts/phone.py state (PHONE_TOKEN, or the token from the+device log; FLEX builds listen on the phone's Tailscale address too, token required). Needs a FLEX build (make install FLEX=1) open in the foreground: iOS suspends it in the background.+"""+import base64+import json+import os+import re+import shutil+import socket+import subprocess+import sys+import time+import urllib.error+import urllib.parse+import urllib.request++PORT = int(os.environ.get("PHONE_PORT", "8085"))+# Over Tailscale instead of the cable: the phone's tailnet address and the token the app logs at launch+# ("driver: tailnet listener on 100.x.y.z:8085, token ..."), read from the device log when not given.+HOST = os.environ.get("PHONE_HOST")+LOG = "/tmp/claude-501/device.log"+++def token():+ if os.environ.get("PHONE_TOKEN"):+ return os.environ["PHONE_TOKEN"]+ try:+ with open(LOG, errors="replace") as f:+ found = re.findall(r"driver: tailnet listener on \S+, token ([0-9a-f]{32})", f.read())+ except OSError:+ found = []+ if not found:+ sys.exit(f"no PHONE_TOKEN and no token in {LOG}: run once over the cable, or read it from the app's log")+ return found[-1]+# The argument a command takes without a flag.+POSITIONAL = {"type": "text", "menu.pick": "title", "seek": "seconds", "tab": "arg", "settings.page": "title"}+++def port_open():+ try:+ with socket.create_connection(("127.0.0.1", PORT), timeout=0.5):+ return True+ except OSError:+ return False+++def ensure_tunnel():+ if port_open():+ return+ if not shutil.which("iproxy"):+ sys.exit(f"nothing answers on 127.0.0.1:{PORT} and there is no iproxy (brew install libimobiledevice)")+ subprocess.Popen(["iproxy", f"{PORT}:{PORT}"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)+ for _ in range(30):+ if port_open():+ return+ time.sleep(0.1)+ sys.exit(f"iproxy did not open 127.0.0.1:{PORT}: is the iPhone plugged in and trusted?")+++def fetch(path, timeout):+ # The header the app asks of a driver command, which a web page on the phone cannot send.+ headers = {"X-Phone-Driver": "1"}+ if HOST:+ headers["X-Phone-Token"] = token()+ url = f"http://{HOST}:{PORT}/{path}"+ else:+ ensure_tunnel()+ url = f"http://127.0.0.1:{PORT}/{path}"+ request = urllib.request.Request(url, headers=headers)+ try:+ with urllib.request.urlopen(request, timeout=timeout) as response:+ return response.read().decode("utf-8")+ except urllib.error.HTTPError as e:+ return e.read().decode("utf-8", "replace")+ except (urllib.error.URLError, ConnectionError, socket.timeout) as e:+ sys.exit(f"the app did not answer ({e}): is a FLEX build open in the foreground?")+++def parse(args):+ """`cmd --key value ... [positional]` into the command and its query items."""+ params, rest = {}, list(args)+ while rest:+ arg = rest.pop(0)+ if arg.startswith("--") and len(arg) > 2:+ key = arg[2:]+ params[key] = rest.pop(0) if rest and not rest[0].startswith("--") else "1"+ else:+ params.setdefault("arg", arg)+ return params+++def run(argv):+ if not argv or argv[0] in ("-h", "--help", "help"):+ print(__doc__)+ return 0+ command, params = argv[0], parse(argv[1:])+ if command == "tree":+ print(fetch("tree", 15), end="")+ return 0+ out = None+ if command == "screenshot":+ out = params.pop("arg", None)+ if not out:+ sys.exit("screenshot takes the path to save the PNG to")+ if "arg" in params and command in POSITIONAL:+ params[POSITIONAL[command]] = params.pop("arg")+ timeout = float(params.get("timeout", 5)) + float(params.get("duration", 0)) + 15+ text = fetch(f"{urllib.parse.quote(command)}?{urllib.parse.urlencode(params, quote_via=urllib.parse.quote)}", timeout)+ try:+ answer = json.loads(text)+ except ValueError:+ sys.exit(f"not a driver answer (a build without the driver serves only the tree):\n{text[:300]}")+ if out and answer.get("ok"):+ with open(out, "wb") as f:+ f.write(base64.b64decode(answer.pop("png")))+ answer["saved"] = os.path.abspath(out)+ print(json.dumps(answer, indent=1, ensure_ascii=False, sort_keys=True))+ return 0 if answer.get("ok") else 1+++if __name__ == "__main__":+ sys.exit(run(sys.argv[1:]))diff --git a/scripts/pipeline.sh b/scripts/pipeline.shindex 3d6f24ae..fee132f6 100755--- a/scripts/pipeline.sh+++ b/scripts/pipeline.sh@@ -83,7 +83,8 @@ if ! xcrun -sdk iphoneos --find clang >/dev/null 2>&1; thenTARGET_STRIP=strip TARGET_LIPO=lipo TARGET_CODESIGN_ALLOCATE=codesign_allocate TARGET_LIBTOOL=libtoolfi# Theos builds its Swift support tools only at MAKELEVEL 0, and `make release` hands this script MAKELEVEL 1.-env -u MAKELEVEL gmake -C "$ROOT/tweak" clean package >/dev/null+# A build with FLEX carries the phone driver too (tweak/Makefile, SG_DRIVER); --no-flex leaves it out.+env -u MAKELEVEL gmake -C "$ROOT/tweak" clean package SG_DRIVER="$WITH_FLEX" >/dev/nullTWEAK_DEB="$(ls -t "$ROOT"/tweak/packages/*.deb | head -1)"echo " $TWEAK_DEB"diff --git a/tweak/Makefile b/tweak/Makefileindex 5a0acea2..fcf49958 100644--- a/tweak/Makefile+++ b/tweak/Makefile@@ -23,6 +23,10 @@ TWEAK_NAME := spotifyglass# is generated from the IPA by scripts/extract-flags.py.spotifyglass_FILES := $(shell find Sources -name '*.m' -o -name '*.x' -o -name '*.swift' | sort)spotifyglass_CFLAGS := -fobjc-arc -ISources -DSG_VERSION=\"$(SG_VERSION)\"+# The phone driver (Diagnostics/Driver.m) goes only into builds that carry FLEX: scripts/pipeline.sh sets+# SG_DRIVER=1 for those, and every other build (make release, the release workflow's .deb) leaves it out.+SG_DRIVER ?= 0+spotifyglass_CFLAGS += -DSG_DRIVER=$(SG_DRIVER)spotifyglass_FRAMEWORKS := UIKit QuartzCore UniformTypeIdentifiers MediaPlayer CoreImage AudioToolbox AVFoundation CoreHaptics Accelerate CoreMotion CoreML CoreText# zlib unpacks KuGou's KRC lyrics (Shared/LyricsSources/KuGou.m).spotifyglass_LIBRARIES := zdiff --git a/tweak/Sources/Core/SGLog.h b/tweak/Sources/Core/SGLog.hindex 2d8c6b3a..68db729e 100644--- a/tweak/Sources/Core/SGLog.h+++ b/tweak/Sources/Core/SGLog.h@@ -2,7 +2,21 @@#import <os/log.h>// %{public}s so idevicesyslog on the Mac sees the text instead of <private>.+#if SG_DRIVER+// A build with the phone driver (FLEX builds, SG_DRIVER in tweak/Makefile) also keeps the last lines in+// memory, for the driver's log and wait commands (Diagnostics/Driver.m).+#define SGLog(fmt, ...) do { \+ NSString *sg_log_line_ = [NSString stringWithFormat:(fmt), ##__VA_ARGS__]; \+ os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_DEFAULT, "[spotifyglass] %{public}s", sg_log_line_.UTF8String); \+ SGLogRemember(sg_log_line_); \+} while (0)+void SGLogRemember(NSString *line);+// The remembered lines numbered after `after`, oldest first, each {seq, t (seconds since 1970), text}.+NSArray<NSDictionary *> *SGLogRecent(uint64_t after);+uint64_t SGLogLastSeq(void);+#else#define SGLog(fmt, ...) os_log_with_type(OS_LOG_DEFAULT, OS_LOG_TYPE_DEFAULT, "[spotifyglass] %{public}s", [NSString stringWithFormat:(fmt), ##__VA_ARGS__].UTF8String)+#endif// Long dumps, split into numbered parts under the unified log's size cap.void SGLogLong(NSString *tag, NSString *text);diff --git a/tweak/Sources/Core/SGLog.m b/tweak/Sources/Core/SGLog.mindex a0144e8d..601b7177 100644--- a/tweak/Sources/Core/SGLog.m+++ b/tweak/Sources/Core/SGLog.m@@ -1,4 +1,41 @@#import "SGLog.h"+#import <os/lock.h>++#if SG_DRIVER+// The last lines, for the phone driver. They come from any thread, so a lock, never the main queue.+static const NSUInteger kRemembered = 2000;+static os_unfair_lock sg_logLock = OS_UNFAIR_LOCK_INIT;+static NSMutableArray<NSDictionary *> *sg_logLines;+static uint64_t sg_logSeq;++void SGLogRemember(NSString *line) {+ // The player's state dumps run to hundreds of parts each and would push every other line out.+ if ([line hasPrefix:@"player state "] || [line hasPrefix:@"player track metadata"]) return;+ NSTimeInterval now = NSDate.date.timeIntervalSince1970;+ os_unfair_lock_lock(&sg_logLock);+ if (!sg_logLines) sg_logLines = [NSMutableArray array];+ [sg_logLines addObject:@{@"seq" : @(++sg_logSeq), @"t" : @(now), @"text" : line ?: @""}];+ if (sg_logLines.count > kRemembered) [sg_logLines removeObjectAtIndex:0];+ os_unfair_lock_unlock(&sg_logLock);+}++NSArray<NSDictionary *> *SGLogRecent(uint64_t after) {+ os_unfair_lock_lock(&sg_logLock);+ NSIndexSet *newer = [sg_logLines indexesOfObjectsPassingTest:^BOOL(NSDictionary *entry, NSUInteger i, BOOL *stop) {+ return [entry[@"seq"] unsignedLongLongValue] > after;+ }];+ NSArray<NSDictionary *> *lines = [sg_logLines objectsAtIndexes:newer] ?: @[];+ os_unfair_lock_unlock(&sg_logLock);+ return lines;+}++uint64_t SGLogLastSeq(void) {+ os_unfair_lock_lock(&sg_logLock);+ uint64_t seq = sg_logSeq;+ os_unfair_lock_unlock(&sg_logLock);+ return seq;+}+#endif// The unified log cuts a message at about 1 KB, so long dumps go out as numbered parts.void SGLogLong(NSString *tag, NSString *text) {diff --git a/tweak/Sources/Diagnostics/Diagnostics.h b/tweak/Sources/Diagnostics/Diagnostics.hindex 73e50f27..c8b61db6 100644--- a/tweak/Sources/Diagnostics/Diagnostics.h+++ b/tweak/Sources/Diagnostics/Diagnostics.h@@ -1,7 +1,21 @@// Screen dumps for FLEX builds: the visible screen's view tree, served over the phone's loopback// and logged when the app goes to the background.-#import <Foundation/Foundation.h>+#import <UIKit/UIKit.h>BOOL SGIsDebugBuild(void);NSString *SGScreenTree(void);void SGDumpScreen(NSString *reason);+// Runs `block` on the main thread and waits for it at most `timeout` seconds; for any thread but main,+// where it just runs the block. NO when main did not get to it in time (the block still runs later).+BOOL SGRunOnMain(NSTimeInterval timeout, dispatch_block_t block);+// The app's SPTEsperantoPlayer, nil until something asked it for its state (PlayerState.x).+id SGDiagnosticsPlayer(void);++#if SG_DRIVER+// The phone driver (Driver.m): whether `command`, a request path without its slash, is one of its+// commands, and running one with the request's query items, answering JSON. Called off the main thread.+BOOL SGDriverHandles(NSString *command);+NSData *SGDriverRun(NSString *command, NSDictionary<NSString *, NSString *> *params);+// Defined in App/ModSettings.x: declared low, defined high.+void SGOpenModSettings(UIView *source);+#endifdiff --git a/tweak/Sources/Diagnostics/Diagnostics.x b/tweak/Sources/Diagnostics/Diagnostics.xindex d244d32e..eca64516 100644--- a/tweak/Sources/Diagnostics/Diagnostics.x+++ b/tweak/Sources/Diagnostics/Diagnostics.x@@ -4,10 +4,16 @@#import "Core/SGCore.h"#import "Diagnostics.h"#import <sys/socket.h>+#import <ifaddrs.h>+#import <arpa/inet.h>#import <netinet/in.h>#import <unistd.h>-static const uint16_t kTreePort = 8085;+// harness/driver builds the server on another port, so it leaves a tunnel to the phone on 8085 alone.+#ifndef SG_TREE_PORT+#define SG_TREE_PORT 8085+#endif+static const uint16_t kTreePort = SG_TREE_PORT;static NSString *hexColor(CGColorRef color) {CGFloat r = 0, g = 0, b = 0, a = 0;@@ -25,7 +31,10 @@ static void appendTree(UIView *view, NSUInteger depth, NSMutableString *out) {if (view.layer.mask) [line appendString:@" masked"];if (view.clipsToBounds) [line appendString:@" clips"];if (view.accessibilityIdentifier.length) [line appendFormat:@" id=%@", view.accessibilityIdentifier];- if ([view isKindOfClass:UIControl.class] && view.accessibilityLabel.length) [line appendFormat:@" a11y=\"%@\"", view.accessibilityLabel];+ // Controls and accessibility elements, the views the phone driver's `tap --label` finds.+ if (([view isKindOfClass:UIControl.class] || (view.isAccessibilityElement && ![view isKindOfClass:UILabel.class])) && view.accessibilityLabel.length) {+ [line appendFormat:@" a11y=\"%@\"", view.accessibilityLabel];+ }if ([view isKindOfClass:UILabel.class]) {UILabel *label = (UILabel *)view;[line appendFormat:@" \"%@\" %.0fpt %@", label.text, label.font.pointSize, hexColor(label.textColor.CGColor)];@@ -96,7 +105,23 @@ void SGDumpScreen(NSString *reason) {SGLogLong([@"screen dump " stringByAppendingString:reason], SGScreenTree());}-// GET anything on 127.0.0.1:kTreePort answers with the current screen's tree as text/plain.+BOOL SGRunOnMain(NSTimeInterval timeout, dispatch_block_t block) {+ if (NSThread.isMainThread) {+ block();+ return YES;+ }+ dispatch_semaphore_t done = dispatch_semaphore_create(0);+ dispatch_async(dispatch_get_main_queue(), ^{+ block();+ dispatch_semaphore_signal(done);+ });+ return dispatch_semaphore_wait(done, dispatch_time(DISPATCH_TIME_NOW, (int64_t)(timeout * NSEC_PER_SEC))) == 0;+}++// The server: GET /tree (or any path that is not a driver command) answers with the current screen's+// tree as text/plain; in a build with the phone driver, GET /<command>?<params> runs that command and+// answers JSON (Driver.m). It listens on 127.0.0.1 only, the phone's loopback, which the Mac reaches+// through usbmux (iproxy) over the cable and nothing on Wi-Fi reaches, and it starts only in FLEX builds.static void sendAll(int client, NSData *data) {const uint8_t *p = data.bytes;size_t left = data.length;@@ -108,38 +133,185 @@ static void sendAll(int client, NSData *data) {}}-static void startTreeServer(void) {+static void respond(int client, NSString *status, NSString *type, NSData *data) {+ NSString *head = [NSString stringWithFormat:@"HTTP/1.0 %@\r\nContent-Type: %@; charset=utf-8\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n",+ status, type, (unsigned long)data.length];+ sendAll(client, [head dataUsingEncoding:NSUTF8StringEncoding]);+ sendAll(client, data);+}++// The request line and headers, up to the blank line that ends them; a GET has no body.+static NSString *readRequest(int client) {+ NSMutableData *request = [NSMutableData data];+ char buffer[4096];+ while (request.length < 64 * 1024) {+ ssize_t n = recv(client, buffer, sizeof(buffer), 0);+ if (n <= 0) break;+ [request appendBytes:buffer length:(NSUInteger)n];+ if ([request rangeOfData:[@"\r\n\r\n" dataUsingEncoding:NSUTF8StringEncoding] options:0 range:NSMakeRange(0, request.length)].location != NSNotFound) break;+ }+ return [[NSString alloc] initWithData:request encoding:NSUTF8StringEncoding] ?: @"";+}++#if SG_DRIVER+static NSString *driverToken(void);+#endif++// `tailnet`: the client came in on the Tailscale listener, which answers nothing without the token.+static void serve(int client, BOOL tailnet) {+ struct timeval timeout = {2, 0};+ setsockopt(client, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));+ int no = 1;+ setsockopt(client, SOL_SOCKET, SO_NOSIGPIPE, &no, sizeof(no));+#if !SG_DRIVER+ readRequest(client);+#else+ NSString *request = readRequest(client);+ if (tailnet) {+ NSString *token = [NSString stringWithFormat:@"\r\nX-Phone-Token: %@\r\n", driverToken()];+ if ([request rangeOfString:token options:NSCaseInsensitiveSearch].location == NSNotFound) {+ respond(client, @"403 Forbidden", @"application/json", [@"{\"ok\":false,\"error\":\"the tailnet listener needs the X-Phone-Token header\"}" dataUsingEncoding:NSUTF8StringEncoding]);+ close(client);+ return;+ }+ }+ NSArray<NSString *> *first = [[request componentsSeparatedByString:@"\r\n"].firstObject componentsSeparatedByString:@" "];+ NSURLComponents *url = first.count > 1 ? [NSURLComponents componentsWithString:[@"http://phone" stringByAppendingString:first[1]]] : nil;+ NSString *command = [url.path stringByTrimmingCharactersInSet:[NSCharacterSet characterSetWithCharactersInString:@"/"]] ?: @"";+ if (SGDriverHandles(command)) {+ // The phone's loopback is every app's on the phone, Safari's pages included. A page can send a GET+ // anywhere but cannot add a header of its own without a CORS preflight, which this server never+ // allows, so a command needs the header scripts/phone.py sends.+ if ([request rangeOfString:@"\r\nX-Phone-Driver:" options:NSCaseInsensitiveSearch].location == NSNotFound) {+ respond(client, @"403 Forbidden", @"application/json", [@"{\"ok\":false,\"error\":\"a driver command needs the X-Phone-Driver header\"}" dataUsingEncoding:NSUTF8StringEncoding]);+ close(client);+ return;+ }+ NSMutableDictionary<NSString *, NSString *> *params = [NSMutableDictionary dictionary];+ for (NSURLQueryItem *item in url.queryItems) params[item.name] = item.value ?: @"1";+ // One command at a time, so two clients' touches never interleave; the tree is served alongside.+ static dispatch_queue_t commands;+ static dispatch_once_t once;+ dispatch_once(&once, ^{ commands = dispatch_queue_create("spotifyglass.driver", DISPATCH_QUEUE_SERIAL); });+ __block NSData *answer = nil;+ dispatch_sync(commands, ^{ answer = SGDriverRun(command, params); });+ respond(client, @"200 OK", @"application/json", answer);+ close(client);+ return;+ }+#endif+ __block NSString *body = nil;+ if (!SGRunOnMain(5, ^{ body = SGScreenTree(); })) {+ respond(client, @"503 Service Unavailable", @"text/plain", [@"the main thread did not answer in 5 s\n" dataUsingEncoding:NSUTF8StringEncoding]);+ } else {+ respond(client, @"200 OK", @"text/plain", [body dataUsingEncoding:NSUTF8StringEncoding]);+ }+ close(client);+}++// Listens on `address` (network order) and serves each client on a thread of its own: a command may wait+// seconds (wait, longpress). Answers the listening socket, -1 when it could not listen.+static int listenOn(in_addr_t address, BOOL tailnet, dispatch_source_t __strong *source) {int fd = socket(AF_INET, SOCK_STREAM, 0);int yes = 1;setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));struct sockaddr_in addr = {0};addr.sin_family = AF_INET;addr.sin_port = htons(kTreePort);- addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);+ addr.sin_addr.s_addr = address;if (bind(fd, (struct sockaddr *)&addr, sizeof(addr)) != 0 || listen(fd, 4) != 0) {- SGLog(@"tree server: could not listen on %u", kTreePort);close(fd);- return;+ return -1;}- static dispatch_source_t source;- source = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, (uintptr_t)fd, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));- dispatch_source_set_event_handler(source, ^{+ *source = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, (uintptr_t)fd, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));+ dispatch_source_set_event_handler(*source, ^{int client = accept(fd, NULL, NULL);if (client < 0) return;- struct timeval timeout = {2, 0};- setsockopt(client, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));- char request[1024];- recv(client, request, sizeof(request), 0);- __block NSString *body = nil;- dispatch_sync(dispatch_get_main_queue(), ^{ body = SGScreenTree(); });- NSData *data = [body dataUsingEncoding:NSUTF8StringEncoding];- NSString *head = [NSString stringWithFormat:@"HTTP/1.0 200 OK\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n", (unsigned long)data.length];- sendAll(client, [head dataUsingEncoding:NSUTF8StringEncoding]);- sendAll(client, data);- close(client);+ dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0), ^{ serve(client, tailnet); });+ });+ dispatch_source_set_cancel_handler(*source, ^{ close(fd); });+ dispatch_resume(*source);+ return fd;+}++#if SG_DRIVER+// The tailnet listener: the user asked to drive the app from their Mac over Tailscale. It binds the phone's+// exact Tailscale address (a utun interface in 100.64.0.0/10), never every interface and never Wi-Fi or+// cellular, so only the user's own tailnet reaches it, and every request on it needs the token, made once and+// kept in the app's defaults, which the log prints at launch for scripts/phone.py.+static NSString *driverToken(void) {+ static NSString *token;+ static dispatch_once_t once;+ dispatch_once(&once, ^{+ NSString *const key = @"spotifyglass.driver.token";+ token = [NSUserDefaults.standardUserDefaults stringForKey:key];+ if (token.length < 32) {+ uint8_t bytes[16];+ arc4random_buf(bytes, sizeof(bytes)); // cryptographically random on Darwin+ NSMutableString *hex = [NSMutableString string];+ for (size_t i = 0; i < sizeof(bytes); i++) [hex appendFormat:@"%02x", bytes[i]];+ token = hex;+ [NSUserDefaults.standardUserDefaults setObject:token forKey:key];+ }});- dispatch_resume(source);- SGLog(@"tree server on 127.0.0.1:%u; on the Mac: iproxy %u:%u, then GET http://127.0.0.1:%u/tree", kTreePort, kTreePort, kTreePort, kTreePort);+ return token;+}++static in_addr_t tailnetAddress(void) {+ struct ifaddrs *list = NULL;+ in_addr_t found = 0;+ if (getifaddrs(&list) != 0) return 0;+ for (struct ifaddrs *i = list; i && !found; i = i->ifa_next) {+ if (!i->ifa_addr || i->ifa_addr->sa_family != AF_INET || strncmp(i->ifa_name, "utun", 4) != 0) continue;+ in_addr_t address = ((struct sockaddr_in *)i->ifa_addr)->sin_addr.s_addr;+ if ((ntohl(address) & 0xFFC00000) == 0x64400000) found = address; // 100.64.0.0/10+ }+ freeifaddrs(list);+ return found;+}++// The VPN can come up, go away or change address after launch: looked at again every 10 s.+static void watchTailnet(void) {+ static dispatch_source_t timer, source;+ static in_addr_t bound;+ timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, dispatch_get_global_queue(QOS_CLASS_UTILITY, 0));+ dispatch_source_set_timer(timer, DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC, NSEC_PER_SEC);+ dispatch_source_set_event_handler(timer, ^{+ in_addr_t address = tailnetAddress();+ if (address == bound) return;+ if (source) {+ dispatch_source_cancel(source);+ source = nil;+ }+ bound = 0;+ char text[INET_ADDRSTRLEN] = "";+ inet_ntop(AF_INET, &address, text, sizeof(text));+ if (!address) {+ SGLog(@"driver: no Tailscale address, the tailnet listener is off");+ return;+ }+ if (listenOn(address, YES, &source) < 0) {+ SGLog(@"driver: could not listen on %s:%u", text, kTreePort);+ return;+ }+ bound = address;+ SGLog(@"driver: tailnet listener on %s:%u, token %@", text, kTreePort, driverToken());+ });+ dispatch_resume(timer);+}+#endif++static void startTreeServer(void) {+ static dispatch_source_t source;+ if (listenOn(htonl(INADDR_LOOPBACK), NO, &source) < 0) { // loopback, which only the cable's iproxy reaches+ SGLog(@"tree server: could not listen on %u", kTreePort);+ return;+ }+ SGLog(@"tree server on 127.0.0.1:%u; on the Mac: iproxy %u:%u, then GET http://127.0.0.1:%u/tree%@", kTreePort, kTreePort, kTreePort, kTreePort,+ SG_DRIVER ? @", or drive the app with scripts/phone.py" : @"");+#if SG_DRIVER+ watchTailnet();+#endif}%hook _TtC21NowPlaying_ScrollImpl23NPVScrollViewControllerdiff --git a/tweak/Sources/Diagnostics/Driver.m b/tweak/Sources/Diagnostics/Driver.mnew file mode 100644index 00000000..ad839e9a--- /dev/null+++ b/tweak/Sources/Diagnostics/Driver.m@@ -0,0 +1,861 @@+// The phone driver: commands on the tree server (Diagnostics.x) that work the app the way a finger+// would, so an agent on the Mac can test a FLEX build without anyone touching the phone.+// scripts/phone.py is its client; docs/tweaks.md (Diagnostics) lists the commands.+//+// Debug only: compiled with SG_DRIVER alone (tweak/Makefile; scripts/pipeline.sh sets it for builds that+// carry FLEX, never for make release or the release workflow's .deb), and reached only through the tree+// server, which starts only when FLEX is in the app and listens on 127.0.0.1, never on Wi-Fi.+//+// Threading: a command runs on the server's thread. What reads or touches UIKit goes to main through+// SGRunOnMain (dispatch_async and a semaphore with a timeout, never waited on from main), and the pauses+// between the phases of a touch are slept on the server's thread, so main runs freely between them.+//+// Touches are made the way harness/tabbar/touch.m makes them: a UITouch in the app's touches event,+// carrying the IOHIDEvent UIKit's gesture recognizers read, sent through -[UIApplication sendEvent:].+// Controls get their control events, recognizers recognize, and a button whose menu is its primary action+// opens it on the touch down, as with a finger. The IOHIDEvent functions are looked up at run time, so the+// dylib links nothing new.+//+// Views are addressed the way the tree (GET /tree) prints them: id= (accessibilityIdentifier), a11y=+// (accessibilityLabel), the quoted text of a label, or the class name with an index counting that+// class's views in the tree's order, windows included.+#if SG_DRIVER+#import "Core/SGCore.h"+#import "Diagnostics.h"+#import "Headers/SPTPlayer.h"+#import <dlfcn.h>+#import <mach/mach_time.h>+#import <objc/message.h>++static const NSTimeInterval kMainTimeout = 5;++// A command that cannot go on throws, and SGDriverRun answers with the reason.+static void fail(NSString *format, ...) NS_FORMAT_FUNCTION(1, 2) __attribute__((noreturn));+static void fail(NSString *format, ...) {+ va_list args;+ va_start(args, format);+ NSString *reason = [[NSString alloc] initWithFormat:format arguments:args];+ va_end(args);+ @throw [NSException exceptionWithName:@"SGDriver" reason:reason userInfo:nil];+}++// Runs `block` on main and hands back what it returned; an exception inside comes back out here.+static id onMain(id (^block)(void)) {+ __block id result = nil;+ __block NSException *raised = nil;+ BOOL answered = SGRunOnMain(kMainTimeout, ^{+ @try {+ result = block();+ } @catch (NSException *exception) {+ raised = exception;+ }+ });+ if (!answered) fail(@"the main thread did not answer in %.0f s", kMainTimeout);+ if (raised) @throw raised;+ return result;+}++static void pause_(NSTimeInterval seconds) {+ if (seconds > 0) usleep((useconds_t)(seconds * 1e6));+}++#pragma mark - touches++@interface UITouch (SGDriver)+- (void)setWindow:(UIWindow *)window;+- (void)setView:(UIView *)view;+- (void)setPhase:(UITouchPhase)phase;+- (void)setTimestamp:(NSTimeInterval)timestamp;+- (void)setTapCount:(NSUInteger)count;+- (void)_setLocationInWindow:(CGPoint)location resetPrevious:(BOOL)reset;+- (void)_setIsFirstTouchForView:(BOOL)first;+- (void)_setHidEvent:(void *)event;+- (void)setGestureView:(UIView *)view;+@end++@interface UIEvent (SGDriver)+- (void)_clearTouches;+- (void)_addTouch:(UITouch *)touch forDelayedDelivery:(BOOL)delayed;+- (void)_setHIDEvent:(void *)event;+@end++@interface UIApplication (SGDriver)+- (UIEvent *)_touchesEvent;+@end++typedef void *IOHIDEventRef;+static IOHIDEventRef (*sg_createDigitizer)(CFAllocatorRef, uint64_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, double, double, double,+ double, double, Boolean, Boolean, uint32_t);+static IOHIDEventRef (*sg_createFinger)(CFAllocatorRef, uint64_t, uint32_t, uint32_t, uint32_t, double, double, double, double, double, double,+ double, double, double, double, Boolean, Boolean, uint32_t);+static void (*sg_appendEvent)(IOHIDEventRef, IOHIDEventRef, uint32_t);+static void (*sg_setInteger)(IOHIDEventRef, uint32_t, long);++static void requireTouches(void) {+ static dispatch_once_t once;+ dispatch_once(&once, ^{+ sg_createDigitizer = dlsym(RTLD_DEFAULT, "IOHIDEventCreateDigitizerEvent");+ sg_createFinger = dlsym(RTLD_DEFAULT, "IOHIDEventCreateDigitizerFingerEventWithQuality");+ sg_appendEvent = dlsym(RTLD_DEFAULT, "IOHIDEventAppendEvent");+ sg_setInteger = dlsym(RTLD_DEFAULT, "IOHIDEventSetIntegerValue");+ });+ if (!sg_createDigitizer || !sg_createFinger || !sg_appendEvent || !sg_setInteger) fail(@"IOHIDEvent functions not found, touches cannot be made");+ if (![UIApplication.sharedApplication respondsToSelector:@selector(_touchesEvent)] || ![UITouch instancesRespondToSelector:@selector(_setHidEvent:)]) {+ fail(@"UIKit's private touch setters are missing on this iOS");+ }+}++static const uint32_t kDigitizer = 11, kHand = 3, kRange = 1 << 0, kTouch = 1 << 1, kPosition = 1 << 2;+static const uint32_t kDisplayIntegrated = (kDigitizer << 16) + 25;++static IOHIDEventRef hidFor(UITouch *touch) {+ uint64_t now = mach_absolute_time();+ IOHIDEventRef hand = sg_createDigitizer(kCFAllocatorDefault, now, kHand, 0, 0, kTouch, 0, 0, 0, 0, 0, 0, 0, 1, 0);+ sg_setInteger(hand, kDisplayIntegrated, 1);+ BOOL moved = touch.phase == UITouchPhaseMoved, down = touch.phase != UITouchPhaseEnded;+ CGPoint at = [touch locationInView:touch.window];+ IOHIDEventRef finger = sg_createFinger(kCFAllocatorDefault, now, 1, 2, moved ? kPosition : kRange | kTouch, at.x, at.y, 0, 0, 0, 5, 5, 1, 1, 1,+ down, down, 0);+ sg_setInteger(finger, kDisplayIntegrated, 1);+ sg_appendEvent(hand, finger, 0);+ CFRelease(finger);+ return hand;+}++static void deliver(UITouch *touch) {+ IOHIDEventRef hid = hidFor(touch);+ [touch _setHidEvent:hid];+ UIEvent *event = [UIApplication.sharedApplication _touchesEvent];+ [event _clearTouches];+ [event _setHIDEvent:hid];+ [event _addTouch:touch forDelayedDelivery:NO];+ CFRelease(hid);+ [UIApplication.sharedApplication sendEvent:event];+}++static NSArray<UIWindow *> *windows(void);++// The window a finger at `screen` lands in: the highest one that takes a touch there.+static UIWindow *windowAt(CGPoint screen) {+ // Highest level first, and of one level the later window, which is the one in front.+ NSArray<UIWindow *> *all = [windows().reverseObjectEnumerator.allObjects sortedArrayWithOptions:NSSortStable+ usingComparator:^NSComparisonResult(UIWindow *a, UIWindow *b) {+ return a.windowLevel > b.windowLevel ? NSOrderedAscending : a.windowLevel < b.windowLevel ? NSOrderedDescending : NSOrderedSame;+ }];+ for (UIWindow *window in all) {+ if ([window hitTest:[window convertPoint:screen fromWindow:nil] withEvent:nil]) return window;+ }+ return nil;+}++static NSArray *pair(CGPoint p) {+ return @[@(round(p.x * 10) / 10), @(round(p.y * 10) / 10)];+}++// A finger down at `from`, moved to `to` over `duration` s at 60 Hz, held still `hold` s, lifted at `to`.+// With no hold the last move is the lift, so a quick one flings.+static NSDictionary *finger(CGPoint from, CGPoint to, NSTimeInterval duration, NSTimeInterval hold) {+ requireTouches();+ __block UITouch *touch = nil;+ NSDictionary *down = onMain(^id {+ UIWindow *window = windowAt(from);+ if (!window) fail(@"no window takes a touch at {%.0f, %.0f}", from.x, from.y);+ CGPoint at = [window convertPoint:from fromWindow:nil];+ UIView *view = [window hitTest:at withEvent:nil];+ touch = [UITouch new];+ [touch setWindow:window];+ [touch _setLocationInWindow:at resetPrevious:YES];+ [touch setView:view];+ if ([touch respondsToSelector:@selector(setGestureView:)]) [touch setGestureView:view];+ [touch setPhase:UITouchPhaseBegan];+ [touch _setIsFirstTouchForView:YES];+ [touch setTapCount:1];+ [touch setTimestamp:NSProcessInfo.processInfo.systemUptime];+ deliver(touch);+ return @{@"window" : NSStringFromClass(window.class), @"hit" : NSStringFromClass(view.class) ?: @"nil"};+ });+ void (^move)(CGPoint, UITouchPhase) = ^(CGPoint point, UITouchPhase phase) {+ onMain(^id {+ [touch setTimestamp:NSProcessInfo.processInfo.systemUptime];+ [touch _setLocationInWindow:[touch.window convertPoint:point fromWindow:nil] resetPrevious:NO];+ [touch setPhase:phase];+ deliver(touch);+ return nil;+ });+ };+ @try {+ NSUInteger steps = duration > 0 ? MAX(2, (NSUInteger)(duration * 60)) : 0;+ for (NSUInteger i = 1; i <= steps; i++) {+ pause_(1.0 / 60);+ CGFloat t = (CGFloat)i / steps;+ CGPoint at = CGPointMake(from.x + (to.x - from.x) * t, from.y + (to.y - from.y) * t);+ if (i == steps && hold <= 0) {+ move(at, UITouchPhaseEnded);+ return down;+ }+ move(at, UITouchPhaseMoved);+ }+ pause_(hold);+ move(to, UITouchPhaseEnded);+ } @catch (NSException *exception) {+ // A finger left down would hold every later touch; lift it whatever happened.+ dispatch_async(dispatch_get_main_queue(), ^{+ [touch setPhase:UITouchPhaseCancelled];+ deliver(touch);+ });+ @throw exception;+ }+ return down;+}++#pragma mark - finding views++static NSArray<UIWindow *> *windows(void) {+ NSMutableArray<UIWindow *> *all = [NSMutableArray array];+ for (UIScene *scene in UIApplication.sharedApplication.connectedScenes) {+ if (![scene isKindOfClass:UIWindowScene.class]) continue;+ for (UIWindow *window in ((UIWindowScene *)scene).windows) {+ if (window.hidden || [NSStringFromClass(window.class) containsString:@"FLEX"]) continue;+ [all addObject:window];+ }+ }+ return all;+}++static UIWindow *keyWindow(void) {+ for (UIWindow *window in windows()) {+ if (window.isKeyWindow) return window;+ }+ return windows().firstObject;+}++static NSString *textOf(UIView *view) {+ if ([view isKindOfClass:UIButton.class]) return ((UIButton *)view).currentTitle;+ if (![view respondsToSelector:@selector(text)] || [view methodSignatureForSelector:@selector(text)].methodReturnType[0] != '@') return nil;+ id text = ((id (*)(id, SEL))objc_msgSend)(view, @selector(text));+ return [text isKindOfClass:NSString.class] ? text : nil;+}++static BOOL isShown(UIView *view) {+ if (!view.window) return NO;+ for (UIView *v = view; v; v = v.superview) {+ if (v.hidden || v.alpha < 0.01) return NO;+ }+ CGRect frame = [view convertRect:view.bounds toView:nil];+ return frame.size.width > 0 && frame.size.height > 0 && CGRectIntersectsRect(frame, view.window.bounds);+}++static CGRect screenFrame(UIView *view) {+ return [view.window convertRect:[view convertRect:view.bounds toView:nil] toWindow:nil];+}++static CGPoint screenCenter(UIView *view) {+ CGRect frame = screenFrame(view);+ return CGPointMake(CGRectGetMidX(frame), CGRectGetMidY(frame));+}++static BOOL same(NSString *a, NSString *b, BOOL contains) {+ if (!a || !b) return NO;+ a = [a stringByTrimmingCharactersInSet:NSCharacterSet.whitespaceAndNewlineCharacterSet];+ if (contains) return [a rangeOfString:b options:NSCaseInsensitiveSearch].location != NSNotFound;+ return [a caseInsensitiveCompare:b] == NSOrderedSame;+}++static BOOL hasSelector(NSDictionary *p) {+ return p[@"id"] || p[@"label"] || p[@"text"] || p[@"class"];+}++static BOOL matches(UIView *view, NSDictionary *p) {+ BOOL contains = p[@"contains"] != nil;+ if (p[@"class"] && ![NSStringFromClass(view.class) isEqualToString:p[@"class"]]) return NO;+ if (p[@"id"] && ![view.accessibilityIdentifier isEqualToString:p[@"id"]]) return NO;+ if (p[@"label"] && !same(view.accessibilityLabel, p[@"label"], contains)) return NO;+ if (p[@"text"] && !same(textOf(view), p[@"text"], contains)) return NO;+ return YES;+}++// Every view the selector matches, in the tree's order.+static NSArray<UIView *> *findAll(NSDictionary *p, UIView *root) {+ NSMutableArray<UIView *> *found = [NSMutableArray array];+ for (UIView *top in root ? @[root] : windows()) {+ SGForEachView(top, ^(UIView *view) {+ if (matches(view, p)) [found addObject:view];+ });+ }+ return found;+}++// The view a selector means: with an index, that one of the matches in the tree's order; without, the+// first one shown.+static UIView *pick(NSDictionary *p, UIView *root) {+ NSArray<UIView *> *found = findAll(p, root);+ if (p[@"index"]) {+ NSInteger index = [p[@"index"] integerValue];+ return index >= 0 && index < (NSInteger)found.count ? found[(NSUInteger)index] : nil;+ }+ for (UIView *view in found) {+ if (isShown(view)) return view;+ }+ return nil;+}++static NSString *selectorText(NSDictionary *p) {+ NSMutableArray<NSString *> *parts = [NSMutableArray array];+ for (NSString *key in @[@"id", @"label", @"text", @"class", @"index"]) {+ if (p[key]) [parts addObject:[NSString stringWithFormat:@"%@=%@", key, p[key]]];+ }+ return [parts componentsJoinedByString:@" "];+}++static UIView *require(NSDictionary *p, UIView *root) {+ UIView *view = pick(p, root);+ if (view) return view;+ NSUInteger count = findAll(p, root).count;+ if (count) fail(@"%@ matches %lu views, none of them shown (index= takes a hidden one)", selectorText(p), (unsigned long)count);+ fail(@"no view matches %@", selectorText(p));+}++static NSDictionary *describe(UIView *view) {+ NSMutableDictionary *d = [NSMutableDictionary dictionary];+ d[@"class"] = NSStringFromClass(view.class);+ CGRect frame = screenFrame(view);+ d[@"frame"] = @[@(round(frame.origin.x)), @(round(frame.origin.y)), @(round(frame.size.width)), @(round(frame.size.height))];+ d[@"shown"] = @(isShown(view));+ if (view.accessibilityIdentifier.length) d[@"id"] = view.accessibilityIdentifier;+ if (view.accessibilityLabel.length) d[@"label"] = view.accessibilityLabel;+ NSString *text = textOf(view);+ if (text.length) d[@"text"] = text;+ if ([view isKindOfClass:UIControl.class]) {+ d[@"enabled"] = @(((UIControl *)view).enabled);+ d[@"selected"] = @(((UIControl *)view).selected);+ }+ if ([view isKindOfClass:UISwitch.class]) d[@"on"] = @(((UISwitch *)view).on);+ if ([view isKindOfClass:UIScrollView.class]) {+ UIScrollView *scroll = (UIScrollView *)view;+ d[@"contentOffset"] = pair(scroll.contentOffset);+ d[@"contentSize"] = pair(CGPointMake(scroll.contentSize.width, scroll.contentSize.height));+ }+ return d;+}++static CGPoint parsePoint(NSString *text, NSString *name) {+ NSArray<NSString *> *parts = [text componentsSeparatedByString:@","];+ if (parts.count != 2) fail(@"%@ takes x,y, not %@", name, text);+ return CGPointMake(parts[0].doubleValue, parts[1].doubleValue);+}++static double number(NSDictionary *p, NSString *key, double fallback) {+ return p[key] ? [p[key] doubleValue] : fallback;+}++// Where a command's finger goes: at=x,y, or the middle of the view the selector picks.+static NSDictionary *target(NSDictionary *p) {+ return onMain(^id {+ if (p[@"at"]) return @{@"point" : [NSValue valueWithCGPoint:parsePoint(p[@"at"], @"at")]};+ if (!hasSelector(p)) fail(@"say where: at=x,y, or a view by id, label, text or class (with index)");+ UIView *view = require(p, nil);+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(view)], @"target" : describe(view)};+ });+}++static NSDictionary *touchAt(NSDictionary *where, NSTimeInterval hold) {+ CGPoint point = [where[@"point"] CGPointValue];+ NSMutableDictionary *result = [finger(point, point, 0, hold) mutableCopy];+ result[@"point"] = pair(point);+ if (where[@"target"]) result[@"target"] = where[@"target"];+ return result;+}++#pragma mark - the app's own parts++static NSString *const kPlayerClass = @"_TtC21NowPlaying_ScrollImpl23NPVScrollViewController";+static NSString *const kBarClass = @"_TtC18NowPlaying_BarImpl27NowPlayingBarViewController";++static BOOL insideController(UIView *view, Class cls) {+ for (UIResponder *r = view; r; r = r.nextResponder) {+ if (cls && [r isKindOfClass:cls]) return YES;+ }+ return NO;+}++// A shown view that is some controller's own view, of class `name`.+static UIView *controllerView(NSString *name) {+ Class cls = NSClassFromString(name);+ if (!cls) return nil;+ for (UIWindow *window in windows()) {+ __block UIView *found = nil;+ SGForEachView(window, ^(UIView *view) {+ if (!found && [view.nextResponder isKindOfClass:cls] && isShown(view)) found = view;+ });+ if (found) return found;+ }+ return nil;+}++static BOOL playerShown(void) {+ return controllerView(kPlayerClass) != nil;+}++// The system's menu, for the menus of UIButton and UIContextMenuInteraction.+static BOOL isMenuPart(UIView *view) {+ NSString *name = NSStringFromClass(view.class);+ return [name hasPrefix:@"_UIContextMenu"] || [name hasPrefix:@"_UIMenu"];+}++static UIView *shownMenu(void) {+ for (UIWindow *window in windows()) {+ __block UIView *found = nil;+ SGForEachView(window, ^(UIView *view) {+ if (!found && [NSStringFromClass(view.class) hasPrefix:@"_UIContextMenuView"] && isShown(view)) found = view;+ });+ if (found) return found;+ }+ return nil;+}++static BOOL insideMenu(UIView *view) {+ for (UIView *v = view; v; v = v.superview) {+ if (isMenuPart(v)) return YES;+ }+ return NO;+}++// The view showing `title`, by its text or its accessibility label, shown and passing `filter`.+static UIView *titled(NSString *title, BOOL (^filter)(UIView *)) {+ for (NSString *key in @[@"text", @"label"]) {+ for (UIView *view in findAll(@{key : title}, nil)) {+ if (isShown(view) && (!filter || filter(view))) return view;+ }+ }+ return nil;+}++static UIViewController *topController(NSMutableArray<NSString *> *path, NSMutableArray<NSString *> *presented) {+ UIViewController *top = keyWindow().rootViewController;+ while (top) {+ [path addObject:NSStringFromClass(top.class)];+ UIViewController *next = nil;+ if (top.presentedViewController && !top.presentedViewController.isBeingDismissed) {+ next = top.presentedViewController;+ [presented addObject:NSStringFromClass(next.class)];+ } else if ([top isKindOfClass:UINavigationController.class]) {+ next = ((UINavigationController *)top).topViewController;+ } else if ([top isKindOfClass:UITabBarController.class]) {+ next = ((UITabBarController *)top).selectedViewController;+ } else {+ // A container of Spotify's own: the child that covers most of it, not a bar inside it.+ CGRect own = CGRectIntersection(screenFrame(top.viewIfLoaded), keyWindow().bounds);+ CGFloat best = own.size.width * own.size.height / 2;+ for (UIViewController *child in top.childViewControllers) {+ if (!child.isViewLoaded || !isShown(child.view)) continue;+ CGRect frame = CGRectIntersection(screenFrame(child.view), keyWindow().bounds);+ CGFloat area = frame.size.width * frame.size.height;+ if (area > best) {+ best = area;+ next = child;+ }+ }+ }+ if (!next) break;+ top = next;+ }+ return top;+}++static NSDictionary *nowPlaying(void) {+ id player = SGDiagnosticsPlayer();+ SPTPlayerState *state = [player respondsToSelector:@selector(state)] ? [(id<SPTPlayer>)player state] : nil;+ if (!state) return nil;+ NSMutableDictionary *d = [NSMutableDictionary dictionary];+ SPTPlayerTrack *track = state.track;+ if (track.trackTitle) d[@"title"] = track.trackTitle;+ if (track.artistName) d[@"artist"] = track.artistName;+ if (track.URI) d[@"uri"] = [track.URI description];+ d[@"position"] = @(round((state.isPaused ? state.positionAsOfTimestamp : state.position) * 10) / 10);+ d[@"duration"] = @(round(state.duration * 10) / 10);+ d[@"playing"] = @((BOOL)(state.isPlaying && !state.isPaused));+ d[@"paused"] = @(state.isPaused);+ return d;+}++static id<SPTPlayer> requirePlayer(void) {+ id player = SGDiagnosticsPlayer();+ if (!player) fail(@"no player yet: play something first, or open the app's player once");+ return player;+}++static UIView *firstResponder(void) {+ for (UIWindow *window in windows()) {+ id responder = nil;+ @try {+ responder = [window valueForKey:@"firstResponder"];+ } @catch (NSException *exception) {+ }+ if ([responder isKindOfClass:UIView.class]) return responder;+ }+ return nil;+}++// The tab bar's items, leading first: the redesign's UITabBar, or Spotify's own bar's row.+static NSArray<UIView *> *tabItems(NSMutableArray<NSString *> *titles) {+ for (UIWindow *window in windows()) {+ __block UIView *bar = nil;+ SGForEachView(window, ^(UIView *view) {+ if (bar || !isShown(view)) return;+ if ([view isKindOfClass:UITabBar.class] || [NSStringFromClass(view.class) isEqualToString:@"_TtC23NavigationUI_TabBarImpl10TabBarView"]) bar = view;+ });+ if (!bar) continue;+ NSMutableArray<UIView *> *items = [NSMutableArray array];+ if ([bar isKindOfClass:UITabBar.class]) {+ for (UITabBarItem *item in ((UITabBar *)bar).items) {+ UIView *view = item.title ? titled(item.title, ^BOOL(UIView *v) { return [v isDescendantOfView:bar]; }) : nil;+ if (!view) continue;+ [items addObject:view];+ [titles addObject:item.title];+ }+ } else {+ for (UIView *item in SGRowIn(bar).arrangedSubviews) {+ if (!isShown(item)) continue;+ __block NSString *title = item.accessibilityLabel;+ SGForEachView(item, ^(UIView *v) {+ if (!title.length && textOf(v).length) title = textOf(v);+ });+ [items addObject:item];+ [titles addObject:title ?: @""];+ }+ }+ return items;+ }+ return @[];+}++// The largest scroll view shown in the key window, the page's own list.+static UIScrollView *mainScrollView(void) {+ __block UIScrollView *best = nil;+ __block CGFloat bestArea = 0;+ SGForEachView(keyWindow(), ^(UIView *view) {+ if (![view isKindOfClass:UIScrollView.class] || !isShown(view)) return;+ CGRect frame = screenFrame(view);+ if (frame.size.width * frame.size.height > bestArea) {+ bestArea = frame.size.width * frame.size.height;+ best = (UIScrollView *)view;+ }+ });+ return best;+}++static CGPoint clampedOffset(UIScrollView *scroll, CGPoint offset) {+ UIEdgeInsets inset = scroll.adjustedContentInset;+ CGFloat maxX = MAX(-inset.left, scroll.contentSize.width + inset.right - scroll.bounds.size.width);+ CGFloat maxY = MAX(-inset.top, scroll.contentSize.height + inset.bottom - scroll.bounds.size.height);+ return CGPointMake(MIN(MAX(offset.x, -inset.left), maxX), MIN(MAX(offset.y, -inset.top), maxY));+}++#pragma mark - commands++typedef NSDictionary *(^SGCommand)(NSDictionary *p);++// The log number when the last command that does something began: a log wait looks from there, so+// `player.more` and then `wait log=...` sees a line that came before the wait did.+static uint64_t sg_actionSeq;++static NSDictionary<NSString *, SGCommand> *commands(void) {+ static NSDictionary<NSString *, SGCommand> *all;+ static dispatch_once_t once;+ dispatch_once(&once, ^{+ all = @{+ @"tap" : ^NSDictionary *(NSDictionary *p) {+ return touchAt(target(p), 0.08);+ },+ @"longpress" : ^NSDictionary *(NSDictionary *p) {+ return touchAt(target(p), number(p, @"duration", 1.0));+ },+ @"swipe" : ^NSDictionary *(NSDictionary *p) {+ NSDictionary *start = p[@"from"] ? @{@"point" : [NSValue valueWithCGPoint:parsePoint(p[@"from"], @"from")]} : target(p);+ CGPoint from = [start[@"point"] CGPointValue];+ CGPoint to;+ if (p[@"to"]) {+ to = parsePoint(p[@"to"], @"to");+ } else if (p[@"by"]) {+ CGPoint by = parsePoint(p[@"by"], @"by");+ to = CGPointMake(from.x + by.x, from.y + by.y);+ } else {+ fail(@"swipe takes to=x,y or by=dx,dy");+ }+ BOOL drag = [p[@"drag"] boolValue];+ NSMutableDictionary *result = [finger(from, to, number(p, @"duration", drag ? 0.6 : 0.2), number(p, @"hold", drag ? 0.3 : 0)) mutableCopy];+ result[@"from"] = pair(from);+ result[@"to"] = pair(to);+ if (start[@"target"]) result[@"target"] = start[@"target"];+ return result;+ },+ @"scroll" : ^NSDictionary *(NSDictionary *p) {+ if (!p[@"by"]) fail(@"scroll takes by=dx,dy");+ CGPoint by = parsePoint(p[@"by"], @"by");+ return onMain(^id {+ UIView *view = nil;+ if (hasSelector(p)) {+ view = require(p, nil);+ } else if (p[@"at"]) {+ CGPoint at = parsePoint(p[@"at"], @"at");+ UIWindow *window = windowAt(at);+ view = [window hitTest:[window convertPoint:at fromWindow:nil] withEvent:nil];+ } else {+ view = mainScrollView();+ }+ while (view && ![view isKindOfClass:UIScrollView.class]) view = view.superview;+ if (!view) fail(@"no scroll view there");+ UIScrollView *scroll = (UIScrollView *)view;+ CGPoint before = scroll.contentOffset;+ CGPoint after = clampedOffset(scroll, CGPointMake(before.x + by.x, before.y + by.y));+ [scroll setContentOffset:after animated:[p[@"animated"] boolValue]];+ return @{@"scrollView" : describe(scroll), @"before" : pair(before), @"after" : pair(after)};+ });+ },+ @"type" : ^NSDictionary *(NSDictionary *p) {+ NSString *text = p[@"text"] ?: p[@"arg"];+ if (!text) fail(@"type takes the text");+ return onMain(^id {+ UIView *responder = firstResponder();+ if (![responder conformsToProtocol:@protocol(UIKeyInput)]) fail(@"nothing takes typing: tap a text field first");+ [(id<UIKeyInput>)responder insertText:text];+ return @{@"into" : describe(responder)};+ });+ },+ @"find" : ^NSDictionary *(NSDictionary *p) {+ if (!hasSelector(p)) fail(@"find takes id, label, text or class");+ return onMain(^id {+ NSMutableArray *views = [NSMutableArray array];+ NSArray<UIView *> *found = findAll(p, nil);+ for (NSUInteger i = 0; i < found.count && i < 100; i++) {+ NSMutableDictionary *d = [describe(found[i]) mutableCopy];+ d[@"index"] = @(i);+ [views addObject:d];+ }+ return @{@"count" : @(found.count), @"views" : views};+ });+ },+ @"state" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ NSMutableArray<NSString *> *path = [NSMutableArray array], *presented = [NSMutableArray array];+ UIViewController *top = topController(path, presented);+ NSMutableDictionary *d = [NSMutableDictionary dictionary];+ d[@"top"] = top ? NSStringFromClass(top.class) : [NSNull null];+ if (top.title.length) d[@"topTitle"] = top.title;+ d[@"controllers"] = path;+ d[@"presented"] = presented;+ d[@"menu"] = @((BOOL)(shownMenu() != nil));+ d[@"player"] = @(playerShown());+ d[@"nowPlaying"] = nowPlaying() ?: [NSNull null];+ UIView *responder = firstResponder();+ if (responder) d[@"firstResponder"] = NSStringFromClass(responder.class);+ d[@"keyWindow"] = NSStringFromClass(keyWindow().class) ?: [NSNull null];+ d[@"logSeq"] = @(SGLogLastSeq());+ return d;+ });+ },+ @"screenshot" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ UIWindow *key = keyWindow();+ if (!key) fail(@"no window");+ UIGraphicsImageRendererFormat *format = [UIGraphicsImageRendererFormat preferredFormat];+ if (p[@"scale"]) format.scale = [p[@"scale"] doubleValue];+ // Every window, lowest level first, so a menu in a window of its own is in the picture.+ NSArray<UIWindow *> *all = [windows() sortedArrayWithOptions:NSSortStable usingComparator:^NSComparisonResult(UIWindow *a, UIWindow *b) {+ return a.windowLevel < b.windowLevel ? NSOrderedAscending : a.windowLevel > b.windowLevel ? NSOrderedDescending : NSOrderedSame;+ }];+ UIGraphicsImageRenderer *renderer = [[UIGraphicsImageRenderer alloc] initWithBounds:key.bounds format:format];+ NSData *png = [renderer PNGDataWithActions:^(UIGraphicsImageRendererContext *context) {+ for (UIWindow *window in all) [window drawViewHierarchyInRect:window.frame afterScreenUpdates:NO];+ }];+ return @{@"png" : [png base64EncodedStringWithOptions:0], @"size" : pair(CGPointMake(key.bounds.size.width * format.scale, key.bounds.size.height * format.scale))};+ });+ },+ @"log" : ^NSDictionary *(NSDictionary *p) {+ uint64_t last = SGLogLastSeq();+ uint64_t since = p[@"since"] ? (uint64_t)[p[@"since"] longLongValue] : (last > 100 ? last - 100 : 0);+ return @{@"lines" : SGLogRecent(since), @"last" : @(last)};+ },+ @"wait" : ^NSDictionary *(NSDictionary *p) {+ NSTimeInterval timeout = number(p, @"timeout", 5);+ NSDate *start = NSDate.date;+ BOOL gone = [p[@"gone"] boolValue];+ if (!p[@"log"] && !hasSelector(p) && !p[@"menu"]) fail(@"wait takes log=text, menu=1, or a view by id, label, text or class");+ uint64_t since = p[@"since"] ? (uint64_t)[p[@"since"] longLongValue] : sg_actionSeq;+ while (YES) {+ if (p[@"log"]) {+ for (NSDictionary *line in SGLogRecent(since)) {+ if ([line[@"text"] containsString:p[@"log"]]) return @{@"line" : line, @"after" : @(-start.timeIntervalSinceNow)};+ }+ } else {+ NSDictionary *seen = onMain(^id {+ UIView *view = p[@"menu"] ? shownMenu() : pick(p, nil);+ if (view && !isShown(view)) view = nil;+ if (gone) return view ? nil : @{};+ return view ? describe(view) : nil;+ });+ if (seen) return @{@"view" : seen.count ? seen : [NSNull null], @"after" : @(-start.timeIntervalSinceNow)};+ }+ if (-start.timeIntervalSinceNow > timeout) {+ if (p[@"log"]) fail(@"no log line with \"%@\" within %.1f s", p[@"log"], timeout);+ NSString *what = p[@"menu"] ? @"the menu" : selectorText(p);+ fail(@"%@ %@ after %.1f s", what, gone ? @"still shown" : @"not shown", timeout);+ }+ pause_(0.1);+ }+ },+ @"player.open" : ^NSDictionary *(NSDictionary *p) {+ NSDictionary *where = onMain(^id {+ if (playerShown()) return @{};+ UIView *bar = controllerView(kBarClass);+ if (!bar) fail(@"no now playing bar shown");+ // Left of the middle: the artwork and the title, away from the bar's buttons.+ CGRect frame = screenFrame(bar);+ return @{@"point" : [NSValue valueWithCGPoint:CGPointMake(CGRectGetMinX(frame) + frame.size.width * 0.3, CGRectGetMidY(frame))],+ @"target" : describe(bar)};+ });+ if (!where.count) return @{@"note" : @"the player was already open"};+ return touchAt(where, 0.08);+ },+ @"player.close" : ^NSDictionary *(NSDictionary *p) {+ NSDictionary *where = onMain(^id {+ for (NSString *identifier in @[@"now-playing-minimize-button", @"mobile-nowplaying-close-button"]) {+ UIView *button = pick(@{@"id" : identifier}, nil);+ if (button) return @{@"point" : [NSValue valueWithCGPoint:screenCenter(button)], @"target" : describe(button)};+ }+ if (!playerShown()) return @{};+ fail(@"the player is open but its down arrow is not found");+ });+ if (!where.count) return @{@"note" : @"the player was not open"};+ return touchAt(where, 0.08);+ },+ @"player.more" : ^NSDictionary *(NSDictionary *p) {+ NSDictionary *where = onMain(^id {+ Class player = NSClassFromString(kPlayerClass);+ UIView *more = nil;+ for (UIView *view in findAll(@{@"id" : @"Context menu"}, nil)) {+ if (isShown(view) && insideController(view, player)) more = view;+ }+ if (!more) fail(@"%@", playerShown() ? @"the player's ⋯ (id=Context menu) is not shown" : @"the player is not open (player.open)");+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(more)], @"target" : describe(more)};+ });+ return touchAt(where, 0.08);+ },+ @"menu.pick" : ^NSDictionary *(NSDictionary *p) {+ NSString *title = p[@"title"] ?: p[@"arg"];+ if (!title) fail(@"menu.pick takes the row's title");+ NSDictionary *where = onMain(^id {+ if (!shownMenu()) fail(@"no menu is up");+ UIView *row = titled(title, ^BOOL(UIView *v) { return insideMenu(v); });+ if (!row) fail(@"the menu has no row \"%@\"", title);+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(row)], @"target" : describe(row)};+ });+ return touchAt(where, 0.08);+ },+ @"seek" : ^NSDictionary *(NSDictionary *p) {+ double seconds = [(p[@"seconds"] ?: p[@"arg"]) doubleValue];+ return onMain(^id {+ [requirePlayer() seekTo:seconds];+ return @{@"seconds" : @(seconds)};+ });+ },+ @"play" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ [requirePlayer() resume:nil];+ return @{};+ });+ },+ @"pause" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ [requirePlayer() pause:nil];+ return @{};+ });+ },+ @"next" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ [requirePlayer() skipToNextTrackWithOptions:nil];+ return @{};+ });+ },+ @"tab" : ^NSDictionary *(NSDictionary *p) {+ NSString *which = p[@"title"] ?: p[@"index"] ?: p[@"arg"];+ if (!which) fail(@"tab takes an index (0 is the leading tab) or a title");+ NSDictionary *where = onMain(^id {+ NSMutableArray<NSString *> *titles = [NSMutableArray array];+ NSArray<UIView *> *items = tabItems(titles);+ if (!items.count) fail(@"no tab bar shown");+ NSUInteger index = [titles indexOfObjectPassingTest:^BOOL(NSString *t, NSUInteger i, BOOL *stop) { return same(t, which, NO); }];+ if (index == NSNotFound && !p[@"title"] && [which rangeOfCharacterFromSet:NSCharacterSet.decimalDigitCharacterSet.invertedSet].location == NSNotFound) {+ index = (NSUInteger)which.integerValue;+ }+ if (index == NSNotFound || index >= items.count) fail(@"no tab %@; the tabs are %@", which, [titles componentsJoinedByString:@", "]);+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(items[index])], @"target" : describe(items[index]), @"tabs" : titles};+ });+ NSMutableDictionary *result = [touchAt(where, 0.08) mutableCopy];+ result[@"tabs"] = where[@"tabs"];+ return result;+ },+ @"settings.open" : ^NSDictionary *(NSDictionary *p) {+ return onMain(^id {+ UIViewController *top = topController([NSMutableArray array], [NSMutableArray array]);+ SGOpenModSettings(top.viewIfLoaded ?: keyWindow());+ return @{};+ });+ },+ @"settings.page" : ^NSDictionary *(NSDictionary *p) {+ NSString *title = p[@"title"] ?: p[@"arg"];+ if (!title) fail(@"settings.page takes the row's title");+ NSDictionary *where = onMain(^id {+ UIView *row = titled(title, nil);+ UIScrollView *list = mainScrollView();+ // Down the page a screen at a time until the row is laid out.+ if (!row && list) {+ CGPoint offset = clampedOffset(list, CGPointMake(list.contentOffset.x, -CGFLOAT_MAX));+ for (NSUInteger step = 0; step < 50 && !row; step++) {+ [list setContentOffset:offset animated:NO];+ [list layoutIfNeeded];+ row = titled(title, nil);+ CGPoint next = clampedOffset(list, CGPointMake(offset.x, offset.y + list.bounds.size.height * 0.6));+ if (CGPointEqualToPoint(next, offset)) break;+ offset = next;+ }+ }+ if (!row) fail(@"no row \"%@\" on this page", title);+ if (list && [row isDescendantOfView:list]) {+ CGRect frame = [row convertRect:row.bounds toView:list];+ CGPoint offset = clampedOffset(list, CGPointMake(list.contentOffset.x, CGRectGetMidY(frame) - list.bounds.size.height / 2));+ [list setContentOffset:offset animated:NO];+ [list layoutIfNeeded];+ }+ return @{@"point" : [NSValue valueWithCGPoint:screenCenter(row)], @"target" : describe(row)};+ });+ return touchAt(where, 0.08);+ },+ };+ });+ return all;+}++static NSSet<NSString *> *readOnly(void) {+ return [NSSet setWithArray:@[@"find", @"state", @"screenshot", @"log", @"wait"]];+}++BOOL SGDriverHandles(NSString *command) {+ return commands()[command] != nil;+}++NSData *SGDriverRun(NSString *command, NSDictionary<NSString *, NSString *> *params) {+ NSMutableDictionary *answer = [NSMutableDictionary dictionaryWithObject:command forKey:@"cmd"];+ if (![readOnly() containsObject:command]) sg_actionSeq = SGLogLastSeq();+ @try {+ [answer addEntriesFromDictionary:commands()[command](params) ?: @{}];+ answer[@"ok"] = @YES;+ } @catch (NSException *exception) {+ answer[@"ok"] = @NO;+ answer[@"error"] = exception.reason ?: exception.name;+ }+ NSData *json = [NSJSONSerialization isValidJSONObject:answer] ? [NSJSONSerialization dataWithJSONObject:answer options:0 error:nil] : nil;+ if (!json) json = [NSJSONSerialization dataWithJSONObject:@{@"cmd" : command, @"ok" : @NO, @"error" : @"the answer was not JSON"} options:0 error:nil];+ return json;+}+#endifdiff --git a/tweak/Sources/Diagnostics/PlayerState.x b/tweak/Sources/Diagnostics/PlayerState.xindex 0b068576..7c2b497f 100644--- a/tweak/Sources/Diagnostics/PlayerState.x+++ b/tweak/Sources/Diagnostics/PlayerState.x@@ -87,9 +87,16 @@ static void report(SPTPlayerState *state) {if (track) SGLogLong(@"player track metadata", track.metadata.description);}+static __weak id sg_player;++id SGDiagnosticsPlayer(void) {+ return sg_player;+}+%hook SPTEsperantoPlayer- (id)state {id state = %orig;+ sg_player = self;if (SGIsDebugBuild() && [state isKindOfClass:NSClassFromString(@"SPTPlayerState")]) report(state);return state;}