Liquid Glass and Apple Music touches for the Spotify iOS app. A GPL-3.0 fork of spoti.pw 0.21.1.
C
feat: Karaoke turns a song's vocals down from the lyrics or its own page
My-Name-Is-Jefff2b936d52986
diff --git a/docs/tweaks.md b/docs/tweaks.mdindex ab4a5d01..e94c6228 100644--- a/docs/tweaks.md+++ b/docs/tweaks.md@@ -153,6 +153,15 @@ Shared:master switch on.Tested on the Mac against harness/audio-effects/ and harness/autoeq/, the hook in the simulatoragainst its sim/, the page against harness/audio-effects-page/+ Sing/ Sing, a song's vocals turned down while it plays (Sing.h lists its files): the voice model (Mel-Band+ RoFormer, MIT, from Hugging Face) downloaded and checked file by file, a stop keeping what came in for the+ next download to carry on from, the STFT around it on Accelerate,+ and an engine that stands in Speed and pitch's chain between Spotify's mixer and its output+ (SGPlayerSetStage), pulls the mixer a few seconds ahead of what plays, separates two-second windows+ there on a worker thread and mixes the vocals down on the render thread. Spotify's clock has the lead+ taken off (SPTPlayerState's positionAsOfTimestamp), and a seek, a skip or a stop drops it. The mic is+ on the redesign's lyrics (Redesigned/Lyrics/SGRSingButton.m). Tested on the Mac against harness/sing/,+ the download against its download/Haptics/ Vibrations (Haptics.h lists its files): a tap of UIKit's feedback generators for the player's and the nowplaying bar's controls, the scrubber's tenths and ends, cover swipes, gestures and the lyrics page's tap toseek, at the strength set for them (ControlHaptics.x, SGFeedback.m); and Music Haptics, Core Haptics@@ -312,10 +321,15 @@ is set. Last comes the Font, under either look and on any iOS, below 26 too. Red(Settings/SGGlowSwitch), its ⓘ says what it changes, and flipping it offers to restart Spotify.The pages show only what the stored look has: a page opened after flipping the switch already showswhat the restart will bring. Then a card of parts. Navbar: the tab editor of the stored look, each with-its own list of tabs. Player: Gestures, Lyrics (the ordered list of lyrics sources, lyrics for every track,-naming the source in the redesign, the lock screen, and glass lyrics in the native look; in the redesign also-which of the lyrics, their pronunciation and their translation is set largest, and the translation's language), Blocked artists (with the count on the row) and Lock screen widget (on iOS 26 its full-screen artwork: Off,-Moving artwork, Lyrics or Every song, and the lyrics' style, Still or Animated), which work with either look;+its own list of tabs. Lyrics, on the main page of its own: Sing first, then the ordered list of lyrics sources,+lyrics for every track, naming the source in the redesign, the lock screen, and glass lyrics in the native look; in+the redesign also which of the lyrics, their pronunciation and their translation is set largest, and the+translation's language. Sing, under either look, on the main page and as Lyrics' first row, the row reading out On,+Off or how far the voice model's download has come, kept up to date while the page shows: Sing's switch, which turns+the mic on and off at once, its Status (a tap says more), the Vocals slider (gone, as sung, the vocals alone), the voice model's download, Runs on (GPU, GPU+and Neural Engine, Neural Engine) and Ignore heat warnings, all applying straight away. Player: Gestures, AirPods gestures, Blocked+artists (with the count on the row) and Lock screen widget (Moving artwork, Lyrics or Every song, and the+lyrics' style, Still or Animated), which work with either look;in the native look also Now playing bar (its device button and its flags), Queue & devices, andSpotify's own player screen (artwork background, glass header buttons, Disable Canvas and the sheet,header, slider and sticky header flags, the cards under the player and the lyrics preview and playerdiff --git a/harness/lyrics/build.sh b/harness/lyrics/build.shindex 008d25b4..1fc3e28e 100755--- a/harness/lyrics/build.sh+++ b/harness/lyrics/build.sh@@ -23,7 +23,8 @@ xcrun -sdk iphonesimulator clang -target arm64-apple-ios17.0-simulator -fobjc-ar"$SRC"/Shared/LyricsSources/SGTTML.m "$SRC"/Redesigned/Kit/SGRTokens.m \$( [ -f "$SRC"/Shared/LyricsMeanings/Meanings.m ] && echo "$SRC"/Shared/LyricsMeanings/Meanings.m "$SRC"/Redesigned/Lyrics/MeaningSheet.m ) \"$SRC"/Core/SGLog.m "$SRC"/Core/SGPrefs.m "$SRC"/Core/SGGlass.m \- -framework UIKit -framework QuartzCore -framework CoreGraphics -framework CoreText -framework Foundation \+ $( [ -f "$SRC"/Redesigned/Lyrics/SGRSingButton.m ] && echo "$SRC"/Redesigned/Lyrics/SGRSingButton.m "$SRC"/Redesigned/Kit/SGRGlass.m "$HERE"/../sing/button/sing-stubs.m ) \+ -framework UIKit -framework QuartzCore -framework CoreGraphics -framework CoreText -framework Foundation -framework Symbols \-o "$APP/LyricsHarness"cp "$HERE"/fixtures/*.ttml "$HERE"/fixtures/*.lrc "$HERE"/fixtures/*.json "$HERE"/fixtures/*.txt "$APP/"diff --git a/harness/lyrics/main.m b/harness/lyrics/main.mindex 9a78659c..bc57ef74 100644--- a/harness/lyrics/main.m+++ b/harness/lyrics/main.m@@ -19,6 +19,9 @@// -translateIn S keeps copies of the lines S seconds in with every other one translated, as// Musixmatch's community translations come in after the lines on the phone// -gemini 1 a Gemini key is set, so the menu offers Translate with Gemini (stubs.m answers it)+// -geminiDelay S stubs.m's Gemini answers S seconds later instead of at once+// -check 1 asserts the view's fixes (see runChecks), prints PASS and FAIL lines and quits with the+// number of failures; run it with -song duet -at 20000 -gemini 1 -geminiDelay 1// and the lyrics' own settings by their keys: -spotifyglass.lyricsSimulateWords 1 (sweep line timed// lines on the estimate), -spotifyglass.redesign.lyricsPronunciation 1,// -spotifyglass.redesign.lyricsTranslation 1, -spotifyglass.redesign.lyricsTextOrder '(translation, lyrics, pronunciation)'.@@ -27,6 +30,8 @@#import <objc/message.h>#import <mach/mach_time.h>#import "Redesigned/Lyrics/SGRKaraokeView.h"+#import "Redesigned/Lyrics/SGRSingButton.h"+#import "Shared/Sing/Sing.h"// HEAD's sources (build.sh old) may be from before the lyrics had anything but their words.#if __has_include("Redesigned/Lyrics/LyricsText.h")#import "Redesigned/Lyrics/LyricsText.h"@@ -178,6 +183,110 @@ static void timedTick(id self, SEL _cmd) {sg_sampleCount = 0;}+#pragma mark - the checks++static int sg_failures;++static void expect(BOOL ok, NSString *what) {+ printf("%s %s\n", ok ? "PASS" : "FAIL", what.UTF8String);+ fflush(stdout);+ if (!ok) sg_failures++;+}++static id findView(UIView *root, Class cls) {+ if ([root isKindOfClass:cls]) return root;+ for (UIView *view in root.subviews) {+ id found = findView(view, cls);+ if (found) return found;+ }+ return nil;+}++static void after(double seconds, dispatch_block_t block) {+ dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(seconds * NSEC_PER_SEC)), dispatch_get_main_queue(), block);+}++// Where the line the stack is arranged around starts, against where the anchor puts it for the page's height.+static CGFloat anchorMiss(SGRKaraokeView *karaoke) {+ NSInteger focus = [[karaoke valueForKey:@"focus"] integerValue];+ UIView *view = [karaoke valueForKey:@"shown"][@(focus)];+ if (!view) return CGFLOAT_MAX;+ return fabs(view.center.y - view.bounds.size.height / 2 - karaoke.bounds.size.height * 0.28);+}++static void runChecks(SGRKaraokeView *karaoke, UIView *host) {+ // A held last word keeps its glow as its line goes dim, and loses it once the line has.+ SGKaraokeLine *line = timed(0, @"hold this", @"v1");+ line.words.lastObject.end = line.words.lastObject.start + 3000;+ line.end = line.words.lastObject.end;+ id style = ((id (*)(id, SEL, CGFloat, NSArray *, BOOL, BOOL))objc_msgSend)([NSClassFromString(@"SGRKaraokeStyle") alloc],+ NSSelectorFromString(@"initWithSize:order:pronunciation:translation:"), 30, SGRLyricsTextOrder(), NO, NO);+ UIView *lineView = ((id (*)(id, SEL, id, CGFloat, id, id, BOOL, BOOL))objc_msgSend)([NSClassFromString(@"SGRKaraokeLineView") alloc],+ NSSelectorFromString(@"initWithLine:width:style:under:blurred:sweepsEstimates:"), line, 300, style, nil, NO, NO);+ [karaoke.window addSubview:lineView];+ [lineView setValue:@YES forKey:@"active"];+ ((void (*)(id, SEL, double))objc_msgSend)(lineView, NSSelectorFromString(@"showTime:"), line.words.lastObject.start + 2500.0);+ UILabel *lit = [[[lineView valueForKey:@"words"] lastObject] valueForKey:@"lit"];+ float glow = lit.layer.shadowOpacity;+ expect(glow > 0.3, [NSString stringWithFormat:@"a word held 3 s glows (%.2f)", glow]);+ [lineView setValue:@NO forKey:@"active"];+ expect(lit.layer.shadowOpacity == glow, @"the glow stays on as its line goes out, for the fade to carry it");+ after(0.8, ^{+ expect(lit.layer.shadowOpacity == 0 && lit.hidden, @"the glow is off once the line is dim");+ [lineView removeFromSuperview];+ });++ // The sung line stays on the anchor through a change of height, and is placed for the new one at once.+ expect(anchorMiss(karaoke) < 2, [NSString stringWithFormat:@"the sung line is at the anchor (off by %.1f)", anchorMiss(karaoke)]);+ CGRect frame = host.frame;+ host.frame = CGRectMake(frame.origin.x, frame.origin.y - 75, frame.size.width, frame.size.height + 300);+ [karaoke layoutIfNeeded];+ expect(anchorMiss(karaoke) < 2, [NSString stringWithFormat:@"after 300 pt more height it is at the new anchor (off by %.1f)", anchorMiss(karaoke)]);+ host.frame = frame;+ [karaoke layoutIfNeeded];+ expect(anchorMiss(karaoke) < 2, [NSString stringWithFormat:@"and back (off by %.1f)", anchorMiss(karaoke)]);++ // The mic to VoiceOver: one adjustable button, two of the slider's steps a swipe.+ SGRSingButton *mic = findView(karaoke, SGRSingButton.class);+ expect(mic.isAccessibilityElement && (mic.accessibilityTraits & UIAccessibilityTraitAdjustable), @"the mic is adjustable");+ float level = SGSingLevel();+ [mic accessibilityIncrement];+ expect(fabsf(SGSingLevel() - (level + 0.1f)) < 0.001f, [NSString stringWithFormat:@"a swipe up takes the level from %.2f to %.2f", level, SGSingLevel()]);+ expect([mic.accessibilityValue containsString:SGSingLevelText(SGSingLevel())], [NSString stringWithFormat:@"its value reads the level: %@", mic.accessibilityValue]);+ [mic accessibilityDecrement];+ expect(fabsf(SGSingLevel() - level) < 0.001f, @"a swipe down takes it back");++ // The corner buttons go by their glass's effect, never an alpha over the glass.+ UIView *box = [karaoke valueForKey:@"extrasBox"];+ UIButton *extras = [karaoke valueForKey:@"extras"];+ UIVisualEffectView *micGlass = mic.subviews.firstObject, *extrasGlass = box.subviews.firstObject;+ expect([micGlass isKindOfClass:UIVisualEffectView.class] && [extrasGlass isKindOfClass:UIVisualEffectView.class], @"both have the Kit's glass behind them");+ karaoke.extrasHidden = YES;+ expect(!micGlass.effect && !extrasGlass.effect && mic.alpha == 1 && box.alpha == 1, @"hidden, their glass has no effect and nothing over it fades");+ expect(extras.alpha == 0 && !mic.userInteractionEnabled && !box.userInteractionEnabled, @"hidden, the glyphs are gone and take no touches");+ karaoke.extrasHidden = NO;+ expect(micGlass.effect && extrasGlass.effect && extras.alpha == 1 && mic.userInteractionEnabled, @"shown again, all of it is back");++ // Gemini: the item says where the lyrics go, a spinner while it works and no second request.+ UIAction *gemini = nil;+ for (UIMenuElement *item in extras.menu.children) {+ if ([item.title isEqualToString:@"Translate with Gemini"]) gemini = (UIAction *)item;+ }+ expect(gemini.subtitle.length > 0, [NSString stringWithFormat:@"the Gemini item says: %@", gemini.subtitle]);+ ((void (*)(id, SEL))objc_msgSend)(karaoke, NSSelectorFromString(@"translateWithGemini"));+ BOOL offered = NO;+ for (UIMenuElement *item in extras.menu.children) offered |= [item.title isEqualToString:@"Translate with Gemini"];+ expect(extras.configuration.showsActivityIndicator && !offered, @"while Gemini works the button spins and the item is gone");+ after([NSUserDefaults.standardUserDefaults doubleForKey:@"geminiDelay"] + 0.5, ^{+ expect(!extras.configuration.showsActivityIndicator, @"once it answers the spinner is gone");+ after(0.5, ^{+ printf("%d failed\n", sg_failures);+ fflush(stdout);+ exit(sg_failures);+ });+ });+}+#pragma mark - the app@interface SGHarnessDelegate : UIResponder <UIApplicationDelegate>@@ -231,9 +340,11 @@ static void timedTick(id self, SEL _cmd) {}];self.window.rootViewController = root;[self.window makeKeyAndVisible];+ if ([args boolForKey:@"check"]) after(2, ^{ runChecks(karaoke, host); });if ([args objectForKey:@"openMenu"]) {dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)([args doubleForKey:@"openMenu"] * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{- for (UIView *view in karaoke.subviews) {+ // The button sits in a box with its glass, as the mic does.+ for (UIView *view in [karaoke.subviews valueForKeyPath:@"@unionOfArrays.subviews"]) {if (![view isKindOfClass:UIButton.class]) continue;UIContextMenuInteraction *menu = ((UIButton *)view).contextMenuInteraction;SEL present = NSSelectorFromString(@"_presentMenuAtLocation:");diff --git a/harness/lyrics/stubs.m b/harness/lyrics/stubs.mindex 287976b5..a82e97ac 100644--- a/harness/lyrics/stubs.m+++ b/harness/lyrics/stubs.m@@ -23,7 +23,13 @@ void SGLyricsTranslateWithGemini(NSString *trackID, NSArray<SGKaraokeLine *> *livoid (^done)(NSArray<NSString *> *translations, NSString *error)) {NSMutableArray<NSString *> *translations = [NSMutableArray array];for (SGKaraokeLine *line in lines) [translations addObject:[@"Gemini: " stringByAppendingString:SGKaraokeLineText(line)]];- done(translations, nil);+ // -geminiDelay S: as late as a whole song through the real API can be.+ double delay = [NSUserDefaults.standardUserDefaults doubleForKey:@"geminiDelay"];+ if (delay <= 0) {+ done(translations, nil);+ return;+ }+ dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(delay * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{ done(translations, nil); });}// Line meanings: -title and -artist name the track Genius is searched for, and the setting's keydiff --git a/harness/sing/README.md b/harness/sing/README.mdnew file mode 100644index 00000000..5eb7478b--- /dev/null+++ b/harness/sing/README.md@@ -0,0 +1,94 @@+# Sing harness++Three parts: the separator and the engine on the Mac, Sing end to end in the simulator, and the mic button.++## The separator and the engine on the Mac (`main.m`)++Sing's separator and engine (`tweak/Sources/Shared/Sing/SGSingSeparator.m`, `SGSingEngine.m`) as the tweak+compiles them, on the Mac, over the voice model as the app downloads it (a `separator.mlmodelc` folder of the five+files from https://huggingface.co/My-Name-Is-Jeff/vitrine-sing, a mirror of Darkkos/spoti-sing, MIT). Each check prints a line:++- the STFT: a cosine's peak bin at torch.stft's size (the amplitude times the Hann window's sum over two), noise+ through the STFT and back unchanged, its edges included;+- the model: its input and output shapes, the plan Core ML makes for the compute units asked for (how many+ operations go to the CPU, the GPU and the Neural Engine), the time a two second window takes, and a voice mixed+ over synthetic chords taken apart: the vocals found scored against the voice, beside the mix's own score;+- the engine: the mix pulled through `SGSingEngineRender` in buffers of 1024 on a thread that keeps real time,+ with Sing on from the start: the mix plays dry while the lead fills, then exactly the mix less the vocals the+ offline pass finds, the lead given back after Sing is switched off and the mix playing on where it was, the+ top of the slider playing the vocals alone, a flush dropping the lead, and no allocation on the render thread+ (`malloc_logger`).++The voice is macOS's own speech, `say -o speech.aiff "..."` (any file ExtAudioFile reads will do), repeated a+second apart over 30 seconds.++ ./build.sh && build/sing <separator.mlmodelc> <voice> <out dir> [all|cpu|gpu|ane]+ ./build.sh thread && build/sing-thread <separator.mlmodelc> <voice> <out dir> gpu (ThreadSanitizer)++The out dir gets `mix.wav`, `vocals.wav`, `accompaniment.wav` and `engine.wav` (what played).++2026-10-05, a MacBook with an M4, macOS 27: everything passes under `gpu` and under ThreadSanitizer.++| compute units | Core ML's plan | a window, warm |+|---|---|---|+| `gpu` (CPU and GPU) | 1996 operations on the GPU | 405-525 ms |+| `all` | 1303 on the GPU, 693 on the Neural Engine | 2500 ms |+| `ane` (CPU and Neural Engine) | 1755 on the CPU | 6250 ms |++The model card keeps normalization, attention, softmax and the matrix products in float32, which the Neural+Engine does not run, so splitting the graph between it and the GPU costs more than it saves on the Mac; Sing asks+for the CPU and GPU unless the Sing page's Runs on says otherwise (`SGKeySingComputeUnits`). The vocals found score 28.1 dB against the voice where the mix scores+7.6 dB, and the mix less them 20.5 dB against the chords where the mix scores -7.6 dB. In the engine the lead+settles at 2.84 s (a window plus 1.5 times the model's time and a quarter second).++Not measured: the iPhone. Its GPU is slower than the M4's, and a window must take under 1.5 s (the hop) for the+vocals to keep up; when they do not, what plays is dry until they catch up.++## Sing end to end in the simulator (`sim/main.m`)++The voice model downloaded from Hugging Face by `SGSingModel.m` and checked file by file, loaded by `Sing.x`, and+Spotify's audio chain rebuilt with real units (a converter fed by a render callback, a mixer, RemoteIO, wired with+MakeConnection) with `SpeedPitch.x` and `Sing.x` compiled in (logos, internal generator). The harness is the main+executable, so the rebound imports take as in Spotify and Sing's stage runs in Speed and pitch's chain. The+decoder hands over the voice over the chords; a mock SPTPlayerState computes -position the way Spotify's does+and a mock SPTEsperantoPlayer moves the decoder on seekTo:. A render notify on RemoteIO records what plays.+The script turns Sing on at the vocals' 0, plays, seeks to 5 s at 20 s, and turns Sing off at 30 s; it checks the+download, the model loading, the position Spotify shows against what is heard, the seek's result handed back and+the lead dropped, the vocals down again after it, the lead given back, and what played against the chords alone.++ THEOS=$HOME/theos VOICE=speech.aiff ./build-sim.sh+ xcrun simctl install <udid> build/sim/SingHarness.app+ xcrun simctl launch --console-pty <udid> com.vojta.singharness++2026-10-05, iPhone 17 Pro simulator, iOS 27: the four small files came in from Hugging Face and were checked and+kept; the weights reached 52% before the run was stopped, and were then put in the staging folder from a copy+checked on the Mac, so the move into place ran but the weights' own check in the app did not. The model loads in+1.6-2.1 s. The decoder is drained at 2x until 3.75 s ahead, then at 1x; the position Spotify shows is what is+heard to the hundredth (13.98 s against 13.98 s, with 19.98 s decoded); the seek hands back its result and drops+the 6 s held ahead; switched off, the decoder is drained at 0.5x until the lead is given back. The simulator runs+Core ML on the CPU, 15 s a window, so Sing reads Too slow there and the three vocals checks are skipped: the Mac+harness is where they pass.++## The download on the Mac (`download/`)++`SGSingModel.m` as the tweak compiles it, built for Mac Catalyst (Sing.h imports UIKit) with `download/shim` standing+in for the mod's core so its log goes to stderr, against Hugging Face. The four small files and a fifth of the weights+come in, the download is stopped (its resume data kept beside the staging folder), and the next download carries the+weights on from there through the checks and the move into place. `CFFIXED_USER_HOME` keeps the 489 MB out of the+Mac's own Application Support.++ download/build.sh && CFFIXED_USER_HOME="$(mktemp -d)" build/download 0.2++2026-10-05, a MacBook with an M4, macOS 27: everything passes. The stop is at 20%, the next download's first bytes+read 21%, the server answers the resumed weights 206 and the whole 488986336 bytes are handed over and checked. An+earlier run lost the network for a few seconds and its three retries failed at once, which is why the session now+waits for connectivity. Not run: an expired resume (the server's signed address lasts a day), which starts the file+over.++## The mic button (`button/`)++`SGRSingButton.m` on a field like the lyrics', in the state the launch line asks for, for screenshots.+`button/sing-stubs.m` answers Sing's calls; the lyrics harness links it too.++ button/build.sh && xcrun simctl install <udid> button/build/SingButtonHarness.app+ xcrun simctl launch <udid> com.vojta.singbuttonharness -state 7 -panel 1 -level 0.4diff --git a/harness/sing/build-sim.sh b/harness/sing/build-sim.shnew file mode 100755index 00000000..d1a4922a--- /dev/null+++ b/harness/sing/build-sim.sh@@ -0,0 +1,47 @@+#!/bin/sh+# Builds the simulator harness (sim/main.m): SpeedPitch.x and Sing.x generated by logos with the internal+# generator as the tweak builds them, Sing's model, separator and engine, in an app whose main executable is the+# harness, so the rebinding of AudioUnitSetProperty and AudioOutputUnitStart takes as it does in Spotify. VOICE is+# the voice laid over the chords (macOS's `say -o speech.aiff "..."`), bundled as voice.aiff.+# THEOS=$HOME/theos VOICE=speech.aiff ./build-sim.sh+set -e+cd "$(dirname "$0")"+SRC=../../tweak/Sources+SING=$SRC/Shared/Sing+OUT=build/sim+rm -rf "$OUT"+mkdir -p "$OUT/gen" "$OUT/SingHarness.app"+"$THEOS/bin/logos.pl" -c generator=internal "$SRC/Shared/Player/SpeedPitch.x" > "$OUT/gen/SpeedPitch.m"+"$THEOS/bin/logos.pl" -c generator=internal "$SING/Sing.x" > "$OUT/gen/Sing.m"+SDK=$(xcrun --sdk iphonesimulator --show-sdk-path)+xcrun -sdk iphonesimulator clang -target arm64-apple-ios18.0-simulator -fobjc-arc -g -O1 -isysroot "$SDK" -Wno-deprecated-declarations \+ -I"$SRC" -I"$SING" -I"$SRC/Shared/Player" \+ sim/main.m ../scene.m "$OUT/gen/SpeedPitch.m" "$OUT/gen/Sing.m" "$SRC/Shared/Player/SGTimePitch.m" \+ "$SING/SGSingModel.m" "$SING/SGSingSeparator.m" "$SING/SGSingEngine.m" \+ "$SRC/Core/SGRebind.m" "$SRC/Core/SGLog.m" "$SRC/Core/SGPrefs.m" "$SRC/Core/SGUIMode.m" "$SRC/Core/SGFlagForce.m" \+ -framework UIKit -framework QuartzCore -framework AudioToolbox -framework AVFoundation -framework Accelerate -framework CoreML -framework Foundation \+ -o "$OUT/SingHarness.app/SingHarness"+if [ -n "$VOICE" ]; then cp "$VOICE" "$OUT/SingHarness.app/voice.aiff"; fi+cat > "$OUT/SingHarness.app/Info.plist" <<'PLIST'+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0"><dict>+<key>CFBundleExecutable</key><string>SingHarness</string>+<key>CFBundleIdentifier</key><string>com.vojta.singharness</string>+<key>CFBundleName</key><string>SingHarness</string>+<key>CFBundleVersion</key><string>1</string>+<key>CFBundleShortVersionString</key><string>1.0</string>+<key>UILaunchScreen</key><dict/>+<key>UIBackgroundModes</key><array><string>audio</string></array>+<key>UIApplicationSceneManifest</key><dict>+ <key>UIApplicationSupportsMultipleScenes</key><false/>+ <key>UISceneConfigurations</key><dict>+ <key>UIWindowSceneSessionRoleApplication</key><array><dict>+ <key>UISceneConfigurationName</key><string>Default</string>+ <key>UISceneDelegateClassName</key><string>SGRHarnessScene</string>+ </dict></array>+ </dict>+</dict>+</dict></plist>+PLIST+echo "built $OUT/SingHarness.app"diff --git a/harness/sing/build.sh b/harness/sing/build.shnew file mode 100755index 00000000..e9243e35--- /dev/null+++ b/harness/sing/build.sh@@ -0,0 +1,15 @@+#!/bin/sh+# Builds the Sing harness for the Mac: the separator and the engine (tweak/Sources/Shared/Sing) as the tweak+# compiles them, and main.m.+# ./build.sh build/sing+# ./build.sh thread build/sing-thread, under ThreadSanitizer+set -e+cd "$(dirname "$0")"+SING=../../tweak/Sources/Shared/Sing+SANITIZE=$1+OUT=build/sing${SANITIZE:+-$SANITIZE}+mkdir -p build+xcrun clang -fobjc-arc -O2 -g -Wall -Werror -target arm64-apple-macos15.0 ${SANITIZE:+-fsanitize=$SANITIZE -O1 -fno-omit-frame-pointer} \+ -I ../../tweak/Sources main.m "$SING"/SGSingSeparator.m "$SING"/SGSingEngine.m \+ -framework Foundation -framework AudioToolbox -framework Accelerate -framework CoreML -o "$OUT"+echo "built $OUT"diff --git a/harness/sing/button/build.sh b/harness/sing/button/build.shnew file mode 100755index 00000000..4fd0d3fd--- /dev/null+++ b/harness/sing/button/build.sh@@ -0,0 +1,38 @@+#!/bin/sh+# Builds the mic button harness for the simulator: SGRSingButton.m with the Kit's glass, and Sing's calls stood in+# for by sing-stubs.m.+set -e+HERE=$(cd "$(dirname "$0")" && pwd)+SRC=$HERE/../../../tweak/Sources+APP=$HERE/build/SingButtonHarness.app+rm -rf "$APP"; mkdir -p "$APP"+SDK=$(xcrun --sdk iphonesimulator --show-sdk-path)+xcrun -sdk iphonesimulator clang -target arm64-apple-ios17.0-simulator -fobjc-arc -g -O0 -isysroot "$SDK" -Wall -Wno-deprecated-declarations \+ -Wno-undeclared-selector -I"$SRC" "$HERE/main.m" "$HERE/sing-stubs.m" "$HERE/../../scene.m" \+ "$SRC"/Redesigned/Lyrics/SGRSingButton.m "$SRC"/Redesigned/Kit/SGRGlass.m "$SRC"/Redesigned/Kit/SGRTokens.m \+ "$SRC"/Core/SGGlass.m "$SRC"/Core/SGLog.m "$SRC"/Core/SGPrefs.m \+ -framework UIKit -framework QuartzCore -framework CoreGraphics -framework Foundation -framework Symbols -o "$APP/SingButtonHarness"+cat > "$APP/Info.plist" <<'PLIST'+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0"><dict>+<key>CFBundleExecutable</key><string>SingButtonHarness</string>+<key>CFBundleIdentifier</key><string>com.vojta.singbuttonharness</string>+<key>CFBundleName</key><string>Sing button</string>+<key>CFBundleVersion</key><string>1</string>+<key>CFBundleShortVersionString</key><string>1.0</string>+<key>UIUserInterfaceStyle</key><string>Dark</string>+<key>UILaunchScreen</key><dict/>+<key>UIApplicationSceneManifest</key><dict>+ <key>UIApplicationSupportsMultipleScenes</key><false/>+ <key>UISceneConfigurations</key><dict>+ <key>UIWindowSceneSessionRoleApplication</key><array><dict>+ <key>UISceneConfigurationName</key><string>Default</string>+ <key>UISceneDelegateClassName</key><string>SGRHarnessScene</string>+ </dict></array>+ </dict>+</dict>+</dict></plist>+PLIST+codesign -f -s - "$APP" >/dev/null 2>&1+echo "built $APP"diff --git a/harness/sing/button/build/SingButtonHarness.app/Info.plist b/harness/sing/button/build/SingButtonHarness.app/Info.plistnew file mode 100644index 00000000..ea2d1a4f--- /dev/null+++ b/harness/sing/button/build/SingButtonHarness.app/Info.plist@@ -0,0 +1,20 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0"><dict>+<key>CFBundleExecutable</key><string>SingButtonHarness</string>+<key>CFBundleIdentifier</key><string>com.vojta.singbuttonharness</string>+<key>CFBundleName</key><string>Sing button</string>+<key>CFBundleVersion</key><string>1</string>+<key>CFBundleShortVersionString</key><string>1.0</string>+<key>UIUserInterfaceStyle</key><string>Dark</string>+<key>UILaunchScreen</key><dict/>+<key>UIApplicationSceneManifest</key><dict>+ <key>UIApplicationSupportsMultipleScenes</key><false/>+ <key>UISceneConfigurations</key><dict>+ <key>UIWindowSceneSessionRoleApplication</key><array><dict>+ <key>UISceneConfigurationName</key><string>Default</string>+ <key>UISceneDelegateClassName</key><string>SGRHarnessScene</string>+ </dict></array>+ </dict>+</dict>+</dict></plist>diff --git a/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarnessnew file mode 100755index 00000000..0823303dBinary files /dev/null and b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness differdiff --git a/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Info.plist b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Info.plistnew file mode 100644index 00000000..cac440b6--- /dev/null+++ b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Info.plist@@ -0,0 +1,20 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+ <dict>+ <key>CFBundleDevelopmentRegion</key>+ <string>English</string>+ <key>CFBundleIdentifier</key>+ <string>com.apple.xcode.dsym.SingButtonHarness</string>+ <key>CFBundleInfoDictionaryVersion</key>+ <string>6.0</string>+ <key>CFBundlePackageType</key>+ <string>dSYM</string>+ <key>CFBundleSignature</key>+ <string>????</string>+ <key>CFBundleShortVersionString</key>+ <string>1.0</string>+ <key>CFBundleVersion</key>+ <string>1</string>+ </dict>+</plist>diff --git a/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/DWARF/SingButtonHarness b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/DWARF/SingButtonHarnessnew file mode 100644index 00000000..58138cb2Binary files /dev/null and b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/DWARF/SingButtonHarness differdiff --git a/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/Relocations/aarch64/SingButtonHarness.yml b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/Relocations/aarch64/SingButtonHarness.ymlnew file mode 100644index 00000000..c2d4cd2e--- /dev/null+++ b/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness.dSYM/Contents/Resources/Relocations/aarch64/SingButtonHarness.yml@@ -0,0 +1,137 @@+---+triple: 'arm64-apple-darwin'+binary-path: '/Users/r/GitRepositories/spotifyglass-gpl-wt-sing/harness/sing/button/build/SingButtonHarness.app/SingButtonHarness'+relocations:+ - { offset: 0x27, size: 0x8, addend: 0x0, symName: _SGRAccentColor, symObjAddr: 0x0, symBinAddr: 0x100000ED0, symSize: 0x8 }+ - { offset: 0x209C, size: 0x8, addend: 0x0, symName: _SGRAccentColor, symObjAddr: 0x0, symBinAddr: 0x100000ED0, symSize: 0x8 }+ - { offset: 0x20B5, size: 0x8, addend: 0x0, symName: _SGTopController, symObjAddr: 0x8, symBinAddr: 0x100000ED8, symSize: 0x184 }+ - { offset: 0x2108, size: 0x8, addend: 0x0, symName: '-[SGHarnessDelegate application:didFinishLaunchingWithOptions:]', symObjAddr: 0x18C, symBinAddr: 0x10000105C, symSize: 0x5B8 }+ - { offset: 0x21AA, size: 0x8, addend: 0x0, symName: _CGRectMake, symObjAddr: 0x744, symBinAddr: 0x100001614, symSize: 0x4C }+ - { offset: 0x2210, size: 0x8, addend: 0x0, symName: '___63-[SGHarnessDelegate application:didFinishLaunchingWithOptions:]_block_invoke', symObjAddr: 0x790, symBinAddr: 0x100001660, symSize: 0x38 }+ - { offset: 0x2249, size: 0x8, addend: 0x0, symName: ___copy_helper_block_e8_32s, symObjAddr: 0x7C8, symBinAddr: 0x100001698, symSize: 0x38 }+ - { offset: 0x226D, size: 0x8, addend: 0x0, symName: ___destroy_helper_block_e8_32s, symObjAddr: 0x800, symBinAddr: 0x1000016D0, symSize: 0x2C }+ - { offset: 0x2289, size: 0x8, addend: 0x0, symName: '-[SGHarnessDelegate window]', symObjAddr: 0x82C, symBinAddr: 0x1000016FC, symSize: 0x24 }+ - { offset: 0x22BF, size: 0x8, addend: 0x0, symName: '-[SGHarnessDelegate setWindow:]', symObjAddr: 0x850, symBinAddr: 0x100001720, symSize: 0x3C }+ - { offset: 0x22FD, size: 0x8, addend: 0x0, symName: '-[SGHarnessDelegate .cxx_destruct]', symObjAddr: 0x88C, symBinAddr: 0x10000175C, symSize: 0x38 }+ - { offset: 0x232F, size: 0x8, addend: 0x0, symName: _main, symObjAddr: 0x8C4, symBinAddr: 0x100001794, symSize: 0x90 }+ - { offset: 0x2B4C, size: 0x8, addend: 0x0, symName: _SGSingCurrentState, symObjAddr: 0x0, symBinAddr: 0x100001824, symSize: 0xF0 }+ - { offset: 0x2B65, size: 0x8, addend: 0x0, symName: _SGSingChangedNotification, symObjAddr: 0xAB8, symBinAddr: 0x100010030, symSize: 0x0 }+ - { offset: 0x2C33, size: 0x8, addend: 0x0, symName: _SGSingCurrentState, symObjAddr: 0x0, symBinAddr: 0x100001824, symSize: 0xF0 }+ - { offset: 0x2C4C, size: 0x8, addend: 0x0, symName: _defaults, symObjAddr: 0xF0, symBinAddr: 0x100001914, symSize: 0x1C }+ - { offset: 0x2C65, size: 0x8, addend: 0x0, symName: _SGSingStatusText, symObjAddr: 0x10C, symBinAddr: 0x100001930, symSize: 0x180 }+ - { offset: 0x2C8D, size: 0x8, addend: 0x0, symName: _names, symObjAddr: 0x28C, symBinAddr: 0x100001AB0, symSize: 0xC }+ - { offset: 0x2CA6, size: 0x8, addend: 0x0, symName: _SGSingStatusDetail, symObjAddr: 0x298, symBinAddr: 0x100001ABC, symSize: 0x14 }+ - { offset: 0x2CBF, size: 0x8, addend: 0x0, symName: _SGSingMissing, symObjAddr: 0x2AC, symBinAddr: 0x100001AD0, symSize: 0xC }+ - { offset: 0x2CD8, size: 0x8, addend: 0x0, symName: _SGSingOn, symObjAddr: 0x2B8, symBinAddr: 0x100001ADC, symSize: 0x1C }+ - { offset: 0x2CF1, size: 0x8, addend: 0x0, symName: _SGSetSingOn, symObjAddr: 0x2D4, symBinAddr: 0x100001AF8, symSize: 0xA4 }+ - { offset: 0x2D15, size: 0x8, addend: 0x0, symName: _SGSingLevel, symObjAddr: 0x378, symBinAddr: 0x100001B9C, symSize: 0xF8 }+ - { offset: 0x2D2E, size: 0x8, addend: 0x0, symName: _SGSetSingLevel, symObjAddr: 0x470, symBinAddr: 0x100001C94, symSize: 0x50 }+ - { offset: 0x2D52, size: 0x8, addend: 0x0, symName: _SGSingLevelText, symObjAddr: 0x4C0, symBinAddr: 0x100001CE4, symSize: 0x18C }+ - { offset: 0x2D7A, size: 0x8, addend: 0x0, symName: _SGSingModelProgress, symObjAddr: 0x64C, symBinAddr: 0x100001E70, symSize: 0x50 }+ - { offset: 0x2D93, size: 0x8, addend: 0x0, symName: _SGSingModelSizeText, symObjAddr: 0x69C, symBinAddr: 0x100001EC0, symSize: 0xC }+ - { offset: 0x2DAC, size: 0x8, addend: 0x0, symName: _SGSingModelError, symObjAddr: 0x6A8, symBinAddr: 0x100001ECC, symSize: 0x8 }+ - { offset: 0x2DC5, size: 0x8, addend: 0x0, symName: _SGSingDownloadModel, symObjAddr: 0x6B0, symBinAddr: 0x100001ED4, symSize: 0x4 }+ - { offset: 0x2DDA, size: 0x8, addend: 0x0, symName: _SGSingCancelModelDownload, symObjAddr: 0x6B4, symBinAddr: 0x100001ED8, symSize: 0x4 }+ - { offset: 0x2EF4, size: 0x8, addend: 0x0, symName: '-[SGRHarnessScene scene:willConnectToSession:options:]', symObjAddr: 0x0, symBinAddr: 0x100001EDC, symSize: 0x13C }+ - { offset: 0x4DAA, size: 0x8, addend: 0x0, symName: '-[SGRHarnessScene scene:willConnectToSession:options:]', symObjAddr: 0x0, symBinAddr: 0x100001EDC, symSize: 0x13C }+ - { offset: 0x4E14, size: 0x8, addend: 0x0, symName: '-[SGRHarnessScene window]', symObjAddr: 0x13C, symBinAddr: 0x100002018, symSize: 0x24 }+ - { offset: 0x4E4A, size: 0x8, addend: 0x0, symName: '-[SGRHarnessScene setWindow:]', symObjAddr: 0x160, symBinAddr: 0x10000203C, symSize: 0x3C }+ - { offset: 0x4E88, size: 0x8, addend: 0x0, symName: '-[SGRHarnessScene .cxx_destruct]', symObjAddr: 0x19C, symBinAddr: 0x100002078, symSize: 0x38 }+ - { offset: 0x5032, size: 0x8, addend: 0x0, symName: '-[SGRSingButton initWithFrame:]', symObjAddr: 0x0, symBinAddr: 0x1000020B0, symSize: 0x530 }+ - { offset: 0x5142, size: 0x8, addend: 0x0, symName: _kPanelGlassKey, symObjAddr: 0x123F5, symBinAddr: 0x100015660, symSize: 0x0 }+ - { offset: 0x8663, size: 0x8, addend: 0x0, symName: '-[SGRSingButton initWithFrame:]', symObjAddr: 0x0, symBinAddr: 0x1000020B0, symSize: 0x530 }+ - { offset: 0x86B6, size: 0x8, addend: 0x0, symName: '-[SGRSingButton dealloc]', symObjAddr: 0x530, symBinAddr: 0x1000025E0, symSize: 0x94 }+ - { offset: 0x86E8, size: 0x8, addend: 0x0, symName: '-[SGRSingButton layoutSubviews]', symObjAddr: 0x5C4, symBinAddr: 0x100002674, symSize: 0x260 }+ - { offset: 0x8764, size: 0x8, addend: 0x0, symName: _CGPointMake, symObjAddr: 0x824, symBinAddr: 0x1000028D4, symSize: 0x2C }+ - { offset: 0x87AC, size: 0x8, addend: 0x0, symName: '-[SGRSingButton refresh]', symObjAddr: 0x850, symBinAddr: 0x100002900, symSize: 0x648 }+ - { offset: 0x8884, size: 0x8, addend: 0x0, symName: '-[SGRSingButton tapped]', symObjAddr: 0xEC4, symBinAddr: 0x100002F74, symSize: 0x2AC }+ - { offset: 0x88B6, size: 0x8, addend: 0x0, symName: _tell, symObjAddr: 0x1170, symBinAddr: 0x100003220, symSize: 0x23C }+ - { offset: 0x8904, size: 0x8, addend: 0x0, symName: '___23-[SGRSingButton tapped]_block_invoke', symObjAddr: 0x13AC, symBinAddr: 0x10000345C, symSize: 0x34 }+ - { offset: 0x892C, size: 0x8, addend: 0x0, symName: '___23-[SGRSingButton tapped]_block_invoke_2', symObjAddr: 0x13E0, symBinAddr: 0x100003490, symSize: 0x28 }+ - { offset: 0x8954, size: 0x8, addend: 0x0, symName: '___23-[SGRSingButton tapped]_block_invoke_3', symObjAddr: 0x1408, symBinAddr: 0x1000034B8, symSize: 0x30 }+ - { offset: 0x897C, size: 0x8, addend: 0x0, symName: '-[SGRSingButton held:]', symObjAddr: 0x1438, symBinAddr: 0x1000034E8, symSize: 0x78 }+ - { offset: 0x89BC, size: 0x8, addend: 0x0, symName: '-[SGRSingButton showPanel]', symObjAddr: 0x14B0, symBinAddr: 0x100003560, symSize: 0x548 }+ - { offset: 0x89EE, size: 0x8, addend: 0x0, symName: '___26-[SGRSingButton showPanel]_block_invoke', symObjAddr: 0x19F8, symBinAddr: 0x100003AA8, symSize: 0x68 }+ - { offset: 0x8A2B, size: 0x8, addend: 0x0, symName: '-[SGRSingButton placePanel]', symObjAddr: 0x1AC4, symBinAddr: 0x100003B10, symSize: 0x290 }+ - { offset: 0x8A6B, size: 0x8, addend: 0x0, symName: _CGRectMake, symObjAddr: 0x1D54, symBinAddr: 0x100003DA0, symSize: 0x4C }+ - { offset: 0x8AD1, size: 0x8, addend: 0x0, symName: '-[SGRSingButton showLevel]', symObjAddr: 0x1DA0, symBinAddr: 0x100003DEC, symSize: 0xD0 }+ - { offset: 0x8B03, size: 0x8, addend: 0x0, symName: '-[SGRSingButton slid]', symObjAddr: 0x1E70, symBinAddr: 0x100003EBC, symSize: 0x68 }+ - { offset: 0x8B43, size: 0x8, addend: 0x0, symName: '-[SGRSingButton holdPanel]', symObjAddr: 0x1ED8, symBinAddr: 0x100003F24, symSize: 0x54 }+ - { offset: 0x8B75, size: 0x8, addend: 0x0, symName: '-[SGRSingButton letPanelGo]', symObjAddr: 0x1F2C, symBinAddr: 0x100003F78, symSize: 0x138 }+ - { offset: 0x8BB5, size: 0x8, addend: 0x0, symName: '___27-[SGRSingButton letPanelGo]_block_invoke', symObjAddr: 0x2064, symBinAddr: 0x1000040B0, symSize: 0x70 }+ - { offset: 0x8BFC, size: 0x8, addend: 0x0, symName: ___copy_helper_block_e8_32w, symObjAddr: 0x20D4, symBinAddr: 0x100004120, symSize: 0x34 }+ - { offset: 0x8C20, size: 0x8, addend: 0x0, symName: ___destroy_helper_block_e8_32w, symObjAddr: 0x2108, symBinAddr: 0x100004154, symSize: 0x28 }+ - { offset: 0x8C3C, size: 0x8, addend: 0x0, symName: '-[SGRSingButton hidePanel]', symObjAddr: 0x2130, symBinAddr: 0x10000417C, symSize: 0xB0 }+ - { offset: 0x8C6E, size: 0x8, addend: 0x0, symName: '___26-[SGRSingButton hidePanel]_block_invoke', symObjAddr: 0x21E0, symBinAddr: 0x10000422C, symSize: 0x68 }+ - { offset: 0x8CAB, size: 0x8, addend: 0x0, symName: '-[SGRSingButton pointInside:withEvent:]', symObjAddr: 0x2248, symBinAddr: 0x100004294, symSize: 0x11C }+ - { offset: 0x8CFD, size: 0x8, addend: 0x0, symName: '-[SGRSingButton .cxx_destruct]', symObjAddr: 0x2364, symBinAddr: 0x1000043B0, symSize: 0xB8 }+ - { offset: 0x8D2F, size: 0x8, addend: 0x0, symName: ___tell_block_invoke, symObjAddr: 0x241C, symBinAddr: 0x100004468, symSize: 0x5C }+ - { offset: 0x8D76, size: 0x8, addend: 0x0, symName: ___copy_helper_block_e8_32b, symObjAddr: 0x2478, symBinAddr: 0x1000044C4, symSize: 0x38 }+ - { offset: 0x99BD, size: 0x8, addend: 0x0, symName: _SGRGlassInside, symObjAddr: 0x0, symBinAddr: 0x1000044FC, symSize: 0xBC }+ - { offset: 0x99D6, size: 0x8, addend: 0x0, symName: _kInsideModeKey, symObjAddr: 0x4838, symBinAddr: 0x100015661, symSize: 0x0 }+ - { offset: 0x99F2, size: 0x8, addend: 0x0, symName: _kFilmKey, symObjAddr: 0x4839, symBinAddr: 0x100015662, symSize: 0x0 }+ - { offset: 0xA38B, size: 0x8, addend: 0x0, symName: _SGRGlassInside, symObjAddr: 0x0, symBinAddr: 0x1000044FC, symSize: 0xBC }+ - { offset: 0xA3CF, size: 0x8, addend: 0x0, symName: _glassInside, symObjAddr: 0xBC, symBinAddr: 0x1000045B8, symSize: 0x58C }+ - { offset: 0xA497, size: 0x8, addend: 0x0, symName: _CGSizeMake, symObjAddr: 0x648, symBinAddr: 0x100004B44, symSize: 0x2C }+ - { offset: 0xA4DF, size: 0x8, addend: 0x0, symName: _SGRGlassCapsuleInside, symObjAddr: 0x674, symBinAddr: 0x100004B70, symSize: 0xA0 }+ - { offset: 0xA531, size: 0x8, addend: 0x0, symName: _glassMode, symObjAddr: 0x714, symBinAddr: 0x100004C10, symSize: 0x64 }+ - { offset: 0xA54A, size: 0x8, addend: 0x0, symName: _newShape, symObjAddr: 0x778, symBinAddr: 0x100004C74, symSize: 0x1C4 }+ - { offset: 0xA580, size: 0x8, addend: 0x0, symName: _CGRectMake, symObjAddr: 0x93C, symBinAddr: 0x100004E38, symSize: 0x4C }+ - { offset: 0xA5E6, size: 0x8, addend: 0x0, symName: _keepFilm, symObjAddr: 0x988, symBinAddr: 0x100004E84, symSize: 0x5B4 }+ - { offset: 0xA670, size: 0x8, addend: 0x0, symName: _CGPointMake, symObjAddr: 0xF3C, symBinAddr: 0x100005438, symSize: 0x2C }+ - { offset: 0xA6B8, size: 0x8, addend: 0x0, symName: ___CGPointEqualToPoint, symObjAddr: 0xF68, symBinAddr: 0x100005464, symSize: 0x58 }+ - { offset: 0xA729, size: 0x8, addend: 0x0, symName: _SGRPrimary, symObjAddr: 0x0, symBinAddr: 0x1000054BC, symSize: 0x1C }+ - { offset: 0xA742, size: 0x8, addend: 0x0, symName: _SGRSideMargin, symObjAddr: 0x9E8, symBinAddr: 0x10000C818, symSize: 0x0 }+ - { offset: 0xA76E, size: 0x8, addend: 0x0, symName: _SGRGrid, symObjAddr: 0x9F0, symBinAddr: 0x10000C820, symSize: 0x0 }+ - { offset: 0xA783, size: 0x8, addend: 0x0, symName: _SGRRadiusArtwork, symObjAddr: 0x9F8, symBinAddr: 0x10000C828, symSize: 0x0 }+ - { offset: 0xA798, size: 0x8, addend: 0x0, symName: _SGRRadiusCard, symObjAddr: 0xA00, symBinAddr: 0x10000C830, symSize: 0x0 }+ - { offset: 0xA7AD, size: 0x8, addend: 0x0, symName: _SGRRadiusCover, symObjAddr: 0xA08, symBinAddr: 0x10000C838, symSize: 0x0 }+ - { offset: 0xA7C2, size: 0x8, addend: 0x0, symName: _SGRRadiusThumb, symObjAddr: 0xA10, symBinAddr: 0x10000C840, symSize: 0x0 }+ - { offset: 0xA7D7, size: 0x8, addend: 0x0, symName: _SGRGlassCircleSize, symObjAddr: 0xA18, symBinAddr: 0x10000C848, symSize: 0x0 }+ - { offset: 0xA7EC, size: 0x8, addend: 0x0, symName: _SGRActionHeight, symObjAddr: 0xA20, symBinAddr: 0x10000C850, symSize: 0x0 }+ - { offset: 0xA801, size: 0x8, addend: 0x0, symName: _SGRActionSpacing, symObjAddr: 0xA28, symBinAddr: 0x10000C858, symSize: 0x0 }+ - { offset: 0xA816, size: 0x8, addend: 0x0, symName: _SGRGlassSpacing, symObjAddr: 0xA30, symBinAddr: 0x10000C860, symSize: 0x0 }+ - { offset: 0xA82B, size: 0x8, addend: 0x0, symName: _SGRCrossfade, symObjAddr: 0xA38, symBinAddr: 0x10000C868, symSize: 0x0 }+ - { offset: 0xACC8, size: 0x8, addend: 0x0, symName: _SGRPrimary, symObjAddr: 0x0, symBinAddr: 0x1000054BC, symSize: 0x1C }+ - { offset: 0xACE1, size: 0x8, addend: 0x0, symName: _SGRSecondary, symObjAddr: 0x1C, symBinAddr: 0x1000054D8, symSize: 0x64 }+ - { offset: 0xACFA, size: 0x8, addend: 0x0, symName: _SGRIncreaseContrast, symObjAddr: 0x80, symBinAddr: 0x10000553C, symSize: 0x14 }+ - { offset: 0xAD13, size: 0x8, addend: 0x0, symName: _SGRTertiary, symObjAddr: 0x94, symBinAddr: 0x100005550, symSize: 0x60 }+ - { offset: 0xAD2C, size: 0x8, addend: 0x0, symName: _SGRAccent, symObjAddr: 0xF4, symBinAddr: 0x1000055B0, symSize: 0xFC }+ - { offset: 0xAD45, size: 0x8, addend: 0x0, symName: _SGRNeutralField, symObjAddr: 0x1F0, symBinAddr: 0x1000056AC, symSize: 0x3C }+ - { offset: 0xAD5E, size: 0x8, addend: 0x0, symName: _SGRSolidGlassFill, symObjAddr: 0x22C, symBinAddr: 0x1000056E8, symSize: 0x34 }+ - { offset: 0xAD77, size: 0x8, addend: 0x0, symName: _SGRHairline, symObjAddr: 0x260, symBinAddr: 0x10000571C, symSize: 0x68 }+ - { offset: 0xAD90, size: 0x8, addend: 0x0, symName: _SGRElevated, symObjAddr: 0x2C8, symBinAddr: 0x100005784, symSize: 0x134 }+ - { offset: 0xADFE, size: 0x8, addend: 0x0, symName: _SGRFont, symObjAddr: 0x3FC, symBinAddr: 0x1000058B8, symSize: 0x1AC }+ - { offset: 0xAE6C, size: 0x8, addend: 0x0, symName: _SGRMonospacedDigitsFont, symObjAddr: 0x5A8, symBinAddr: 0x100005A64, symSize: 0x240 }+ - { offset: 0xAEB3, size: 0x8, addend: 0x0, symName: _SGRReduceMotion, symObjAddr: 0x7E8, symBinAddr: 0x100005CA4, symSize: 0x14 }+ - { offset: 0xAECC, size: 0x8, addend: 0x0, symName: _SGRReduceTransparency, symObjAddr: 0x7FC, symBinAddr: 0x100005CB8, symSize: 0x14 }+ - { offset: 0xAEE5, size: 0x8, addend: 0x0, symName: _SGRAnimate, symObjAddr: 0x810, symBinAddr: 0x100005CCC, symSize: 0x1D4 }+ - { offset: 0xB6BA, size: 0x8, addend: 0x0, symName: _SGGlassEffect, symObjAddr: 0x0, symBinAddr: 0x100005EA0, symSize: 0x88 }+ - { offset: 0xB6D3, size: 0x8, addend: 0x0, symName: _kPanesKey, symObjAddr: 0x39D3, symBinAddr: 0x100015663, symSize: 0x0 }+ - { offset: 0xB7F2, size: 0x8, addend: 0x0, symName: _SGGlassEffect, symObjAddr: 0x0, symBinAddr: 0x100005EA0, symSize: 0x88 }+ - { offset: 0xB81A, size: 0x8, addend: 0x0, symName: _SGGlassFor, symObjAddr: 0x88, symBinAddr: 0x100005F28, symSize: 0x134 }+ - { offset: 0xB85E, size: 0x8, addend: 0x0, symName: _newPane, symObjAddr: 0x1BC, symBinAddr: 0x10000605C, symSize: 0xD8 }+ - { offset: 0xB886, size: 0x8, addend: 0x0, symName: _SGGlassAt, symObjAddr: 0x294, symBinAddr: 0x100006134, symSize: 0x1E0 }+ - { offset: 0xB8D8, size: 0x8, addend: 0x0, symName: _SGHideGlassFrom, symObjAddr: 0x474, symBinAddr: 0x100006314, symSize: 0xF4 }+ - { offset: 0xB934, size: 0x8, addend: 0x0, symName: _SGShapeGlass, symObjAddr: 0x568, symBinAddr: 0x100006408, symSize: 0x2B4 }+ - { offset: 0xC286, size: 0x8, addend: 0x0, symName: _SGLogLong, symObjAddr: 0x0, symBinAddr: 0x1000066BC, symSize: 0x3AC }+ - { offset: 0xC293, size: 0x8, addend: 0x0, symName: ___SGLogLong_block_invoke, symObjAddr: 0x3AC, symBinAddr: 0x100006A68, symSize: 0x174 }+ - { offset: 0xC37C, size: 0x8, addend: 0x0, symName: _SGRequireClasses, symObjAddr: 0x5FC, symBinAddr: 0x100006CB8, symSize: 0x268 }+ - { offset: 0xC465, size: 0x8, addend: 0x0, symName: _SGLogLong, symObjAddr: 0x0, symBinAddr: 0x1000066BC, symSize: 0x3AC }+ - { offset: 0xC4CC, size: 0x8, addend: 0x0, symName: ___os_log_helper_16_2_1_8_34, symObjAddr: 0x520, symBinAddr: 0x100006BDC, symSize: 0x40 }+ - { offset: 0xC4F8, size: 0x8, addend: 0x0, symName: ___copy_helper_block_e8_32s40s, symObjAddr: 0x560, symBinAddr: 0x100006C1C, symSize: 0x58 }+ - { offset: 0xC51C, size: 0x8, addend: 0x0, symName: ___destroy_helper_block_e8_32s40s, symObjAddr: 0x5B8, symBinAddr: 0x100006C74, symSize: 0x44 }+ - { offset: 0xC704, size: 0x8, addend: 0x0, symName: _SGFlag, symObjAddr: 0x0, symBinAddr: 0x100006F20, symSize: 0x120 }+ - { offset: 0xC71D, size: 0x8, addend: 0x0, symName: _SGFlagOverridePrefix, symObjAddr: 0x868, symBinAddr: 0x100010148, symSize: 0x0 }+ - { offset: 0xC790, size: 0x8, addend: 0x0, symName: _SGFlag, symObjAddr: 0x0, symBinAddr: 0x100006F20, symSize: 0x120 }+ - { offset: 0xC7E2, size: 0x8, addend: 0x0, symName: _SGEnabled, symObjAddr: 0x120, symBinAddr: 0x100007040, symSize: 0x5C }+ - { offset: 0xC80A, size: 0x8, addend: 0x0, symName: _SGHidden, symObjAddr: 0x17C, symBinAddr: 0x10000709C, symSize: 0x5C }+ - { offset: 0xC832, size: 0x8, addend: 0x0, symName: _SGSetEnabled, symObjAddr: 0x1D8, symBinAddr: 0x1000070F8, symSize: 0x94 }+ - { offset: 0xC864, size: 0x8, addend: 0x0, symName: _SGInt, symObjAddr: 0x26C, symBinAddr: 0x10000718C, symSize: 0xD8 }+ - { offset: 0xC8A8, size: 0x8, addend: 0x0, symName: _SGSetInt, symObjAddr: 0x344, symBinAddr: 0x100007264, symSize: 0x90 }+ - { offset: 0xC8DA, size: 0x8, addend: 0x0, symName: _SGFlagOverride, symObjAddr: 0x3D4, symBinAddr: 0x1000072F4, symSize: 0xC0 }+ - { offset: 0xC902, size: 0x8, addend: 0x0, symName: _SGSetFlagOverride, symObjAddr: 0x494, symBinAddr: 0x1000073B4, symSize: 0x11C }+ - { offset: 0xC934, size: 0x8, addend: 0x0, symName: _SGMigrateKey, symObjAddr: 0x5B0, symBinAddr: 0x1000074D0, symSize: 0x180 }+ - { offset: 0xC99F, size: 0x8, addend: 0x0, symName: _SGRestartSpotify, symObjAddr: 0x730, symBinAddr: 0x100007650, symSize: 0xA0 }+ - { offset: 0xC9C9, size: 0x8, addend: 0x0, symName: ___SGRestartSpotify_block_invoke, symObjAddr: 0x7D0, symBinAddr: 0x1000076F0, symSize: 0x20 }+...diff --git a/harness/sing/button/build/SingButtonHarness.app/_CodeSignature/CodeResources b/harness/sing/button/build/SingButtonHarness.app/_CodeSignature/CodeResourcesnew file mode 100644index 00000000..25f54b35--- /dev/null+++ b/harness/sing/button/build/SingButtonHarness.app/_CodeSignature/CodeResources@@ -0,0 +1,135 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>files</key>+ <dict>+ <key>Info.plist</key>+ <data>+ OgtQnE1cCuWWlgLJXofOYXRrfiA=+ </data>+ <key>SingButtonHarness.dSYM/Contents/Info.plist</key>+ <data>+ lu6HGfL+/j0TTMDpTSYBJPGyz70=+ </data>+ <key>SingButtonHarness.dSYM/Contents/Resources/DWARF/SingButtonHarness</key>+ <data>+ Mb9tY258C2IJT27/7s8+L7wFhtM=+ </data>+ <key>SingButtonHarness.dSYM/Contents/Resources/Relocations/aarch64/SingButtonHarness.yml</key>+ <data>+ zPwCbQ1Q8Cu0fQ7am6E08K9pOYw=+ </data>+ </dict>+ <key>files2</key>+ <dict>+ <key>SingButtonHarness.dSYM/Contents/Info.plist</key>+ <dict>+ <key>hash2</key>+ <data>+ UrVyO4JExqAB1o9xILnb6vN8sI5P0WhP4mZHf7g7XKs=+ </data>+ </dict>+ <key>SingButtonHarness.dSYM/Contents/Resources/DWARF/SingButtonHarness</key>+ <dict>+ <key>hash2</key>+ <data>+ F9nIh1L1RoVIg79oF0HZGUU6dYp0PPy5Evji4/OkZFw=+ </data>+ </dict>+ <key>SingButtonHarness.dSYM/Contents/Resources/Relocations/aarch64/SingButtonHarness.yml</key>+ <dict>+ <key>hash2</key>+ <data>+ P6oZ4BvqY2OydbkB6x1o5/1kWrBDE0QFUVUru3wYoHY=+ </data>+ </dict>+ </dict>+ <key>rules</key>+ <dict>+ <key>^.*</key>+ <true/>+ <key>^.*\.lproj/</key>+ <dict>+ <key>optional</key>+ <true/>+ <key>weight</key>+ <real>1000</real>+ </dict>+ <key>^.*\.lproj/locversion.plist$</key>+ <dict>+ <key>omit</key>+ <true/>+ <key>weight</key>+ <real>1100</real>+ </dict>+ <key>^Base\.lproj/</key>+ <dict>+ <key>weight</key>+ <real>1010</real>+ </dict>+ <key>^version.plist$</key>+ <true/>+ </dict>+ <key>rules2</key>+ <dict>+ <key>.*\.dSYM($|/)</key>+ <dict>+ <key>weight</key>+ <real>11</real>+ </dict>+ <key>^(.*/)?\.DS_Store$</key>+ <dict>+ <key>omit</key>+ <true/>+ <key>weight</key>+ <real>2000</real>+ </dict>+ <key>^.*</key>+ <true/>+ <key>^.*\.lproj/</key>+ <dict>+ <key>optional</key>+ <true/>+ <key>weight</key>+ <real>1000</real>+ </dict>+ <key>^.*\.lproj/locversion.plist$</key>+ <dict>+ <key>omit</key>+ <true/>+ <key>weight</key>+ <real>1100</real>+ </dict>+ <key>^Base\.lproj/</key>+ <dict>+ <key>weight</key>+ <real>1010</real>+ </dict>+ <key>^Info\.plist$</key>+ <dict>+ <key>omit</key>+ <true/>+ <key>weight</key>+ <real>20</real>+ </dict>+ <key>^PkgInfo$</key>+ <dict>+ <key>omit</key>+ <true/>+ <key>weight</key>+ <real>20</real>+ </dict>+ <key>^embedded\.provisionprofile$</key>+ <dict>+ <key>weight</key>+ <real>20</real>+ </dict>+ <key>^version\.plist$</key>+ <dict>+ <key>weight</key>+ <real>20</real>+ </dict>+ </dict>+</dict>+</plist>diff --git a/harness/sing/button/main.m b/harness/sing/button/main.mnew file mode 100644index 00000000..c2264ff4--- /dev/null+++ b/harness/sing/button/main.m@@ -0,0 +1,54 @@+// Sing's mic button (Redesigned/Lyrics/SGRSingButton.m) on a field like the player's lyrics, in the state the+// launch line asks for (sing-stubs.m), its slider opened with -panel 1, for screenshots in the simulator.+//+// ./build.sh && xcrun simctl install <udid> build/SingButtonHarness.app+// xcrun simctl launch <udid> com.vojta.singbuttonharness -state 7 -panel 1 -level 0.4+// xcrun simctl io <udid> screenshot shot.png+#import <UIKit/UIKit.h>+#import "Redesigned/Lyrics/SGRSingButton.h"++UIColor *SGRAccentColor(void) { return nil; }++UIViewController *SGTopController(void) {+ UIWindowScene *scene = (UIWindowScene *)UIApplication.sharedApplication.connectedScenes.anyObject;+ UIViewController *top = scene.windows.firstObject.rootViewController;+ while (top.presentedViewController) top = top.presentedViewController;+ return top;+}++@interface SGHarnessDelegate : UIResponder <UIApplicationDelegate>+@property (nonatomic, strong) UIWindow *window;+@end++@implementation SGHarnessDelegate++- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)options {+ self.window = [[UIWindow alloc] initWithFrame:UIScreen.mainScreen.bounds];+ UIViewController *root = [UIViewController new];+ root.view.backgroundColor = [UIColor colorWithRed:0.16 green:0.1 blue:0.22 alpha:1];+ CGSize size = UIScreen.mainScreen.bounds.size;+ UILabel *line = [[UILabel alloc] initWithFrame:CGRectMake(24, size.height * 0.3, size.width - 48, 80)];+ line.text = @"And the voice should come out clean";+ line.numberOfLines = 2;+ line.font = [UIFont systemFontOfSize:30 weight:UIFontWeightBold];+ line.textColor = UIColor.whiteColor;+ [root.view addSubview:line];+ SGRSingButton *button = [[SGRSingButton alloc] initWithFrame:CGRectMake(size.width - 24 - 44, size.height - 44 - 12 - 120, 44, 44)];+ [root.view addSubview:button];+ self.window.rootViewController = root;+ [self.window makeKeyAndVisible];+ if ([NSUserDefaults.standardUserDefaults boolForKey:@"panel"]) {+ dispatch_after(dispatch_time(DISPATCH_TIME_NOW, NSEC_PER_SEC / 2), dispatch_get_main_queue(), ^{+ [button performSelector:@selector(showPanel)];+ });+ }+ return YES;+}++@end++int main(int argc, char *argv[]) {+ @autoreleasepool {+ return UIApplicationMain(argc, argv, nil, NSStringFromClass(SGHarnessDelegate.class));+ }+}diff --git a/harness/sing/button/sing-stubs.m b/harness/sing/button/sing-stubs.mnew file mode 100644index 00000000..f1fbbda6--- /dev/null+++ b/harness/sing/button/sing-stubs.m@@ -0,0 +1,51 @@+// Shared/Sing's calls as the mic button makes them, answering from the launch line instead of the model and the+// audio: -state is an SGSingState (Sing.h), -progress the download's fraction. The lyrics harness links it too,+// since SGRKaraokeView puts the mic on the lyrics.+#import <UIKit/UIKit.h>+#import "Shared/Sing/Sing.h"++NSString *const SGSingChangedNotification = @"SGSingChangedNotification";++static NSUserDefaults *defaults(void) {+ return NSUserDefaults.standardUserDefaults;+}++SGSingState SGSingCurrentState(void) {+ return [defaults() objectForKey:@"state"] ? [defaults() integerForKey:@"state"] : SGSingStateOff;+}++static NSArray<NSString *> *names(void) {+ return @[@"Unavailable", @"Needs its voice model", @"Downloading", @"Off", @"Preparing", @"Waiting for Spotify", @"Listening ahead",+ @"On", @"Too slow", @"Held, too hot", @"Failed"];+}++NSString *SGSingStatusText(void) {+ NSInteger state = SGSingCurrentState();+ return state >= 0 && state < (NSInteger)names().count ? names()[state] : @"";+}++NSString *SGSingStatusDetail(void) { return SGSingStatusText(); }+NSString *SGSingMissing(void) { return @"Sing needs iOS 18, the first its voice model runs on."; }+BOOL SGSingOn(void) { return SGSingCurrentState() >= SGSingStatePreparing; }++void SGSetSingOn(BOOL on) {+ [defaults() setInteger:on ? SGSingStateSinging : SGSingStateOff forKey:@"state"];+ [NSNotificationCenter.defaultCenter postNotificationName:SGSingChangedNotification object:nil];+}++float SGSingLevel(void) { return [defaults() objectForKey:@"level"] ? [defaults() floatForKey:@"level"] : 0.15f; }+void SGSetSingLevel(float level) { [defaults() setFloat:level forKey:@"level"]; }++NSString *SGSingLevelText(double level) {+ if (level <= 0.001) return @"Gone";+ if (level < 0.999) return [NSString stringWithFormat:@"%.0f%%", level * 100];+ if (level <= 1.001) return @"As sung";+ if (level >= 1.999) return @"Vocals only";+ return [NSString stringWithFormat:@"Backing %.0f%%", (2 - level) * 100];+}++double SGSingModelProgress(void) { return [defaults() doubleForKey:@"progress"]; }+NSString *SGSingModelSizeText(void) { return @"489 MB"; }+NSString *SGSingModelError(void) { return nil; }+void SGSingDownloadModel(void) {}+void SGSingCancelModelDownload(void) {}diff --git a/harness/sing/download/build.sh b/harness/sing/download/build.shnew file mode 100755index 00000000..01755e9e--- /dev/null+++ b/harness/sing/download/build.sh@@ -0,0 +1,14 @@+#!/bin/sh+# Builds the download check (main.m) for the Mac, through Mac Catalyst since Sing.h imports UIKit: SGSingModel.m+# as the tweak compiles it, with shim/Core/SGCore.h standing in for the mod's core.+# ./build.sh && CFFIXED_USER_HOME="$(mktemp -d)" ../build/download 0.2+set -e+cd "$(dirname "$0")"+SRC=../../../tweak/Sources+mkdir -p ../build+xcrun clang -fobjc-arc -O1 -g -Wall -Werror -target arm64-apple-ios18.0-macabi \+ -isysroot "$(xcrun --sdk macosx --show-sdk-path)" \+ -iframework "$(xcrun --sdk macosx --show-sdk-path)/System/iOSSupport/System/Library/Frameworks" \+ -I shim -I "$SRC" main.m "$SRC/Shared/Sing/SGSingModel.m" \+ -framework Foundation -o ../build/download+echo "built ../build/download"diff --git a/harness/sing/download/main.m b/harness/sing/download/main.mnew file mode 100644index 00000000..85192428--- /dev/null+++ b/harness/sing/download/main.m@@ -0,0 +1,53 @@+// The voice model's download (SGSingModel.m) as the tweak compiles it, run on the Mac against Hugging Face: the+// four small files and part of the weights come in, the download is stopped, and the next one carries the weights+// on from where the stop left them (a 206 from the server) through to the move into place.+//+// ./build.sh && CFFIXED_USER_HOME=<empty dir> ../build/download [stop at, 0 to 1 of the weights]+#import <Foundation/Foundation.h>+#import "Shared/Sing/Sing.h"++static int failures;+#define CHECK(ok, ...) do { BOOL _ok = (ok); if (!_ok) failures++; printf("%s %s\n", _ok ? "PASS" : "FAIL", [NSString stringWithFormat:__VA_ARGS__].UTF8String); } while (0)++static void runUntil(BOOL (^done)(void), NSTimeInterval limit) {+ NSDate *end = [NSDate dateWithTimeIntervalSinceNow:limit];+ while (!done() && end.timeIntervalSinceNow > 0) CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0.25, false);+}++int main(int argc, char **argv) {+ @autoreleasepool {+ setvbuf(stdout, NULL, _IOLBF, 0);+ double stopAt = argc > 1 ? atof(argv[1]) : 0.2;+ NSURL *sing = [[[NSFileManager.defaultManager URLsForDirectory:NSApplicationSupportDirectory inDomains:NSUserDomainMask].firstObject+ URLByAppendingPathComponent:@"Vitrine"] URLByAppendingPathComponent:@"Sing"];+ printf("the model goes to %s\n", sing.path.UTF8String);+ CHECK(SGSingModelCurrentState() == SGSingModelMissing, @"no model to start with");++ // The small files are 0.14% of the bytes, so the progress is nearly the weights' own.+ SGSingDownloadModel();+ runUntil(^BOOL { return SGSingModelProgress() >= stopAt || SGSingModelCurrentState() != SGSingModelDownloading; }, 900);+ double stopped = SGSingModelProgress();+ CHECK(SGSingModelCurrentState() == SGSingModelDownloading, @"the download runs into the weights (%.0f%%)", stopped * 100);+ SGSingCancelModelDownload();+ NSURL *resume = [sing URLByAppendingPathComponent:@"download-4.resume"];+ runUntil(^BOOL { return [NSFileManager.defaultManager fileExistsAtPath:resume.path]; }, 10);+ CHECK([NSFileManager.defaultManager fileExistsAtPath:resume.path], @"the stop keeps the weights' resume data");+ CHECK(SGSingModelCurrentState() == SGSingModelMissing, @"stopped, the model is still missing");++ // Started over, the first bytes read well under 1%; carried on, the stop's share.+ SGSingDownloadModel();+ __block double first = -1;+ runUntil(^BOOL {+ if (first < 0 && SGSingModelProgress() > 0.005) first = SGSingModelProgress();+ return SGSingModelCurrentState() != SGSingModelDownloading;+ }, 1800);+ CHECK(first >= stopped - 0.01, @"the next download carries on from %.0f%%, not from the start", first * 100);+ CHECK(![NSFileManager.defaultManager fileExistsAtPath:resume.path], @"the resume data is used once");+ CHECK(SGSingModelCurrentState() == SGSingModelReady, @"the model is in place and checked (%@)", SGSingModelError() ?: @"no error");+ NSArray *inside = [NSFileManager.defaultManager contentsOfDirectoryAtPath:SGSingModelURL().path error:nil];+ CHECK(inside && ![[inside componentsJoinedByString:@" "] containsString:@"resume"], @"nothing but the model in its folder (%@)",+ [inside componentsJoinedByString:@", "]);+ printf("%s\n", failures ? "FAILED" : "all passed");+ return failures ? 1 : 0;+ }+}diff --git a/harness/sing/download/shim/Core/SGCore.h b/harness/sing/download/shim/Core/SGCore.hnew file mode 100644index 00000000..48818f9e--- /dev/null+++ b/harness/sing/download/shim/Core/SGCore.h@@ -0,0 +1,5 @@+// Stands in for tweak/Sources/Core/SGCore.h in the download check: SGSingModel.m needs only SGLog, here printed+// to stderr so the run shows it.+#import <Foundation/Foundation.h>++#define SGLog(fmt, ...) fprintf(stderr, "[spotifyglass] %s\n", [NSString stringWithFormat:(fmt), ##__VA_ARGS__].UTF8String)diff --git a/harness/sing/main.m b/harness/sing/main.mnew file mode 100644index 00000000..ba7d4f08--- /dev/null+++ b/harness/sing/main.m@@ -0,0 +1,407 @@+// Runs Sing's separator and engine (tweak/Sources/Shared/Sing) on the Mac the way the tweak runs them. Each+// check prints a line.+//+// - the STFT: a sine's peak bin at the size torch.stft gives it (amplitude times the window's sum over two), and+// noise through the STFT and back unchanged;+// - the model: loaded from a separator.mlmodelc folder at the compute units asked for, the time a window takes,+// and a voice mixed over chords taken apart: the vocals it finds against the voice, beside the mix's own score;+// - the engine: Spotify's mixer stood in for by the mix, pulled through SGSingEngineRender in IO buffers on a+// thread that keeps real time, while the engine's worker separates ahead: the lead it builds, every frame+// out in order, the vocals level applied, a flush, the lead given back when Sing is switched off, and+// no allocation on the render thread.+//+// ./build.sh && build/sing <separator.mlmodelc> <voice> <out dir> [all|cpu|gpu|ane]+#import <Foundation/Foundation.h>+#import <Accelerate/Accelerate.h>+#import <AudioToolbox/AudioToolbox.h>+#import <CoreML/CoreML.h>+#import <pthread.h>+#import <stdatomic.h>+#import "Shared/Sing/SGSingSeparator.h"+#import "Shared/Sing/SGSingEngine.h"++static int sg_failures;+static NSString *sg_outDir;++#define CHECK(ok, ...) do { bool _ok = (ok); if (!_ok) sg_failures++; printf("%s ", _ok ? " ok " : "FAILED"); printf(__VA_ARGS__); printf("\n"); } while (0)++#pragma mark - audio++typedef struct {+ float *left, *right;+ size_t frames;+} Audio;++static Audio makeAudio(size_t frames) {+ return (Audio){calloc(frames, sizeof(float)), calloc(frames, sizeof(float)), frames};+}++static AudioStreamBasicDescription floatFormat(double rate, UInt32 channels, bool interleaved) {+ UInt32 bytes = interleaved ? 4 * channels : 4;+ return (AudioStreamBasicDescription){rate, kAudioFormatLinearPCM,+ kAudioFormatFlagsNativeFloatPacked | (interleaved ? 0 : kAudioFormatFlagIsNonInterleaved), bytes, 1, bytes, channels, 32, 0};+}++static Audio readAudio(NSString *path) {+ ExtAudioFileRef file;+ if (ExtAudioFileOpenURL((__bridge CFURLRef)[NSURL fileURLWithPath:path], &file)) {+ printf("cannot open %s\n", path.UTF8String);+ exit(1);+ }+ AudioStreamBasicDescription format = floatFormat(kSGSingRate, 2, false);+ ExtAudioFileSetProperty(file, kExtAudioFileProperty_ClientDataFormat, sizeof format, &format);+ Audio audio = makeAudio(kSGSingRate * 60);+ size_t done = 0;+ while (done < audio.frames) {+ UInt32 frames = (UInt32)MIN((size_t)8192, audio.frames - done);+ struct { AudioBufferList list; AudioBuffer second; } buffers = {{2, {{1, frames * 4, audio.left + done}}}, {1, frames * 4, audio.right + done}};+ if (ExtAudioFileRead(file, &frames, &buffers.list) || !frames) break;+ done += frames;+ }+ ExtAudioFileDispose(file);+ audio.frames = done;+ return audio;+}++static void writeWAV(NSString *name, Audio audio) {+ NSString *path = [sg_outDir stringByAppendingPathComponent:name];+ AudioStreamBasicDescription stored = floatFormat(kSGSingRate, 2, true);+ ExtAudioFileRef file;+ if (ExtAudioFileCreateWithURL((__bridge CFURLRef)[NSURL fileURLWithPath:path], kAudioFileWAVEType, &stored, NULL, kAudioFileFlags_EraseFile, &file)) return;+ AudioStreamBasicDescription client = floatFormat(kSGSingRate, 2, false);+ ExtAudioFileSetProperty(file, kExtAudioFileProperty_ClientDataFormat, sizeof client, &client);+ struct { AudioBufferList list; AudioBuffer second; } buffers = {{2, {{1, (UInt32)audio.frames * 4, audio.left}}}, {1, (UInt32)audio.frames * 4, audio.right}};+ ExtAudioFileWrite(file, (UInt32)audio.frames, &buffers.list);+ ExtAudioFileDispose(file);+}++// Signal to error in dB of `estimate` against `truth` over [from, to).+static double snr(Audio truth, Audio estimate, size_t from, size_t to) {+ double signal = 0, noise = 0;+ for (size_t i = from; i < to; i++) {+ double l = truth.left[i], r = truth.right[i];+ double el = estimate.left[i] - l, er = estimate.right[i] - r;+ signal += l * l + r * r;+ noise += el * el + er * er;+ }+ return 10 * log10(signal / fmax(noise, 1e-20));+}++// Block chords with a decay, a note every half second, in a fifth of the stereo field each side.+static Audio chords(size_t frames) {+ Audio audio = makeAudio(frames);+ static const double roots[] = {130.81, 174.61, 196.00, 110.00};+ size_t beat = kSGSingRate / 2;+ for (size_t i = 0; i < frames; i++) {+ size_t bar = i / (beat * 4), within = i % beat;+ double root = roots[bar % 4], env = exp(-3.0 * within / beat), t = (double)i / kSGSingRate;+ double l = 0, r = 0;+ static const double ratios[] = {1, 1.26, 1.5, 2};+ for (int n = 0; n < 4; n++) {+ double f = root * ratios[n];+ double tone = sin(2 * M_PI * f * t) + 0.4 * sin(4 * M_PI * f * t) + 0.2 * sin(6 * M_PI * f * t);+ l += tone * (n % 2 ? 0.6 : 1.0);+ r += tone * (n % 2 ? 1.0 : 0.6);+ }+ audio.left[i] = (float)(0.06 * env * l);+ audio.right[i] = (float)(0.06 * env * r);+ }+ return audio;+}++#pragma mark - the STFT++static void checkSTFT(void) {+ SGSingSeparator *stft = [[SGSingSeparator alloc] initWithModel:nil];+ float *spectrum = calloc(kSGSingSpectrumFloats, sizeof(float));+ Audio in = makeAudio(kSGSingWindowFrames), out = makeAudio(kSGSingWindowFrames);+ int bin = 100;+ double frequency = bin * (double)kSGSingRate / kSGSingFFT;+ for (int i = 0; i < kSGSingWindowFrames; i++) {+ in.left[i] = (float)(0.5 * cos(2 * M_PI * frequency * i / kSGSingRate));+ in.right[i] = (float)(0.25 * cos(2 * M_PI * frequency * i / kSGSingRate));+ }+ [stft analyzeLeft:in.left right:in.right into:spectrum];+ int frame = 100;+ const float *left = spectrum + ((size_t)(2 * bin) * kSGSingSTFTFrames + frame) * 2;+ const float *right = spectrum + ((size_t)(2 * bin + 1) * kSGSingSTFTFrames + frame) * 2;+ double magnitude = hypot(left[0], left[1]), rightMagnitude = hypot(right[0], right[1]);+ // The periodic Hann window of 2048 sums to 1024, so a cosine of amplitude A peaks at A * 1024 / 2.+ CHECK(fabs(magnitude - 256) < 1 && fabs(rightMagnitude - 128) < 0.5, "a 0.5 cosine on bin %d peaks at %.2f (256 expected), 0.25 at %.2f in the right channel's slot",+ bin, magnitude, rightMagnitude);+ srand48(7);+ for (int i = 0; i < kSGSingWindowFrames; i++) {+ in.left[i] = (float)(drand48() * 2 - 1) * 0.5f;+ in.right[i] = (float)(0.3 * sin(i * 0.01) + (drand48() - 0.5) * 0.1);+ }+ [stft analyzeLeft:in.left right:in.right into:spectrum];+ [stft synthesize:spectrum left:out.left right:out.right];+ float worst = 0;+ for (int i = 0; i < kSGSingWindowFrames; i++) worst = fmaxf(worst, fmaxf(fabsf(out.left[i] - in.left[i]), fabsf(out.right[i] - in.right[i])));+ CHECK(worst < 1e-5, "noise through the STFT and back, edges included, within %.2g", worst);+ free(spectrum);+}++#pragma mark - the model++static MLComputeUnits unitsNamed(NSString *name) {+ if ([name isEqualToString:@"cpu"]) return MLComputeUnitsCPUOnly;+ if ([name isEqualToString:@"gpu"]) return MLComputeUnitsCPUAndGPU;+ if ([name isEqualToString:@"ane"]) return MLComputeUnitsCPUAndNeuralEngine;+ return MLComputeUnitsAll;+}++static double now(void) {+ return CFAbsoluteTimeGetCurrent();+}++// The offline pass the engine makes as it plays: windows a hop apart, their overlaps crossfaded.+static Audio separateOffline(SGSingSeparator *separator, Audio mix, double *slowest, double *average) {+ Audio vocals = makeAudio(mix.frames);+ Audio window = makeAudio(kSGSingWindowFrames);+ int count = 0;+ double total = 0;+ *slowest = 0;+ size_t previousEnd = 0;+ for (size_t start = 0; start + kSGSingWindowFrames <= mix.frames; start += kSGSingEngineHop) {+ double began = now();+ NSError *error;+ if (![separator separateLeft:mix.left + start right:mix.right + start vocalsLeft:window.left vocalsRight:window.right error:&error]) {+ printf("the model failed: %s\n", error.localizedDescription.UTF8String);+ exit(1);+ }+ double took = now() - began;+ total += took;+ *slowest = fmax(*slowest, took);+ count++;+ for (size_t i = 0; i < kSGSingWindowFrames; i++) {+ size_t p = start + i;+ float a = p < previousEnd ? (float)(p - start) / (float)(previousEnd - start) : 1;+ vocals.left[p] = vocals.left[p] * (1 - a) + window.left[i] * a;+ vocals.right[p] = vocals.right[p] * (1 - a) + window.right[i] * a;+ }+ previousEnd = start + kSGSingWindowFrames;+ }+ *average = total / count;+ return vocals;+}++static MLModel *loadModel(NSString *path, NSString *units) {+ MLModelConfiguration *configuration = [MLModelConfiguration new];+ configuration.computeUnits = unitsNamed(units);+ double began = now();+ NSError *error;+ MLModel *model = [MLModel modelWithContentsOfURL:[NSURL fileURLWithPath:path] configuration:configuration error:&error];+ CHECK(model != nil, "the model loads (%s compute units) in %.1f s%s%s", units.UTF8String, now() - began, error ? ": " : "",+ error.localizedDescription.UTF8String ?: "");+ if (!model) exit(1);+ MLFeatureDescription *input = model.modelDescription.inputDescriptionsByName[@"spectrum"];+ MLFeatureDescription *output = model.modelDescription.outputDescriptionsByName[@"vocals_spectrum"];+ CHECK([input.multiArrayConstraint.shape isEqualToArray:(@[@1, @2050, @201, @2])] && [output.multiArrayConstraint.shape isEqualToArray:(@[@1, @2050, @201, @2])],+ "its spectrum and vocals_spectrum are [1, 2050, 201, 2]");+ return model;+}++// Which device Core ML means to run each operation on, counted, where the OS can tell.+static void describePlan(NSString *path, NSString *units) {+ if (@available(macOS 14.4, *)) {+ MLModelConfiguration *configuration = [MLModelConfiguration new];+ configuration.computeUnits = unitsNamed(units);+ dispatch_semaphore_t done = dispatch_semaphore_create(0);+ [MLComputePlan loadContentsOfURL:[NSURL fileURLWithPath:path] configuration:configuration completionHandler:^(MLComputePlan *plan, NSError *error) {+ NSMutableDictionary<NSString *, NSNumber *> *counts = [NSMutableDictionary dictionary];+ MLModelStructureProgramFunction *main = plan.modelStructure.program.functions[@"main"];+ for (MLModelStructureProgramOperation *operation in main.block.operations) {+ MLComputePlanDeviceUsage *usage = [plan computeDeviceUsageForMLProgramOperation:operation];+ if (!usage) continue;+ id<MLComputeDeviceProtocol> device = usage.preferredComputeDevice;+ NSString *name = [device isKindOfClass:MLNeuralEngineComputeDevice.class] ? @"Neural Engine"+ : [device isKindOfClass:MLGPUComputeDevice.class] ? @"GPU" : @"CPU";+ counts[name] = @(counts[name].intValue + 1);+ }+ printf(" info the plan at %s: %s\n", units.UTF8String, counts.description.UTF8String);+ dispatch_semaphore_signal(done);+ }];+ dispatch_semaphore_wait(done, DISPATCH_TIME_FOREVER);+ }+}++#pragma mark - the engine++typedef struct {+ Audio source;+ size_t pulled;+} Source;++static OSStatus pullSource(void *context, UInt32 frames, float *left, float *right) {+ Source *source = context;+ for (UInt32 i = 0; i < frames; i++) {+ size_t at = source->pulled + i;+ left[i] = at < source->source.frames ? source->source.left[at] : 0;+ right[i] = at < source->source.frames ? source->source.right[at] : 0;+ }+ source->pulled += frames;+ return noErr;+}++typedef void(malloc_logger_t)(uint32_t type, uintptr_t arg1, uintptr_t arg2, uintptr_t arg3, uintptr_t result, uint32_t skip);+extern malloc_logger_t *malloc_logger;+static pthread_t sg_renderThread;+static atomic_bool sg_inRender;+static atomic_uint sg_renderAllocations;++static void countAllocation(uint32_t type, uintptr_t a, uintptr_t b, uintptr_t c, uintptr_t result, uint32_t skip) {+ if (atomic_load_explicit(&sg_inRender, memory_order_relaxed) && pthread_equal(pthread_self(), sg_renderThread)) {+ atomic_fetch_add_explicit(&sg_renderAllocations, 1, memory_order_relaxed);+ }+}++// Plays `mix` through the engine in real time, in buffers of `slice`; `script` is called once a buffer with the+// seconds played, for the test to switch things.+static Audio play(SGSingEngine *engine, Audio mix, Source *source, UInt32 slice, double seconds, void (^script)(double played)) {+ Audio out = makeAudio((size_t)(seconds * kSGSingRate));+ sg_renderThread = pthread_self();+ double began = now();+ for (size_t done = 0; done + slice <= out.frames; done += slice) {+ script((double)done / kSGSingRate);+ atomic_store(&sg_inRender, true);+ SGSingEngineRender(engine, slice, out.left + done, out.right + done, pullSource, source);+ atomic_store(&sg_inRender, false);+ double due = began + (double)(done + slice) / kSGSingRate;+ double wait = due - now();+ if (wait > 0) usleep((useconds_t)(wait * 1e6));+ }+ return out;+}++static void checkEngine(SGSingSeparator *separator, Audio mix, Audio voice, Audio offlineVocals) {+ SGSingEngine *engine = SGSingEngineCreate();+ SGSingEngineSetSeparator(engine, separator);+ SGSingEngineSetLevel(engine, 0);+ Source source = {mix, 0};+ // On from the first buffer, so the engine's windows fall where the offline pass's do; off at 17 s.+ SGSingEngineSetOn(engine, true);+ __block double leadAtOn = 0, leadAtOff = 0, leadMax = 0;+ malloc_logger = countAllocation;+ Audio out = play(engine, mix, &source, 1024, fmin(mix.frames / (double)kSGSingRate, 28), ^(double played) {+ double lead = SGSingEngineLead(engine);+ leadMax = fmax(leadMax, lead);+ if (played >= 12 && leadAtOn == 0) leadAtOn = lead;+ if (played >= 17 && SGSingEngineOn(engine)) SGSingEngineSetOn(engine, false);+ if (played >= 27) leadAtOff = lead;+ });+ malloc_logger = NULL;+ CHECK(atomic_load(&sg_renderAllocations) == 0, "no allocation on the render thread (%u)", atomic_load(&sg_renderAllocations));+ SGSingEngineStats stats = SGSingEngineReadStats(engine);+ printf(" info lead at 12 s %.2f s (at most %.2f s, target %.2f s), %llu windows at %.0f ms average, %llu frames played dry while on\n",+ leadAtOn, leadMax, stats.targetLead, stats.windows, stats.averageMS, stats.dryFrames);+ CHECK(leadAtOn > 2 && leadAtOn < 7, "with Sing on the engine pulls ahead of what plays (%.2f s)", leadAtOn);+ CHECK(leadAtOff < 0.05, "switched off it gives the lead back (%.3f s left)", leadAtOff);+ // What plays is the mixer's frames in order, from 6 s to 16 s the mix less the vocals the offline pass finds,+ // with the vocals at 0.+ size_t from = 6 * kSGSingRate, to = 16 * kSGSingRate;+ Audio expected = makeAudio(out.frames), instrumental = makeAudio(out.frames);+ for (size_t i = 0; i < out.frames; i++) {+ expected.left[i] = mix.left[i] - offlineVocals.left[i];+ expected.right[i] = mix.right[i] - offlineVocals.right[i];+ instrumental.left[i] = mix.left[i] - voice.left[i];+ instrumental.right[i] = mix.right[i] - voice.right[i];+ }+ double first = 0;+ for (size_t i = 0; i < kSGSingRate; i++) first = fmax(first, fabs(out.left[i] - mix.left[i]) + fabs(out.right[i] - mix.right[i]));+ CHECK(first == 0, "while the lead fills the mix plays dry (%.2g)", first);+ double match = snr(expected, out, from, to);+ CHECK(match > 25, "then what plays is the mix less the vocals the offline pass finds (%.1f dB)", match);+ printf(" info the karaoke against the true instrumental: %.1f dB; the mix's own: %.1f dB\n", snr(instrumental, out, from, to), snr(instrumental, mix, from, to));+ double after = 0;+ size_t firstOff = 0;+ for (size_t i = 26 * kSGSingRate; i < out.frames / 1024 * 1024; i++) {+ double d = fabs(out.left[i] - mix.left[i]) + fabs(out.right[i] - mix.right[i]);+ if (d > 1e-6 && !firstOff) firstOff = i;+ after = fmax(after, d);+ }+ if (firstOff) printf(" info first difference at %.4f s: out %g, mix %g, mix a frame on %g\n", firstOff / (double)kSGSingRate, out.left[firstOff], mix.left[firstOff], mix.left[firstOff + 1]);+ CHECK(after < 1e-6, "off again, the mix plays on where it was (%.2g)", after);+ writeWAV(@"engine.wav", out);++ // The level: at 2 only the vocals play.+ SGSingEngine *loud = SGSingEngineCreate();+ SGSingEngineSetSeparator(loud, separator);+ SGSingEngineSetLevel(loud, 2);+ SGSingEngineSetOn(loud, true);+ Source third = {mix, 0};+ Audio alone = play(loud, mix, &third, 1024, 10, ^(double played) {});+ double vocalsOnly = snr(offlineVocals, alone, 6 * kSGSingRate, 9 * kSGSingRate);+ CHECK(vocalsOnly > 30, "at the top of the slider the vocals play alone (%.1f dB against the offline vocals)", vocalsOnly);+ SGSingEngineDestroy(loud);++ // A flush drops what was pulled ahead: the next frame out is the one the mixer hands over next.+ SGSingEngine *flushing = SGSingEngineCreate();+ SGSingEngineSetSeparator(flushing, separator);+ Source second = {mix, 0}, *secondSource = &second;+ __block size_t pulledAtFlush = 0;+ __block bool flushed = false;+ Audio afterFlush = play(flushing, mix, &second, 512, 8, ^(double played) {+ if (!SGSingEngineOn(flushing)) SGSingEngineSetOn(flushing, true);+ if (played >= 5 && !flushed) {+ flushed = true;+ pulledAtFlush = secondSource->pulled;+ SGSingEngineFlush(flushing);+ }+ });+ size_t at = (size_t)(5 * kSGSingRate / 512 + 1) * 512;+ CHECK(fabsf(afterFlush.left[at] - mix.left[pulledAtFlush]) < 1e-6 && fabsf(afterFlush.left[at + 100] - mix.left[pulledAtFlush + 100]) < 1e-6,+ "after a flush the mixer's next frame plays next (%.0f s of lead dropped)", (pulledAtFlush - (size_t)(5 * kSGSingRate)) / (double)kSGSingRate);+ SGSingEngineDestroy(flushing);+ SGSingEngineDestroy(engine);+}++int main(int argc, char **argv) {+ setvbuf(stdout, NULL, _IOLBF, 0);+ @autoreleasepool {+ if (argc < 4) {+ printf("usage: sing <separator.mlmodelc> <voice> <out dir> [all|cpu|gpu|ane]\n");+ return 2;+ }+ NSString *modelPath = @(argv[1]), *units = argc > 4 ? @(argv[4]) : @"all";+ sg_outDir = @(argv[3]);+ [NSFileManager.defaultManager createDirectoryAtPath:sg_outDir withIntermediateDirectories:YES attributes:nil error:nil];+ checkSTFT();++ MLModel *model = loadModel(modelPath, units);+ describePlan(modelPath, units);+ SGSingSeparator *separator = [[SGSingSeparator alloc] initWithModel:model];++ Audio voice = readAudio(@(argv[2]));+ size_t frames = (size_t)kSGSingRate * 30;+ frames = (frames - kSGSingWindowFrames) / kSGSingEngineHop * kSGSingEngineHop + kSGSingWindowFrames;+ Audio padded = makeAudio(frames), backing = chords(frames), mix = makeAudio(frames);+ for (size_t i = 0; i < frames; i++) {+ // The voice over and over, a second apart, from a second in.+ size_t from = i >= kSGSingRate ? (i - kSGSingRate) % (voice.frames + kSGSingRate) : voice.frames;+ padded.left[i] = padded.right[i] = from < voice.frames ? 0.5f * voice.left[from] : 0;+ mix.left[i] = padded.left[i] + backing.left[i];+ mix.right[i] = padded.right[i] + backing.right[i];+ }+ double slowest, average;+ Audio vocals = separateOffline(separator, mix, &slowest, &average);+ printf(" info a two second window takes %.0f ms on average, %.0f ms at most (the first one warms the model up)\n", average * 1000, slowest * 1000);+ size_t to = frames - kSGSingWindowFrames / 2;+ double found = snr(padded, vocals, kSGSingRate, to), baseline = snr(padded, mix, kSGSingRate, to);+ CHECK(found > baseline + 10, "the vocals it finds score %.1f dB against the voice, the mix itself %.1f dB", found, baseline);+ Audio accompaniment = makeAudio(frames);+ for (size_t i = 0; i < frames; i++) {+ accompaniment.left[i] = mix.left[i] - vocals.left[i];+ accompaniment.right[i] = mix.right[i] - vocals.right[i];+ }+ printf(" info the mix less them scores %.1f dB against the chords, the mix itself %.1f dB\n", snr(backing, accompaniment, kSGSingRate, to),+ snr(backing, mix, kSGSingRate, to));+ writeWAV(@"mix.wav", mix);+ writeWAV(@"vocals.wav", vocals);+ writeWAV(@"accompaniment.wav", accompaniment);++ checkEngine(separator, mix, padded, vocals);+ printf("%s\n", sg_failures ? "FAILED" : "all passed");+ return sg_failures ? 1 : 0;+ }+}diff --git a/harness/sing/sim/main.m b/harness/sing/sim/main.mnew file mode 100644index 00000000..5e260d58--- /dev/null+++ b/harness/sing/sim/main.m@@ -0,0 +1,401 @@+// Sing in the simulator, end to end: the voice model downloaded by SGSingModel.m from Hugging Face and checked,+// loaded by Sing.x, and Spotify's audio chain rebuilt with real Core Audio units (a converter fed by a render+// callback, a mixer and RemoteIO, wired with MakeConnection, slices of 4096) with SpeedPitch.x and Sing.x+// compiled in. The harness is the main executable, so its AudioUnitSetProperty and AudioOutputUnitStart go+// through the rebound import slots as Spotify's do, and Sing's stage runs in Speed and pitch's chain.+//+// The converter's callback stands in for Spotify's decoder: a voice (macOS's speech, bundled by build-sim.sh)+// over synthetic chords. A mock SPTPlayerState computes -position the way Spotify's does (positionAsOfTimestamp+// messaged, disassembly at 0x1057735ec), and a mock SPTEsperantoPlayer moves the decoder on seekTo:. A render+// notify on the RemoteIO unit records what plays. The script: Sing on at the vocals' 0, play, seek, Sing off;+// each check prints a line, and what played is scored against the chords alone.+//+// THEOS=$HOME/theos VOICE=speech.aiff ./build-sim.sh+// xcrun simctl install <udid> build/sim/SingHarness.app+// xcrun simctl launch --console-pty <udid> com.vojta.singharness+#import <UIKit/UIKit.h>+#import <AudioToolbox/AudioToolbox.h>+#import <AVFoundation/AVFoundation.h>+#import <stdatomic.h>+#import "Shared/Sing/Sing.h"++static const double kRate = 44100;+enum { kMixSeconds = 60, kRecordSeconds = 70 };++static int sg_failures;+// The vocals' checks need the model to keep up, which the simulator's Core ML (on the CPU) does not: there they+// are reported as skipped, with the time a window took.+static BOOL sg_modelKeepsUp = YES;+#define VOCALS_CHECK(ok, ...) do { if (sg_modelKeepsUp) CHECK(ok, __VA_ARGS__); else NSLog(@"[harness] skipped %@ (Sing is %@: the model is slower than the song here)", [NSString stringWithFormat:__VA_ARGS__], SGSingStatusText()); } while (0)+#define CHECK(ok, ...) do { bool _ok = (ok); if (!_ok) sg_failures++; NSLog(@"[harness] %@ %@", _ok ? @" ok " : @"FAILED", [NSString stringWithFormat:__VA_ARGS__]); } while (0)++#pragma mark - Spotify's player, as far as Sing reads it++// Shared/Player/PlayerState.x's, which the harness does without: no state is reported to Sing's track watcher.+NSString *SGURIString(id uri) {+ return [uri isKindOfClass:NSString.class] ? uri : nil;+}++void SGAddPlayerStateObserver(id observer) {}++id SGPlayerState(void) {+ return nil;+}++@interface SPTPlayerState : NSObject+@property (nonatomic) double positionAsOfTimestamp;+@property (nonatomic, strong) NSDate *timestamp;+@end++@implementation SPTPlayerState+- (double)playbackSpeed {+ return 1;+}+// As Spotify's: -1 kept, otherwise run on by the time since, never below 0.+- (double)position {+ double asOf = [self positionAsOfTimestamp];+ if (asOf == -1) return asOf;+ return MAX(0, asOf - self.timestamp.timeIntervalSinceNow * [self playbackSpeed]);+}+@end++#pragma mark - the decoder++static float *sg_left, *sg_right, *sg_chordsLeft, *sg_chordsRight;+static size_t sg_mixFrames;+static atomic_uint_fast64_t sg_decoded; // frames handed over+static atomic_uint_fast64_t sg_readAt; // where in the mix the next one comes from++static OSStatus decoder(void *refCon, AudioUnitRenderActionFlags *flags, const AudioTimeStamp *timestamp, UInt32 bus,+ UInt32 frames, AudioBufferList *data) {+ uint64_t at = atomic_fetch_add(&sg_readAt, frames);+ atomic_fetch_add(&sg_decoded, frames);+ float *left = data->mBuffers[0].mData, *right = data->mNumberBuffers > 1 ? data->mBuffers[1].mData : left;+ for (UInt32 i = 0; i < frames; i++) {+ size_t index = (size_t)((at + i) % sg_mixFrames);+ left[i] = sg_left[index];+ right[i] = sg_right[index];+ }+ return noErr;+}++@interface SPTEsperantoPlayer : NSObject+@end++static id sg_seekResult;++// Each command returns an object of Spotify's, which the hook must hand back.+@implementation SPTEsperantoPlayer+- (id)seekTo:(double)position {+ atomic_store(&sg_readAt, (uint64_t)(position * kRate));+ return sg_seekResult;+}+- (id)seekTo:(double)position relative:(long long)relative { return sg_seekResult; }+- (id)seekTo:(double)position options:(id)options { return [self seekTo:position]; }+- (id)seekTo:(double)position relative:(long long)relative options:(id)options { return sg_seekResult; }+- (id)seekTo:(double)position relative:(long long)relative options:(id)options creatorTimestampPositionMs:(double)creator { return sg_seekResult; }+- (id)skipToNextTrackWithOptions:(id)options { return sg_seekResult; }+- (id)skipToPreviousTrackWithOptions:(id)options { return sg_seekResult; }+- (id)skipToNextTrackWithOptions:(id)options track:(id)track { return sg_seekResult; }+- (id)skipToPreviousTrackWithOptions:(id)options track:(id)track { return sg_seekResult; }+- (id)stop { return sg_seekResult; }+@end++#pragma mark - what plays++static float *sg_heardLeft, *sg_heardRight;+static size_t sg_heardCapacity;+static atomic_uint_fast64_t sg_heard;+static double sg_hardwareRate;++static OSStatus record(void *refCon, AudioUnitRenderActionFlags *flags, const AudioTimeStamp *timestamp, UInt32 bus,+ UInt32 frames, AudioBufferList *data) {+ if (!(*flags & kAudioUnitRenderAction_PostRender) || bus != 0 || data->mNumberBuffers < 2) return noErr;+ uint64_t at = atomic_load(&sg_heard);+ for (UInt32 i = 0; i < frames && at + i < sg_heardCapacity; i++) {+ sg_heardLeft[at + i] = ((float *)data->mBuffers[0].mData)[i];+ sg_heardRight[at + i] = ((float *)data->mBuffers[1].mData)[i];+ }+ atomic_store(&sg_heard, at + frames);+ return noErr;+}++#pragma mark - the mix++static void makeMix(void) {+ sg_mixFrames = (size_t)(kMixSeconds * kRate);+ sg_left = calloc(sg_mixFrames, sizeof(float));+ sg_right = calloc(sg_mixFrames, sizeof(float));+ sg_chordsLeft = calloc(sg_mixFrames, sizeof(float));+ sg_chordsRight = calloc(sg_mixFrames, sizeof(float));+ float *voice = calloc(sg_mixFrames, sizeof(float));+ size_t voiceFrames = 0;+ NSURL *url = [NSBundle.mainBundle URLForResource:@"voice" withExtension:@"aiff"];+ ExtAudioFileRef file;+ if (url && !ExtAudioFileOpenURL((__bridge CFURLRef)url, &file)) {+ AudioStreamBasicDescription format = {kRate, kAudioFormatLinearPCM, kAudioFormatFlagsNativeFloatPacked, 4, 1, 4, 1, 32, 0};+ ExtAudioFileSetProperty(file, kExtAudioFileProperty_ClientDataFormat, sizeof format, &format);+ while (voiceFrames < sg_mixFrames) {+ UInt32 frames = (UInt32)MIN((size_t)8192, sg_mixFrames - voiceFrames);+ AudioBufferList list = {1, {{1, frames * 4, voice + voiceFrames}}};+ if (ExtAudioFileRead(file, &frames, &list) || !frames) break;+ voiceFrames += frames;+ }+ ExtAudioFileDispose(file);+ }+ NSLog(@"[harness] the voice is %.1f s long", voiceFrames / kRate);+ static const double roots[] = {130.81, 174.61, 196.00, 110.00}, ratios[] = {1, 1.26, 1.5, 2};+ size_t beat = (size_t)kRate / 2;+ for (size_t i = 0; i < sg_mixFrames; i++) {+ size_t bar = i / (beat * 4), within = i % beat;+ double root = roots[bar % 4], env = exp(-3.0 * within / beat), t = i / kRate, l = 0, r = 0;+ for (int n = 0; n < 4; n++) {+ double f = root * ratios[n], tone = sin(2 * M_PI * f * t) + 0.4 * sin(4 * M_PI * f * t) + 0.2 * sin(6 * M_PI * f * t);+ l += tone * (n % 2 ? 0.6 : 1.0);+ r += tone * (n % 2 ? 1.0 : 0.6);+ }+ sg_chordsLeft[i] = (float)(0.06 * env * l);+ sg_chordsRight[i] = (float)(0.06 * env * r);+ // The voice over and over, a second apart, from a second in.+ size_t from = i >= (size_t)kRate && voiceFrames ? (i - (size_t)kRate) % (voiceFrames + (size_t)kRate) : SIZE_MAX;+ float v = from < voiceFrames ? 0.5f * voice[from] : 0;+ sg_left[i] = sg_chordsLeft[i] + v;+ sg_right[i] = sg_chordsRight[i] + v;+ }+ free(voice);+}++// What played, at the hardware's rate, brought to 44.1 kHz.+static NSArray<NSData *> *heardAt44k(size_t *frames) {+ size_t count = MIN((size_t)atomic_load(&sg_heard), sg_heardCapacity);+ AVAudioFormat *from = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32 sampleRate:sg_hardwareRate channels:2 interleaved:NO];+ AVAudioFormat *to = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32 sampleRate:kRate channels:2 interleaved:NO];+ AVAudioPCMBuffer *input = [[AVAudioPCMBuffer alloc] initWithPCMFormat:from frameCapacity:(AVAudioFrameCount)count];+ memcpy(input.floatChannelData[0], sg_heardLeft, count * sizeof(float));+ memcpy(input.floatChannelData[1], sg_heardRight, count * sizeof(float));+ input.frameLength = (AVAudioFrameCount)count;+ AVAudioConverter *converter = [[AVAudioConverter alloc] initFromFormat:from toFormat:to];+ converter.sampleRateConverterQuality = AVAudioQualityMax;+ AVAudioPCMBuffer *output = [[AVAudioPCMBuffer alloc] initWithPCMFormat:to frameCapacity:(AVAudioFrameCount)(count * kRate / sg_hardwareRate + 4096)];+ __block BOOL given = NO;+ [converter convertToBuffer:output error:nil withInputFromBlock:^AVAudioBuffer *(AVAudioPacketCount packets, AVAudioConverterInputStatus *status) {+ if (given) {+ *status = AVAudioConverterInputStatus_EndOfStream;+ return nil;+ }+ given = YES;+ *status = AVAudioConverterInputStatus_HaveData;+ return input;+ }];+ *frames = output.frameLength;+ return @[[NSData dataWithBytes:output.floatChannelData[0] length:output.frameLength * sizeof(float)],+ [NSData dataWithBytes:output.floatChannelData[1] length:output.frameLength * sizeof(float)]];+}++static double snr(const float *truthL, const float *truthR, const float *left, const float *right, size_t from, size_t to) {+ double signal = 0, noise = 0;+ for (size_t i = from; i < to; i++) {+ signal += truthL[i] * truthL[i] + truthR[i] * truthR[i];+ double l = left[i] - truthL[i], r = right[i] - truthR[i];+ noise += l * l + r * r;+ }+ return 10 * log10(signal / fmax(noise, 1e-20));+}++#pragma mark - the app++static AudioUnit make(OSType type, OSType subType) {+ AudioComponentDescription description = {type, subType, kAudioUnitManufacturer_Apple, 0, 0};+ AudioUnit unit = NULL;+ AudioComponentInstanceNew(AudioComponentFindNext(NULL, &description), &unit);+ return unit;+}++static void check(OSStatus status, const char *what) {+ if (status) NSLog(@"[harness] %s failed: %d", what, (int)status);+}++@interface SGHarnessDelegate : UIResponder <UIApplicationDelegate>+@property (nonatomic, strong) UIWindow *window;+@end++@implementation SGHarnessDelegate {+ AudioUnit _output;+ CFTimeInterval _startedAt;+ uint64_t _lastDecoded;+ CFTimeInterval _lastAt;+ SPTPlayerState *_state;+ double _seekLead;+}++- (void)startChain {+ [AVAudioSession.sharedInstance setCategory:AVAudioSessionCategoryPlayback error:nil];+ [AVAudioSession.sharedInstance setActive:YES error:nil];+ AudioUnit converter = make(kAudioUnitType_FormatConverter, kAudioUnitSubType_AUConverter);+ AudioUnit mixer = make(kAudioUnitType_Mixer, kAudioUnitSubType_MultiChannelMixer);+ _output = make(kAudioUnitType_Output, kAudioUnitSubType_RemoteIO);+ AURenderCallbackStruct callback = {decoder, NULL};+ check(AudioUnitSetProperty(converter, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &callback, sizeof callback), "callback");+ AudioUnitConnection toMixer = {converter, 0, 0}, toOutput = {mixer, 0, 0};+ check(AudioUnitSetProperty(mixer, kAudioUnitProperty_MakeConnection, kAudioUnitScope_Input, 0, &toMixer, sizeof toMixer), "connect mixer");+ check(AudioUnitSetProperty(_output, kAudioUnitProperty_MakeConnection, kAudioUnitScope_Input, 0, &toOutput, sizeof toOutput), "connect output");+ AudioStreamBasicDescription format = {kRate, kAudioFormatLinearPCM, kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved, 4, 1, 4, 2, 32, 0};+ check(AudioUnitSetProperty(converter, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof format), "converter in");+ check(AudioUnitSetProperty(converter, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof format), "converter out");+ check(AudioUnitSetProperty(mixer, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof format), "mixer in");+ check(AudioUnitSetProperty(mixer, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof format), "mixer out");+ check(AudioUnitSetProperty(_output, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof format), "output in");+ UInt32 slice = 4096;+ AudioUnit units[] = {converter, mixer, _output};+ for (int i = 0; i < 3; i++) check(AudioUnitSetProperty(units[i], kAudioUnitProperty_MaximumFramesPerSlice, kAudioUnitScope_Global, 0, &slice, sizeof slice), "slice");+ for (int i = 0; i < 3; i++) check(AudioUnitInitialize(units[i]), "initialize");+ AudioStreamBasicDescription hardware = {0};+ UInt32 size = sizeof hardware;+ AudioUnitGetProperty(_output, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &hardware, &size);+ sg_hardwareRate = hardware.mSampleRate;+ sg_heardCapacity = (size_t)(kRecordSeconds * sg_hardwareRate);+ sg_heardLeft = calloc(sg_heardCapacity, sizeof(float));+ sg_heardRight = calloc(sg_heardCapacity, sizeof(float));+ check(AudioOutputUnitStart(_output), "start");+ check(AudioUnitAddRenderNotify(_output, record, NULL), "record");+ _startedAt = CACurrentMediaTime();+ _lastAt = _startedAt;+ NSLog(@"[harness] the chain runs, the hardware at %.0f Hz", sg_hardwareRate);+}++- (void)after:(double)seconds do:(void (^)(void))block {+ dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(seconds * NSEC_PER_SEC)), dispatch_get_main_queue(), block);+}++// The player reporting what the decoder has handed over, as Spotify's does now and then.+- (void)playerReports {+ _state = [SPTPlayerState new];+ _state.positionAsOfTimestamp = atomic_load(&sg_readAt) / kRate;+ _state.timestamp = [NSDate date];+}++// What is heard now, in seconds of the mix: what the output played, less what a seek skipped.+- (double)heardContent {+ return atomic_load(&sg_heard) / sg_hardwareRate;+}++- (void)report {+ CFTimeInterval now = CACurrentMediaTime();+ uint64_t decoded = atomic_load(&sg_decoded);+ [self playerReports];+ NSLog(@"[harness] %5.1f s %-22@ decoder %.2fx position %.2f s, Spotify's own %.2f s (%.2f s ahead of what plays)",+ now - _startedAt, SGSingStatusText(), (decoded - _lastDecoded) / kRate / (now - _lastAt), _state.position,+ atomic_load(&sg_readAt) / kRate, atomic_load(&sg_readAt) / kRate - _state.position);+ _lastDecoded = decoded;+ _lastAt = now;+}++- (void)waitFor:(BOOL (^)(void))condition every:(double)seconds then:(void (^)(void))then {+ if (condition()) {+ then();+ return;+ }+ [self after:seconds do:^{ [self waitFor:condition every:seconds then:then]; }];+}++- (void)play {+ // Played straight through until the first seek: the position Spotify shows against what is heard.+ for (int s = 1; s <= 46; s++) [self after:s do:^{ [self report]; }];+ [self after:14 do:^{+ double heard = [self heardContent], position = self->_state.position;+ [self playerReports];+ position = self->_state.position;+ sg_modelKeepsUp = SGSingCurrentState() != SGSingStateBehind && SGSingCurrentState() != SGSingStateBuffering;+ VOCALS_CHECK(SGSingCurrentState() == SGSingStateSinging, @"Sing is on with the vocals down at 14 s (%@)", SGSingStatusText());+ CHECK(fabs(position - heard) < 0.15, @"the position Spotify shows, %.2f s, is what is heard, %.2f s, not what was decoded, %.2f s",+ position, heard, atomic_load(&sg_readAt) / kRate);+ }];+ [self after:20 do:^{+ double lead = atomic_load(&sg_readAt) / kRate - self->_state.position;+ self->_seekLead = lead;+ id result = [[SPTEsperantoPlayer new] seekTo:5];+ CHECK(result == sg_seekResult, @"a seek hands Spotify back the result of its seek");+ }];+ [self after:20.5 do:^{+ [self playerReports];+ double ahead = atomic_load(&sg_readAt) / kRate - self->_state.position;+ CHECK(ahead < 1.2, @"the seek dropped the %.1f s held ahead: %.2f s ahead half a second later", self->_seekLead, ahead);+ }];+ [self after:30 do:^{+ VOCALS_CHECK(SGSingCurrentState() == SGSingStateSinging, @"ten seconds after the seek the vocals are down again (%@)", SGSingStatusText());+ SGSetSingOn(NO);+ }];+ [self after:45 do:^{+ [self playerReports];+ double ahead = atomic_load(&sg_readAt) / kRate - self->_state.position;+ CHECK(ahead < 0.05 && SGSingCurrentState() == SGSingStateOff, @"switched off, the lead is given back (%.2f s) and Sing is %@", ahead, SGSingStatusText());+ [self score];+ NSLog(@"[harness] %@", sg_failures ? @"FAILED" : @"all passed");+ exit(sg_failures ? 1 : 0);+ }];+}++// What played from 6 s to 19 s against the chords alone, beside the mix's own score. Before the seek what plays+// is the mix in order, so the two line up but for the resampler's delay, found on the first second (dry).+- (void)score {+ size_t frames;+ NSArray<NSData *> *heard = heardAt44k(&frames);+ const float *left = heard[0].bytes, *right = heard[1].bytes;+ size_t from = (size_t)(6 * kRate), to = (size_t)(19 * kRate);+ if (frames < to + 1000) {+ CHECK(NO, @"enough was heard to score (%.1f s)", frames / kRate);+ return;+ }+ int best = 0;+ double bestScore = -INFINITY;+ for (int lag = -64; lag <= 64; lag++) {+ double dot = 0;+ for (size_t i = (size_t)kRate / 4; i < (size_t)kRate; i++) dot += left[i + lag] * sg_left[i];+ if (dot > bestScore) bestScore = dot, best = lag;+ }+ double dry = snr(sg_left, sg_right, left + best, right + best, (size_t)(kRate / 4), (size_t)kRate);+ NSLog(@"[harness] what plays lags the mix by %d frames after resampling; the first second matches the mix to %.1f dB", best, dry);+ double karaoke = snr(sg_chordsLeft, sg_chordsRight, left + best, right + best, from, to);+ double mix = snr(sg_chordsLeft, sg_chordsRight, sg_left, sg_right, from, to);+ VOCALS_CHECK(karaoke > mix + 15, @"from 6 s to 19 s what plays scores %.1f dB against the chords alone, the mix %.1f dB", karaoke, mix);+}++- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)options {+ self.window = [[UIWindow alloc] initWithFrame:UIScreen.mainScreen.bounds];+ self.window.rootViewController = [UIViewController new];+ sg_seekResult = [NSObject new];+ makeMix();+ NSLog(@"[harness] Sing: %@ (%@)", SGSingStatusText(), SGSingMissing() ?: @"nothing missing");+ CFTimeInterval began = CACurrentMediaTime();+ if (SGSingModelCurrentState() != SGSingModelReady) SGSingDownloadModel();+ __block CFTimeInterval logged = 0;+ [self waitFor:^BOOL {+ if (CACurrentMediaTime() - logged > 5) {+ logged = CACurrentMediaTime();+ NSLog(@"[harness] %@", SGSingStatusText());+ }+ return SGSingModelCurrentState() != SGSingModelDownloading;+ } every:0.5 then:^{+ CHECK(SGSingModelCurrentState() == SGSingModelReady, @"the voice model is on the phone, checked (%@, %.0f s)", SGSingModelError() ?: @"no error",+ CACurrentMediaTime() - began);+ if (SGSingModelCurrentState() != SGSingModelReady) exit(1);+ SGSetSingLevel(0);+ CFTimeInterval loading = CACurrentMediaTime();+ SGSetSingOn(YES);+ [self waitFor:^BOOL { return SGSingCurrentState() != SGSingStatePreparing; } every:0.25 then:^{+ CHECK(SGSingCurrentState() == SGSingStateWaiting, @"the model loads in %.1f s, and Sing waits for Spotify (%@)", CACurrentMediaTime() - loading,+ SGSingStatusText());+ [self startChain];+ [self play];+ }];+ }];+ return YES;+}++@end++int main(int argc, char *argv[]) {+ @autoreleasepool {+ return UIApplicationMain(argc, argv, nil, NSStringFromClass(SGHarnessDelegate.class));+ }+}diff --git a/tweak/Makefile b/tweak/Makefileindex 54266035..370cf163 100644--- a/tweak/Makefile+++ b/tweak/Makefile@@ -16,7 +16,7 @@ TWEAK_NAME := spotifyglass# is generated from the IPA by scripts/extract-flags.py.spotifyglass_FILES := $(shell find Sources -name '*.m' -o -name '*.x' -o -name '*.swift' | sort)spotifyglass_CFLAGS := -fobjc-arc -ISources -DSG_VERSION=\"$(SG_VERSION)\"-spotifyglass_FRAMEWORKS := UIKit QuartzCore UniformTypeIdentifiers MediaPlayer CoreImage AudioToolbox AVFoundation CoreHaptics Accelerate CoreMotion+spotifyglass_FRAMEWORKS := UIKit QuartzCore UniformTypeIdentifiers MediaPlayer CoreImage AudioToolbox AVFoundation CoreHaptics Accelerate CoreMotion CoreML# Runtime swizzling instead of Substrate so the dylib has no extra dependency inside the IPA.spotifyglass_LOGOS_DEFAULT_GENERATOR := internaldiff --git a/tweak/Sources/App/About/Licenses.m b/tweak/Sources/App/About/Licenses.mindex 9472dd0a..6ed46ede 100644--- a/tweak/Sources/App/About/Licenses.m+++ b/tweak/Sources/App/About/Licenses.m@@ -35,11 +35,19 @@ UIViewController *SGLicensesPage(void) {SGModRow *wdl = SGLinkRow(@"WDL", @"Liveprog's EEL2 · zlib License", @"https://github.com/justinfrankel/WDL");// Not shipped: the Headphones page downloads its results when asked, but they are AutoEq's work all the same.SGModRow *autoEq = SGLinkRow(@"AutoEq", @"Headphone corrections · MIT License", @"https://github.com/jaakkopasanen/AutoEq");+ // Sing's voice model is downloaded rather than built in, and its NOTICE travels with it here all the same.+ SGModRow *voice = SGLinkRow(@"Sing's voice model", @"Mel-Band RoFormer · MIT License", @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing");return [[SGModPage alloc] initWithTitle:@"Licenses" intro:@"The mod's own license, and the code from others it includes." sections:@[SGSection(nil, @[mod]),SGNotedSection(nil, @[bs2b], [@"Copyright (c) 2005 Boris Mikhaylov\n\n" stringByAppendingString:kMIT]),SGNotedSection(nil, @[wdl], [@"Copyright (C) 2004-2013 Cockos Incorporated\nCopyright (C) 1999-2003 Nullsoft, Inc.\n\n"stringByAppendingString:kZlib]),SGNotedSection(nil, @[autoEq], [@"Copyright (c) 2018-2022 Jaakko Pasanen\n\n" stringByAppendingString:kMIT]),+ SGNotedSection(nil, @[voice], [@"Mel-Band RoFormer vocal separation, by Ju-Chiang Wang, Wei-Tsung Lu and Minz Won.\n"+ @"Checkpoint: KimberleyJensen, https://huggingface.co/KimberleyJSN/melbandroformer (MIT).\n"+ @"Core ML conversion: john-rocky / mlboydaisuke, https://huggingface.co/mlboydaisuke/MelBandRoformer-Vocal-CoreAI (MIT).\n"+ @"Core ML export: Darkkos, https://huggingface.co/Darkkos/spoti-sing (MIT), mirrored unchanged.\n"+ @"Reference architecture: lucidrains/BS-RoFormer. Training code: ZFTurbo.\n\n"+ @"Copyright (c) 2023 Phil Wang\n\n" stringByAppendingString:kMIT]),] footer:nil];}diff --git a/tweak/Sources/App/ModSettings.x b/tweak/Sources/App/ModSettings.xindex 23a98731..f9704e65 100644--- a/tweak/Sources/App/ModSettings.x+++ b/tweak/Sources/App/ModSettings.x@@ -22,6 +22,7 @@#import "Shared/AudioEffects/AudioEffectsPage.h"#import "Shared/LiveActivity/LiveActivity.h"#import "Shared/ListeningStats/ListeningStats.h"+#import "Shared/Sing/Sing.h"#import "App/About/About.h"#import "Redesigned/Album/Album.h"#import "Pages.h"@@ -46,11 +47,17 @@ static UIViewController *modSettingsPage(void) {// The audio effects work on the sound, so both looks have them, with what they are doing beside the chevron.SGModRow *audioEffects = pageRow(@"Audio effects", @"slider.vertical.3", ^UIViewController *{ return SGDSPSettingsPage(); });audioEffects.value = ^NSString *{ return SGDSPSummary(); };+ // Sing works on the sound, so both looks have it, saying beside the chevron whether it is on or how far its+ // voice model has come.+ SGModRow *sing = pageRow(@"Sing", @"music.mic", ^UIViewController *{ return SGSingSettingsPage(); });+ sing.value = ^NSString *{ return SGSingSummary(); };// Home & Library holds only the native look's switches, so the redesign has no such page; the// Live Activity works under both, and only where ActivityKit's card does.NSMutableArray<SGModRow *> *parts = [NSMutableArray arrayWithArray:@[pageRow(@"Navbar", @"dock.rectangle", ^UIViewController *{ return SGNavbarPage(); }),pageRow(@"Player", @"play.circle", ^UIViewController *{ return SGPlayerSettingsPage(); }),+ pageRow(@"Lyrics", @"quote.bubble", ^UIViewController *{ return SGLyricsSettingsPage(); }),+ sing,audioEffects,]];if (@available(iOS 17.0, *)) {diff --git a/tweak/Sources/App/Pages.h b/tweak/Sources/App/Pages.hindex 09b6ae07..635853b9 100644--- a/tweak/Sources/App/Pages.h+++ b/tweak/Sources/App/Pages.h@@ -11,4 +11,5 @@ extern NSString *const SGRedesignedUIInfo;SGModSection *SGAppearanceSection(void); // the Appearance card at the top of Mod SettingsUIViewController *SGPlayerSettingsPage(void);+UIViewController *SGLyricsSettingsPage(void); // with Sing (Shared/Sing) on itUIViewController *SGNavbarPage(void); // the tab editor of whichever look is storeddiff --git a/tweak/Sources/App/Pages.m b/tweak/Sources/App/Pages.mindex 0e91c744..415177a1 100644--- a/tweak/Sources/App/Pages.m+++ b/tweak/Sources/App/Pages.m@@ -11,6 +11,7 @@#import "Shared/Lyrics/Lyrics.h"#import "Shared/LyricsMeanings/Meanings.h"#import "Shared/Player/PlayerSettings.h"+#import "Shared/Sing/Sing.h"#import "Native/Appearance/Appearance.h"#import "Native/Navbar/Navbar.h"#import "Native/NowPlayingBar/NowPlayingBar.h"@@ -83,7 +84,10 @@ static UIViewController *lyricsPage(void) {BOOL redesigned = SGRedesignedUIStored();NSMutableArray<SGModRow *> *more = [NSMutableArray arrayWithObject:SGLockScreenLyricsRow()];if (!redesigned) [more insertObject:SGGlassLyricsRow() atIndex:0];- NSMutableArray<SGModSection *> *sections = [NSMutableArray arrayWithObject:SGLyricsSourcesSection(redesigned)];+ SGModRow *sing = SGPageRow(@"Sing", ^UIViewController *{ return SGSingSettingsPage(); });+ sing.value = ^NSString *{ return SGSingSummary(); };+ NSMutableArray<SGModSection *> *sections = [NSMutableArray arrayWithObjects:SGSection(nil, @[SGWithSymbol(sing, @"music.mic")]),+ SGLyricsSourcesSection(redesigned), nil];if (redesigned) {[sections addObject:SGSection(@"Display", @[SGLyricsWordTimingRow(), SGRLyricsTextSizesRow(), SGLyricsTranslationLanguageRow(), SGLyricsMeaningsRow(),SGSwitchRow(@"Hide the controls", @"A few seconds after the last touch, the lyrics take the whole player", SGRKeyLyricsAutoHide),@@ -94,6 +98,11 @@ static UIViewController *lyricsPage(void) {return [[SGModPage alloc] initWithTitle:@"Lyrics" intro:SGRestartNote sections:sections footer:nil];}+// Lyrics has its own row on Mod Settings' main page, beside Sing.+UIViewController *SGLyricsSettingsPage(void) {+ return lyricsPage();+}+UIViewController *SGPlayerSettingsPage(void) {SGModRow *blocked = SGPageRow(@"Blocked artists", ^UIViewController *{ return SGArtistBlockSettingsPage(); });blocked.value = ^NSString *{@@ -104,7 +113,6 @@ UIViewController *SGPlayerSettingsPage(void) {NSMutableArray<SGModSection *> *sections = [NSMutableArray arrayWithObject:SGSection(nil, @[SGWithSymbol(SGPageRow(@"Gestures", ^UIViewController *{ return SGGesturesSettingsPage(); }), @"hand.tap"),SGWithSymbol(SGPageRow(@"AirPods gestures", ^UIViewController *{ return SGHeadGesturesSettingsPage(); }), @"airpods.pro"),- SGWithSymbol(SGPageRow(@"Lyrics", ^UIViewController *{ return lyricsPage(); }), @"quote.bubble"),SGWithSymbol(blocked, @"person.crop.circle.badge.xmark"),])];NSMutableArray<SGModRow *> *pages = [NSMutableArray array];diff --git a/tweak/Sources/Redesigned/Kit/SGRTokens.h b/tweak/Sources/Redesigned/Kit/SGRTokens.hindex 9c7d8fb5..3c0ba993 100644--- a/tweak/Sources/Redesigned/Kit/SGRTokens.h+++ b/tweak/Sources/Redesigned/Kit/SGRTokens.h@@ -50,7 +50,10 @@ typedef NS_ENUM(NSInteger, SGRMotion) {SGRMotionLayout, // a spring with no overshoot, for anything moving or resizingSGRMotionPress, // a spring with a little bounce, for press feedbackSGRMotionFade, // an ease in and out over SGRCrossfade, kept under Reduce Motion+ // A strong ease out over 0.2 s, for what comes back or opens at a touch: the eye is on it from the+ // first frame. Kept under Reduce Motion like a fade, so only alpha, colour and a glass effect go in it.+ SGRMotionRespond,};-// Runs `animations` with the motion's timing, or at once under Reduce Motion (except a fade), and+// Runs `animations` with the motion's timing, or at once under Reduce Motion (except a fade or a response), and// always calls `completion`. Only transform, alpha and colour belong in it while a page scrolls.void SGRAnimate(SGRMotion motion, void (^animations)(void), void (^completion)(BOOL finished));diff --git a/tweak/Sources/Redesigned/Kit/SGRTokens.m b/tweak/Sources/Redesigned/Kit/SGRTokens.mindex c6f17fe1..66727212 100644--- a/tweak/Sources/Redesigned/Kit/SGRTokens.m+++ b/tweak/Sources/Redesigned/Kit/SGRTokens.m@@ -18,6 +18,8 @@ const NSTimeInterval SGRCrossfade = 0.35;// A layout spring settles in about this long without overshooting; a press gives a little back.static const NSTimeInterval kLayoutDuration = 0.45, kPressDuration = 0.32;static const CGFloat kPressDamping = 0.62;+// A response lands within the press feedback's budget, on a curve that is most of the way there at once.+static const NSTimeInterval kRespondDuration = 0.2;UIColor *SGRPrimary(void) {return UIColor.whiteColor;@@ -85,7 +87,7 @@ BOOL SGRIncreaseContrast(void) {void SGRAnimate(SGRMotion motion, void (^animations)(void), void (^completion)(BOOL finished)) {if (!animations) return;- if (motion != SGRMotionFade && SGRReduceMotion()) {+ if (motion != SGRMotionFade && motion != SGRMotionRespond && SGRReduceMotion()) {[UIView performWithoutAnimation:animations];if (completion) completion(YES);return;@@ -101,5 +103,16 @@ void SGRAnimate(SGRMotion motion, void (^animations)(void), void (^completion)(Bcase SGRMotionFade:[UIView animateWithDuration:SGRCrossfade delay:0 options:options | UIViewAnimationOptionCurveEaseInOut animations:animations completion:completion];break;+ case SGRMotionRespond: {+ // UIView's own curves are too soft for this; a property animator takes any curve, and like the+ // others it starts from what is on screen, so it can take over a fade halfway out.+ UICubicTimingParameters *curve = [[UICubicTimingParameters alloc] initWithControlPoint1:CGPointMake(0.23, 1)+ controlPoint2:CGPointMake(0.32, 1)];+ UIViewPropertyAnimator *animator = [[UIViewPropertyAnimator alloc] initWithDuration:kRespondDuration timingParameters:curve];+ [animator addAnimations:animations];+ if (completion) [animator addCompletion:^(UIViewAnimatingPosition position) { completion(position == UIViewAnimatingPositionEnd); }];+ [animator startAnimation];+ break;+ }}}diff --git a/tweak/Sources/Redesigned/Lyrics/SGRKaraokeView.m b/tweak/Sources/Redesigned/Lyrics/SGRKaraokeView.mindex 81b4265d..d4e259f2 100644--- a/tweak/Sources/Redesigned/Lyrics/SGRKaraokeView.m+++ b/tweak/Sources/Redesigned/Lyrics/SGRKaraokeView.m@@ -11,11 +11,13 @@#import "SGRKaraokeView.h"#import "LyricsText.h"#import "MeaningSheet.h"+#import "SGRSingButton.h"#import "Shared/LyricsSources/LyricsSources.h"#import "Shared/LyricsTranslation/LyricsTranslation.h"#import "Settings/SGPageStyle.h"#import "Shared/Player/PlayerEvents.h"#import "Shared/Haptics/Haptics.h"+#import "Redesigned/Kit/SGRGlass.h"#import "Redesigned/Kit/SGRTokens.h"static const CGFloat kFontSize = 30, kMargin = 24, kLineGap = 24, kRowTighten = 2;@@ -34,6 +36,7 @@ static const CGFloat kCreditSize = 12, kCreditAlpha = 0.4, kCreditBottom = 10;// The button for the pronunciation and the translation, in the bottom leading corner as Apple Music// has it, and the gap between it and the credit beside it.static const CGFloat kExtrasSide = 44, kExtrasBottom = 12, kExtrasGlyph = 17, kExtrasCreditGap = 12;+static char kExtrasGlassKey;static const NSTimeInterval kRestyleFade = 0.3; // the lines crossfading to a new stylestatic const NSTimeInterval kBrowseHold = 3; // after scrolling by hand, how long until it follows the song againstatic const double kFloatMinMs = 700, kFloatLeadMs = 80; // a short word still floats up this slowly@@ -133,7 +136,9 @@ static UILabel *wordLabel(NSString *text, UIFont *font, UIColor *color, CGRect f@property (nonatomic) double riseStart, riseEnd;- (void)fillTo:(CGFloat)cursor;- (void)floatAt:(double)ms;-- (void)settle;+- (void)settle; // land and unglow at once+- (void)land; // back down from its float+- (void)unglow; // the glow off@end@implementation SGRKaraokeWordView {@@ -220,8 +225,16 @@ static UILabel *wordLabel(NSString *text, UIFont *font, UIColor *color, CGRect f}- (void)settle {+ [self land];+ [self unglow];+}++- (void)land {_lift = 0;self.transform = CGAffineTransformIdentity;+}++- (void)unglow {_glow = 0;_lit.layer.shadowOpacity = 0;}@@ -790,11 +803,12 @@ static const NSUInteger kLinesPerFrame = 4;return;}// A sung line fades back to dim rather than dropping its fill at once, and its words sink back- // on a spring slow enough to still be seen doing it.+ // on a spring slow enough to still be seen doing it. A held word's glow is left on: the spring would+ // drop it in the first frame, and the lit word fading out takes it out with it.[UIView animateWithDuration:0.9 delay:0 usingSpringWithDamping:1 initialSpringVelocity:0options:UIViewAnimationOptionAllowUserInteractionanimations:^{- for (SGRKaraokeWordView *word in self->_words) [word settle];+ for (SGRKaraokeWordView *word in self->_words) [word land];} completion:nil];[UIView animateWithDuration:0.5 delay:0 options:UIViewAnimationOptionCurveEaseOut animations:^{for (SGRKaraokeWordView *word in self->_words) word.lit.alpha = 0;@@ -802,6 +816,7 @@ static const NSUInteger kLinesPerFrame = 4;if (generation != self->_generation) return;for (SGRKaraokeWordView *word in self->_words) {[word fillTo:-CGFLOAT_MAX];+ [word unglow];word.lit.hidden = YES;word.lit.alpha = 1;}@@ -1098,8 +1113,11 @@ typedef struct {BOOL _hasSpoken, _hasTranslation; // whether the song has any line with eitherBOOL _untranslated; // whether a line with words has no translation, which Gemini can fill inNSArray<SGKaraokeLine *> *_passedOver; // lines kept for the song that changed nothing here, not looked at again+ UIView *_extrasBox; // the Kit's glass and _extras over it, which holds none, so its alpha can fadeUIButton *_extras;- CGFloat _builtWidth;+ NSArray<SGKaraokeLine *> *_translating; // the lines Gemini is translating, nil while it is not+ SGRSingButton *_sing; // across from _extras+ CGFloat _builtWidth, _placedHeight;BOOL _showing;CAGradientLayer *_fade;UILabel *_credit;@@ -1146,6 +1164,8 @@ typedef struct {_crediting = SGFlag(SGKeyLyricsCredit, NO);_sweepsEstimates = SGFlag(SGKeyLyricsSimulateWords, NO);[self addSubview:_credit];+ _sing = [SGRSingButton new];+ [self addSubview:_sing];[self addGestureRecognizer:[[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(tapped:)]];[self addGestureRecognizer:[[UILongPressGestureRecognizer alloc] initWithTarget:self action:@selector(held:)]];[NSNotificationCenter.defaultCenter addObserver:self selector:@selector(playerTransitionChanged:) name:SGPlayerTransitionNotification object:nil];@@ -1165,7 +1185,8 @@ typedef struct {}- (void)tapped:(UITapGestureRecognizer *)tap {- if (_extras && !_extras.hidden && CGRectContainsPoint(_extras.frame, [tap locationInView:self])) return;+ if (_extrasBox && !_extrasBox.hidden && CGRectContainsPoint(_extrasBox.frame, [tap locationInView:self])) return;+ if (_sing.userInteractionEnabled && [_sing pointInside:[tap locationInView:_sing] withEvent:nil]) return;CGPoint point = [tap locationInView:_scroll];for (SGRKaraokeLineView *view in _shown.allValues) {CGRect target = view.bubbleTarget;@@ -1185,6 +1206,7 @@ typedef struct {- (void)held:(UILongPressGestureRecognizer *)hold {if (hold.state != UIGestureRecognizerStateBegan || !_meanings.count) return;+ if (_sing.userInteractionEnabled && [_sing pointInside:[hold locationInView:_sing] withEvent:nil]) return;CGPoint point = [hold locationInView:_scroll];for (SGRKaraokeLineView *view in _shown.allValues) {if (!CGRectContainsPoint(CGRectInset(view.frame, -_margin, -_lineGap / 2), point)) continue;@@ -1226,10 +1248,16 @@ typedef struct {for (SGRKaraokeLineView *view in _shown.allValues) view.blur = 0;}+// In the caller's animation: the glass dematerialises by its effect and the glyph fades by its alpha, the+// one view of the two with no glass in it.- (void)setExtrasHidden:(BOOL)hidden {_extrasHidden = hidden;- _extras.alpha = hidden ? 0 : 1;- _extras.userInteractionEnabled = !hidden;+ if (_extrasBox) {+ SGRShowGlass(SGRGlassInside(_extrasBox, &kExtrasGlassKey, kExtrasSide), !hidden);+ _extras.alpha = hidden ? 0 : 1;+ _extrasBox.userInteractionEnabled = !hidden;+ }+ _sing.tucked = hidden;}- (void)scrollViewDidScroll:(UIScrollView *)scrollView {@@ -1318,11 +1346,22 @@ typedef struct {[_credit sizeToFit];_credit.frame = CGRectMake(_margin, self.bounds.size.height - _credit.bounds.size.height - kCreditBottom,_credit.bounds.size.width, _credit.bounds.size.height);- if (_extras && !_extras.hidden) {- _extras.frame = CGRectMake(_margin, self.bounds.size.height - kExtrasSide - kExtrasBottom, kExtrasSide, kExtrasSide);- _credit.center = CGPointMake(CGRectGetMaxX(_extras.frame) + kExtrasCreditGap + _credit.bounds.size.width / 2, _extras.center.y);+ if (_extrasBox && !_extrasBox.hidden) {+ _extrasBox.frame = CGRectMake(_margin, self.bounds.size.height - kExtrasSide - kExtrasBottom, kExtrasSide, kExtrasSide);+ SGRShowGlass(SGRGlassInside(_extrasBox, &kExtrasGlassKey, kExtrasSide), !_extrasHidden);+ _credit.center = CGPointMake(CGRectGetMaxX(_extrasBox.frame) + kExtrasCreditGap + _credit.bounds.size.width / 2, _extrasBox.center.y);+ }+ _sing.frame = CGRectMake(self.bounds.size.width - _margin - kExtrasSide, self.bounds.size.height - kExtrasSide - kExtrasBottom, kExtrasSide, kExtrasSide);+ if (_lines && self.bounds.size.width != _builtWidth) {+ [self rebuild];+ } else if (_tops && self.bounds.size.height != _placedHeight) {+ // A new height moves the anchor (the player's lines going alone and back): the stack is placed for it+ // here, inside whatever animation resized the page, so the sung line rides along with it rather than+ // springing to the new anchor at the next line, and the band the height opens is filled in.+ _sightOffset = -CGFLOAT_MAX;+ [self placeLinesAnimated:NO];}- if (_lines && self.bounds.size.width != _builtWidth) [self rebuild];+ _placedHeight = self.bounds.size.height;}- (void)dropLineViews {@@ -1459,31 +1498,35 @@ static BOOL hasWords(SGKaraokeLine *line) {BOOL gemini = SGGeminiKeySet();BOOL offered = _lines && (_hasSpoken || _hasTranslation || gemini);if (!offered) {- _extras.hidden = YES;+ _extrasBox.hidden = YES;return;}if (!_extras) {- UIImage *glyph = [UIImage systemImageNamed:@"translate" withConfiguration:- [UIImageSymbolConfiguration configurationWithPointSize:kExtrasGlyph weight:UIImageSymbolWeightSemibold]];- // The system's glass, which turns solid under Reduce Transparency by itself; before iOS 26, the- // Kit's solid fill in its place.- UIButtonConfiguration *config;- if (@available(iOS 26.0, *)) {- config = [UIButtonConfiguration glassButtonConfiguration];- } else {- config = [UIButtonConfiguration filledButtonConfiguration];- config.baseBackgroundColor = SGRSolidGlassFill();- }- config.image = glyph;- config.cornerStyle = UIButtonConfigurationCornerStyleCapsule;+ // The Kit's glass, as the mic across from it has (setExtrasHidden:), which turns solid under Reduce+ // Transparency, with a plain button over it.+ _extrasBox = [[UIView alloc] initWithFrame:CGRectMake(0, 0, kExtrasSide, kExtrasSide)];+ _extrasBox.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;+ UIButtonConfiguration *config = [UIButtonConfiguration plainButtonConfiguration];+ config.image = [UIImage systemImageNamed:@"translate"];+ config.preferredSymbolConfigurationForImage = [UIImageSymbolConfiguration configurationWithPointSize:kExtrasGlyph weight:UIImageSymbolWeightSemibold];config.baseForegroundColor = UIColor.whiteColor;_extras = [UIButton buttonWithConfiguration:config primaryAction:nil];- _extras.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;+ _extras.frame = _extrasBox.bounds;+ _extras.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;_extras.showsMenuAsPrimaryAction = YES;_extras.preferredMenuElementOrder = UIContextMenuConfigurationElementOrderFixed;_extras.accessibilityLabel = @"Pronunciation and translation";+ [_extrasBox addSubview:_extras];+ [self addSubview:_extrasBox];self.extrasHidden = _extrasHidden;- [self addSubview:_extras];+ }+ // While Gemini works the glyph turns into a spinner, and the menu has nothing to ask it a second time.+ BOOL translating = _translating && _translating == _lines;+ UIButtonConfiguration *config = _extras.configuration;+ if (config.showsActivityIndicator != translating) {+ config.showsActivityIndicator = translating;+ _extras.configuration = config;+ _extras.accessibilityValue = translating ? @"Translating" : nil;}NSMutableArray<UIMenuElement *> *items = [NSMutableArray array];if (_hasSpoken) {@@ -1499,13 +1542,16 @@ static BOOL hasWords(SGKaraokeLine *line) {handler:^(UIAction *action) { SGRSetLyricsTextShown(SGRLyricsTextTranslation, !on); }]];}// Also for a song a source translated in part, as Musixmatch's community often has: Gemini fills in the rest.- if (gemini && _untranslated) {+ // Said in the item itself: the song's words leave the phone for Google's servers.+ if (gemini && _untranslated && !translating) {__weak SGRKaraokeView *weakSelf = self;- [items addObject:[UIAction actionWithTitle:@"Translate with Gemini" image:[UIImage systemImageNamed:@"sparkles"]- identifier:nil handler:^(UIAction *action) { [weakSelf translateWithGemini]; }]];+ UIAction *gemini = [UIAction actionWithTitle:@"Translate with Gemini" image:[UIImage systemImageNamed:@"sparkles"]+ identifier:nil handler:^(UIAction *action) { [weakSelf translateWithGemini]; }];+ gemini.subtitle = @"Sends the lyrics to Google";+ [items addObject:gemini];}_extras.menu = [UIMenu menuWithChildren:items];- _extras.hidden = NO;+ _extrasBox.hidden = NO;[self setNeedsLayout];}@@ -1514,11 +1560,17 @@ static BOOL hasWords(SGKaraokeLine *line) {- (void)translateWithGemini {NSArray<SGKaraokeLine *> *lines = _lines;NSString *track = _track;- if (!lines.count) return;+ if (!lines.count || _translating == lines) return;+ _translating = lines;+ [self offerExtras];SGLyricsTranslateWithGemini(track, lines, SGLyricsGeminiLanguage(), ^(NSArray<NSString *> *translations, NSString *error) {+ if (self->_translating == lines) self->_translating = nil;+ [self offerExtras];if (!translations) {UIAlertController *alert = [UIAlertController alertControllerWithTitle:@"No translation" message:errorpreferredStyle:UIAlertControllerStyleAlert];+ // Over the player, which is dark whatever the system's appearance.+ alert.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;[alert addAction:[UIAlertAction actionWithTitle:@"OK" style:UIAlertActionStyleCancel handler:nil]];[SGTopController() presentViewController:alert animated:YES completion:nil];return;@@ -1566,15 +1618,18 @@ static BOOL hasWords(SGKaraokeLine *line) {// The view of a line, made the first time it is asked for and placed where the stack has it.- (SGRKaraokeLineView *)viewForLine:(NSInteger)index {- SGRKaraokeLineView *view = _shown[@(index)];+ __block SGRKaraokeLineView *view = _shown[@(index)];if (view || !_tops || index < 0 || index >= (NSInteger)_tops.count) return view;- view = [[SGRKaraokeLineView alloc] initWithLine:_lines[index] width:_builtWidth - 2 * _margin style:_style under:nil- blurred:_maxBlur > 0 && !_plain sweepsEstimates:_sweepsEstimates];- if (_plain) [view showPlain];- [view markMeaning:_meanings[@(index)]];- [_scroll addSubview:view];- _shown[@(index)] = view;- [self placeLine:view at:index animated:NO];+ // Made where it belongs even inside an animation of the page's (a new height), not flown in from the corner.+ [UIView performWithoutAnimation:^{+ view = [[SGRKaraokeLineView alloc] initWithLine:self->_lines[index] width:self->_builtWidth - 2 * self->_margin style:self->_style under:nil+ blurred:self->_maxBlur > 0 && !self->_plain sweepsEstimates:self->_sweepsEstimates];+ if (self->_plain) [view showPlain];+ [view markMeaning:self->_meanings[@(index)]];+ [self->_scroll addSubview:view];+ self->_shown[@(index)] = view;+ [self placeLine:view at:index animated:NO];+ }];return view;}diff --git a/tweak/Sources/Redesigned/Lyrics/SGRSingButton.h b/tweak/Sources/Redesigned/Lyrics/SGRSingButton.hnew file mode 100644index 00000000..72d5e52b--- /dev/null+++ b/tweak/Sources/Redesigned/Lyrics/SGRSingButton.h@@ -0,0 +1,25 @@+// Sing's mic on the redesign's lyrics (Shared/Sing/Sing.h), in the bottom trailing corner of SGRKaraokeView+// across from the pronunciation and translation button, as Apple Music has it. It shows what Sing is doing+// on itself: the mic struck through where Sing cannot run, a ring filling with the voice model's download, a+// ring turning while the model prepares and while Sing listens ahead, the mic lit in the accent colour while+// the vocals are down, a thermometer while the heat holds it, a warning when it failed.+//+// A tap turns Sing on and off, or says what is missing and offers the model's download. Holding it brings up+// its slider, which takes the vocals from gone at the bottom, through the song as sung in the middle, to the+// vocals alone at the top. To VoiceOver the mic is one adjustable button: a swipe up or down moves the+// same level.+//+// Threading: main thread only.+#import <UIKit/UIKit.h>++// Shows or hides a Kit glass shape (Redesigned/Kit/SGRGlass.h) by its effect, never by an alpha on it or on+// a view above it (PlayerMotion.x): UIKit draws the material wrongly or not at all under a fading alpha. The+// solid shape Reduce Transparency puts in its place has no effect and fades by its own alpha. For the mic and+// the pronunciation and translation button across from it; animates in the caller's animation.+void SGRShowGlass(UIView *shape, BOOL shown);++@interface SGRSingButton : UIView+// Goes with the host's controls when they are hidden, in the host's animation: its glass dematerialises+// rather than fading under an alpha, and it takes no touches while tucked.+@property (nonatomic) BOOL tucked;+@enddiff --git a/tweak/Sources/Redesigned/Lyrics/SGRSingButton.m b/tweak/Sources/Redesigned/Lyrics/SGRSingButton.mnew file mode 100644index 00000000..f7f0acbd--- /dev/null+++ b/tweak/Sources/Redesigned/Lyrics/SGRSingButton.m@@ -0,0 +1,369 @@+#import "Core/SGCore.h"+#import "Settings/SGPageStyle.h"+#import "Shared/Sing/Sing.h"+#import "Redesigned/Kit/SGRAccent.h"+#import "Redesigned/Kit/SGRGlass.h"+#import "Redesigned/Kit/SGRTokens.h"+#import "SGRSingButton.h"++static const CGFloat kGlyph = 17, kRingWidth = 2.5, kRingInset = 1.5;+// The slider's capsule over the button, and the gap between them: wider than SGRGlassSpacing (16), so the+// two glass shapes stay apart rather than sampling each other.+static const CGFloat kPanelHeight = 168, kPanelGap = 24, kLevelWidth = 120, kLevelHeight = 20;+// The capsule opens from just under full size at the mic, the way a popover grows out of what opened it.+static const CGFloat kPanelEnterScale = 0.95;+static const NSTimeInterval kPanelStays = 2.5;+static const float kLevelStep = 0.05f;+// As sung (1) holds the thumb this close either side, 4% of the slider's travel: a detent caught with a tick, as the+// Sing page's tall slider ticks there.+static const float kAsSungCatch = 0.08f;+static const CGFloat kMarkWidth = 14;+static char kButtonGlassKey, kPanelGlassKey;++void SGRShowGlass(UIView *shape, BOOL shown) {+ if ([shape isKindOfClass:UIVisualEffectView.class]) {+ UIVisualEffectView *glass = (UIVisualEffectView *)shape;+ if ((glass.effect != nil) != shown) glass.effect = shown ? SGGlassEffect() : nil;+ } else {+ shape.alpha = shown ? 1 : 0;+ }+}++@implementation SGRSingButton {+ UIButton *_button; // holds no glass, so its alpha is free to fade+ CAShapeLayer *_ring;+ BOOL _ringShown;+ NSString *_glyph;+ UIView *_panel;+ UISlider *_slider;+ UIView *_mark; // across the track at As sung+ UISelectionFeedbackGenerator *_feedback;+ BOOL _caught; // the thumb is held at As sung+ UILabel *_level;+ NSTimer *_hide;+ BOOL _panelShown;+}++- (instancetype)initWithFrame:(CGRect)frame {+ if (!(self = [super initWithFrame:frame])) return nil;+ // The Kit's glass behind a plain button (layoutSubviews), the same material as the pronunciation and+ // translation button across from it, so the glass can be faded by its effect when the controls go.+ UIButtonConfiguration *config = [UIButtonConfiguration plainButtonConfiguration];+ config.preferredSymbolConfigurationForImage = [UIImageSymbolConfiguration configurationWithPointSize:kGlyph weight:UIImageSymbolWeightSemibold];+ _button = [UIButton buttonWithConfiguration:config primaryAction:nil];+ _button.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;+ _button.tintColor = UIColor.whiteColor;+ _button.isAccessibilityElement = NO;+ [_button addTarget:self action:@selector(tapped) forControlEvents:UIControlEventTouchUpInside];+ [_button addGestureRecognizer:[[UILongPressGestureRecognizer alloc] initWithTarget:self action:@selector(held:)]];+ [self addSubview:_button];++ // VoiceOver has the mic itself as one adjustable button: the slider shows only for a held finger and+ // goes again on a timer, which a swipe up or down never holds open.+ self.isAccessibilityElement = YES;+ self.accessibilityLabel = @"Sing";+ self.accessibilityHint = @"Turns the vocals down";+ self.accessibilityTraits = UIAccessibilityTraitButton | UIAccessibilityTraitAdjustable;++ _ring = [CAShapeLayer layer];+ _ring.fillColor = UIColor.clearColor.CGColor;+ _ring.strokeColor = UIColor.whiteColor.CGColor;+ _ring.lineWidth = kRingWidth;+ _ring.lineCap = kCALineCapRound;+ _ring.opacity = 0;+ [self.layer addSublayer:_ring];++ [NSNotificationCenter.defaultCenter addObserver:self selector:@selector(refresh) name:SGSingChangedNotification object:nil];+ [NSNotificationCenter.defaultCenter addObserver:self selector:@selector(refresh) name:UIAccessibilityReduceMotionStatusDidChangeNotification object:nil];+ [self refresh];+ return self;+}++- (void)dealloc {+ [NSNotificationCenter.defaultCenter removeObserver:self];+ [_hide invalidate];+}++- (void)layoutSubviews {+ [super layoutSubviews];+ CGRect bounds = self.bounds;+ _button.frame = bounds;+ SGRShowGlass(SGRGlassInside(self, &kButtonGlassKey, MIN(bounds.size.width, bounds.size.height)), !_tucked);+ _ring.frame = bounds;+ // Starting at the top and running clockwise, as a download ring does.+ CGFloat radius = MIN(bounds.size.width, bounds.size.height) / 2 - kRingInset - kRingWidth / 2;+ _ring.path = [UIBezierPath bezierPathWithArcCenter:CGPointMake(CGRectGetMidX(bounds), CGRectGetMidY(bounds)) radius:radius+ startAngle:-M_PI_2 endAngle:3 * M_PI_2 clockwise:YES].CGPath;+ if (_panel) SGRShowGlass([self placePanel], _panelShown);+}++// Goes with the host's controls, in the host's animation: the glass dematerialises, the glyph and the ring+// fade, and an open slider goes with them.+- (void)setTucked:(BOOL)tucked {+ _tucked = tucked;+ SGRShowGlass(SGRGlassInside(self, &kButtonGlassKey, MIN(self.bounds.size.width, self.bounds.size.height)), !tucked);+ _button.alpha = tucked ? 0 : 1;+ [self showRing];+ self.userInteractionEnabled = !tucked;+ if (tucked && _panelShown) [self panelGone];+}++#pragma mark - what Sing is doing++- (void)refresh {+ SGSingState state = SGSingCurrentState();+ NSString *glyph = @"mic.fill";+ UIColor *tint = UIColor.whiteColor;+ CGFloat alpha = 1;+ double progress = -1; // a ring at a fraction; 2 for a turning one, -1 for none+ switch (state) {+ case SGSingStateUnavailable: glyph = @"mic.slash"; alpha = 0.5; break;+ case SGSingStateNoModel: glyph = @"mic"; alpha = 0.7; break;+ case SGSingStateDownloading: glyph = @"arrow.down"; progress = SGSingModelProgress(); break;+ case SGSingStateOff: glyph = @"mic"; break;+ case SGSingStatePreparing:+ case SGSingStateBuffering: progress = 2; break;+ case SGSingStateWaiting: alpha = 0.7; break;+ case SGSingStateSinging: tint = SGRAccentColor() ?: UIColor.systemGreenColor; break;+ case SGSingStateBehind: glyph = @"tortoise.fill"; break;+ case SGSingStateHot: glyph = @"thermometer.high"; tint = UIColor.systemOrangeColor; break;+ case SGSingStateFailed: glyph = @"exclamationmark.triangle.fill"; tint = UIColor.systemYellowColor; break;+ }+ // One glyph replaces the other in place, as the Kit's download glyph does (none under Reduce Motion),+ // and the colour crossfades. Set only on a change: the download's progress refreshes many times a second.+ if (![glyph isEqualToString:_glyph]) {+ _glyph = glyph;+ UIButtonConfiguration *config = _button.configuration;+ if (@available(iOS 26.0, *)) {+ config.symbolContentTransition = SGRReduceMotion() || !self.window ? nil+ : [UISymbolContentTransition transitionWithContentTransition:[NSSymbolReplaceContentTransition replaceDownUpTransition]];+ }+ config.image = [UIImage systemImageNamed:glyph];+ _button.configuration = config;+ }+ UIColor *color = [tint colorWithAlphaComponent:alpha];+ if (![color isEqual:_button.tintColor]) {+ if (self.window) SGRAnimate(SGRMotionFade, ^{ self->_button.tintColor = color; }, nil);+ else _button.tintColor = color;+ }++ // A turning ring under Reduce Motion is a quarter of one standing still: repetitive motion is what it asks+ // to be left out.+ _ringShown = progress >= 0;+ BOOL turns = progress > 1 && !SGRReduceMotion();+ if (progress > 1) {+ [CATransaction begin];+ [CATransaction setDisableActions:YES];+ _ring.strokeEnd = 0.25;+ [CATransaction commit];+ } else if (progress >= 0) {+ _ring.strokeEnd = progress;+ }+ if (turns && ![_ring animationForKey:@"turn"]) {+ CABasicAnimation *turn = [CABasicAnimation animationWithKeyPath:@"transform.rotation.z"];+ turn.fromValue = @0;+ turn.toValue = @(2 * M_PI);+ turn.duration = 1;+ turn.repeatCount = HUGE_VALF;+ turn.removedOnCompletion = NO;+ [_ring addAnimation:turn forKey:@"turn"];+ } else if (!turns && progress >= 0) {+ [_ring removeAnimationForKey:@"turn"];+ }+ [self showRing];+}++// The ring fades rather than popping in and out; a turning one keeps turning until it is gone.+- (void)showRing {+ float opacity = _ringShown && !_tucked ? 1 : 0;+ if (_ring.opacity == opacity) return;+ [CATransaction begin];+ [CATransaction setAnimationDuration:SGRCrossfade];+ [CATransaction setCompletionBlock:^{+ if (self->_ring.opacity == 0) [self->_ring removeAnimationForKey:@"turn"];+ }];+ _ring.opacity = opacity;+ [CATransaction commit];+}++#pragma mark - VoiceOver++- (NSString *)accessibilityValue {+ return [NSString stringWithFormat:@"%@. Vocals level: %@", SGSingStatusText(), SGSingLevelText(SGSingLevel())];+}++- (BOOL)accessibilityActivate {+ [self tapped];+ return YES;+}++// Two of the slider's steps at a time, so the whole range is twenty swipes.+- (void)accessibilityIncrement {+ [self stepLevel:1];+}++- (void)accessibilityDecrement {+ [self stepLevel:-1];+}++- (void)stepLevel:(int)direction {+ SGSetSingLevel(roundf((SGSingLevel() + direction * 2 * kLevelStep) / kLevelStep) * kLevelStep);+ if (_panel) {+ _slider.value = SGSingLevel();+ [self showLevel];+ }+}++#pragma mark - the tap++static void tell(NSString *message, NSString *action, void (^then)(void)) {+ UIAlertController *alert = [UIAlertController alertControllerWithTitle:@"Sing" message:message preferredStyle:UIAlertControllerStyleAlert];+ // Over the player, which is dark whatever the system's appearance.+ alert.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;+ if (action) [alert addAction:[UIAlertAction actionWithTitle:action style:UIAlertActionStyleDefault handler:^(UIAlertAction *a) { then(); }]];+ [alert addAction:[UIAlertAction actionWithTitle:action ? @"Cancel" : @"OK" style:UIAlertActionStyleCancel handler:nil]];+ [SGTopController() presentViewController:alert animated:YES completion:nil];+}++- (void)tapped {+ switch (SGSingCurrentState()) {+ case SGSingStateUnavailable:+ tell(SGSingMissing(), nil, nil);+ return;+ case SGSingStateNoModel:+ tell([NSString stringWithFormat:@"Sing turns the vocals down with a voice model that runs on this iPhone. Download it now (%@)? It is best over Wi-Fi.%@",+ SGSingModelSizeText(), SGSingModelError() ? [NSString stringWithFormat:@"\n\nThe last try failed: %@", SGSingModelError()] : @""],+ @"Download", ^{+ SGSingDownloadModel();+ // Downloaded for the mic, so the mic is on once it is in.+ SGSetSingOn(YES);+ });+ return;+ case SGSingStateDownloading:+ tell(SGSingStatusDetail(), @"Stop the download", ^{ SGSingCancelModelDownload(); });+ return;+ case SGSingStateFailed:+ tell(SGSingStatusDetail(), @"Turn Sing off", ^{ SGSetSingOn(NO); });+ return;+ default:+ SGSetSingOn(!SGSingOn());+ }+}++#pragma mark - the slider++- (void)held:(UILongPressGestureRecognizer *)hold {+ if (hold.state != UIGestureRecognizerStateBegan) return;+ [self showPanel];+}++- (void)showPanel {+ if (!_panel) {+ _panel = [UIView new];+ _panel.overrideUserInterfaceStyle = UIUserInterfaceStyleDark;+ _panel.layer.anchorPoint = CGPointMake(0.5, 1); // its bottom edge, over the mic+ _slider = [UISlider new];+ _slider.minimumValue = 0;+ _slider.maximumValue = 2;+ _slider.transform = CGAffineTransformMakeRotation(-M_PI_2);+ _slider.accessibilityLabel = @"Vocals";+ [_slider addTarget:self action:@selector(slid) forControlEvents:UIControlEventValueChanged];+ [_slider addTarget:self action:@selector(holdPanel) forControlEvents:UIControlEventTouchDown];+ [_slider addTarget:self action:@selector(letPanelGo) forControlEvents:UIControlEventTouchUpInside | UIControlEventTouchUpOutside | UIControlEventTouchCancel];+ [_panel addSubview:_slider];+ _mark = [UIView new];+ _mark.backgroundColor = [UIColor colorWithWhite:1 alpha:0.35];+ _mark.userInteractionEnabled = NO;+ [_panel insertSubview:_mark belowSubview:_slider];+ _feedback = [UISelectionFeedbackGenerator new];+ _level = [UILabel new];+ _level.font = [UIFont systemFontOfSize:12 weight:UIFontWeightSemibold];+ _level.textColor = [UIColor colorWithWhite:1 alpha:0.8];+ _level.textAlignment = NSTextAlignmentRight;+ _level.isAccessibilityElement = NO;+ [self addSubview:_panel];+ [self addSubview:_level];+ _slider.alpha = _mark.alpha = _level.alpha = 0;+ SGRShowGlass([self placePanel], NO);+ if (!SGRReduceMotion()) _panel.transform = CGAffineTransformMakeScale(kPanelEnterScale, kPanelEnterScale);+ }+ _slider.minimumTrackTintColor = SGRAccentColor() ?: UIColor.systemGreenColor;+ _slider.value = SGSingLevel();+ [self showLevel];+ UIView *glass = [self placePanel];+ _panelShown = YES;+ // The fade answers the hold at once; the growth is motion, which Reduce Motion leaves out.+ SGRAnimate(SGRMotionRespond, ^{+ SGRShowGlass(glass, YES);+ self->_slider.alpha = self->_mark.alpha = self->_level.alpha = 1;+ }, nil);+ SGRAnimate(SGRMotionLayout, ^{ self->_panel.transform = CGAffineTransformIdentity; }, nil);+ [self letPanelGo];+}++// Bounds and a centre rather than a frame, since the capsule can be under its entering scale.+- (UIView *)placePanel {+ CGFloat width = self.bounds.size.width, top = -kPanelGap - kPanelHeight;+ _panel.bounds = CGRectMake(0, 0, width, kPanelHeight);+ _panel.center = CGPointMake(width / 2, -kPanelGap);+ UIView *glass = SGRGlassCapsuleInside(_panel, &kPanelGlassKey, _panel.bounds.size, NO);+ _slider.bounds = CGRectMake(0, 0, kPanelHeight - 24, 31);+ _slider.center = CGPointMake(width / 2, kPanelHeight / 2);+ // The slider is centred, so As sung, the middle of its range, is the capsule's middle.+ _mark.frame = CGRectMake((width - kMarkWidth) / 2, kPanelHeight / 2 - 0.5, kMarkWidth, 1);+ _level.frame = CGRectMake(width - kLevelWidth, top - kLevelHeight - 4, kLevelWidth, kLevelHeight);+ return glass;+}++- (void)showLevel {+ _level.text = SGSingLevelText(_slider.value);+ _slider.accessibilityValue = _level.text;+}++// Near As sung the thumb holds there, with a tick as it catches; the finger drags it on out of the band.+- (void)slid {+ BOOL caught = fabsf(_slider.value - 1) <= kAsSungCatch;+ if (caught) _slider.value = 1;+ if (caught && !_caught) [_feedback selectionChanged];+ _caught = caught;+ SGSetSingLevel(roundf(_slider.value / kLevelStep) * kLevelStep);+ [self showLevel];+}++- (void)holdPanel {+ [_hide invalidate];+ _hide = nil;+ _caught = fabsf(_slider.value - 1) <= kAsSungCatch;+ [_feedback prepare];+}++- (void)letPanelGo {+ [_hide invalidate];+ __weak SGRSingButton *weakSelf = self;+ _hide = [NSTimer scheduledTimerWithTimeInterval:kPanelStays repeats:NO block:^(NSTimer *timer) { [weakSelf hidePanel]; }];+}++// Out faster than it came, and without shrinking: nothing is watched on its way out.+- (void)hidePanel {+ if (!_panelShown) return;+ SGRAnimate(SGRMotionRespond, ^{ [self panelGone]; }, ^(BOOL finished) {+ if (!self->_panelShown && !SGRReduceMotion()) self->_panel.transform = CGAffineTransformMakeScale(kPanelEnterScale, kPanelEnterScale);+ });+}++// The capsule's glass and the slider gone, in whatever animation the caller has open.+- (void)panelGone {+ _panelShown = NO;+ [_hide invalidate];+ _hide = nil;+ SGRShowGlass(SGRGlassCapsuleInside(_panel, &kPanelGlassKey, _panel.bounds.size, NO), NO);+ _slider.alpha = _mark.alpha = _level.alpha = 0;+}++// The slider sits outside the button's bounds, above it, and takes touches while it shows.+- (BOOL)pointInside:(CGPoint)point withEvent:(UIEvent *)event {+ if (_panelShown && CGRectContainsPoint(_panel.frame, point)) return YES;+ return [super pointInside:point withEvent:event];+}++@enddiff --git a/tweak/Sources/Settings/SGModPage.m b/tweak/Sources/Settings/SGModPage.mindex ac1dce64..81238574 100644--- a/tweak/Sources/Settings/SGModPage.m+++ b/tweak/Sources/Settings/SGModPage.m@@ -424,9 +424,9 @@ static const CGFloat kSliderTop = 12, kSliderLine = 18, kSliderSubtitle = 14, kS_shown = [self rowsToShow];_intro = intro ? SGNote(intro) : nil;_footer = footer ? SGNote(footer) : nil;- // A page row reads its value out when the page appears rather than on the ticker, so only the- // rows whose numbers climb on their own keep one running.- for (SGModSection *s in sections) for (SGModRow *row in s.rows) _live |= row.value && !row.page;+ // Any row with a value keeps a ticker running: a page row's can move while the page shows (a download's+ // percentage beside the chevron).+ for (SGModSection *s in sections) for (SGModRow *row in s.rows) _live |= row.value != nil;return self;}@@ -522,6 +522,14 @@ static const CGFloat kSliderTop = 12, kSliderLine = 18, kSliderSubtitle = 14, kS- (void)readValues {for (UITableViewCell *cell in self.tableView.visibleCells) {SGModRow *row = [self rowAt:[self.tableView indexPathForCell:cell]];+ if (row.page && row.value) {+ // valueAndChevron is sized to its text, so a new text gets a new one.+ NSString *text = row.value();+ UILabel *shown = (UILabel *)cell.accessoryView.subviews.firstObject;+ if (![shown isKindOfClass:UILabel.class] || [shown.text isEqualToString:text]) continue;+ cell.accessoryView = valueAndChevron(text);+ continue;+ }UILabel *label = (UILabel *)cell.accessoryView;if (!row.value || row.page || ![label isKindOfClass:UILabel.class]) continue;label.text = row.value();diff --git a/tweak/Sources/Shared/Player/SpeedPitch.h b/tweak/Sources/Shared/Player/SpeedPitch.hindex 3127ff96..8968e5cf 100644--- a/tweak/Sources/Shared/Player/SpeedPitch.h+++ b/tweak/Sources/Shared/Player/SpeedPitch.h@@ -10,6 +10,7 @@// Speed and pitch last until Spotify quits; neither is stored.// Threading: main thread only, except what SGTimePitch.h says runs on the render thread.#import <UIKit/UIKit.h>+#import <AudioToolbox/AudioToolbox.h>// Marks a menu opened soon after a tap on `button`, the player's more button, as the player's, so it gets// Speed and pitch (watching it twice does nothing). The redesign's PlayerHeader.x hands its button over;@@ -41,3 +42,9 @@ BOOL SGPlayerPitchAvailable(void);BOOL SGPlayerMenuOffersAnimatedArtwork(void);BOOL SGPlayerMenuAnimatedArtwork(void);void SGPlayerMenuSetAnimatedArtwork(BOOL on);+// A stage between Spotify's mixer and the rest of the chain (Sing's look-ahead, Shared/Sing): it fills the+// chain's buffers, pulling the mixer through `pull` as much as it likes. NULL passes the mixer straight on.+// Called on the render thread, and only while Spotify's connection is taken over (SGPlayerSpeedAllowed).+typedef OSStatus (*SGPlayerPull)(void *context, UInt32 frames, AudioBufferList *data);+typedef OSStatus (*SGPlayerStage)(UInt32 frames, AudioBufferList *data, SGPlayerPull pull, void *context);+void SGPlayerSetStage(SGPlayerStage stage);diff --git a/tweak/Sources/Shared/Player/SpeedPitch.x b/tweak/Sources/Shared/Player/SpeedPitch.xindex bd06082b..d906e921 100644--- a/tweak/Sources/Shared/Player/SpeedPitch.x+++ b/tweak/Sources/Shared/Player/SpeedPitch.x@@ -122,13 +122,37 @@ static OSStatus pullSource(AudioUnit source, const AudioTimeStamp *outputTime, Ureturn noErr;}+static _Atomic(SGPlayerStage) sg_stage;++void SGPlayerSetStage(SGPlayerStage stage) {+ atomic_store(&sg_stage, stage);+}++typedef struct {+ AudioUnit source;+ const AudioTimeStamp *time;+} MixerPull;++static OSStatus pullMixer(void *context, UInt32 frames, AudioBufferList *data) {+ MixerPull *mixer = context;+ return pullSource(mixer->source, mixer->time, frames, data);+}++// The mixer's sound, through the stage when one is set.+static OSStatus pullChain(AudioUnit source, const AudioTimeStamp *time, UInt32 frames, AudioBufferList *data) {+ SGPlayerStage stage = atomic_load_explicit(&sg_stage, memory_order_acquire);+ if (!stage) return pullSource(source, time, frames, data);+ MixerPull mixer = {source, time};+ return stage(frames, data, pullMixer, &mixer);+}+static OSStatus pullForUnit(void *context, UInt32 frames, AudioBufferList *data) {AudioUnit source = atomic_load(&sg_source);if (!source) {for (UInt32 b = 0; b < data->mNumberBuffers; b++) if (data->mBuffers[b].mData) memset(data->mBuffers[b].mData, 0, data->mBuffers[b].mDataByteSize);return noErr;}- return pullSource(source, NULL, frames, data);+ return pullChain(source, NULL, frames, data);}static BOOL fitsUnit(const AudioBufferList *data, UInt32 frames, SGTimePitch *unit) {@@ -152,7 +176,7 @@ static OSStatus feed(void *refCon, AudioUnitRenderActionFlags *flags, const AudiOSStatus status = -1;SGTimePitch *unit = atomic_load(&sg_pull);if (atomic_load(&sg_engaged) && unit && fitsUnit(data, frames, unit)) status = SGTimePitchRender(unit, frames, data);- if (status != noErr) status = pullSource(source, timestamp, frames, data);+ if (status != noErr) status = pullChain(source, timestamp, frames, data);atomic_store(&sg_busy, false);return status;}diff --git a/tweak/Sources/Shared/Sing/SGSingEngine.h b/tweak/Sources/Shared/Sing/SGSingEngine.hnew file mode 100644index 00000000..37bef31b--- /dev/null+++ b/tweak/Sources/Shared/Sing/SGSingEngine.h@@ -0,0 +1,57 @@+// Sing's engine: the vocals of what Spotify plays separated ahead of it, and turned down as it plays.+//+// The model takes two seconds at a time and is far too slow for the render thread, so the engine stands+// between Spotify's mixer and its output and pulls the mixer ahead of what plays: up to twice what a+// render asks for, until the sound it holds (the lead) is a window plus the model's time. A worker thread+// takes the held sound two seconds at a time, every 1.5 s, runs the separator and writes the vocals into a+// second ring beside it, the half second the windows share crossfaded. Each render plays the held sound+// from the oldest frame on, the vocals turned down by the level wherever they are in, and dry wherever+// they are not yet (as the lead fills, or when the model falls behind). Switched off, it pulls half of+// what plays until the lead is given back, and is a straight pull again.+//+// What plays is the lead behind what Spotify's decoder has handed over, which is what Spotify's clock+// counts: Sing.x takes the lead off the player's position, and asks for a flush when Spotify seeks or+// skips, which drops the sound held from before it.+//+// Threading: SGSingEngineRender on the render thread only, allocation and lock free; the worker is the+// engine's own; everything else from any thread.+#import <Foundation/Foundation.h>++@class SGSingSeparator;++enum {+ kSGSingEngineHop = 66150, // 1.5 s between windows, so neighbours share half a second+};++typedef struct SGSingEngine SGSingEngine;++// Fills `frames` frames of each channel with the next of Spotify's sound.+typedef OSStatus (*SGSingPull)(void *context, UInt32 frames, float *left, float *right);++SGSingEngine *SGSingEngineCreate(void);+// Stops the worker and frees the engine; nothing may be rendering through it.+void SGSingEngineDestroy(SGSingEngine *engine);+// The separator the worker runs, nil for none (what plays is then dry).+void SGSingEngineSetSeparator(SGSingEngine *engine, SGSingSeparator *separator);+void SGSingEngineSetOn(SGSingEngine *engine, bool on);+bool SGSingEngineOn(SGSingEngine *engine);+// The vocals' level from 0 (gone) through 1 (as the song has them) to 2 (the vocals alone, the rest gone).+void SGSingEngineSetLevel(SGSingEngine *engine, float level);+// Holds the worker (a hot phone): what plays is dry, and the lead is given back.+void SGSingEngineSetPaused(SGSingEngine *engine, bool paused);+// Drops the sound held ahead, at the next render.+void SGSingEngineFlush(SGSingEngine *engine);+// Seconds of Spotify's sound held ahead of what plays.+double SGSingEngineLead(SGSingEngine *engine);++OSStatus SGSingEngineRender(SGSingEngine *engine, UInt32 frames, float *left, float *right, SGSingPull pull, void *context);++typedef struct {+ double lead, targetLead; // seconds+ double ready; // seconds of the held sound separated, ahead of what plays (negative when behind)+ double averageMS; // a window's separation, a running average+ unsigned long long windows, failures, dryFrames; // dry: frames played unseparated while on+} SGSingEngineStats;+SGSingEngineStats SGSingEngineReadStats(SGSingEngine *engine);+// The model's last error, nil when its last window went through.+NSString *SGSingEngineError(SGSingEngine *engine);diff --git a/tweak/Sources/Shared/Sing/SGSingEngine.m b/tweak/Sources/Shared/Sing/SGSingEngine.mnew file mode 100644index 00000000..b618dce4--- /dev/null+++ b/tweak/Sources/Shared/Sing/SGSingEngine.m@@ -0,0 +1,340 @@+#import <os/lock.h>+#import <pthread.h>+#import <stdatomic.h>+#import "SGSingEngine.h"+#import "SGSingSeparator.h"++enum {+ kRing = kSGSingRate * 10, // what the rings hold: the longest lead and a window, with room to spare+ kLongestLead = kSGSingRate * 6,+ kShortestLead = kSGSingRate * 5 / 2,+};+// The lead asked for over a window: this many times the model's average time, and this much more.+static const double kLeadPerModelTime = 1.5, kLeadSpare = 0.25;+// The vocals come in and go out over this long, so the switch and the lead filling do not click.+static const float kFadeSeconds = 0.1f;+// How often the worker looks for a new window.+static const useconds_t kPoll = 10000;++struct SGSingEngine {+ float *left, *right, *vocalsLeft, *vocalsRight; // rings, a frame at position p at p % kRing++ // Positions in frames of Spotify's sound since the engine was made: what was pulled, what was played,+ // and up to where the vocals are in.+ _Atomic uint64_t written, played, ready;+ _Atomic uint64_t base; // where the worker starts over, at the latest flush+ atomic_uint generation; // counts the flushes+ atomic_bool flushAsked, on, paused, hasSeparator;+ atomic_uint levelBits, targetLead;++ // The render thread's own.+ bool running; // the rings are in use: Sing is on, or the lead is not given back yet+ float amount, vocalsGain, otherGain;++ // The worker's.+ pthread_t worker;+ atomic_bool alive;+ os_unfair_lock lock; // the separator and the error+ void *separator; // retained+ NSString *error;+ float *window[2], *vocals[2];++ _Atomic uint64_t windows, failures, dryFrames, averageMicroseconds;+};++static float loadFloat(atomic_uint *slot) {+ uint32_t bits = atomic_load_explicit(slot, memory_order_relaxed);+ float value;+ memcpy(&value, &bits, sizeof value);+ return value;+}++static void storeFloat(atomic_uint *slot, float value) {+ uint32_t bits;+ memcpy(&bits, &value, sizeof bits);+ atomic_store(slot, bits);+}++#pragma mark - the render thread++// Starts the rings over where the mixer is now.+static void startOver(SGSingEngine *engine) {+ uint64_t written = atomic_load_explicit(&engine->written, memory_order_relaxed);+ atomic_store(&engine->played, written);+ atomic_store(&engine->base, written);+ atomic_fetch_add(&engine->generation, 1);+ engine->amount = 0;+}++// `count` frames from the mixer into the rings at `at`, in up to two runs where the ring wraps.+static OSStatus pullInto(SGSingEngine *engine, uint64_t at, uint64_t count, SGSingPull pull, void *context) {+ while (count) {+ uint64_t index = at % kRing, run = MIN(count, kRing - index);+ OSStatus status = pull(context, (UInt32)run, engine->left + index, engine->right + index);+ if (status != noErr) return status;+ at += run;+ count -= run;+ }+ return noErr;+}++// The gains for a level: the vocals up to 1, then the rest down to nothing at 2.+static void gainsFor(float level, float *vocals, float *other) {+ level = fmaxf(0, fminf(level, 2));+ *vocals = fminf(level, 1);+ *other = level <= 1 ? 1 : 2 - level;+}++OSStatus SGSingEngineRender(SGSingEngine *engine, UInt32 frames, float *left, float *right, SGSingPull pull, void *context) {+ // Larger than any slice an output asks for, and more than the rings leave room for: passed as it is.+ if (frames > kSGSingRate / 4) return pull(context, frames, left, right);+ bool on = atomic_load_explicit(&engine->on, memory_order_relaxed);+ if (!engine->running) {+ if (!on) return pull(context, frames, left, right);+ engine->running = true;+ startOver(engine);+ atomic_store(&engine->flushAsked, false);+ }+ if (atomic_exchange_explicit(&engine->flushAsked, false, memory_order_relaxed)) startOver(engine);++ uint64_t written = atomic_load_explicit(&engine->written, memory_order_relaxed);+ uint64_t played = atomic_load_explicit(&engine->played, memory_order_relaxed);+ uint64_t held = written - played;+ bool working = on && !atomic_load_explicit(&engine->paused, memory_order_relaxed)+ && atomic_load_explicit(&engine->hasSeparator, memory_order_relaxed);+ int64_t want = (working ? (int64_t)atomic_load_explicit(&engine->targetLead, memory_order_relaxed) : 0) - (int64_t)held;+ // Up to twice what plays while the lead fills, half while it is given back.+ int64_t count = (int64_t)frames + MAX(-(int64_t)frames / 2, MIN(want, (int64_t)frames));+ count = MAX(count, (int64_t)frames - (int64_t)held);+ count = MIN(count, (int64_t)(kRing - kSGSingWindowFrames) - (int64_t)held);+ if (count > 0) {+ OSStatus status = pullInto(engine, written, (uint64_t)count, pull, context);+ if (status != noErr) {+ memset(left, 0, frames * sizeof(float));+ memset(right, 0, frames * sizeof(float));+ return status;+ }+ written += (uint64_t)count;+ atomic_store_explicit(&engine->written, written, memory_order_release);+ }++ uint64_t ready = atomic_load_explicit(&engine->ready, memory_order_acquire);+ float vocalsTo, otherTo;+ gainsFor(loadFloat(&engine->levelBits), &vocalsTo, &otherTo);+ float vocalsStep = (vocalsTo - engine->vocalsGain) / frames, otherStep = (otherTo - engine->otherGain) / frames;+ float fade = 1.0f / (kFadeSeconds * kSGSingRate), amountTo = on ? 1 : 0;+ uint64_t dry = 0;+ for (UInt32 i = 0; i < frames; i++) {+ uint64_t position = played + i, index = position % kRing;+ float x = engine->left[index], y = engine->right[index];+ engine->vocalsGain += vocalsStep;+ engine->otherGain += otherStep;+ if (position < ready) {+ engine->amount += fmaxf(-fade, fminf(amountTo - engine->amount, fade));+ float a = engine->amount, vocalsGain = engine->vocalsGain - 1, otherGain = engine->otherGain - 1;+ float v = engine->vocalsLeft[index], w = engine->vocalsRight[index];+ left[i] = x + a * (vocalsGain * v + otherGain * (x - v));+ right[i] = y + a * (vocalsGain * w + otherGain * (y - w));+ } else {+ engine->amount = 0;+ left[i] = x;+ right[i] = y;+ dry += on;+ }+ }+ engine->vocalsGain = vocalsTo;+ engine->otherGain = otherTo;+ played += frames;+ atomic_store_explicit(&engine->played, played, memory_order_release);+ if (dry) atomic_fetch_add_explicit(&engine->dryFrames, dry, memory_order_relaxed);+ // The lead given back and the vocals faded out: a straight pull from the next render on.+ if (!on && played == written) engine->running = false;+ return noErr;+}++#pragma mark - the worker++static void setError(SGSingEngine *engine, NSString *error) {+ os_unfair_lock_lock(&engine->lock);+ engine->error = error;+ os_unfair_lock_unlock(&engine->lock);+}++static SGSingSeparator *currentSeparator(SGSingEngine *engine) {+ os_unfair_lock_lock(&engine->lock);+ SGSingSeparator *separator = (__bridge SGSingSeparator *)engine->separator;+ os_unfair_lock_unlock(&engine->lock);+ return separator;+}++// The lead that keeps the vocals in ahead of what plays: a window, then the model's time and some spare.+static void updateTarget(SGSingEngine *engine, double modelSeconds) {+ double lead = kSGSingWindowFrames + (modelSeconds * kLeadPerModelTime + kLeadSpare) * kSGSingRate;+ atomic_store(&engine->targetLead, (unsigned)fmax(kShortestLead, fmin(lead, kLongestLead)));+}++static void *work(void *argument) {+ SGSingEngine *engine = argument;+ pthread_setname_np("spotifyglass.sing");+ unsigned generation = UINT_MAX;+ uint64_t start = 0, previousEnd = 0;+ while (atomic_load(&engine->alive)) {+ unsigned now = atomic_load(&engine->generation);+ if (now != generation) {+ generation = now;+ start = previousEnd = atomic_load(&engine->base);+ atomic_store_explicit(&engine->ready, start, memory_order_release);+ }+ SGSingSeparator *separator = currentSeparator(engine);+ uint64_t written = atomic_load_explicit(&engine->written, memory_order_acquire);+ uint64_t played = atomic_load_explicit(&engine->played, memory_order_relaxed);+ if (!separator || !atomic_load(&engine->on) || atomic_load(&engine->paused) || written < start + kSGSingWindowFrames) {+ usleep(kPoll);+ continue;+ }+ // Fallen behind what plays: the next window starts where the sound is, with nothing to crossfade with.+ if (start < played) start = previousEnd = played;+ if (written < start + kSGSingWindowFrames) continue;+ for (int c = 0; c < 2; c++) {+ const float *ring = c ? engine->right : engine->left;+ uint64_t index = start % kRing, run = MIN((uint64_t)kSGSingWindowFrames, kRing - index);+ memcpy(engine->window[c], ring + index, run * sizeof(float));+ memcpy(engine->window[c] + run, ring, (kSGSingWindowFrames - run) * sizeof(float));+ }+ CFAbsoluteTime began = CFAbsoluteTimeGetCurrent();+ NSError *error;+ BOOL separated;+ @autoreleasepool {+ separated = [separator separateLeft:engine->window[0] right:engine->window[1] vocalsLeft:engine->vocals[0]+ vocalsRight:engine->vocals[1] error:&error];+ }+ double took = CFAbsoluteTimeGetCurrent() - began;+ if (!separated) {+ atomic_fetch_add(&engine->failures, 1);+ setError(engine, error.localizedDescription ?: @"The model failed");+ usleep(1000000);+ continue;+ }+ setError(engine, nil);+ uint64_t windows = atomic_fetch_add(&engine->windows, 1);+ uint64_t average = atomic_load(&engine->averageMicroseconds);+ average = windows ? (uint64_t)(average * 0.8 + took * 1e6 * 0.2) : (uint64_t)(took * 1e6);+ atomic_store(&engine->averageMicroseconds, average);+ updateTarget(engine, average / 1e6);+ if (atomic_load(&engine->generation) != generation) continue; // flushed meanwhile: the window is gone+ // The rings past `ready` are the worker's: the render thread reads only below it.+ for (uint64_t i = 0; i < kSGSingWindowFrames; i++) {+ uint64_t position = start + i, index = position % kRing;+ float a = position < previousEnd ? (float)(position - start) / (float)(previousEnd - start) : 1;+ engine->vocalsLeft[index] = engine->vocalsLeft[index] * (1 - a) + engine->vocals[0][i] * a;+ engine->vocalsRight[index] = engine->vocalsRight[index] * (1 - a) + engine->vocals[1][i] * a;+ }+ if (atomic_load(&engine->generation) == generation) atomic_store_explicit(&engine->ready, start + kSGSingEngineHop, memory_order_release);+ previousEnd = start + kSGSingWindowFrames;+ start += kSGSingEngineHop;+ }+ return NULL;+}++#pragma mark - any thread++SGSingEngine *SGSingEngineCreate(void) {+ SGSingEngine *engine = calloc(1, sizeof *engine);+ if (!engine) return NULL;+ engine->lock = OS_UNFAIR_LOCK_INIT;+ engine->left = calloc(kRing, sizeof(float));+ engine->right = calloc(kRing, sizeof(float));+ engine->vocalsLeft = calloc(kRing, sizeof(float));+ engine->vocalsRight = calloc(kRing, sizeof(float));+ for (int c = 0; c < 2; c++) {+ engine->window[c] = calloc(kSGSingWindowFrames, sizeof(float));+ engine->vocals[c] = calloc(kSGSingWindowFrames, sizeof(float));+ }+ engine->vocalsGain = engine->otherGain = 1;+ storeFloat(&engine->levelBits, 1);+ updateTarget(engine, 1);+ atomic_store(&engine->alive, true);+ if (!engine->left || !engine->right || !engine->vocalsLeft || !engine->vocalsRight || !engine->window[0] || !engine->window[1]+ || !engine->vocals[0] || !engine->vocals[1] || pthread_create(&engine->worker, NULL, work, engine) != 0) {+ atomic_store(&engine->alive, false);+ engine->worker = NULL;+ SGSingEngineDestroy(engine);+ return NULL;+ }+ return engine;+}++void SGSingEngineDestroy(SGSingEngine *engine) {+ if (!engine) return;+ atomic_store(&engine->alive, false);+ if (engine->worker) pthread_join(engine->worker, NULL);+ if (engine->separator) CFRelease(engine->separator);+ engine->error = nil;+ for (int c = 0; c < 2; c++) {+ free(engine->window[c]);+ free(engine->vocals[c]);+ }+ free(engine->left);+ free(engine->right);+ free(engine->vocalsLeft);+ free(engine->vocalsRight);+ free(engine);+}++void SGSingEngineSetSeparator(SGSingEngine *engine, SGSingSeparator *separator) {+ void *retained = separator ? (void *)CFBridgingRetain(separator) : NULL;+ os_unfair_lock_lock(&engine->lock);+ void *old = engine->separator;+ engine->separator = retained;+ os_unfair_lock_unlock(&engine->lock);+ atomic_store(&engine->hasSeparator, retained != NULL);+ // Released outside the lock; the worker holds its own reference while it runs one.+ if (old) CFRelease(old);+}++void SGSingEngineSetOn(SGSingEngine *engine, bool on) {+ atomic_store(&engine->on, on);+}++bool SGSingEngineOn(SGSingEngine *engine) {+ return atomic_load(&engine->on);+}++void SGSingEngineSetLevel(SGSingEngine *engine, float level) {+ storeFloat(&engine->levelBits, level);+}++void SGSingEngineSetPaused(SGSingEngine *engine, bool paused) {+ atomic_store(&engine->paused, paused);+}++void SGSingEngineFlush(SGSingEngine *engine) {+ atomic_store(&engine->flushAsked, true);+}++double SGSingEngineLead(SGSingEngine *engine) {+ if (!engine) return 0;+ uint64_t played = atomic_load_explicit(&engine->played, memory_order_relaxed);+ uint64_t written = atomic_load_explicit(&engine->written, memory_order_relaxed);+ return written > played ? (double)(written - played) / kSGSingRate : 0;+}++SGSingEngineStats SGSingEngineReadStats(SGSingEngine *engine) {+ uint64_t played = atomic_load(&engine->played), ready = atomic_load(&engine->ready);+ return (SGSingEngineStats){+ .lead = SGSingEngineLead(engine),+ .targetLead = atomic_load(&engine->targetLead) / (double)kSGSingRate,+ .ready = ((double)ready - (double)played) / kSGSingRate,+ .averageMS = atomic_load(&engine->averageMicroseconds) / 1000.0,+ .windows = atomic_load(&engine->windows),+ .failures = atomic_load(&engine->failures),+ .dryFrames = atomic_load(&engine->dryFrames),+ };+}++NSString *SGSingEngineError(SGSingEngine *engine) {+ os_unfair_lock_lock(&engine->lock);+ NSString *error = engine->error;+ os_unfair_lock_unlock(&engine->lock);+ return error;+}diff --git a/tweak/Sources/Shared/Sing/SGSingModel.m b/tweak/Sources/Shared/Sing/SGSingModel.mnew file mode 100644index 00000000..3a0c864b--- /dev/null+++ b/tweak/Sources/Shared/Sing/SGSingModel.m@@ -0,0 +1,367 @@+// The voice model on the phone (Sing.h): the five files of its compiled separator.mlmodelc, downloaded one by one+// from the Hugging Face repo, each kept only once its size and SHA-256 are the ones pinned here, then moved+// together into Application Support/Vitrine/Sing/separator.mlmodelc, which backups leave out (489 MB that can+// be downloaded again). A dropped connection carries on from where it stopped, three times per file, and a stopped+// download keeps the file it was in the middle of (NSURLSession's resume data, kept beside the staging folder)+// for the next one to carry on from. A resumed request is answered 206, with the whole file handed over all the same.+//+// Threading: main thread; the session's delegate runs on a queue of its own and hands back to the main thread.+#import <CommonCrypto/CommonDigest.h>+#import "Core/SGCore.h"+#import "Sing.h"++static NSString *const kRepo = @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing/resolve/main/";+static const int kRetries = 3;++typedef struct {+ NSString *__unsafe_unretained path;+ long long size;+ const char *sha256;+} SGSingFile;++// The model card's table (huggingface.co/My-Name-Is-Jeff/vitrine-sing, a mirror of Darkkos/spoti-sing), the weights last so the small files fail first.+static const SGSingFile kFiles[] = {+ {@"metadata.json", 2431, "52a8d5e3f09e33236d495dbed5bbce1c75bac6f2a6b6097637cf214f37c1de53"},+ {@"coremldata.bin", 507, "2090acaf7a6df72ec83857cb88d101654023a6baad222a25d0173827d3347e28"},+ {@"analytics/coremldata.bin", 243, "f7ee4ec9b5cc1c97171bd5aad93af61e183aa0db1451ef3b775bd4e77f0b7cfd"},+ {@"model.mil", 669061, "966560ed5125174a98f19b94f5de04450a7112ade0e731f2236c202c0280a623"},+ {@"weights/weight.bin", 488986336, "970a99fb4b15724bf76d2918ceb177df592c69265d3e2fabaab6e5ba72738e62"},+};+enum { kFileCount = sizeof kFiles / sizeof kFiles[0] };++NSString *const SGSingChangedNotification = @"SGSingChangedNotification";++// Counts the changes to the model's folder, so SGSingModelURL checks it again only after one.+static NSUInteger sg_modelGeneration;++static long long totalBytes(void) {+ long long total = 0;+ for (int i = 0; i < kFileCount; i++) total += kFiles[i].size;+ return total;+}++static NSURL *singFolder(void) {+ NSURL *support = [NSFileManager.defaultManager URLsForDirectory:NSApplicationSupportDirectory inDomains:NSUserDomainMask].firstObject;+ return [[support URLByAppendingPathComponent:@"Vitrine" isDirectory:YES] URLByAppendingPathComponent:@"Sing" isDirectory:YES];+}++static NSURL *modelFolder(void) {+ return [singFolder() URLByAppendingPathComponent:@"separator.mlmodelc" isDirectory:YES];+}++static NSURL *stagingFolder(void) {+ return [singFolder() URLByAppendingPathComponent:@"download" isDirectory:YES];+}++// A stopped download's resume data for file `index`, beside the staging folder: everything in that folder is moved+// into the model.+static NSURL *resumeFile(int index) {+ return [singFolder() URLByAppendingPathComponent:[NSString stringWithFormat:@"download-%d.resume", index]];+}++static long long sizeOf(NSURL *url) {+ return [[NSFileManager.defaultManager attributesOfItemAtPath:url.path error:nil][NSFileSize] longLongValue];+}++// Whether every file is in `folder` at its size; the hashes were checked as each came in.+static BOOL complete(NSURL *folder, int upTo) {+ for (int i = 0; i < upTo; i++) {+ if (sizeOf([folder URLByAppendingPathComponent:kFiles[i].path]) != kFiles[i].size) return NO;+ }+ return YES;+}++// The file's SHA-256, read a megabyte at a time: the weights are too big to read whole.+static NSString *sha256Of(NSURL *url) {+ NSFileHandle *handle = [NSFileHandle fileHandleForReadingFromURL:url error:nil];+ if (!handle) return nil;+ CC_SHA256_CTX context;+ CC_SHA256_Init(&context);+ for (;;) {+ @autoreleasepool {+ NSData *chunk = [handle readDataUpToLength:1 << 20 error:nil];+ if (!chunk.length) break;+ CC_SHA256_Update(&context, chunk.bytes, (CC_LONG)chunk.length);+ }+ }+ [handle closeFile];+ unsigned char digest[CC_SHA256_DIGEST_LENGTH];+ CC_SHA256_Final(digest, &context);+ NSMutableString *hex = [NSMutableString stringWithCapacity:2 * CC_SHA256_DIGEST_LENGTH];+ for (int i = 0; i < CC_SHA256_DIGEST_LENGTH; i++) [hex appendFormat:@"%02x", digest[i]];+ return hex;+}++static void announce(void) {+ [NSNotificationCenter.defaultCenter postNotificationName:SGSingChangedNotification object:nil];+}++#pragma mark - the download++@interface SGSingDownloader : NSObject <NSURLSessionDownloadDelegate>+@property (nonatomic) int file; // the index of the file coming in+@property (nonatomic) long long received; // its bytes so far+@property (nonatomic, copy) NSString *error;+@property (atomic, strong) NSURLSessionDownloadTask *task; // the one running, so a stop (on the main thread) can keep what it has+@end++static SGSingDownloader *sg_downloader; // while a download runs+static NSString *sg_lastError;+static double sg_progress;+static BOOL sg_waitingForNetwork; // offline: the session holds the request until the iPhone is online++@implementation SGSingDownloader {+ NSURLSession *_session;+ BOOL _resumed; // it carries on from resume data+ BOOL _restart; // resume data the server no longer takes: the file again from the start+ int _attempts;+ BOOL _cancelled;+}++- (void)start {+ NSURLSessionConfiguration *configuration = NSURLSessionConfiguration.defaultSessionConfiguration;+ configuration.timeoutIntervalForRequest = 60;+ // Offline, a request (a retry too) waits for the network rather than failing at once.+ configuration.waitsForConnectivity = YES;+ NSOperationQueue *queue = [NSOperationQueue new];+ queue.maxConcurrentOperationCount = 1;+ _session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:queue];+ NSError *error;+ [NSFileManager.defaultManager createDirectoryAtURL:stagingFolder() withIntermediateDirectories:YES attributes:nil error:&error];+ NSURL *folder = singFolder();+ [folder setResourceValue:@YES forKey:NSURLIsExcludedFromBackupKey error:nil];+ [queue addOperationWithBlock:^{ [self next]; }];+}++// Stopped, the file coming in is kept as resume data for the next download; the session goes once it is written.+- (void)cancel {+ _cancelled = YES;+ NSURLSession *session = _session;+ NSURLSessionDownloadTask *task = self.task;+ if (!task) {+ [session invalidateAndCancel];+ return;+ }+ NSURL *resume = resumeFile(self.file);+ [task cancelByProducingResumeData:^(NSData *data) {+ if (data) [data writeToURL:resume atomically:YES];+ SGLog(@"sing: the download is stopped%@", data ? @", what came in is kept" : @"");+ [session invalidateAndCancel];+ }];+}++- (void)download:(NSURL *)url resumeData:(NSData *)resume {+ _resumed = resume != nil;+ NSURLSessionDownloadTask *task = resume ? [_session downloadTaskWithResumeData:resume] : [_session downloadTaskWithURL:url];+ self.task = task;+ [task resume];+}++- (NSURL *)fileURL {+ return [NSURL URLWithString:[kRepo stringByAppendingString:kFiles[self.file].path]];+}++// The first file not already in the staging folder from an earlier try, or the move into place.+- (void)next {+ if (_cancelled) return;+ while (self.file < kFileCount && sizeOf([stagingFolder() URLByAppendingPathComponent:kFiles[self.file].path]) == kFiles[self.file].size) self.file++;+ self.received = 0;+ _attempts = 0;+ [self report];+ if (self.file == kFileCount) {+ [self finish];+ return;+ }+ // A stop's resume data is used once: if it fails, the file starts over.+ NSData *resume = [NSData dataWithContentsOfURL:resumeFile(self.file)];+ [NSFileManager.defaultManager removeItemAtURL:resumeFile(self.file) error:nil];+ [self download:[self fileURL] resumeData:resume];+ SGLog(@"sing: downloading %@%@", kFiles[self.file].path, resume ? @", carrying on from the last download" : @"");+}++- (void)finish {+ NSFileManager *files = NSFileManager.defaultManager;+ // Resume data a stop wrote after the next download had already looked for it.+ for (int i = 0; i < kFileCount; i++) [files removeItemAtURL:resumeFile(i) error:nil];+ [files removeItemAtURL:modelFolder() error:nil];+ NSError *error;+ BOOL moved = [files moveItemAtURL:stagingFolder() toURL:modelFolder() error:&error];+ [self endWithError:moved ? nil : [NSString stringWithFormat:@"The model could not be put in place (%@)", error.localizedDescription]];+}++- (void)endWithError:(NSString *)error {+ [_session finishTasksAndInvalidate];+ if (error) SGLog(@"sing: the model's download failed: %@", error);+ else SGLog(@"sing: the model is downloaded and checked");+ dispatch_async(dispatch_get_main_queue(), ^{+ if (sg_downloader != self) return;+ sg_downloader = nil;+ sg_waitingForNetwork = NO;+ sg_lastError = error;+ sg_modelGeneration++;+ announce();+ });+}++- (void)report {+ long long done = self.received;+ for (int i = 0; i < self.file && i < kFileCount; i++) done += kFiles[i].size;+ double progress = (double)done / totalBytes();+ // A few times a second is enough for the row and the button.+ static CFAbsoluteTime reported;+ CFAbsoluteTime now = CFAbsoluteTimeGetCurrent();+ if (now - reported < 0.25 && progress < 1) return;+ reported = now;+ dispatch_async(dispatch_get_main_queue(), ^{+ sg_progress = progress;+ sg_waitingForNetwork = NO;+ announce();+ });+}++- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)task didWriteData:(int64_t)written+ totalBytesWritten:(int64_t)totalWritten totalBytesExpectedToWrite:(int64_t)expected {+ self.received = MIN(totalWritten, kFiles[self.file].size);+ [self report];+}++- (void)URLSession:(NSURLSession *)session taskIsWaitingForConnectivity:(NSURLSessionTask *)task {+ SGLog(@"sing: waiting for the network to download %@", kFiles[self.file].path);+ dispatch_async(dispatch_get_main_queue(), ^{+ if (sg_downloader != self) return;+ sg_waitingForNetwork = YES;+ announce();+ });+}++// The file is checked here, before the session deletes it, and kept in the staging folder if it is the one+// pinned.+- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)task didFinishDownloadingToURL:(NSURL *)location {+ SGSingFile file = kFiles[self.file];+ NSInteger status = [task.response isKindOfClass:NSHTTPURLResponse.class] ? ((NSHTTPURLResponse *)task.response).statusCode : 0;+ BOOL answered = status == 200 || status == 206;+ if (!answered && _resumed) {+ // Resume data the server no longer takes, its signed address run out after a day: the file again.+ SGLog(@"sing: the server answered %ld to carrying on %@, so it starts over", (long)status, file.path);+ _restart = YES;+ return;+ }+ long long size = sizeOf(location);+ NSString *hash = size == file.size ? sha256Of(location) : nil;+ SGLog(@"sing: %@ came in, %ld, %lld bytes", file.path, (long)status, size);+ if (!answered || size != file.size || ![hash isEqualToString:@(file.sha256)]) {+ self.error = !answered ? [NSString stringWithFormat:@"The server answered %ld for %@", (long)status, file.path]+ : [NSString stringWithFormat:@"%@ is not the file Sing expects (%lld bytes%@)", file.path, size, hash ? @", another checksum" : @""];+ return;+ }+ NSURL *target = [stagingFolder() URLByAppendingPathComponent:file.path];+ [NSFileManager.defaultManager createDirectoryAtURL:target.URLByDeletingLastPathComponent withIntermediateDirectories:YES attributes:nil error:nil];+ [NSFileManager.defaultManager removeItemAtURL:target error:nil];+ NSError *error;+ if (![NSFileManager.defaultManager moveItemAtURL:location toURL:target error:&error]) {+ self.error = [NSString stringWithFormat:@"%@ could not be kept (%@)", file.path, error.localizedDescription];+ }+}++- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error {+ if (_cancelled) return;+ if (_restart) {+ _restart = NO;+ [self download:[self fileURL] resumeData:nil];+ return;+ }+ if (self.error) {+ NSString *message = self.error;+ self.error = nil;+ [self endWithError:message];+ return;+ }+ if (!error) {+ self.file++;+ [self next];+ return;+ }+ NSData *resume = error.userInfo[NSURLSessionDownloadTaskResumeData];+ if (++_attempts <= kRetries) {+ SGLog(@"sing: %@ stopped (%@), trying again%@", kFiles[self.file].path, error.localizedDescription, resume ? @" from where it was" : @"");+ [self download:[self fileURL] resumeData:resume];+ return;+ }+ [self endWithError:error.localizedDescription];+}++@end++#pragma mark - what Sing asks++SGSingModelState SGSingModelCurrentState(void) {+ if (sg_downloader) return SGSingModelDownloading;+ return SGSingModelURL() ? SGSingModelReady : SGSingModelMissing;+}++NSURL *SGSingModelURL(void) {+ // Checked by size once a launch, and again after a download or a delete changes it.+ static int known = -1;+ static NSUInteger knownGeneration = NSUIntegerMax;+ if (knownGeneration != sg_modelGeneration) {+ knownGeneration = sg_modelGeneration;+ known = complete(modelFolder(), kFileCount);+ }+ return known ? modelFolder() : nil;+}++double SGSingModelProgress(void) {+ return sg_downloader ? sg_progress : SGSingModelURL() ? 1 : 0;+}++BOOL SGSingModelWaitingForNetwork(void) {+ return sg_downloader && sg_waitingForNetwork;+}++NSString *SGSingModelError(void) {+ return sg_lastError;+}++NSString *SGSingModelSizeText(void) {+ return [NSByteCountFormatter stringFromByteCount:totalBytes() countStyle:NSByteCountFormatterCountStyleFile];+}++void SGSingDownloadModel(void) {+ if (sg_downloader || SGSingModelURL()) return;+ sg_lastError = nil;+ sg_progress = 0;+ sg_waitingForNetwork = NO;+ sg_downloader = [SGSingDownloader new];+ [sg_downloader start];+ announce();+}++void SGSingCancelModelDownload(void) {+ [sg_downloader cancel];+ sg_downloader = nil;+ sg_modelGeneration++;+ announce();+}++void SGSingDeleteModel(void) {+ SGSingCancelModelDownload();+ [NSFileManager.defaultManager removeItemAtURL:singFolder() error:nil];+ sg_modelGeneration++;+ announce();+}++NSArray<NSString *> *SGSingComputeUnitNames(void) {+ return @[@"GPU", @"GPU and Neural Engine", @"Neural Engine"];+}++BOOL SGSingOSSupported(void) {+ if (@available(iOS 18.0, *)) return YES;+ return NO;+}++// The model is 489 MB of weights, held while Sing is on beside Spotify's own memory: an iPhone with less than+// 6 GB (which report a little under 6) would have Spotify closed under it.+// ponytail: a memory floor stands in for a list of phones; measure on a 6 GB phone and a 4 GB one to place it.+BOOL SGSingDeviceSupported(void) {+ return NSProcessInfo.processInfo.physicalMemory >= 5ull * 1000 * 1000 * 1000;+}diff --git a/tweak/Sources/Shared/Sing/SGSingSeparator.h b/tweak/Sources/Shared/Sing/SGSingSeparator.hnew file mode 100644index 00000000..512b5f35--- /dev/null+++ b/tweak/Sources/Shared/Sing/SGSingSeparator.h@@ -0,0 +1,40 @@+// Sing's separator: two seconds of a song in, its vocals out, through the voice model (Mel-Band RoFormer+// with KimberleyJensen's vocal checkpoint, MIT; Sing.h says where it comes from). The model is the spectral+// core only, so the STFT around it is done here, as the checkpoint's reference implementation does it+// (its configs/config_vocals_mel_band_roformer.yaml, and model.mil read for the layout):+//+// 44.1 kHz stereo, n_fft 2048, hop 441, a periodic Hann window of 2048, unnormalized, centred with+// reflect padding (torch.stft's defaults), so two seconds (88200 frames) make 201 frames of 1025 bins.+// The model's `spectrum` and `vocals_spectrum` are float32 [1, 2050, 201, 2]: axis 1 is bin-major with+// the two channels interleaved (bin f of channel c at 2f + c, which its band split's gather indices+// pair up), the last axis real then imaginary (its complex mask multiply reads them so).+//+// The vocals come back from the model's complex mask times the spectrum, through the inverse STFT+// (torch.istft: overlap-add of the windowed frames divided by the window's squared sum).+//+// Threading: one separator is used by one thread at a time (its buffers are its own).+#import <Foundation/Foundation.h>++@class MLModel;++enum {+ kSGSingRate = 44100,+ kSGSingWindowFrames = 88200, // two seconds, what the model takes+ kSGSingFFT = 2048,+ kSGSingHop = 441,+ kSGSingBins = kSGSingFFT / 2 + 1,+ kSGSingSTFTFrames = kSGSingWindowFrames / kSGSingHop + 1,+ kSGSingSpectrumFloats = kSGSingBins * 2 * kSGSingSTFTFrames * 2,+};++@interface SGSingSeparator : NSObject+// Nil `model` makes one that only does the STFT, for the harness.+- (instancetype)initWithModel:(MLModel *)model;+// The window's spectrum in the model's layout, `spectrum` holding kSGSingSpectrumFloats.+- (void)analyzeLeft:(const float *)left right:(const float *)right into:(float *)spectrum;+// The inverse: kSGSingWindowFrames frames per channel out of a spectrum in the model's layout.+- (void)synthesize:(const float *)spectrum left:(float *)left right:(float *)right;+// The vocals of kSGSingWindowFrames frames of stereo; NO and `error` when the model failed.+- (BOOL)separateLeft:(const float *)left right:(const float *)right vocalsLeft:(float *)vocalsLeft+ vocalsRight:(float *)vocalsRight error:(NSError **)error;+@enddiff --git a/tweak/Sources/Shared/Sing/SGSingSeparator.m b/tweak/Sources/Shared/Sing/SGSingSeparator.mnew file mode 100644index 00000000..556bb9e1--- /dev/null+++ b/tweak/Sources/Shared/Sing/SGSingSeparator.m@@ -0,0 +1,176 @@+#import <Accelerate/Accelerate.h>+#import <CoreML/CoreML.h>+#import "SGSingSeparator.h"++enum {+ kPad = kSGSingFFT / 2, // the centring's reflect padding on each side+ kPadded = kSGSingWindowFrames + 2 * kPad,+ kHalf = kSGSingFFT / 2,+};++static NSString *const kInput = @"spectrum", *const kOutput = @"vocals_spectrum";++// Where bin `bin` of channel `channel` at STFT frame `frame` starts in the model's layout (its real part;+// the imaginary part follows it).+static inline size_t at(int bin, int channel, int frame) {+ return ((size_t)(2 * bin + channel) * kSGSingSTFTFrames + frame) * 2;+}++@implementation SGSingSeparator {+ MLModel *_model;+ MLMultiArray *_input;+ vDSP_DFT_Setup _forward, _inverse;+ float _window[kSGSingFFT];+ float _envelope[kSGSingWindowFrames]; // 1 / the squared window's overlap-add, per output frame+ float _padded[kPadded];+ float _frame[kSGSingFFT];+ float _real[kHalf], _imag[kHalf], _outReal[kHalf], _outImag[kHalf];+ float _sum[kPadded];+ float *_spectrum, *_vocals;+}++- (instancetype)initWithModel:(MLModel *)model {+ if (!(self = [super init])) return nil;+ _model = model;+ _forward = vDSP_DFT_zrop_CreateSetup(NULL, kSGSingFFT, vDSP_DFT_FORWARD);+ _inverse = vDSP_DFT_zrop_CreateSetup(_forward, kSGSingFFT, vDSP_DFT_INVERSE);+ _spectrum = calloc(kSGSingSpectrumFloats, sizeof(float));+ _vocals = calloc(kSGSingSpectrumFloats, sizeof(float));+ if (!_forward || !_inverse || !_spectrum || !_vocals) return nil;+ // torch.hann_window's default, periodic: the denominator is N, not N - 1.+ for (int n = 0; n < kSGSingFFT; n++) _window[n] = 0.5f * (1 - cosf(2 * (float)M_PI * n / kSGSingFFT));+ float squares[kSGSingFFT];+ vDSP_vsq(_window, 1, squares, 1, kSGSingFFT);+ memset(_sum, 0, sizeof _sum);+ for (int t = 0; t < kSGSingSTFTFrames; t++) vDSP_vadd(_sum + t * kSGSingHop, 1, squares, 1, _sum + t * kSGSingHop, 1, kSGSingFFT);+ for (int i = 0; i < kSGSingWindowFrames; i++) _envelope[i] = 1 / _sum[i + kPad];+ if (model) {+ NSError *error;+ _input = [[MLMultiArray alloc] initWithDataPointer:_spectrum shape:@[@1, @(2 * kSGSingBins), @(kSGSingSTFTFrames), @2]+ dataType:MLMultiArrayDataTypeFloat32+ strides:@[@(2 * kSGSingBins * kSGSingSTFTFrames * 2), @(kSGSingSTFTFrames * 2), @2, @1]+ deallocator:nil error:&error];+ if (!_input) return nil;+ }+ return self;+}++- (void)dealloc {+ if (_inverse) vDSP_DFT_DestroySetup(_inverse);+ if (_forward) vDSP_DFT_DestroySetup(_forward);+ free(_spectrum);+ free(_vocals);+}++#pragma mark - the STFT++- (void)analyzeChannel:(const float *)samples channel:(int)channel into:(float *)spectrum {+ // Reflect padding, the edge sample itself not repeated (torch's "reflect").+ memcpy(_padded + kPad, samples, kSGSingWindowFrames * sizeof(float));+ for (int i = 1; i <= kPad; i++) {+ _padded[kPad - i] = samples[i];+ _padded[kPad + kSGSingWindowFrames - 1 + i] = samples[kSGSingWindowFrames - 1 - i];+ }+ DSPSplitComplex split = {_real, _imag};+ for (int t = 0; t < kSGSingSTFTFrames; t++) {+ vDSP_vmul(_padded + t * kSGSingHop, 1, _window, 1, _frame, 1, kSGSingFFT);+ vDSP_ctoz((const DSPComplex *)_frame, 2, &split, 1, kHalf);+ vDSP_DFT_Execute(_forward, _real, _imag, _outReal, _outImag);+ // vDSP's real transform comes out twice the DFT, with the Nyquist bin's real part where the DC+ // bin's imaginary part would be.+ float *dc = spectrum + at(0, channel, t), *nyquist = spectrum + at(kHalf, channel, t);+ dc[0] = _outReal[0] * 0.5f;+ dc[1] = 0;+ nyquist[0] = _outImag[0] * 0.5f;+ nyquist[1] = 0;+ for (int k = 1; k < kHalf; k++) {+ float *bin = spectrum + at(k, channel, t);+ bin[0] = _outReal[k] * 0.5f;+ bin[1] = _outImag[k] * 0.5f;+ }+ }+}++- (void)analyzeLeft:(const float *)left right:(const float *)right into:(float *)spectrum {+ [self analyzeChannel:left channel:0 into:spectrum];+ [self analyzeChannel:right channel:1 into:spectrum];+}++- (void)synthesizeChannel:(int)channel from:(const float *)spectrum into:(float *)samples {+ memset(_sum, 0, sizeof _sum);+ DSPSplitComplex split = {_outReal, _outImag};+ // The inverse of a true DFT comes out N times the signal, so 1/N with the window applied again.+ float scale = 1.0f / kSGSingFFT;+ for (int t = 0; t < kSGSingSTFTFrames; t++) {+ _real[0] = spectrum[at(0, channel, t)];+ _imag[0] = spectrum[at(kHalf, channel, t)];+ for (int k = 1; k < kHalf; k++) {+ const float *bin = spectrum + at(k, channel, t);+ _real[k] = bin[0];+ _imag[k] = bin[1];+ }+ vDSP_DFT_Execute(_inverse, _real, _imag, _outReal, _outImag);+ vDSP_ztoc(&split, 1, (DSPComplex *)_frame, 2, kHalf);+ vDSP_vmul(_frame, 1, _window, 1, _frame, 1, kSGSingFFT);+ vDSP_vsma(_frame, 1, &scale, _sum + t * kSGSingHop, 1, _sum + t * kSGSingHop, 1, kSGSingFFT);+ }+ vDSP_vmul(_sum + kPad, 1, _envelope, 1, samples, 1, kSGSingWindowFrames);+}++- (void)synthesize:(const float *)spectrum left:(float *)left right:(float *)right {+ [self synthesizeChannel:0 from:spectrum into:left];+ [self synthesizeChannel:1 from:spectrum into:right];+}++#pragma mark - the model++// The model's output into `_vocals`, by its strides, whatever they are and whether it is float32 or 16.+static BOOL readOutput(MLMultiArray *array, float *into) {+ NSArray<NSNumber *> *shape = array.shape, *strides = array.strides;+ if (shape.count != 4 || shape[1].intValue != 2 * kSGSingBins || shape[2].intValue != kSGSingSTFTFrames || shape[3].intValue != 2) return NO;+ MLMultiArrayDataType type = array.dataType;+ if (type != MLMultiArrayDataTypeFloat32 && type != MLMultiArrayDataTypeFloat16) return NO;+ NSInteger s1 = strides[1].integerValue, s2 = strides[2].integerValue, s3 = strides[3].integerValue;+ __block BOOL ok = YES;+ [array getBytesWithHandler:^(const void *bytes, NSInteger size) {+ NSInteger elementSize = type == MLMultiArrayDataTypeFloat32 ? 4 : 2;+ NSInteger last = ((2 * kSGSingBins - 1) * s1 + (kSGSingSTFTFrames - 1) * s2 + s3) * elementSize;+ if (last >= size) {+ ok = NO;+ return;+ }+ BOOL contiguous = s3 == 1 && s2 == 2 && s1 == kSGSingSTFTFrames * 2;+ if (contiguous && type == MLMultiArrayDataTypeFloat32) {+ memcpy(into, bytes, kSGSingSpectrumFloats * sizeof(float));+ return;+ }+ for (NSInteger row = 0; row < 2 * kSGSingBins; row++) {+ for (NSInteger frame = 0; frame < kSGSingSTFTFrames; frame++) {+ for (NSInteger part = 0; part < 2; part++) {+ NSInteger index = row * s1 + frame * s2 + part * s3;+ float value = type == MLMultiArrayDataTypeFloat32 ? ((const float *)bytes)[index] : (float)((const _Float16 *)bytes)[index];+ into[(row * kSGSingSTFTFrames + frame) * 2 + part] = value;+ }+ }+ }+ }];+ return ok;+}++- (BOOL)separateLeft:(const float *)left right:(const float *)right vocalsLeft:(float *)vocalsLeft+ vocalsRight:(float *)vocalsRight error:(NSError **)error {+ if (!_model) return NO;+ [self analyzeLeft:left right:right into:_spectrum];+ MLDictionaryFeatureProvider *features = [[MLDictionaryFeatureProvider alloc] initWithDictionary:@{kInput: _input} error:error];+ if (!features) return NO;+ id<MLFeatureProvider> result = [_model predictionFromFeatures:features error:error];+ MLMultiArray *output = [result featureValueForName:kOutput].multiArrayValue;+ if (!output || !readOutput(output, _vocals)) {+ if (error && !*error) *error = [NSError errorWithDomain:@"SGSing" code:1 userInfo:@{NSLocalizedDescriptionKey: @"The model answered in a shape Sing does not read"}];+ return NO;+ }+ [self synthesize:_vocals left:vocalsLeft right:vocalsRight];+ return YES;+}++@enddiff --git a/tweak/Sources/Shared/Sing/Sing.h b/tweak/Sources/Shared/Sing/Sing.hnew file mode 100644index 00000000..5890a632--- /dev/null+++ b/tweak/Sources/Shared/Sing/Sing.h@@ -0,0 +1,90 @@+// Sing: a song's vocals turned down while it plays, from a mic button on the redesigned player's lyrics+// (Redesigned/Lyrics/SGRSingButton.m) and the Sing page (Sing on Mod Settings' main page, and Lyrics > Karaoke),+// under either look: the work is on Spotify's sound.+//+// SGSingModel.m the voice model: downloaded, each file checked against its size and SHA-256, loaded+// SGSingSeparator.m the STFT around the model: two seconds in, their vocals out+// SGSingEngine.m Spotify's mixer pulled ahead of what plays, the vocals separated in the lead and mixed down+// Sing.x the engine put between Spotify's mixer and its output (through Shared/Player/SpeedPitch.x's+// chain), Spotify's clock and seeks kept true to what plays, the heat, and what Sing is doing+// SingSettings.m the Sing page+//+// The voice model is public and MIT licensed: Mel-Band RoFormer (Ju-Chiang Wang, Wei-Tsung Lu, Minz Won) with+// KimberleyJensen's vocal checkpoint, its spectral core exported for Core ML with two-second windows, from+// https://huggingface.co/My-Name-Is-Jeff/vitrine-sing, a mirror of Darkkos/spoti-sing (App/About/Licenses.m carries+// its notice). Its floor is iOS 18, the Core ML it was built with. No audio leaves the phone.+//+// Threading: main thread, but for what Sing.x's render side and the engine say.+#import <UIKit/UIKit.h>++// The mic: on and off at once, from the button or the Sing page; off until switched on.+#define SGKeySing @"spotifyglass.sing"+// The vocals' level, 0 (gone) through 1 (as the song has them) to 2 (the vocals alone) at the slider's top.+#define SGKeySingLevel @"spotifyglass.sing.level"+// Keeps the model running on a hot iPhone, which otherwise holds it from Serious on.+#define SGKeySingIgnoreHeat @"spotifyglass.sing.ignoreHeat"+// Where the model runs: an index into SGSingComputeUnitNames(), read as it loads.+#define SGKeySingComputeUnits @"spotifyglass.sing.computeUnits"++// Posted on the main thread whenever what Sing is doing, or its model's download, changes.+extern NSString *const SGSingChangedNotification;++typedef NS_ENUM(NSInteger, SGSingState) {+ SGSingStateUnavailable, // the OS or the iPhone cannot run it (SGSingMissing says which)+ SGSingStateNoModel, // the model is not downloaded+ SGSingStateDownloading,+ SGSingStateOff,+ SGSingStatePreparing, // the model loading, which Core ML specialises for the phone the first time+ SGSingStateWaiting, // on, Spotify not playing through the chain yet+ SGSingStateBuffering, // on, the lead filling: what plays is dry+ SGSingStateSinging, // on, the vocals down+ SGSingStateBehind, // on, the model slower than the song: what plays is dry until it catches up+ SGSingStateHot, // on, held for the heat+ SGSingStateFailed, // the model or Spotify's output failed (SGSingStatusDetail says how)+};+SGSingState SGSingCurrentState(void);+// What Sing is doing, a few words for the button's label and the page, and what it means where there is+// more to say (what is missing, why it failed), nil otherwise.+NSString *SGSingStatusText(void);+NSString *SGSingStatusDetail(void);+// Which of the OS, the voice model or the iPhone Sing was built for is missing; nil when none is.+NSString *SGSingMissing(void);++BOOL SGSingOn(void);+void SGSetSingOn(BOOL on);+float SGSingLevel(void);+void SGSetSingLevel(float level);+// A level as the slider reads it out: Gone, a percentage, As sung, Backing and a percentage, Vocals only.+NSString *SGSingLevelText(double level);+void SGSetSingIgnoresHeat(BOOL ignores);+// Loads the model again on the compute units now stored, if it is loaded.+void SGSingComputeUnitsChanged(void);+// What the main page's Sing row reads out: On, Off, or how far the model has come.+NSString *SGSingSummary(void);+UIViewController *SGSingSettingsPage(void);++#pragma mark - the model (SGSingModel.m)++typedef NS_ENUM(NSInteger, SGSingModelState) {+ SGSingModelMissing,+ SGSingModelDownloading,+ SGSingModelReady, // on the phone, every file checked+};+SGSingModelState SGSingModelCurrentState(void);+// 0 to 1 while downloading.+double SGSingModelProgress(void);+// Whether the download is held until the iPhone is online again.+BOOL SGSingModelWaitingForNetwork(void);+// The last download's failure, nil when there was none.+NSString *SGSingModelError(void);+// The download's size, for the row that offers it.+NSString *SGSingModelSizeText(void);+void SGSingDownloadModel(void);+void SGSingCancelModelDownload(void);+void SGSingDeleteModel(void);+// The compiled model's folder, nil until it is ready.+NSURL *SGSingModelURL(void);+NSArray<NSString *> *SGSingComputeUnitNames(void);+// Whether this OS can load the model (iOS 18) and this iPhone was built to run it.+BOOL SGSingOSSupported(void);+BOOL SGSingDeviceSupported(void);diff --git a/tweak/Sources/Shared/Sing/Sing.x b/tweak/Sources/Shared/Sing/Sing.xnew file mode 100644index 00000000..7a75d1b4--- /dev/null+++ b/tweak/Sources/Shared/Sing/Sing.x@@ -0,0 +1,478 @@+// Sing on Spotify's sound (Sing.h): the engine put between Spotify's mixer and its output, the voice model+// loaded while the mic is on, Spotify's clock and seeks kept true to what plays, the heat, and what Sing is+// doing for the button and the page.+//+// The engine sits in the chain Speed and pitch takes over (Shared/Player/SpeedPitch.x): its stage is handed+// the mixer's pull, so the engine can pull the mixer ahead of what plays. It takes Spotify's sound as Spotify+// hands it to the RemoteIO unit, which this file reads when Spotify starts the unit (Spotify's import of+// AudioOutputUnitStart rebound, as Music Haptics and the audio effects do): Sing needs it at the model's+// 44.1 kHz, float, a buffer per channel, two channels, and otherwise passes it as it is.+//+// The lead is sound Spotify's decoder has handed over and the speaker has not played yet, and Spotify's clock+// counts what the decoder handed over. -[SPTPlayerState position] is [self positionAsOfTimestamp] run on by+// the time since (disassembly at 0x1057735ec, its -1 kept for no position), so positionAsOfTimestamp is+// hooked to take the lead off: the scrubber, the lyrics and the lock screen follow what is heard. A seek, a+// skip or a stop drops the lead (-[SPTEsperantoPlayer seekTo:...] and skipTo...TrackWithOptions:track:loggingParams:+// in the binary's method list, each returning Spotify's own result for the command; the shorter skips are+// trampolines into those two through objc_msgSend, 0x1096da9e4-0x1096daa08), and so does a track changing well+// before the last one ended: something new was played.+//+// The model is held only while the mic is on, and loaded on a queue of its own. From the thermal state+// Serious on, the engine is held and plays dry, unless Ignore heat warnings is on.+//+// Threading: the stage on the render thread; Spotify starts its output on a thread of its own; the rest main.+#import <AudioToolbox/AudioToolbox.h>+#import <CoreML/CoreML.h>+#import <pthread.h>+#import <stdatomic.h>+#import "Core/SGCore.h"+#import "Core/SGRebind.h"+#import "Shared/Player/PlayerState.h"+#import "Shared/Player/SpeedPitch.h"+#import "Sing.h"+#import "SGSingEngine.h"+#import "SGSingSeparator.h"++// The vocals' level until one is chosen: down to a guide.+static const float kDefaultLevel = 0.15f;+// How often what Sing is doing is read while the mic is on.+static const NSTimeInterval kWatchEvery = 0.5;+// A track changing more than this before the last one's end is something new played, not the next track.+static const double kEndSlack = 2;++static _Atomic(SGSingEngine *) sg_engine; // made the first time the mic is on, never freed+static atomic_bool sg_formatTaken; // Spotify's output is in a format Sing takes+static atomic_bool sg_staged; // the stage has run: Spotify's sound comes through the chain+static atomic_bool sg_formatRefused; // the stage was handed buffers it does not take+static atomic_bool sg_outputStarted; // Spotify has started its RemoteIO unit++#pragma mark - the render thread++typedef struct {+ SGPlayerPull pull;+ void *context;+} ChainPull;++static OSStatus pullChain(void *context, UInt32 frames, float *left, float *right) {+ ChainPull *chain = context;+ struct { AudioBufferList list; AudioBuffer second; } buffers = {{2, {{1, frames * 4, left}}}, {1, frames * 4, right}};+ return chain->pull(chain->context, frames, &buffers.list);+}++static OSStatus stage(UInt32 frames, AudioBufferList *data, SGPlayerPull pull, void *context) {+ SGSingEngine *engine = atomic_load_explicit(&sg_engine, memory_order_acquire);+ if (!engine) return pull(context, frames, data);+ atomic_store_explicit(&sg_staged, true, memory_order_relaxed);+ BOOL takes = atomic_load_explicit(&sg_formatTaken, memory_order_relaxed) && data->mNumberBuffers == 2;+ for (UInt32 b = 0; takes && b < 2; b++) {+ takes = data->mBuffers[b].mData && data->mBuffers[b].mNumberChannels == 1 && data->mBuffers[b].mDataByteSize == frames * sizeof(float);+ }+ if (!takes) {+ atomic_store_explicit(&sg_formatRefused, true, memory_order_relaxed);+ return pull(context, frames, data);+ }+ ChainPull chain = {pull, context};+ return SGSingEngineRender(engine, frames, data->mBuffers[0].mData, data->mBuffers[1].mData, pullChain, &chain);+}++#pragma mark - Spotify's audio thread++static OSStatus (*sg_startOutput)(AudioUnit unit);++static OSStatus startOutput(AudioUnit unit) {+ AudioComponentDescription description = {0};+ if (unit && AudioComponentGetDescription(AudioComponentInstanceGetComponent(unit), &description) == noErr+ && description.componentType == kAudioUnitType_Output && description.componentSubType == kAudioUnitSubType_RemoteIO) {+ atomic_store_explicit(&sg_outputStarted, true, memory_order_relaxed);+ AudioStreamBasicDescription format = {0};+ UInt32 size = sizeof format;+ OSStatus status = AudioUnitGetProperty(unit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, &size);+ BOOL takes = status == noErr && format.mFormatID == kAudioFormatLinearPCM && format.mSampleRate == kSGSingRate+ && format.mChannelsPerFrame == 2 && format.mBitsPerChannel == 32 && (format.mFormatFlags & kAudioFormatFlagIsFloat)+ && (format.mFormatFlags & kAudioFormatFlagIsNonInterleaved);+ if (takes != atomic_exchange(&sg_formatTaken, takes) || !takes) {+ SGLog(@"sing: Spotify hands its output %.0f Hz, %u channels, %u bits, flags 0x%x: %@", format.mSampleRate,+ (unsigned)format.mChannelsPerFrame, (unsigned)format.mBitsPerChannel, (unsigned)format.mFormatFlags,+ takes ? @"Sing takes it" : @"not a format Sing takes, its sound passes as it is");+ }+ }+ return sg_startOutput(unit);+}++#pragma mark - the model++static SGSingSeparator *sg_separator;+static BOOL sg_loading;+static NSString *sg_loadError;+static NSUInteger sg_loadGeneration; // counts the mic's switches, so a load finished after it went off is dropped+static BOOL sg_hot;++static void announce(void) {+ [NSNotificationCenter.defaultCenter postNotificationName:SGSingChangedNotification object:nil];+}++static dispatch_queue_t loadQueue(void) {+ static dispatch_queue_t queue;+ static dispatch_once_t once;+ dispatch_once(&once, ^{+ queue = dispatch_queue_create("spotifyglass.sing.load", dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_USER_INITIATED, 0));+ });+ return queue;+}++static MLComputeUnits computeUnits(void) {+ switch (SGInt(SGKeySingComputeUnits, 0)) {+ case 1: return MLComputeUnitsAll;+ case 2: return MLComputeUnitsCPUAndNeuralEngine;+ default: return MLComputeUnitsCPUAndGPU;+ }+}++static void loadModel(void) {+ NSURL *url = SGSingModelURL();+ if (sg_separator || sg_loading || !url || SGSingMissing()) return;+ sg_loading = YES;+ sg_loadError = nil;+ NSUInteger generation = sg_loadGeneration;+ MLComputeUnits units = computeUnits();+ SGLog(@"sing: loading the voice model");+ dispatch_async(loadQueue(), ^{+ CFAbsoluteTime began = CFAbsoluteTimeGetCurrent();+ MLModelConfiguration *configuration = [MLModelConfiguration new];+ configuration.computeUnits = units;+ NSError *error;+ MLModel *model = [MLModel modelWithContentsOfURL:url configuration:configuration error:&error];+ SGSingSeparator *separator = model ? [[SGSingSeparator alloc] initWithModel:model] : nil;+ double took = CFAbsoluteTimeGetCurrent() - began;+ dispatch_async(dispatch_get_main_queue(), ^{+ sg_loading = NO;+ if (!separator) {+ sg_loadError = error.localizedDescription ?: @"The voice model could not be loaded";+ SGLog(@"sing: the voice model did not load: %@", sg_loadError);+ } else if (generation == sg_loadGeneration) {+ sg_separator = separator;+ SGSingEngineSetSeparator(atomic_load(&sg_engine), separator);+ SGLog(@"sing: the voice model is loaded in %.1f s", took);+ }+ announce();+ });+ });+}++static void unloadModel(void) {+ sg_loadGeneration++;+ sg_separator = nil;+ SGSingEngine *engine = atomic_load(&sg_engine);+ if (engine) SGSingEngineSetSeparator(engine, nil);+}++#pragma mark - the state++static void watch(void);++static void apply(void) {+ BOOL on = SGSingOn() && !SGSingMissing();+ SGSingEngine *engine = atomic_load(&sg_engine);+ if (on && !engine) {+ engine = SGSingEngineCreate();+ if (!engine) {+ sg_loadError = @"Sing could not set aside memory for the song";+ return;+ }+ SGSingEngineSetLevel(engine, SGSingLevel());+ atomic_store_explicit(&sg_engine, engine, memory_order_release);+ }+ if (!engine) return;+ SGSingEngineSetPaused(engine, sg_hot && !SGHidden(SGKeySingIgnoreHeat));+ SGSingEngineSetOn(engine, on);+ if (on) loadModel();+ else unloadModel();+ watch();+}++BOOL SGSingOn(void) {+ return SGHidden(SGKeySing);+}++void SGSetSingOn(BOOL on) {+ SGSetEnabled(SGKeySing, on);+ sg_loadError = nil; // switched on again, a failed load is tried again+ SGLog(@"sing: the mic is %@", on ? @"on" : @"off");+ apply();+ announce();+}++float SGSingLevel(void) {+ NSNumber *stored = [NSUserDefaults.standardUserDefaults objectForKey:SGKeySingLevel];+ return stored ? fmaxf(0, fminf(stored.floatValue, 2)) : kDefaultLevel;+}++void SGSetSingLevel(float level) {+ level = fmaxf(0, fminf(level, 2));+ [NSUserDefaults.standardUserDefaults setFloat:level forKey:SGKeySingLevel];+ SGSingEngine *engine = atomic_load(&sg_engine);+ if (engine) SGSingEngineSetLevel(engine, level);+}++void SGSingComputeUnitsChanged(void) {+ unloadModel();+ apply();+ announce();+}++void SGSetSingIgnoresHeat(BOOL ignores) {+ SGSetEnabled(SGKeySingIgnoreHeat, ignores);+ apply();+ announce();+}++NSString *SGSingMissing(void) {+ if (!SGSingOSSupported()) return [NSString stringWithFormat:@"Sing needs iOS 18, the first its voice model runs on. This iPhone has iOS %@.", UIDevice.currentDevice.systemVersion];+ if (!SGSingDeviceSupported()) return @"Sing needs an iPhone with 6 GB of memory or more, which the voice model was built for. This one has less.";+ if (SGSingModelCurrentState() != SGSingModelReady) return [NSString stringWithFormat:@"Sing needs its voice model, which is downloaded from Mod Settings > Sing (%@).", SGSingModelSizeText()];+ return nil;+}++SGSingState SGSingCurrentState(void) {+ if (!SGSingOSSupported() || !SGSingDeviceSupported()) return SGSingStateUnavailable;+ SGSingModelState model = SGSingModelCurrentState();+ if (model == SGSingModelDownloading) return SGSingStateDownloading;+ if (model != SGSingModelReady) return SGSingStateNoModel;+ if (!SGSingOn()) return SGSingStateOff;+ SGSingEngine *engine = atomic_load(&sg_engine);+ if (sg_loadError || !engine || (engine && SGSingEngineError(engine))) return SGSingStateFailed;+ if (sg_loading || !sg_separator) return SGSingStatePreparing;+ if (sg_hot && !SGHidden(SGKeySingIgnoreHeat)) return SGSingStateHot;+ // Spotify plays and Speed and pitch never took its output over: the stage, and so Sing, never runs.+ if (!atomic_load(&sg_staged) && atomic_load(&sg_outputStarted) && !SGPlayerSpeedAllowed()) return SGSingStateFailed;+ if (!sg_startOutput || (atomic_load(&sg_staged) && atomic_load(&sg_formatRefused) && !atomic_load(&sg_formatTaken))) return SGSingStateFailed;+ SGSingEngineStats stats = SGSingEngineReadStats(engine);+ if (!atomic_load(&sg_staged) || stats.lead < 0.05) return SGSingStateWaiting;+ if (stats.ready > 0) return SGSingStateSinging;+ // The lead is full and the vocals are still not in: the model is slower than the song.+ if (stats.windows && (stats.averageMS > kSGSingEngineHop * 1000.0 / kSGSingRate || stats.lead >= stats.targetLead - 0.25)) return SGSingStateBehind;+ return SGSingStateBuffering;+}++NSString *SGSingStatusText(void) {+ switch (SGSingCurrentState()) {+ case SGSingStateUnavailable: return @"Unavailable";+ case SGSingStateNoModel: return @"Needs its voice model";+ case SGSingStateDownloading:+ if (SGSingModelWaitingForNetwork()) return @"Waiting for the network";+ return [NSString stringWithFormat:@"Downloading %.0f%%", SGSingModelProgress() * 100];+ case SGSingStateOff: return @"Off";+ case SGSingStatePreparing: return @"Preparing";+ case SGSingStateWaiting: return @"Waiting for Spotify";+ case SGSingStateBuffering: return @"Listening ahead";+ case SGSingStateSinging: return @"On";+ case SGSingStateBehind: return @"Too slow";+ case SGSingStateHot: return @"Held, too hot";+ case SGSingStateFailed: return @"Failed";+ }+ return @"";+}++NSString *SGSingStatusDetail(void) {+ switch (SGSingCurrentState()) {+ case SGSingStateUnavailable:+ case SGSingStateNoModel:+ return SGSingMissing();+ case SGSingStateDownloading:+ if (SGSingModelWaitingForNetwork()) {+ return [NSString stringWithFormat:@"The iPhone is offline. The voice model's download carries on from %.0f%% of %@ once it is online again.",+ SGSingModelProgress() * 100, SGSingModelSizeText()];+ }+ return [NSString stringWithFormat:@"The voice model is coming in, %.0f%% of %@. Sing starts once it is checked.", SGSingModelProgress() * 100, SGSingModelSizeText()];+ case SGSingStateWaiting:+ return @"Sing starts once Spotify plays a song.";+ case SGSingStatePreparing:+ return @"Core ML prepares the voice model for this iPhone. The first time, that can take a minute.";+ case SGSingStateBuffering:+ return @"Sing listens a few seconds ahead of what plays, so the vocals are separated before you hear them. Until then the song plays as it is.";+ case SGSingStateBehind: {+ SGSingEngine *engine = atomic_load(&sg_engine);+ double ms = engine ? SGSingEngineReadStats(engine).averageMS : 0;+ return [NSString stringWithFormat:@"The voice model takes %.1f s for every 1.5 s of song on this iPhone, so the vocals stay in until it catches up. Another choice under Runs on may be faster.", ms / 1000];+ }+ case SGSingStateHot:+ return @"The iPhone is hot, so Sing is held and the song plays as it is until it cools down. Ignore heat warnings keeps Sing going.";+ case SGSingStateFailed: {+ SGSingEngine *engine = atomic_load(&sg_engine);+ NSString *error = sg_loadError ?: (engine ? SGSingEngineError(engine) : nil);+ if (error) return error;+ if (!sg_startOutput) return @"Spotify's output could not be reached.";+ if (!atomic_load(&sg_staged) && !SGPlayerSpeedAllowed()) return @"Spotify's output could not be taken over, so Sing cannot reach its sound.";+ return @"Spotify's output is not 44.1 kHz stereo, which the voice model needs, so the song plays as it is.";+ }+ default:+ return nil;+ }+}++NSString *SGSingSummary(void) {+ switch (SGSingCurrentState()) {+ case SGSingStateUnavailable: return @"Unavailable";+ case SGSingStateNoModel: return @"Off";+ case SGSingStateDownloading: return [NSString stringWithFormat:@"%.0f%%", SGSingModelProgress() * 100];+ case SGSingStateOff: return @"Off";+ default: return @"On";+ }+}++#pragma mark - watching while the mic is on++static NSTimer *sg_watch;+static SGSingState sg_shown = -1;+static NSString *sg_shownText;+static NSString *sg_lastTrack;+static double sg_lastRaw = -1, sg_lastDuration;+static CFAbsoluteTime sg_lastFlush;+static __thread BOOL sg_rawPosition; // the hook below hands Spotify's own value back++static double rawPosition(SPTPlayerState *state) {+ sg_rawPosition = YES;+ double position = state.position;+ sg_rawPosition = NO;+ return position;+}++static void flush(NSString *why) {+ SGSingEngine *engine = atomic_load(&sg_engine);+ if (!engine || SGSingEngineLead(engine) <= 0) return;+ sg_lastFlush = CFAbsoluteTimeGetCurrent();+ SGSingEngineFlush(engine);+ SGLog(@"sing: %@, the %.1f s held ahead are dropped", why, SGSingEngineLead(engine));+}++static void tick(void) {+ SGSingState state = SGSingCurrentState();+ NSString *text = SGSingStatusText();+ if (state != sg_shown || ![text isEqualToString:sg_shownText]) {+ sg_shown = state;+ sg_shownText = text;+ announce();+ }+ SPTPlayerState *player = SGPlayerState();+ if (player) {+ sg_lastRaw = rawPosition(player);+ sg_lastDuration = player.duration;+ }+ SGSingEngine *engine = atomic_load(&sg_engine);+ // Now and then while on, what the model costs on this iPhone.+ static CFAbsoluteTime summarized;+ if (engine && SGSingOn() && CFAbsoluteTimeGetCurrent() - summarized >= 30) {+ summarized = CFAbsoluteTimeGetCurrent();+ SGSingEngineStats stats = SGSingEngineReadStats(engine);+ SGLog(@"sing: %@, %llu windows at %.0f ms each (1500 ms keeps up), lead %.2f s of %.2f s, %.2f s separated ahead, %llu frames dry, %llu failures",+ SGSingStatusText(), stats.windows, stats.averageMS, stats.lead, stats.targetLead, stats.ready, stats.dryFrames, stats.failures);+ }+ // Off, with the lead given back: nothing left to watch.+ if (!SGSingOn() && (!engine || SGSingEngineLead(engine) <= 0)) {+ [sg_watch invalidate];+ sg_watch = nil;+ }+}++static void watch(void) {+ if (sg_watch) return;+ sg_watch = [NSTimer scheduledTimerWithTimeInterval:kWatchEvery repeats:YES block:^(NSTimer *timer) { tick(); }];+ tick();+}++@interface SGSingTrackWatcher : NSObject <SGPlayerStateObserver>+@end++@implementation SGSingTrackWatcher+// A new track well before the last one's end was played, not reached: the lead is the old track's.+- (void)playerStateDidChange:(SPTPlayerState *)state {+ NSString *track = SGURIString(state.track.URI);+ if (track && sg_lastTrack && ![track isEqualToString:sg_lastTrack] && sg_lastRaw >= 0 && sg_lastDuration > 0+ && sg_lastDuration - sg_lastRaw > kEndSlack + kWatchEvery && CFAbsoluteTimeGetCurrent() - sg_lastFlush > 1) {+ flush(@"another track was played");+ }+ if (track) sg_lastTrack = track;+}+@end++#pragma mark - the heat++static void readHeat(void) {+ BOOL hot = NSProcessInfo.processInfo.thermalState >= NSProcessInfoThermalStateSerious;+ if (hot == sg_hot) return;+ sg_hot = hot;+ SGLog(@"sing: the iPhone is %@%@", hot ? @"hot" : @"cool again", hot && SGHidden(SGKeySingIgnoreHeat) ? @", and heat warnings are ignored" : @"");+ apply();+ announce();+}++#pragma mark - Spotify's clock and commands++%hook SPTPlayerState+- (double)positionAsOfTimestamp {+ double position = %orig;+ if (sg_rawPosition || position < 0) return position;+ SGSingEngine *engine = atomic_load_explicit(&sg_engine, memory_order_acquire);+ double lead = engine ? SGSingEngineLead(engine) : 0;+ return lead > 0 ? MAX(0, position - lead) : position;+}+%end++%hook SPTEsperantoPlayer+- (id)seekTo:(double)position {+ flush(@"Spotify seeks");+ return %orig;+}+- (id)seekTo:(double)position relative:(long long)relative {+ flush(@"Spotify seeks");+ return %orig;+}+- (id)seekTo:(double)position options:(id)options {+ flush(@"Spotify seeks");+ return %orig;+}+- (id)seekTo:(double)position relative:(long long)relative options:(id)options {+ flush(@"Spotify seeks");+ return %orig;+}+- (id)seekTo:(double)position relative:(long long)relative options:(id)options creatorTimestampPositionMs:(double)creator {+ flush(@"Spotify seeks");+ return %orig;+}+// Every skip of Spotify's ends up in one of these two.+- (id)skipToNextTrackWithOptions:(id)options track:(id)track loggingParams:(id)params {+ flush(@"Spotify skips");+ return %orig;+}+- (id)skipToPreviousTrackWithOptions:(id)options track:(id)track loggingParams:(id)params {+ flush(@"Spotify skips back");+ return %orig;+}+- (id)stop {+ flush(@"Spotify stops");+ return %orig;+}+%end++%ctor {+ if (!SGRebindImport("AudioOutputUnitStart", startOutput, (void **)&sg_startOutput) || !sg_startOutput) {+ sg_startOutput = NULL;+ SGLog(@"sing: Spotify does not import AudioOutputUnitStart, Sing cannot read its output");+ }+ SGPlayerSetStage(stage);+ %init;+ SGRequireClasses(@[@"SPTPlayerState", @"SPTEsperantoPlayer"]);+ dispatch_async(dispatch_get_main_queue(), ^{+ static SGSingTrackWatcher *watcher;+ watcher = [SGSingTrackWatcher new];+ SGAddPlayerStateObserver(watcher);+ [NSNotificationCenter.defaultCenter addObserverForName:NSProcessInfoThermalStateDidChangeNotification object:nil queue:NSOperationQueue.mainQueue+ usingBlock:^(NSNotification *note) { readHeat(); }];+ [NSNotificationCenter.defaultCenter addObserverForName:SGSingChangedNotification object:nil queue:nil usingBlock:^(NSNotification *note) {+ // A download finishing with the mic already on loads the model.+ if (SGSingOn() && !sg_separator && !sg_loading && !sg_loadError) apply();+ }];+ readHeat();+ if (SGSingOn()) apply();+ });+}diff --git a/tweak/Sources/Shared/Sing/SingSettings.m b/tweak/Sources/Shared/Sing/SingSettings.mnew file mode 100644index 00000000..95184060--- /dev/null+++ b/tweak/Sources/Shared/Sing/SingSettings.m@@ -0,0 +1,91 @@+// The Sing page (Sing on Mod Settings' main page, and Lyrics > Karaoke), under either look: Sing's switch, which+// turns the mic on and off at once, what Sing is doing, the vocals' level, and the voice model.+#import "Core/SGCore.h"+#import "Settings/SGModPage.h"+#import "Settings/SGPageStyle.h"+#import "Sing.h"++static void tell(NSString *title, NSString *message) {+ UIAlertController *alert = [UIAlertController alertControllerWithTitle:title message:message preferredStyle:UIAlertControllerStyleAlert];+ [alert addAction:[UIAlertAction actionWithTitle:@"OK" style:UIAlertActionStyleCancel handler:nil]];+ [SGTopController() presentViewController:alert animated:YES completion:nil];+}++static void ask(NSString *title, NSString *message, NSString *action, BOOL destructive, void (^then)(void)) {+ UIAlertController *alert = [UIAlertController alertControllerWithTitle:title message:message preferredStyle:UIAlertControllerStyleAlert];+ [alert addAction:[UIAlertAction actionWithTitle:@"Cancel" style:UIAlertActionStyleCancel handler:nil]];+ [alert addAction:[UIAlertAction actionWithTitle:action style:destructive ? UIAlertActionStyleDestructive : UIAlertActionStyleDefault+ handler:^(UIAlertAction *a) { then(); }]];+ [SGTopController() presentViewController:alert animated:YES completion:nil];+}++NSString *SGSingLevelText(double level) {+ if (level <= 0.001) return @"Gone";+ if (level < 0.999) return [NSString stringWithFormat:@"%.0f%%", level * 100];+ if (level <= 1.001) return @"As sung";+ if (level >= 1.999) return @"Vocals only";+ return [NSString stringWithFormat:@"Backing %.0f%%", (2 - level) * 100];+}++static NSString *modelValue(void) {+ if (!SGSingOSSupported() || !SGSingDeviceSupported()) return @"Unavailable";+ switch (SGSingModelCurrentState()) {+ case SGSingModelDownloading:+ return SGSingModelWaitingForNetwork() ? @"Waiting for the network" : [NSString stringWithFormat:@"%.0f%%", SGSingModelProgress() * 100];+ case SGSingModelReady: return @"On the iPhone";+ case SGSingModelMissing: return SGSingModelError() ? @"Failed, try again" : [NSString stringWithFormat:@"Download, %@", SGSingModelSizeText()];+ }+ return @"";+}++static void modelTapped(UITableViewController *page) {+ if (!SGSingOSSupported() || !SGSingDeviceSupported()) {+ tell(@"Voice model", SGSingMissing());+ return;+ }+ switch (SGSingModelCurrentState()) {+ case SGSingModelMissing: {+ NSString *message = [NSString stringWithFormat:@"%@%@ from Hugging Face, best over Wi-Fi. It stays on this iPhone, and no audio leaves it.",+ SGSingModelError() ? [NSString stringWithFormat:@"The last try failed: %@\n\n", SGSingModelError()] : @"", SGSingModelSizeText()];+ ask(@"Download the voice model?", message, @"Download", NO, ^{ SGSingDownloadModel(); });+ break;+ }+ case SGSingModelDownloading:+ ask(@"Stop the download?", @"What has come in is kept, so the next download carries on from it.", @"Stop", YES, ^{ SGSingCancelModelDownload(); });+ break;+ case SGSingModelReady:+ ask(@"Delete the voice model?", [NSString stringWithFormat:@"It frees %@. Sing needs it again to turn the vocals down.", SGSingModelSizeText()],+ @"Delete", YES, ^{+ SGSetSingOn(NO);+ SGSingDeleteModel();+ // Sing's switch goes off with it.+ [page.tableView reloadData];+ });+ break;+ }+}++UIViewController *SGSingSettingsPage(void) {+ SGModRow *sing = SGOptionRow(@"Sing", @"Turns the vocals down as the song plays, like the mic on the lyrics", SGKeySing);+ sing.changed = ^(BOOL on) { SGSetSingOn(on); };+ SGModRow *status = SGStatActionRow(@"Status", nil, ^NSString *{ return SGSingStatusText(); }, ^{+ NSString *detail = SGSingStatusDetail();+ if (detail) tell(@"Sing", detail);+ });+ SGModRow *level = SGSliderRow(@"Vocals", @"From gone, through the song as sung, to the vocals alone", 0, 2, 0.05,+ ^double { return SGSingLevel(); }, ^(double value) { SGSetSingLevel((float)value); }, ^NSString *(double value) { return SGSingLevelText(value); });+ __block __weak SGModPage *page;+ SGModRow *model = SGStatActionRow(@"Voice model", @"Mel-Band RoFormer, run on the iPhone", ^NSString *{ return modelValue(); }, ^{ modelTapped(page); });+ SGModRow *units = SGChoiceRow(@"Runs on", @"Read as the model loads", SGKeySingComputeUnits, SGSingComputeUnitNames(), 0);+ units.chosen = ^(NSInteger index) { SGSingComputeUnitsChanged(); };+ SGModRow *heat = SGOptionRow(@"Ignore heat warnings", @"Keeps Sing going on a hot iPhone, which then gets hotter", SGKeySingIgnoreHeat);+ heat.changed = ^(BOOL on) { SGSetSingIgnoresHeat(on); };+ SGModPage *made = [[SGModPage alloc] initWithTitle:@"Sing" intro:nil sections:@[+ SGSection(nil, @[SGWithSymbol(sing, @"music.mic"), status, level]),+ SGNotedSection(@"Voice model", @[model, units, heat],+ @"Sing listens a few seconds ahead of what plays and separates the vocals there, so a song takes a few seconds to "+ @"turn its vocals down after it starts or after a seek. Everything runs on the iPhone; no audio leaves it."),+ ] footer:nil];+ page = made;+ return made;+}