Liquid Glass and Apple Music touches for the Spotify iOS app. A GPL-3.0 fork of spoti.pw 0.21.1.
C
feat: the lock screen can show the lyrics over the moving artwork
My-Name-Is-Jeff1aef54bf72e9
diff --git a/docs/tweaks.md b/docs/tweaks.mdindex abb6ceec..5c4a624c 100644--- a/docs/tweaks.md+++ b/docs/tweaks.md@@ -150,7 +150,21 @@ Shared:Still (one frame) or Animated (15 fps). The backdrop is blurred small and drawn large, on the CPU as thelock screen asks in the background; previews are drawn on a queue apart from the clips, and while thelock screen is asking, the next line's preview and clip are drawn ahead and a ready preview is handed- over at once. iOS still shows the plain cover for about half a second at each change of artwork. Picking it or another artwork applies at once. Rendered on the Mac against harness/lyrics-clip/. What reaches the+ over at once. iOS still shows the plain cover for about half a second at each change of artwork. Picking it or another artwork applies at once.+ Moving artwork with lyrics (the choice's fifth value, 4) draws each line over the track's moving+ artwork, which LockScreenMotion.x finds and hands over (SGLyricsArtworkSetMotion): the clip is the+ stretch of the video the song is at while the line shows (its start taken round the video's length,+ from the line's start to the next's plus a second, 2 to 8 s; a video no longer than that is written+ whole, turned to start there, so its loop has no seam), at the video's own 3:4 cut and frame rate up+ to 30 fps, with the line sung last dimmed above (dim, bright, dim). The video is decoded as BGRA, the+ rows behind the words are blurred (Accelerate's tent blur) and faded in, every row of the cut is+ darkened in one pass (0.25, and 0.6 across the words), and the words with a tight dark shadow are+ drawn on; the encoder cuts the 3:4 middle as SGMotionClip.m does, so nothing is scaled on the CPU.+ The IDs end in m and a tag of the file, so the clip arriving, or a Canvas after an animated cover,+ re-offers the line showing. Until it comes, and for a track+ with no moving artwork or one that will not read (a turned video), the lines are drawn as Lyrics+ draws them, in the chosen style. Rendered on the Mac against harness/lyrics-clip/ (`motion`, over a+ bright busy, a dark and any given videos). What reaches thesystem's now playing through this hook and Player/NowPlayingExtras.x is checked by harness/now-playing/AnimatedArtwork/ moving artwork (AnimatedArtwork.h): the track's Canvas, else Apple Music's animated album cover(SGMotionCatalog.m: searched without the album's edition, the edition of the same name first, and@@ -165,8 +179,10 @@ Shared:shape when it is not, with its still filled to the size asked (SGMotionClip.m), and a change of thechoice applies at once, Lyrics too. Switch with the star (spotifyglass.lockscreen.star, off;LockScreenStar.x) puts a star on the lock screen widget, MediaPlayer's like command, whose tap swaps- the choice between the lyrics and the moving artwork it left (Moving artwork or Every song, kept under- spotifyglass.lockscreen.star.back), filled while the lyrics show; the choice is the setting, so the+ the choice for the one the star last left (kept under spotifyglass.lockscreen.star.back) when that one+ is on the other side, with lyrics or without, else for Lyrics or Moving artwork: Lyrics swaps with the+ moving artwork it left (Moving artwork or Every song), and Moving artwork with lyrics counts as lyrics,+ so it swaps with Moving artwork. The star is filled while the lyrics show; the choice is the setting, so themode outlives a relaunch and the page's row follows it. The star takes the slot of Spotify's likeand dislike buttons, so their flag is forced off at launch while the switch is on, over the row'sown override, and the row shows it locked. The cut and the service's wire format are checked on@@ -759,8 +775,8 @@ model with the full one, is on the iPhone), Toggle from lyrics (the redesign's mIgnore heat warnings, and under Advanced Runs on (Automatic: the Neural Enginebeside the CPU; CPU only) and Prepare after updates, all applying straight away. Lock screen, on the mainpage under either look, opens the lock screen widget's page, titled Lock screen: Widget (the line in place of the artist,-and Spotify's like and dislike buttons' flag, locked off while the star is on) then Artwork (Off, Moving artwork, Lyrics-or Every song, the lyrics' style, Still or Animated, Switch with the star, the sources and Low Data Mode; below iOS 26+and Spotify's like and dislike buttons' flag, locked off while the star is on) then Artwork (Off, Moving artwork, Lyrics,+Every song or Moving artwork with lyrics, the lyrics' style, Still or Animated, Switch with the star, the sources and Low Data Mode; below iOS 26one row saying why not). Spotify's podcast, audiobook and artwork flags stay in All flags. Player: Gestures and Blocked artists (with the count on the row), which work with either look;in the native look also Now playing bar (its device button and its flags), Queue & devices, andSpotify's own player screen (artwork background, glass header buttons, Disable Canvas and the sheet,diff --git a/harness/fluid-clip/build.sh b/harness/fluid-clip/build.shindex 3d31d0fa..476e37e3 100755--- a/harness/fluid-clip/build.sh+++ b/harness/fluid-clip/build.sh@@ -5,6 +5,6 @@ cd "$(dirname "$0")"mkdir -p buildxcrun clang -fobjc-arc -O2 -Wno-deprecated-declarations -I ../../tweak/Sources \-framework Foundation -framework AVFoundation -framework CoreMedia -framework CoreVideo -framework CoreImage \- -framework CoreText -framework CoreGraphics -framework ImageIO -framework UniformTypeIdentifiers \+ -framework CoreText -framework CoreGraphics -framework ImageIO -framework UniformTypeIdentifiers -framework Accelerate \main.m ../../tweak/Sources/Shared/AnimatedArtwork/SGFluidClip.m ../../tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m \-o build/fluid-clipdiff --git a/harness/lyrics-clip/README.md b/harness/lyrics-clip/README.mdindex a649634a..a992c181 100644--- a/harness/lyrics-clip/README.md+++ b/harness/lyrics-clip/README.md@@ -4,10 +4,26 @@short line, a long one, a CJK one, a break (the next line only), the last line and an empty clip, eachas a Still and as Animated, over a stand-in cover or the image given. Every file is read back withAVFoundation and checked: its length (2 s for a still, 4 s for an animated loop), its 3:4 size, H.264,-how many frames it holds (1 or 96), white words in the middle band and none under the clock or the+how many frames it holds (1 or 60), white words in the middle band and none under the clock or thecontrols, and the words the same at the end as at the start. It prints how long each clip took towrite. The first frame of each clip is saved next to it as a PNG../build.sh && build/lyrics-clip [cover.jpg]The clips land in `build/clips/`.++`motion` renders Moving artwork with lyrics (`SGLyricsClipWriteOverMotion`, `motion.m`) over two stand-ins+it makes, a bright busy flag (720x1280, 30 fps, a Canvas's shape) and a dark field (904x1206, 24 fps, an+Apple Music cover's), and over any videos given: a line with the one before and after, a long line, a+stretch round the video's end, one longer than the video (written whole), a break and no words. Each is+read back and checked: length, the 3:4 cut's size, H.264, frames at up to 30 fps, words in the middle and+nothing white under the clock or the controls, the picture moving, and the preview the same as the first+frame. It prints each clip's time a frame against the video cut alone (`SGMotionClipShape`) and the+Animated lyrics clip, the contrast of the words with the brightest of the picture behind them, and the+memory used.++ ./build.sh && build/lyrics-clip motion [video.mp4 ...]+ build/lyrics-clip motion one video.mp4 # one line's clip alone, for its memory peak+ build/lyrics-clip motion cut video.mp4 # the video cut alone, to compare++These land in `build/motion/`. AVFoundation's encoder needs the sandbox off.diff --git a/harness/lyrics-clip/build.sh b/harness/lyrics-clip/build.shindex 240e80c4..4ff788a7 100755--- a/harness/lyrics-clip/build.sh+++ b/harness/lyrics-clip/build.sh@@ -1,9 +1,12 @@#!/bin/sh-# Builds the lyrics clip harness for the Mac: SGLyricsClip.m as the tweak compiles it, and main.m.+# Builds the lyrics clip harness for the Mac: SGLyricsClip.m as the tweak compiles it, and main.m. motion.m+# times the words over moving artwork against SGMotionClip.m's cut, which needs Protobuf.m to link.set -ecd "$(dirname "$0")"mkdir -p buildxcrun clang -fobjc-arc -O2 -Wno-deprecated-declarations -I ../../tweak/Sources \-framework Foundation -framework AVFoundation -framework CoreMedia -framework CoreVideo -framework CoreImage \- -framework CoreText -framework CoreGraphics -framework ImageIO -framework UniformTypeIdentifiers \- main.m ../../tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m -o build/lyrics-clip+ -framework CoreText -framework CoreGraphics -framework ImageIO -framework UniformTypeIdentifiers -framework Accelerate \+ main.m motion.m ../../tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m \+ ../../tweak/Sources/Shared/AnimatedArtwork/SGMotionClip.m ../../tweak/Sources/Shared/AdBlock/Protobuf.m \+ -o build/lyrics-clipdiff --git a/harness/lyrics-clip/main.m b/harness/lyrics-clip/main.mindex 319a0386..9ed1ae9d 100644--- a/harness/lyrics-clip/main.m+++ b/harness/lyrics-clip/main.m@@ -95,7 +95,7 @@ static void verify(NSURL *url, CGImageRef backdrop, CGSize size, SGLyricsClipStyif (CMSampleBufferGetNumSamples(sample)) samples++;CFRelease(sample);}- int wantedSamples = style == SGLyricsClipAnimated ? 96 : 1;+ int wantedSamples = style == SGLyricsClipAnimated ? 60 : 1;check(samples == wantedSamples, [NSString stringWithFormat:@"holds %d frames (want %d)", samples, wantedSamples]);AVAssetImageGenerator *generator = [AVAssetImageGenerator assetImageGeneratorWithAsset:asset];@@ -119,9 +119,16 @@ static void verify(NSURL *url, CGImageRef backdrop, CGSize size, SGLyricsClipStyCGImageRelease(late);}+int motionMain(NSURL *out, int argc, char **argv);+int main(int argc, char **argv) {@autoreleasepool {NSString *here = [[NSString stringWithUTF8String:argv[0]] stringByDeletingLastPathComponent];+ if (argc > 1 && !strcmp(argv[1], "motion")) {+ NSURL *motionOut = [NSURL fileURLWithPath:[here stringByAppendingPathComponent:@"motion"] isDirectory:YES];+ [NSFileManager.defaultManager createDirectoryAtURL:motionOut withIntermediateDirectories:YES attributes:nil error:nil];+ return motionMain(motionOut, argc - 2, argv + 2);+ }NSURL *out = [NSURL fileURLWithPath:[here stringByAppendingPathComponent:@"clips"] isDirectory:YES];[NSFileManager.defaultManager createDirectoryAtURL:out withIntermediateDirectories:YES attributes:nil error:nil];CGImageRef cover = argc > 1 ? newImageAt([NSString stringWithUTF8String:argv[1]]) : newCover();diff --git a/harness/lyrics-clip/motion.m b/harness/lyrics-clip/motion.mnew file mode 100644index 00000000..2162386c--- /dev/null+++ b/harness/lyrics-clip/motion.m@@ -0,0 +1,340 @@+// `lyrics-clip motion [video ...]`: the words over moving artwork (SGLyricsClipWriteOverMotion). Makes two+// stand-in videos, a bright busy one (a waving pink and white flag with white stars, 720 x 1280, 30 fps, the+// shape of a Canvas) and a dark one (dim drifting blobs and specks, 904 x 1206, 24 fps, the shape of an Apple+// Music cover at SGMotionPixels), takes any videos given too, and for each writes lines over it and checks+// the file: length, 3:4 size, H.264, frame count, words in the middle and none under the clock or the+// controls, the picture moving, the preview matching the first frame. It times each clip against the video+// cut alone (SGMotionClipShape) and the Animated lyrics clip, and prints the contrast of the words with the+// brightest of the picture behind them. Exits 1 on any failure.+#import <AVFoundation/AVFoundation.h>+#import <ImageIO/ImageIO.h>+#import <UniformTypeIdentifiers/UniformTypeIdentifiers.h>+#import <mach/mach.h>+#import "Shared/AnimatedArtwork/SGMotionClip.h"+#import "Shared/LockScreenLyrics/SGLyricsClip.h"++static int failures;++static void check(BOOL ok, NSString *what) {+ printf(" %s %s\n", ok ? "ok " : "FAIL", what.UTF8String);+ if (!ok) failures++;+}++static double footprintMB(BOOL peak) {+ task_vm_info_data_t info;+ mach_msg_type_number_t count = TASK_VM_INFO_COUNT;+ if (task_info(mach_task_self(), TASK_VM_INFO, (task_info_t)&info, &count) != KERN_SUCCESS) return -1;+ return (peak ? info.ledger_phys_footprint_peak : info.phys_footprint) / 1048576.0;+}++static CGContextRef newContext(size_t width, size_t height, void *data, size_t bytesPerRow) {+ CGColorSpaceRef space = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);+ CGContextRef context = CGBitmapContextCreate(data, width, height, 8, bytesPerRow, space,+ kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little);+ CGColorSpaceRelease(space);+ return context;+}++static void star(CGContextRef c, CGFloat x, CGFloat y, CGFloat r) {+ for (int i = 0; i < 10; i++) {+ CGFloat a = M_PI_2 + i * M_PI / 5, l = i % 2 ? r * 0.4 : r;+ if (i) CGContextAddLineToPoint(c, x + l * cos(a), y + l * sin(a));+ else CGContextMoveToPoint(c, x + l * cos(a), y + l * sin(a));+ }+ CGContextClosePath(c);+ CGContextFillPath(c);+}++// A frame of the bright stand-in at `t` seconds: pink and white stripes waving across, a navy field of+// white stars at the top left, a white figure in the middle, the way a flag cover is busy and light.+static void drawFlag(CGContextRef c, size_t w, size_t h, double t) {+ CGContextSetRGBFillColor(c, 1, 1, 1, 1);+ CGContextFillRect(c, CGRectMake(0, 0, w, h));+ CGFloat stripe = h / 13.0;+ CGContextSetRGBFillColor(c, 0.88, 0.22, 0.58, 1);+ for (int i = 0; i < 14; i += 2) {+ CGContextMoveToPoint(c, 0, i * stripe);+ for (int x = 0; x <= (int)w; x += 8) {+ CGFloat y = i * stripe + stripe * 0.35 * sin(x / (CGFloat)w * 2 * M_PI + t * 2.2);+ CGContextAddLineToPoint(c, x, y);+ }+ for (int x = (int)w; x >= 0; x -= 8) {+ CGFloat y = (i + 1) * stripe + stripe * 0.35 * sin(x / (CGFloat)w * 2 * M_PI + t * 2.2);+ CGContextAddLineToPoint(c, x, y);+ }+ CGContextClosePath(c);+ CGContextFillPath(c);+ }+ CGFloat sway = 12 * sin(t * 2.2);+ CGRect field = CGRectMake(-20, h * 0.42 + sway, w * 0.62, h * 0.6);+ CGContextSetRGBFillColor(c, 0.06, 0.07, 0.16, 1);+ CGContextFillRect(c, field);+ CGContextSetRGBFillColor(c, 1, 1, 1, 1);+ for (int row = 0; row < 9; row++) {+ for (int col = 0; col < 5; col++) {+ star(c, 30 + col * w * 0.12 + (row % 2) * w * 0.06, h * 0.45 + sway + row * h * 0.062, w * 0.035);+ }+ }+ CGContextSetRGBFillColor(c, 0.97, 0.97, 0.97, 1);+ CGContextFillEllipseInRect(c, CGRectMake(w * 0.38, h * 0.18, w * 0.24, h * 0.22));+}++// A frame of the dark stand-in: a near-black field, slow dim colored blobs, and small bright specks.+static void drawDark(CGContextRef c, size_t w, size_t h, double t) {+ CGContextSetRGBFillColor(c, 0.03, 0.03, 0.06, 1);+ CGContextFillRect(c, CGRectMake(0, 0, w, h));+ CGFloat colors[][3] = {{0.35, 0.1, 0.45}, {0.1, 0.2, 0.45}, {0.45, 0.15, 0.1}};+ for (int i = 0; i < 3; i++) {+ CGFloat x = w * (0.3 + 0.25 * i + 0.1 * sin(t * 0.8 + i)), y = h * (0.3 + 0.2 * i + 0.08 * cos(t * 0.8 + i * 2));+ CGContextSetRGBFillColor(c, colors[i][0], colors[i][1], colors[i][2], 0.6);+ CGContextFillEllipseInRect(c, CGRectMake(x - w * 0.3, y - w * 0.3, w * 0.6, w * 0.6));+ }+ CGContextSetRGBFillColor(c, 1, 1, 1, 0.9);+ for (int i = 0; i < 60; i++) {+ CGFloat x = fmod(i * 97.3 + t * 20, w), y = fmod(i * 211.7, h), r = 1.5 + (i % 3);+ CGContextFillEllipseInRect(c, CGRectMake(x, y, r * 2, r * 2));+ }+}++static BOOL makeVideo(NSURL *url, size_t w, size_t h, int fps, int seconds, void (*draw)(CGContextRef, size_t, size_t, double)) {+ [NSFileManager.defaultManager removeItemAtURL:url error:nil];+ AVAssetWriter *writer = [AVAssetWriter assetWriterWithURL:url fileType:AVFileTypeMPEG4 error:nil];+ AVAssetWriterInput *input = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:@{+ AVVideoCodecKey: AVVideoCodecTypeH264, AVVideoWidthKey: @(w), AVVideoHeightKey: @(h),+ AVVideoCompressionPropertiesKey: @{AVVideoAverageBitRateKey: @(w * h * 3), AVVideoMaxKeyFrameIntervalKey: @(fps * 2)},+ }];+ AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:input+ sourcePixelBufferAttributes:@{(id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32BGRA),+ (id)kCVPixelBufferWidthKey: @(w), (id)kCVPixelBufferHeightKey: @(h)}];+ [writer addInput:input];+ [writer startWriting];+ [writer startSessionAtSourceTime:kCMTimeZero];+ for (int i = 0; i < fps * seconds; i++) {+ while (!input.readyForMoreMediaData) [NSThread sleepForTimeInterval:0.002];+ CVPixelBufferRef buffer = NULL;+ CVPixelBufferPoolCreatePixelBuffer(NULL, adaptor.pixelBufferPool, &buffer);+ CVPixelBufferLockBaseAddress(buffer, 0);+ CGContextRef context = newContext(w, h, CVPixelBufferGetBaseAddress(buffer), CVPixelBufferGetBytesPerRow(buffer));+ draw(context, w, h, (double)i / fps);+ CGContextRelease(context);+ CVPixelBufferUnlockBaseAddress(buffer, 0);+ [adaptor appendPixelBuffer:buffer withPresentationTime:CMTimeMake(i, fps)];+ CVPixelBufferRelease(buffer);+ }+ [input markAsFinished];+ dispatch_semaphore_t done = dispatch_semaphore_create(0);+ [writer finishWritingWithCompletionHandler:^{ dispatch_semaphore_signal(done); }];+ dispatch_semaphore_wait(done, DISPATCH_TIME_FOREVER);+ return writer.status == AVAssetWriterStatusCompleted;+}++static void savePNG(CGImageRef image, NSURL *url) {+ CGImageDestinationRef destination = CGImageDestinationCreateWithURL((__bridge CFURLRef)url, (__bridge CFStringRef)UTTypePNG.identifier, 1, NULL);+ CGImageDestinationAddImage(destination, image, NULL);+ CGImageDestinationFinalize(destination);+ CFRelease(destination);+}++static CGImageRef frameAt(NSURL *url, double seconds) CF_RETURNS_RETAINED {+ AVAssetImageGenerator *generator = [AVAssetImageGenerator assetImageGeneratorWithAsset:[AVURLAsset URLAssetWithURL:url options:nil]];+ generator.requestedTimeToleranceBefore = generator.requestedTimeToleranceAfter = kCMTimeZero;+ return [generator copyCGImageAtTime:CMTimeMakeWithSeconds(seconds, 600) actualTime:NULL error:nil];+}++// RGBA bytes of `image`, freed by the caller.+static uint8_t *pixelsOf(CGImageRef image, size_t *width, size_t *height) {+ *width = CGImageGetWidth(image);+ *height = CGImageGetHeight(image);+ CGColorSpaceRef space = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);+ uint8_t *data = calloc(*width * *height, 4);+ CGContextRef context = CGBitmapContextCreate(data, *width, *height, 8, *width * 4, space, (CGBitmapInfo)kCGImageAlphaPremultipliedLast);+ CGContextDrawImage(context, CGRectMake(0, 0, *width, *height), image);+ CGContextRelease(context);+ CGColorSpaceRelease(space);+ return data;+}++// Share of pixels lighter than `above` in every channel in rows `from` to `to` of the height, counted from the+// top. The picture is dimmed to 0.75 of white (kMotionDim), and to 0.4 under the words (kBandDim), so past 225+// only the words reach, and past 120 in the band only the words, dimmed ones too.+static double whiteIn(CGImageRef image, double from, double to, int above) {+ size_t w, h, lit = 0, all = 0;+ uint8_t *p = pixelsOf(image, &w, &h);+ for (size_t y = from * h; y < to * h; y++) {+ for (size_t x = 0; x < w; x++, all++) {+ uint8_t *q = p + (y * w + x) * 4;+ if (q[0] > above && q[1] > above && q[2] > above) lit++;+ }+ }+ free(p);+ return all ? (double)lit / all : 0;+}++static double linear(double c) {+ c /= 255;+ return c <= 0.04045 ? c / 12.92 : pow((c + 0.055) / 1.055, 2.4);+}++// The relative luminance the brightest 5% of the picture reaches in rows `from` to `to`, and their mean.+static double brightIn(CGImageRef image, double from, double to, double *mean) {+ size_t w, h;+ uint8_t *p = pixelsOf(image, &w, &h);+ NSMutableArray<NSNumber *> *all = [NSMutableArray array];+ double sum = 0;+ for (size_t y = from * h; y < to * h; y += 2) {+ for (size_t x = 0; x < w; x += 2) {+ uint8_t *q = p + (y * w + x) * 4;+ double l = 0.2126 * linear(q[0]) + 0.7152 * linear(q[1]) + 0.0722 * linear(q[2]);+ [all addObject:@(l)];+ sum += l;+ }+ }+ free(p);+ [all sortUsingSelector:@selector(compare:)];+ *mean = sum / all.count;+ return all[(NSUInteger)(all.count * 0.95)].doubleValue;+}++// Mean absolute difference of two images of the same size, per channel, 0 to 255.+static double differ(CGImageRef a, CGImageRef b) {+ size_t w, h, w2, h2;+ uint8_t *p = pixelsOf(a, &w, &h), *q = pixelsOf(b, &w2, &h2);+ double sum = 0;+ if (w == w2 && h == h2) {+ for (size_t i = 0; i < w * h * 4; i++) if (i % 4 != 3) sum += abs(p[i] - q[i]);+ }+ free(p);+ free(q);+ return w == w2 && h == h2 ? sum / (w * h * 3) : 255;+}++typedef struct { double seconds; int frames; CGSize size; BOOL h264; } Read;++static Read readBack(NSURL *url) {+ AVURLAsset *asset = [AVURLAsset URLAssetWithURL:url options:@{AVURLAssetPreferPreciseDurationAndTimingKey: @YES}];+ AVAssetTrack *track = [asset tracksWithMediaType:AVMediaTypeVideo].firstObject;+ Read read = {CMTimeGetSeconds(asset.duration), 0, track.naturalSize, NO};+ if (!track) return read;+ read.h264 = CMFormatDescriptionGetMediaSubType((__bridge CMFormatDescriptionRef)track.formatDescriptions.firstObject) == kCMVideoCodecType_H264;+ AVAssetReader *reader = [AVAssetReader assetReaderWithAsset:asset error:nil];+ AVAssetReaderTrackOutput *output = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track outputSettings:nil];+ [reader addOutput:output];+ [reader startReading];+ CMSampleBufferRef sample;+ while ((sample = [output copyNextSampleBuffer])) {+ if (CMSampleBufferGetNumSamples(sample)) read.frames++;+ CFRelease(sample);+ }+ return read;+}++static double contrast(double lighter, double darker) {+ return (lighter + 0.05) / (darker + 0.05);+}++int motionMain(NSURL *out, int argc, char **argv) {+ printf("footprint at start %.1f MB\n", footprintMB(NO));+ // `motion one video` and `motion cut video`: a single line's clip, or the video cut alone as LockScreenMotion.x+ // cuts it, in a fresh process, for the memory one takes.+ if (argc == 2 && (!strcmp(argv[0], "one") || !strcmp(argv[0], "cut"))) {+ NSURL *video = [NSURL fileURLWithPath:[NSString stringWithUTF8String:argv[1]]];+ CFAbsoluteTime start = CFAbsoluteTimeGetCurrent();+ BOOL ok = !strcmp(argv[0], "one")+ ? SGLyricsClipWriteOverMotion([out URLByAppendingPathComponent:@"one.mp4"], video, @"(And I turn into a stalker)",+ @"I stare at the moon", @"When nighttime comes", 1, 4.5)+ : SGMotionClipShape(video, 0.75, [out URLByAppendingPathComponent:@"one-cut.mp4"]);+ printf("%s %s in %.0f ms; footprint now %.1f MB, peak %.1f MB\n", argv[0], ok ? "written" : "FAILED",+ (CFAbsoluteTimeGetCurrent() - start) * 1000, footprintMB(NO), footprintMB(YES));+ return ok ? 0 : 1;+ }+ NSMutableArray<NSURL *> *videos = [NSMutableArray array];+ NSURL *flag = [out URLByAppendingPathComponent:@"source-flag.mp4"], *dark = [out URLByAppendingPathComponent:@"source-dark.mp4"];+ check(makeVideo(flag, 720, 1280, 30, 6, drawFlag), @"made the bright stand-in, 720x1280, 30 fps, 6 s");+ check(makeVideo(dark, 904, 1206, 24, 8, drawDark), @"made the dark stand-in, 904x1206, 24 fps, 8 s");+ [videos addObjectsFromArray:@[flag, dark]];+ for (int i = 0; i < argc; i++) [videos addObject:[NSURL fileURLWithPath:[NSString stringWithUTF8String:argv[i]]]];++ // The lyrics clip as it is, Animated, for the Mac-to-phone ratio: the phone wrote 96 such frames in about 4 s.+ CGImageRef plain = SGLyricsClipBackdrop(NULL, SGLyricsClipSize(904));+ CFAbsoluteTime start = CFAbsoluteTimeGetCurrent();+ SGLyricsClipWrite([out URLByAppendingPathComponent:@"animated.mp4"], plain, @"Hold on", @"we're almost there", SGLyricsClipSize(904), SGLyricsClipAnimated);+ double animatedMs = (CFAbsoluteTimeGetCurrent() - start) * 1000;+ CGImageRelease(plain);+ printf("the Animated lyrics clip, 60 frames at 896x1200: %.0f ms, %.1f ms a frame\n", animatedMs, animatedMs / 60);++ NSArray *cases = @[+ // name, previous, line, next, from, seconds+ @[@"line", @"(And I turn into a stalker)", @"I stare at the moon", @"When nighttime comes", @1.0, @3.5],+ @[@"long", @"Hold on", @"And every word I never said keeps ringing out across the quiet of the night", @"until the morning finds us", @2.0, @5.0],+ @[@"wrap", @"", @"Round the end of the video", @"and on from its start", @-1.5, @3.0],+ @[@"whole", @"Before", @"A line longer than the video", @"", @3.0, @20.0],+ @[@"rest", @"", @"", @"The first line, coming up", @0.0, @4.0],+ @[@"bare", @"", @"", @"", @2.0, @2.0],+ ];+ for (NSURL *video in videos) {+ NSString *name = video.lastPathComponent.stringByDeletingPathExtension;+ AVURLAsset *asset = [AVURLAsset URLAssetWithURL:video options:@{AVURLAssetPreferPreciseDurationAndTimingKey: @YES}];+ AVAssetTrack *track = [asset tracksWithMediaType:AVMediaTypeVideo].firstObject;+ double length = CMTimeGetSeconds(asset.duration), fps = MIN(round(track.nominalFrameRate), 30); // kept to 30 fps+ CGSize cut = SGMotionClipCut(track.naturalSize, 0.75);+ printf("\n%s: %.0fx%.0f, %.2f s at %.0f fps, cut to %.0fx%.0f\n", name.UTF8String, track.naturalSize.width, track.naturalSize.height,+ length, fps, cut.width, cut.height);+ // The video cut alone, the decode and encode with nothing drawn, to tell the drawing's share.+ start = CFAbsoluteTimeGetCurrent();+ SGMotionClipShape(video, 0.75, [out URLByAppendingPathComponent:[name stringByAppendingString:@"-cut.mp4"]]);+ double cutMs = (CFAbsoluteTimeGetCurrent() - start) * 1000;+ printf(" the cut alone: %.0f ms for %.2f s, %.1f ms a frame\n", cutMs, length, cutMs / (length * fps));+ for (NSArray *c in cases) @autoreleasepool {+ double from = [c[4] doubleValue] < 0 ? length + [c[4] doubleValue] : [c[4] doubleValue], seconds = [c[5] doubleValue];+ double wanted = MIN(seconds, length);+ NSURL *url = [out URLByAppendingPathComponent:[NSString stringWithFormat:@"%@-%@.mp4", name, c[0]]];+ start = CFAbsoluteTimeGetCurrent();+ BOOL written = SGLyricsClipWriteOverMotion(url, video, c[1], c[2], c[3], from, seconds);+ double ms = (CFAbsoluteTimeGetCurrent() - start) * 1000;+ printf(" footprint after writing %.1f MB\n", footprintMB(NO));+ printf(" %s\n", url.lastPathComponent.UTF8String);+ check(written, [NSString stringWithFormat:@"written in %.0f ms, %.1f ms a frame, %.1f s of clip a second",+ ms, ms / (wanted * fps), wanted / (ms / 1000)]);+ if (!written) continue;+ Read read = readBack(url);+ check(fabs(read.seconds - wanted) < 0.05, [NSString stringWithFormat:@"lasts %.3f s (want %.3f)", read.seconds, wanted]);+ check(CGSizeEqualToSize(read.size, cut), [NSString stringWithFormat:@"is %.0fx%.0f", read.size.width, read.size.height]);+ check(read.h264, @"is H.264");+ check(abs(read.frames - (int)round(wanted * fps)) <= 1, [NSString stringWithFormat:@"holds %d frames (want %.0f)", read.frames, wanted * fps]);+ CGImageRef first = frameAt(url, 0), middle = frameAt(url, wanted / 2);+ if (!first || !middle) {+ check(NO, @"frames read back");+ continue;+ }+ savePNG(first, [url URLByAppendingPathExtension:@"png"]);+ BOOL words = [c[1] length] || [c[2] length] || [c[3] length];+ double band = [c[2] length] || ![c[3] length] ? whiteIn(first, 0.3, 0.7, 225) : whiteIn(first, 0.42, 0.58, 120);+ if (words) check(band > 0.003, [NSString stringWithFormat:@"words in the middle band (%.2f%% white)", band * 100]);+ else check(band < 0.0005, [NSString stringWithFormat:@"no words when there are none (%.3f%% white)", band * 100]);+ check(whiteIn(first, 0, 0.15, 225) < 0.0005 && whiteIn(first, 0.85, 1, 225) < 0.0005, @"nothing white under the clock or the controls");+ double moved = differ(first, middle);+ check(moved > 2, [NSString stringWithFormat:@"the picture moves (%.1f of 255 between the start and the middle)", moved]);+ CGImageRef preview = SGLyricsClipMotionFrame(video, c[1], c[2], c[3], from);+ double off = preview ? differ(preview, first) : 255;+ check(off < 10, [NSString stringWithFormat:@"the preview is the first frame (%.1f of 255 apart, the encode's loss)", off]);+ if (preview && [c[0] isEqual:@"line"]) savePNG(preview, [out URLByAppendingPathComponent:[name stringByAppendingString:@"-preview.png"]]);+ CGImageRelease(preview);+ if ([c[0] isEqual:@"bare"]) {+ // The picture under the words, as dimmed: white on its brightest 5%, and the dimmed lines' 40% white over it.+ double mean, top = brightIn(first, 0.42, 0.58, &mean);+ double dimTop = linear(255 * (0.4 + 0.6 * pow(top, 1 / 2.2)));+ printf(" under the words: brightest 5%% at luminance %.3f, mean %.3f; white on it %.1f:1, white on the mean %.1f:1, "+ "a dimmed line on the brightest %.1f:1\n", top, mean, contrast(1, top), contrast(1, mean), contrast(dimTop, top));+ CGImageRef source = frameAt(video, from);+ double sourceMean, sourceTop = brightIn(source, 0.42, 0.58, &sourceMean);+ printf(" the same rows undimmed: brightest 5%% at %.3f, white on it %.1f:1\n", sourceTop, contrast(1, sourceTop));+ CGImageRelease(source);+ }+ CGImageRelease(first);+ CGImageRelease(middle);+ }+ }+ printf("\nfootprint now %.1f MB, peak %.1f MB\n", footprintMB(NO), footprintMB(YES));+ printf("%s, clips in %s\n", failures ? "FAILED" : "all passed", out.path.UTF8String);+ return failures ? 1 : 0;+}diff --git a/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtwork.h b/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtwork.hindex 131318cd..2a39f447 100644--- a/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtwork.h+++ b/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtwork.h@@ -13,14 +13,20 @@ typedef NS_ENUM(NSInteger, SGLockArtwork) {SGLockArtworkMotion, // the Canvas, else Apple Music's animated cover (LockScreenMotion.x)SGLockArtworkLyrics, // a clip per line of lyrics (Shared/LockScreenLyrics/LyricsArtwork.x)SGLockArtworkEverySong, // Moving artwork, else the cover over a moving blur of it (LockScreenMotion.x)+ // The lyrics over the moving artwork, else Lyrics: LockScreenMotion.x finds the clip, LyricsArtwork.x+ // draws the lines over it.+ SGLockArtworkMotionLyrics,};// Off below iOS 26, which has no animated artwork on the lock screen.SGLockArtwork SGLockScreenArtwork(void);+static inline BOOL SGLockArtworkShowsLyrics(SGLockArtwork artwork) {+ return artwork == SGLockArtworkLyrics || artwork == SGLockArtworkMotionLyrics;+}// How the lyrics are drawn, stored as the index of SGLyricsClipStyle (Still unless changed).#define SGKeyLockScreenLyricsStyle @"spotifyglass.lockscreen.lyricsStyle"-// Off until switched on: the lock screen widget's star switches the choice between moving artwork and-// Lyrics (LockScreenStar.x), and takes the place of Spotify's like and dislike buttons. StarBack is the-// choice the star left for Lyrics, Moving artwork or Every song, to go back to.+// Off until switched on: the lock screen widget's star switches the choice between a choice with lyrics and+// one without (LockScreenStar.x), and takes the place of Spotify's like and dislike buttons. StarBack is the+// choice the star left, to go back to.#define SGKeyLockScreenStar @"spotifyglass.lockscreen.star"#define SGKeyLockScreenStarBack @"spotifyglass.lockscreen.star.back"// Off until switched on: download in Low Data Mode too.diff --git a/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtworkSettings.m b/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtworkSettings.mindex 2fae6e16..1d21898b 100644--- a/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtworkSettings.m+++ b/tweak/Sources/Shared/AnimatedArtwork/AnimatedArtworkSettings.m@@ -18,13 +18,14 @@ SGLockArtwork SGLockScreenArtwork(void) {NSArray *SGLockScreenMotionRows(void) {if (@available(iOS 26.0, *)) {SGLockScreenArtwork(); // moves the old switch over before the row reads the new key- SGModRow *style = SGChoiceRow(@"Lyrics style", @"Still draws each line once; Animated breathes the cover behind it",+ SGModRow *style = SGChoiceRow(@"Lyrics style", @"Still draws each line once; Animated breathes the cover behind it. "+ @"With moving artwork, for songs that have none",SGKeyLockScreenLyricsStyle, @[@"Still", @"Animated"], 0);- style.visible = ^BOOL { return SGLockScreenArtwork() == SGLockArtworkLyrics; };+ style.visible = ^BOOL { return SGLockArtworkShowsLyrics(SGLockScreenArtwork()); };SGModRow *lowData = SGOptionRow(@"Download in Low Data Mode", @"Moving artwork, up to about 7 MB a song", SGKeyMotionLowData);lowData.visible = ^BOOL {SGLockArtwork artwork = SGLockScreenArtwork();- return artwork == SGLockArtworkMotion || artwork == SGLockArtworkEverySong;+ return artwork == SGLockArtworkMotion || artwork == SGLockArtworkEverySong || artwork == SGLockArtworkMotionLyrics;};SGModRow *sources = SGMotionSourcesRow();sources.visible = lowData.visible;@@ -34,8 +35,10 @@ NSArray *SGLockScreenMotionRows(void) {star.visible = ^BOOL { return SGLockScreenArtwork() != SGLockArtworkOff; };return @[SGChoiceRow(@"Full-screen artwork", @"The Canvas or Apple Music's animated cover, or the lyrics a line at a time. "- @"Every song puts the cover over a moving blur of it where a song has neither",- SGKeyLockScreenArtwork, @[@"Off", @"Moving artwork", @"Lyrics", @"Every song"], SGLockArtworkOff),+ @"Every song puts the cover over a moving blur of it where a song has neither. "+ @"Moving artwork with lyrics puts the lines over the Canvas or animated cover",+ SGKeyLockScreenArtwork, @[@"Off", @"Moving artwork", @"Lyrics", @"Every song", @"Moving artwork with lyrics"],+ SGLockArtworkOff),style,star,sources,diff --git a/tweak/Sources/Shared/AnimatedArtwork/LockScreenMotion.x b/tweak/Sources/Shared/AnimatedArtwork/LockScreenMotion.xindex d3ae9b5f..79903110 100644--- a/tweak/Sources/Shared/AnimatedArtwork/LockScreenMotion.x+++ b/tweak/Sources/Shared/AnimatedArtwork/LockScreenMotion.x@@ -11,12 +11,14 @@//// The choice is read on every track, and a change applies at once: Off takes the clip away, Moving artwork// and Every song look the playing track up. Lyrics is LyricsArtwork.x's, which reads it the same way, so the-// lock screen widget's star (LockScreenStar.x) can swap the two without a restart.+// lock screen widget's star (LockScreenStar.x) can swap the two without a restart. Moving artwork with+// lyrics looks the track up too and hands its clip to LyricsArtwork.x, which draws the lines over it.#import <MediaPlayer/MediaPlayer.h>#import "Core/SGCore.h"#import "AnimatedArtwork.h"#import "SGFluidClip.h"#import "SGMotionClip.h"+#import "Shared/LockScreenLyrics/LockScreenLyrics.h"#import "Shared/LockScreenLyrics/SGLyricsClip.h"#import "Shared/Player/NowPlayingExtras.h"#import "Shared/Player/PlayerState.h"@@ -31,8 +33,9 @@ static SGLockArtwork sg_choice;static dispatch_queue_t sg_queue; // cuts clips and makes the cover's, one at a timestatic SGMotionFollower *sg_follower;+// Moving artwork with lyrics walks the sources too, and hands the clip to LyricsArtwork.x.static BOOL motionOn(SGLockArtwork choice) {- return choice == SGLockArtworkMotion || choice == SGLockArtworkEverySong;+ return choice == SGLockArtworkMotion || choice == SGLockArtworkEverySong || choice == SGLockArtworkMotionLyrics;}// The key a clip shown at `shown` goes under, and the shape it is cut to (width over height): 1:1 for a@@ -163,7 +166,11 @@ static void showCover(NSString *picture) {sg_choice = SGLockScreenArtwork();return motionOn(sg_choice);} found:^(NSString *uri, NSURL *file) {- if (file) show(file);+ // Under the lyrics instead, and never as this file's extra too: both would fill the same 3:4 key.+ if (sg_choice == SGLockArtworkMotionLyrics) {+ if (file) SGLyricsArtworkSetMotion(uri, file);+ SGLog(@"lock motion: %@ for the lyrics of %@", file ? file.lastPathComponent : @"no clip", uri);+ } else if (file) show(file);else if (sg_choice == SGLockArtworkEverySong) showCover(sg_picture);}];// Heard on the writing thread and handed to main without waiting: a main-queue observer makes everydiff --git a/tweak/Sources/Shared/AnimatedArtwork/LockScreenStar.x b/tweak/Sources/Shared/AnimatedArtwork/LockScreenStar.xindex 800353d4..2b599dc7 100644--- a/tweak/Sources/Shared/AnimatedArtwork/LockScreenStar.x+++ b/tweak/Sources/Shared/AnimatedArtwork/LockScreenStar.x@@ -1,7 +1,7 @@// The lock screen widget's star: a tap switches the full-screen artwork between moving artwork and the// lyrics, at once, by writing the choice LockScreenMotion.x and LyricsArtwork.x both watch. The star is// MediaPlayer's like command, the slot Apple Music's Favorite star takes; it is filled while the lyrics-// show. Spotify's own like and dislike buttons (its like_dislike flag) take the same slot, so the flag+// show, under Lyrics or Moving artwork with lyrics. Spotify's own like and dislike buttons (its like_dislike flag) take the same slot, so the flag// is forced off while the switch is on and the Lock screen page's row shows it locked. iOS 26 only, as// the artwork is.#import <MediaPlayer/MediaPlayer.h>@@ -21,12 +21,13 @@ static MPRemoteCommandHandlerStatus tapped(void) {SGLockArtwork choice = SGLockScreenArtwork();if (choice == SGLockArtworkOff) return MPRemoteCommandHandlerStatusCommandFailed;dispatch_async(dispatch_get_main_queue(), ^{- if (choice == SGLockArtworkLyrics) {- SGSetInt(SGKeyLockScreenArtwork, SGInt(SGKeyLockScreenStarBack, SGLockArtworkMotion));- } else {- SGSetInt(SGKeyLockScreenStarBack, choice);- SGSetInt(SGKeyLockScreenArtwork, SGLockArtworkLyrics);- }+ // Back to the choice the star left when that one is on the other side, with lyrics or without; else to+ // Lyrics or Moving artwork. So Lyrics and Moving artwork with lyrics each swap with the moving artwork.+ BOOL lyrics = SGLockArtworkShowsLyrics(choice);+ SGLockArtwork back = (SGLockArtwork)SGInt(SGKeyLockScreenStarBack, SGLockArtworkOff);+ BOOL valid = back > SGLockArtworkOff && back <= SGLockArtworkMotionLyrics && SGLockArtworkShowsLyrics(back) != lyrics;+ SGSetInt(SGKeyLockScreenStarBack, choice);+ SGSetInt(SGKeyLockScreenArtwork, valid ? back : lyrics ? SGLockArtworkMotion : SGLockArtworkLyrics);SGLog(@"lock star: tapped, the lock screen takes %ld", (long)SGLockScreenArtwork());});return MPRemoteCommandHandlerStatusSuccess;@@ -36,7 +37,7 @@ static MPRemoteCommandHandlerStatus tapped(void) {// changed, or when Spotify's lock screen policy turned it off under the star, which the log says.static void arm(void) {static BOOL lastOn, lastActive;- BOOL on = starOn(), active = on && SGLockScreenArtwork() == SGLockArtworkLyrics;+ BOOL on = starOn(), active = on && SGLockArtworkShowsLyrics(SGLockScreenArtwork());if (sg_like && !sg_like.enabled) {static int told;if (told++ < 3) SGLog(@"lock star: the like command was off under the star, set again");diff --git a/tweak/Sources/Shared/LockScreenLyrics/LockScreenLyrics.h b/tweak/Sources/Shared/LockScreenLyrics/LockScreenLyrics.hindex 4e92f637..fa1e17e4 100644--- a/tweak/Sources/Shared/LockScreenLyrics/LockScreenLyrics.h+++ b/tweak/Sources/Shared/LockScreenLyrics/LockScreenLyrics.h@@ -4,3 +4,7 @@#import <Foundation/Foundation.h>#define SGKeyLockScreenLyrics @"spotifyglass.lockScreenLyrics"++// Moving artwork with lyrics: the moving artwork clip found for the track `uri`, a local file, which+// LyricsArtwork.x draws the lines over while that track plays. Main queue.+void SGLyricsArtworkSetMotion(NSString *uri, NSURL *file);diff --git a/tweak/Sources/Shared/LockScreenLyrics/LyricsArtwork.x b/tweak/Sources/Shared/LockScreenLyrics/LyricsArtwork.xindex 0e885955..d2900a0d 100644--- a/tweak/Sources/Shared/LockScreenLyrics/LyricsArtwork.x+++ b/tweak/Sources/Shared/LockScreenLyrics/LyricsArtwork.x@@ -3,8 +3,15 @@// changes. Nothing is drawn until the lock screen asks for that artwork's video, so lines sung with the// screen off cost nothing. It rides on Spotify's now playing info as an extra (NowPlayingExtras.h), like// the moving artwork it takes the place of.+//+// Moving artwork with lyrics draws the lines over the track's moving artwork instead, which+// LockScreenMotion.x finds and hands over (SGLyricsArtworkSetMotion): each line's clip is the stretch of it+// the song is at while the line shows, taken round its length, with the line sung last dimmed above. Until the+// clip comes, and for a track that has none, the lines are drawn as Lyrics draws them; the clip coming+// changes every ID, so the line showing is offered again over it.#import <MediaPlayer/MediaPlayer.h>#import "Core/SGCore.h"+#import "LockScreenLyrics.h"#import "SGLyricsClip.h"#import "Shared/AnimatedArtwork/AnimatedArtwork.h"#import "Shared/Lyrics/Lyrics.h"@@ -15,8 +22,25 @@ static const NSTimeInterval kTick = 0.25;// Past a line's sung end by this much, with the next line at least this far off, the line is let go and// only the next one shows, dimmed (as the artist row does in LockScreenLyrics.x).static const NSInteger kBreakMs = 4000;+// A line's clip over moving artwork lasts from the line's start to the next one's, a second longer for the+// lock screen showing it late, and between these: the cost of a clip is its length.+static const double kShortest = 2, kLongest = 8;++// What a line's artwork shows, kept by its request blocks.+@interface SGLockLine : NSObject+@property (nonatomic, copy) NSString *previous, *line, *next; // nil for none+@property (nonatomic) UIImage *cover;+@property (nonatomic) SGLyricsClipStyle style;+@property (nonatomic) NSURL *motion; // the moving artwork under the words, nil for the blurred cover+@property (nonatomic) double from, seconds; // where in the song the clip starts and how long it is+@end++@implementation SGLockLine+@endstatic dispatch_source_t sg_timer;+static NSString *sg_motionURI; // the track the moving artwork is for, and its file (SGLyricsArtworkSetMotion)+static NSURL *sg_motion;static NSString *sg_track; // the track the clips are forstatic UIImage *sg_cover; // its cover, once Spotify's now playing info has one for itstatic NSString *sg_shownID; // the artwork handed over last, nil for none; read on sg_queue too@@ -70,44 +94,55 @@ static NSURL *clipFile(NSString *artworkID) {return [folder() URLByAppendingPathComponent:[artworkID stringByAppendingPathExtension:@"mp4"]];}-// On sg_previewQueue.-static UIImage *previewFor(NSString *artworkID, NSString *line, NSString *next, UIImage *cover, CGSize size) {+// On sg_previewQueue. Over moving artwork that will not read, the blurred cover's.+static UIImage *previewFor(NSString *artworkID, SGLockLine *line, CGSize size) {UIImage *preview = [sg_previews objectForKey:artworkID];if (preview) return preview;- CGImageRef backdrop = copyBackdropFor(cover, size);- CGImageRef frame = SGLyricsClipFrame(backdrop, line, next, size);- CGImageRelease(backdrop);+ CGImageRef frame = line.motion ? SGLyricsClipMotionFrame(line.motion, line.previous, line.line, line.next, line.from) : NULL;+ if (!frame) {+ CGImageRef backdrop = copyBackdropFor(line.cover, size);+ frame = SGLyricsClipFrame(backdrop, line.line, line.next, size);+ CGImageRelease(backdrop);+ }if (frame) [sg_previews setObject:(preview = [UIImage imageWithCGImage:frame]) forKey:artworkID];CGImageRelease(frame);return preview;}-// On sg_queue. A clip drawn ahead is found on disk and handed over at once.-static BOOL writeClip(NSString *artworkID, NSString *line, NSString *next, UIImage *cover, SGLyricsClipStyle style, CGSize size) {+// On sg_queue. A clip drawn ahead is found on disk and handed over at once. Over moving artwork that will not+// read, it is drawn over the blurred cover.+static BOOL writeClip(NSString *artworkID, SGLockLine *line, CGSize size) {NSURL *file = clipFile(artworkID);if ([NSFileManager.defaultManager fileExistsAtPath:file.path]) return YES;CFAbsoluteTime start = CFAbsoluteTimeGetCurrent();- CGImageRef backdrop = copyBackdropFor(cover, size);- BOOL ok = SGLyricsClipWrite(file, backdrop, line, next, size, style);- CGImageRelease(backdrop);- // The first few of a launch, and any that fails: one a line would fill the log.- static NSUInteger told;- if (!ok || told++ < 5) SGLog(@"lock lyrics: %@ %@ in %.0f ms", artworkID, ok ? @"written" : @"not written", (CFAbsoluteTimeGetCurrent() - start) * 1000);+ BOOL over = line.motion && SGLyricsClipWriteOverMotion(file, line.motion, line.previous, line.line, line.next, line.from, line.seconds);+ BOOL ok = over;+ if (!ok) {+ if (line.motion) SGLog(@"lock lyrics: %@ could not be drawn over %@, drawn over the cover", artworkID, line.motion.lastPathComponent);+ CGImageRef backdrop = copyBackdropFor(line.cover, size);+ ok = SGLyricsClipWrite(file, backdrop, line.line, line.next, size, line.style);+ CGImageRelease(backdrop);+ }+ // The first few of a launch of each kind, and any that fails: one a line would fill the log.+ static NSUInteger told[2];+ if (!ok || told[over]++ < 5) {+ SGLog(@"lock lyrics: %@ %@ in %.0f ms%@", artworkID, ok ? @"written" : @"not written", (CFAbsoluteTimeGetCurrent() - start) * 1000,+ over ? [NSString stringWithFormat:@", up to %.1f s of %@ from %.1f s", line.seconds, line.motion.lastPathComponent, line.from] : @"");+ }return ok;}// The next line's preview and clip, drawn while this one shows: a line change then hands both over at once, where// drawing them on request left the lock screen on the plain cover for half a second.-static void drawAhead(NSString *artworkID, NSString *line, NSString *next, UIImage *cover, SGLyricsClipStyle style) {+static void drawAhead(NSString *artworkID, SGLockLine *line) {CFAbsoluteTime time = CFAbsoluteTimeGetCurrent();CGSize size = SGLyricsClipSize(SGMotionPixels());- if (time - sg_previewAskedAt <= 15) dispatch_async(sg_previewQueue, ^{ previewFor(artworkID, line, next, cover, size); });- if (time - sg_askedAt <= 15) dispatch_async(sg_queue, ^{ writeClip(artworkID, line, next, cover, style, size); });+ if (time - sg_previewAskedAt <= 15) dispatch_async(sg_previewQueue, ^{ previewFor(artworkID, line, size); });+ if (time - sg_askedAt <= 15) dispatch_async(sg_queue, ^{ writeClip(artworkID, line, size); });}API_AVAILABLE(ios(26.0))-static MPMediaItemAnimatedArtwork *artworkFor(NSString *artworkID, NSString *line, NSString *next, UIImage *cover,- SGLyricsClipStyle style) {+static MPMediaItemAnimatedArtwork *artworkFor(NSString *artworkID, SGLockLine *line) {CGSize size = SGLyricsClipSize(SGMotionPixels());return [[MPMediaItemAnimatedArtwork alloc] initWithArtworkID:artworkIDpreviewImageRequestHandler:^(CGSize wanted, void (^completion)(UIImage *)) {@@ -119,7 +154,7 @@ static MPMediaItemAnimatedArtwork *artworkFor(NSString *artworkID, NSString *linreturn;}dispatch_async(sg_previewQueue, ^{- completion(previewFor(artworkID, line, next, cover, size));+ completion(previewFor(artworkID, line, size));static NSUInteger told;if (told++ < 10) SGLog(@"lock lyrics: preview of %@ was not drawn ahead", artworkID);});@@ -132,7 +167,7 @@ static MPMediaItemAnimatedArtwork *artworkFor(NSString *artworkID, NSString *lincompletion(nil);return;}- completion(writeClip(artworkID, line, next, cover, style, size) ? clipFile(artworkID) : nil);+ completion(writeClip(artworkID, line, size) ? clipFile(artworkID) : nil);});}];}@@ -147,6 +182,30 @@ static NSString *textAt(NSArray<SGKaraokeLine *> *lines, NSInteger index) {return index >= 0 && index < (NSInteger)lines.count ? SGKaraokeLineText(lines[index]) : nil;}+void SGLyricsArtworkSetMotion(NSString *uri, NSURL *file) {+ sg_motionURI = uri;+ sg_motion = file;+}++// What the artwork for line `index` shows, `resting` once its break has come (index -1, before the first line,+// always rests). Over moving artwork, the stretch of song from when it shows until the next artwork does.+static SGLockLine *lineAt(NSArray<SGKaraokeLine *> *lines, NSInteger index, BOOL resting, NSURL *motion,+ UIImage *cover, SGLyricsClipStyle style) {+ SGLockLine *shown = [SGLockLine new];+ // Resting, the line sung is let go and the one coming shows alone.+ shown.line = resting ? nil : textAt(lines, index);+ shown.next = textAt(lines, index + 1);+ shown.previous = motion && !resting ? textAt(lines, index - 1) : nil;+ shown.cover = cover;+ shown.style = style;+ shown.motion = motion;+ NSInteger start = index < 0 ? 0 : resting ? SGKaraokeSungEnd(lines[index]) + kBreakMs : lines[index].start;+ NSInteger end = index + 1 < (NSInteger)lines.count ? lines[index + 1].start : index < 0 ? start : SGKaraokeSungEnd(lines[index]) + kBreakMs;+ shown.from = start / 1000.0;+ shown.seconds = MIN(MAX((end - start) / 1000.0 + 1, kShortest), kLongest);+ return shown;+}+static void tick(void) {if (@available(iOS 26.0, *)) {SPTPlayerState *state = SGPlayerState();@@ -183,7 +242,13 @@ static void tick(void) {BOOL nextFarOff = index + 1 == (NSInteger)lines.count || lines[index + 1].start - position > kBreakMs;BOOL resting = index < 0 || (position > SGKaraokeSungEnd(lines[index]) + kBreakMs && nextFarOff);SGLyricsClipStyle style = (SGLyricsClipStyle)SGInt(SGKeyLockScreenLyricsStyle, SGLyricsClipStill);- NSString *artworkID = [NSString stringWithFormat:@"%@-%ld%@-%ld", trackID, (long)index, resting ? @"r" : @"", (long)style];+ NSURL *motion = SGLockScreenArtwork() == SGLockArtworkMotionLyrics && [sg_motionURI isEqualToString:SGURIString(state.track.URI)]+ ? sg_motion : nil;+ // The look ends the ID: m and a tag of the file over moving artwork, so a Canvas that comes after an animated+ // cover redraws the line showing too; else the style.+ NSString *look = motion ? [NSString stringWithFormat:@"m%04lx", (unsigned long)(motion.lastPathComponent.hash & 0xffff)]+ : [NSString stringWithFormat:@"%ld", (long)style];+ NSString *artworkID = [NSString stringWithFormat:@"%@-%ld%@-%@", trackID, (long)index, resting ? @"r" : @"", look];if ([artworkID isEqualToString:sg_shownID]) return;// Spotify's request block for its cover, read on main and kept once it is this track's.if (!sg_cover) {@@ -193,15 +258,15 @@ static void tick(void) {sg_cover = [artwork imageWithSize:CGSizeMake(600, 600)];}}- // Resting, the line sung is let go and the one coming shows alone.- NSString *line = resting ? nil : textAt(lines, index), *next = textAt(lines, index + 1);if (![sg_shownID hasPrefix:trackID]) SGLog(@"lock lyrics: lines offered for %@", trackID);+ if (motion && ![sg_shownID hasSuffix:[@"-" stringByAppendingString:look]]) SGLog(@"lock lyrics: the lines of %@ over %@", trackID, motion.lastPathComponent);setShown(artworkID);- SGNowPlayingSetExtras(@"lyrics", @{MPNowPlayingInfoProperty3x4AnimatedArtwork: artworkFor(artworkID, line, next, sg_cover, style)},+ SGNowPlayingSetExtras(@"lyrics", @{MPNowPlayingInfoProperty3x4AnimatedArtwork:+ artworkFor(artworkID, lineAt(lines, index, resting, motion, sg_cover, style))},state.track.trackTitle);if (index + 1 < (NSInteger)lines.count) {- NSString *ahead = [NSString stringWithFormat:@"%@-%ld-%ld", trackID, (long)index + 1, (long)style];- drawAhead(ahead, textAt(lines, index + 1), textAt(lines, index + 2), sg_cover, style);+ NSString *ahead = [NSString stringWithFormat:@"%@-%ld-%@", trackID, (long)index + 1, look];+ drawAhead(ahead, lineAt(lines, index + 1, NO, motion, sg_cover, style));}}}@@ -237,7 +302,7 @@ static void setTicking(BOOL on) {// A paused player's line does not move, and the lock screen keeps the clip it was left on. Picked or not is read// each time, so a change on the Lock screen page applies at once.- (void)playerStateDidChange:(SPTPlayerState *)state {- BOOL on = SGLockScreenArtwork() == SGLockArtworkLyrics;+ BOOL on = SGLockArtworkShowsLyrics(SGLockScreenArtwork());setTicking(on && !state.isPaused);if (on) tick();else clear();@@ -262,11 +327,11 @@ static void setTicking(BOOL on) {dispatch_async(dispatch_get_main_queue(), ^{if (SGLockScreenArtwork() == choice) return;choice = SGLockScreenArtwork();- SGLog(@"lock lyrics: %@", choice == SGLockArtworkLyrics ? @"picked" : @"no longer picked");+ SGLog(@"lock lyrics: %@", !SGLockArtworkShowsLyrics(choice) ? @"no longer picked" : choice == SGLockArtworkLyrics ? @"picked" : @"picked over the moving artwork");[observer playerStateDidChange:SGPlayerState()];});}];- SGLog(@"lock lyrics: %@, the lock screen takes %@", choice == SGLockArtworkLyrics ? @"on" : @"off",+ SGLog(@"lock lyrics: %@, the lock screen takes %@", !SGLockArtworkShowsLyrics(choice) ? @"off" : choice == SGLockArtworkLyrics ? @"on" : @"on over the moving artwork",MPNowPlayingInfoCenter.supportedAnimatedArtworkKeys);}}diff --git a/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.h b/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.hindex ef594429..b3260822 100644--- a/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.h+++ b/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.h@@ -1,5 +1,5 @@// A line of lyrics as a short video for the lock screen's full-screen artwork (LyricsArtwork.x): the-// line, and the next one dimmed under it, over the cover blurred. Core Graphics, Core Text, Core Image+// line, and the next one dimmed under it, over the cover blurred or over the moving artwork. Core Graphics, Core Text, Core Image// and AVFoundation only, so harness/lyrics-clip renders the same files on the Mac.#import <CoreGraphics/CoreGraphics.h>#import <Foundation/Foundation.h>@@ -17,3 +17,14 @@ CGImageRef SGLyricsClipBackdrop(CGImageRef cover, CGSize size) CF_RETURNS_RETAINCGImageRef SGLyricsClipFrame(CGImageRef backdrop, NSString *line, NSString *next, CGSize size) CF_RETURNS_RETAINED;// Writes the clip as H.264 to `file`, replacing it, and returns once it is written; NO if it was not.BOOL SGLyricsClipWrite(NSURL *file, CGImageRef backdrop, NSString *line, NSString *next, CGSize size, SGLyricsClipStyle style);++// The words over moving artwork instead: `motion` is a local video, played from `from` seconds into the+// song, taken round its length, for `seconds`, round its end as often as needed; when the video is no+// longer than that, the whole of it, turned to start there, so the lock screen's loop has no seam. The+// line sung last shows dimmed above the line. Written at the largest 3:4 cut of the video, about its+// middle, at the video's frame rate up to 30 fps. NO for a video stood on its side by its transform, or+// one that does not read. Blocks, so never on the main queue.+BOOL SGLyricsClipWriteOverMotion(NSURL *file, NSURL *motion, NSString *previous, NSString *line, NSString *next,+ double from, double seconds);+// That clip's first frame, for the preview image, NULL where the clip would not be written.+CGImageRef SGLyricsClipMotionFrame(NSURL *motion, NSString *previous, NSString *line, NSString *next, double from) CF_RETURNS_RETAINED;diff --git a/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m b/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.mindex e22ffd16..c03ec946 100644--- a/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m+++ b/tweak/Sources/Shared/LockScreenLyrics/SGLyricsClip.m@@ -1,5 +1,6 @@// SGLyricsClip.h says what this draws. The words are laid out once per clip and only the backdrop moves,// so an animated clip costs a scaled image draw and an encode a frame.+#import <Accelerate/Accelerate.h>#import <AVFoundation/AVFoundation.h>#import <CoreImage/CoreImage.h>#import <CoreText/CoreText.h>@@ -23,6 +24,15 @@ static const int32_t kStillSeconds = 2;static const int32_t kLoopSeconds = 4;static const int32_t kFPS = 15; // the breath is slow; at 24 a line took about 4 s to write on the CPUstatic const CGFloat kBreath = 0.06; // how much bigger the cover grows at the top of a breath+// Over moving artwork, judged on bright, busy and dark clips in harness/lyrics-clip: the picture is dimmed by+// kMotionDim, and by kBandDim where the words sit, fading between the two over kBandReach of the height above+// and below the middle, so white in it stays under 0.4 of the words' white there; behind the words it is+// blurred, a tent kSoftWidth of the width across, so its edges do not cross the letters; the words carry a+// tight dark shadow kShadowBlur of the font size wide, and the dimmed lines are kMotionNextAlpha white.+static const CGFloat kMotionDim = 0.25, kBandDim = 0.6, kBandReach = 0.3, kSoftWidth = 0.06;+static const CGFloat kShadowAlpha = 0.9, kShadowBlur = 0.15, kMotionNextAlpha = 0.55;+// A faster video keeps only this many frames a second: each costs a draw on the CPU.+static const double kMotionFPS = 30;CGSize SGLyricsClipSize(CGFloat pixels) {CGFloat width = MAX(floor(pixels / 16) * 16, 16);@@ -74,7 +84,7 @@ CGImageRef SGLyricsClipBackdrop(CGImageRef cover, CGSize size) {return backdrop;}-static NSAttributedString *wordsAt(NSString *line, NSString *next, CGFloat fontSize) {+static NSAttributedString *wordsAt(NSString *previous, NSString *line, NSString *next, CGFloat fontSize, CGFloat dimAlpha) {CTFontRef font = CTFontCreateUIFontForLanguage(kCTFontUIFontEmphasizedSystem, fontSize, NULL);CGFloat spacing = fontSize * 0.6;CTParagraphStyleSetting settings[] = {@@ -82,19 +92,16 @@ static NSAttributedString *wordsAt(NSString *line, NSString *next, CGFloat fontS};CTParagraphStyleRef paragraph = CTParagraphStyleCreate(settings, 1);CGColorSpaceRef space = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);- CGFloat white[] = {1, 1, 1, 1}, dim[] = {1, 1, 1, kNextAlpha};+ CGFloat white[] = {1, 1, 1, 1}, dim[] = {1, 1, 1, dimAlpha};CGColorRef lit = CGColorCreate(space, white), unlit = CGColorCreate(space, dim);NSMutableAttributedString *words = [NSMutableAttributedString new];NSDictionary *base = @{(id)kCTFontAttributeName: (__bridge id)font, (id)kCTParagraphStyleAttributeName: (__bridge id)paragraph};- if (line.length) {+ NSString *texts[] = {previous, line, next};+ for (int i = 0; i < 3; i++) {+ if (!texts[i].length) continue;NSMutableDictionary *attributes = [base mutableCopy];- attributes[(id)kCTForegroundColorAttributeName] = (__bridge id)lit;- [words appendAttributedString:[[NSAttributedString alloc] initWithString:line attributes:attributes]];- }- if (next.length) {- NSMutableDictionary *attributes = [base mutableCopy];- attributes[(id)kCTForegroundColorAttributeName] = (__bridge id)unlit;- NSString *text = words.length ? [@"\n" stringByAppendingString:next] : next;+ attributes[(id)kCTForegroundColorAttributeName] = (__bridge id)(i == 1 ? lit : unlit);+ NSString *text = words.length ? [@"\n" stringByAppendingString:texts[i]] : texts[i];[words appendAttributedString:[[NSAttributedString alloc] initWithString:text attributes:attributes]];}CFRelease(font);@@ -105,9 +112,11 @@ static NSAttributedString *wordsAt(NSString *line, NSString *next, CGFloat fontSreturn words;}-// The words alone on a clear image the size of the clip, nil when there are none.-static CGImageRef newWords(NSString *line, NSString *next, CGSize size) CF_RETURNS_RETAINED {- if (!line.length && !next.length) return NULL;+// The words alone on a clear image the size of the clip, nil when there are none. Over moving artwork they+// carry a shadow, the dimmed lines are less dim, and the image is only the words' box, which `drawn` gets+// the place of in the clip.+static CGImageRef newWords(NSString *previous, NSString *line, NSString *next, CGSize size, CGRect *drawn) CF_RETURNS_RETAINED {+ if (!previous.length && !line.length && !next.length) return NULL;CGFloat width = round(size.width * kColumn);CGFloat fontSize = size.width * kFont;CTFramesetterRef setter = NULL;@@ -115,7 +124,7 @@ static CGImageRef newWords(NSString *line, NSString *next, CGSize size) CF_RETUR// A long line is set smaller until the two lines fit; past the smallest size it runs taller.for (CGFloat scale = 1; scale >= kSmallest - 0.01; scale -= 0.1) {if (setter) CFRelease(setter);- setter = CTFramesetterCreateWithAttributedString((__bridge CFAttributedStringRef)wordsAt(line, next, fontSize * scale));+ setter = CTFramesetterCreateWithAttributedString((__bridge CFAttributedStringRef)wordsAt(previous, line, next, fontSize * scale, drawn ? kMotionNextAlpha : kNextAlpha));fits = CTFramesetterSuggestFrameSizeWithConstraints(setter, CFRangeMake(0, 0), NULL, CGSizeMake(width, CGFLOAT_MAX), NULL);if (fits.height <= size.height * kTallest) break;}@@ -125,9 +134,25 @@ static CGImageRef newWords(NSString *line, NSString *next, CGSize size) CF_RETURCGPathRef path = CGPathCreateWithRect(box, NULL);CTFrameRef frame = CTFramesetterCreateFrame(setter, CFRangeMake(0, 0), path, NULL);CGContextRef context = newContext(size, NULL, 0);+ if (drawn) {+ CGColorSpaceRef space = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);+ CGFloat shade[] = {0, 0, 0, kShadowAlpha};+ CGColorRef color = CGColorCreate(space, shade);+ CGContextSetShadowWithColor(context, CGSizeZero, fontSize * kShadowBlur, color);+ CGColorRelease(color);+ CGColorSpaceRelease(space);+ }CTFrameDraw(frame, context);CGImageRef words = CGBitmapContextCreateImage(context);CGContextRelease(context);+ if (drawn) {+ // Out to the shadow's reach, inside the clip; an image is cut from the top, Core Graphics counts from the bottom.+ *drawn = CGRectIntersection(CGRectIntegral(CGRectInset(box, -fontSize, -fontSize)), (CGRect){CGPointZero, size});+ CGImageRef cut = CGImageCreateWithImageInRect(words, CGRectMake(drawn->origin.x, size.height - CGRectGetMaxY(*drawn),+ drawn->size.width, drawn->size.height));+ CGImageRelease(words);+ words = cut;+ }CFRelease(frame);CGPathRelease(path);CFRelease(setter);@@ -143,7 +168,7 @@ static void drawFrame(CGContextRef context, CGImageRef backdrop, CGImageRef word}CGImageRef SGLyricsClipFrame(CGImageRef backdrop, NSString *line, NSString *next, CGSize size) {- CGImageRef words = newWords(line, next, size);+ CGImageRef words = newWords(nil, line, next, size, NULL);CGContextRef context = newContext(size, NULL, 0);drawFrame(context, backdrop, words, size, 0);CGImageRef frame = CGBitmapContextCreateImage(context);@@ -176,7 +201,7 @@ BOOL SGLyricsClipWrite(NSURL *file, CGImageRef backdrop, NSString *line, NSStrin[writer addInput:input];if (![writer startWriting]) return NO;[writer startSessionAtSourceTime:kCMTimeZero];- CGImageRef words = newWords(line, next, size);+ CGImageRef words = newWords(nil, line, next, size, NULL);BOOL ok = YES;for (int32_t i = 0; i < frames && ok; i++) {while (!input.readyForMoreMediaData && writer.status == AVAssetWriterStatusWriting) [NSThread sleepForTimeInterval:0.002];@@ -207,3 +232,254 @@ BOOL SGLyricsClipWrite(NSURL *file, CGImageRef backdrop, NSString *line, NSStrin[NSFileManager.defaultManager removeItemAtURL:file error:nil];return [NSFileManager.defaultManager moveItemAtURL:partial toURL:file error:nil];}++// Over moving artwork. The video is read as BGRA, the dimming and the words are drawn into each frame where+// it lies, at the video's own size, and the encoder cuts the 3:4 middle out as SGMotionClip.m's cut does,+// so the picture is never scaled on the CPU: a frame costs a fill, a gradient and one image draw.++typedef struct {+ CGSize stored, cut; // the video's size, and the 3:4 middle of it the clip keeps+ double length; // the video's, in seconds+} Motion;++// The video's track, nil when it has none or a transform turns it.+static AVAssetTrack *motionTrack(AVURLAsset *asset, Motion *motion) {+ __block AVAssetTrack *track = nil;+ dispatch_semaphore_t loaded = dispatch_semaphore_create(0);+ [asset loadTracksWithMediaType:AVMediaTypeVideo completionHandler:^(NSArray<AVAssetTrack *> *tracks, NSError *error) {+ track = tracks.firstObject;+ dispatch_semaphore_signal(loaded);+ }];+ dispatch_semaphore_wait(loaded, DISPATCH_TIME_FOREVER);+ // shortcut: a turned video (none seen among Canvases or Apple Music's covers) falls back to the blurred+ // cover; drawing the words turned with it is the step up if one turns up.+ if (!track || !CGAffineTransformIsIdentity(track.preferredTransform)) return nil;+ CGSize stored = track.naturalSize;+ double length = CMTimeGetSeconds(track.timeRange.duration);+ if (stored.width < 16 || stored.height < 16 || !(length > 0)) return nil;+ CGSize cut = stored.width / stored.height > 0.75 ? CGSizeMake(stored.height * 0.75, stored.height)+ : CGSizeMake(stored.width, stored.width / 0.75);+ // H.264 takes even sides, rounded down so the cut stays inside the picture.+ *motion = (Motion){stored, CGSizeMake(floor(cut.width / 2) * 2, floor(cut.height / 2) * 2), length};+ return track;+}++// The cut's place in the stored frame, which Core Graphics counts from the bottom; it is the middle, so the+// same from the top.+static CGRect cutIn(Motion motion) {+ return CGRectMake(round((motion.stored.width - motion.cut.width) / 2), round((motion.stored.height - motion.cut.height) / 2),+ motion.cut.width, motion.cut.height);+}++// What is drawn over every frame of a clip, made once for it.+typedef struct {+ CGImageRef words; // the words' box, NULL for none+ CGRect drawn; // where it goes in the cut+ uint16_t *keep; // how much of the picture each row of the cut keeps, of 256, from the top: the dimming+ CGRect soft; // the rows behind the words, across the cut, in the stored frame+ uint8_t *blurred; // those rows blurred, NULL with no words+ uint16_t *feather; // how much of the blurred rows each row takes, of 256, fading in and out at the top and bottom+} Over;++// How dark row `y` of `height` (from the top) is made: kBandDim across the middle 40% of the band about the+// words, kMotionDim past it, straight between.+static CGFloat dimAt(size_t y, size_t height) {+ CGFloat reach = height * kBandReach, t = (y - (height * kCentreY - reach)) / (2 * reach);+ if (t <= 0 || t >= 1) return kMotionDim;+ CGFloat edge = MIN(MIN(t, 1 - t) / 0.3, 1);+ return kMotionDim + (kBandDim - kMotionDim) * edge;+}++static Over makeOver(Motion motion, NSString *previous, NSString *line, NSString *next) {+ Over over = {0};+ over.words = newWords(previous, line, next, motion.cut, &over.drawn);+ size_t rows = (size_t)motion.cut.height;+ over.keep = malloc(rows * sizeof(uint16_t));+ for (size_t y = 0; y < rows; y++) over.keep[y] = (uint16_t)round(256 * (1 - dimAt(y, rows)));+ if (!over.words) return over;+ CGRect cut = cutIn(motion);+ over.soft = CGRectMake(cut.origin.x, cut.origin.y + over.drawn.origin.y, cut.size.width, over.drawn.size.height);+ // Faded over the margin newWords leaves about the words, a font size, so every line is on blurred picture.+ size_t width = (size_t)over.soft.size.width, height = (size_t)over.soft.size.height;+ size_t fade = MAX(MIN((size_t)(motion.cut.width * kFont), height / 2), 1);+ over.blurred = malloc(width * height * 4);+ over.feather = malloc(height * sizeof(uint16_t));+ for (size_t y = 0; y < height; y++) over.feather[y] = (uint16_t)(256 * MIN(1.0, MIN(y, height - 1 - y) / (double)fade));+ return over;+}++static void freeOver(Over over) {+ CGImageRelease(over.words);+ free(over.keep);+ free(over.blurred);+ free(over.feather);+}++// The softening, the dimming and the words, drawn into `buffer`, a BGRA frame of the video: the rows behind+// the words blurred (Accelerate's tent blur) and faded in, every row of the cut darkened by its share in one+// pass, and one image the size of the words. All on the CPU.+static void drawOver(CVPixelBufferRef buffer, Motion motion, Over over) {+ CVPixelBufferLockBaseAddress(buffer, 0);+ uint8_t *base = CVPixelBufferGetBaseAddress(buffer);+ size_t bytesPerRow = CVPixelBufferGetBytesPerRow(buffer);+ CGRect cut = cutIn(motion);+ // Every write below is placed by the track's natural size, which the file states: a decoded frame of another+ // size or format is left as it is rather than written past its end.+ size_t frameWidth = CVPixelBufferGetWidth(buffer), frameHeight = CVPixelBufferGetHeight(buffer);+ CGRect frame = CGRectMake(0, 0, frameWidth, frameHeight);+ if (!base || CVPixelBufferIsPlanar(buffer) || CVPixelBufferGetPixelFormatType(buffer) != kCVPixelFormatType_32BGRA+ || frameWidth != (size_t)motion.stored.width || frameHeight != (size_t)motion.stored.height || bytesPerRow < frameWidth * 4+ || !CGRectContainsRect(frame, cut) || (over.blurred && !CGRectContainsRect(frame, over.soft))) {+ CVPixelBufferUnlockBaseAddress(buffer, 0);+ return;+ }+ if (over.blurred) {+ // An image counts its rows from the top, Core Graphics from the bottom.+ size_t top = (size_t)(motion.stored.height - CGRectGetMaxY(over.soft)), height = (size_t)over.soft.size.height;+ size_t width = (size_t)over.soft.size.width, rowBytes = width * 4;+ uint8_t *rows = base + top * bytesPerRow + (size_t)over.soft.origin.x * 4;+ vImage_Buffer from = {rows, height, width, bytesPerRow}, to = {over.blurred, height, width, rowBytes};+ uint32_t kernel = (uint32_t)(width * kSoftWidth) | 1;+ if (vImageTentConvolve_ARGB8888(&from, &to, NULL, 0, 0, kernel, kernel, NULL, kvImageEdgeExtend) == kvImageNoError) {+ for (size_t y = 0; y < height; y++) {+ uint8_t *sharp = rows + y * bytesPerRow;+ const uint8_t *soft = over.blurred + y * rowBytes;+ uint16_t share = over.feather[y];+ if (share == 256) memcpy(sharp, soft, rowBytes);+ else if (share) for (size_t i = 0; i < rowBytes; i++) sharp[i] += (uint8_t)(((soft[i] - sharp[i]) * share) >> 8);+ }+ }+ }+ // Blue and red, then green, each scaled in its own lane of the pixel; alpha is left as it is.+ size_t top = (size_t)cut.origin.y, width = (size_t)cut.size.width;+ for (size_t y = 0; y < (size_t)cut.size.height; y++) {+ uint32_t *pixel = (uint32_t *)(base + (top + y) * bytesPerRow) + (size_t)cut.origin.x, keep = over.keep[y];+ for (size_t x = 0; x < width; x++) {+ uint32_t v = pixel[x];+ pixel[x] = (((v & 0x00FF00FFu) * keep >> 8) & 0x00FF00FFu) | (((v & 0x0000FF00u) * keep >> 8) & 0x0000FF00u) | (v & 0xFF000000u);+ }+ }+ if (over.words) {+ CGContextRef context = newContext(motion.stored, base, bytesPerRow);+ CGContextDrawImage(context, CGRectOffset(over.drawn, cut.origin.x, cut.origin.y), over.words);+ CGContextRelease(context);+ }+ CVPixelBufferUnlockBaseAddress(buffer, 0);+}++// Each frame of the video from `from` seconds (taken round its length) for `seconds`, round its end as often+// as needed, timed from 0, to `each`, which returns NO to stop. NO when a read failed.+static BOOL eachFrame(AVURLAsset *asset, AVAssetTrack *track, Motion motion, double from, double seconds,+ BOOL (^each)(CMSampleBufferRef sample)) {+ int32_t scale = 600; // a tick that 24, 25, 30 and 60 fps all land on+ double at = fmod(MAX(from, 0), motion.length), done = 0;+ // When the next frame is due: kMotionFPS on a schedule, so a 60 fps video keeps every other frame.+ double due = 0;+ while (seconds - done > 0.01) {+ double take = MIN(motion.length - at, seconds - done);+ AVAssetReader *reader = [AVAssetReader assetReaderWithAsset:asset error:nil];+ AVAssetReaderTrackOutput *output = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track outputSettings:@{+ (id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32BGRA),+ }];+ // Copies, so drawing into one never touches a frame the decoder still refers to.+ output.alwaysCopiesSampleData = YES;+ if (!reader || ![reader canAddOutput:output]) return NO;+ [reader addOutput:output];+ reader.timeRange = CMTimeRangeMake(CMTimeMakeWithSeconds(at, scale), CMTimeMakeWithSeconds(take, scale));+ if (![reader startReading]) return NO;+ CMSampleBufferRef sample;+ while ((sample = [output copyNextSampleBuffer])) {+ // The first frame may start before `at`; it shows from the start of this stretch.+ CMTime time = CMTimeMakeWithSeconds(done + MAX(CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(sample)) - at, 0), scale);+ CMSampleBufferRef timed = NULL;+ CMSampleTimingInfo timing = {kCMTimeInvalid, time, kCMTimeInvalid};+ BOOL keep = CMTimeGetSeconds(time) >= due - 0.002 && CMTimeGetSeconds(time) < done + take+ && CMSampleBufferGetImageBuffer(sample)+ && CMSampleBufferCreateCopyWithNewTiming(NULL, sample, 1, &timing, &timed) == noErr;+ CFRelease(sample);+ if (!keep) continue;+ // A jump in the video (its loop) starts the schedule again rather than keeping a burst of frames.+ due = due + 1.0 / kMotionFPS < CMTimeGetSeconds(time) ? CMTimeGetSeconds(time) + 0.5 / kMotionFPS : due + 1.0 / kMotionFPS;+ BOOL more = each(timed);+ CFRelease(timed);+ if (!more) {+ [reader cancelReading];+ return YES;+ }+ }+ if (reader.status != AVAssetReaderStatusCompleted) return NO;+ done += take;+ at = 0;+ }+ return YES;+}++CGImageRef SGLyricsClipMotionFrame(NSURL *motionFile, NSString *previous, NSString *line, NSString *next, double from) {+ AVURLAsset *asset = [AVURLAsset URLAssetWithURL:motionFile options:nil];+ Motion motion;+ AVAssetTrack *track = motionTrack(asset, &motion);+ if (!track) return NULL;+ Over over = makeOver(motion, previous, line, next);+ __block CGImageRef frame = NULL;+ eachFrame(asset, track, motion, from, motion.length, ^BOOL(CMSampleBufferRef sample) {+ CVPixelBufferRef buffer = CMSampleBufferGetImageBuffer(sample);+ drawOver(buffer, motion, over);+ CVPixelBufferLockBaseAddress(buffer, kCVPixelBufferLock_ReadOnly);+ CGContextRef context = newContext(motion.stored, CVPixelBufferGetBaseAddress(buffer), CVPixelBufferGetBytesPerRow(buffer));+ CGImageRef whole = CGBitmapContextCreateImage(context);+ CGContextRelease(context);+ CVPixelBufferUnlockBaseAddress(buffer, kCVPixelBufferLock_ReadOnly);+ frame = CGImageCreateWithImageInRect(whole, cutIn(motion));+ CGImageRelease(whole);+ return NO;+ });+ freeOver(over);+ return frame;+}++BOOL SGLyricsClipWriteOverMotion(NSURL *file, NSURL *motionFile, NSString *previous, NSString *line, NSString *next,+ double from, double seconds) {+ AVURLAsset *asset = [AVURLAsset URLAssetWithURL:motionFile options:nil];+ Motion motion;+ AVAssetTrack *track = motionTrack(asset, &motion);+ if (!track) return NO;+ // No longer than the video, which is then written whole and loops as it was made to.+ seconds = MIN(seconds, motion.length);+ NSURL *partial = [file URLByAppendingPathExtension:@"partial"];+ [NSFileManager.defaultManager removeItemAtURL:partial error:nil];+ AVAssetWriter *writer = [AVAssetWriter assetWriterWithURL:partial fileType:AVFileTypeMPEG4 error:nil];+ AVAssetWriterInput *input = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:@{+ AVVideoCodecKey: AVVideoCodecTypeH264,+ AVVideoWidthKey: @(motion.cut.width),+ AVVideoHeightKey: @(motion.cut.height),+ AVVideoScalingModeKey: AVVideoScalingModeResizeAspectFill,+ // As SGMotionClip.m's cut: about 2 Mbit/s at a Canvas's 720 x 960.+ AVVideoCompressionPropertiesKey: @{AVVideoAverageBitRateKey: @(motion.cut.width * motion.cut.height * 3)},+ }];+ if (!writer || ![writer canAddInput:input]) return NO;+ [writer addInput:input];+ if (![writer startWriting]) return NO;+ [writer startSessionAtSourceTime:kCMTimeZero];+ Over over = makeOver(motion, previous, line, next);+ __block BOOL ok = YES, any = NO;+ BOOL read = eachFrame(asset, track, motion, from, seconds, ^BOOL(CMSampleBufferRef sample) {+ drawOver(CMSampleBufferGetImageBuffer(sample), motion, over);+ while (!input.readyForMoreMediaData && writer.status == AVAssetWriterStatusWriting) [NSThread sleepForTimeInterval:0.002];+ ok = writer.status == AVAssetWriterStatusWriting && [input appendSampleBuffer:sample];+ any = YES;+ return ok;+ });+ freeOver(over);+ if (!read || !ok || !any) {+ [writer cancelWriting];+ return NO;+ }+ [input markAsFinished];+ [writer endSessionAtSourceTime:CMTimeMakeWithSeconds(seconds, 600)];+ dispatch_semaphore_t written = dispatch_semaphore_create(0);+ [writer finishWritingWithCompletionHandler:^{ dispatch_semaphore_signal(written); }];+ dispatch_semaphore_wait(written, DISPATCH_TIME_FOREVER);+ if (writer.status != AVAssetWriterStatusCompleted) return NO;+ [NSFileManager.defaultManager removeItemAtURL:file error:nil];+ return [NSFileManager.defaultManager moveItemAtURL:partial toURL:file error:nil];+}diff --git a/tweak/Sources/Shared/Lyrics/KaraokeSource.x b/tweak/Sources/Shared/Lyrics/KaraokeSource.xindex 5ce9fdf3..80503b90 100644--- a/tweak/Sources/Shared/Lyrics/KaraokeSource.x+++ b/tweak/Sources/Shared/Lyrics/KaraokeSource.x@@ -559,7 +559,7 @@ static SGKaraokeTrackWatcher *sg_trackWatcher;// The sources that search by name learn the name from the player, so the player is caught// whenever one is on, not only for the redesign's lyrics and the lock screen.if (!SGRedesignedUI() && !SGFlag(SGKeyLockScreenLyrics, NO) && !SGLyricsEnabled()- && SGLockScreenArtwork() != SGLockArtworkLyrics) {+ && !SGLockArtworkShowsLyrics(SGLockScreenArtwork())) {%init(SGSpclientHeadersOnly);return;}