Skip to content

Liquid Glass and Apple Music touches for the Spotify iOS app. A GPL-3.0 fork of spoti.pw 0.21.1.

C

feat: Karaoke can use a light voice model for older iPhones

My-Name-Is-Jeffa663fcf83896
diff --git a/docs/tweaks.md b/docs/tweaks.md
index e3ef346d..abb6ceec 100644
--- a/docs/tweaks.md
+++ b/docs/tweaks.md
@@ -262,9 +262,16 @@ Shared:
Tested on the Mac against harness/audio-effects/ and harness/autoeq/, the hook in the simulator
against its sim/, the page against harness/audio-effects-page/
Sing/ Karaoke (Sing in the code and its keys), a song's vocals turned down while it plays (Sing.h lists
- its files): one voice model (Mel-Band RoFormer, MIT, exported by Vitrine for the Neural Engine and
- palettized to 6 bits: separator-ane.mlmodelc, 210 MB, from Hugging Face; on a FLEX build a copy in
- Sing/dev/ comes first) downloaded and checked file by file over Wi-Fi unless cellular is allowed,
+ its files): the voice model Voice model chooses (spotifyglass.sing.model, read live: Best quality,
+ the default, is Mel-Band RoFormer, MIT, exported by Vitrine for the Neural Engine and palettized to
+ 6 bits: separator-ane.mlmodelc, 210 MB; Light is the vocals of HiDolen's Mini-BS-RoFormer-18M, MIT,
+ exported in float16 by harness/sing/export_light.py with the same input and output:
+ separator-light.mlmodelc, 31 MB, more of the voice left in and the music dipping under it; both
+ from the same Hugging Face repo, each with its own pins, staging folder, resume data and paused
+ bytes; on a FLEX build a copy of the chosen one in Sing/dev/<its folder> comes first; a switch
+ stops a running download, keeping what came in, and Sing.x loads the chosen model at once, no
+ restart, or lets the old one go if the chosen one is not downloaded) downloaded and checked file by
+ file over Wi-Fi unless cellular is allowed,
a stop keeping what came in for the next download to carry on from; the 489 MB model downloaded
before is kept and loaded on the CPU alone until this one is in (the Voice model row then reads
Update available), and deleted as its download ends or at a launch that finds both
@@ -745,8 +752,10 @@ and the rest traced live from the engine's loudness, Karaoke running on the play
from gone through as sung to the vocals alone, with Sing along, Original and Vocals only under it), Spatial voice
(where the iPhone reads headphone motion; a page of its own, reading out On or Off, with a live preview at its top that
follows the head through AirPods, or sways gently without them and holds still under Reduce Motion, a line under it
-saying which, then the switch; its row is on the main page too, under Karaoke's), the voice model's download (Paused and
-Checking among its states) and its removal, Toggle from lyrics (the redesign's mic, on until switched off, issue #12),
+saying which, then the switch; its row is on the main page too, under Karaoke's), a Voice model section (Best quality
+and Light, a checkmark against the chosen one, each with its size; then the chosen model's download, Paused and Checking
+among its states, and its removal; and Remove the other model, with its size, while the one not chosen, or the old
+model with the full one, is on the iPhone), Toggle from lyrics (the redesign's mic, on until switched off, issue #12),
Ignore heat warnings, and under Advanced Runs on (Automatic: the Neural Engine
beside the CPU; CPU only) and Prepare after updates, all applying straight away. Lock screen, on the main
page under either look, opens the lock screen widget's page, titled Lock screen: Widget (the line in place of the artist,
diff --git a/harness/sing/README.md b/harness/sing/README.md
index 11b8f42b..c3bf723e 100644
--- a/harness/sing/README.md
+++ b/harness/sing/README.md
@@ -250,11 +250,25 @@ cleanup), and the next download carries the weights on from there through the ch
else left in `Sing/`. `CFFIXED_USER_HOME` keeps the 210 MB out of the Mac's own Application Support. `update` starts
from the old model (`separator.mlmodelc`, stood in by sparse files of its sizes) instead: kept at launch and the model
(Ready, the download its update) while the update downloads, deleted as soon as it is in, and deleted by a launch that
-finds both. `dev` checks the FLEX build's dev folder.
+finds both. `dev` checks the FLEX build's dev folder. `light` chooses the light model with the full one and the old
+one stood in: it reads missing (the full one does not stand in for it), downloads from `separator-light.mlmodelc`, is
+left alone by the launch's cleanup, switches to the full model and back with no download, and the other model's
+removal and its own each leave the other alone. Every mode starts on the full model: the defaults outlive the
+`CFFIXED_USER_HOME` folder.
download/build.sh && CFFIXED_USER_HOME="$(mktemp -d)" build/download 0.2
download/build.sh && CFFIXED_USER_HOME="$(mktemp -d)" build/download update
download/build.sh && CFFIXED_USER_HOME="$(mktemp -d)" build/download dev
+ download/build.sh && CFFIXED_USER_HOME="$(mktemp -d)" build/download light
+
+2026-10-12, with the choice of model: every mode passes. `light`: Light's size reads 30.8 MB, the full model's 210.4
+MB; the five files come in from Hugging Face and are checked against their pins.
+
+The light model is built by `export_light.py` (`uv run harness/sing/export_light.py <out dir> [audio]`): HiDolen's
+Mini-BS-RoFormer-18M at a pinned revision, its vocals' mask alone, every RMSNorm made safe for float16, exported for
+iOS 18 and compiled. 2026-10-11: the rewrite matches the published model at 135 dB in float32, and the float16 Core ML
+model on the CPU matches it at 52.7 dB. `build/sing <separator-light.mlmodelc> <voice> <out dir> cpu` passes every
+check on it: 330-365 ms a window on an M4's CPU, the vocals it finds at 24.3 dB against the voice.
2026-10-06, one model, the old one kept until it is in: every mode passes. The size row reads 210.4 MB
(NSByteCountFormatter's decimal megabytes; 201 MiB), the old model 489.7 MB. Default: the old download's leftovers go
diff --git a/harness/sing/download/main.m b/harness/sing/download/main.m
index 19e01748..7ebead3e 100644
--- a/harness/sing/download/main.m
+++ b/harness/sing/download/main.m
@@ -14,6 +14,11 @@
// the model then Ready without a download.
//
// ./build.sh && CFFIXED_USER_HOME=<empty dir> ../build/download dev
+//
+// `light` chooses the light model (separator-light.mlmodelc) with the full one on the iPhone, downloads it, switches
+// back and forth and removes each (checkLight).
+//
+// ./build.sh && CFFIXED_USER_HOME=<empty dir> ../build/download light
#import <Foundation/Foundation.h>
#import <objc/runtime.h>
#import "Shared/Sing/Sing.h"
@@ -96,6 +101,43 @@ static int checkUpdate(NSURL *sing) {
return failures ? 1 : 0;
}
+// Light chosen, with the full model and the old one on the iPhone (stood in by files of their sizes): the light model
+// downloads from its own folder and is the model; the launch's cleanup leaves it alone; switching back and forth needs
+// no download; removing the other model leaves the chosen one, and removing the chosen one leaves Sing/ empty.
+static int checkLight(NSURL *sing) {
+ NSURL *full = [sing URLByAppendingPathComponent:@"separator-ane.mlmodelc"], *light = [sing URLByAppendingPathComponent:@"separator-light.mlmodelc"];
+ NSURL *old = [sing URLByAppendingPathComponent:@"separator.mlmodelc"];
+ standIn(full, @{@"metadata.json": @2333, @"coremldata.bin": @388, @"analytics/coremldata.bin": @243, @"model.mil": @1346642,
+ @"weights/weight.bin": @209077208});
+ standIn(old, @{@"model.mil": @669061});
+ CHECK(SGSingChosenModel() == SGSingModelFull && [SGSingModelURL().path isEqualToString:full.path], @"the full model is the default");
+ SGSetSingChosenModel(SGSingModelLight);
+ CHECK(SGSingModelCurrentState() == SGSingModelMissing && !SGSingModelURL(), @"chosen, Light reads missing: the full model does not stand in for it");
+ CHECK(SGSingOtherModelBytes() > 209077208, @"the other model's bytes count the full model and the old one (%lld)", SGSingOtherModelBytes());
+ printf("Light's size reads %s, the full model's %s\n", SGSingModelSizeText().UTF8String, SGSingModelSizeTextOf(SGSingModelFull).UTF8String);
+
+ SGSingDownloadModelOverCellular();
+ runUntil(^BOOL { return SGSingModelCurrentState() != SGSingModelDownloading; }, 900);
+ CHECK(SGSingModelCurrentState() == SGSingModelReady && [SGSingModelURL().path isEqualToString:light.path],
+ @"Light downloads from separator-light.mlmodelc, checked, and is the model (%@)", SGSingModelError() ?: SGSingModelURL().lastPathComponent);
+ CHECK(exists(full), @"the full model is kept");
+ SGSingRemoveOldModel();
+ CHECK(exists(light) && SGSingModelCurrentState() == SGSingModelReady, @"the launch's cleanup leaves the light model alone");
+
+ SGSetSingChosenModel(SGSingModelFull);
+ CHECK([SGSingModelURL().path isEqualToString:full.path] && SGSingModelPausedBytes() == 0, @"switched back, the full model is the model at once");
+ SGSetSingChosenModel(SGSingModelLight);
+ CHECK([SGSingModelURL().path isEqualToString:light.path], @"and Light again");
+
+ SGSingDeleteOtherModel();
+ CHECK(!exists(full) && !exists(old) && exists(light) && SGSingOtherModelBytes() == 0, @"removing the other model removes the full one and the old one, not Light");
+ SGSingDeleteModel();
+ NSArray *left = [NSFileManager.defaultManager contentsOfDirectoryAtPath:sing.path error:nil];
+ CHECK(!left.count && SGSingModelCurrentState() == SGSingModelMissing, @"removing Light leaves Sing/ empty (%@)", [left componentsJoinedByString:@", "]);
+ printf("%s\n", failures ? "FAILED" : "all passed");
+ return failures ? 1 : 0;
+}
+
int main(int argc, char **argv) {
@autoreleasepool {
setvbuf(stdout, NULL, _IOLBF, 0);
@@ -103,7 +145,10 @@ int main(int argc, char **argv) {
NSURL *sing = [[[NSFileManager.defaultManager URLsForDirectory:NSApplicationSupportDirectory inDomains:NSUserDomainMask].firstObject
URLByAppendingPathComponent:@"Vitrine"] URLByAppendingPathComponent:@"Sing"];
printf("the model goes to %s\n", sing.path.UTF8String);
+ // The defaults outlive CFFIXED_USER_HOME's folder (cfprefsd keeps them): every check starts on the full model.
+ [NSUserDefaults.standardUserDefaults removeObjectForKey:SGKeySingModel];
if (argc > 1 && !strcmp(argv[1], "dev")) return checkDev(sing);
+ if (argc > 1 && !strcmp(argv[1], "light")) return checkLight(sing);
if (argc > 1 && !strcmp(argv[1], "update")) return checkUpdate(sing);
diff --git a/harness/sing/export_light.py b/harness/sing/export_light.py
new file mode 100644
index 00000000..cd03b51d
--- /dev/null
+++ b/harness/sing/export_light.py
@@ -0,0 +1,213 @@
+# /// script
+# requires-python = ">=3.11,<3.13"
+# dependencies = [
+# "torch==2.7.1",
+# "transformers==4.55.4",
+# "coremltools==9.0",
+# "einops==0.8.1",
+# "safetensors",
+# "huggingface_hub",
+# "numpy<2.3",
+# ]
+# ///
+"""Karaoke's light voice model: Mini-BS-RoFormer-18M's vocal stem exported for Core ML as a drop-in for
+separator-ane.mlmodelc (SGSingSeparator.m's contract: `spectrum` [1, 2050, 201, 2] in, the layout
+(2 * bin + channel, frame, re/im), and `vocals_spectrum` of the same shape out, the STFT around it done by the app).
+
+ uv run harness/sing/export_light.py <out dir> [<audio for the check, any file afconvert reads>]
+
+writes <out dir>/separator-light.mlpackage, compiles it to <out dir>/separator-light.mlmodelc (xcrun coremlcompiler)
+and checks the Core ML model on the CPU against the PyTorch model in float32 on that audio's spectrum (or a
+synthetic mix). Changes from the published model, none of which change its maths:
+- only the vocals' mask estimator (stem 3) is kept, the others' 3.2 M parameters dropped;
+- every RMSNorm divides its row by the row's largest value first, so float16 does not overflow on the raw
+ spectrum (|X| reaches 1024): RMSNorm ignores scale, and eps is scaled with it;
+- the register tokens are concatenated instead of written into a slice, and attention is written out (its K/V
+ heads repeated for the 8 query heads), so the graph traces;
+- the model is fed the app's STFT (hop 441, 201 frames for 2 s) rather than its own (hop 512), as the model tested
+ on the phone on 2026-10-08 was.
+"""
+import json
+import math
+import os
+import subprocess
+import sys
+import tempfile
+import types
+import wave
+
+import coremltools as ct
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from huggingface_hub import snapshot_download
+from transformers import AutoModel
+
+REPO = "HiDolen/Mini-BS-RoFormer-18M"
+REVISION = "d3ff400db0c72c236775ecc82e038f3117e52bf8"
+VOCALS = 3
+FRAMES, BINS, FFT, HOP, SECONDS = 201, 1025, 2048, 441, 88200
+
+
+class SafeRMSNorm(nn.Module):
+ def __init__(self, norm: nn.RMSNorm):
+ super().__init__()
+ self.weight = norm.weight
+ self.root_eps = math.sqrt(norm.eps)
+
+ def forward(self, x):
+ m = x.abs().amax(dim=-1, keepdim=True).clamp(min=self.root_eps)
+ y = x / m
+ # eps / m^2 written as (sqrt(eps) / m)^2: at most 1, so it cannot overflow either.
+ return y * torch.rsqrt(y.pow(2).mean(dim=-1, keepdim=True) + (self.root_eps / m).pow(2)) * self.weight
+
+
+def safe_norms(module: nn.Module):
+ for name, child in module.named_children():
+ if isinstance(child, nn.RMSNorm):
+ setattr(module, name, SafeRMSNorm(child))
+ else:
+ safe_norms(child)
+
+
+def attention_forward(self, hidden_states, position_embeddings, attention_mask=None):
+ b, s, _ = hidden_states.shape
+ heads, kv, d = self.config.num_attention_heads, self.config.num_key_value_heads, self.head_dim
+ q = self.q_proj(hidden_states).view(b, s, heads, d).transpose(1, 2)
+ k = self.k_proj(hidden_states).view(b, s, kv, d).transpose(1, 2)
+ v = self.v_proj(hidden_states).view(b, s, kv, d).transpose(1, 2)
+ cos, sin = position_embeddings
+ cos, sin = cos.unsqueeze(1), sin.unsqueeze(1)
+
+ def rotate(x):
+ return torch.cat((-x[..., d // 2:], x[..., : d // 2]), dim=-1)
+
+ q, k = q * cos + rotate(q) * sin, k * cos + rotate(k) * sin
+ k = k.repeat_interleave(heads // kv, dim=1)
+ v = v.repeat_interleave(heads // kv, dim=1)
+ weights = torch.softmax(torch.matmul(q, k.transpose(-1, -2)) * self.scaling, dim=-1)
+ out = torch.matmul(weights, v).transpose(1, 2).reshape(b, s, heads * d)
+ return self.o_proj(out), None
+
+
+class Core(nn.Module):
+ """BSRoformerModel.forward with the register tokens concatenated and the rotary tables precomputed."""
+
+ def __init__(self, model):
+ super().__init__()
+ self.m = model
+ rn = model.config.register_token_num
+ n = len(model.config.freqs_per_bands)
+ probe = torch.zeros(1, 1, 1, model.config.hidden_size)
+
+ def table(length):
+ cos, sin = model.rotary_emb(probe, torch.arange(length).unsqueeze(0))
+ return F.pad(cos, (0, 0, 0, rn), value=1.0), F.pad(sin, (0, 0, 0, rn), value=0.0)
+
+ self.register_buffer("tcos", table(FRAMES)[0]), self.register_buffer("tsin", table(FRAMES)[1])
+ self.register_buffer("fcos", table(n)[0]), self.register_buffer("fsin", table(n)[1])
+
+ def forward(self, x):
+ m = self.m
+ h = m.band_split(x)
+ b, t, n, d = h.shape
+ rn = m.config.register_token_num
+ top = torch.cat([h, torch.zeros(b, t, rn, d, dtype=h.dtype)], dim=2)
+ bottom = torch.cat([torch.zeros(b, rn, n, d, dtype=h.dtype), m.register_tokens.unsqueeze(0).expand(b, -1, -1, -1)], dim=2)
+ h = torch.cat([top, bottom], dim=1)
+ for time_transformer, freq_transformer in m.layers:
+ h = time_transformer(h, position_embeddings=(self.tcos, self.tsin), attention_mask=None)
+ h = freq_transformer(h, position_embeddings=(self.fcos, self.fsin), attention_mask=None)
+ return m.final_norm(h[:, :t, :n, :])
+
+
+class Separator(nn.Module):
+ def __init__(self, full):
+ super().__init__()
+ self.core = Core(full.model)
+ self.mask = full.mask_estimators[VOCALS]
+
+ def forward(self, spectrum):
+ # (2 * bin + channel, frame, re/im) -> (frame, (bin channel re/im)), the model's "b t (f c T)".
+ x = spectrum.permute(0, 2, 1, 3).reshape(1, FRAMES, 4 * BINS)
+ mask = self.mask(self.core(x)).reshape(1, FRAMES, 2 * BINS, 2).permute(0, 2, 1, 3)
+ sr, si, mr, mi = spectrum[..., 0], spectrum[..., 1], mask[..., 0], mask[..., 1]
+ return torch.stack([sr * mr - si * mi, sr * mi + si * mr], dim=-1)
+
+
+def reference(full, spectrum):
+ """The published model's own path: its forward from the merged spectrum on, vocals only, float32."""
+ x = spectrum.permute(0, 2, 1, 3).reshape(1, FRAMES, 4 * BINS)
+ mask = full.mask_estimators[VOCALS](full.model(x)).reshape(1, FRAMES, 2 * BINS, 2).permute(0, 2, 1, 3)
+ return torch.view_as_real(torch.view_as_complex(spectrum.contiguous()) * torch.view_as_complex(mask.contiguous()))
+
+
+def spectrum_of(audio):
+ """The app's STFT: torch's periodic Hann, unnormalised, centered, (2 * bin + channel, frame, re/im)."""
+ s = torch.stft(audio, FFT, HOP, FFT, window=torch.hann_window(FFT), center=True, return_complex=True) # [2, 1025, 201]
+ s = torch.view_as_real(s).permute(1, 0, 2, 3).reshape(1, 2 * BINS, FRAMES, 2)
+ return s.contiguous()
+
+
+def test_audio(path):
+ if path:
+ with tempfile.TemporaryDirectory() as tmp:
+ raw = os.path.join(tmp, "a.wav")
+ subprocess.run(["afconvert", "-f", "WAVE", "-d", "LEI16@44100", "-c", "2", path, raw], check=True)
+ with wave.open(raw) as file:
+ data = np.frombuffer(file.readframes(file.getnframes()), dtype="<i2").astype(np.float32) / 32768
+ frames = data.reshape(-1, 2)
+ start = max(0, (len(frames) - SECONDS) // 2) # two seconds from the middle
+ frames = np.pad(frames[start:start + SECONDS], ((0, max(0, SECONDS - len(frames) + start)), (0, 0)))
+ return torch.from_numpy(frames.T.copy())
+ t = torch.arange(SECONDS) / 44100
+ rng = torch.Generator().manual_seed(1)
+ chords = sum(0.15 * torch.sin(2 * math.pi * f * t) for f in (110, 220, 277.2, 329.6))
+ voice = 0.3 * torch.sin(2 * math.pi * 440 * t + 3 * torch.sin(2 * math.pi * 5 * t)) * (torch.sin(2 * math.pi * 0.7 * t) > 0)
+ noise = 0.02 * torch.randn(2, SECONDS, generator=rng)
+ return torch.stack([chords + voice, chords + 0.8 * voice]) + noise
+
+
+def main():
+ out = os.path.abspath(sys.argv[1])
+ os.makedirs(out, exist_ok=True)
+ folder = snapshot_download(REPO, revision=REVISION, allow_patterns=["*.json", "*.py", "model.safetensors"])
+ full = AutoModel.from_pretrained(folder, trust_remote_code=True, torch_dtype=torch.float32).eval()
+ spectrum = spectrum_of(test_audio(sys.argv[2] if len(sys.argv) > 2 else None))
+ with torch.no_grad():
+ expected = reference(full, spectrum)
+ for module in full.modules():
+ if type(module).__name__ == "BSRoformerAttention":
+ module.forward = types.MethodType(attention_forward, module)
+ safe_norms(full)
+ separator = Separator(full).eval()
+ rewritten = separator(spectrum)
+ print(f"rewrite vs published (float32): {snr(expected, rewritten):.1f} dB")
+
+ traced = torch.jit.trace(separator, spectrum)
+ package = ct.convert(
+ traced,
+ inputs=[ct.TensorType(name="spectrum", shape=(1, 2 * BINS, FRAMES, 2), dtype=np.float32)],
+ outputs=[ct.TensorType(name="vocals_spectrum", dtype=np.float16)],
+ convert_to="mlprogram",
+ compute_precision=ct.precision.FLOAT16,
+ minimum_deployment_target=ct.target.iOS18,
+ )
+ package.short_description = "Karaoke's light voice model: Mini-BS-RoFormer-18M's vocals (HiDolen, MIT), exported by Vitrine"
+ path = os.path.join(out, "separator-light.mlpackage")
+ package.save(path)
+ subprocess.run(["xcrun", "coremlcompiler", "compile", path, out], check=True)
+
+ cpu = ct.models.MLModel(path, compute_units=ct.ComputeUnit.CPU_ONLY)
+ got = torch.from_numpy(cpu.predict({"spectrum": spectrum.numpy()})["vocals_spectrum"].astype(np.float32))
+ print(f"Core ML float16 on the CPU vs published (float32): {snr(expected, got):.1f} dB")
+ print(json.dumps({"vocals_energy_ratio": float(expected.pow(2).sum() / spectrum.pow(2).sum())}))
+
+
+def snr(reference, test):
+ return 10 * math.log10(float(reference.pow(2).sum()) / max(float((reference - test).pow(2).sum()), 1e-30))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/harness/spatial-page/stubs.m b/harness/spatial-page/stubs.m
index db057013..866c2778 100644
--- a/harness/spatial-page/stubs.m
+++ b/harness/spatial-page/stubs.m
@@ -80,6 +80,11 @@ NSString *SGSingModelInUseSizeText(void) { return SGSingModelUpdateAvailable() ?
void SGSingDownloadModel(void) {}
void SGSingCancelModelDownload(void) {}
void SGSingDeleteModel(void) {}
+SGSingModelKind SGSingChosenModel(void) { return [NSUserDefaults.standardUserDefaults integerForKey:SGKeySingModel] == 1 ? SGSingModelLight : SGSingModelFull; }
+void SGSetSingChosenModel(SGSingModelKind kind) { [NSUserDefaults.standardUserDefaults setInteger:kind forKey:SGKeySingModel]; }
+NSString *SGSingModelSizeTextOf(SGSingModelKind kind) { return kind == SGSingModelLight ? @"30.8 MB" : @"210.4 MB"; }
+long long SGSingOtherModelBytes(void) { return 0; }
+void SGSingDeleteOtherModel(void) {}
NSArray<NSString *> *SGSingComputeUnitNames(void) { return @[@"Automatic", @"CPU only"]; }
BOOL SGSingOSSupported(void) { return YES; }
BOOL SGSingDeviceSupported(void) { return YES; }
diff --git a/tweak/Sources/App/About/Licenses.m b/tweak/Sources/App/About/Licenses.m
index d6602340..20a92b76 100644
--- a/tweak/Sources/App/About/Licenses.m
+++ b/tweak/Sources/App/About/Licenses.m
@@ -39,6 +39,8 @@ UIViewController *SGLicensesPage(void) {
SGModRow *autoEq = SGLinkRow(@"AutoEq", @"Headphone corrections · MIT License", @"https://github.com/jaakkopasanen/AutoEq");
// Sing's voice model is downloaded rather than built in, and its NOTICE travels with it here all the same.
SGModRow *voice = SGLinkRow(@"Karaoke's voice model", @"Mel-Band RoFormer · MIT License", @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing");
+ SGModRow *lightVoice = SGLinkRow(@"Karaoke's light voice model", @"Mini-BS-RoFormer-18M · MIT License",
+ @"https://huggingface.co/HiDolen/Mini-BS-RoFormer-18M");
return [[SGModPage alloc] initWithTitle:@"Licenses" intro:@"The mod's own license, and the code from others it includes." sections:@[
SGSection(nil, @[mod]),
SGNotedSection(nil, @[eevee], @"By whoeevee (whoeevee/EeveeSpotify), under the GNU General Public License v3.0, as Vitrine is.\n\n"
@@ -57,5 +59,11 @@ UIViewController *SGLicensesPage(void) {
@"Neural Engine export, the model Karaoke downloads: Vitrine, from the same checkpoint, palettized to 6 bits.\n"
@"Reference architecture: lucidrains/BS-RoFormer. Training code: ZFTurbo.\n\n"
@"Copyright (c) 2023 Phil Wang\n\n" stringByAppendingString:kMIT]),
+ SGNotedSection(nil, @[lightVoice], [@"Mini-BS-RoFormer-18M by HiDolen, revision d3ff400db0c72c236775ecc82e038f3117e52bf8, whose model card "
+ @"declares the MIT License. Its code is adapted from lucidrains/BS-RoFormer (MIT, Copyright (c) 2023 "
+ @"Phil Wang, above).\n"
+ @"Core ML export of its vocals, the light model Karaoke downloads: Vitrine, in float16, from "
+ @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing.\n\n"
+ @"Copyright (c) HiDolen (the model card gives no year)\n\n" stringByAppendingString:kMIT]),
] footer:nil];
}
diff --git a/tweak/Sources/Settings/SGModPage.h b/tweak/Sources/Settings/SGModPage.h
index 4e1ca997..f1c71aa6 100644
--- a/tweak/Sources/Settings/SGModPage.h
+++ b/tweak/Sources/Settings/SGModPage.h
@@ -50,6 +50,9 @@
@property (nonatomic, copy) NSArray<NSString *> *menu;
// A small rounded square of this color before the row's value.
@property (nonatomic, copy) UIColor *(^swatch)(void);
+// One of a few rows picked from on the page itself (an action row each): a checkmark against it while this answers
+// YES, read as the row is drawn, so its action reloads the table.
+@property (nonatomic, copy) BOOL (^checked)(void);
@end
@interface SGModSection : NSObject
diff --git a/tweak/Sources/Settings/SGModPage.m b/tweak/Sources/Settings/SGModPage.m
index 5b9e5d44..6dbf0157 100644
--- a/tweak/Sources/Settings/SGModPage.m
+++ b/tweak/Sources/Settings/SGModPage.m
@@ -876,7 +876,8 @@ static void showWaiting(UITableViewCell *cell, BOOL waiting, NSString *switchTit
showWaiting(cell, [self isWaiting:row], [self switchFor:row].title);
return cell;
}
- UITableViewCell *cell = SGDequeueCell(table, @"row");
+ // A checked row's cells set their own accessory and traits, which no other row resets.
+ UITableViewCell *cell = SGDequeueCell(table, row.checked ? @"checked" : @"row");
SGFillCell(cell, row.title, row.subtitle, row.color, row.symbol);
UIListContentConfiguration *content = (UIListContentConfiguration *)cell.contentConfiguration;
if (SGAccessibilityTextSize() && row.value) content.secondaryText = stackedValue(row);
@@ -924,6 +925,14 @@ static void showWaiting(UITableViewCell *cell, BOOL waiting, NSString *switchTit
} else if (row.action) {
cell.selectionStyle = UITableViewCellSelectionStyleDefault;
}
+ if (row.checked) {
+ BOOL checked = row.checked();
+ // SGChoicePage's checkmark.
+ UIImageView *tick = checked ? SGSymbolView(@"checkmark", 13, UIImageSymbolWeightSemibold, 16) : nil;
+ tick.tintColor = SGGreen();
+ cell.accessoryView = tick;
+ cell.accessibilityTraits = UIAccessibilityTraitButton | (checked ? UIAccessibilityTraitSelected : 0);
+ }
BOOL waiting = [self isWaiting:row];
showWaiting(cell, waiting, [self switchFor:row].title);
if (waiting) cell.selectionStyle = UITableViewCellSelectionStyleNone;
diff --git a/tweak/Sources/Shared/Sing/SGSingModel.m b/tweak/Sources/Shared/Sing/SGSingModel.m
index 1d195e7c..0251f8f5 100644
--- a/tweak/Sources/Shared/Sing/SGSingModel.m
+++ b/tweak/Sources/Shared/Sing/SGSingModel.m
@@ -1,10 +1,12 @@
-// The voice model on the phone (Sing.h): the five files of its compiled separator-ane.mlmodelc, downloaded one by one
-// from the Hugging Face repo's separator-ane.mlmodelc folder, each kept only once its size and SHA-256 are the ones
-// pinned here, then moved together into Application Support/Vitrine/Sing/separator-ane.mlmodelc, which backups leave
-// out (210 MB that can be downloaded again). A dropped connection carries on from where it stopped, three times per
-// file, and a stopped download keeps the file it was in the middle of (NSURLSession's resume data, kept beside the
+// The voice model on the phone (Sing.h): the one Voice model chooses (SGKeySingModel), the full separator-ane.mlmodelc
+// or the light separator-light.mlmodelc, each the five files of a compiled model, downloaded one by one from the
+// Hugging Face repo's folder of that name, each kept only once its size and SHA-256 are the ones pinned here, then
+// moved together into Application Support/Vitrine/Sing/<folder>, which backups leave out (210 or 31 MB that can be
+// downloaded again). Each model has its own staging folder, resume data and paused bytes, so either can be on the
+// iPhone, downloaded or removed without the other. A dropped connection carries on from where it stopped, three times
+// per file, and a stopped download keeps the file it was in the middle of (NSURLSession's resume data, kept beside the
// staging folder) for the next one to carry on from. A resumed request is answered 206, with the whole file handed
-// over all the same. On a FLEX build a copy put in Sing/dev/ by hand comes first.
+// over all the same. On a FLEX build a copy put in Sing/dev/<folder> by hand comes first.
//
// The model Karaoke downloaded before (separator.mlmodelc, 489 MB, the same checkpoint with operations in float32 that
// the Neural Engine does not run) is kept and loaded until this one is in, so Karaoke works through the update: it is
@@ -18,10 +20,8 @@
#import "Core/SGCore.h"
#import "Sing.h"
-static NSString *const kRepo = @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing/resolve/main/separator-ane.mlmodelc/";
+static NSString *const kRepo = @"https://huggingface.co/My-Name-Is-Jeff/vitrine-sing/resolve/main/";
static const int kRetries = 3;
-// Bytes a stopped download has kept, for the row that offers to carry on with it or remove it.
-static NSString *const kPausedKey = @"spotifyglass.sing.downloadPaused";
// Room the download leaves on the disk besides itself.
static const long long kSpareSpace = 64ll * 1000 * 1000;
@@ -31,16 +31,37 @@ typedef struct {
const char *sha256;
} SGSingFile;
-// The repo's separator-ane.mlmodelc (manifest.json beside the compiled folder; upload those exact files, as a
-// recompile changes coremldata.bin), the weights last so the small files fail first.
-static const SGSingFile kFiles[] = {
+// The repo's folders (each with the README's table and, for the light one, a manifest.json; upload those exact
+// files, as a recompile changes coremldata.bin), the weights last so the small files fail first.
+enum { kFileCount = 5 };
+static const SGSingFile kFullFiles[kFileCount] = {
{@"metadata.json", 2333, "101714d1a37ad29c70bf1705ec25ec99584c3cbce27f2245858ace06dacbfb44"},
{@"coremldata.bin", 388, "bcfd38a8b121681dfb9fdfe6f9d88ffb29068e0dd6d24d4012f4c3a7bdcbc81b"},
{@"analytics/coremldata.bin", 243, "55f78aab64f5b1250ce4d6016dff23aee43fb97feb77bbceab4e36b3d9ccffb3"},
{@"model.mil", 1346642, "957024c1075fe331f7ca3410b8bf5e2f83a58c61763caf154729829ce0a92d08"},
{@"weights/weight.bin", 209077208, "bb4a0effafb5121b9aaa5ea96371fa14d0d8dc30a5ae44a93e0c15a2f4077635"},
};
-enum { kFileCount = sizeof kFiles / sizeof kFiles[0] };
+// Mini-BS-RoFormer-18M's vocals, harness/sing/export_light.py.
+static const SGSingFile kLightFiles[kFileCount] = {
+ {@"metadata.json", 2383, "757200c870bd4c5e5b70130c9edbd60029bd81c779b504c57ddcd624597f30e9"},
+ {@"coremldata.bin", 484, "e533e0c22c3f185e5b8871d2b897111e64420736daa7dc7c239206ca2519d428"},
+ {@"analytics/coremldata.bin", 243, "205b1d1d16603c69b4abdfa54b29df71164a16ca4f16a1cf436e7e33ce0b1407"},
+ {@"model.mil", 715945, "d3b62a7972344d1a8eec5de8a21130a19272738bce1135871d8bab098585a841"},
+ {@"weights/weight.bin", 30040968, "ad460549f68898eab7fe946e9675efaa2551d37b93f73796958c73d4881897fc"},
+};
+
+typedef struct {
+ NSString *__unsafe_unretained folder; // the compiled model, in the repo and in Sing/
+ NSString *__unsafe_unretained staging; // its download's folder in Sing/, and its resume data's prefix
+ NSString *__unsafe_unretained pausedKey; // bytes a stopped download kept, for the row that offers to carry on
+ const SGSingFile *files;
+} SGSingModelFiles;
+
+// Indexed by SGSingModelKind. The full model's names are the ones it had before there was a choice.
+static const SGSingModelFiles kModels[] = {
+ {@"separator-ane.mlmodelc", @"download-ane", @"spotifyglass.sing.downloadPaused", kFullFiles},
+ {@"separator-light.mlmodelc", @"download-light", @"spotifyglass.sing.light.downloadPaused", kLightFiles},
+};
// The old model's, checked by size only: it was checked as it came in.
static const SGSingFile kOldFiles[] = {
{@"metadata.json", 2431, NULL},
@@ -55,9 +76,17 @@ NSString *const SGSingChangedNotification = @"SGSingChangedNotification";
// Counts the changes to the model's folder, so SGSingModelURL checks it again only after one.
static NSUInteger sg_modelGeneration;
-static long long totalBytes(void) {
+SGSingModelKind SGSingChosenModel(void) {
+ return [NSUserDefaults.standardUserDefaults integerForKey:SGKeySingModel] == SGSingModelLight ? SGSingModelLight : SGSingModelFull;
+}
+
+static const SGSingModelFiles *chosen(void) {
+ return &kModels[SGSingChosenModel()];
+}
+
+static long long totalBytes(const SGSingModelFiles *model) {
long long total = 0;
- for (int i = 0; i < kFileCount; i++) total += kFiles[i].size;
+ for (int i = 0; i < kFileCount; i++) total += model->files[i].size;
return total;
}
@@ -66,22 +95,27 @@ static NSURL *singFolder(void) {
return [[support URLByAppendingPathComponent:@"Vitrine" isDirectory:YES] URLByAppendingPathComponent:@"Sing" isDirectory:YES];
}
-static NSURL *modelFolder(void) {
- return [singFolder() URLByAppendingPathComponent:@"separator-ane.mlmodelc" isDirectory:YES];
+static NSURL *modelFolder(const SGSingModelFiles *model) {
+ return [singFolder() URLByAppendingPathComponent:model->folder isDirectory:YES];
+}
+
+// A FLEX build's hand-put copy of `model` (SGSingModelURL).
+static NSURL *devFolder(const SGSingModelFiles *model) {
+ return [[singFolder() URLByAppendingPathComponent:@"dev" isDirectory:YES] URLByAppendingPathComponent:model->folder isDirectory:YES];
}
static NSURL *oldFolder(void) {
return [singFolder() URLByAppendingPathComponent:@"separator.mlmodelc" isDirectory:YES];
}
-static NSURL *stagingFolder(void) {
- return [singFolder() URLByAppendingPathComponent:@"download-ane" isDirectory:YES];
+static NSURL *stagingFolder(const SGSingModelFiles *model) {
+ return [singFolder() URLByAppendingPathComponent:model->staging isDirectory:YES];
}
// A stopped download's resume data for file `index`, beside the staging folder: everything in that folder is moved
// into the model.
-static NSURL *resumeFile(int index) {
- return [singFolder() URLByAppendingPathComponent:[NSString stringWithFormat:@"download-ane-%d.resume", index]];
+static NSURL *resumeFile(const SGSingModelFiles *model, int index) {
+ return [singFolder() URLByAppendingPathComponent:[NSString stringWithFormat:@"%@-%d.resume", model->staging, index]];
}
static long long sizeOf(NSURL *url) {
@@ -96,8 +130,8 @@ static BOOL completeWith(NSURL *folder, const SGSingFile *files, int count) {
return YES;
}
-static BOOL complete(NSURL *folder) {
- return completeWith(folder, kFiles, kFileCount);
+static BOOL complete(const SGSingModelFiles *model) {
+ return completeWith(modelFolder(model), model->files, kFileCount);
}
static BOOL oldComplete(void) {
@@ -132,6 +166,7 @@ static void announce(void) {
#pragma mark - the download
@interface SGSingDownloader : NSObject <NSURLSessionDownloadDelegate>
+@property (nonatomic) const SGSingModelFiles *model; // the one chosen when it started; a switch stops it
@property (nonatomic) int file; // the index of the file coming in
@property (nonatomic) long long received; // its bytes so far
@property (nonatomic, copy) NSString *error;
@@ -165,7 +200,7 @@ static BOOL sg_checking; // the weights' checksum is being read
queue.maxConcurrentOperationCount = 1;
_session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:queue];
NSError *error;
- [NSFileManager.defaultManager createDirectoryAtURL:stagingFolder() withIntermediateDirectories:YES attributes:nil error:&error];
+ [NSFileManager.defaultManager createDirectoryAtURL:stagingFolder(self.model) withIntermediateDirectories:YES attributes:nil error:&error];
NSURL *folder = singFolder();
[folder setResourceValue:@YES forKey:NSURLIsExcludedFromBackupKey error:nil];
[queue addOperationWithBlock:^{ [self next]; }];
@@ -180,7 +215,7 @@ static BOOL sg_checking; // the weights' checksum is being read
[session invalidateAndCancel];
return;
}
- NSURL *resume = resumeFile(self.file);
+ NSURL *resume = resumeFile(self.model, self.file);
[task cancelByProducingResumeData:^(NSData *data) {
if (data) [data writeToURL:resume atomically:YES];
SGLog(@"sing: the download is stopped%@", data ? @", what came in is kept" : @"");
@@ -196,13 +231,13 @@ static BOOL sg_checking; // the weights' checksum is being read
}
- (NSURL *)fileURL {
- return [NSURL URLWithString:[kRepo stringByAppendingString:kFiles[self.file].path]];
+ return [NSURL URLWithString:[NSString stringWithFormat:@"%@%@/%@", kRepo, self.model->folder, self.model->files[self.file].path]];
}
// The first file not already in the staging folder from an earlier try, or the move into place.
- (void)next {
if (_cancelled) return;
- while (self.file < kFileCount && sizeOf([stagingFolder() URLByAppendingPathComponent:kFiles[self.file].path]) == kFiles[self.file].size) self.file++;
+ while (self.file < kFileCount && sizeOf([stagingFolder(self.model) URLByAppendingPathComponent:self.model->files[self.file].path]) == self.model->files[self.file].size) self.file++;
self.received = 0;
_attempts = 0;
[self report];
@@ -211,32 +246,32 @@ static BOOL sg_checking; // the weights' checksum is being read
return;
}
// A stop's resume data is used once: if it fails, the file starts over.
- NSData *resume = [NSData dataWithContentsOfURL:resumeFile(self.file)];
- [NSFileManager.defaultManager removeItemAtURL:resumeFile(self.file) error:nil];
+ NSData *resume = [NSData dataWithContentsOfURL:resumeFile(self.model, self.file)];
+ [NSFileManager.defaultManager removeItemAtURL:resumeFile(self.model, self.file) error:nil];
[self download:[self fileURL] resumeData:resume];
- SGLog(@"sing: downloading %@%@", kFiles[self.file].path, resume ? @", carrying on from the last download" : @"");
+ SGLog(@"sing: downloading %@/%@%@", self.model->folder, self.model->files[self.file].path, resume ? @", carrying on from the last download" : @"");
}
- (void)finish {
NSFileManager *files = NSFileManager.defaultManager;
// Resume data a stop wrote after the next download had already looked for it.
- for (int i = 0; i < kFileCount; i++) [files removeItemAtURL:resumeFile(i) error:nil];
- [files removeItemAtURL:modelFolder() error:nil];
+ for (int i = 0; i < kFileCount; i++) [files removeItemAtURL:resumeFile(self.model, i) error:nil];
+ [files removeItemAtURL:modelFolder(self.model) error:nil];
NSError *error;
- BOOL moved = [files moveItemAtURL:stagingFolder() toURL:modelFolder() error:&error];
+ BOOL moved = [files moveItemAtURL:stagingFolder(self.model) toURL:modelFolder(self.model) error:&error];
[self endWithError:moved ? nil : [NSString stringWithFormat:@"The model could not be put in place (%@)", error.localizedDescription]];
}
- (void)endWithError:(NSString *)error {
[_session finishTasksAndInvalidate];
- if (error) SGLog(@"sing: the model's download failed: %@", error);
- else SGLog(@"sing: the model is downloaded and checked");
+ if (error) SGLog(@"sing: the download of %@ failed: %@", self.model->folder, error);
+ else SGLog(@"sing: %@ is downloaded and checked", self.model->folder);
dispatch_async(dispatch_get_main_queue(), ^{
if (sg_downloader != self) return;
sg_downloader = nil;
sg_waitingForNetwork = NO;
sg_lastError = error;
- if (!error) [NSUserDefaults.standardUserDefaults removeObjectForKey:kPausedKey];
+ if (!error) [NSUserDefaults.standardUserDefaults removeObjectForKey:self.model->pausedKey];
// The update in: the old model goes now, and Sing.x loads this one the next time it wants the model.
if (!error) SGSingRemoveOldModel();
sg_modelGeneration++;
@@ -246,8 +281,8 @@ static BOOL sg_checking; // the weights' checksum is being read
- (void)report {
long long done = self.received;
- for (int i = 0; i < self.file && i < kFileCount; i++) done += kFiles[i].size;
- double progress = (double)done / totalBytes();
+ for (int i = 0; i < self.file && i < kFileCount; i++) done += self.model->files[i].size;
+ double progress = (double)done / totalBytes(self.model);
// A few times a second is enough for the row and the button.
static CFAbsoluteTime reported;
CFAbsoluteTime now = CFAbsoluteTimeGetCurrent();
@@ -262,12 +297,12 @@ static BOOL sg_checking; // the weights' checksum is being read
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)task didWriteData:(int64_t)written
totalBytesWritten:(int64_t)totalWritten totalBytesExpectedToWrite:(int64_t)expected {
- self.received = MIN(totalWritten, kFiles[self.file].size);
+ self.received = MIN(totalWritten, self.model->files[self.file].size);
[self report];
}
- (void)URLSession:(NSURLSession *)session taskIsWaitingForConnectivity:(NSURLSessionTask *)task {
- SGLog(@"sing: waiting for the network to download %@", kFiles[self.file].path);
+ SGLog(@"sing: waiting for the network to download %@", self.model->files[self.file].path);
dispatch_async(dispatch_get_main_queue(), ^{
if (sg_downloader != self) return;
sg_waitingForNetwork = YES;
@@ -278,7 +313,7 @@ static BOOL sg_checking; // the weights' checksum is being read
// The file is checked here, before the session deletes it, and kept in the staging folder if it is the one
// pinned.
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)task didFinishDownloadingToURL:(NSURL *)location {
- SGSingFile file = kFiles[self.file];
+ SGSingFile file = self.model->files[self.file];
NSInteger status = [task.response isKindOfClass:NSHTTPURLResponse.class] ? ((NSHTTPURLResponse *)task.response).statusCode : 0;
BOOL answered = status == 200 || status == 206;
if (!answered && _resumed) {
@@ -298,7 +333,7 @@ static BOOL sg_checking; // the weights' checksum is being read
: [NSString stringWithFormat:@"%@ is not the file Karaoke expects (%lld bytes%@)", file.path, size, hash ? @", another checksum" : @""];
return;
}
- NSURL *target = [stagingFolder() URLByAppendingPathComponent:file.path];
+ NSURL *target = [stagingFolder(self.model) URLByAppendingPathComponent:file.path];
[NSFileManager.defaultManager createDirectoryAtURL:target.URLByDeletingLastPathComponent withIntermediateDirectories:YES attributes:nil error:nil];
[NSFileManager.defaultManager removeItemAtURL:target error:nil];
NSError *error;
@@ -327,7 +362,7 @@ static BOOL sg_checking; // the weights' checksum is being read
}
NSData *resume = error.userInfo[NSURLSessionDownloadTaskResumeData];
if (++_attempts <= kRetries) {
- SGLog(@"sing: %@ stopped (%@), trying again%@", kFiles[self.file].path, error.localizedDescription, resume ? @" from where it was" : @"");
+ SGLog(@"sing: %@ stopped (%@), trying again%@", self.model->files[self.file].path, error.localizedDescription, resume ? @" from where it was" : @"");
[self download:[self fileURL] resumeData:resume];
return;
}
@@ -345,28 +380,43 @@ SGSingModelState SGSingModelCurrentState(void) {
}
NSURL *SGSingModelURL(void) {
- // Checked by size once a launch, and again after a download or a delete changes it.
+ // Checked by size once a launch, and again after a download, a delete or a switch changes it.
static int known = -1, oldKnown = -1;
static NSURL *dev;
static NSUInteger knownGeneration = NSUIntegerMax;
+ const SGSingModelFiles *model = chosen();
if (knownGeneration != sg_modelGeneration) {
knownGeneration = sg_modelGeneration;
- known = complete(modelFolder());
- oldKnown = oldComplete();
- // FLEX builds only (Diagnostics' SGIsDebugBuild, which the harnesses do not link): a copy put in Sing/dev/ by
- // hand, for a model not on Hugging Face yet, neither downloaded nor checked. Core ML crashes the process on a
- // compiled model with a file missing (model.mil, weights/weight.bin; a short file fails cleanly), so a copy
- // cut off halfway is not loaded.
- NSURL *folder = [[singFolder() URLByAppendingPathComponent:@"dev" isDirectory:YES] URLByAppendingPathComponent:@"separator-ane.mlmodelc" isDirectory:YES];
+ known = complete(model);
+ // The old model stands in for the full one alone: it is the full one's earlier export.
+ oldKnown = model == &kModels[SGSingModelFull] && oldComplete();
+ // FLEX builds only (Diagnostics' SGIsDebugBuild, which the harnesses do not link): a copy of the chosen model
+ // put in Sing/dev/ by hand, for a model not on Hugging Face yet, neither downloaded nor checked. Core ML crashes
+ // the process on a compiled model with a file missing (model.mil, weights/weight.bin; a short file fails
+ // cleanly), so a copy cut off halfway is not loaded.
+ NSURL *folder = devFolder(model);
int present = 0;
- for (int i = 0; i < kFileCount; i++) present += sizeOf([folder URLByAppendingPathComponent:kFiles[i].path]) > 0;
+ for (int i = 0; i < kFileCount; i++) present += sizeOf([folder URLByAppendingPathComponent:model->files[i].path]) > 0;
BOOL there = present == kFileCount, flex = NSClassFromString(@"FLEXManager") != nil;
- if (there && flex && !dev) SGLog(@"sing: DEV MODEL: the voice model is the unchecked copy in %@, not a download (a FLEX build)", folder.path);
+ if (there && flex && ![dev isEqual:folder]) SGLog(@"sing: DEV MODEL: the voice model is the unchecked copy in %@, not a download (a FLEX build)", folder.path);
if (there && !flex) SGLog(@"sing: a dev model is in %@, and only a FLEX build loads it", folder.path);
if (present && !there) SGLog(@"sing: the dev model in %@ has %d of its %d files, so it is not loaded", folder.path, present, kFileCount);
dev = there && flex ? folder : nil;
}
- return dev ?: known ? modelFolder() : oldKnown ? oldFolder() : nil;
+ return dev ?: known ? modelFolder(model) : oldKnown ? oldFolder() : nil;
+}
+
+void SGSetSingChosenModel(SGSingModelKind kind) {
+ if (kind == SGSingChosenModel()) return;
+ // The other model's download stops, keeping what came in under its own name for when it is chosen again.
+ if (sg_downloader) SGSingCancelModelDownload();
+ [NSUserDefaults.standardUserDefaults setInteger:kind forKey:SGKeySingModel];
+ sg_lastError = nil;
+ sg_modelGeneration++;
+ SGLog(@"sing: the voice model is now %@ (%@)", kModels[kind].folder,
+ SGSingModelURL() ? @"on the iPhone, Karaoke loads it" : @"not downloaded");
+ // Sing.x loads it in place of the one before, or lets that one go if this one is not here.
+ announce();
}
// This model in place, the dev copy or the download: nothing to download.
@@ -387,14 +437,19 @@ NSString *SGSingModelError(void) {
}
NSString *SGSingModelSizeText(void) {
- return [NSByteCountFormatter stringFromByteCount:totalBytes() countStyle:NSByteCountFormatterCountStyleFile];
+ return SGSingModelSizeTextOf(SGSingChosenModel());
+}
+
+NSString *SGSingModelSizeTextOf(SGSingModelKind kind) {
+ return [NSByteCountFormatter stringFromByteCount:totalBytes(&kModels[kind]) countStyle:NSByteCountFormatterCountStyleFile];
}
// What is still to come: the files not in the staging folder yet.
static long long bytesToCome(void) {
+ const SGSingModelFiles *model = chosen();
long long left = 0;
for (int i = 0; i < kFileCount; i++) {
- if (sizeOf([stagingFolder() URLByAppendingPathComponent:kFiles[i].path]) != kFiles[i].size) left += kFiles[i].size;
+ if (sizeOf([stagingFolder(model) URLByAppendingPathComponent:model->files[i].path]) != model->files[i].size) left += model->files[i].size;
}
return left;
}
@@ -418,6 +473,7 @@ void SGSingDownloadModel(void) {
sg_progress = 0;
sg_waitingForNetwork = NO;
sg_downloader = [SGSingDownloader new];
+ sg_downloader.model = chosen();
[sg_downloader start];
announce();
}
@@ -444,23 +500,22 @@ BOOL SGSingModelChecking(void) {
}
long long SGSingModelPausedBytes(void) {
- return sg_downloader || hasModel() ? 0 : [NSUserDefaults.standardUserDefaults integerForKey:kPausedKey];
+ return sg_downloader || hasModel() ? 0 : [NSUserDefaults.standardUserDefaults integerForKey:chosen()->pausedKey];
}
void SGSingCancelModelDownload(void) {
- if (sg_downloader) [NSUserDefaults.standardUserDefaults setInteger:(NSInteger)(sg_progress * totalBytes()) forKey:kPausedKey];
- [sg_downloader cancel];
+ SGSingDownloader *downloader = sg_downloader;
+ if (downloader) [NSUserDefaults.standardUserDefaults setInteger:(NSInteger)(sg_progress * totalBytes(downloader.model)) forKey:downloader.model->pausedKey];
+ [downloader cancel];
sg_downloader = nil;
sg_modelGeneration++;
announce();
}
-void SGSingDeleteModel(void) {
- SGSingCancelModelDownload();
- [NSFileManager.defaultManager removeItemAtURL:singFolder() error:nil];
- [NSUserDefaults.standardUserDefaults removeObjectForKey:kPausedKey];
- sg_modelGeneration++;
- announce();
+static long long bytesIn(NSURL *url) {
+ long long bytes = sizeOf(url);
+ for (NSURL *file in [NSFileManager.defaultManager enumeratorAtURL:url includingPropertiesForKeys:@[NSURLFileSizeKey] options:0 errorHandler:nil]) bytes += sizeOf(file);
+ return bytes;
}
// Deletes what of `urls` is there, adding the bytes freed to `freed`; whether there was any.
@@ -470,31 +525,82 @@ static BOOL removeAll(NSArray<NSURL *> *urls, long long *freed) {
for (NSURL *url in urls) {
if (![files fileExistsAtPath:url.path]) continue;
found = YES;
- *freed += sizeOf(url);
- for (NSURL *file in [files enumeratorAtURL:url includingPropertiesForKeys:@[NSURLFileSizeKey] options:0 errorHandler:nil]) *freed += sizeOf(file);
+ *freed += bytesIn(url);
[files removeItemAtURL:url error:nil];
}
return found;
}
-void SGSingRemoveOldModel(void) {
- // What the old model's download left: its staging folder and resume data, one per file (named one by one:
- // download-ane-N.resume is this model's own). The bytes a stopped download kept were the old model's.
+// What the old model's download left: its staging folder and resume data, one per file (named one by one:
+// download-ane-N.resume and download-light-N.resume are the two models' own).
+static NSArray<NSURL *> *oldLeftovers(void) {
NSMutableArray<NSURL *> *leftovers = [NSMutableArray arrayWithObject:[singFolder() URLByAppendingPathComponent:@"download"]];
for (int i = 0; i < 5; i++) [leftovers addObject:[singFolder() URLByAppendingPathComponent:[NSString stringWithFormat:@"download-%d.resume", i]]];
+ return leftovers;
+}
+
+// Everything of `model` in Sing/: the compiled model, its download's folder and resume data and a FLEX build's copy,
+// and for the full model the old one it replaced, with what that one's download left.
+static NSArray<NSURL *> *filesOf(const SGSingModelFiles *model) {
+ NSMutableArray<NSURL *> *urls = [NSMutableArray arrayWithObjects:modelFolder(model), stagingFolder(model), devFolder(model), nil];
+ for (int i = 0; i < kFileCount; i++) [urls addObject:resumeFile(model, i)];
+ if (model == &kModels[SGSingModelFull]) {
+ [urls addObject:oldFolder()];
+ [urls addObjectsFromArray:oldLeftovers()];
+ }
+ return urls;
+}
+
+static const SGSingModelFiles *unchosen(void) {
+ return &kModels[SGSingChosenModel() == SGSingModelFull ? SGSingModelLight : SGSingModelFull];
+}
+
+static void removeModel(const SGSingModelFiles *model) {
+ long long freed = 0;
+ removeAll(filesOf(model), &freed);
+ [NSUserDefaults.standardUserDefaults removeObjectForKey:model->pausedKey];
+ SGLog(@"sing: %@ is removed, %@ freed", model->folder, [NSByteCountFormatter stringFromByteCount:freed countStyle:NSByteCountFormatterCountStyleFile]);
+ sg_modelGeneration++;
+ announce();
+}
+
+void SGSingDeleteModel(void) {
+ SGSingCancelModelDownload();
+ removeModel(chosen());
+}
+
+long long SGSingOtherModelBytes(void) {
+ NSFileManager *files = NSFileManager.defaultManager;
+ long long bytes = 0;
+ for (NSURL *url in filesOf(unchosen())) {
+ if ([files fileExistsAtPath:url.path]) bytes += bytesIn(url);
+ }
+ return bytes;
+}
+
+// No download of it runs: choosing the other model stopped it.
+void SGSingDeleteOtherModel(void) {
+ removeModel(unchosen());
+}
+
+void SGSingRemoveOldModel(void) {
+ // The bytes a stopped download kept were the old model's.
+ const SGSingModelFiles *full = &kModels[SGSingModelFull];
long long freed = 0;
- BOOL left = removeAll(leftovers, &freed);
- if (left) [NSUserDefaults.standardUserDefaults removeObjectForKey:kPausedKey];
- // The old model itself only once this one is in: until then Karaoke runs on it.
- BOOL there = [NSFileManager.defaultManager fileExistsAtPath:oldFolder().path], replaced = complete(modelFolder());
+ BOOL left = removeAll(oldLeftovers(), &freed);
+ if (left) [NSUserDefaults.standardUserDefaults removeObjectForKey:full->pausedKey];
+ // The old model itself only once the full model is in: until then Karaoke runs on it. The light model is left alone.
+ BOOL there = [NSFileManager.defaultManager fileExistsAtPath:oldFolder().path], replaced = complete(full);
BOOL old = there && replaced && removeAll(@[oldFolder()], &freed);
NSString *size = [NSByteCountFormatter stringFromByteCount:freed countStyle:NSByteCountFormatterCountStyleFile];
- if (old) SGLog(@"sing: the old voice model (separator.mlmodelc) is deleted, %@ freed; the model is separator-ane.mlmodelc", size);
+ if (old) SGLog(@"sing: the old voice model (separator.mlmodelc) is deleted, %@ freed; the full model is separator-ane.mlmodelc", size);
else if (left) SGLog(@"sing: what the old voice model's download left is deleted, %@ freed", size);
- // Which model is in use: a FLEX build's dev copy of the new one comes before the old.
+ // Which model is in use: a FLEX build's dev copy of the new one comes before the old, and Light needs neither.
if (there && !replaced) {
SGLog(SGSingModelUpdateAvailable()
? @"sing: the old voice model (separator.mlmodelc) is kept until separator-ane.mlmodelc is downloaded, and Karaoke runs on it, on the CPU alone"
+ : SGSingChosenModel() == SGSingModelLight
+ ? @"sing: the old voice model (separator.mlmodelc) is kept until separator-ane.mlmodelc is downloaded; Voice model is Light, so it is not in use"
: @"sing: the old voice model (separator.mlmodelc) is kept until separator-ane.mlmodelc is downloaded; the dev copy is in use");
}
}
@@ -510,7 +616,7 @@ BOOL SGSingModelUpdateDownloading(void) {
NSString *SGSingModelInUseSizeText(void) {
long long total = 0;
for (size_t i = 0; i < sizeof kOldFiles / sizeof kOldFiles[0]; i++) total += kOldFiles[i].size;
- return [NSByteCountFormatter stringFromByteCount:SGSingModelUpdateAvailable() ? total : totalBytes() countStyle:NSByteCountFormatterCountStyleFile];
+ return [NSByteCountFormatter stringFromByteCount:SGSingModelUpdateAvailable() ? total : totalBytes(chosen()) countStyle:NSByteCountFormatterCountStyleFile];
}
NSArray<NSString *> *SGSingComputeUnitNames(void) {
diff --git a/tweak/Sources/Shared/Sing/Sing.h b/tweak/Sources/Shared/Sing/Sing.h
index 8d2fbb32..8d8b1351 100644
--- a/tweak/Sources/Shared/Sing/Sing.h
+++ b/tweak/Sources/Shared/Sing/Sing.h
@@ -21,7 +21,9 @@
// notice): Vitrine's export of that checkpoint for the Neural Engine (its layout (B, C, 1, S), 1x1 convolutions,
// attention per head, an RMSNorm that divides by the row's largest value first so half precision does not overflow),
// palettized to 6 bits. Every operation runs on the Neural Engine, and the CPU runs it too, more slowly. Its floor is
-// iOS 18, the Core ML it was built with. No audio leaves the phone.
+// iOS 18, the Core ML it was built with. No audio leaves the phone. Voice model can choose a light model instead, from
+// the same repo's separator-light.mlmodelc: the vocals of HiDolen's Mini-BS-RoFormer-18M (MIT), exported in float16
+// by harness/sing/export_light.py with the same input and output, smaller and less clean.
//
// Threading: main thread, but for what Sing.x's render side and the engine say.
#import <UIKit/UIKit.h>
@@ -133,6 +135,24 @@ UIViewController *SGSingSettingsPage(void);
#pragma mark - the model (SGSingModel.m)
+// Which voice model Karaoke downloads and runs, an SGSingModelKind: the full one until Light is chosen. Read live, so
+// a switch loads the other model for what plays next, without a restart.
+#define SGKeySingModel @"spotifyglass.sing.model"
+typedef NS_ENUM(NSInteger, SGSingModelKind) {
+ SGSingModelFull, // separator-ane.mlmodelc, 210 MB: the cleanest separation
+ SGSingModelLight, // separator-light.mlmodelc, 31 MB: Mini-BS-RoFormer-18M's vocals, for older iPhones
+};
+SGSingModelKind SGSingChosenModel(void);
+// Stored, a running download stopped (it keeps what came in for when its model is chosen again), and announced.
+void SGSetSingChosenModel(SGSingModelKind kind);
+// A model's download size, for the rows that choose it.
+NSString *SGSingModelSizeTextOf(SGSingModelKind kind);
+// The bytes the model not chosen takes on the iPhone, downloaded or partly (the old model with the full one), and
+// their removal.
+long long SGSingOtherModelBytes(void);
+void SGSingDeleteOtherModel(void);
+// Everything below is about the chosen model.
+
typedef NS_ENUM(NSInteger, SGSingModelState) {
SGSingModelMissing,
SGSingModelDownloading, // with no model to run meanwhile (the old one runs on through its update's download)
@@ -158,9 +178,9 @@ NSString *SGSingModelSizeText(void);
void SGSingDownloadModel(void);
void SGSingCancelModelDownload(void);
void SGSingDeleteModel(void);
-// The compiled model's folder: separator-ane.mlmodelc once it is downloaded and checked, else the old model
-// (separator.mlmodelc) where it is still on the iPhone, else nil. On a FLEX build a copy put in Application
-// Support/Vitrine/Sing/dev/ by hand comes first, unchecked.
+// The chosen model's compiled folder once it is downloaded and checked; for the full model, else the old model
+// (separator.mlmodelc) where it is still on the iPhone; else nil. On a FLEX build a copy put in Application
+// Support/Vitrine/Sing/dev/<separator-ane or separator-light>.mlmodelc by hand comes first, unchecked.
NSURL *SGSingModelURL(void);
// The model is the old one: it runs on the CPU alone, and the download is its update.
BOOL SGSingModelUpdateAvailable(void);
@@ -169,7 +189,7 @@ BOOL SGSingModelUpdateDownloading(void);
// The size of the model on the iPhone now, for the row that removes it.
NSString *SGSingModelInUseSizeText(void);
// Deletes what the old model's download left, and the old model (separator.mlmodelc) itself once separator-ane.mlmodelc
-// is in, logging the space freed; from any thread, at launch and as the download ends.
+// is in, logging the space freed; from any thread, at launch and as the download ends. The light model is not touched.
void SGSingRemoveOldModel(void);
NSArray<NSString *> *SGSingComputeUnitNames(void);
// Whether this OS can load the model (iOS 18). Memory is checked as the model loads (SGSingLoader.m).
diff --git a/tweak/Sources/Shared/Sing/SingSettings.m b/tweak/Sources/Shared/Sing/SingSettings.m
index f33f7748..99d82810 100644
--- a/tweak/Sources/Shared/Sing/SingSettings.m
+++ b/tweak/Sources/Shared/Sing/SingSettings.m
@@ -1,7 +1,7 @@
// The Sing page (Sing on Mod Settings' main page, and Lyrics > Karaoke), under either look: the card at its top
// (SGSingCard.m: the song, what Sing is doing, the vocals and the rest traced live, play and pause, the vocals'
// level and its three stops), Sing's switch, which turns the mic on and off at once, Ignore heat warnings, the voice
-// model and its removal, spatial voice's page (its preview, SGSpatialPreview.m, and its switch), and Runs on under
+// model (Best quality or Light, the chosen one's download and removal, and the other's removal), spatial voice's page (its preview, SGSpatialPreview.m, and its switch), and Runs on under
// Advanced. While the page is on screen with Spotify in front, the card's reads of its lines keep Sing separating
// the playing song even at As sung, where it otherwise rests (SGSingReadLevels), so what the card shows and what
// its slider sets are heard at once; popped, covered or in the background, the page no longer counts.
@@ -84,7 +84,11 @@ static void modelTapped(UITableViewController *page) {
SGSingModelError() ? [NSString stringWithFormat:@"The last try failed: %@\n\n", SGSingModelError()] : @"",
update ? @"The new voice model runs on the Neural Engine, faster and cooler, and Karaoke keeps the one it has until it is in. " : @"",
SGSingModelSizeText()];
- if (untestedChip()) message = [message stringByAppendingString:@"\n\nKaraoke is tested on the iPhone 15 Pro. On iPhones older than the iPhone 12 it may not keep up."];
+ if (untestedChip()) {
+ message = [message stringByAppendingString:SGSingChosenModel() == SGSingModelLight
+ ? @"\n\nKaraoke is tested on the iPhone 15 Pro. On iPhones older than the iPhone 12 it may not keep up, even with the light model."
+ : @"\n\nKaraoke is tested on the iPhone 15 Pro. On iPhones older than the iPhone 12 it may not keep up; the light model has a better chance."];
+ }
NSString *title = SGSingModelPausedBytes() ? (update ? @"Carry on with the update?" : @"Carry on with the download?")
: (update ? @"Download the update?" : @"Download the voice model?");
ask(title, message, @"Download", NO, ^{ SGSingDownloadModel(); });
@@ -224,6 +228,36 @@ UIViewController *SGSingSettingsPage(void) {
});
remove.color = SGRed();
remove.visible = ^BOOL { return SGSingModelCurrentState() == SGSingModelReady || SGSingModelPausedBytes() > 0; };
+ // The two models, picked on the page: the rows below follow the one with the checkmark.
+ SGModRow *(^choice)(NSString *, NSString *, SGSingModelKind) = ^SGModRow *(NSString *title, NSString *format, SGSingModelKind kind) {
+ SGModRow *row = SGActionRow(title, [NSString stringWithFormat:format, SGSingModelSizeTextOf(kind)], ^{
+ if (SGSingChosenModel() == kind) return;
+ void (^choose)(void) = ^{
+ SGSetSingChosenModel(kind);
+ [page.tableView reloadData];
+ [page refreshVisibility];
+ };
+ BOOL downloading = SGSingModelCurrentState() == SGSingModelDownloading || SGSingModelUpdateDownloading();
+ if (!downloading) choose();
+ else ask(@"Stop the download?", @"The voice model coming in stops, and what has come in is kept for when you choose it again.", @"Switch", NO, choose);
+ });
+ row.checked = ^BOOL { return SGSingChosenModel() == kind; };
+ return row;
+ };
+ SGModRow *full = choice(@"Best quality", @"The full model, %@: the cleanest result, for recent iPhones", SGSingModelFull);
+ SGModRow *light = choice(@"Light", @"A small model, %@, for older iPhones: more of the voice is left in, and the music dips under it",
+ SGSingModelLight);
+ SGModRow *other = SGStatActionRow(@"Remove the other model", nil, ^NSString *{ return bytes(SGSingOtherModelBytes()); }, ^{
+ NSString *name = SGSingChosenModel() == SGSingModelFull ? @"Light" : @"Best quality";
+ ask(@"Remove the other model?", [NSString stringWithFormat:@"%@ is not in use. Removing it frees %@, and choosing it again downloads it again.",
+ name, bytes(SGSingOtherModelBytes())],
+ @"Remove", YES, ^{
+ SGSingDeleteOtherModel();
+ [page refreshVisibility];
+ });
+ });
+ other.color = SGRed();
+ other.visible = ^BOOL { return SGSingOtherModelBytes() > 0; };
SGModRow *spatial = SGPageRow(@"Spatial voice", ^UIViewController *{ return SGSpatialVoiceSettingsPage(); });
spatial.subtitle = @"With AirPods, the voice stays in front of you as you turn your head";
spatial.value = ^NSString *{ return SGSingSpatial() ? @"On" : @"Off"; };
@@ -240,7 +274,10 @@ UIViewController *SGSingSettingsPage(void) {
SGModRow *prepare = SGSwitchRow(@"Prepare after updates",
@"Readies the Neural Engine in the background after Vitrine or iOS updates, so Karaoke starts at full speed", SGKeySingPrepareAhead);
SGSingPage *made = [[SGSingPage alloc] initWithTitle:@"Karaoke" intro:nil sections:@[
- SGSection(nil, @[sing, button, heat, memory, model, remove]),
+ SGSection(nil, @[sing, button, heat, memory]),
+ SGNotedSection(@"Voice model", @[full, light, model, remove, other],
+ @"Each model is downloaded and kept on its own, and Karaoke needs only the chosen one. A switch takes effect "
+ @"without restarting Spotify: the model loads in a few seconds, and the song plays as it is until it is in."),
SGSection(nil, @[spatial]),
SGNotedSection(@"Advanced", @[units, prepare],
@"Karaoke turns a song's vocals down to sing over, or the rest down to hear the vocals alone, with a voice model that "