Native Android client for Spotify, built with Jetpack Compose
Kotlin ad-freeadblockandroidjetpack-composekotlinlyricsmusic-playermusic-videomyspotispotifyspotify-adblockerspotify-clientspotify-connectspotui
AutoMix rewritten: whole-song analysis (Beat This small0 + onset grid), two decks that swap roles (no song handed between players), speed through the AudioTrack, a phase lock while the next song is still muted, bass swap on a DynamicsProcessing EQ, mix on Next, and no mix between songs that follow each other on their album
kota-zoea0d025ce181
diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.mdindex 0a89de0f..d6876b5a 100644--- a/THIRD_PARTY_NOTICES.md+++ b/THIRD_PARTY_NOTICES.md@@ -3,6 +3,22 @@Myspoti is free software under the GNU General Public License v3.0 (see `LICENSE`).It builds on the projects below; their licences are compatible with the GPL-3.0.+## Beat This! — MIT+https://github.com/CPJKU/beat_this++By Francesco Foscarin, Jan Schlüter and Gerhard Widmer (CPJKU, Johannes Kepler University Linz),+"Beat This! Accurate beat tracking without DBN postprocessing" (ISMIR 2024). AutoMix ships the+`small0` checkpoint, exported to ONNX and quantized to int8 (`base/app/src/main/assets/automix/+beat-this-small0-int8.onnx`); upstream publishes the code and the weights under the MIT licence.+The log-mel front end in `base/automix/.../BeatThis.kt` follows its input contract. The training+recordings are not bundled.++## ONNX Runtime — MIT+https://github.com/microsoft/onnxruntime++Copyright Microsoft Corporation. `com.microsoft.onnxruntime:onnxruntime-android` 1.23.2 runs the+Beat This model on the phone (the desktop bench uses the JVM build of the same version).+## Spotui — GPL-3.0https://github.com/Spotui/Spotuidiff --git a/base/app/build.gradle.kts b/base/app/build.gradle.ktsindex bd5c3845..0565f4db 100644--- a/base/app/build.gradle.kts+++ b/base/app/build.gradle.kts@@ -110,6 +110,9 @@ android {dependencies {implementation(project(":host-android"))+ // AutoMix: song analysis and transition planning (pure Kotlin), Beat This on ONNX Runtime.+ implementation(project(":automix"))+ implementation("com.microsoft.onnxruntime:onnxruntime-android:1.23.2")// Third-party extensions: the sandbox (Rhino), manifests, packages and catalogues.implementation(project(":extensions"))// Jam: scan the invite QR code (Google code scanner, no camera permission).diff --git a/base/app/src/main/assets/automix/beat-this-small0-int8.onnx b/base/app/src/main/assets/automix/beat-this-small0-int8.onnxnew file mode 100644index 00000000..05ce7fdcBinary files /dev/null and b/base/app/src/main/assets/automix/beat-this-small0-int8.onnx differdiff --git a/base/app/src/main/java/com/music/spotui/automix/AutoMix.kt b/base/app/src/main/java/com/music/spotui/automix/AutoMix.ktnew file mode 100644index 00000000..d0e1c699--- /dev/null+++ b/base/app/src/main/java/com/music/spotui/automix/AutoMix.kt@@ -0,0 +1,161 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package com.music.spotui.automix++import android.net.Uri+import androidx.compose.runtime.getValue+import androidx.compose.runtime.mutableStateOf+import androidx.compose.runtime.setValue+import app.myspoti.automix.Analyzer+import app.myspoti.automix.BeatThis+import app.myspoti.automix.MixPlan+import app.myspoti.automix.MixPlanner+import app.myspoti.automix.TrackAnalysis+import com.music.spotui.di.Graph+import com.music.spotui.extensions.ExtensionAudio+import java.io.File+import java.security.MessageDigest+import java.util.concurrent.Executors++/**+ * AutoMix: DJ-style transitions between the local player's songs (extension streams, downloads,+ * the phone's files). Spotify songs play in the Web Player, whose audio the app never has.+ *+ * Each song is analysed once in the background (see [Analyzer]) and the result kept in the cache;+ * [planFor] gives the transition between two analysed songs. LocalPlayer plays it (see its AutoMix part).+ */+object AutoMix {+ private const val PREFS = "myspoti_prefs"+ private const val KEY = "automix_on"++ /** The setting, Compose state (off by default). */+ var on by mutableStateOf(Graph.app.getSharedPreferences(PREFS, 0).getBoolean(KEY, false))+ private set++ fun set(value: Boolean) {+ on = value+ Graph.app.getSharedPreferences(PREFS, 0).edit().putBoolean(KEY, value).apply()+ journal("AutoMix ${if (value) "on" else "off"}")+ }++ // ── Journal (last lines, for the command line and the stats sheet) ──++ private val lines = ArrayDeque<String>()+ fun journal(line: String) {+ app.myspoti.host.android.Logbook.log("automix: $line")+ synchronized(lines) { lines.addLast(line); while (lines.size > 60) lines.removeFirst() }+ }+ fun journalLines(): List<String> = synchronized(lines) { lines.toList() }++ // ── Analysis ──++ private val worker = Executors.newSingleThreadExecutor { r -> Thread(r, "automix").apply { priority = Thread.MIN_PRIORITY; isDaemon = true } }+ private val memory = object : LinkedHashMap<String, TrackAnalysis>(16, .75f, true) {+ override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, TrackAnalysis>?) = size > 24+ }+ private val asked = HashSet<String>()+ private val failed = HashSet<String>()+ @Volatile private var wanted: Set<String> = emptySet()++ /** The analysis of [uri] if it is ready (memory or cache), else null. */+ fun analysis(uri: Uri): TrackAnalysis? {+ val key = key(uri)+ synchronized(memory) { memory[key]?.let { return it } }+ return null+ }++ /**+ * Asks for the analysis of [uris] (the song playing and the next ones), dropping the asks for+ * any other song; [ready] runs on the worker when one is done.+ */+ fun request(uris: List<Uri>, ready: () -> Unit) {+ wanted = uris.map(::key).toSet()+ for (uri in uris) {+ val key = key(uri)+ synchronized(memory) { if (memory.containsKey(key) || key in failed || !asked.add(key)) continue }+ worker.execute {+ try {+ if (key !in wanted) return@execute+ val a = load(key) ?: analyse(uri, key)+ if (a != null) { synchronized(memory) { memory[key] = a }; ready() }+ } finally { synchronized(memory) { asked -= key } }+ }+ }+ }++ private fun analyse(uri: Uri, key: String): TrackAnalysis? {+ // A stream is read a second time for the analysis: on unmetered networks only.+ if (uri.scheme == ExtensionAudio.SCHEME) {+ if (com.music.spotui.data.DataSaver.metered) { journal("not analysed (metered network): ${uri.lastPathSegment}"); return null }+ val s = runCatching { ExtensionAudio.resolve(uri, ahead = true) }.getOrNull()+ if (s == null || s.manifest != null) { synchronized(memory) { failed += key }; journal("not analysed (${if (s == null) "no stream" else "DASH stream"}): ${uri.lastPathSegment}"); return null }+ }+ val t0 = System.currentTimeMillis()+ return try {+ val cancelled = { key !in wanted }+ val pcm = TrackDecoder.decode(uri, cancelled)+ val t1 = System.currentTimeMillis()+ val a = Analyzer.analyze(pcm.mid, pcm.side, model(), cancelled, THREADS)+ save(key, a)+ journal(String.format("analysed %s: %.1f BPM%s, %d bars, key %d, decode %d ms, analysis %d ms", label(uri), a.bpm,+ if (a.steady) " steady" else (if (a.tail != null) " (tail grid)" else " (no grid)"), a.bars, a.key, t1 - t0, System.currentTimeMillis() - t1))+ a+ } catch (e: InterruptedException) {+ null+ } catch (e: Throwable) {+ synchronized(memory) { failed += key }+ journal("analysis failed for ${label(uri)}: ${e.javaClass.simpleName} ${e.message?.take(120)}")+ null+ }+ }++ /** Cores for one analysis: all but two (the player and the screen keep theirs). */+ private val THREADS = (Runtime.getRuntime().availableProcessors() - 2).coerceIn(2, 6)++ private var beatThis: BeatThis? = null+ private var modelFailed = false++ /** Beat This, loaded once on the worker (null if it cannot run here: the onsets then decide alone). */+ private fun model(): BeatThis? {+ if (beatThis != null || modelFailed) return beatThis+ return try {+ BeatThis(Graph.app.assets.open("automix/beat-this-small0-int8.onnx").use { it.readBytes() }, THREADS).also { beatThis = it }+ } catch (e: Throwable) {+ modelFailed = true+ journal("Beat This unavailable: ${e.javaClass.simpleName} ${e.message?.take(120)}")+ null+ }+ }++ // ── Plans ──++ fun planFor(a: TrackAnalysis, b: TrackAnalysis): MixPlan = MixPlanner.plan(a, b)++ // ── Cache ──++ private fun dir() = File(Graph.app.cacheDir, "automix4").apply { mkdirs() }++ private fun load(key: String): TrackAnalysis? =+ runCatching { File(dir(), key).takeIf { it.exists() }?.inputStream()?.buffered()?.use { TrackAnalysis.read(it) } }.getOrNull()++ private fun save(key: String, a: TrackAnalysis) {+ runCatching {+ val f = File(dir(), key)+ val tmp = File(dir(), "$key.tmp")+ tmp.outputStream().buffered().use { a.write(it) }+ tmp.renameTo(f)+ }+ }++ /** An extension stream is keyed by its track (its address changes); a file by its address. */+ fun key(uri: Uri): String {+ val id = if (uri.scheme == ExtensionAudio.SCHEME) "ext:" + uri.lastPathSegment.orEmpty() else uri.toString()+ return MessageDigest.getInstance("SHA-1").digest(id.toByteArray()).joinToString("") { "%02x".format(it) }.take(24)+ }++ private fun label(uri: Uri) = uri.lastPathSegment?.takeLast(48).orEmpty()++ fun clearCache() {+ synchronized(memory) { memory.clear(); failed.clear() }+ dir().listFiles()?.forEach { it.delete() }+ }+}diff --git a/base/app/src/main/java/com/music/spotui/automix/DeckEq.kt b/base/app/src/main/java/com/music/spotui/automix/DeckEq.ktnew file mode 100644index 00000000..9da87fc8--- /dev/null+++ b/base/app/src/main/java/com/music/spotui/automix/DeckEq.kt@@ -0,0 +1,53 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package com.music.spotui.automix++import android.media.audiofx.DynamicsProcessing+import app.myspoti.automix.Features++/**+ * A player's three-band EQ for the mixes (low under 180 Hz, mid, high over 2.5 kHz), as Android's+ * DynamicsProcessing on its audio session: it acts on what is heard now, not after the player's+ * buffer (an AudioProcessor would be late by the whole buffer). Neutral (0 dB) outside a mix. Null+ * when the phone refuses the effect: the mixes then use the gains alone.+ */+class DeckEq private constructor(private val dp: DynamicsProcessing) {+ private var low = 0f; private var high = 0f++ /** Band levels in dB (0 = untouched, down to −30). */+ fun set(lowDb: Double, highDb: Double) {+ val l = lowDb.toFloat().coerceIn(-40f, 0f); val h = highDb.toFloat().coerceIn(-40f, 0f)+ if (kotlin.math.abs(l - low) < 0.25f && kotlin.math.abs(h - high) < 0.25f) return+ runCatching {+ if (kotlin.math.abs(l - low) >= 0.25f) dp.setPreEqBandAllChannelsTo(0, DynamicsProcessing.EqBand(true, Features.LOW_HZ.toFloat(), l))+ if (kotlin.math.abs(h - high) >= 0.25f) dp.setPreEqBandAllChannelsTo(2, DynamicsProcessing.EqBand(true, 20_000f, h))+ low = l; high = h+ }+ }++ fun neutral() = set(0.0, 0.0)++ fun release() { runCatching { dp.release() } }++ companion object {+ fun attach(sessionId: Int): DeckEq? {+ if (sessionId <= 0) return null+ return try {+ val cfg = DynamicsProcessing.Config.Builder(DynamicsProcessing.VARIANT_FAVOR_TIME_RESOLUTION, 2,+ true, 3, false, 0, false, 0, false)+ .setPreferredFrameDuration(10f)+ .build()+ val eq = DynamicsProcessing.Eq(true, true, 3)+ eq.setBand(0, DynamicsProcessing.EqBand(true, Features.LOW_HZ.toFloat(), 0f))+ eq.setBand(1, DynamicsProcessing.EqBand(true, Features.HIGH_HZ.toFloat(), 0f))+ eq.setBand(2, DynamicsProcessing.EqBand(true, 20_000f, 0f))+ cfg.setPreEqAllChannelsTo(eq)+ val dp = DynamicsProcessing(0, sessionId, cfg)+ dp.enabled = true+ DeckEq(dp)+ } catch (e: Throwable) {+ AutoMix.journal("EQ unavailable on session $sessionId: ${e.javaClass.simpleName}")+ null+ }+ }+ }+}diff --git a/base/app/src/main/java/com/music/spotui/automix/TrackDecoder.kt b/base/app/src/main/java/com/music/spotui/automix/TrackDecoder.ktnew file mode 100644index 00000000..c7244bf3--- /dev/null+++ b/base/app/src/main/java/com/music/spotui/automix/TrackDecoder.kt@@ -0,0 +1,197 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package com.music.spotui.automix++import android.media.AudioFormat+import android.media.MediaCodec+import android.media.MediaDataSource+import android.media.MediaExtractor+import android.media.MediaFormat+import android.net.Uri+import androidx.media3.common.C+import androidx.media3.common.util.UnstableApi+import androidx.media3.datasource.DataSpec+import com.music.spotui.data.offline.OfflineCrypto+import com.music.spotui.di.Graph+import java.io.IOException+import java.nio.ByteBuffer+import java.nio.ByteOrder++/**+ * Decodes a whole song to the analysis signals: mid ((L+R)/2) and side ((L-R)/2) at 22.05 kHz.+ * The bytes come through the player's own data source (downloads are decrypted there, extension+ * streams resolved), so a song is read the way it plays. Not for DASH answers (no single file).+ */[email protected](UnstableApi::class)+object TrackDecoder {+ const val RATE = app.myspoti.automix.Features.RATE+ /** Longer songs are not analysed (memory: 4 bytes × 2 signals × 22 050 per second). */+ const val MAX_SECONDS = 15 * 60++ class Pcm(val mid: FloatArray, val side: FloatArray)++ fun decode(uri: Uri, cancelled: () -> Boolean): Pcm {+ val source = ByteSource(uri, cancelled)+ val extractor = MediaExtractor()+ try {+ extractor.setDataSource(source)+ val track = (0 until extractor.trackCount).firstOrNull {+ extractor.getTrackFormat(it).getString(MediaFormat.KEY_MIME)?.startsWith("audio/") == true+ } ?: throw IOException("no audio track")+ extractor.selectTrack(track)+ val format = extractor.getTrackFormat(track)+ val mime = format.getString(MediaFormat.KEY_MIME) ?: throw IOException("no audio codec")+ val durationUs = if (format.containsKey(MediaFormat.KEY_DURATION)) format.getLong(MediaFormat.KEY_DURATION) else -1L+ if (durationUs > MAX_SECONDS * 1_000_000L) throw IOException("song too long to analyse")+ return run(extractor, format, mime, durationUs, cancelled)+ } finally {+ runCatching { extractor.release() }+ source.close()+ }+ }++ private fun run(extractor: MediaExtractor, format: MediaFormat, mime: String, durationUs: Long, cancelled: () -> Boolean): Pcm {+ val out = Resampler(format.getInteger(MediaFormat.KEY_SAMPLE_RATE), durationUs)+ var channels = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT)+ var encoding = if (format.containsKey(MediaFormat.KEY_PCM_ENCODING)) format.getInteger(MediaFormat.KEY_PCM_ENCODING) else AudioFormat.ENCODING_PCM_16BIT+ var shorts = ShortArray(0); var floats = FloatArray(0)+ // Copied out in one go per buffer (a value at a time through the ByteBuffer is slow).+ fun take(buffer: ByteBuffer) {+ val float = encoding == AudioFormat.ENCODING_PCM_FLOAT+ if (!float && encoding != AudioFormat.ENCODING_PCM_16BIT) throw IOException("unsupported decoded PCM")+ val values = buffer.remaining() / (if (float) 4 else 2)+ val frames = values / channels+ if (float) { if (floats.size < values) floats = FloatArray(values); buffer.asFloatBuffer().get(floats, 0, values) }+ else { if (shorts.size < values) shorts = ShortArray(values); buffer.asShortBuffer().get(shorts, 0, values) }+ for (i in 0 until frames) {+ val at = i * channels+ val l = if (float) floats[at] else shorts[at] / 32768f+ val r = if (channels == 1) l else if (float) floats[at + 1] else shorts[at + 1] / 32768f+ out.push(if (l.isFinite()) l else 0f, if (r.isFinite()) r else 0f)+ }+ }+ if (mime == "audio/raw") {+ val packet = ByteBuffer.allocateDirect(1 shl 20).order(ByteOrder.LITTLE_ENDIAN)+ while (true) {+ if (cancelled()) throw InterruptedException("cancelled")+ packet.clear()+ val size = extractor.readSampleData(packet, 0)+ if (size < 0) break+ packet.position(0); packet.limit(size)+ take(packet)+ extractor.advance()+ }+ return out.result()+ }+ val codec = MediaCodec.createDecoderByType(mime)+ var started = false+ try {+ codec.configure(format, null, null, 0)+ codec.start(); started = true+ var inputDone = false+ var ended = false+ val info = MediaCodec.BufferInfo()+ // Every free input slot filled, every ready output taken, without waiting on either while+ // the other has work (a fixed 5-ms wait per packet made a song take 10 s).+ while (!ended) {+ if (cancelled()) throw InterruptedException("cancelled")+ while (!inputDone) {+ val slot = codec.dequeueInputBuffer(0)+ if (slot < 0) break+ val input = codec.getInputBuffer(slot) ?: throw IOException("no decoder input")+ val size = extractor.readSampleData(input, 0)+ if (size < 0) { codec.queueInputBuffer(slot, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM); inputDone = true }+ else { codec.queueInputBuffer(slot, 0, size, extractor.sampleTime, 0); extractor.advance() }+ }+ var slot = codec.dequeueOutputBuffer(info, 2000)+ while (slot >= 0 || slot == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED || slot == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {+ if (slot == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {+ val f = codec.outputFormat+ channels = f.getInteger(MediaFormat.KEY_CHANNEL_COUNT)+ encoding = if (f.containsKey(MediaFormat.KEY_PCM_ENCODING)) f.getInteger(MediaFormat.KEY_PCM_ENCODING) else AudioFormat.ENCODING_PCM_16BIT+ out.rate(f.getInteger(MediaFormat.KEY_SAMPLE_RATE))+ } else if (slot >= 0) {+ val buf = codec.getOutputBuffer(slot)?.order(ByteOrder.LITTLE_ENDIAN)+ if (buf != null && info.size > 0) { buf.position(info.offset); buf.limit(info.offset + info.size); take(buf) }+ codec.releaseOutputBuffer(slot, false)+ if (info.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM != 0) { ended = true; break }+ }+ slot = codec.dequeueOutputBuffer(info, 0)+ }+ }+ } finally {+ if (started) runCatching { codec.stop() }+ codec.release()+ }+ return out.result()+ }++ /**+ * Down to 22.05 kHz: a box average over the source samples of each output sample (enough+ * against aliasing for onsets and the mel bands up to 11 kHz), linear between them.+ */+ private class Resampler(private var inRate: Int, durationUs: Long) {+ private var mid = FloatArray(if (durationUs > 0) (durationUs * RATE / 1_000_000L).toInt() + RATE else RATE * 240)+ private var side = FloatArray(mid.size)+ private var n = 0+ private var pos = 0.0+ private var accM = 0.0; private var accS = 0.0; private var cnt = 0++ fun rate(r: Int) { if (r in 8000..192000) inRate = r }++ fun push(l: Float, r: Float) {+ accM += (l + r) * 0.5; accS += (l - r) * 0.5; cnt+++ pos += RATE.toDouble() / inRate+ while (pos >= 1.0) {+ pos -= 1.0+ if (n >= mid.size) {+ if (n >= MAX_SECONDS * RATE) throw IOException("song too long to analyse")+ mid = mid.copyOf(mid.size * 3 / 2); side = side.copyOf(mid.size)+ }+ mid[n] = (accM / cnt).toFloat(); side[n] = (accS / cnt).toFloat(); n+++ if (pos < 1.0) { accM = 0.0; accS = 0.0; cnt = 0 }+ }+ }++ fun result(): Pcm {+ if (n < RATE * 5) throw IOException("decoded audio too short")+ return Pcm(mid.copyOf(n), side.copyOf(n))+ }+ }++ /** Random-access adapter over the player's data source: each seek reopens it at the new position. */+ private class ByteSource(private val uri: Uri, private val cancelled: () -> Boolean) : MediaDataSource() {+ private val data = OfflineCrypto.Factory(Graph.app).createDataSource()+ private var open = false+ private var cursor = 0L+ private var size = -1L+ private var sizeKnown = false++ @Synchronized override fun readAt(position: Long, buffer: ByteArray, offset: Int, length: Int): Int {+ if (cancelled()) throw IOException("cancelled")+ if (length == 0) return 0+ if (position < 0) return -1+ if (!open || cursor != position) {+ if (open) runCatching { data.close() }+ val remaining = data.open(DataSpec.Builder().setUri(uri).setPosition(position).build())+ if (!sizeKnown) { size = if (remaining >= 0) position + remaining else -1; sizeKnown = true }+ open = true; cursor = position+ }+ val n = data.read(buffer, offset, length)+ if (n == C.RESULT_END_OF_INPUT) return -1+ if (n > 0) cursor += n+ return n+ }++ @Synchronized override fun getSize(): Long {+ if (!sizeKnown) {+ if (open) runCatching { data.close() }+ val total = data.open(DataSpec.Builder().setUri(uri).build())+ size = if (total >= 0) total else -1+ sizeKnown = true; open = true; cursor = 0+ }+ return size+ }++ @Synchronized override fun close() { runCatching { data.close() }; open = false }+ }+}diff --git a/base/app/src/main/java/com/music/spotui/cli/Cli.kt b/base/app/src/main/java/com/music/spotui/cli/Cli.ktindex a0cc1b23..025984e4 100644--- a/base/app/src/main/java/com/music/spotui/cli/Cli.kt+++ b/base/app/src/main/java/com/music/spotui/cli/Cli.kt@@ -38,6 +38,12 @@ sealed interface CliCommand {data object Next : CliCommanddata object Previous : CliCommanddata class Nav(val route: String) : CliCommand+ /** AutoMix on or off; null: as it is. */+ data class AutoMixSet(val on: Boolean?) : CliCommand+ data object AutoMixStatus : CliCommand+ /** Phone songs (a piece of each one's MediaStore folder + title) played as the local queue. */+ data class AutoMixFiles(val names: List<String>) : CliCommand+ data class AutoMixSeek(val ms: Long) : CliCommand}class CliError(message: String) : Exception(message)@@ -47,7 +53,7 @@ object CliParser {"status", "ext.list", "ext.install <zip path>", "ext.uninstall <id>", "ext.enable <id>", "ext.disable <id>","ext.setting <id> <key> <value|null>", "ext.press <id> <key>", "ext.call <id> <hook> [json]","ext.log <id> [lines]", "ext.storage <id>", "ext.why [n]", "play <spotify uri> [context uri]", "pause", "resume", "next", "previous",- "nav <route>",+ "nav <route>", "automix [on|off]", "automix.status", "automix.files <folder+title piece> <folder+title piece>...", "automix.seek <ms>",)private val ID = Regex("[A-Za-z0-9][A-Za-z0-9._-]{0,127}")@@ -95,6 +101,10 @@ object CliParser {val uri = arg(0, "spotify uri").also { if (!URI.matches(it)) throw CliError("play: not a spotify: uri") }CliCommand.Play(uri, args.getOrNull(1)?.takeIf { it.isNotBlank() }?.also { if (!URI.matches(it)) throw CliError("play: the context is not a spotify: uri") })}+ "automix" -> CliCommand.AutoMixSet(when (args.getOrNull(0)) { null -> null; "on" -> true; "off" -> false; else -> throw CliError("automix: on or off") }).also { most(1) }+ "automix.status" -> CliCommand.AutoMixStatus.also { most(0) }+ "automix.files" -> CliCommand.AutoMixFiles(args.filter { it.isNotBlank() }.also { if (it.size < 2) throw CliError("automix.files: at least two files") })+ "automix.seek" -> CliCommand.AutoMixSeek(arg(0, "ms").toLongOrNull()?.takeIf { it >= 0 } ?: throw CliError("automix.seek: <ms> is a number")).also { most(1) }"pause" -> CliCommand.Pause.also { most(0) }"resume" -> CliCommand.Resume.also { most(0) }"next" -> CliCommand.Next.also { most(0) }diff --git a/base/app/src/main/java/com/music/spotui/cli/CliReceiver.kt b/base/app/src/main/java/com/music/spotui/cli/CliReceiver.ktindex 5e947cf8..4faea3d6 100644--- a/base/app/src/main/java/com/music/spotui/cli/CliReceiver.kt+++ b/base/app/src/main/java/com/music/spotui/cli/CliReceiver.kt@@ -195,6 +195,24 @@ class CliReceiver : BroadcastReceiver() {}}is CliCommand.Play -> onMain { SongPlayer.playSong(c.uri, contextUri = c.context); JsonNull }+ is CliCommand.AutoMixSet -> onMain {+ c.on?.let { com.music.spotui.automix.AutoMix.set(it); com.music.spotui.di.LocalPlayer.autoMixChanged(it) }+ JsonPrimitive(com.music.spotui.di.LocalPlayer.autoMixState())+ }+ CliCommand.AutoMixStatus -> onMain {+ buildJsonObject {+ put("state", com.music.spotui.di.LocalPlayer.autoMixState())+ put("journal", JsonArray(com.music.spotui.automix.AutoMix.journalLines().takeLast(30).map(::JsonPrimitive)))+ }+ }+ is CliCommand.AutoMixFiles -> {+ val all = com.music.spotui.data.local.LocalMusic.tracks(ctx, refresh = true)+ // Each name: a piece of "folder + title" (e.g. "Spotiflaccc/24").+ val picked = c.names.map { n -> all.firstOrNull { (it.folder + it.title).contains(n, ignoreCase = true) }+ ?: throw CliError("automix.files: no phone song matches $n") }+ onMain { com.music.spotui.di.LocalPlayer.play(picked, 0, context = "AutoMix test"); JsonArray(picked.map { JsonPrimitive(it.title) }) }+ }+ is CliCommand.AutoMixSeek -> onMain { com.music.spotui.di.LocalPlayer.seekTo(c.ms); JsonNull }CliCommand.Pause -> onMain { SongPlayer.pause(by = "cli"); JsonNull }CliCommand.Resume -> onMain { SongPlayer.play(by = "cli"); JsonNull }CliCommand.Next -> onMain { SongPlayer.next(); JsonNull }diff --git a/base/app/src/main/java/com/music/spotui/data/local/LocalMusic.kt b/base/app/src/main/java/com/music/spotui/data/local/LocalMusic.ktindex ac36b945..1a40c8ed 100644--- a/base/app/src/main/java/com/music/spotui/data/local/LocalMusic.kt+++ b/base/app/src/main/java/com/music/spotui/data/local/LocalMusic.kt@@ -21,6 +21,8 @@ data class LocalTrack(val artUri: Uri,/** The file's folder on the phone, e.g. "Music/Album/" (MediaStore relative path). */val folder: String = "",+ /** MediaStore's track number (disc × 1000 + track), 0 unknown (streams and downloads have none). */+ val trackNo: Int = 0,)/** The device's music files, read through MediaStore (no file access beyond audio). */@@ -53,7 +55,7 @@ object LocalMusic {val projection = arrayOf(MediaStore.Audio.Media._ID, MediaStore.Audio.Media.TITLE, MediaStore.Audio.Media.ARTIST,MediaStore.Audio.Media.ALBUM, MediaStore.Audio.Media.DURATION, MediaStore.Audio.Media.ALBUM_ID,- MediaStore.Audio.Media.RELATIVE_PATH,+ MediaStore.Audio.Media.RELATIVE_PATH, MediaStore.Audio.Media.TRACK,)val out = ArrayList<LocalTrack>()// Android's index decides what is "music": a phone full of recorded or downloaded audio can@@ -77,6 +79,7 @@ object LocalMusic {val iDur = c.getColumnIndexOrThrow(MediaStore.Audio.Media.DURATION)val iAlbumId = c.getColumnIndexOrThrow(MediaStore.Audio.Media.ALBUM_ID)val iPath = c.getColumnIndex(MediaStore.Audio.Media.RELATIVE_PATH)+ val iTrack = c.getColumnIndex(MediaStore.Audio.Media.TRACK)while (c.moveToNext()) {val id = c.getLong(iId)val artist = c.getString(iArtist)?.takeIf { it != MediaStore.UNKNOWN_STRING }.orEmpty()@@ -89,6 +92,7 @@ object LocalMusic {durationMs = c.getLong(iDur),artUri = ContentUris.withAppendedId(Uri.parse("content://media/external/audio/albumart"), c.getLong(iAlbumId)),folder = if (iPath >= 0) c.getString(iPath).orEmpty() else "",+ trackNo = if (iTrack >= 0 && !c.isNull(iTrack)) c.getInt(iTrack).coerceAtLeast(0) else 0,)}}diff --git a/base/app/src/main/java/com/music/spotui/di/LocalPlayer.kt b/base/app/src/main/java/com/music/spotui/di/LocalPlayer.ktindex 858fe5f8..b470c254 100644--- a/base/app/src/main/java/com/music/spotui/di/LocalPlayer.kt+++ b/base/app/src/main/java/com/music/spotui/di/LocalPlayer.kt@@ -61,7 +61,7 @@ object LocalPlayer {}@androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)- private fun build(main: Boolean): ExoPlayer = ExoPlayer.Builder(Graph.app, Renderers)+ private fun build(main: Boolean, deck: Boolean = false): ExoPlayer = ExoPlayer.Builder(Graph.app, if (deck) DeckRenderers else Renderers)// The fading-out copy of a crossfade owns neither the audio focus nor the noisy-audio pause:// the main player has them and the fade follows its play state..setAudioAttributes(audioAttributes, main && !mirrorActive)@@ -78,7 +78,7 @@ object LocalPlayer {* EqualizerProcessor), ahead of the silence skipping and the speed the sink adds after it.*/@androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)- private object Renderers : androidx.media3.exoplayer.DefaultRenderersFactory(Graph.app) {+ private open class AudioRenderers : androidx.media3.exoplayer.DefaultRenderersFactory(Graph.app) {override fun buildAudioSink(context: android.content.Context, enableFloatOutput: Boolean, enableAudioTrackPlaybackParams: Boolean): androidx.media3.exoplayer.audio.AudioSink =androidx.media3.exoplayer.audio.DefaultAudioSink.Builder(context).setEnableFloatOutput(enableFloatOutput)@@ -86,6 +86,17 @@ object LocalPlayer {.setAudioProcessors(arrayOf(EqualizerProcessor())).build()}+ private object Renderers : AudioRenderers()++ /**+ * An AutoMix deck's renderers: its speed is the AudioTrack's own (set while it plays, at once and+ * without a break), not the sink's Sonic, which drains and starts over at each change (a click,+ * heard on some phones) and is heard only after the whole buffer.+ */+ @androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)+ private object DeckRenderers : AudioRenderers() {+ init { setEnableAudioTrackPlaybackParams(true) }+ }private fun tune(p: ExoPlayer) {val pref = com.music.spotui.data.preferences.EqualizerPref@@ -96,6 +107,8 @@ object LocalPlayer {/** The speed or the silence skipping changed in the settings: the song playing (and a fade) takes it now. */fun applyPlayback() {exo?.let(::tune); outgoing?.let(::tune)+ // A deck in a mix takes its own speed again at the next step.+ mix?.lastSpeed = 0flog("speed ${rate()}×, skip silences ${if (com.music.spotui.data.preferences.EqualizerPref.skipSilence) "on" else "off"}")}@@ -104,100 +117,109 @@ object LocalPlayer {@androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)private fun player(): ExoPlayer = exo ?: build(true).also { p ->- p.addListener(object : Player.Listener {- override fun onEvents(player: Player, events: Player.Events) = publish()-- // Each change of what the player does, one line each, so a song that never starts says why.- override fun onPlaybackStateChanged(state: Int) {- log("state ${stateName(state)} at ${p.currentPosition} ms, item ${p.currentMediaItemIndex + 1}/${p.mediaItemCount}")- // Casting: this player keeps the queue only, stopped; the song plays on the device.- if (CastPlayback.active && state != Player.STATE_IDLE) { p.stop(); return }- if (state == Player.STATE_READY) checkLength(p)- if (state == Player.STATE_ENDED && !playAgain(p)) atEnd(p)- }- override fun onPlayWhenReadyChanged(playWhenReady: Boolean, reason: Int) {- log("playWhenReady $playWhenReady (${PLAY_WHEN_READY_REASONS.getOrElse(reason - 1) { "reason $reason" }})")- // A song on repeat paused at its end (see LocalRepeat): it starts again at 0:00.- if (reason == Player.PLAY_WHEN_READY_CHANGE_REASON_END_OF_MEDIA_ITEM && playAgain(p)) return- // A pause (or a lost focus) during a fade stops the fading-out song too.- outgoing?.playWhenReady = playWhenReady- if (playWhenReady && CastPlayback.active) { p.pause(); CastPlayback.play() }- }- // Moves the player makes by itself (not a seek, not a song's end): logged, so a song- // jumping elsewhere shows where it went and why.- override fun onPositionDiscontinuity(old: Player.PositionInfo, new: Player.PositionInfo, reason: Int) {- if (reason == Player.DISCONTINUITY_REASON_SEEK || reason == Player.DISCONTINUITY_REASON_AUTO_TRANSITION) return- log("moved by the player from ${old.positionMs} to ${new.positionMs} ms (reason $reason), item ${new.mediaItemIndex + 1}")- }- // The queue grew or shrank: a queue of one song repeated loops it as repeat one does.- override fun onTimelineChanged(timeline: androidx.media3.common.Timeline, reason: Int) = applyRepeat(p)- override fun onIsPlayingChanged(isPlaying: Boolean) {- log("isPlaying $isPlaying at ${p.currentPosition} ms")- if (isPlaying) errors.played()- if (isPlaying) tick.run()- }- override fun onPlaybackSuppressionReasonChanged(reason: Int) =- log("suppressed: ${if (reason == Player.PLAYBACK_SUPPRESSION_REASON_NONE) "no" else "reason $reason"}")-- // The next song an extension streams is asked for as soon as this one starts: it then- // starts at once instead of waiting for the extension when this one ends.- // The last one starting asks what goes on after it (see continueIfLast).- override fun onMediaItemTransition(mediaItem: androidx.media3.common.MediaItem?, reason: Int) {- // Anything but the transition the fade itself makes (a skip, a tap in the queue) ends the fade.- if (!startingFade) endFade()- // The song kept from fading out is left behind: the next one fades as usual.- if (p.currentMediaItemIndex != noFadeFor) noFadeFor = -1- // A song to play again that the player went past by itself: put back, never followed.- if (reason == Player.MEDIA_ITEM_TRANSITION_REASON_AUTO && wentPastRepeat(p, p.previousMediaItemIndex)) return- if (reason == Player.MEDIA_ITEM_TRANSITION_REASON_AUTO || reason == Player.MEDIA_ITEM_TRANSITION_REASON_REPEAT) startsAtZero(p)- if (skipHidden(p, reason)) return- prefetchNext(p); continueIfLast(p)- }-- // A song whose audio an extension did not give or that would not play (audio-source- // hook) is never skipped silently: asked again once if the stream itself failed,- // then Spotify plays it, where it was, and goes on; a queue that keeps failing stops.- override fun onPlayerError(error: androidx.media3.common.PlaybackException) {- log("error ${error.errorCodeName}: ${generic(error.cause?.toString() ?: error.message.orEmpty())}")- val item = p.currentMediaItem?.localConfiguration?.uri ?: return- if (item.scheme == com.music.spotui.extensions.ExtensionAudio.SCHEME) com.music.spotui.extensions.Extensions.noteAttempt(item.lastPathSegment) { it.copy(error = "${error.errorCodeName}: ${error.cause ?: error.message.orEmpty()}") }- // A file that is gone (a download deleted while the queue played): never left stopped on it.- if (item.scheme != com.music.spotui.extensions.ExtensionAudio.SCHEME && DeletedDownloads.isMissingFile(error.errorCode, error.cause)) return fileGone(p, item)- if (item.scheme != com.music.spotui.extensions.ExtensionAudio.SCHEME) return- val t = current() ?: return- val noAudio = com.music.spotui.extensions.AudioRoute.isNoAudio(error)- // No extension had it (the data source already asked twice): Spotify, no new ask.- val again = errors.alreadyRetried(item.toString())- if (!noAudio && !again) {- log("${t.title}: the stream failed, the extensions are asked again once")- val at = p.currentPosition- prefetch.launch {- runCatching { com.music.spotui.extensions.ExtensionAudio.resolve(item, again = true) }- handler.post { if (exo === p && p.currentMediaItem?.localConfiguration?.uri == item) { p.prepare(); p.seekTo(at); p.play() } }- }- return- }- if (SongPlayer.playUnavailableOnSpotify(t, p.currentPosition)) return- when (errors.failed(item.toString(), p.hasNextMediaItem())) {- StreamErrors.Step.SKIP -> {- log("skipped ${t.title}: no extension had it and Spotify cannot play it now")- android.widget.Toast.makeText(Graph.app, Graph.app.getString(app.myspoti.engine.R.string.song_unavailable_skipped, t.title), android.widget.Toast.LENGTH_SHORT).show()- p.seekToNextMediaItem()- p.prepare()- p.play()- }- else -> {- log("stopped on ${t.title}: it cannot be played, and the songs after it failed too or none follows")- android.widget.Toast.makeText(Graph.app, Graph.app.getString(app.myspoti.engine.R.string.song_unavailable, t.title), android.widget.Toast.LENGTH_LONG).show()- }- }- }- })+ mainListener = listenerFor(p).also(p::addListener)exo = p+ deckEq = if (com.music.spotui.automix.AutoMix.on) com.music.spotui.automix.DeckEq.attach(p.audioSessionId) else null// A new local session takes the repeat the player shows (it started off).setRepeatMode(SongPlayer.repeatMode.value)}+ /** The main player's listener: it moves to the deck that takes over at an AutoMix (see [takeOver]). */+ private var mainListener: Player.Listener? = null++ private fun listenerFor(p: ExoPlayer): Player.Listener = object : Player.Listener {+ override fun onEvents(player: Player, events: Player.Events) = publish()++ // Each change of what the player does, one line each, so a song that never starts says why.+ override fun onPlaybackStateChanged(state: Int) {+ log("state ${stateName(state)} at ${p.currentPosition} ms, item ${p.currentMediaItemIndex + 1}/${p.mediaItemCount}")+ // Casting: this player keeps the queue only, stopped; the song plays on the device.+ if (CastPlayback.active && state != Player.STATE_IDLE) { p.stop(); return }+ if (state == Player.STATE_READY) checkLength(p)+ if (state == Player.STATE_ENDED && !playAgain(p)) atEnd(p)+ }+ override fun onPlayWhenReadyChanged(playWhenReady: Boolean, reason: Int) {+ log("playWhenReady $playWhenReady (${PLAY_WHEN_READY_REASONS.getOrElse(reason - 1) { "reason $reason" }})")+ // A song on repeat paused at its end (see LocalRepeat): it starts again at 0:00.+ if (reason == Player.PLAY_WHEN_READY_CHANGE_REASON_END_OF_MEDIA_ITEM && playAgain(p)) return+ // A pause (or a lost focus) during a fade stops the fading-out song too.+ outgoing?.playWhenReady = playWhenReady+ if (!playWhenReady) mixPaused()+ if (playWhenReady && CastPlayback.active) { p.pause(); CastPlayback.play() }+ }+ // Moves the player makes by itself (not a seek, not a song's end): logged, so a song+ // jumping elsewhere shows where it went and why.+ override fun onPositionDiscontinuity(old: Player.PositionInfo, new: Player.PositionInfo, reason: Int) {+ if (reason == Player.DISCONTINUITY_REASON_SEEK) { mixMoved(); return }+ if (reason == Player.DISCONTINUITY_REASON_AUTO_TRANSITION) return+ log("moved by the player from ${old.positionMs} to ${new.positionMs} ms (reason $reason), item ${new.mediaItemIndex + 1}")+ }+ // The queue grew or shrank: a queue of one song repeated loops it as repeat one does.+ override fun onTimelineChanged(timeline: androidx.media3.common.Timeline, reason: Int) = applyRepeat(p)+ override fun onIsPlayingChanged(isPlaying: Boolean) {+ log("isPlaying $isPlaying at ${p.currentPosition} ms")+ if (isPlaying) errors.played()+ if (isPlaying) tick.run()+ }+ override fun onPlaybackSuppressionReasonChanged(reason: Int) =+ log("suppressed: ${if (reason == Player.PLAYBACK_SUPPRESSION_REASON_NONE) "no" else "reason $reason"}")++ // The next song an extension streams is asked for as soon as this one starts: it then+ // starts at once instead of waiting for the extension when this one ends.+ // The last one starting asks what goes on after it (see continueIfLast).+ override fun onMediaItemTransition(mediaItem: androidx.media3.common.MediaItem?, reason: Int) {+ // Anything but the transition the fade itself makes (a skip, a tap in the queue) ends the fade.+ if (!startingFade) endFade()+ planned = null; dropped = null+ // The song kept from fading out is left behind: the next one fades as usual.+ if (p.currentMediaItemIndex != noFadeFor) noFadeFor = -1+ // A song to play again that the player went past by itself: put back, never followed.+ if (reason == Player.MEDIA_ITEM_TRANSITION_REASON_AUTO && wentPastRepeat(p, p.previousMediaItemIndex)) return+ if (reason == Player.MEDIA_ITEM_TRANSITION_REASON_AUTO || reason == Player.MEDIA_ITEM_TRANSITION_REASON_REPEAT) startsAtZero(p)+ if (skipHidden(p, reason)) return+ prefetchNext(p); continueIfLast(p)+ }++ // A song whose audio an extension did not give or that would not play (audio-source+ // hook) is never skipped silently: asked again once if the stream itself failed,+ // then Spotify plays it, where it was, and goes on; a queue that keeps failing stops.+ override fun onPlayerError(error: androidx.media3.common.PlaybackException) {+ log("error ${error.errorCodeName}: ${generic(error.cause?.toString() ?: error.message.orEmpty())}")+ val item = p.currentMediaItem?.localConfiguration?.uri ?: return+ if (item.scheme == com.music.spotui.extensions.ExtensionAudio.SCHEME) com.music.spotui.extensions.Extensions.noteAttempt(item.lastPathSegment) { it.copy(error = "${error.errorCodeName}: ${error.cause ?: error.message.orEmpty()}") }+ // A file that is gone (a download deleted while the queue played): never left stopped on it.+ if (item.scheme != com.music.spotui.extensions.ExtensionAudio.SCHEME && DeletedDownloads.isMissingFile(error.errorCode, error.cause)) return fileGone(p, item)+ if (item.scheme != com.music.spotui.extensions.ExtensionAudio.SCHEME) return+ val t = current() ?: return+ val noAudio = com.music.spotui.extensions.AudioRoute.isNoAudio(error)+ // No extension had it (the data source already asked twice): Spotify, no new ask.+ val again = errors.alreadyRetried(item.toString())+ if (!noAudio && !again) {+ log("${t.title}: the stream failed, the extensions are asked again once")+ val at = p.currentPosition+ prefetch.launch {+ runCatching { com.music.spotui.extensions.ExtensionAudio.resolve(item, again = true) }+ handler.post { if (exo === p && p.currentMediaItem?.localConfiguration?.uri == item) { p.prepare(); p.seekTo(at); p.play() } }+ }+ return+ }+ if (SongPlayer.playUnavailableOnSpotify(t, p.currentPosition)) return+ when (errors.failed(item.toString(), p.hasNextMediaItem())) {+ StreamErrors.Step.SKIP -> {+ log("skipped ${t.title}: no extension had it and Spotify cannot play it now")+ android.widget.Toast.makeText(Graph.app, Graph.app.getString(app.myspoti.engine.R.string.song_unavailable_skipped, t.title), android.widget.Toast.LENGTH_SHORT).show()+ p.seekToNextMediaItem()+ p.prepare()+ p.play()+ }+ else -> {+ log("stopped on ${t.title}: it cannot be played, and the songs after it failed too or none follows")+ android.widget.Toast.makeText(Graph.app, Graph.app.getString(app.myspoti.engine.R.string.song_unavailable, t.title), android.widget.Toast.LENGTH_LONG).show()+ }+ }+ }+ }+/*** The player's load errors: a song no extension gave usable audio for fails at once (each* retry would ask the extensions again, five asks in two seconds were seen); others as usual.@@ -421,6 +443,417 @@ object LocalPlayer {}+ // ── AutoMix (see com.music.spotui.automix.AutoMix) ──+ // Two decks, no song ever handed from one player to another: the next song is prepared on a+ // second player (the deck) at its entry point, started muted a little before the mix and brought+ // in step with the song playing by nudging its speed (a phase-locked loop on both positions).+ // When the mix starts the deck becomes the main player (queue, listener, audio focus: see+ // [takeOver]) and the old one plays out the mix as the crossfade's outgoing player, then stops.+ // Gains are the players' volumes, the bass swap and filters the decks' DynamicsProcessing: both+ // act on what is heard now. The deck's speed goes back to the song's own after the mix.++ enum class MixState { PREPARING, READY, PREROLL, MIXING, RESTORING }++ private class MixRun(+ var plan: app.myspoti.automix.MixPlan,+ val toIndex: Int,+ val toId: String,+ val deck: ExoPlayer,+ val eq: com.music.spotui.automix.DeckEq?,+ /** Where the deck starts (B's time, ms): the entry point less the muted run-up. */+ val startMs: Long,+ /** A mix for Next: started from A's next downbeat once the deck is ready (see [mixOnSkip]). */+ val skip: Boolean = false,+ ) {+ val createdAt = android.os.SystemClock.uptimeMillis()+ /** The run-up, in the outgoing song's ms: only as long as B has audio before its entry point. */+ val runUpA get() = (plan.entryB * 1000 - startMs) / plan.speedB+ var firstLocked = false+ var moves = 0+ var startedAt = 0L+ var lastTickAt = 0L+ /** The deck's position moved by the speed nudges since its start (ms of B). */+ var corrected = 0.0+ var state = MixState.PREPARING+ var a: ExoPlayer? = null+ var eqA: com.music.spotui.automix.DeckEq? = null+ var lastSpeed = 0f+ val errors = ArrayDeque<Double>()+ var locked = false+ var worstMs = 0.0+ var loggedAt = 0L+ /** Progress of the mix from the outgoing song's position. */+ fun progress(): Double = a?.let { ((it.currentPosition / 1000.0) - plan.exitA) / plan.length } ?: 1.0+ }++ /** The plan for the song playing → the next one, once both are analysed. */+ private class Planned(val fromIndex: Int, val toId: String, val plan: app.myspoti.automix.MixPlan)+ @Volatile private var planned: Planned? = null+ private var mix: MixRun? = null+ /** The main player's EQ (a deck's once it took over). */+ private var deckEq: com.music.spotui.automix.DeckEq? = null+ private var asking: Pair<Int, String>? = null+ /** The pair whose deck was dropped: no new deck for it. */+ private var dropped: Pair<Int, String>? = null++ private const val PREPARE_LEAD_MS = 15_000L+ /**+ * How long a deck takes from play to sound (ms), learned at each start (about a second in, once+ * the AudioTrack's timestamps hold) and kept: the deck is started that much earlier.+ */+ private var startLead = Graph.app.getSharedPreferences("myspoti_prefs", 0).getFloat("automix_start_lead", 120f).toDouble()+ private const val RUN_UP_MS = 2_500L++ /** One AutoMix step (every 200 ms, every 20 ms around a mix). False: no more ticks. */+ private fun mixTick(p: ExoPlayer): Boolean {+ val m = mix+ if (m != null && m.state >= MixState.MIXING) return mixing(m)+ if (!p.isPlaying) return false+ val next = p.nextMediaItemIndex+ val canMix = next != C.INDEX_UNSET && !LocalRepeat.loopsSong(repeat, p.mediaItemCount) && p.playWhenReady+ if (!canMix) { if (m != null) cancelMix("no next song"); return true }+ val nextId = p.getMediaItemAt(next).mediaId+ // The next song changed (queue edited): the deck prepared for another one goes.+ if (m != null && (m.toIndex != next || m.toId != nextId)) cancelMix("the next song changed")+ val pos = p.currentPosition+ val run = mix+ if (run == null) {+ val plan = planFor(p, next, nextId) ?: return true+ val exitMs = (plan.exitA * 1000).toLong()+ if (pos >= exitMs - RUN_UP_MS - PREPARE_LEAD_MS && pos < exitMs - RUN_UP_MS / 2 && dropped != next to nextId) prepareDeck(p, next, nextId, plan)+ return true+ }+ val exitMs = (run.plan.exitA * 1000).toLong()+ val startAt = exitMs - run.runUpA - startLead+ when (run.state) {+ MixState.PREPARING -> {+ // Too late for the deck: the song ends as usual (a skip that waited too long: the plain skip).+ if (run.skip && android.os.SystemClock.uptimeMillis() - run.createdAt > SKIP_WAIT_MS) {+ cancelMix("not ready in time for the skip"); p.seekToNextMediaItem()+ } else if (!run.skip && pos > startAt) cancelMix("the next song was not ready in time")+ }+ MixState.READY -> {+ // A skip's deck ready later than its plan: the mix moves to A's next downbeat.+ if (run.skip && pos > startAt + 20) {+ // Never more than twice: a Next always leaves the song within a few seconds.+ if (++run.moves > 2) { cancelMix("the skip could not start"); p.seekToNextMediaItem(); return true }+ run.plan = skipPlan(p, next, pos)+ com.music.spotui.automix.AutoMix.journal("skip moved: ${run.plan}")+ } else if (pos >= startAt) startRunUp(run)+ }+ MixState.PREROLL -> {+ lock(run, p, preroll = true)+ if (pos >= exitMs) takeOver(run, p)+ }+ else -> Unit+ }+ return true+ }++ /** The plan for this pair: the analysed one, else (once the end is near) a fade on the songs' lengths. */+ private fun planFor(p: ExoPlayer, next: Int, nextId: String): app.myspoti.automix.MixPlan? {+ val cur = p.currentMediaItemIndex+ planned?.takeIf { it.fromIndex == cur && it.toId == nextId }?.let { return it.plan }+ val a = tracks.getOrNull(cur) ?: return null+ val b = tracks.getOrNull(next) ?: return null+ if (asking != cur to nextId) {+ asking = cur to nextId+ // The song after the next one too: a skip then finds both of its songs analysed.+ com.music.spotui.automix.AutoMix.request(listOfNotNull(a.uri, b.uri, tracks.getOrNull(next + 1)?.uri)) {}+ if (followsOnAlbum(a, b)) com.music.spotui.automix.AutoMix.journal("${a.title} → ${b.title}: next on the album, no mix")+ }+ // B right after A on their album: they run into each other as the album does (a skip still mixes).+ if (followsOnAlbum(a, b)) return null+ val ra = com.music.spotui.automix.AutoMix.analysis(a.uri); val rb = com.music.spotui.automix.AutoMix.analysis(b.uri)+ if (ra != null && rb != null) {+ val plan = com.music.spotui.automix.AutoMix.planFor(ra, rb)+ planned = Planned(cur, nextId, plan)+ com.music.spotui.automix.AutoMix.journal("plan ${a.title} → ${b.title}: $plan")+ return plan+ }+ // Not analysed yet with the end near: an equal-power fade of 6 s on the lengths.+ val dur = p.duration.takeIf { it > 0 } ?: return null+ if (dur - p.currentPosition > PREPARE_LEAD_MS + RUN_UP_MS + 1000) return null+ val len = minOf(6.0, dur / 4000.0)+ val plan = app.myspoti.automix.MixPlan(app.myspoti.automix.MixPlan.Style.FADE, dur / 1000.0 - len, 0.0, len, 1.0, 0, 0.5, 0.1, 1.0, 0.0, "fade: not analysed in time")+ planned = Planned(cur, nextId, plan)+ com.music.spotui.automix.AutoMix.journal("plan ${a.title} → ${b.title}: $plan")+ return plan+ }++ /**+ * [b] is the song after [a] on their album: by the files' track numbers (the next track, or the+ * first of the next disc); without numbers (streams, downloads), when the album itself is the+ * list and [b] follows [a] in its own order (the shuffle aside). Two songs of one album that do+ * not follow each other there are mixed as any others.+ */+ private fun followsOnAlbum(a: LocalTrack, b: LocalTrack): Boolean {+ val x = a.album.trim(); val y = b.album.trim()+ if (x.isEmpty() || x.equals("<unknown>", true) || !x.equals(y, ignoreCase = true)) return false+ if (a.trackNo > 0 && b.trackNo > 0) {+ val da = a.trackNo / 1000; val ta = a.trackNo % 1000; val db = b.trackNo / 1000; val tb = b.trackNo % 1000+ return (db == da && tb == ta + 1) || (db == da + 1 && tb == 1)+ }+ if (!listUri.startsWith("spotify:album:")) return false+ val order = unshuffled ?: tracks+ val i = order.indexOfFirst { it === a }+ return i >= 0 && order.getOrNull(i + 1) === b+ }++ private const val SKIP_WAIT_MS = 2500L+ /** How long a skip waits from the deck's readiness to the mix (the run-up). */+ private const val SKIP_RUN_UP_MS = 600L++ /** The skip mix from A's time [posMs] (ms), on what is analysed of both songs. */+ private fun skipPlan(p: ExoPlayer, next: Int, posMs: Long): app.myspoti.automix.MixPlan {+ val a = tracks.getOrNull(p.currentMediaItemIndex)?.let { com.music.spotui.automix.AutoMix.analysis(it.uri) }+ val b = tracks.getOrNull(next)?.let { com.music.spotui.automix.AutoMix.analysis(it.uri) }+ // A skip may stretch more than a natural mix (12 %): it is short, and B is back to its tempo soon after.+ return app.myspoti.automix.MixPlanner.planSkip(a, b, posMs / 1000.0, (SKIP_RUN_UP_MS + startLead) / 1000.0,+ app.myspoti.automix.MixPlanner.Settings(maxStretch = 0.12))+ }++ /**+ * Next with AutoMix on: the next song is prepared on the deck and mixed in from A's next+ * downbeat (a short beat-matched blend, a cut on the beat, or a quick fade). False: a plain skip+ * (nothing to mix into, casting, a mix already playing). A second Next skips at once.+ */+ private fun mixOnSkip(): Boolean {+ val p = exo ?: return false+ if (!com.music.spotui.automix.AutoMix.on || CastPlayback.active || !p.isPlaying || outgoing != null) return false+ mix?.let { if (it.skip || it.state >= MixState.PREROLL) return false }+ val next = p.nextMediaItemIndex+ if (next == C.INDEX_UNSET || LocalRepeat.loopsSong(repeat, p.mediaItemCount)) return false+ cancelMix("skip", again = true)+ val plan = skipPlan(p, next, p.currentPosition)+ com.music.spotui.automix.AutoMix.journal("skip: $plan")+ prepareDeck(p, next, p.getMediaItemAt(next).mediaId, plan, skip = true)+ handler.removeCallbacks(tick); handler.post(tick)+ return true+ }++ @androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)+ private fun prepareDeck(p: ExoPlayer, next: Int, nextId: String, plan: app.myspoti.automix.MixPlan, skip: Boolean = false) {+ val t = tracks.getOrNull(next) ?: return+ val deck = build(false, deck = true)+ deck.volume = 0f+ deck.skipSilenceEnabled = false+ deck.playbackParameters = androidx.media3.common.PlaybackParameters((plan.speedB * rate()).toFloat())+ val runUpB = minOf(plan.entryB * 1000, (if (skip) SKIP_RUN_UP_MS else RUN_UP_MS) * plan.speedB)+ val start = (plan.entryB * 1000 - runUpB).toLong().coerceAtLeast(0L)+ val run = MixRun(plan, next, nextId, deck, com.music.spotui.automix.DeckEq.attach(deck.audioSessionId), start, skip)+ run.eq?.set(app.myspoti.automix.MixPlan.KILL, 0.0)+ run.lastSpeed = deck.playbackParameters.speed+ run.a = p; run.eqA = deckEq+ mix = run+ deck.addListener(object : Player.Listener {+ override fun onPlaybackStateChanged(state: Int) {+ if (state == Player.STATE_READY && run.state == MixState.PREPARING) {+ run.state = MixState.READY+ com.music.spotui.automix.AutoMix.journal("deck ready on ${t.title} at $start ms")+ }+ }+ override fun onPlayerError(error: androidx.media3.common.PlaybackException) {+ if (mix === run && run.state < MixState.MIXING) cancelMix("deck error ${error.errorCodeName}")+ }+ })+ deck.setMediaItem(item(t), start)+ deck.prepare()+ com.music.spotui.automix.AutoMix.journal("deck preparing ${t.title} from $start ms")+ }++ private fun startRunUp(run: MixRun) {+ run.state = MixState.PREROLL+ run.errors.clear(); run.locked = false+ run.deck.play()+ run.startedAt = android.os.SystemClock.uptimeMillis()+ com.music.spotui.automix.AutoMix.journal(String.format("run-up started (%.0f ms, start lead %.0f ms)", run.runUpA, startLead))+ }++ /**+ * Keeps the deck on its place in the mix: its position against where the outgoing song's says it+ * should be, corrected by a speed nudge (a seek when far off during the muted run-up).+ */+ private fun lock(run: MixRun, a: ExoPlayer, preroll: Boolean, quiet: Boolean = false) {+ val plan = run.plan+ if (!plan.synced && !preroll) return+ // A cut between tempos too far apart: B's first downbeat on A's is all there is to hold; after+ // it the two run apart anyway (A is gone within two beats), so B is left at its own pace.+ if (!preroll && plan.style == app.myspoti.automix.MixPlan.Style.CUT && plan.restoreSec == 0.0) return+ val d = run.deck+ if (!d.isPlaying) return+ val aPos = a.currentPosition; val bPos = d.currentPosition+ val target = plan.entryB * 1000 + (aPos - plan.exitA * 1000) * plan.speedB+ // Started well before its time (a pause and play moved the outgoing song back): wait for it again.+ // (Started early by the start delay on purpose: up to that much before its start is expected.)+ if (preroll && target < run.startMs - startLead * plan.speedB - 300) { d.pause(); d.seekTo(run.startMs); run.state = MixState.READY; run.errors.clear(); return }+ val err = bPos - target+ run.errors.addLast(err); while (run.errors.size > 3) run.errors.removeFirst()+ val e = run.errors.sorted()[run.errors.size / 2]+ val base = plan.speedB * rate()+ // A second after the start, the deck's distance says how late it sounded: the next starts that much earlier.+ val sinceStart = android.os.SystemClock.uptimeMillis() - run.startedAt+ if (!run.firstLocked && run.startedAt > 0 && sinceStart in 900..1600 && run.errors.size == 3) {+ run.firstLocked = true+ // The correction already made by the speed since the start counts too.+ startLead = (startLead - 0.6 * (e - run.corrected) / plan.speedB).coerceIn(0.0, 500.0)+ Graph.app.getSharedPreferences("myspoti_prefs", 0).edit().putFloat("automix_start_lead", startLead.toFloat()).apply()+ }+ if (preroll && kotlin.math.abs(e) > 400 && run.errors.size == 3) {+ d.seekTo((target + 120).toLong().coerceAtLeast(0L)); run.errors.clear(); return+ }+ run.locked = kotlin.math.abs(e) < 15+ if (!preroll) run.worstMs = maxOf(run.worstMs, kotlin.math.abs(e))+ // Muted, or still low in the mix: fast and strong; heard: gentle.+ val fast = preroll || quiet+ val dead = if (preroll) 2.0 else 6.0+ val horizon = if (fast) 300.0 else 1500.0+ val limit = if (preroll) 0.08 else if (quiet) 0.05 else 0.02+ val nudge = if (kotlin.math.abs(e) <= dead) 0.0 else (-e / horizon).coerceIn(-limit, limit)+ val speed = (base * (1 + nudge)).toFloat()+ // What the nudges moved the deck by so far (ms of B), for the start delay learnt above.+ if (run.lastTickAt > 0 && run.lastSpeed > 0) run.corrected += (run.lastSpeed / base - 1) * (android.os.SystemClock.uptimeMillis() - run.lastTickAt) * plan.speedB+ run.lastTickAt = android.os.SystemClock.uptimeMillis()+ if (kotlin.math.abs(speed - run.lastSpeed) > 0.0005f) {+ d.playbackParameters = androidx.media3.common.PlaybackParameters(speed)+ run.lastSpeed = speed+ }+ // Once a second in the journal: how far the deck is from its place.+ val now = android.os.SystemClock.uptimeMillis()+ if (now - run.loggedAt >= 1000) {+ run.loggedAt = now+ com.music.spotui.automix.AutoMix.journal(String.format("%s: deck %+.0f ms, speed %.4f", if (preroll) "run-up" else "mix", e, speed))+ }+ }++ /**+ * The mix starts: the deck becomes the main player (the queue around its song, the listener, the+ * audio focus), the old main player plays the mix out as the outgoing one.+ */+ @androidx.annotation.OptIn(androidx.media3.common.util.UnstableApi::class)+ private fun takeOver(run: MixRun, a: ExoPlayer) {+ val d = run.deck+ if (a.getMediaItemAt(run.toIndex).mediaId != run.toId) return cancelMix("the queue moved")+ val err = run.errors.lastOrNull() ?: 0.0+ val items = (0 until a.mediaItemCount).map(a::getMediaItemAt)+ if (run.toIndex > 0) d.addMediaItems(0, items.subList(0, run.toIndex))+ if (run.toIndex + 1 < items.size) d.addMediaItems(items.subList(run.toIndex + 1, items.size))+ d.repeatMode = a.repeatMode+ d.shuffleModeEnabled = false+ mainListener?.let(a::removeListener)+ // The focus moves with the main role: given up by the old player first, so it does not pause.+ a.setAudioAttributes(audioAttributes, false); a.setHandleAudioBecomingNoisy(false)+ d.setAudioAttributes(audioAttributes, !mirrorActive); d.setHandleAudioBecomingNoisy(true)+ mainListener = listenerFor(d).also(d::addListener)+ exo = d+ outgoing = a+ run.eqA = deckEq+ deckEq = run.eq+ run.state = MixState.MIXING+ SongPlayer.onLocalFade(true)+ com.music.spotui.automix.AutoMix.journal(String.format("mix started (%s), deck %+.0f ms from its place%s", run.plan.style, err, if (run.locked) ", locked" else ""))+ // What a transition of the main player does (not the move to 0:00: the deck starts where the plan says).+ if (skipHidden(d, Player.MEDIA_ITEM_TRANSITION_REASON_AUTO)) return+ prefetchNext(d); continueIfLast(d)+ publish()+ }++ /** During and after the mix: gains, EQ, the lock, then the deck's speed and gain back to its own. */+ private fun mixing(run: MixRun): Boolean {+ val d = run.deck+ if (exo !== d) { endMix(); return false }+ val plan = run.plan+ val a = run.a+ if (run.state == MixState.MIXING && a != null) {+ val x = run.progress()+ if (x >= 1.0) {+ // A is gone.+ com.music.spotui.automix.AutoMix.journal(String.format("mix done, largest gap %.0f ms", run.worstMs))+ handler.removeCallbacks(tick)+ outgoing?.let { it.stop(); it.release() }+ outgoing = null+ SongPlayer.onLocalFade(false)+ run.eqA?.release(); run.eqA = null; run.a = null+ run.state = MixState.RESTORING+ } else {+ val lv = plan.levels(x)+ a.volume = lv.gainA.toFloat(); d.volume = lv.gainB.toFloat()+ run.eqA?.set(lv.lowA, lv.highA); run.eq?.set(lv.lowB, lv.highB)+ // B low in the mix, or a cut (B whole at once, A out within two beats): corrected fast.+ if (a.isPlaying) lock(run, a, preroll = false, quiet = lv.gainB < 0.35 || plan.style == app.myspoti.automix.MixPlan.Style.CUT)+ // Paused: the ticks start again with the play (onIsPlayingChanged).+ return d.playWhenReady+ }+ }+ // Restoring: the song's own speed and full gain.+ val tB = d.currentPosition / 1000.0+ run.eq?.set(0.0, 0.0)+ d.volume = plan.gainAfter(tB).toFloat()+ val speed = (plan.speedAfter(tB) * rate()).toFloat()+ if (kotlin.math.abs(speed - run.lastSpeed) > 0.0005f) { d.playbackParameters = androidx.media3.common.PlaybackParameters(speed); run.lastSpeed = speed }+ if (plan.speedAfter(tB) == 1.0 && plan.gainAfter(tB) >= 0.999) {+ mix = null+ d.volume = 1f+ tune(d)+ com.music.spotui.automix.AutoMix.journal("back to the song's own speed")+ return d.isPlaying+ }+ return true+ }++ /** A pause before the mix: the run-up starts again when the song plays on. */+ private fun mixPaused() {+ val run = mix ?: return+ if (run.state == MixState.PREROLL) {+ run.deck.pause(); run.deck.seekTo(run.startMs); run.state = MixState.READY+ }+ }++ /** The user moved in the song before the mix: the deck goes (prepared again if the end is still ahead). */+ private fun mixMoved() {+ val run = mix ?: return+ if (run.state < MixState.MIXING) cancelMix("seek", again = true)+ }++ private fun cancelMix(why: String, again: Boolean = false) {+ val run = mix ?: return+ if (run.state >= MixState.MIXING) return+ mix = null+ // Not prepared again for this pair (a deck that failed would fail again), unless the user moved.+ if (!again) dropped = run.toIndex to run.toId+ run.eq?.release()+ run.deck.release()+ com.music.spotui.automix.AutoMix.journal("deck dropped: $why")+ }++ /** A user action (skip, seek, stop) during or after the mix: the deck plays on as the song, at its own speed. */+ private fun endMix() {+ val run = mix ?: return+ if (run.state < MixState.MIXING) return cancelMix("stopped")+ mix = null+ run.eqA?.release()+ run.eq?.neutral()+ if (exo === run.deck) tune(run.deck)+ com.music.spotui.automix.AutoMix.journal("mix ended early")+ }++ /** AutoMix turned on or off in the settings. */+ fun autoMixChanged(on: Boolean) {+ val p = exo ?: return+ if (on && deckEq == null) deckEq = com.music.spotui.automix.DeckEq.attach(p.audioSessionId)+ if (!on) { cancelMix("AutoMix off"); planned = null }+ if (p.isPlaying) { handler.removeCallbacks(tick); handler.post(tick) }+ }++ /** For the command line: where the mix stands. */+ fun autoMixState(): String {+ val run = mix+ val p = exo+ return "AutoMix ${if (com.music.spotui.automix.AutoMix.on) "on" else "off"}, " ++ (if (run == null) "no deck" else "deck ${run.state} ${run.plan.style}, progress ${"%.2f".format(run.progress())}") ++ (planned?.let { ", plan ${it.plan}" } ?: "") + (p?.let { ", at ${it.currentPosition} ms of ${it.duration}" } ?: "")+ }+// ── Crossfade (real overlap: every song of this player is a local-player one) ──// Near the end of a song, a second player takes over its last seconds (fading out) while the// main player moves on to the next item at once (fading in). The main player thus sees an@@ -437,13 +870,18 @@ object LocalPlayer {override fun run() {handler.removeCallbacks(this)val p = exo ?: return- if (outgoing != null) {+ if (outgoing != null && mix == null) {val t = CrossfadeRamp.progress(p.currentPosition, fadeMs)p.volume = CrossfadeRamp.incomingGain(t)outgoing?.volume = CrossfadeRamp.outgoingGain(t)if (t >= 1f) endFade() else handler.postDelayed(this, 40)return}+ if (mix != null || (com.music.spotui.automix.AutoMix.on && !CastPlayback.active)) {+ // Every 20 ms from the deck's readiness: its start and the lock depend on it.+ if (mixTick(p)) handler.postDelayed(this, if (mix?.let { it.state >= MixState.READY } == true) 20 else 200)+ return+ }if (!p.isPlaying) returnif (!preparingFade && p.currentMediaItemIndex != noFadeFor && canFade(p) &&CrossfadeRamp.shouldStart(CrossfadePref.seconds, p.duration.takeIf { it > 0 } ?: 0L, p.currentPosition)) beginFade(p)@@ -515,6 +953,7 @@ object LocalPlayer {outgoing?.let { it.stop(); it.release() }if (outgoing != null) SongPlayer.onLocalFade(false)outgoing = null+ endMix()exo?.let { it.volume = 1f; if (it.isPlaying) handler.postDelayed(tick, 200) }}@@ -524,6 +963,7 @@ object LocalPlayer {*/fun fadeProgress(): Float {val p = exo ?: return 1f+ mix?.takeIf { it.state == MixState.MIXING }?.let { m -> return m.progress().toFloat().coerceIn(0f, 1f) }return if (outgoing == null) 1f else CrossfadeRamp.progress(p.currentPosition, fadeMs)}/** Where the song fading out is (0 with no fade). */@@ -537,7 +977,7 @@ object LocalPlayer {p.play()}/** The sleep timer's fade: the player's volume (a crossfade owns it while it runs, the fade then goes on after it). */- fun setGain(gain: Float) { if (outgoing == null) exo?.volume = gain }+ fun setGain(gain: Float) { if (outgoing == null && mix == null) exo?.volume = gain }fun pause() { if (CastPlayback.active) CastPlayback.pause() else exo?.pause() }fun next() {// In the first half of a crossfade the main player is on the next song already, while the@@ -548,6 +988,11 @@ object LocalPlayer {log("skip during the crossfade: the song coming in plays")return endFade()}+ if (mixOnSkip()) {+ cancelOnce("next")+ com.music.spotui.extensions.Extensions.skipAsked("forward")+ return+ }endFade()cancelOnce("next")exo?.let {diff --git a/base/app/src/main/java/com/music/spotui/ui/immersive/pages/ImmersiveSettingsScreen.kt b/base/app/src/main/java/com/music/spotui/ui/immersive/pages/ImmersiveSettingsScreen.ktindex 9c1388ec..4b233a3b 100644--- a/base/app/src/main/java/com/music/spotui/ui/immersive/pages/ImmersiveSettingsScreen.kt+++ b/base/app/src/main/java/com/music/spotui/ui/immersive/pages/ImmersiveSettingsScreen.kt@@ -244,7 +244,8 @@ private fun PlaybackPage(navController: NavController, open: (SettingsPage) -> Urow {ImmersiveGroupRow(stringResource(R.string.settings_crossfade_title), detail = stringResource(R.string.settings_crossfade_detail),- value = if (crossfade == 0) stringResource(R.string.settings_crossfade_off) else stringResource(R.string.settings_crossfade_seconds, crossfade),+ value = if (com.music.spotui.automix.AutoMix.on) stringResource(R.string.settings_automix_title)+ else if (crossfade == 0) stringResource(R.string.settings_crossfade_off) else stringResource(R.string.settings_crossfade_seconds, crossfade),chevron = true, onClick = { open(SettingsPage.Crossfade) },)}@@ -339,6 +340,12 @@ private fun CrossfadePage(back: () -> Unit) {val ctx = LocalContext.currentval pref = com.music.spotui.data.preferences.CrossfadePrefImmersivePage(title = stringResource(R.string.settings_crossfade_title), onBack = back) {+ immersiveGroup("automix", footerRes = R.string.settings_automix_detail) {+ row {+ ImmersiveSwitchRow(stringResource(R.string.settings_automix_title), checked = com.music.spotui.automix.AutoMix.on,+ onChecked = { com.music.spotui.automix.AutoMix.set(it); com.music.spotui.di.LocalPlayer.autoMixChanged(it) })+ }+ }immersiveGroup("choices", footerRes = R.string.settings_crossfade_detail) {for (v in listOf(0, 3, 6, 9, 12)) {row {diff --git a/base/app/src/main/java/com/music/spotui/ui/screens/SettingsScreen.kt b/base/app/src/main/java/com/music/spotui/ui/screens/SettingsScreen.ktindex 0f5346d3..fc45b47f 100644--- a/base/app/src/main/java/com/music/spotui/ui/screens/SettingsScreen.kt+++ b/base/app/src/main/java/com/music/spotui/ui/screens/SettingsScreen.kt@@ -126,6 +126,12 @@ fun SettingsScreen(navController: NavController) {onChecked = { com.music.spotui.data.preferences.AppShufflePref.set(ctx, it) },)CrossfadeRow()+ OptionRow(+ label = stringResource(R.string.settings_automix_title),+ detail = stringResource(R.string.settings_automix_detail),+ checked = com.music.spotui.automix.AutoMix.on,+ onChecked = { com.music.spotui.automix.AutoMix.set(it); com.music.spotui.di.LocalPlayer.autoMixChanged(it) },+ )// The local player's equalizer, speed and silence skipping (never Spotify's own songs).Column(Modifier.fillMaxWidth().clickable { navController.navigate(com.music.spotui.ui.navigation.Routes.Equalizer.route) }diff --git a/base/app/src/main/res/values-fr/myspoti_strings.xml b/base/app/src/main/res/values-fr/myspoti_strings.xmlindex 7c8a797a..d3cd86a5 100644--- a/base/app/src/main/res/values-fr/myspoti_strings.xml+++ b/base/app/src/main/res/values-fr/myspoti_strings.xml@@ -395,6 +395,8 @@<string name="myspoti_shuffle_title">Lecture aléatoire Myspoti</string><string name="myspoti_shuffle_detail">Avec la lecture aléatoire activée, une playlist, un album ou les titres likés jouent chaque titre une fois, dans un ordre aléatoire, avant de recommencer. Désactivée : la lecture aléatoire de Spotify. Un compte gratuit peut garder celle de Spotify sur certaines listes.</string><string name="myspoti_shuffle_imposed">Spotify garde sa propre lecture aléatoire sur cette liste pour votre compte : la lecture aléatoire Myspoti y est désactivée.</string>+ <string name="settings_automix_title">AutoMix</string>+ <string name="settings_automix_detail">Transitions façon DJ entre les morceaux des extensions, des téléchargements et des fichiers locaux : chaque morceau est analysé une fois (temps, mesures, phrases, tonalité) et le suivant entre sur le temps, au même tempo, avec un échange des basses. Les morceaux impossibles à caler ont une coupe sur le temps ou un fondu. Les flux ne sont analysés que sur un réseau non limité.</string><string name="settings_crossfade_title">Fondu enchaîné</string><string name="settings_crossfade_detail">Les morceaux issus d\'extensions, de téléchargements et de fichiers locaux se chevauchent : la fin de l\'un s\'efface pendant que le suivant monte. Les flux Spotify ne peuvent pas se chevaucher (un seul flux audio) : ils s\'effacent puis remontent seulement.</string><string name="settings_crossfade_off">Désactivé</string>diff --git a/base/app/src/main/res/values/myspoti_strings.xml b/base/app/src/main/res/values/myspoti_strings.xmlindex 2c2fce75..931c7b80 100644--- a/base/app/src/main/res/values/myspoti_strings.xml+++ b/base/app/src/main/res/values/myspoti_strings.xml@@ -400,6 +400,8 @@<string name="myspoti_shuffle_title">Myspoti shuffle</string><string name="myspoti_shuffle_detail">With shuffle on, a playlist, album or Liked Songs plays every song once, in a random order, before starting over. Off: Spotify\'s shuffle. A free account may keep Spotify\'s shuffle on some lists.</string><string name="myspoti_shuffle_imposed">Spotify keeps its own shuffle on this list for your account: Myspoti shuffle is off for it.</string>+ <string name="settings_automix_title">AutoMix</string>+ <string name="settings_automix_detail">DJ-style transitions between songs from extensions, downloads and local files: each song is analysed once (beats, bars, phrases, key) and the next one comes in on the beat, tempo matched, with a bass swap. Songs that cannot be beat-matched get a cut on the beat or a fade. Streams are analysed on unmetered networks only.</string><string name="settings_crossfade_title">Crossfade</string><string name="settings_crossfade_detail">Songs from extensions, downloads and local files overlap: the end of one fades out while the next fades in. Spotify streams cannot overlap (one audio stream): they only fade out and in.</string><string name="settings_crossfade_off">Off</string>diff --git a/base/automix/build.gradle.kts b/base/automix/build.gradle.ktsnew file mode 100644index 00000000..60bbf090--- /dev/null+++ b/base/automix/build.gradle.kts@@ -0,0 +1,27 @@+// AutoMix: song analysis (beats, bars, phrases, key, loudness) and transition planning.+// Pure Kotlin, no Android: the phone and the desktop bench (tools/automix) run the same code.+plugins {+ alias(libs.plugins.kotlin.jvm)+}++kotlin {+ jvmToolchain(17)+}++dependencies {+ // Beat This runs on ONNX Runtime: the app brings onnxruntime-android, the desktop bench the JVM build+ // (same Java API, same package).+ compileOnly("com.microsoft.onnxruntime:onnxruntime:1.23.2")+ testImplementation("com.microsoft.onnxruntime:onnxruntime:1.23.2")+ testImplementation("junit:junit:4.13.2")+}++// Desktop bench on real songs: ./gradlew :automix:bench -Pargs="<dir> ..." (see tools/automix/README.md).+tasks.register<JavaExec>("bench") {+ classpath = sourceSets["test"].runtimeClasspath+ mainClass.set("app.myspoti.automix.BenchKt")+ args = (findProperty("args") as String?)?.split(" ")?.filter { it.isNotBlank() } ?: emptyList()+ jvmArgs("-Xmx4g")+ environment("AUTOMIX_THREADS", (findProperty("threads") as String?) ?: "2")+ environment("AUTOMIX_MODEL", rootProject.file("app/src/main/assets/automix/beat-this-small0-int8.onnx").path)+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/Analyzer.kt b/base/automix/src/main/kotlin/app/myspoti/automix/Analyzer.ktnew file mode 100644index 00000000..381f952a--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/Analyzer.kt@@ -0,0 +1,320 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.abs+import kotlin.math.max+import kotlin.math.min+import kotlin.math.sqrt++/** Song analysis: beats, downbeats, bars, phrases, key, loudness and the audible bounds. */+object Analyzer {+ /** Last global grid fitted (bench diagnostics only). */+ @Volatile var lastGlobal: Grid? = null+ @Volatile var lastModelBeats: DoubleArray = DoubleArray(0)+ @Volatile var lastWhole: BeatTracker.Quality? = null++ private const val FPS = Features.FPS+ /** Seconds of the song's start and end the head and tail grids are fitted on. */+ const val HEAD_SEC = 75.0+ const val TAIL_SEC = 90.0++ /** Seconds of the start and of the end that Beat This reads. */+ const val MODEL_HEAD_SEC = 30.0+ const val MODEL_TAIL_SEC = 40.0++ /**+ * [mid] and [side] (or null) at [Features.RATE] Hz. With [model], Beat This reads the song's first+ * and last seconds for the metrical level and the downbeats; without it (or when it finds no+ * beats) the onsets alone decide.+ */+ fun analyze(mid: FloatArray, side: FloatArray?, model: BeatThis? = null, cancelled: () -> Boolean = { false }, threads: Int = 1): TrackAnalysis {+ // The onset features and Beat This side by side (the model reads the whole file; its beats+ // outside the audible part are dropped below).+ var feats: Features? = null+ var act: BeatThis.Activations? = null+ val split = max(1, threads / 2)+ parallel(if (model != null) 2 else 1, 2) { task ->+ if (task == 0) feats = Features.of(mid, side, cancelled, if (model != null) split else threads)+ else act = model!!.run(mid, 0, mid.size, cancelled, max(1, threads - split))+ }+ val f = feats!!+ val n = f.frames+ val env = BeatTracker.envelope(f)+ val level = BeatTracker.smooth(f.rmsDb, 4.0)+ val peak = percentile(level, 0.95)+ val gate = max(peak - 45f, -60f)+ val startF = (0 until n).firstOrNull { level[it] > gate } ?: 0+ val endF = (n - 1 downTo 0).firstOrNull { level[it] > gate } ?: (n - 1)+ val contentStart = startF / FPS+ val contentEnd = min(f.durationSec, (endF + 1) / FPS)++ // Beat This: metrical level, downbeats, and the beats of a song whose tempo moves.+ val inside = { t: Double -> t >= contentStart - 0.1 && t <= contentEnd + 0.1 }+ val modelBeats = act?.let { a -> BeatThis.peaks(a, a.beat).filter { inside(it.first) } }.orEmpty()+ val modelDowns = act?.let { a -> BeatThis.peaks(a, a.downbeat).filter { inside(it.first) } }.orEmpty()+ val mb = modelBeats.map { it.first }.toDoubleArray()+ val modelPeriod = modelPeriod(mb)++ val p0 = modelPeriod?.times(FPS) ?: BeatTracker.period(env, startF, endF + 1)+ val global = BeatTracker.fit(env, startF, endF + 1, p0)?.let { onModel(it, mb) }+ lastGlobal = global; lastModelBeats = mb+ // The global grid where it holds near the start and near the end, else a grid of its own there.+ val headSpan = contentStart to min(contentEnd, contentStart + HEAD_SEC)+ val tailSpan = max(contentStart, contentEnd - TAIL_SEC) to contentEnd+ fun holds(g: Grid, span: Pair<Double, Double>): Boolean {+ val q = BeatTracker.quality(env, g, span.first, span.second)+ return q.beats >= 16 && q.driftMs <= 40 && (q.steadiness >= 0.8 || (q.steadiness >= 0.6 && agreement(g, mb, span) >= 0.75))+ }+ fun local(span: Pair<Double, Double>): Grid? =+ BeatTracker.fit(env, (span.first * FPS).toInt(), min(n, (span.second * FPS).toInt()), global?.period?.times(FPS) ?: p0)+ ?.let { onModel(it, mb) }?.takeIf { holds(it, span) }+ // One grid for the whole song (made on a fixed tempo, as most are): it holds in quiet intros too.+ val whole = global?.let { BeatTracker.quality(env, it, contentStart, contentEnd) }+ lastWhole = whole+ val steady = global != null && whole != null && whole.driftMs <= 40 &&+ (whole.steadiness >= 0.75 || (whole.steadiness >= 0.6 && agreement(global, mb, contentStart to contentEnd) >= 0.75))+ val head = if (steady || (global != null && holds(global, headSpan))) global else local(headSpan)+ val tail = if (steady || (global != null && holds(global, tailSpan))) global else local(tailSpan)+ val period = global?.period?.times(FPS) ?: p0+ val beats = (when {+ steady -> DoubleArray(global!!.count) { global.beat(it) }+ mb.size >= 16 -> mb+ else -> BeatTracker.track(env, period).map { it + ONSET_LEAD }.toDoubleArray()+ }).filter { it >= contentStart - 0.05 && it < contentEnd }.toDoubleArray()+ val bpm = global?.bpm ?: (FPS * 60 / p0)++ // Downbeats: Beat This votes near the start and near the end; the onsets otherwise.+ val headPhase = votePhase(beats, modelDowns.filter { it.first < contentStart + HEAD_SEC })+ val tailPhase = votePhase(beats, modelDowns.filter { it.first > contentEnd - TAIL_SEC })+ val dspPhase = if (headPhase == null || tailPhase == null) downbeatPhase(beats, f, env) else 0+ val switchAt = (contentStart + contentEnd) / 2+ val starts = ArrayList<Double>()+ for (k in beats.indices) {+ val ph = (if (beats[k] < switchAt) headPhase else tailPhase) ?: dspPhase+ if (Math.floorMod(k - ph, 4) == 0) starts.add(beats[k])+ }+ val barStarts = starts.toDoubleArray()+ val bars = barStarts.size+ val barDb = FloatArray(bars); val barLow = FloatArray(bars); val barHigh = FloatArray(bars); val barMid = FloatArray(bars)+ val barOnset = FloatArray(bars); val barCentre = FloatArray(bars)+ val barChroma = Array(bars) { FloatArray(12) }+ val centreMedian = if (f.stereo) percentile(f.centre, 0.5) else 0f+ for (i in 0 until bars) {+ val a = barStarts[i]+ val b = if (i + 1 < bars) barStarts[i + 1] else min(contentEnd, a + 4 * 60 / bpm)+ val fa = (a * FPS).toInt().coerceIn(0, n - 1); val fb = (b * FPS).toInt().coerceIn(fa + 1, n)+ barDb[i] = powerMean(f.rmsDb, fa, fb); barLow[i] = powerMean(f.lowDb, fa, fb)+ barHigh[i] = powerMean(f.highDb, fa, fb); barMid[i] = powerMean(f.midDb, fa, fb)+ var o = 0.0; for (j in fa until fb) o += env[j]; barOnset[i] = (o / (fb - fa)).toFloat()+ val ca = (a * Features.CHROMA_FPS).toInt().coerceIn(0, f.chroma.size - 1)+ val cb = (b * Features.CHROMA_FPS).toInt().coerceIn(ca + 1, f.chroma.size)+ var c = 0.0+ for (j in ca until cb) { for (p in 0 until 12) barChroma[i][p] += f.chroma[j][p]; c += f.centre[j] }+ for (p in 0 until 12) barChroma[i][p] /= (cb - ca)+ barCentre[i] = if (f.stereo) (c / (cb - ca)).toFloat() - centreMedian else 0f+ }+ // A downbeat on the head and on the tail grids: the model's downbeats voted on grid beats, else+ // the bar start nearest to the grid.+ fun downOn(g: Grid?, span: Pair<Double, Double>): Double {+ if (g == null) return Double.NaN+ val vote = DoubleArray(4)+ for ((t, v) in modelDowns) if (t in span.first..span.second) {+ val k = Math.round(g.index(t)).toInt()+ if (abs(g.beat(k) - t) < g.period * 0.3) vote[Math.floorMod(k, 4)] += v.toDouble()+ }+ if (vote.sum() > 0) return g.beat(vote.indices.maxBy { vote[it] })+ val bar = barStarts.firstOrNull { it in span.first..span.second } ?: return Double.NaN+ return g.beat(Math.round(g.index(bar)).toInt())+ }+ val headDown = downOn(head, headSpan); val tailDown = downOn(tail, tailSpan)+ val novelty = novelty(arrayOf(barDb, barLow, barMid, barHigh, barOnset), barChroma)+ val phraseOffset = phraseOffset(novelty, barDb)+ val (key, keyStrength) = key(f, level, gate)+ val loud = loudness(f.rmsDb, peak - 20f)+ return TrackAnalysis(f.durationSec, contentStart, contentEnd, loud, key, keyStrength, bpm, steady, head, tail,+ beats, barStarts, barDb, barLow, barHigh, barCentre, novelty, phraseOffset, f.stereo, headDown, tailDown)+ }++ /**+ * Which of every 4 beats starts a bar: the beat phase with the most kick (low-band onsets), the+ * most harmonic change and the most level change from before it to after it.+ */+ fun downbeatPhase(beats: DoubleArray, f: Features, env: FloatArray): Int {+ if (beats.size < 8) return 0+ val nb = beats.size+ val bass = DoubleArray(nb); val chroma = Array(nb) { DoubleArray(12) }; val low = DoubleArray(nb)+ for (k in 0 until nb) {+ val c = (beats[k] * FPS).toInt()+ var m = 0f+ for (j in c - 3..c + 3) m = max(m, f.bassOnset.getOrElse(j) { 0f })+ bass[k] = m.toDouble()+ val end = if (k + 1 < nb) beats[k + 1] else beats[k] + (beats[k] - beats[k - 1])+ val ca = (beats[k] * Features.CHROMA_FPS).toInt().coerceIn(0, f.chroma.size - 1)+ val cb = (end * Features.CHROMA_FPS).toInt().coerceIn(ca + 1, f.chroma.size)+ for (j in ca until cb) for (p in 0 until 12) chroma[k][p] += f.chroma[j][p].toDouble()+ val fa = (beats[k] * FPS).toInt().coerceIn(0, f.frames - 1); val fb = (end * FPS).toInt().coerceIn(fa + 1, f.frames)+ low[k] = powerMean(f.lowDb, fa, fb).toDouble()+ }+ // Harmonic change: the two beats from here against the two before.+ val change = DoubleArray(nb)+ for (k in 2 until nb - 2) {+ val a = DoubleArray(12) { chroma[k - 2][it] + chroma[k - 1][it] }+ val b = DoubleArray(12) { chroma[k][it] + chroma[k + 1][it] }+ change[k] = 1 - cosine(a, b)+ }+ val lowStep = DoubleArray(nb)+ for (k in 1 until nb) lowStep[k] = max(0.0, low[k] - low[k - 1])+ val scores = DoubleArray(4)+ for (feature in listOf(bass to 1.0, change to 1.0, lowStep to 0.5)) {+ val per = DoubleArray(4); val cnt = IntArray(4)+ for (k in 0 until nb) { per[k % 4] += feature.first[k]; cnt[k % 4]++ }+ for (p in 0 until 4) per[p] /= max(1, cnt[p])+ val mean = per.average(); val sd = sqrt(per.sumOf { (it - mean) * (it - mean) } / 4).takeIf { it > 1e-9 } ?: continue+ for (p in 0 until 4) scores[p] += feature.second * (per[p] - mean) / sd+ }+ return scores.indices.maxBy { scores[it] }+ }++ /**+ * Onset-flux peaks come about 27 ms before the beat a listener (and Beat This) places; grids are+ * moved by that much without the model, so that every song's beats mean the same instant.+ */+ private const val ONSET_LEAD = 0.027++ /** Mean beat interval of the model's beats near their median (seconds), or null with too few. */+ private fun modelPeriod(beats: DoubleArray): Double? {+ val gaps = beats.toList().zipWithNext { a, b -> b - a }.filter { it in 60 / BeatTracker.MAX_BPM..60 / BeatTracker.MIN_BPM }+ if (gaps.size < 12) return null+ val m = gaps.sorted()[gaps.size / 2]+ return gaps.filter { abs(it - m) < m * 0.1 }.average()+ }++ /**+ * The onset grid put on the model's beats: moved by half a beat when they fall between its beats+ * (an off-beat lock), then by the median gap to them. Without the model, by [ONSET_LEAD].+ */+ private fun onModel(grid: Grid, modelBeats: DoubleArray): Grid {+ if (modelBeats.size < 8) return grid.copy(first = grid.first + ONSET_LEAD)+ var g = grid+ var c = 0.0; var s = 0.0+ for (t in modelBeats) { val a = 2 * Math.PI * g.index(t); c += Math.cos(a); s += Math.sin(a) }+ if (abs(Math.atan2(s, c) / (2 * Math.PI)) > 0.25) g = g.copy(first = g.first + g.period / 2, count = g.count - 1)+ val gaps = modelBeats.filter { it >= g.first - g.period / 2 && it <= g.last + g.period / 2 }+ .map { t -> (t - g.beat(Math.round(g.index(t)).toInt())) }.filter { abs(it) < g.period * 0.25 }.sorted()+ return if (gaps.size < 8) g else g.copy(first = g.first + gaps[gaps.size / 2])+ }++ /** Share of the model's beats (in [span] and the grid's) within 50 ms of a grid beat; 1 without the model. */+ private fun agreement(g: Grid, modelBeats: DoubleArray, span: Pair<Double, Double>): Double {+ val inside = modelBeats.filter { it >= max(span.first, g.first - 0.05) && it <= min(span.second, g.last + 0.05) }+ if (inside.size < 8) return 1.0+ return inside.count { t -> abs(t - g.beat(Math.round(g.index(t)).toInt())) <= 0.05 }.toDouble() / inside.size+ }++ /** The phase (beat index mod 4) that most of the model's downbeats land on, weighted by their logits. */+ private fun votePhase(beats: DoubleArray, downbeats: List<Pair<Double, Float>>): Int? {+ if (beats.size < 8 || downbeats.size < 2) return null+ val vote = DoubleArray(4)+ for ((t, v) in downbeats) {+ val k = nearest(beats, t)+ val gap = if (k + 1 < beats.size) beats[k + 1] - beats[k] else beats[k] - beats[k - 1]+ if (abs(beats[k] - t) < gap * 0.3) vote[k % 4] += v.toDouble()+ }+ return if (vote.sum() <= 0) null else vote.indices.maxBy { vote[it] }+ }++ private fun nearest(x: DoubleArray, t: Double): Int {+ var lo = 0; var hi = x.size - 1+ while (hi - lo > 1) { val m = (lo + hi) ushr 1; if (x[m] <= t) lo = m else hi = m }+ return if (abs(x[lo] - t) <= abs(x[hi] - t)) lo else hi+ }++ /** Per bar: how far the mean feature vector of the 4 bars from it is from that of the 4 before. */+ private fun novelty(features: Array<FloatArray>, chroma: Array<FloatArray>): FloatArray {+ val bars = chroma.size+ if (bars < 9) return FloatArray(bars)+ val dims = features.size + 12+ val v = Array(bars) { i -> DoubleArray(dims) { d -> if (d < features.size) features[d][i].toDouble() else chroma[i][d - features.size].toDouble() } }+ // Standardised per dimension, chroma weighted down (12 dimensions of one kind).+ for (d in 0 until dims) {+ val m = v.sumOf { it[d] } / bars+ val sd = sqrt(v.sumOf { (it[d] - m) * (it[d] - m) } / bars).takeIf { it > 1e-9 } ?: 1.0+ val w = if (d < features.size) 1.0 else 0.35+ for (i in 0 until bars) v[i][d] = (v[i][d] - m) / sd * w+ }+ val out = FloatArray(bars)+ for (i in 1 until bars) {+ val span = min(4, min(i, bars - i))+ var s = 0.0+ for (d in 0 until dims) {+ var a = 0.0; var b = 0.0+ for (j in 0 until span) { a += v[i - 1 - j][d]; b += v[i + j][d] }+ val diff = (b - a) / span+ s += diff * diff+ }+ out[i] = sqrt(s).toFloat() * if (span < 4) span / 4f else 1f+ }+ return out+ }++ /** The bar offset of 8-bar phrases: where the novelty peaks recur every 8 bars. */+ private fun phraseOffset(novelty: FloatArray, barDb: FloatArray): Int {+ if (novelty.size < 16) return 0+ val per8 = DoubleArray(8)+ for (i in novelty.indices) per8[i % 8] += novelty[i].toDouble()+ return per8.indices.maxBy { per8[it] }+ }++ private val MAJOR = doubleArrayOf(6.35, 2.23, 3.48, 2.33, 4.38, 4.09, 2.52, 5.19, 2.39, 3.66, 2.29, 2.88)+ private val MINOR = doubleArrayOf(6.33, 2.68, 3.52, 5.38, 2.60, 3.53, 2.54, 4.75, 3.98, 2.69, 3.34, 3.17)++ /** Krumhansl–Kessler key of the song's summed chroma, with its correlation. */+ private fun key(f: Features, level: FloatArray, gate: Float): Pair<Int, Double> {+ val sum = DoubleArray(12)+ for (j in f.chroma.indices) {+ val lf = (j / Features.CHROMA_FPS * FPS).toInt().coerceIn(0, level.size - 1)+ if (level[lf] > gate + 10) for (p in 0 until 12) sum[p] += f.chroma[j][p]+ }+ if (sum.sum() <= 0) return -1 to 0.0+ var best = -1; var bestR = -2.0+ for (mode in 0..1) for (tonic in 0 until 12) {+ val prof = if (mode == 0) MAJOR else MINOR+ val r = pearson(DoubleArray(12) { sum[it] }, DoubleArray(12) { prof[Math.floorMod(it - tonic, 12)] })+ if (r > bestR) { bestR = r; best = mode * 12 + tonic }+ }+ // Minor keys are numbered by tonic + 12 (A minor = 21).+ return best to bestR+ }++ private fun loudness(db: FloatArray, gate: Float): Double {+ var s = 0.0; var c = 0+ for (v in db) if (v > gate) { s += Math.pow(10.0, v / 10.0); c++ }+ return if (c == 0) -60.0 else 10 * Math.log10(s / c)+ }++ internal fun powerMean(db: FloatArray, from: Int, to: Int): Float {+ var s = 0.0+ val a = from.coerceIn(0, db.size - 1); val b = to.coerceIn(a + 1, db.size)+ for (j in a until b) s += Math.pow(10.0, db[j] / 10.0)+ return Features.db(s / (b - a))+ }++ internal fun percentile(x: FloatArray, q: Double): Float {+ if (x.isEmpty()) return 0f+ val s = x.copyOf(); s.sort()+ return s[((s.size - 1) * q).toInt()]+ }++ private fun cosine(a: DoubleArray, b: DoubleArray): Double {+ var ab = 0.0; var aa = 0.0; var bb = 0.0+ for (i in a.indices) { ab += a[i] * b[i]; aa += a[i] * a[i]; bb += b[i] * b[i] }+ return if (aa <= 0 || bb <= 0) 1.0 else ab / sqrt(aa * bb)+ }++ private fun pearson(a: DoubleArray, b: DoubleArray): Double {+ val ma = a.average(); val mb = b.average()+ var ab = 0.0; var aa = 0.0; var bb = 0.0+ for (i in a.indices) { ab += (a[i] - ma) * (b[i] - mb); aa += (a[i] - ma) * (a[i] - ma); bb += (b[i] - mb) * (b[i] - mb) }+ return if (aa <= 0 || bb <= 0) 0.0 else ab / sqrt(aa * bb)+ }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/BeatThis.kt b/base/automix/src/main/kotlin/app/myspoti/automix/BeatThis.ktnew file mode 100644index 00000000..98879f0f--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/BeatThis.kt@@ -0,0 +1,169 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import ai.onnxruntime.OnnxTensor+import ai.onnxruntime.OrtEnvironment+import ai.onnxruntime.OrtSession+import java.nio.FloatBuffer+import kotlin.math.ln+import kotlin.math.max+import kotlin.math.min++/**+ * Beat This! small0 (CPJKU, MIT code and weights), int8 ONNX, run through ONNX Runtime: beat and+ * downbeat logits at [FPS] frames per second. The phone (onnxruntime-android) and the desktop bench+ * (onnxruntime) share this code. It decides the metrical level and the downbeats; the onset grid of+ * [BeatTracker] gives the precise beat times.+ */+class BeatThis(model: ByteArray, private val defaultThreads: Int = 4) : AutoCloseable {+ private val env = OrtEnvironment.getEnvironment()+ // One thread per run: the song's chunks run side by side instead (a run uses a core badly on its own).+ private val session = OrtSession.SessionOptions().use { o ->+ o.setIntraOpNumThreads(1)+ o.setInterOpNumThreads(1)+ o.setOptimizationLevel(OrtSession.SessionOptions.OptLevel.ALL_OPT)+ env.createSession(model, o)+ }++ /** Beat and downbeat logits of mono [RATE]-Hz samples [from, to). */+ class Activations(val start: Double, val beat: FloatArray, val downbeat: FloatArray) {+ fun time(frame: Double) = start + frame / FPS+ }++ fun run(mono: FloatArray, from: Int, to: Int, cancelled: () -> Boolean = { false }, threads: Int = defaultThreads): Activations {+ val mel = MelFrontend.spectrogram(mono, from, to, threads)+ val n = mel.size+ val beat = FloatArray(n); val down = FloatArray(n)+ // Chunks of 768 frames that overlap by 12; each keeps all but its 6 edge frames.+ val starts = ArrayList<Int>()+ var start = -EDGE+ while (true) {+ starts.add(start)+ val frames = max(2 * EDGE + 1, min(CHUNK, n + EDGE - start))+ if (start + frames - EDGE >= n) break+ start += CHUNK - 2 * EDGE+ }+ parallel(starts.size, threads) { c ->+ if (cancelled()) throw InterruptedException("analysis cancelled")+ val st = starts[c]+ val frames = max(2 * EDGE + 1, min(CHUNK, n + EDGE - st))+ val flat = FloatArray(frames * MelFrontend.MELS)+ for (i in 0 until frames) if (st + i in 0 until n) mel[st + i].copyInto(flat, i * MelFrontend.MELS)+ OnnxTensor.createTensor(env, FloatBuffer.wrap(flat), longArrayOf(1, frames.toLong(), MelFrontend.MELS.toLong())).use { input ->+ session.run(mapOf("spect" to input)).use { out ->+ val b = (out.get("beat").get() as OnnxTensor).floatBuffer+ val d = (out.get("downbeat").get() as OnnxTensor).floatBuffer+ // Each chunk writes its own frames (the kept ranges do not overlap).+ for (i in EDGE until frames - EDGE) {+ val at = st + i+ if (at in 0 until n) { beat[at] = b.get(i); down[at] = d.get(i) }+ }+ }+ }+ }+ return Activations(from.toDouble() / RATE, beat, down)+ }++ override fun close() = session.close()++ companion object {+ const val RATE = MelFrontend.RATE+ const val FPS = MelFrontend.FPS+ private const val CHUNK = 768+ private const val EDGE = 6++ /** Peaks of [logits] over 0 that are the largest within ±3 frames, in seconds, parabolically refined. */+ fun peaks(a: Activations, logits: FloatArray): List<Pair<Double, Float>> {+ val out = ArrayList<Pair<Double, Float>>()+ for (i in logits.indices) {+ val v = logits[i]+ if (v <= 0f) continue+ var top = true+ for (j in max(0, i - 3)..min(logits.size - 1, i + 3)) if (logits[j] > v || (logits[j] == v && j < i)) { top = false; break }+ if (!top) continue+ val y0 = logits.getOrElse(i - 1) { v }; val y2 = logits.getOrElse(i + 1) { v }+ val den = y0 - 2 * v + y2+ val d = if (den != 0f) (0.5 * (y0 - y2) / den).coerceIn(-0.5, 0.5) else 0.0+ out.add(a.time(i + d) to v)+ }+ return out+ }+ }+}++/**+ * Beat This input: Slaney-style 128-band magnitude log-mel of mono 22.05-kHz audio, 1024-point+ * FFT, hop 441 (50 frames per second), centred frames with reflected edges, log(1 + 1000 x).+ * Matches the upstream torchaudio front end (checked to 1e-4 in the earlier AutoMix work).+ */+object MelFrontend {+ const val RATE = 22050+ const val FFT_N = 1024+ const val HOP = 441+ const val MELS = 128+ const val FPS = RATE.toDouble() / HOP++ private val filters = Array(MELS) { DoubleArray(FFT_N / 2 + 1) }+ private val first = IntArray(MELS); private val last = IntArray(MELS)+ private val window = hann(FFT_N)++ init {+ val lo = mel(30.0); val hi = mel(11000.0)+ val edges = DoubleArray(MELS + 2) { hz(lo + (hi - lo) * it / (MELS + 1)) }+ for (m in 0 until MELS) {+ first[m] = FFT_N / 2 + 1+ for (k in 0..FFT_N / 2) {+ val f = k * RATE.toDouble() / FFT_N+ filters[m][k] = max(0.0, min((f - edges[m]) / (edges[m + 1] - edges[m]), (edges[m + 2] - f) / (edges[m + 2] - edges[m + 1])))+ if (filters[m][k] > 0) { first[m] = min(first[m], k); last[m] = k + 1 }+ }+ }+ }++ private fun mel(f: Double) = if (f < 1000) f / (200.0 / 3) else 15 + ln(f / 1000) / (ln(6.4) / 27)+ private fun hz(m: Double) = if (m < 15) m * (200.0 / 3) else 1000 * Math.exp((m - 15) * (ln(6.4) / 27))++ fun spectrogram(mono: FloatArray, from: Int, to: Int, threads: Int = 1): Array<FloatArray> {+ val n = to - from+ require(n >= FFT_N) { "audio window too short" }+ val frames = 1 + n / HOP+ val out = Array(frames) { FloatArray(MELS) }+ val parts = max(1, threads)+ parallel(parts, parts) { part -> slice(mono, from, n, out, frames * part / parts, frames * (part + 1) / parts) }+ return out+ }++ private fun slice(mono: FloatArray, from: Int, n: Int, out: Array<FloatArray>, t0: Int, t1: Int) {+ val fft = Fft(FFT_N)+ val re = DoubleArray(FFT_N); val im = DoubleArray(FFT_N); val mag = DoubleArray(FFT_N / 2 + 1)+ for (t in t0 until t1) {+ for (i in 0 until FFT_N) {+ var at = t * HOP + i - FFT_N / 2+ while (at < 0 || at >= n) at = if (at < 0) -at else 2 * n - 2 - at+ re[i] = mono[from + at] * window[i]; im[i] = 0.0+ }+ fft.transform(re, im)+ for (k in 0..FFT_N / 2) mag[k] = Math.hypot(re[k], im[k]) / 32+ for (m in 0 until MELS) {+ var s = 0.0+ for (k in first[m] until last[m]) s += mag[k] * filters[m][k]+ out[t][m] = Math.log1p(1000 * s).toFloat()+ }+ }+ }+}++/** Runs [task] for 0 until [count] on up to [threads] threads; the first failure is thrown. */+internal fun parallel(count: Int, threads: Int, task: (Int) -> Unit) {+ if (threads <= 1 || count <= 1) { for (i in 0 until count) task(i); return }+ val next = java.util.concurrent.atomic.AtomicInteger()+ val error = java.util.concurrent.atomic.AtomicReference<Throwable?>()+ val workers = List(min(threads, count) - 1) {+ Thread {+ while (error.get() == null) { val i = next.getAndIncrement(); if (i >= count) break; try { task(i) } catch (t: Throwable) { error.compareAndSet(null, t) } }+ }.apply { priority = Thread.currentThread().priority; isDaemon = true; start() }+ }+ while (error.get() == null) { val i = next.getAndIncrement(); if (i >= count) break; try { task(i) } catch (t: Throwable) { error.compareAndSet(null, t) } }+ workers.forEach { it.join() }+ error.get()?.let { throw it }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/BeatTracker.kt b/base/automix/src/main/kotlin/app/myspoti/automix/BeatTracker.ktnew file mode 100644index 00000000..e1a95ecd--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/BeatTracker.kt@@ -0,0 +1,274 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.abs+import kotlin.math.exp+import kotlin.math.floor+import kotlin.math.ln+import kotlin.math.max+import kotlin.math.min+import kotlin.math.roundToInt+import kotlin.math.sqrt++/**+ * A constant-tempo beat grid over part of a song: beat k is at [first] + k × [period] seconds.+ * [steadiness] is the share of beats whose onset peak lies within 5 % of a period (and 35 ms) of+ * the grid; [driftMs] the largest gap between the grid and the onsets, averaged over 8 beats;+ * [contrast] the onset level on the beats over that half a beat off them.+ */+data class Grid(+ val first: Double,+ val period: Double,+ val count: Int,+ val steadiness: Double,+ val driftMs: Double,+ val contrast: Double,+) {+ val bpm get() = 60.0 / period+ fun beat(k: Int) = first + k * period+ val last get() = beat(count - 1)+ /** Fractional beat index of time [t] (seconds). */+ fun index(t: Double) = (t - first) / period+ /** Steady enough to beat-match: most beats on the grid, no drift, clear onsets. */+ val reliable get() = count >= 16 && steadiness >= 0.6 && driftMs <= 30+}++/** Tempo and beats from an onset envelope at [Features.FPS] frames per second. */+object BeatTracker {+ private const val FPS = Features.FPS+ const val MIN_BPM = 55.0+ const val MAX_BPM = 210.0++ /** The onset envelope used for beats: both fluxes, local mean removed, unit variance. */+ fun envelope(f: Features): FloatArray {+ val n = f.frames+ val raw = FloatArray(n) { f.onset[it] + 1.5f * f.bassOnset[it] }+ val half = 24+ val out = FloatArray(n)+ // Running mean over ±half frames, subtracted (onsets stand out of a loud passage too).+ var sum = 0.0+ var lo = 0; var hi = -1+ for (i in 0 until n) {+ while (hi < min(n - 1, i + half)) { hi++; sum += raw[hi] }+ while (lo < i - half) { sum -= raw[lo]; lo++ }+ out[i] = max(0f, raw[i] - (sum / (hi - lo + 1)).toFloat())+ }+ var m = 0.0; for (v in out) m += v * v+ val sd = sqrt(m / n).toFloat()+ if (sd > 0) for (i in 0 until n) out[i] /= sd+ return out+ }++ /**+ * Beat period in frames over [from, to): the autocorrelation peak, weighted by a log-normal tempo+ * prior around [preferBpm] and by the bar-level (2 × lag) correlation.+ */+ fun period(env: FloatArray, from: Int = 0, to: Int = env.size, preferBpm: Double = 100.0): Double {+ val minLag = floor(FPS * 60 / MAX_BPM).toInt()+ val maxLag = min((FPS * 60 / MIN_BPM).toInt() + 1, (to - from) / 6)+ if (maxLag <= minLag + 2) return FPS * 60 / 120+ // Autocorrelation of the envelope less its mean (else every lag shares a large constant).+ var mean = 0.0+ for (i in from until to) mean += env[i]+ mean /= (to - from)+ val e = DoubleArray(to - from) { env[from + it] - mean }+ val acf = DoubleArray(4 * maxLag + 2)+ for (lag in 0 until acf.size) {+ var s = 0.0+ for (i in 0 until e.size - lag) s += e[i] * e[i + lag]+ acf[lag] = s / (e.size - lag).coerceAtLeast(1)+ }+ fun at(l: Double): Double { val i = l.toInt(); val f = l - i; return if (i + 1 >= acf.size) 0.0 else acf[i] * (1 - f) + acf[i + 1] * f }+ // Peak around a lag (onsets jitter by a frame or so).+ fun peak(l: Double): Double = maxOf(at(l - 1), at(l), at(l + 1))+ var best = -1; var bestScore = Double.NEGATIVE_INFINITY+ for (lag in minLag..maxLag) {+ if (!(acf[lag] >= acf[lag - 1] && acf[lag] >= acf[lag + 1])) continue+ val bpm = FPS * 60 / lag+ val prior = exp(-0.5 * (ln(bpm / preferBpm) / ln(2.0) / 1.0).let { it * it })+ // A beat period recurs at 2 and 4 beats (half bar, bar); a triplet or dotted period+ // (3/4 or 4/3 of the beat) does not.+ val score = prior * (acf[lag] + peak(2.0 * lag) + peak(4.0 * lag))+ if (score > bestScore) { bestScore = score; best = lag }+ }+ if (best < 0) return FPS * 60 / 120+ // Parabolic interpolation of the peak.+ val a = acf[best - 1]; val b = acf[best]; val c = acf[best + 1]+ val d = (a - 2 * b + c).let { if (it == 0.0) 0.0 else 0.5 * (a - c) / it }+ return best + d.coerceIn(-0.5, 0.5)+ }++ /**+ * The constant grid that best fits the onsets of [from, to) frames, period near [period0]:+ * a coarse search of period and phase, then a weighted least-squares fit on the onset peaks+ * found near each grid beat, outliers dropped.+ */+ fun fit(env: FloatArray, from: Int, to: Int, period0: Double): Grid? {+ if (to - from < period0 * 8) return null+ val s = smooth(env, 1.5)+ // Coarse: period within ±3 %, phase in half frames.+ var bestP = period0; var bestPhase = 0.0; var bestScore = Double.NEGATIVE_INFINITY+ var p = period0 * 0.97+ while (p <= period0 * 1.03) {+ var ph = 0.0+ while (ph < p) {+ var sum = 0.0; var cnt = 0+ var t = from + ph+ while (t < to - 1) { sum += interp(s, t); cnt++; t += p }+ val score = sum / max(1, cnt)+ if (score > bestScore) { bestScore = score; bestP = p; bestPhase = from + ph }+ ph += 0.5+ }+ p += period0 * 0.0005+ }+ // Fine: least squares on the peaks next to each beat.+ var first = bestPhase; var per = bestP+ var kept = 0; var steady = 0.0; var drift = 0.0+ val count = floor((to - 1 - first) / per).toInt() + 1+ if (count < 8) return null+ val peakT = DoubleArray(count); val peakW = DoubleArray(count)+ repeat(3) { pass ->+ val win = per * (if (pass == 0) 0.15 else 0.1)+ for (k in 0 until count) {+ val c = first + k * per+ val lo = max(from, floor(c - win).toInt()); val hi = min(to - 2, (c + win).toInt() + 1)+ var at = -1; var v = 0f+ for (i in lo..hi) if (s[i] > v && s[i] >= s.getOrElse(i - 1) { 0f } && s[i] >= s.getOrElse(i + 1) { 0f }) { v = s[i]; at = i }+ if (at < 0 || v <= 0f) { peakW[k] = 0.0; continue }+ val y0 = s.getOrElse(at - 1) { v }; val y2 = s.getOrElse(at + 1) { v }+ val den = y0 - 2 * v + y2+ peakT[k] = at + if (den != 0f) (0.5 * (y0 - y2) / den).coerceIn(-0.5, 0.5) else 0.0+ peakW[k] = v.toDouble()+ }+ // Robust: weights cut for residuals over 6 % of a period.+ var sw = 0.0; var sk = 0.0; var st = 0.0; var skk = 0.0; var skt = 0.0+ for (k in 0 until count) {+ if (peakW[k] <= 0) continue+ val r = peakT[k] - (first + k * per)+ if (pass > 0 && abs(r) > per * 0.06) continue+ val w = peakW[k]+ sw += w; sk += w * k; st += w * peakT[k]; skk += w * k * k; skt += w * k * peakT[k]+ }+ val den = sw * skk - sk * sk+ if (sw > 0 && den > 1e-9) {+ val newPer = (sw * skt - sk * st) / den+ if (newPer in period0 * 0.95..period0 * 1.05) { per = newPer; first = (st - per * sk) / sw }+ }+ }+ // Metrics.+ val tol = min(per * 0.05, 0.035 * FPS)+ val residual = DoubleArray(count)+ var weightAll = 0.0; var weightOn = 0.0+ for (k in 0 until count) {+ residual[k] = if (peakW[k] > 0) peakT[k] - (first + k * per) else Double.NaN+ val w = max(peakW[k], 0.2)+ weightAll += w+ if (peakW[k] > 0 && abs(residual[k]) <= tol) { weightOn += w; kept++ }+ }+ steady = weightOn / weightAll+ // Drift: weighted mean residual over 8 consecutive beats, largest absolute value.+ for (k in 0..count - 8) {+ var sw = 0.0; var sr = 0.0+ for (j in k until k + 8) if (!residual[j].isNaN() && abs(residual[j]) < per * 0.2) { sw += peakW[j]; sr += peakW[j] * residual[j] }+ if (sw > 0) drift = max(drift, abs(sr / sw))+ }+ // Contrast: onset level on the beats over the level half a beat off them.+ var on = 0.0; var off = 0.0+ for (k in 0 until count - 1) { on += interp(s, first + k * per); off += interp(s, first + (k + 0.5) * per) }+ val contrast = if (off > 0) on / off else 9.0+ while (first - per >= from) first -= per+ val n = floor((to - 1 - first) / per).toInt() + 1+ return Grid(first / FPS, per / FPS, n, steady, drift / FPS * 1000, contrast)+ }++ /** How well [grid] holds over [fromSec, toSec): its steadiness and drift there, its constant offset to the onsets taken out. */+ class Quality(val steadiness: Double, val driftMs: Double, val beats: Int) {+ val good get() = beats >= 16 && steadiness >= 0.7 && driftMs <= 40+ override fun toString() = String.format("%.2f/%.0fms/%d", steadiness, driftMs, beats)+ }++ fun quality(env: FloatArray, grid: Grid, fromSec: Double, toSec: Double): Quality {+ val s = smooth(env, 1.5)+ val per = grid.period * FPS+ val k0 = Math.ceil(grid.index(fromSec)).toInt().coerceAtLeast(0)+ val k1 = floor(grid.index(toSec)).toInt().coerceAtMost(grid.count - 1)+ if (k1 - k0 < 8) return Quality(0.0, 999.0, max(0, k1 - k0 + 1))+ val count = k1 - k0 + 1+ val res = DoubleArray(count) { Double.NaN }; val w = DoubleArray(count)+ val win = per * 0.12+ for (i in 0 until count) {+ val c = grid.beat(k0 + i) * FPS+ val lo = max(0, floor(c - win).toInt()); val hi = min(s.size - 2, (c + win).toInt() + 1)+ var at = -1; var v = 0f+ for (j in lo..hi) if (s[j] > v && s[j] >= s.getOrElse(j - 1) { 0f } && s[j] >= s.getOrElse(j + 1) { 0f }) { v = s[j]; at = j }+ if (at < 0) continue+ res[i] = at - c; w[i] = v.toDouble()+ }+ val valid = (0 until count).filter { !res[it].isNaN() }+ if (valid.size < 8) return Quality(0.0, 999.0, count)+ val offset = valid.map { res[it] }.sorted()[valid.size / 2]+ for (i in valid) res[i] -= offset+ val tol = min(per * 0.05, 0.035 * FPS)+ var all = 0.0; var on = 0.0+ for (i in 0 until count) { val wi = max(w[i], 0.2); all += wi; if (!res[i].isNaN() && abs(res[i]) <= tol) on += wi }+ var drift = 0.0+ for (k in 0..count - 8) {+ var sw = 0.0; var sr = 0.0+ var m = 0+ for (j in k until k + 8) if (!res[j].isNaN() && abs(res[j]) < per * 0.08) { sw += w[j]; sr += w[j] * res[j]; m++ }+ if (m >= 4) drift = max(drift, abs(sr / sw))+ }+ return Quality(on / all, drift / FPS * 1000, count)+ }++ /**+ * Dynamic-programming beat tracker (Ellis 2007) for songs whose tempo moves: each beat is the+ * best previous beat about one [period] earlier plus this frame's onset.+ */+ fun track(env: FloatArray, period: Double, tightness: Double = 120.0): DoubleArray {+ val n = env.size+ if (n < period * 4) return DoubleArray(0)+ val s = smooth(env, 1.5)+ val score = DoubleArray(n); val back = IntArray(n) { -1 }+ val lo = (period * 0.5).roundToInt(); val hi = (period * 2).roundToInt()+ for (i in 0 until n) {+ var best = 0.0; var arg = -1+ for (d in lo..hi) {+ val j = i - d+ if (j < 0) break+ val pen = ln(d / period).let { -tightness * it * it }+ val v = score[j] + pen+ if (arg < 0 || v > best) { best = v; arg = j }+ }+ score[i] = s[i] + if (arg >= 0) max(best, 0.0) else 0.0+ back[i] = if (arg >= 0 && best > 0) arg else -1+ }+ // Best end in the last period, followed back.+ var end = n - 1; var bestEnd = Double.NEGATIVE_INFINITY+ for (i in max(0, n - hi) until n) if (score[i] > bestEnd) { bestEnd = score[i]; end = i }+ val beats = ArrayList<Double>()+ var i = end+ while (i >= 0) { beats.add(i / FPS); i = back[i] }+ beats.reverse()+ return beats.toDoubleArray()+ }++ internal fun smooth(x: FloatArray, sigma: Double): FloatArray {+ val r = (sigma * 3).toInt()+ val k = DoubleArray(2 * r + 1) { exp(-0.5 * ((it - r) / sigma).let { v -> v * v }) }+ val ks = k.sum()+ return FloatArray(x.size) { i ->+ var s = 0.0+ for (j in -r..r) s += k[j + r] * x.getOrElse(i + j) { 0f }+ (s / ks).toFloat()+ }+ }++ internal fun interp(x: FloatArray, t: Double): Double {+ if (t <= 0) return x.firstOrNull()?.toDouble() ?: 0.0+ val i = t.toInt()+ if (i >= x.size - 1) return x.lastOrNull()?.toDouble() ?: 0.0+ val f = t - i+ return x[i] * (1 - f) + x[i + 1] * f+ }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/Features.kt b/base/automix/src/main/kotlin/app/myspoti/automix/Features.ktnew file mode 100644index 00000000..af24e68f--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/Features.kt@@ -0,0 +1,149 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.ln+import kotlin.math.log10+import kotlin.math.log2+import kotlin.math.max+import kotlin.math.pow+import kotlin.math.roundToInt++/**+ * Frame features of a song, from its mid (L+R)/2 and side (L-R)/2 signals at [RATE] Hz.+ * Onset frames are [HOP] samples apart ([FPS] per second).+ */+class Features private constructor(+ /** Spectral flux on log bands, all bands. */+ val onset: FloatArray,+ /** Spectral flux of the bands under 150 Hz (kicks, 808s). */+ val bassOnset: FloatArray,+ /** Band levels in dB per onset frame: low (< 180 Hz), mid (180 Hz – 2.5 kHz), high (> 2.5 kHz), all. */+ val lowDb: FloatArray, val midDb: FloatArray, val highDb: FloatArray, val rmsDb: FloatArray,+ /** Pitch-class profile, 12 values per chroma frame ([CHROMA_HOP] samples apart). */+ val chroma: Array<FloatArray>,+ /** Voice-band (300 Hz – 3.4 kHz) level of the mid signal over the side signal, in dB per chroma frame. */+ val centre: FloatArray,+ /** True when the song has a side signal at all (not mono): [centre] means something only then. */+ val stereo: Boolean,+ val durationSec: Double,+) {+ val frames get() = onset.size++ companion object {+ const val RATE = 22050+ const val HOP = 256+ const val N = 1024+ const val CHROMA_N = 4096+ const val CHROMA_HOP = 2048+ const val FPS = RATE.toDouble() / HOP+ const val CHROMA_FPS = RATE.toDouble() / CHROMA_HOP+ const val LOW_HZ = 180.0+ const val HIGH_HZ = 2500.0+ private const val BANDS = 40++ /** [threads]: the onset frames are cut into that many slices, computed side by side (and the chroma beside them). */+ fun of(mid: FloatArray, side: FloatArray?, cancelled: () -> Boolean = { false }, threads: Int = 1): Features {+ val win = hann(N)+ val frames = max(1, (mid.size - N) / HOP + 1)+ val binHz = RATE.toDouble() / N+ // Log-spaced bands, 30 Hz to 10 kHz, each at least one bin wide.+ val edges = IntArray(BANDS + 1) { i ->+ val hz = 30.0 * (10_000.0 / 30.0).pow(i.toDouble() / BANDS)+ (hz / binHz).roundToInt().coerceIn(1, N / 2)+ }+ for (i in 1..BANDS) if (edges[i] <= edges[i - 1]) edges[i] = minOf(edges[i - 1] + 1, N / 2)+ val bassBands = (0 until BANDS).count { edges[it + 1] * binHz <= 150.0 }.coerceAtLeast(2)+ val lowBin = (LOW_HZ / binHz).toInt()+ val highBin = (HIGH_HZ / binHz).toInt()++ val onset = FloatArray(frames)+ val bassOnset = FloatArray(frames)+ val lowDb = FloatArray(frames); val midDb = FloatArray(frames); val highDb = FloatArray(frames); val rmsDb = FloatArray(frames)+ var chromaResult: Triple<Array<FloatArray>, FloatArray, Boolean>? = null+ val slices = max(1, threads)+ // Task 0: the chroma; tasks 1..slices: the onset frames, a slice each (from the frame before it, for the flux).+ parallel(slices + 1, slices + 1) { task ->+ if (task == 0) chromaResult = chromaAndCentre(mid, side, cancelled) else {+ val f0 = frames.toLong() * (task - 1) / slices; val f1 = frames.toLong() * task / slices+ val fft = Fft(N)+ val frame = DoubleArray(N); val mag = DoubleArray(N / 2 + 1)+ val re = DoubleArray(N); val im = DoubleArray(N)+ var prev = DoubleArray(BANDS); var cur = DoubleArray(BANDS)+ for (f in max(0, f0.toInt() - 1) until f1.toInt()) {+ if (f and 1023 == 0 && cancelled()) throw InterruptedException("analysis cancelled")+ val start = f * HOP+ for (i in 0 until N) frame[i] = (mid.getOrElse(start + i) { 0f }) * win[i]+ fft.magnitudes(frame, mag, re, im)+ var low = 0.0; var mi = 0.0; var hi = 0.0+ for (k in 1..N / 2) {+ val p = mag[k] * mag[k]+ when { k < lowBin -> low += p; k < highBin -> mi += p; else -> hi += p }+ }+ // Power to dB of a full-scale sine ~ 0 dB (Hann window sum N/2, one-sided spectrum).+ val norm = (N / 4.0).pow(2) / 2+ lowDb[f] = db(low / norm); midDb[f] = db(mi / norm); highDb[f] = db(hi / norm); rmsDb[f] = db((low + mi + hi) / norm)+ for (b in 0 until BANDS) {+ var s = 0.0+ for (k in edges[b] until edges[b + 1]) s += mag[k]+ cur[b] = ln(1.0 + 1000.0 * s / (N / 4.0))+ }+ if (f > 0 && f >= f0) {+ var all = 0.0; var bass = 0.0+ for (b in 0 until BANDS) {+ val d = cur[b] - prev[b]+ if (d > 0) { all += d; if (b < bassBands) bass += d }+ }+ onset[f] = all.toFloat(); bassOnset[f] = bass.toFloat()+ }+ val t = prev; prev = cur; cur = t+ }+ }+ }++ val (chroma, centre, stereo) = chromaResult!!+ return Features(onset, bassOnset, lowDb, midDb, highDb, rmsDb, chroma, centre, stereo, mid.size.toDouble() / RATE)+ }++ private fun chromaAndCentre(mid: FloatArray, side: FloatArray?, cancelled: () -> Boolean): Triple<Array<FloatArray>, FloatArray, Boolean> {+ val fft = Fft(CHROMA_N)+ val win = hann(CHROMA_N)+ val frames = max(1, (mid.size - CHROMA_N) / CHROMA_HOP + 1)+ val binHz = RATE.toDouble() / CHROMA_N+ // Pitch class of each bin from 65 Hz (C2) to 2.1 kHz (C7); weights favour the bins near a semitone centre.+ val first = (65.0 / binHz).toInt(); val last = (2100.0 / binHz).toInt()+ val pc = IntArray(last + 1) { k -> if (k < first) -1 else Math.floorMod((12 * log2(k * binHz / 440.0)).roundToInt() + 9, 12) }+ val voiceLo = (300 / binHz).toInt(); val voiceHi = (3400 / binHz).toInt()+ val chroma = Array(frames) { FloatArray(12) }+ val centre = FloatArray(frames)+ val frame = DoubleArray(CHROMA_N); val mag = DoubleArray(CHROMA_N / 2 + 1); val smag = DoubleArray(CHROMA_N / 2 + 1)+ val re = DoubleArray(CHROMA_N); val im = DoubleArray(CHROMA_N)+ var sideEnergy = 0.0; var midEnergy = 0.0+ for (f in 0 until frames) {+ if (f and 255 == 0 && cancelled()) throw InterruptedException("analysis cancelled")+ val start = f * CHROMA_HOP+ for (i in 0 until CHROMA_N) frame[i] = mid.getOrElse(start + i) { 0f } * win[i]+ fft.magnitudes(frame, mag, re, im)+ val c = chroma[f]+ for (k in first..last) c[pc[k]] += ln(1.0 + 100.0 * mag[k] / (CHROMA_N / 4.0)).toFloat()+ var m = 0.0+ for (k in voiceLo..voiceHi) m += mag[k] * mag[k]+ midEnergy += m+ if (side != null) {+ for (i in 0 until CHROMA_N) frame[i] = side.getOrElse(start + i) { 0f } * win[i]+ fft.magnitudes(frame, smag, re, im)+ var s = 0.0+ for (k in voiceLo..voiceHi) s += smag[k] * smag[k]+ sideEnergy += s+ centre[f] = (db(m + 1e-9) - db(s + 1e-9)).toFloat()+ }+ val norm = c.max()+ if (norm > 0) for (i in 0 until 12) c[i] /= norm+ }+ // A side signal 30 dB under the mid one is a mono master: no centre measure.+ val stereo = side != null && sideEnergy > midEnergy * 1e-3+ return Triple(chroma, centre, stereo)+ }++ fun db(power: Double): Float = (10 * log10(power + 1e-12)).toFloat().coerceAtLeast(-120f)+ }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/Fft.kt b/base/automix/src/main/kotlin/app/myspoti/automix/Fft.ktnew file mode 100644index 00000000..9bd1e571--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/Fft.kt@@ -0,0 +1,58 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.sin++/** In-place radix-2 complex FFT of size [n] (a power of two), tables built once. */+internal class Fft(val n: Int) {+ init { require(n >= 2 && n and (n - 1) == 0) { "FFT size $n" } }++ private val cosT = DoubleArray(n / 2) { cos(2 * PI * it / n) }+ private val sinT = DoubleArray(n / 2) { -sin(2 * PI * it / n) }+ private val rev = IntArray(n).also { r ->+ val bits = Integer.numberOfTrailingZeros(n)+ for (i in 0 until n) r[i] = Integer.reverse(i) ushr (32 - bits)+ }++ fun transform(re: DoubleArray, im: DoubleArray) {+ for (i in 0 until n) {+ val j = rev[i]+ if (j > i) {+ var t = re[i]; re[i] = re[j]; re[j] = t+ t = im[i]; im[i] = im[j]; im[j] = t+ }+ }+ var size = 2+ while (size <= n) {+ val half = size / 2+ val step = n / size+ var start = 0+ while (start < n) {+ var k = 0+ for (j in start until start + half) {+ val wr = cosT[k]; val wi = sinT[k]+ val l = j + half+ val tr = re[l] * wr - im[l] * wi+ val ti = re[l] * wi + im[l] * wr+ re[l] = re[j] - tr; im[l] = im[j] - ti+ re[j] += tr; im[j] += ti+ k += step+ }+ start += size+ }+ size *= 2+ }+ }++ /** Magnitudes of bins 0..n/2 of a real frame (Hann-windowed by the caller or not). */+ fun magnitudes(frame: DoubleArray, out: DoubleArray, re: DoubleArray = DoubleArray(n), im: DoubleArray = DoubleArray(n)) {+ System.arraycopy(frame, 0, re, 0, n)+ java.util.Arrays.fill(im, 0.0)+ transform(re, im)+ for (k in 0..n / 2) out[k] = Math.sqrt(re[k] * re[k] + im[k] * im[k])+ }+}++internal fun hann(n: Int) = DoubleArray(n) { 0.5 - 0.5 * cos(2 * PI * it / n) }diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/MixPlan.kt b/base/automix/src/main/kotlin/app/myspoti/automix/MixPlan.ktnew file mode 100644index 00000000..431df2cd--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/MixPlan.kt@@ -0,0 +1,104 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.PI+import kotlin.math.cos+import kotlin.math.pow+import kotlin.math.sin++/**+ * A transition from song A (going out) to song B (coming in).+ *+ * At A's time [exitA] B is heard from its time [entryB]; the mix lasts [length] seconds of A's+ * timeline, after which A stops. B plays at [speedB] (its beats then fall on A's) from the start of+ * the mix until A is gone, then goes back to 1 over [restoreSec]. [levels] gives both decks' gains+ * and EQ along the mix.+ */+class MixPlan(+ val style: Style,+ val exitA: Double,+ val entryB: Double,+ val length: Double,+ val speedB: Double,+ /** Beats of A in the mix (0 for an unsynced fade). */+ val beats: Int,+ /** Where the bass moves from A to B, 0..1 along the mix (a downbeat). */+ val swapAt: Double,+ /** One beat of A as a share of the mix (bass swap and cut ramps last a beat). */+ val beatShare: Double,+ /** B's gain during the mix (B louder than A is brought down to A's level, then back to 1 after). */+ val matchB: Double,+ val restoreSec: Double,+ val note: String,+) {+ enum class Style {+ /** Beat-matched blend over several bars with a bass swap. */+ BLEND,+ /** Beat-aligned swap on a downbeat: B comes in whole, A leaves within a beat or two. */+ CUT,+ /** No common beat: an equal-power fade, A's top end cut on its way out. */+ FADE,+ }++ val synced get() = style != Style.FADE+ val endA get() = exitA + length+ /** B's time when A is gone. */+ val resumeB get() = entryB + length * speedB++ /** Gains (linear) and band levels (dB, 0 = untouched) of both decks. */+ class Levels(val gainA: Double, val gainB: Double, val lowA: Double, val lowB: Double, val highA: Double, val highB: Double)++ /** Levels at [x] (0 = start of the mix, 1 = A gone). Outside the mix: A alone before, B alone after. */+ fun levels(x: Double): Levels {+ if (x <= 0) return Levels(1.0, 0.0, 0.0, KILL, 0.0, 0.0)+ if (x >= 1) return Levels(0.0, matchB, KILL, 0.0, 0.0, 0.0)+ return when (style) {+ Style.BLEND -> {+ val s = swapAt+ val ramp = beatShare.coerceAtLeast(0.01)+ // B's fader rises over the first half with its bass cut, sitting under A.+ val gB = if (x < s) 0.8 * sin(PI / 2 * (x / s)).pow(0.8) else 0.8 + 0.2 * ((x - s) / (1 - s)).coerceIn(0.0, 1.0)+ // A stays up until the swap, then fades out (equal power) over the second half.+ val gA = if (x < s) 1.0 else cos(PI / 2 * ((x - s) / (1 - s)).coerceIn(0.0, 1.0))+ // The bass swap: B's low band comes in as A's goes, over one beat from the swap.+ val swap = ((x - s) / ramp).coerceIn(0.0, 1.0)+ val lowA = KILL * smooth(swap); val lowB = KILL * (1 - smooth(swap))+ // A's top end goes down over the last quarter (as a DJ closes A's high EQ).+ val highA = if (x > 0.7) -15.0 * ((x - 0.7) / 0.3) else 0.0+ // B's top end slightly held back at first so A's hi-hats lead until the middle.+ val highB = if (x < s) -6.0 * (1 - x / s) else 0.0+ Levels(gA, gB * matchB, lowA, lowB, highA, highB)+ }+ Style.CUT -> {+ // B whole from the downbeat; A's bass gone at once, A out over the mix.+ val gA = cos(PI / 2 * x)+ // A few tens of ms of ramp on B: no click if the downbeat lands between two steps.+ Levels(gA, matchB * (x / 0.04).coerceAtMost(1.0), KILL, 0.0, -10.0 * x, 0.0)+ }+ Style.FADE -> {+ val gA = cos(PI / 2 * x); val gB = sin(PI / 2 * x)+ Levels(gA, gB * matchB, KILL * smooth(((x - 0.4) / 0.3).coerceIn(0.0, 1.0)), KILL * (1 - smooth((x / 0.6).coerceIn(0.0, 1.0))), -12.0 * x, 0.0)+ }+ }+ }++ /** B's speed at its own time [tB] after the mix: back to 1 over [restoreSec] seconds of B. */+ fun speedAfter(tB: Double): Double {+ val past = tB - resumeB+ if (past <= 0) return speedB+ if (past >= restoreSec || restoreSec <= 0) return 1.0+ return speedB + (1 - speedB) * smooth(past / restoreSec)+ }++ /** B's gain after the mix: [matchB] back to 1 over 8 s of B. */+ fun gainAfter(tB: Double): Double = matchB + (1 - matchB) * smooth(((tB - resumeB) / 8.0).coerceIn(0.0, 1.0))++ override fun toString() = String.format("%s A %.2f→%.2f (%.1f s, %d beats) B from %.2f ×%.4f swap %.2f gainB %.2f | %s",+ style, exitA, endA, length, beats, entryB, speedB, swapAt, matchB, note)++ companion object {+ /** A band "off", in dB. */+ const val KILL = -30.0+ private fun smooth(t: Double) = t * t * (3 - 2 * t)+ }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/MixPlanner.kt b/base/automix/src/main/kotlin/app/myspoti/automix/MixPlanner.ktnew file mode 100644index 00000000..f7c515fc--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/MixPlanner.kt@@ -0,0 +1,284 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import kotlin.math.abs+import kotlin.math.ceil+import kotlin.math.floor+import kotlin.math.ln+import kotlin.math.max+import kotlin.math.min+import kotlin.math.pow+import kotlin.math.roundToInt++/**+ * Picks the transition between two analysed songs: where A goes out, where B comes in, how long the+ * mix lasts and its style.+ *+ * Rules (from DJ practice, Mixxx's intro/outro model and Zehren et al. 2020 on cue points):+ * - Cues sit on downbeats, preferably on 8-bar phrase starts; the mix lasts 2 to 16 bars.+ * - A goes out in its outro: late, where its level drops, without cutting a loud part short.+ * - B comes in at its first downbeat (or a later phrase start within its intro), and its first big+ * change (the drop, the first verse) should land when A is gone.+ * - Beat-matching only with clear grids on both sides and a tempo change within [Settings.maxStretch]+ * (half and double time count); keys a fifth apart or more and two voices at once shorten the mix.+ * - Otherwise a cut on A's downbeat (A has a grid) or a plain fade.+ */+object MixPlanner {+ class Settings(+ /** Largest tempo change applied to B, as a share (0.08 = 8 %). */+ val maxStretch: Double = 0.08,+ /** Longest mix, seconds. */+ val maxSec: Double = 32.0,+ /** Fade length without a beat grid, seconds. */+ val fadeSec: Double = 6.0,+ )++ fun plan(a: TrackAnalysis, b: TrackAnalysis, s: Settings = Settings()): MixPlan {+ val ga = a.tail; val gb = b.head+ val match = if (ga != null && gb != null && !a.tailDown.isNaN() && !b.headDown.isNaN()) tempoMatch(ga.bpm, gb.bpm) else null+ if (match != null && abs(match.speed - 1) <= s.maxStretch) {+ blend(a, b, ga!!, gb!!, match, s)?.let { return it }+ }+ if (ga != null && !a.tailDown.isNaN()) cut(a, b, ga, gb, match, s)?.let { return it }+ return fade(a, b, s, if (match == null) "no beat grid" else String.format("tempo %.1f → %.1f too far", ga!!.bpm, gb!!.bpm))+ }++ /**+ * The mix for a skip (the user pressed Next at A's time [now]; the next song can be heard [lead]+ * seconds later at the earliest): short, from A's next downbeat. Two bars beat-matched (one when+ * two are over 5.5 s), else a cut on the downbeat, else a 2.5 s fade. Either analysis may be missing.+ */+ fun planSkip(a: TrackAnalysis?, b: TrackAnalysis?, now: Double, lead: Double, s: Settings = Settings()): MixPlan {+ val from = now + lead+ val ga = a?.let { gridAt(it, from) }+ val gb = b?.head?.takeIf { !b.headDown.isNaN() }+ val entry = if (b != null && gb != null) firstDown(gb, b.headDown, b.contentStart - 0.08) else b?.contentStart ?: 0.0+ val gain = if (a != null && b != null) matchGain(a, b) else 1.0+ if (a != null && ga != null) {+ val exit = firstDown(ga.first, ga.second, from)+ val barA = 4 * ga.first.period+ val m = gb?.let { tempoMatch(ga.first.bpm, it.bpm) }+ if (m != null && abs(m.speed - 1) <= s.maxStretch) {+ val bars = if (2 * barA <= 5.5) 2 else 1+ return MixPlan(MixPlan.Style.BLEND, exit, entry, bars * barA, m.speed, bars * 4, 0.5, 1.0 / (bars * 4), gain,+ restore(m.speed, gb.period), String.format("skip: %d bar(s) from A's downbeat at %.1f, %.1f→%.1f BPM", bars, exit, ga.first.bpm, gb.bpm))+ }+ return MixPlan(MixPlan.Style.CUT, exit, entry, 2 * ga.first.period, 1.0, 2, 0.0, 0.5, gain, 0.0,+ String.format("skip: cut on A's downbeat at %.1f", exit))+ }+ return MixPlan(MixPlan.Style.FADE, from, entry, 2.5, 1.0, 0, 0.5, 0.1, gain, 0.0, "skip: fade (no beat grid here)")+ }++ /** The grid (and one of its downbeats) holding at A's time [t]: the tail's near the end, the head's near the start, the one grid of a steady song. */+ private fun gridAt(a: TrackAnalysis, t: Double): Pair<Grid, Double>? {+ val tail = a.tail?.takeIf { !a.tailDown.isNaN() && t >= max(it.first, a.contentEnd - Analyzer.TAIL_SEC) && t < it.last }+ if (tail != null) return tail to a.tailDown+ val head = a.head?.takeIf { !a.headDown.isNaN() && t < it.last && (a.steady || t <= a.contentStart + Analyzer.HEAD_SEC) }+ return head?.let { it to a.headDown }+ }++ /** B's speed so its beat (or half or double beat) falls on A's; [beatsPerB] A beats per B beat. */+ class TempoMatch(val speed: Double, val beatsPerB: Double)++ fun tempoMatch(bpmA: Double, bpmB: Double): TempoMatch =+ listOf(0.5, 1.0, 2.0).map { m -> TempoMatch(bpmA / (bpmB * m), m) }.minBy { abs(ln(it.speed)) }++ private class Cue(val time: Double, val score: Double, val why: String)++ private fun blend(a: TrackAnalysis, b: TrackAnalysis, ga: Grid, gb: Grid, m: TempoMatch, s: Settings): MixPlan? {+ val barA = 4 * ga.period+ val exits = exitCues(a, ga)+ val entries = entryCues(b, gb)+ if (exits.isEmpty() || entries.isEmpty()) return null+ val keyPenalty = keyDistance(a, b)+ var best: MixPlan? = null; var bestScore = Double.NEGATIVE_INFINITY+ for (e in exits) for (n in entries) for (bars in intArrayOf(16, 8, 4, 2)) {+ val len = bars * barA+ if (len > s.maxSec + 0.01 && bars > 2) continue+ // A must still be playing (its audible part) for most of the mix.+ if (e.time + len > musicEnd(a) + barA) continue+ val bEnd = n.time + len * m.speed+ if (bEnd > b.contentEnd - 10) continue+ var score = e.score + n.score+ // Long mixes suit songs in compatible keys with steady grids; clashing keys favour short ones.+ score += when (bars) { 16 -> 1.0; 8 -> 1.5; 4 -> 0.8; else -> 0.0 } - keyPenalty * bars / 8.0+ // Strong tempo changes are heard: shorter mixes then.+ score -= abs(m.speed - 1) / 0.08 * bars / 16.0+ // A's loud bars after the mix (cut short) and a long silence of A inside the mix.+ score -= loudCut(a, e.time + len) * 0.6+ score -= max(0.0, e.time + len - musicEnd(a)) / barA * 0.5+ // B's change (drop, first verse) when A is gone: a novelty peak near the end of the mix.+ score += changeNear(b, bEnd, 4 * gb.period * m.beatsPerB) * 1.2+ // Two voices at once.+ score -= voiceClash(a, e.time, e.time + len, b, n.time, bEnd) * bars / 8.0+ // Two loud songs over each other for long: worse than a loud A over a quiet intro.+ score -= max(0.0, levelAt(a, e.time, e.time + len) + levelAt(b, n.time, bEnd)) * 0.08 * bars / 8.0+ if (score > bestScore) {+ bestScore = score+ val swapBar = (bars / 2.0).let { if (bars >= 4) floor(it) else it }+ best = MixPlan(MixPlan.Style.BLEND, e.time, n.time, len, m.speed, bars * 4, swapBar / bars, 1.0 / (bars * 4),+ matchGain(a, b), restore(m.speed, gb.period), String.format("%s, %s, %.1f→%.1f BPM, key %s, score %.1f", e.why, n.why, ga.bpm, gb.bpm, keyText(keyPenalty), score))+ }+ }+ return best+ }++ private fun cut(a: TrackAnalysis, b: TrackAnalysis, ga: Grid, gb: Grid?, m: TempoMatch?, s: Settings): MixPlan? {+ val exits = exitCues(a, ga)+ if (exits.isEmpty()) return null+ // B starts on its first downbeat when it has one, else where its audio starts.+ val entry = if (gb != null && !b.headDown.isNaN()) entryCues(b, gb).maxBy { it.score }.time else b.contentStart+ val e = exits.maxBy { it.score - loudCut(a, it.time + 2 * ga.period) * 0.6 }+ val synced = m != null && abs(m.speed - 1) <= s.maxStretch+ val speed = if (synced) m!!.speed else 1.0+ return MixPlan(MixPlan.Style.CUT, e.time, entry, 2 * ga.period, speed, 2, 0.0, 0.5, matchGain(a, b),+ if (synced) restore(speed, gb!!.period) else 0.0, "cut on ${e.why}" + if (synced) "" else ", tempos apart")+ }++ private fun fade(a: TrackAnalysis, b: TrackAnalysis, s: Settings, why: String): MixPlan {+ val len = min(s.fadeSec, (a.contentEnd - a.contentStart) / 4)+ val exit = max(a.contentStart, musicEnd(a) - len * 0.6)+ return MixPlan(MixPlan.Style.FADE, exit, b.contentStart, len, 1.0, 0, 0.5, 0.1, matchGain(a, b), 0.0, "fade: $why")+ }++ /**+ * A's exit cues: downbeats of its tail grid in its last minute and a half, scored for phrase starts,+ * a level drop after them (the outro) and being late in the song.+ */+ private fun exitCues(a: TrackAnalysis, g: Grid): List<Cue> {+ val end = musicEnd(a)+ val from = max(a.contentStart + (end - a.contentStart) * 0.5, end - 90)+ val to = end - 2 * g.period+ val out = ArrayList<Cue>()+ val loud = loudLevel(a)+ var t = firstDown(g, a.tailDown, from)+ while (t <= to) {+ val bar = a.barAt(t + 0.05)+ var score = 0.0; var why = ""+ if (bar >= 0) {+ val pos = a.phrasePos(bar)+ score += when (pos) { 0 -> 2.0; 4 -> 1.0; else -> 0.0 }+ score += (a.barNovelty.getOrElse(bar) { 0f }).toDouble().coerceAtMost(3.0) * 0.4+ val before = levelBars(a, bar - 4, bar); val after = levelBars(a, bar, bar + 4)+ if (before - after > 3) { score += 1.5; why = "outro" }+ why = (if (pos == 0) "A phrase" else if (pos == 4) "A half phrase" else "A bar") + (if (why.isEmpty()) "" else " ($why)") ++ String.format(" at %.1f", t)+ // Already quiet (the song's end fading): fine, but no later than needed.+ if (after < loud - 12) score -= 0.5+ } else why = String.format("A bar at %.1f", t)+ // Later is better: less of A is lost.+ score += (t - from) / max(1.0, to - from) * 1.5+ out.add(Cue(t, score, why))+ t += 4 * g.period+ }+ return out+ }++ /** B's entry cues: its first downbeat, and phrase starts in its first 32 s (each skip costs). */+ private fun entryCues(b: TrackAnalysis, g: Grid): List<Cue> {+ val first = firstDown(g, b.headDown, b.contentStart - 0.08)+ val out = arrayListOf(Cue(first, 1.0, String.format("B first downbeat %.1f", first)))+ var t = first + 4 * g.period+ while (t < b.contentStart + 32 && t < b.contentEnd / 3) {+ val bar = b.barAt(t + 0.05)+ if (bar >= 0 && b.phrasePos(bar) == 0) {+ // Skipping a quiet intro is fine, skipping music costs.+ val skipped = t - b.contentStart+ val quiet = levelBars(b, 0, bar) < loudLevel(b) - 9+ out.add(Cue(t, 1.5 - skipped / (if (quiet) 30.0 else 12.0), String.format("B phrase %.1f", t)))+ }+ t += 4 * g.period+ }+ return out+ }++ /**+ * Where A's music really ends: the end of its last bar within 15 dB of its loud level (a long+ * fade-out or a quiet tail is left to the mix).+ */+ fun musicEnd(a: TrackAnalysis): Double {+ val loud = loudLevel(a)+ val last = (a.bars - 1 downTo 0).firstOrNull { a.barDb[it] > loud - 15 } ?: return a.contentEnd+ return min(a.contentEnd, a.barEnd(last))+ }++ /** The first downbeat of the grid at or after [from]. */+ fun firstDown(g: Grid, down: Double, from: Double): Double {+ val bar = 4 * g.period+ return down + ceil((from - down) / bar - 1e-6) * bar+ }++ private fun loudLevel(a: TrackAnalysis): Double = a.barDb.sorted().let { if (it.isEmpty()) -20.0 else it[(it.size * 0.7).toInt()].toDouble() }++ private fun levelBars(a: TrackAnalysis, from: Int, to: Int): Double {+ var s = 0.0; var c = 0+ for (i in max(0, from) until min(a.bars, to)) { s += 10.0.pow(a.barDb[i] / 10.0); c++ }+ return if (c == 0) -60.0 else 10 * Math.log10(s / c)+ }++ private fun levelAt(a: TrackAnalysis, from: Double, to: Double): Double {+ val i = max(0, a.barAt(from)); val j = max(i + 1, a.barAt(to))+ return levelBars(a, i, j) - loudLevel(a)+ }++ /** Bars of A at full level that the mix ending at [end] leaves out. */+ private fun loudCut(a: TrackAnalysis, end: Double): Double {+ val loud = loudLevel(a)+ var n = 0.0+ for (i in max(0, a.barAt(end) + 1) until a.bars) if (a.barDb[i] > loud - 4) n += 1.0+ return n+ }++ /** A large novelty (a new section) within a bar of [t] in B. */+ private fun changeNear(b: TrackAnalysis, t: Double, bar: Double): Double {+ val i = b.barAt(t + 0.05)+ if (i < 0) return 0.0+ val mean = b.barNovelty.average().takeIf { it > 0 } ?: return 0.0+ var top = 0.0+ for (j in max(0, i - 1)..min(b.bars - 1, i + 1)) {+ if (abs(b.barStarts[j] - t) <= bar * 1.01) top = max(top, b.barNovelty[j] / mean)+ }+ return (top - 1).coerceIn(0.0, 2.0)+ }++ /** Both songs' voice bands high at once (stereo songs only): 0 (none) to 1 (all along). */+ private fun voiceClash(a: TrackAnalysis, a0: Double, a1: Double, b: TrackAnalysis, b0: Double, b1: Double): Double {+ if (!a.stereo || !b.stereo) return 0.0+ fun share(x: TrackAnalysis, t0: Double, t1: Double): Double {+ val i = max(0, x.barAt(t0)); val j = max(i + 1, x.barAt(t1))+ var c = 0; var v = 0+ for (k in i until min(x.bars, j)) { c++; if (x.barCentre[k] > 3f) v++ }+ return if (c == 0) 0.0 else v.toDouble() / c+ }+ return share(a, a0, a1) * share(b, b0, b1)+ }++ /** 0: same or neighbouring Camelot keys; 1: two steps; 2: further. Unknown or weak keys: 0.5. */+ fun keyDistance(a: TrackAnalysis, b: TrackAnalysis): Double {+ if (a.key < 0 || b.key < 0 || a.keyStrength < 0.5 || b.keyStrength < 0.5) return 0.5+ val (na, la) = camelot(a.key); val (nb, lb) = camelot(b.key)+ val d = min(Math.floorMod(na - nb, 12), Math.floorMod(nb - na, 12)) + if (la != lb) 1 else 0+ return when { d <= 1 -> 0.0; d == 2 -> 1.0; else -> 2.0 }+ }++ /** Camelot number (1..12) and letter (true = B, major). */+ fun camelot(key: Int): Pair<Int, Boolean> {+ val major = key < 12+ val tonic = if (major) key else Math.floorMod(key - 12 + 3, 12)+ return (Math.floorMod(tonic * 7 + 7, 12) + 1) to major+ }++ private fun keyText(d: Double) = when (d) { 0.0 -> "compatible"; 0.5 -> "unsure"; 1.0 -> "near"; else -> "clash" }++ /** B louder than A by more than 1.5 dB is brought down to A's level during the mix (down to −6 dB). */+ private fun matchGain(a: TrackAnalysis, b: TrackAnalysis): Double {+ val d = a.loudnessDb - b.loudnessDb+ return if (d < -1.5) 10.0.pow(max(d, -6.0) / 20) else 1.0+ }++ /** B's speed back to 1: about 0.3 % per beat, at least 8 beats. */+ private fun restore(speed: Double, period: Double): Double {+ val beats = max(8.0, abs(speed - 1) / 0.003)+ return if (abs(speed - 1) < 1e-4) 0.0 else (beats * period).let { (it * 10).roundToInt() / 10.0 }+ }+}diff --git a/base/automix/src/main/kotlin/app/myspoti/automix/TrackAnalysis.kt b/base/automix/src/main/kotlin/app/myspoti/automix/TrackAnalysis.ktnew file mode 100644index 00000000..fd48f2f0--- /dev/null+++ b/base/automix/src/main/kotlin/app/myspoti/automix/TrackAnalysis.kt@@ -0,0 +1,96 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import java.io.DataInputStream+import java.io.DataOutputStream+import java.io.InputStream+import java.io.OutputStream++/**+ * What AutoMix knows of a song. Times in seconds of the song's own timeline.+ *+ * Bars are 4 beats: bar i starts at [barStarts][i]. Per bar: level (dB), low-band level (dB),+ * high-band level (dB), centre (voice-band mid over side, dB above the song's median) and novelty+ * (how much the 4 bars from here differ from the 4 before). [phraseOffset]: bars i with+ * (i - phraseOffset) % 8 == 0 start 8-bar phrases.+ */+class TrackAnalysis(+ val duration: Double,+ val contentStart: Double,+ val contentEnd: Double,+ val loudnessDb: Double,+ /** 0..11 major (C = 0), 12..23 minor (A minor = 21); -1 unknown. */+ val key: Int,+ val keyStrength: Double,+ val bpm: Double,+ /** The whole song follows one grid (most produced music). */+ val steady: Boolean,+ /** Grids fitted on the song's first and last minute or so (beat-matching uses these). */+ val head: Grid?,+ val tail: Grid?,+ val beats: DoubleArray,+ val barStarts: DoubleArray,+ val barDb: FloatArray,+ val barLowDb: FloatArray,+ val barHighDb: FloatArray,+ val barCentre: FloatArray,+ val barNovelty: FloatArray,+ val phraseOffset: Int,+ val stereo: Boolean,+ /** A downbeat of the [head] and of the [tail] grid (NaN without the grid): bars there start every 4 beats from it. */+ val headDown: Double,+ val tailDown: Double,+) {+ val bars get() = barStarts.size++ /** The bar containing [t], or -1 before the first. */+ fun barAt(t: Double): Int {+ var lo = 0; var hi = bars - 1; var ans = -1+ while (lo <= hi) { val m = (lo + hi) ushr 1; if (barStarts[m] <= t) { ans = m; lo = m + 1 } else hi = m - 1 }+ return ans+ }++ fun barEnd(i: Int): Double = if (i + 1 < bars) barStarts[i + 1] else contentEnd++ /** 0 on an 8-bar phrase start, 4 on its middle (a 4-bar start), else other. */+ fun phrasePos(i: Int): Int = Math.floorMod(i - phraseOffset, 8)++ fun write(out: OutputStream) {+ val o = DataOutputStream(out)+ o.writeInt(VERSION)+ o.writeDouble(duration); o.writeDouble(contentStart); o.writeDouble(contentEnd); o.writeDouble(loudnessDb)+ o.writeInt(key); o.writeDouble(keyStrength); o.writeDouble(bpm); o.writeBoolean(steady)+ writeGrid(o, head); writeGrid(o, tail)+ o.writeInt(beats.size); beats.forEach(o::writeDouble)+ o.writeInt(bars); barStarts.forEach(o::writeDouble)+ for (a in listOf(barDb, barLowDb, barHighDb, barCentre, barNovelty)) a.forEach(o::writeFloat)+ o.writeInt(phraseOffset); o.writeBoolean(stereo); o.writeDouble(headDown); o.writeDouble(tailDown)+ o.flush()+ }++ companion object {+ const val VERSION = 4_02++ fun read(input: InputStream): TrackAnalysis? {+ val i = DataInputStream(input)+ if (i.readInt() != VERSION) return null+ val duration = i.readDouble(); val cs = i.readDouble(); val ce = i.readDouble(); val loud = i.readDouble()+ val key = i.readInt(); val ks = i.readDouble(); val bpm = i.readDouble(); val steady = i.readBoolean()+ val head = readGrid(i); val tail = readGrid(i)+ val beats = DoubleArray(i.readInt()) { i.readDouble() }+ val n = i.readInt()+ val starts = DoubleArray(n) { i.readDouble() }+ val arrays = List(5) { FloatArray(n) { i.readFloat() } }+ return TrackAnalysis(duration, cs, ce, loud, key, ks, bpm, steady, head, tail, beats, starts,+ arrays[0], arrays[1], arrays[2], arrays[3], arrays[4], i.readInt(), i.readBoolean(), i.readDouble(), i.readDouble())+ }++ private fun writeGrid(o: DataOutputStream, g: Grid?) {+ o.writeBoolean(g != null)+ if (g != null) { o.writeDouble(g.first); o.writeDouble(g.period); o.writeInt(g.count); o.writeDouble(g.steadiness); o.writeDouble(g.driftMs); o.writeDouble(g.contrast) }+ }++ private fun readGrid(i: DataInputStream): Grid? =+ if (!i.readBoolean()) null else Grid(i.readDouble(), i.readDouble(), i.readInt(), i.readDouble(), i.readDouble(), i.readDouble())+ }+}diff --git a/base/automix/src/test/kotlin/app/myspoti/automix/AutoMixTest.kt b/base/automix/src/test/kotlin/app/myspoti/automix/AutoMixTest.ktnew file mode 100644index 00000000..e1d0512e--- /dev/null+++ b/base/automix/src/test/kotlin/app/myspoti/automix/AutoMixTest.kt@@ -0,0 +1,127 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import org.junit.Assert.assertEquals+import org.junit.Assert.assertTrue+import org.junit.Test+import kotlin.math.PI+import kotlin.math.abs+import kotlin.math.exp+import kotlin.math.sin++class AutoMixTest {+ /** A drum loop at [bpm]: kick on 1 and 3 (low), snare on 2 and 4 (noise), hat on eighths, a bass note per bar. */+ private fun loop(bpm: Double, seconds: Double, offset: Double = 0.5, seed: Long = 1): FloatArray {+ val sr = Features.RATE+ val x = FloatArray((seconds * sr).toInt())+ val rnd = java.util.Random(seed)+ val beat = 60 / bpm+ var k = 0+ while (offset + k * beat / 2 < seconds) {+ val t0 = offset + k * beat / 2+ val i0 = (t0 * sr).toInt()+ val inBar = k % 8+ for (i in 0 until (0.25 * sr).toInt()) {+ if (i0 + i >= x.size) break+ val t = i.toDouble() / sr+ var v = 0.0+ if (inBar == 0 || inBar == 4) v += 0.8 * sin(2 * PI * (50 + 80 * exp(-t * 30)) * t) * exp(-t * 12)+ if (inBar == 2 || inBar == 6) v += 0.4 * (rnd.nextDouble() * 2 - 1) * exp(-t * 25)+ v += 0.08 * (rnd.nextDouble() * 2 - 1) * exp(-t * 80)+ x[i0 + i] += v.toFloat()+ }+ k+++ }+ // A bass note changing every bar (pitch classes C, F, G, C).+ val notes = doubleArrayOf(65.4, 87.3, 98.0, 65.4)+ for (i in x.indices) {+ val t = i.toDouble() / sr - offset+ if (t < 0) continue+ val bar = (t / (4 * beat)).toInt()+ x[i] += (0.15 * sin(2 * PI * notes[bar % 4] * t)).toFloat()+ }+ return x+ }++ @Test fun gridFindsTempoAndPhase() {+ val a = Analyzer.analyze(loop(124.0, 60.0), null)+ assertEquals(124.0, a.bpm, 0.3)+ assertTrue("steady", a.steady)+ val g = a.tail!!+ // A beat every 60/124 s from 0.5 s (onset lead aside, within 25 ms).+ val k = Math.round(g.index(0.5 + 40 * 60 / 124.0)).toInt()+ assertEquals(0.5 + 40 * 60 / 124.0, g.beat(k), 0.035)+ }++ @Test fun blendBetweenNearTempos() {+ val a = Analyzer.analyze(loop(124.0, 90.0, seed = 1), null)+ val b = Analyzer.analyze(loop(122.0, 90.0, offset = 0.3, seed = 2), null)+ val p = MixPlanner.plan(a, b)+ assertEquals(MixPlan.Style.BLEND, p.style)+ assertEquals(a.tail!!.bpm / b.head!!.bpm, p.speedB, 0.01)+ assertTrue("A goes out late", p.exitA > 45)+ assertTrue("mix inside A", p.endA <= a.contentEnd + 4 * 60 / 124.0)+ // B comes in on one of its beats.+ val gb = b.head!!+ assertEquals(0.0, abs(gb.index(p.entryB) - Math.round(gb.index(p.entryB))), 0.02)+ }++ @Test fun farTemposAreNotStretched() {+ val a = Analyzer.analyze(loop(124.0, 90.0), null)+ val b = Analyzer.analyze(loop(100.0, 90.0, seed = 3), null)+ val p = MixPlanner.plan(a, b)+ assertTrue(p.style != MixPlan.Style.BLEND)+ assertEquals(1.0, p.speedB, 1e-9)+ }++ @Test fun halfTimeCountsAsAMatch() {+ val m = MixPlanner.tempoMatch(140.0, 71.0)+ assertEquals(140.0 / 142.0, m.speed, 1e-9)+ assertEquals(2.0, m.beatsPerB, 0.0)+ }++ @Test fun levelsAreContinuous() {+ for (style in MixPlan.Style.values()) {+ val p = MixPlan(style, 100.0, 1.0, 16.0, 1.02, 32, 0.5, 1.0 / 32, 0.8, 4.0, "")+ var prev = p.levels(0.0)+ var x = 0.0+ while (x <= 1.0) {+ val l = p.levels(x)+ assertTrue("$style gainA step at $x", abs(l.gainA - prev.gainA) < 0.06)+ assertTrue("$style gainB step at $x", abs(l.gainB - prev.gainB) < if (style == MixPlan.Style.CUT) 0.045 else 0.06)+ assertTrue("$style power at $x", l.gainA * l.gainA + l.gainB * l.gainB < 1.7)+ prev = l+ x += 0.002+ }+ assertEquals(0.0, p.levels(1.0).gainA, 1e-9)+ assertEquals(1.02, p.speedAfter(p.resumeB), 1e-9)+ assertEquals(1.0, p.speedAfter(p.resumeB + 5), 1e-9)+ }+ }++ @Test fun camelot() {+ assertEquals(8 to true, MixPlanner.camelot(0)) // C major 8B+ assertEquals(8 to false, MixPlanner.camelot(21)) // A minor 8A+ assertEquals(9 to true, MixPlanner.camelot(7)) // G major 9B+ }++ @Test fun analysisRoundTrip() {+ val a = Analyzer.analyze(loop(124.0, 40.0), null)+ val out = java.io.ByteArrayOutputStream(); a.write(out)+ val b = TrackAnalysis.read(java.io.ByteArrayInputStream(out.toByteArray()))!!+ assertEquals(a.bpm, b.bpm, 0.0); assertEquals(a.bars, b.bars); assertEquals(a.tailDown, b.tailDown, 0.0)+ }++ @Test fun skipMixStartsOnTheNextDownbeat() {+ val a = Analyzer.analyze(loop(124.0, 90.0, seed = 1), null)+ val b = Analyzer.analyze(loop(122.0, 90.0, offset = 0.3, seed = 2), null)+ val p = MixPlanner.planSkip(a, b, now = 40.0, lead = 0.5)+ assertEquals(MixPlan.Style.BLEND, p.style)+ assertTrue("after the lead", p.exitA >= 40.5)+ assertTrue("within a bar", p.exitA < 40.5 + 4 * 60 / 124.0 + 0.01)+ val g = a.tail!!+ assertEquals(0.0, Math.floorMod(Math.round(g.index(p.exitA) - g.index(a.tailDown)).toInt(), 4).toDouble(), 0.0)+ // Nothing known: a short fade.+ assertEquals(MixPlan.Style.FADE, MixPlanner.planSkip(null, null, 40.0, 0.5).style)+ }+}diff --git a/base/automix/src/test/kotlin/app/myspoti/automix/Bench.kt b/base/automix/src/test/kotlin/app/myspoti/automix/Bench.ktnew file mode 100644index 00000000..fd2b3fb3--- /dev/null+++ b/base/automix/src/test/kotlin/app/myspoti/automix/Bench.kt@@ -0,0 +1,195 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import java.io.File+import java.nio.ByteBuffer+import java.nio.ByteOrder+import kotlin.math.abs++/**+ * Desktop bench on real songs (not part of the app). `<dir>` holds `sNN.f32` (stereo float 44.1 kHz,+ * from ffmpeg) and optionally `sNN.beats` (Beat This output: time and beat number per line) as the+ * reference. Commands: `analyze <dir>` prints tempo, beat and downbeat agreement with the reference.+ */+fun main(args: Array<String>) {+ val cmd = args.getOrNull(0) ?: return println("usage: analyze <dir> | plan <dir> <a> <b> [out.f32]")+ val dir = File(args[1])+ when (cmd) {+ "analyze" -> {+ val names = dir.listFiles { f -> f.name.endsWith(".f32") }!!.map { it.nameWithoutExtension }.sorted()+ for (n in names) analyzeOne(dir, n)+ }+ "plan" -> {+ // plan <dir> a,b a,b ... [render]: plans (and renders) each pair; checks beat alignment on the reference beats.+ val pairs = args.drop(2).filter { it.contains(',') }.map { it.split(',').let { p -> p[0] to p[1] } }+ val render = args.contains("render")+ val cache = HashMap<String, TrackAnalysis>()+ for ((x, y) in pairs) {+ val a = cache.getOrPut(x) { analysisOf(dir, x) }; val b = cache.getOrPut(y) { analysisOf(dir, y) }+ val plan = MixPlanner.plan(a, b)+ println("$x→$y " + plan)+ println(" " + alignment(dir, x, y, plan))+ if (render) Render.render(File(dir, "$x.f32"), File(dir, "$y.f32"), plan, File(dir, "mix-$x-$y.wav"))+ }+ }+ "model" -> dir.listFiles { f -> f.name.endsWith(".f32") }!!.map { it.nameWithoutExtension }.sorted().forEach { modelDebug(dir, it) }+ "profile" -> args.drop(2).forEach { profile(dir, it) }+ "tempo" -> dir.listFiles { f -> f.name.endsWith(".f32") }!!.map { it.nameWithoutExtension }.sorted().forEach { tempoDebug(dir, it) }+ else -> println("unknown $cmd")+ }+}++/** Stereo float 44.1 kHz → mid and side at 22.05 kHz (pairs averaged). */+fun loadPcm(file: File): Pair<FloatArray, FloatArray> {+ val bytes = file.readBytes()+ val fb = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).asFloatBuffer()+ val frames = fb.remaining() / 2+ val n = frames / 2+ val mid = FloatArray(n); val side = FloatArray(n)+ for (i in 0 until n) {+ val l0 = fb.get(4 * i); val r0 = fb.get(4 * i + 1); val l1 = fb.get(4 * i + 2); val r1 = fb.get(4 * i + 3)+ mid[i] = (l0 + r0 + l1 + r1) / 4; side[i] = (l0 - r0 + l1 - r1) / 4+ }+ return mid to side+}++/** Beat This, from AUTOMIX_MODEL (the app's asset), or none. */+val model: BeatThis? by lazy { System.getenv("AUTOMIX_MODEL")?.let { BeatThis(File(it).readBytes(), defaultThreads = System.getenv("AUTOMIX_THREADS")?.toInt() ?: 2) } }++fun analysisOf(dir: File, name: String): TrackAnalysis {+ val (mid, side) = loadPcm(File(dir, "$name.f32"))+ return Analyzer.analyze(mid, side, model, threads = System.getenv("AUTOMIX_THREADS")?.toInt() ?: 2)+}++private fun analyzeOne(dir: File, name: String) {+ val (mid, side) = loadPcm(File(dir, "$name.f32"))+ val t0 = System.nanoTime()+ val a = Analyzer.analyze(mid, side, model, threads = System.getenv("AUTOMIX_THREADS")?.toInt() ?: 2)+ val ms = (System.nanoTime() - t0) / 1e6+ val ref = File(dir, "$name.beats").takeIf { it.exists() }?.readLines()?.filter { it.isNotBlank() }?.map { l -> l.trim().split(Regex("\\s+")).let { it[0].toDouble() to it[1].toInt() } }+ val sb = StringBuilder()+ sb.append(String.format("%s %5.0fms bpm %6.2f steady %-5s head %s tail %s key %2d(%.2f) loud %5.1f content %.2f-%.2f phrase %d",+ name, ms, a.bpm, a.steady, g(a.head), g(a.tail), a.key, a.keyStrength, a.loudnessDb, a.contentStart, a.contentEnd, a.phraseOffset))+ if (ref != null) {+ val rb = ref.map { it.first }.toDoubleArray()+ val rd = ref.filter { it.second == 1 }.map { it.first }.toDoubleArray()+ val rbpm = 60.0 / median(DoubleArray(rb.size - 1) { rb[it + 1] - rb[it] })+ val f = fMeasure(a.beats, rb); val off = meanOffset(a.beats, rb)+ val fHalf = fMeasure(a.beats.filterIndexed { i, _ -> i % 2 == 0 }.toDoubleArray(), rb)+ val fDouble = fMeasure(a.beats, rb.flatMapIndexed { i, t -> if (i + 1 < rb.size) listOf(t, (t + rb[i + 1]) / 2) else listOf(t) }.toDoubleArray())+ val fd = fMeasure(a.barStarts, rd)+ val gl = Analyzer.lastGlobal+ if (gl != null) {+ val gb = DoubleArray(gl.count) { gl.beat(it) }+ fun region(x: DoubleArray, a: Double, b: Double) = x.filter { it in a..b }.toDoubleArray()+ val h = a.contentStart to a.contentStart + 75; val t = a.contentEnd - 90 to a.contentEnd+ val mb = Analyzer.lastModelBeats+ sb.append(String.format("\n grid %s q %s gridF head %.2f tail %.2f | modelF head %.2f tail %.2f | grid-vs-model head %.2f tail %.2f",+ g(gl), Analyzer.lastWhole, fMeasure(region(gb, h.first, h.second), region(rb, h.first, h.second)), fMeasure(region(gb, t.first, t.second), region(rb, t.first, t.second)),+ fMeasure(region(mb, h.first, h.second), region(rb, h.first, h.second)), fMeasure(region(mb, t.first, t.second), region(rb, t.first, t.second)),+ fMeasure(region(gb, h.first, h.second), region(mb, h.first, h.second)), fMeasure(region(gb, t.first, t.second), region(mb, t.first, t.second))))+ }+ // Downbeat check on the head and tail grids: the grid's downbeats there against the reference ones.+ fun downF(g: Grid?, d: Double, a0: Double, b0: Double): String {+ if (g == null || d.isNaN()) return "-"+ val k0 = Math.ceil(g.index(a0)).toInt(); val k1 = Math.floor(g.index(b0)).toInt()+ val db = (k0..k1).filter { Math.floorMod(it - Math.round(g.index(d)).toInt(), 4) == 0 }.map { g.beat(it) }.toDoubleArray()+ return String.format("%.2f", fMeasure(db, rd.filter { it in a0..b0 }.toDoubleArray()))+ }+ sb.append("\n head/tail grid downF " + downF(a.head, a.headDown, a.contentStart, a.contentStart + 75) + " / " + downF(a.tail, a.tailDown, a.contentEnd - 90, a.contentEnd))+ sb.append(String.format("\n ref bpm %6.2f | beatF %.2f (half %.2f, double %.2f) offset %+5.1fms | downF %.2f", rbpm, f, fHalf, fDouble, off * 1000, fd))+ }+ println(sb)+}++private fun g(g: Grid?) = if (g == null) "-" else String.format("%.2f/%.2f/%.0fms/%.1f%s", g.bpm, g.steadiness, g.driftMs, g.contrast, if (g.reliable) "✓" else "✗")++fun median(x: DoubleArray): Double { val s = x.sorted(); return s[s.size / 2] }++fun fMeasure(est: DoubleArray, ref: DoubleArray, tol: Double = 0.07): Double {+ if (est.isEmpty() || ref.isEmpty()) return 0.0+ var hit = 0; var j = 0+ val used = BooleanArray(ref.size)+ for (t in est) {+ while (j < ref.size && ref[j] < t - tol) j+++ var k = j+ while (k < ref.size && ref[k] <= t + tol) { if (!used[k]) { used[k] = true; hit++; break }; k++ }+ }+ val p = hit.toDouble() / est.size; val r = hit.toDouble() / ref.size+ return if (p + r == 0.0) 0.0 else 2 * p * r / (p + r)+}++fun meanOffset(est: DoubleArray, ref: DoubleArray, tol: Double = 0.07): Double {+ var s = 0.0; var c = 0+ for (t in est) {+ val r = ref.minByOrNull { abs(it - t) } ?: continue+ if (abs(r - t) <= tol) { s += t - r; c++ }+ }+ return if (c == 0) 0.0 else s / c+}++/** Tempo debugging: autocorrelation peaks of a song's onset envelope. */+fun tempoDebug(dir: File, name: String) {+ val (mid, side) = loadPcm(File(dir, "$name.f32"))+ val f = Features.of(mid, side)+ for ((label, env) in listOf("env" to BeatTracker.envelope(f))) {+ val n = env.size+ val m = env.average()+ val acf = DoubleArray(400) { lag -> var s = 0.0; for (i in 0 until n - lag) s += (env[i] - m) * (env[i + lag] - m); s / (n - lag) }+ val peaks = (20 until 399).filter { acf[it] >= acf[it - 1] && acf[it] >= acf[it + 1] && Features.FPS * 60 / it in 50.0..220.0 }+ .sortedByDescending { acf[it] }.take(8)+ println("$name $label " + peaks.take(5).joinToString(" ") { lag ->+ val g = BeatTracker.fit(env, 0, n, lag.toDouble())+ String.format("%.1f:%.2f[st%.2f c%.2f d%.0f]", Features.FPS * 60 / lag, acf[lag], g?.steadiness ?: 0.0, g?.contrast ?: 0.0, g?.driftMs ?: 0.0)+ })+ }+}++/** Beat This small alone on the whole song against the reference file. */+fun modelDebug(dir: File, name: String) {+ val (mid, _) = loadPcm(File(dir, "$name.f32"))+ val t0 = System.nanoTime()+ val act = model!!.run(mid, 0, mid.size)+ val ms = (System.nanoTime() - t0) / 1e6+ val beats = BeatThis.peaks(act, act.beat).map { it.first }.toDoubleArray()+ val downs = BeatThis.peaks(act, act.downbeat).map { it.first }.toDoubleArray()+ val ref = File(dir, "$name.beats").readLines().filter { it.isNotBlank() }.map { l -> l.trim().split(Regex("\\s+")).let { it[0].toDouble() to it[1].toInt() } }+ val rb = ref.map { it.first }.toDoubleArray(); val rd = ref.filter { it.second == 1 }.map { it.first }.toDoubleArray()+ val gaps = beats.toList().zipWithNext { a, b -> b - a }.sorted()+ println(String.format("%s small0 %5.0fms bpm %.2f beatF %.2f downF %.2f (%d beats, %d downs)", name, ms, 60 / gaps[gaps.size / 2], fMeasure(beats, rb), fMeasure(downs, rd), beats.size, downs.size))+}++/** Beat alignment in the mix, on the reference (Beat This final0) beats of both songs mapped through the plan. */+fun alignment(dir: File, x: String, y: String, plan: MixPlan): String {+ fun ref(n: String) = File(dir, "$n.beats").takeIf { it.exists() }?.readLines()?.filter { it.isNotBlank() }+ ?.map { l -> l.trim().split(Regex("\\s+")).let { it[0].toDouble() to it[1].toInt() } }+ val ra = ref(x) ?: return "no reference"; val rb = ref(y) ?: return "no reference"+ if (!plan.synced) return "unsynced"+ val ua = ra.map { (it.first - plan.exitA) to it.second }.filter { it.first in -1.0..plan.length + 1 }+ val ub = rb.map { ((it.first - plan.entryB) / plan.speedB) to it.second }.filter { it.first in 0.0..plan.length }+ if (ua.size < 2 || ub.isEmpty()) return "too few reference beats"+ val aBeats = ua.map { it.first }.let { l -> l + l.zipWithNext { p, q -> (p + q) / 2 } }+ val d = ub.map { b -> aBeats.minOf { abs(it - b.first) } * 1000 }.sorted()+ val aDown = ua.filter { it.second == 1 }.map { it.first }+ val bDown = ub.filter { it.second == 1 }.map { it.first }+ val dd = bDown.mapNotNull { b -> aDown.minOfOrNull { abs(it - b) } }.map { it * 1000 }.sorted()+ return String.format("ref beats: median %.0f ms, 90%% %.0f ms over %d B beats | downbeats on A downbeats: median %s ms", d[d.size / 2], d[(d.size * 0.9).toInt().coerceAtMost(d.size - 1)], d.size,+ if (dd.isEmpty()) "-" else String.format("%.0f", dd[dd.size / 2]))+}++/** Time per analysis stage on one song (desktop). */+fun profile(dir: File, name: String) {+ val (mid, side) = loadPcm(File(dir, "$name.f32"))+ fun <T> t(label: String, f: () -> T): T { val t0 = System.nanoTime(); val r = f(); print(String.format(" %s %.0fms", label, (System.nanoTime() - t0) / 1e6)); return r }+ print(name)+ repeat(2) {+ val f = t("features") { Features.of(mid, side) }+ val env = t("env") { BeatTracker.envelope(f) }+ t("model") { model!!.run(mid, 0, mid.size) }+ t("mel") { MelFrontend.spectrogram(mid, 0, mid.size) }+ t("fit") { BeatTracker.fit(env, 0, env.size, Features.FPS * 60 / 120) }+ t("track") { BeatTracker.track(env, Features.FPS * 60 / 120) }+ t("total") { Analyzer.analyze(mid, side, model, threads = System.getenv("AUTOMIX_THREADS")?.toInt() ?: 2) }+ println()+ }+}diff --git a/base/automix/src/test/kotlin/app/myspoti/automix/Render.kt b/base/automix/src/test/kotlin/app/myspoti/automix/Render.ktnew file mode 100644index 00000000..44c5ebad--- /dev/null+++ b/base/automix/src/test/kotlin/app/myspoti/automix/Render.kt@@ -0,0 +1,97 @@+/* SPDX-License-Identifier: GPL-3.0-only */+package app.myspoti.automix++import java.io.File+import java.io.RandomAccessFile+import java.nio.ByteBuffer+import java.nio.ByteOrder+import kotlin.math.abs+import kotlin.math.PI+import kotlin.math.pow+import kotlin.math.sqrt+import kotlin.math.tan++/**+ * Desktop preview of a plan, the way the phone plays it: A at 1×, B stretched by ffmpeg's atempo+ * (pitch kept, as the phone's AudioTrack time stretch), gains and 3-band levels per [MixPlan.levels].+ * B's speed stays at speedB after the mix here (the phone brings it back to 1 over restoreSec).+ */+object Render {+ const val SR = 44100++ fun stereo(file: File): FloatArray {+ val fb = ByteBuffer.wrap(file.readBytes()).order(ByteOrder.LITTLE_ENDIAN).asFloatBuffer()+ return FloatArray(fb.remaining()).also { fb.get(it) }+ }++ /** B from [from] seconds on, at [speed] (pitch kept), as stereo float. */+ private fun stretched(src: File, from: Double, seconds: Double, speed: Double, tmp: File): FloatArray {+ val cmd = listOf("ffmpeg", "-v", "error", "-y", "-f", "f32le", "-ar", "$SR", "-ac", "2", "-ss", "%.6f".format(from), "-t", "%.3f".format(seconds * speed + 1),+ "-i", src.path, "-af", "atempo=%.6f".format(speed), "-f", "f32le", "-ar", "$SR", "-ac", "2", tmp.path)+ check(ProcessBuilder(cmd).inheritIO().start().waitFor() == 0) { "ffmpeg failed" }+ return stereo(tmp)+ }++ fun render(a: File, b: File, plan: MixPlan, out: File, pre: Double = 12.0, post: Double = 15.0) {+ val pcmA = stereo(a)+ val total = pre + plan.length + post+ val bPart = stretched(b, plan.entryB, plan.length + post, plan.speedB, File(out.parentFile, "b.tmp.f32"))+ val n = (total * SR).toInt()+ val o = FloatArray(2 * n)+ val eqA = Eq(); val eqB = Eq()+ var lv = plan.levels(-1.0)+ for (i in 0 until n) {+ val u = i.toDouble() / SR - pre+ if (i % 64 == 0) lv = plan.levels(u / plan.length)+ val ia = ((plan.exitA + u) * SR).toInt()+ val ib = (u * SR).toInt()+ val gainB = if (u >= plan.length) plan.gainAfter(plan.resumeB + (u - plan.length) * plan.speedB) else lv.gainB+ for (c in 0..1) {+ val xa = if (u < plan.length + 0.05 && ia >= 0 && 2 * ia + c < pcmA.size) pcmA[2 * ia + c] else 0f+ val xb = if (ib >= 0 && 2 * ib + c < bPart.size) bPart[2 * ib + c] else 0f+ val ya = eqA.process(c, xa.toDouble(), lv.lowA, 0.0, lv.highA) * lv.gainA+ val yb = eqB.process(c, xb.toDouble(), lv.lowB, 0.0, lv.highB) * gainB+ o[2 * i + c] = (ya + yb).toFloat()+ }+ }+ writeWav(out, o)+ }++ /** Three bands: low under 180 Hz, high over 2.5 kHz (4th-order Linkwitz–Riley splits), mid the rest. */+ class Eq {+ private val lp1 = Array(2) { Biquad.lowPass(180.0) }; private val lp1b = Array(2) { Biquad.lowPass(180.0) }+ private val lp2 = Array(2) { Biquad.lowPass(2500.0) }; private val lp2b = Array(2) { Biquad.lowPass(2500.0) }+ fun process(c: Int, x: Double, lowDb: Double, midDb: Double, highDb: Double): Double {+ val low = lp1b[c].run(lp1[c].run(x))+ val lowMid = lp2b[c].run(lp2[c].run(x))+ val high = x - lowMid; val mid = lowMid - low+ return low * db(lowDb) + mid * db(midDb) + high * db(highDb)+ }+ private fun db(v: Double) = 10.0.pow(v / 20)+ }++ class Biquad(private val b0: Double, private val b1: Double, private val b2: Double, private val a1: Double, private val a2: Double) {+ private var z1 = 0.0; private var z2 = 0.0+ fun run(x: Double): Double { val y = b0 * x + z1; z1 = b1 * x - a1 * y + z2; z2 = b2 * x - a2 * y; return y }+ companion object {+ fun lowPass(f: Double): Biquad {+ val k = tan(PI * f / SR); val q = 1 / sqrt(2.0); val norm = 1 / (1 + k / q + k * k)+ return Biquad(k * k * norm, 2 * k * k * norm, k * k * norm, 2 * (k * k - 1) * norm, (1 - k / q + k * k) * norm)+ }+ }+ }++ fun writeWav(f: File, x: FloatArray) {+ var peak = 0f; for (v in x) peak = maxOf(peak, abs(v))+ val g = if (peak > 0.99f) 0.99f / peak else 1f+ RandomAccessFile(f, "rw").use { r ->+ r.setLength(0)+ val bb = ByteBuffer.allocate(44 + x.size * 2).order(ByteOrder.LITTLE_ENDIAN)+ bb.put("RIFF".toByteArray()); bb.putInt(36 + x.size * 2); bb.put("WAVEfmt ".toByteArray())+ bb.putInt(16); bb.putShort(1); bb.putShort(2); bb.putInt(SR); bb.putInt(SR * 4); bb.putShort(4); bb.putShort(16)+ bb.put("data".toByteArray()); bb.putInt(x.size * 2)+ for (v in x) bb.putShort((v * g * 32767).toInt().coerceIn(-32768, 32767).toShort())+ r.write(bb.array())+ }+ }+}diff --git a/base/settings.gradle.kts b/base/settings.gradle.ktsindex 2c158551..9d1db908 100644--- a/base/settings.gradle.kts+++ b/base/settings.gradle.kts@@ -28,6 +28,7 @@ dependencyResolutionManagement {rootProject.name = "Myspoti"include(":app")include(":api")+include(":automix")include(":extensions")include(":extensions-cli")include(":api-spotify")diff --git a/docs/automix/README.md b/docs/automix/README.mdnew file mode 100644index 00000000..51c43111--- /dev/null+++ b/docs/automix/README.md@@ -0,0 +1,106 @@+# AutoMix++DJ-style transitions between the songs of the local player: extension streams, downloads and the+phone's files. Spotify songs play in the Web Player, whose audio the app never has: no AutoMix there.+Settings → Crossfade → AutoMix (or the AutoMix switch under Crossfade in the classic settings).++This is the fourth AutoMix and a rewrite: nothing is kept from the earlier ones' playback.++## Why the earlier AutoMix broke the sound++The earlier one rendered each transition into a WAV file and spliced it into the queue between the+two songs (the outgoing song clipped while it played, the WAV, then the incoming song from an+offset). Each seam sat on a decoder packet, not a sample, so the audio could jump by up to a packet+(23 ms for AAC, 93 ms for a 4096-sample FLAC block) at both ends of every transition, and a sample+rate change between songs reopened the audio output. It also measured only short windows of each+song, with strict checks, so most pairs fell back to a plain fade.++Two Media3 facts shaped the new design (read in Media3 1.8.0, the version the app uses):+- A speed change through the sink's Sonic drains the processors and starts a new Sonic: the+ join is a hard switch, and the new speed is heard only after the AudioTrack buffer+ (250–750 ms). BitChord turned its live phase correction off for the same reason+ ("cracked on some phones"). With `setEnableAudioTrackPlaybackParams(true)` the speed is the+ AudioTrack's own: applied at once, without a drain.+- An AudioProcessor sees no timestamps and acts a whole buffer before the sound is heard; a+ player's volume (`AudioTrack.setVolume`) and an audio effect on its session act on what is heard now.++## How it works now++**Analysis** (`base/automix`, pure Kotlin, the same code on the phone and on the desktop bench).+Each song is decoded once through the player's own data source (downloads decrypted, extension+streams resolved; streams only on unmetered networks, no DASH answers), to mid and side at 22.05 kHz.+- Onsets: spectral flux on 40 log bands, plus the bands under 150 Hz (kicks, 808s).+- Beat This! small0 (MIT, int8 ONNX, ONNX Runtime) on the whole song gives the metrical level+ (pure onset autocorrelation hesitates between the beat and ¾ or 4⁄3 of it on trap hi-hats) and+ the downbeats.+- A constant grid is fitted on the onsets with the model's period (coarse search, then weighted+ least squares on the onset peaks), moved onto the model's beats (same instant for every song),+ and checked: the share of beats on the grid, the drift over 8 beats, and the agreement with the+ model. A song made on a fixed tempo gets one grid; otherwise the start and the end get grids of+ their own where they hold.+- Bars (downbeats voted by the model near the start and near the end), per-bar level, low and+ high band, voice-band centre (mid over side), novelty (4 bars against the 4 before) and the+ 8-bar phrase offset; key (Krumhansl–Kessler on the chroma), loudness, where the sound starts and ends.+- The result is cached (`cache/automix4/`), about 10 KB a song.+- Speed: on the S24, 5 to 11 s a song (decoding 3–6 s, analysis 3–5 s). The decoder never waits on+ the codec while it has work; the onset features (in slices) and Beat This (its 15-s chunks side by+ side, one ONNX thread each) run at once on all cores but two. The song after the next one is+ analysed too, so a skip finds both of its songs ready. Beat This final0 (10× the parameters) would+ take 40–80 s a song for better metrical decisions on a few hard songs: not used.++**Planning** (`MixPlanner`): exit cues on A's downbeats in its last minute and a half (phrase starts,+a level drop after them, late; never past where its music really ends, before a long fade-out),+entry cues on B's first downbeat or a phrase start in its intro (a skip costs, less over a quiet+intro); 16, 8, 4 or 2 bars, scored for: B's next section starting when A is gone, A's loud bars+left out, two voices at once, two loud songs at once, key distance on the Camelot wheel, size of+the tempo change. Styles:+- **BLEND** when both grids hold and the tempo change is at most 8 % (half and double time count):+ B comes in under A with its bass cut and its top end a little down, the bass moves from A to B+ over one beat at a downbeat in the middle, A fades out (equal power) over the second half with+ its top end going down. B louder than A is brought to A's level, then back to its own over 8 s.+- **CUT** when A has a grid but the tempos are too far apart: B starts whole on A's downbeat (on+ its own first downbeat or a later phrase), A's bass goes at once and A is out within two beats.+- **FADE** without a grid: 6 s equal power, A's bass out halfway.++**Playback** (`LocalPlayer`, "AutoMix" part): two decks, no song ever handed from one player to the other.+- 15 s before the mix the next song is prepared on a second player at its entry point less a+ 2.5 s run-up, its speed set (AudioTrack playback parameters) so its beats fall on A's, volume 0.+- 2.5 s before the mix it starts, muted, and a phase-locked loop compares both positions every+ 20 ms and nudges its speed (up to 8 % while muted, 2 % once heard; a seek if it is 400 ms off).+- At the exit point the deck becomes the main player: the queue around its song, the listener,+ the audio focus. The old main player plays the mix out as the outgoing one, then stops.+- Gains are the two players' volumes; the bass swap and the top-end cuts are a three-band+ DynamicsProcessing EQ on each player's audio session (low under 180 Hz, high over 2.5 kHz).+ The phone refusing the effect leaves the gains alone.+- After the mix the deck's speed goes back to the song's own, about 0.3 % a beat.+- Not analysed in time: a 6 s fade on the songs' lengths, still on two decks.+- **Album order kept**: at the end of a song, no mix into the next song of the same album (the+ phone's files: the next track number, or the first track of the next disc; streams and downloads,+ which carry no track number: the album played as the list, the next song in its own order). The+ two songs run into each other as on the album. Two songs of one album that do not follow each+ other there are mixed as any others.+- **Next mixes too**: Next prepares the next song on the deck and mixes it in from A's next+ downbeat: two bars beat-matched (one when two last over 5.5 s), else a cut on the beat, else a+ 2.5 s fade; this holds for songs of one album as well. Not ready within 2.5 s: the plain skip. A+ second Next skips at once.+- Seek, previous, a queue change or AutoMix off: the deck is dropped (before the mix) or the song+ coming in plays on at its own speed (during the mix).++## Checking it++Desktop bench (real songs, no phone): see [tools/automix/README.md](../../tools/automix/README.md).+On the 12 songs it was tuned on (rap, trap, pop, an orchestral piece), against the Beat This final0+model as the reference: the tempo level matches on all 12; 10 get one grid for the whole song; the+beat-matched mixes put B's reference beats within 11–16 ms (median) of A's, about the reference's+own 20-ms resolution.++On the phone, through the command line (`docs/cli.md`): `automix on`, `automix.files <piece>…`+(phone songs as the queue), `automix.seek <ms>`, `automix.status` (state and journal: analyses,+plans, the deck's distance to its place once a second, the mix's largest gap).++## Sources++Methods only, no code copied: Mixxx (intro/outro model, sync), Zehren, Alunno and Bientinesi,+"Automatic detection of cue points for DJ mixing" (2020), Foscarin et al., "Beat This!" (2024),+Ellis, "Beat tracking by dynamic programming" (2007), Krumhansl and Kessler key profiles, BitChord's+two-player automix (GPL-3.0; its design only), kckDeepak/AI-DJ-Mixing-System (MIT; its rules).diff --git a/docs/cli.md b/docs/cli.mdindex e61724ac..46d7a3cd 100644--- a/docs/cli.md+++ b/docs/cli.md@@ -37,6 +37,10 @@ the defaults (the beta is `-p app.myspoti`).| `ext.why [n]` | The last `n` (1 by default, 20 at most) plays through the extensions: sources asked, outcome and time, skipped and why, chosen, how its address opened, announced and detected quality, fallback, error. Addresses without query, masked. Debug builds only. || `play <spotify:…> [context]`, `pause`, `resume`, `next`, `previous` | The player's own commands. || `nav <route>` | The open screen navigates to a route (`settings`, `extensions`, `extension/<id>`, `queue`…). |+| `automix [on\|off]` | Turns AutoMix on or off (as the setting) and prints where it stands. |+| `automix.status` | AutoMix's state (deck, plan, position) and its journal's last 30 lines: analyses, plans, the deck's distance to its place once a second during a mix, the mix's largest gap. See `docs/automix/README.md`. |+| `automix.files <piece> <piece>…` | Plays the phone songs whose folder + title contains each piece, in that order, as the local queue (e.g. `Spotiflaccc/24`). |+| `automix.seek <ms>` | Moves in the local player's song. |## What never leaves the phonediff --git a/tools/automix/README.md b/tools/automix/README.mdnew file mode 100644index 00000000..84adcb8f--- /dev/null+++ b/tools/automix/README.md@@ -0,0 +1,21 @@+# AutoMix desktop bench++Runs the phone's analysis and planning code (`base/automix`) on real songs and renders the+transitions to WAV files, the way the phone mixes them (B stretched with ffmpeg's `atempo`, pitch kept).++Prepare a folder (songs are not in the repository):++```sh+tools/automix/prepare.sh <songs folder> <work folder> # sNN.f32 per song, plus Beat This final0 references if beat_this is installed+```++Then, from `base/`:++```sh+./gradlew :automix:bench -Pargs="analyze <work folder>" # tempo, grids, downbeats; agreement with the references+./gradlew :automix:bench -Pargs="plan <work folder> s01,s02 s03,s06" # plans; beat alignment on the references+./gradlew :automix:bench -Pargs="plan <work folder> s01,s02 render" # also mix-s01-s02.wav (12 s before, 15 s after)+```++The bench reads the app's Beat This model (`app/src/main/assets/automix/`) through the JVM build of+ONNX Runtime. References: `pip install beat_this` (PyTorch), then `beat_this <work folder>/*.wav`.diff --git a/tools/automix/prepare.sh b/tools/automix/prepare.shnew file mode 100755index 00000000..543ee88a--- /dev/null+++ b/tools/automix/prepare.sh@@ -0,0 +1,15 @@+#!/bin/sh+# Decodes every song of $1 into $2: sNN.f32 (stereo float 44.1 kHz, the bench's input) and sNN.wav+# (mono 22.05 kHz, for the Beat This references); names.txt maps sNN to the file.+set -e+src=$1; out=$2+[ -d "$src" ] && [ -n "$out" ] || { echo "usage: $0 <songs folder> <work folder>"; exit 1; }+mkdir -p "$out"; : > "$out/names.txt"+i=0+for f in "$src"/*; do+ i=$((i + 1)); n=$(printf "s%02d" $i)+ echo "$n|$(basename "$f")" >> "$out/names.txt"+ ffmpeg -v error -y -i "$f" -ac 2 -ar 44100 -f f32le "$out/$n.f32"+ ffmpeg -v error -y -i "$f" -ac 1 -ar 22050 "$out/$n.wav"+done+if command -v beat_this >/dev/null; then beat_this "$out"/*.wav; fi