Your Android phone and your Windows PC, working as one. Local-first, end-to-end encrypted, no account.
Control media across PC and phone
Beef4ccdf20c85e
diff --git a/README.md b/README.mdindex e177a52..bf00770 100644--- a/README.md+++ b/README.md@@ -84,6 +84,10 @@ place and tested; the user-facing features of Phase 1 come next.phone, share them from any app or pick them in Nectarlink. They land in `Downloads\Nectarlink` on the PC and`Download/Nectarlink` on the phone, with progress, cancelling, and transfersthat pick up where they left off after a dropped connection.+- **Media** both ways: what plays on the phone shows on the PC, in the app and+ in Windows' own media flyout (so the keyboard's media keys work too), and+ what plays on the PC shows in the phone's media controls, with artwork and+ a seek bar.- **Capability matrix**: both apps compute which features work for a pair ofdevices and what would enable the rest, so the interface never offerssomething that can't work.@@ -95,8 +99,7 @@ place and tested; the user-facing features of Phase 1 come next.connection service, and an identity key protected by the Android Keystore.- **Command-line client** for development and testing (`nectarlink`).-**Next (Phase 1)**: media controls, notification filters and history, and a-Windows installer. Mirroring, messages, calls and the other+**Next (Phase 1)**: notification filters and history, and a Windows installer. Mirroring, messages, calls and the otherfeatures in the plan follow in later phases.## Screenshotsdiff --git a/android/README.md b/android/README.mdindex 5096a3e..834eeb0 100644--- a/android/README.md+++ b/android/README.md@@ -45,6 +45,9 @@ sources; nothing generated is checked in except the theme tokens- `files/`: opening files to send, publishing received files toDownload/Nectarlink (MediaStore on Android 10+), and the transfernotifications with progress and Cancel.+- `media/`: this phone's media sessions for PCs (`PhoneMedia`, with+ notification access), and a PC's player in Android's media controls+ (`PcMedia`, a media session and its notification).- `service/ConnectionService`: a "connected device" foreground service thatkeeps the phone reachable and holds the Wi-Fi multicast lock localdiscovery needs.diff --git a/android/app/src/main/AndroidManifest.xml b/android/app/src/main/AndroidManifest.xmlindex d49bd3a..3c40e3e 100644--- a/android/app/src/main/AndroidManifest.xml+++ b/android/app/src/main/AndroidManifest.xml@@ -86,6 +86,11 @@android:name=".files.TransferNotifications$CancelReceiver"android:exported="false" />+ <!-- The PC media notification's buttons (Android 12 and earlier). -->+ <receiver+ android:name=".media.PcMedia$CommandReceiver"+ android:exported="false" />+<!-- Opening received files on Android 8–9 (later versions use MediaStore),and images from a PC on the clipboard. --><providerdiff --git a/android/app/src/main/kotlin/app/nectarlink/android/core/Core.kt b/android/app/src/main/kotlin/app/nectarlink/android/core/Core.ktindex abe4650..91a3f02 100644--- a/android/app/src/main/kotlin/app/nectarlink/android/core/Core.kt+++ b/android/app/src/main/kotlin/app/nectarlink/android/core/Core.kt@@ -8,6 +8,8 @@ import android.util.Logimport android.net.Uriimport app.nectarlink.android.BuildConfigimport app.nectarlink.android.clipboard.PhoneClipboard+import app.nectarlink.android.media.PcMedia+import app.nectarlink.android.media.PhoneMediaimport app.nectarlink.android.files.OutgoingFilesimport app.nectarlink.android.files.ReceivedFilesimport app.nectarlink.android.files.TransferNotifications@@ -15,6 +17,7 @@ import app.nectarlink.android.Rimport app.nectarlink.android.notifications.NotificationListenerimport app.nectarlink.core.Eventimport app.nectarlink.core.EventListener+import app.nectarlink.core.Linkimport app.nectarlink.core.NectarlinkExceptionimport app.nectarlink.core.NectarlinkNodeimport app.nectarlink.core.NodeOptions@@ -48,7 +51,16 @@ import kotlinx.coroutines.withContextclass Core(context: Context, private val scope: CoroutineScope) : EventListener {private val context = context.applicationContextprivate val ringer = Ringer(this.context)- private val platform = PhonePlatform(this.context, ringer)+ /** What plays on this phone, for PCs (needs notification access). */+ private val phoneMedia = PhoneMedia(this.context) { players ->+ notificationOps.trySend { it.mediaChanged(players) }+ }+ private val platform = PhonePlatform(this.context, ringer, phoneMedia)++ /** What plays on the PCs, in Android's media controls. */+ val pcMedia = PcMedia(this.context) { pc, player, action, position ->+ command { it.mediaCommand(pc, player, action, position) }+ }/*** Notification changes, in order: a removal must never overtake the post* it removes. Runs once the node is up.@@ -128,6 +140,7 @@ class Core(context: Context, private val scope: CoroutineScope) : EventListener*/fun notificationAccessChanged(granted: Boolean, showing: List<Notification>) {_state.update { it.copy(notificationAccess = granted) }+ scope.launch(Dispatchers.Main) { if (granted) phoneMedia.start() else phoneMedia.stop() }notificationOps.trySend { node ->node.updatePower(PowerLevel.BASIC, capabilities(granted))node.notificationsReset(showing)@@ -158,7 +171,15 @@ class Core(context: Context, private val scope: CoroutineScope) : EventListeneroverride fun onEvent(event: Event) {_state.update { it.reduce(event) }- if (event is Event.Transfer) onTransfer(event.transfer)+ when (event) {+ is Event.Transfer -> onTransfer(event.transfer)+ is Event.MediaChanged -> pcMedia.update(event.id, _state.value.nameOf(event.id).orEmpty(), event.players)+ // A PC that's gone isn't playing for this phone anymore; it+ // sends what plays when it's back.+ is Event.LinkChanged -> if (event.link is Link.Offline) pcMedia.update(event.id, "", emptyList())+ is Event.DeviceRemoved -> pcMedia.update(event.id, "", emptyList())+ else -> {}+ }}// ---- Files ----@@ -253,7 +274,7 @@ class Core(context: Context, private val scope: CoroutineScope) : EventListener/** What this phone offers PCs (docs/protocol/capabilities.md). */private fun capabilities(notificationAccess: Boolean): List<String> =- CLIPBOARD_CAPABILITIES + if (notificationAccess) NOTIFICATION_CAPABILITIES else emptyList()+ CLIPBOARD_CAPABILITIES + if (notificationAccess) NOTIFICATION_CAPABILITIES + MEDIA_CAPABILITIES else emptyList()// ---- Pairing ----@@ -352,6 +373,8 @@ class Core(context: Context, private val scope: CoroutineScope) : EventListenerconst val TAG = "Nectarlink"/** Offered while notification access is granted (docs/protocol/capabilities.md). */val NOTIFICATION_CAPABILITIES = listOf("notify.mirror", "notify.reply")+ /** Sharing this phone's players, which also needs notification access. */+ val MEDIA_CAPABILITIES = listOf("media.control")/** Accepting the PC's clipboard, and sending this one when asked. */val CLIPBOARD_CAPABILITIES = listOf("clip.write", "clip.share")}diff --git a/android/app/src/main/kotlin/app/nectarlink/android/core/CoreState.kt b/android/app/src/main/kotlin/app/nectarlink/android/core/CoreState.ktindex df98a14..37f844b 100644--- a/android/app/src/main/kotlin/app/nectarlink/android/core/CoreState.kt+++ b/android/app/src/main/kotlin/app/nectarlink/android/core/CoreState.kt@@ -102,6 +102,8 @@ data class CoreState(// Android shows its own "copied" confirmation.is Event.ClipboardReceived -> thisis Event.Transfer -> copy(transfers = withTransfer(event.transfer))+ // Shown in Android's media controls (see media/PcMedia).+ is Event.MediaChanged -> this}private fun withTransfer(transfer: Transfer): List<Transfer> {diff --git a/android/app/src/main/kotlin/app/nectarlink/android/core/PhonePlatform.kt b/android/app/src/main/kotlin/app/nectarlink/android/core/PhonePlatform.ktindex db5805e..ca7afa1 100644--- a/android/app/src/main/kotlin/app/nectarlink/android/core/PhonePlatform.kt+++ b/android/app/src/main/kotlin/app/nectarlink/android/core/PhonePlatform.kt@@ -10,6 +10,8 @@ import android.os.Buildimport android.os.Bundleimport app.nectarlink.android.clipboard.PhoneClipboardimport app.nectarlink.android.notifications.NotificationListener+import app.nectarlink.android.media.PhoneMedia+import app.nectarlink.core.MediaActionimport app.nectarlink.core.NotificationFailureimport app.nectarlink.core.Platform@@ -17,7 +19,11 @@ import app.nectarlink.core.Platform* What the core asks of the phone: ring it, and act on its notifications* for a PC (dismiss, reply, run an action).*/-internal class PhonePlatform(context: Context, private val ringer: Ringer) : Platform {+internal class PhonePlatform(+ context: Context,+ private val ringer: Ringer,+ private val media: PhoneMedia,+) : Platform {private val context = context.applicationContextoverride fun startRinging() = ringer.startRinging()@@ -29,6 +35,9 @@ internal class PhonePlatform(context: Context, private val ringer: Ringer) : Plaoverride fun setClipboardImage(mime: String, bytes: ByteArray): Boolean =PhoneClipboard.writeImage(context, mime, bytes)+ override fun mediaCommand(player: String, action: MediaAction, position: ULong?) =+ media.command(player, action, position)+override fun dismissNotification(key: String) {val listener = NotificationListener.instance ?: throw NotificationFailure.Unsupported()listener.dismiss(key)diff --git a/android/app/src/main/kotlin/app/nectarlink/android/media/PcMedia.kt b/android/app/src/main/kotlin/app/nectarlink/android/media/PcMedia.ktnew file mode 100644index 0000000..c5336b9--- /dev/null+++ b/android/app/src/main/kotlin/app/nectarlink/android/media/PcMedia.kt@@ -0,0 +1,211 @@+// SPDX-License-Identifier: GPL-3.0-or-later+package app.nectarlink.android.media++import android.Manifest+import android.app.Notification+import android.app.NotificationChannel+import android.app.NotificationManager+import android.app.PendingIntent+import android.content.BroadcastReceiver+import android.content.Context+import android.content.Intent+import android.content.pm.PackageManager+import android.graphics.Bitmap+import android.graphics.BitmapFactory+import android.graphics.drawable.Icon+import android.media.MediaMetadata+import android.media.session.MediaSession+import android.media.session.PlaybackState+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import androidx.core.content.ContextCompat+import app.nectarlink.android.NectarlinkApplication+import app.nectarlink.android.R+import app.nectarlink.android.ui.MainActivity+import app.nectarlink.core.MediaAction+import app.nectarlink.core.MediaPlayer++/**+ * What plays on a paired PC, in Android's own media controls (the+ * notification shade and Quick Settings player): a media session that+ * stands for the PC's player, and its notification. What the user presses+ * goes to the PC through [send]; the PC's new state comes back as an update.+ *+ * Everything runs on the main thread.+ */+class PcMedia(+ context: Context,+ private val send: (pc: String, player: String, action: MediaAction, position: ULong?) -> Unit,+) {+ private val context = context.applicationContext+ private val main = Handler(Looper.getMainLooper())++ /** Players per PC, in the PC's order, and the PCs' names. */+ private val players = linkedMapOf<String, List<MediaPlayer>>()+ private val names = mutableMapOf<String, String>()+ /** Decoded artwork by PC and key (artwork arrives once per key). */+ private val art = object : LinkedHashMap<String, Bitmap>(8, 0.75f, true) {+ override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, Bitmap>) = size > 8+ }++ private var session: MediaSession? = null+ private var shown: Pair<String, MediaPlayer>? = null++ private val callback = object : MediaSession.Callback() {+ override fun onPlay() = command(MediaAction.PLAY)+ override fun onPause() = command(MediaAction.PAUSE)+ override fun onSkipToNext() = command(MediaAction.NEXT)+ override fun onSkipToPrevious() = command(MediaAction.PREVIOUS)+ override fun onSeekTo(pos: Long) = command(MediaAction.SEEK, pos.coerceAtLeast(0).toULong())+ }++ /** A PC's players changed (empty: nothing plays there, or it's offline). */+ fun update(pc: String, pcName: String, list: List<MediaPlayer>) = main.post {+ names[pc] = pcName+ for (player in list) {+ val key = player.artKey ?: continue+ val bytes = player.art ?: continue+ BitmapFactory.decodeByteArray(bytes, 0, bytes.size)?.let { art["$pc/$key"] = it }+ }+ if (list.isEmpty()) players.remove(pc) else players[pc] = list+ render()+ }++ private fun command(action: MediaAction, position: ULong? = null) {+ val (pc, player) = shown ?: return+ send(pc, player.id, action, position)+ }++ private fun render() {+ val first = players.entries.firstNotNullOfOrNull { (pc, list) -> list.firstOrNull()?.let { pc to it } }+ if (first == null) return hide()+ val (pc, player) = first+ shown = first+ val session = session ?: MediaSession(context, "Nectarlink PC media").also {+ it.setCallback(callback, main)+ it.setSessionActivity(openApp())+ session = it+ }+ val picture = player.artKey?.let { art["$pc/$it"] }+ val where = context.getString(R.string.media_on, player.app, names[pc].orEmpty())+ session.setMetadata(+ MediaMetadata.Builder()+ .putString(MediaMetadata.METADATA_KEY_TITLE, player.title ?: player.app)+ .putString(MediaMetadata.METADATA_KEY_ARTIST, player.artist ?: where)+ .putString(MediaMetadata.METADATA_KEY_ALBUM, player.album)+ .putString(MediaMetadata.METADATA_KEY_DISPLAY_SUBTITLE, where)+ .apply { player.duration?.let { putLong(MediaMetadata.METADATA_KEY_DURATION, it.toLong()) } }+ .apply { picture?.let { putBitmap(MediaMetadata.METADATA_KEY_ALBUM_ART, it) } }+ .build(),+ )+ var actions = 0L+ if (MediaAction.PLAY in player.actions) actions = actions or PlaybackState.ACTION_PLAY+ if (MediaAction.PAUSE in player.actions) actions = actions or PlaybackState.ACTION_PAUSE+ if (MediaAction.PLAY in player.actions && MediaAction.PAUSE in player.actions) {+ actions = actions or PlaybackState.ACTION_PLAY_PAUSE+ }+ if (MediaAction.NEXT in player.actions) actions = actions or PlaybackState.ACTION_SKIP_TO_NEXT+ if (MediaAction.PREVIOUS in player.actions) actions = actions or PlaybackState.ACTION_SKIP_TO_PREVIOUS+ if (MediaAction.SEEK in player.actions) actions = actions or PlaybackState.ACTION_SEEK_TO+ session.setPlaybackState(+ PlaybackState.Builder()+ .setActions(actions)+ .setState(+ if (player.playing) PlaybackState.STATE_PLAYING else PlaybackState.STATE_PAUSED,+ player.position?.toLong() ?: PlaybackState.PLAYBACK_POSITION_UNKNOWN,+ if (player.playing) 1f else 0f,+ SystemClock.elapsedRealtime(),+ )+ .build(),+ )+ session.isActive = true+ post(notification(session, player, where, picture))+ }++ private fun hide() {+ shown = null+ session?.run {+ isActive = false+ release()+ }+ session = null+ context.getSystemService(NotificationManager::class.java).cancel(NOTIFICATION_TAG, NOTIFICATION_ID)+ }++ private fun notification(session: MediaSession, player: MediaPlayer, where: String, picture: Bitmap?): Notification {+ val buttons = buildList {+ if (MediaAction.PREVIOUS in player.actions) add(button(R.drawable.ic_skip_previous, R.string.media_previous, MediaAction.PREVIOUS))+ if (player.playing && MediaAction.PAUSE in player.actions) add(button(R.drawable.ic_pause, R.string.media_pause, MediaAction.PAUSE))+ if (!player.playing && MediaAction.PLAY in player.actions) add(button(R.drawable.ic_play, R.string.media_play, MediaAction.PLAY))+ if (MediaAction.NEXT in player.actions) add(button(R.drawable.ic_skip_next, R.string.media_next, MediaAction.NEXT))+ }+ return Notification.Builder(context, CHANNEL)+ .setSmallIcon(R.drawable.ic_notification)+ .setContentTitle(player.title ?: player.app)+ .setContentText(listOfNotNull(player.artist, where).joinToString(" · "))+ .setLargeIcon(picture)+ .setContentIntent(openApp())+ .setOnlyAlertOnce(true)+ .setShowWhen(false)+ .setVisibility(Notification.VISIBILITY_PUBLIC)+ .setCategory(Notification.CATEGORY_TRANSPORT)+ .apply { buttons.forEach(::addAction) }+ .setStyle(+ Notification.MediaStyle()+ .setMediaSession(session.sessionToken)+ .setShowActionsInCompactView(*buttons.indices.take(3).toIntArray()),+ )+ .build()+ }++ private fun button(icon: Int, label: Int, action: MediaAction): Notification.Action {+ val intent = Intent(context, CommandReceiver::class.java).putExtra(EXTRA_ACTION, action.name)+ val pending = PendingIntent.getBroadcast(+ context,+ action.ordinal,+ intent,+ PendingIntent.FLAG_IMMUTABLE or PendingIntent.FLAG_UPDATE_CURRENT,+ )+ return Notification.Action.Builder(Icon.createWithResource(context, icon), context.getString(label), pending).build()+ }++ private fun openApp(): PendingIntent = PendingIntent.getActivity(+ context,+ 0,+ Intent(context, MainActivity::class.java).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK),+ PendingIntent.FLAG_IMMUTABLE,+ )++ private fun post(notification: Notification) {+ if (ContextCompat.checkSelfPermission(context, Manifest.permission.POST_NOTIFICATIONS) != PackageManager.PERMISSION_GRANTED) {+ return+ }+ context.getSystemService(NotificationManager::class.java).notify(NOTIFICATION_TAG, NOTIFICATION_ID, notification)+ }++ /** The notification's buttons on Android 12 and earlier (later ones use the session). */+ class CommandReceiver : BroadcastReceiver() {+ override fun onReceive(context: Context, intent: Intent) {+ val action = intent.getStringExtra(EXTRA_ACTION)?.let { name -> MediaAction.entries.find { it.name == name } } ?: return+ (context.applicationContext as NectarlinkApplication).core.pcMedia.pressed(action)+ }+ }++ /** A button in the notification. */+ internal fun pressed(action: MediaAction) = main.post { command(action) }++ companion object {+ private const val CHANNEL = "pc_media"+ private const val NOTIFICATION_TAG = "pc_media"+ private const val NOTIFICATION_ID = 1+ private const val EXTRA_ACTION = "action"++ fun createChannel(context: Context) {+ context.getSystemService(NotificationManager::class.java).createNotificationChannel(+ NotificationChannel(CHANNEL, context.getString(R.string.channel_pc_media), NotificationManager.IMPORTANCE_LOW)+ .apply { setShowBadge(false) },+ )+ }+ }+}diff --git a/android/app/src/main/kotlin/app/nectarlink/android/media/PhoneMedia.kt b/android/app/src/main/kotlin/app/nectarlink/android/media/PhoneMedia.ktnew file mode 100644index 0000000..87d6af1--- /dev/null+++ b/android/app/src/main/kotlin/app/nectarlink/android/media/PhoneMedia.kt@@ -0,0 +1,214 @@+// SPDX-License-Identifier: GPL-3.0-or-later+package app.nectarlink.android.media++import android.content.Context+import android.graphics.Bitmap+import android.media.MediaMetadata+import android.media.session.MediaController+import android.media.session.MediaSessionManager+import android.media.session.PlaybackState+import android.os.Handler+import android.os.Looper+import android.os.SystemClock+import android.util.Log+import androidx.core.graphics.scale+import app.nectarlink.android.notifications.NotificationListener+import app.nectarlink.core.MediaAction+import app.nectarlink.core.MediaFailure+import app.nectarlink.core.MediaPlayer+import java.io.ByteArrayOutputStream++/**+ * What plays on this phone, for paired PCs (docs/protocol/media.md): the+ * apps' media sessions, read with notification access, and the PCs'+ * commands for them.+ *+ * Runs on the main thread. Nectarlink's own session (a PC's media, see+ * [PcMedia]) is left out, so a PC never gets its own music back.+ */+class PhoneMedia(context: Context, private val onChange: (List<MediaPlayer>) -> Unit) {+ private val context = context.applicationContext+ private val manager = context.getSystemService(MediaSessionManager::class.java)+ private val main = Handler(Looper.getMainLooper())++ /** Watched sessions, with their callbacks. */+ private val watched = mutableMapOf<MediaController, MediaController.Callback>()+ /** The players last sent, by ID, for commands. */+ @Volatile+ private var controllers: Map<String, MediaController> = emptyMap()+ private val art = ArtCache()+ private var running = false++ private val sessionsChanged = MediaSessionManager.OnActiveSessionsChangedListener { list ->+ watch(list.orEmpty())+ }+ private val publish = Runnable { publishNow() }++ /** Starts watching; needs notification access. */+ fun start() {+ if (running) return+ val component = NotificationListener.component(context)+ try {+ manager.addOnActiveSessionsChangedListener(sessionsChanged, component, main)+ watch(manager.getActiveSessions(component))+ running = true+ } catch (e: SecurityException) {+ Log.w(TAG, "no access to media sessions", e)+ }+ }++ fun stop() {+ if (!running) return+ running = false+ manager.removeOnActiveSessionsChangedListener(sessionsChanged)+ watched.forEach { (controller, callback) -> controller.unregisterCallback(callback) }+ watched.clear()+ controllers = emptyMap()+ main.removeCallbacks(publish)+ onChange(emptyList())+ }++ private fun watch(sessions: List<MediaController>) {+ val current = sessions.filter { it.packageName != context.packageName }+ watched.keys.filter { old -> current.none { it.sessionToken == old.sessionToken } }.forEach { gone ->+ watched.remove(gone)?.let(gone::unregisterCallback)+ }+ for (controller in current) {+ if (watched.keys.any { it.sessionToken == controller.sessionToken }) continue+ val callback = object : MediaController.Callback() {+ override fun onMetadataChanged(metadata: MediaMetadata?) = schedule()+ override fun onPlaybackStateChanged(state: PlaybackState?) = schedule()+ override fun onSessionDestroyed() = schedule()+ }+ controller.registerCallback(callback, main)+ watched[controller] = callback+ }+ schedule()+ }++ /** Changes come in bursts (metadata, then state, then artwork). */+ private fun schedule() {+ main.removeCallbacks(publish)+ main.postDelayed(publish, SETTLE_MS)+ }++ private fun publishNow() {+ val ids = mutableMapOf<String, MediaController>()+ val players = watched.keys+ .mapNotNull { controller -> read(controller)?.let { controller to it } }+ .sortedWith(+ compareByDescending<Pair<MediaController, Reading>> { it.second.playing }+ .thenByDescending { it.second.updated },+ )+ .take(MAX_PLAYERS)+ .map { (controller, reading) ->+ // One app may have more than one session.+ var id = controller.packageName+ var n = 2+ while (id in ids) id = "${controller.packageName}#${n++}"+ ids[id] = controller+ reading.player.copy(id = id)+ }+ controllers = ids+ onChange(players)+ }++ private class Reading(val player: MediaPlayer, val playing: Boolean, val updated: Long)++ private fun read(controller: MediaController): Reading? {+ val metadata = controller.metadata ?: return null+ val title = metadata.text(MediaMetadata.METADATA_KEY_TITLE)+ ?: metadata.text(MediaMetadata.METADATA_KEY_DISPLAY_TITLE)+ ?: return null+ val state = controller.playbackState+ val playing = state?.state == PlaybackState.STATE_PLAYING ||+ state?.state == PlaybackState.STATE_BUFFERING+ val duration = metadata.getLong(MediaMetadata.METADATA_KEY_DURATION).takeIf { it > 0 }+ val position = state?.let { s ->+ val moved = if (playing && s.lastPositionUpdateTime > 0) {+ ((SystemClock.elapsedRealtime() - s.lastPositionUpdateTime) * s.playbackSpeed).toLong()+ } else {+ 0L+ }+ (s.position + moved).coerceAtLeast(0).let { if (duration != null) it.coerceAtMost(duration) else it }+ }+ val bits = state?.actions ?: 0L+ val actions = buildList {+ if (bits and (PlaybackState.ACTION_PLAY or PlaybackState.ACTION_PLAY_PAUSE) != 0L) add(MediaAction.PLAY)+ if (bits and (PlaybackState.ACTION_PAUSE or PlaybackState.ACTION_PLAY_PAUSE) != 0L) add(MediaAction.PAUSE)+ if (bits and PlaybackState.ACTION_SKIP_TO_NEXT != 0L) add(MediaAction.NEXT)+ if (bits and PlaybackState.ACTION_SKIP_TO_PREVIOUS != 0L) add(MediaAction.PREVIOUS)+ if (bits and PlaybackState.ACTION_SEEK_TO != 0L && duration != null) add(MediaAction.SEEK)+ }+ val picture = metadata.getBitmap(MediaMetadata.METADATA_KEY_ART)+ ?: metadata.getBitmap(MediaMetadata.METADATA_KEY_ALBUM_ART)+ ?: metadata.getBitmap(MediaMetadata.METADATA_KEY_DISPLAY_ICON)+ val (artKey, artBytes) = picture?.let(art::get) ?: (null to null)+ val player = MediaPlayer(+ id = controller.packageName,+ app = appName(controller.packageName),+ title = title,+ artist = metadata.text(MediaMetadata.METADATA_KEY_ARTIST)+ ?: metadata.text(MediaMetadata.METADATA_KEY_ALBUM_ARTIST)+ ?: metadata.text(MediaMetadata.METADATA_KEY_DISPLAY_SUBTITLE),+ album = metadata.text(MediaMetadata.METADATA_KEY_ALBUM),+ playing = playing,+ duration = duration?.toULong(),+ position = position?.toULong(),+ actions = actions,+ artKey = artKey,+ art = artBytes,+ )+ return Reading(player, playing, state?.lastPositionUpdateTime ?: 0)+ }++ private fun MediaMetadata.text(key: String): String? = getString(key)?.trim()?.takeIf { it.isNotEmpty() }++ private fun appName(packageName: String): String = runCatching {+ val pm = context.packageManager+ pm.getApplicationLabel(pm.getApplicationInfo(packageName, 0)).toString()+ }.getOrDefault(packageName)++ /** A PC's command for one of the players last sent. */+ fun command(player: String, action: MediaAction, position: ULong?) {+ val controller = controllers[player] ?: throw MediaFailure.NotFound()+ val controls = controller.transportControls+ when (action) {+ MediaAction.PLAY -> controls.play()+ MediaAction.PAUSE -> controls.pause()+ MediaAction.NEXT -> controls.skipToNext()+ MediaAction.PREVIOUS -> controls.skipToPrevious()+ MediaAction.SEEK -> controls.seekTo(position?.toLong() ?: throw MediaFailure.Unsupported())+ }+ }++ /** Artwork as a small JPEG, made once per picture. */+ private class ArtCache {+ private val made = object : LinkedHashMap<String, Pair<String, ByteArray>>(8, 0.75f, true) {+ override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, Pair<String, ByteArray>>) = size > 8+ }++ fun get(bitmap: Bitmap): Pair<String, ByteArray> {+ val identity = "${System.identityHashCode(bitmap)}:${bitmap.generationId}"+ made[identity]?.let { return it }+ val scale = minOf(1f, ART_SIZE.toFloat() / maxOf(bitmap.width, bitmap.height))+ val small = if (scale < 1f) {+ bitmap.scale((bitmap.width * scale).toInt().coerceAtLeast(1), (bitmap.height * scale).toInt().coerceAtLeast(1))+ } else {+ bitmap+ }+ val jpeg = ByteArrayOutputStream().also { small.compress(Bitmap.CompressFormat.JPEG, 88, it) }.toByteArray()+ if (small !== bitmap) small.recycle()+ val key = jpeg.contentHashCode().toUInt().toString(16) + "-" + jpeg.size.toString(16)+ return (key to jpeg).also { made[identity] = it }+ }+ }++ private companion object {+ const val TAG = "PhoneMedia"+ const val SETTLE_MS = 200L+ const val MAX_PLAYERS = 4+ /** Artwork's longest side, in pixels. */+ const val ART_SIZE = 360+ }+}diff --git a/android/app/src/main/kotlin/app/nectarlink/android/service/ConnectionService.kt b/android/app/src/main/kotlin/app/nectarlink/android/service/ConnectionService.ktindex d20c52a..0d3d4a5 100644--- a/android/app/src/main/kotlin/app/nectarlink/android/service/ConnectionService.kt+++ b/android/app/src/main/kotlin/app/nectarlink/android/service/ConnectionService.kt@@ -37,6 +37,7 @@ class ConnectionService : LifecycleService() {super.onCreate()createChannels(this)app.nectarlink.android.files.TransferNotifications.createChannels(this)+ app.nectarlink.android.media.PcMedia.createChannel(this)val core = (application as NectarlinkApplication).corestartInForeground(core.state.value)multicastLock = getSystemService(WifiManager::class.java)diff --git a/android/app/src/main/res/drawable/ic_pause.xml b/android/app/src/main/res/drawable/ic_pause.xmlnew file mode 100644index 0000000..4e1117b--- /dev/null+++ b/android/app/src/main/res/drawable/ic_pause.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->+<vector xmlns:android="http://schemas.android.com/apk/res/android"+ android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">+ <path android:fillColor="#FFFFFFFF" android:pathData="M7,5h3.5v14H7zM13.5,5H17v14h-3.5z" />+</vector>diff --git a/android/app/src/main/res/drawable/ic_play.xml b/android/app/src/main/res/drawable/ic_play.xmlnew file mode 100644index 0000000..58cc10b--- /dev/null+++ b/android/app/src/main/res/drawable/ic_play.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->+<vector xmlns:android="http://schemas.android.com/apk/res/android"+ android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">+ <path android:fillColor="#FFFFFFFF" android:pathData="M8,5.14v13.72a1,1 0,0 0,1.5 0.86l11.2,-6.86a1,1 0,0 0,0 -1.72L9.5,4.28A1,1 0,0 0,8 5.14z" />+</vector>diff --git a/android/app/src/main/res/drawable/ic_skip_next.xml b/android/app/src/main/res/drawable/ic_skip_next.xmlnew file mode 100644index 0000000..67550b0--- /dev/null+++ b/android/app/src/main/res/drawable/ic_skip_next.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->+<vector xmlns:android="http://schemas.android.com/apk/res/android"+ android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">+ <path android:fillColor="#FFFFFFFF" android:pathData="M6,6.5v11a1,1 0,0 0,1.55 0.83l8.2,-5.5a1,1 0,0 0,0 -1.66l-8.2,-5.5A1,1 0,0 0,6 6.5zM16.5,6h2v12h-2z" />+</vector>diff --git a/android/app/src/main/res/drawable/ic_skip_previous.xml b/android/app/src/main/res/drawable/ic_skip_previous.xmlnew file mode 100644index 0000000..28e3645--- /dev/null+++ b/android/app/src/main/res/drawable/ic_skip_previous.xml@@ -0,0 +1,6 @@+<?xml version="1.0" encoding="utf-8"?>+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->+<vector xmlns:android="http://schemas.android.com/apk/res/android"+ android:width="24dp" android:height="24dp" android:viewportWidth="24" android:viewportHeight="24">+ <path android:fillColor="#FFFFFFFF" android:pathData="M18,6.5v11a1,1 0,0 1,-1.55 0.83l-8.2,-5.5a1,1 0,0 1,0 -1.66l8.2,-5.5A1,1 0,0 1,18 6.5zM5.5,6h2v12h-2z" />+</vector>diff --git a/android/app/src/main/res/values/strings.xml b/android/app/src/main/res/values/strings.xmlindex 535600b..0c8e4a2 100644--- a/android/app/src/main/res/values/strings.xml+++ b/android/app/src/main/res/values/strings.xml@@ -59,6 +59,12 @@<string name="clip_share_label">Send to PC</string><string name="channel_transfers">File transfers</string><string name="channel_received">Received files</string>+ <string name="channel_pc_media">Media on your PC</string>+ <string name="media_on">%1$s on %2$s</string>+ <string name="media_play">Play</string>+ <string name="media_pause">Pause</string>+ <string name="media_next">Next</string>+ <string name="media_previous">Previous</string><string name="action_send_files">Send files</string><string name="transfers_title">Transfers</string><string name="transfer_sending">Sending to %1$s</string>diff --git a/core/nectarlink-cli/src/main.rs b/core/nectarlink-cli/src/main.rsindex f6df376..ae404a8 100644--- a/core/nectarlink-cli/src/main.rs+++ b/core/nectarlink-cli/src/main.rs@@ -10,8 +10,9 @@ use anyhow::{Context, Result, bail};use clap::{Parser, Subcommand, ValueEnum};use nectarlink_core::{Battery, ConnectionPath, DeviceId, DeviceInfo, DeviceKind, Direction, FeatureState, FileSource,- LinkState, Node, NodeConfig, NodeEvent, Notification, NotificationAction, NotificationError,- OutgoingFile, PairedDevice, PairingEvent, Platform, PowerLevel, TransferState,+ LinkState, MediaAction, MediaError, MediaPlayer, Node, NodeConfig, NodeEvent, Notification,+ NotificationAction, NotificationError, OutgoingFile, PairedDevice, PairingEvent, Platform, PowerLevel,+ TransferState,features::{Effort, FEATURES, Role, UnsupportedReason, Upgrade, UpgradeAction},};use tokio::sync::broadcast::error::RecvError;@@ -140,6 +141,35 @@ enum Command {},/// Show a notification on paired PCs as if this were a phone (use with/// --as-phone), then stay online to show what the PC does with it.+ /// Control what plays on a paired device: play, pause, next, previous,+ /// or seek with --position.+ Media {+ device: String,+ #[arg(value_enum)]+ action: MediaCommandArg,+ /// The player (`run` prints them); default: the device's first.+ #[arg(long)]+ player: Option<String>,+ /// Where to seek to, in seconds.+ #[arg(long, required_if_eq("action", "seek"))]+ position: Option<u64>,+ },+ /// Pretend to play a song and share it with paired devices, which can+ /// control it; stays online until Ctrl+C.+ Play {+ title: String,+ #[arg(long, default_value = "Test Artist")]+ artist: String,+ /// App name shown with it.+ #[arg(long, default_value = "Music")]+ app: String,+ /// Length, in seconds.+ #[arg(long, default_value_t = 215)]+ length: u64,+ /// Artwork (JPEG or PNG).+ #[arg(long)]+ art: Option<PathBuf>,+ },Notify {title: String,text: String,@@ -162,6 +192,31 @@ enum Power {Elevated,}+#[derive(Debug, Clone, Copy, PartialEq, Eq, ValueEnum)]+enum MediaCommandArg {+ Play,+ Pause,+ Next,+ Previous,+ Seek,+}++impl From<MediaCommandArg> for MediaAction {+ fn from(a: MediaCommandArg) -> Self {+ match a {+ MediaCommandArg::Play => MediaAction::Play,+ MediaCommandArg::Pause => MediaAction::Pause,+ MediaCommandArg::Next => MediaAction::Next,+ MediaCommandArg::Previous => MediaAction::Previous,+ MediaCommandArg::Seek => MediaAction::Seek,+ }+ }+}++/// Media commands from paired devices, for `play`.+static MEDIA_COMMANDS: std::sync::OnceLock<tokio::sync::mpsc::UnboundedSender<(MediaAction, Option<u64>)>> =+ std::sync::OnceLock::new();+#[derive(Debug, Clone, Copy, ValueEnum)]enum OnOff {On,@@ -190,6 +245,23 @@ impl Platform for TerminalPlatform {println!("Clipboard image from a paired device ({} bytes): {}", bytes.len(), path.display());Ok(())}+ fn media_command(+ &self,+ player: &str,+ action: MediaAction,+ position: Option<u64>,+ ) -> Result<(), MediaError> {+ println!(+ "A paired device asked {player} to {}{}",+ action.as_str(),+ match position {+ Some(ms) => format!(" to {}", clock(ms)),+ None => String::new(),+ }+ );+ let sender = MEDIA_COMMANDS.get().ok_or(MediaError::NotFound)?;+ sender.send((action, position)).map_err(|_| MediaError::NotFound)+ }fn dismiss_notification(&self, key: &str) -> Result<(), NotificationError> {println!("The PC dismissed {key}");Ok(())@@ -387,6 +459,24 @@ async fn run(cli: &Cli, node: &Node) -> Result<()> {println!("{name} is now {}", if on { "on" } else { "off" });}Command::Run => watch(node).await?,+ Command::Media { device, action, player, position } => {+ let id = resolve(node, device)?;+ wait_until_online(node, id).await?;+ let player = match player {+ Some(player) => player.clone(),+ None => first_player(node, id).await?,+ };+ node.media_command(id, player.clone(), (*action).into(), position.map(|s| s * 1000))+ .await+ .context("media command failed")?;+ println!("Done ({player}).");+ }+ Command::Play { title, artist, app, length, art } => {+ let mut offers = cli.offers.clone();+ offers.push("media.control".into());+ node.update_power(node_power(cli), offers).await;+ play(node, title, artist, app, *length, art.as_deref()).await?;+ }Command::Notify { title, text, app, reply, silent } => {if !cli.as_phone {bail!("notifications come from phones: add --as-phone");@@ -514,6 +604,87 @@ async fn send_files(node: &Node, device: DeviceId, paths: &[PathBuf]) -> Result<}}+/// "3:07".+fn clock(ms: u64) -> String {+ let s = ms / 1000;+ format!("{}:{:02}", s / 60, s % 60)+}++/// The first player a device reports (it reports them when connecting).+async fn first_player(node: &Node, device: DeviceId) -> Result<String> {+ let mut events = node.events();+ // Ask again in case the report came before we listened.+ node.set_device_toggle(device, "media", true)?;+ let wait = async {+ loop {+ if let Ok(NodeEvent::MediaChanged { device: d, players }) = events.recv().await+ && d == device+ && let Some(p) = players.first()+ {+ return p.id.clone();+ }+ }+ };+ tokio::time::timeout(Duration::from_secs(10), wait).await.context("nothing is playing on that device")+}++/// A pretend player that follows the commands it gets.+async fn play(+ node: &Node,+ title: &str,+ artist: &str,+ app: &str,+ length: u64,+ art: Option<&std::path::Path>,+) -> Result<()> {+ let (sender, mut commands) = tokio::sync::mpsc::unbounded_channel();+ let _ = MEDIA_COMMANDS.set(sender);+ let art = art.map(std::fs::read).transpose().context("can't read the artwork")?;+ let art_key = art.as_ref().map(|a| format!("{:x}", a.len()));+ let length = length * 1000;+ let mut track = 1u32;+ let mut position = 0u64;+ let mut playing = true;+ let mut since = std::time::Instant::now();+ let player = |track: u32, position: u64, playing: bool| MediaPlayer {+ id: "cli.player".into(),+ app: app.into(),+ title: Some(if track == 1 { title.to_owned() } else { format!("{title} ({track})") }),+ artist: Some(artist.into()),+ album: Some("Nectarlink Test Album".into()),+ playing,+ duration: Some(length),+ position: Some(position),+ actions: ["play", "pause", "next", "previous", "seek"].map(String::from).to_vec(),+ art_key: art_key.clone(),+ art: art.clone(),+ };+ node.media_changed(vec![player(track, position, playing)]).await;+ println!("Playing \"{title}\". Paired devices can control it; Ctrl+C to stop.");+ loop {+ tokio::select! {+ _ = tokio::signal::ctrl_c() => {+ node.media_changed(Vec::new()).await;+ return Ok(());+ }+ Some((action, to)) = commands.recv() => {+ if playing {+ position = (position + since.elapsed().as_millis() as u64).min(length);+ }+ since = std::time::Instant::now();+ match action {+ MediaAction::Play => playing = true,+ MediaAction::Pause => playing = false,+ MediaAction::Next => { track += 1; position = 0; }+ MediaAction::Previous => { track = track.saturating_sub(1).max(1); position = 0; }+ MediaAction::Seek => position = to.unwrap_or(position).min(length),+ }+ node.media_changed(vec![player(track, position, playing)]).await;+ }+ }+ }+}+async fn watch(node: &Node) -> Result<()> {println!("Online as {}. Press Ctrl+C to stop.", node.device_id());for d in node.paired_devices()? {@@ -581,6 +752,25 @@ fn print_event(node: &Node, event: &NodeEvent) {println!("{}: a notification went away", name(device))}NodeEvent::ClipboardReceived { device } => println!("{}: sent its clipboard", name(device)),+ NodeEvent::MediaChanged { device, players } if players.is_empty() => {+ println!("{}: nothing playing", name(device));+ }+ NodeEvent::MediaChanged { device, players } => {+ for p in players {+ println!(+ "{}: {} {} – {} ({}, {} of {}{}) [player {}]",+ name(device),+ if p.playing { "▶" } else { "⏸" },+ p.title.as_deref().unwrap_or("?"),+ p.artist.as_deref().unwrap_or("?"),+ p.app,+ clock(p.position.unwrap_or(0)),+ clock(p.duration.unwrap_or(0)),+ if p.art.is_some() { ", with artwork" } else { "" },+ p.id,+ );+ }+ }NodeEvent::Transfer(t) if t.direction == Direction::Incoming => match &t.state {TransferState::Done { saved } => {for path in saved {diff --git a/core/nectarlink-core/src/config.rs b/core/nectarlink-core/src/config.rsindex ecae6d6..e033f04 100644--- a/core/nectarlink-core/src/config.rs+++ b/core/nectarlink-core/src/config.rs@@ -3,6 +3,7 @@ use std::{fmt, path::PathBuf};use nectarlink_protocol::messages::{DeviceInfo, PowerLevel};+use crate::media::{MediaAction, MediaError};use crate::{identity::KeyProtector, notifications::NotificationError};/// Configuration for starting a [`Node`](crate::Node).@@ -30,6 +31,9 @@ pub struct NodeConfig {/// Where received files go once complete. `None`: a `received` folder/// in `data_dir`.pub downloads_dir: Option<PathBuf>,+ /// Capabilities this device offers beyond the ones every build does+ /// (docs/protocol/capabilities.md), e.g. `media.control`.+ pub capabilities: Vec<String>,}impl NodeConfig {@@ -44,6 +48,7 @@ impl NodeConfig {port: 0,key_protector: None,downloads_dir: None,+ capabilities: Vec::new(),}}}@@ -99,6 +104,17 @@ pub trait Platform: Send + Sync + 'static {fn set_clipboard_image(&self, _mime: &str, _bytes: &[u8]) -> Result<(), String> {Err("this device has no clipboard".into())}++ /// Run a command on one of this device's media players (a paired device+ /// asked); `position` is set for [`MediaAction::Seek`].+ fn media_command(+ &self,+ _player: &str,+ _action: MediaAction,+ _position: Option<u64>,+ ) -> Result<(), MediaError> {+ Err(MediaError::Unsupported)+ }}/// A platform that does nothing; useful for tests and headless tools.diff --git a/core/nectarlink-core/src/events.rs b/core/nectarlink-core/src/events.rsindex 97392da..fb4f20f 100644--- a/core/nectarlink-core/src/events.rs+++ b/core/nectarlink-core/src/events.rs@@ -3,7 +3,7 @@use nectarlink_protocol::{DeviceId,- messages::{Battery, DeviceInfo, Notification, PowerLevel},+ messages::{Battery, DeviceInfo, MediaPlayer, Notification, PowerLevel},};use crate::{features::CapabilityMatrix, transfer::Transfer};@@ -121,4 +121,11 @@ pub enum NodeEvent {},/// A file transfer started, progressed or finished.Transfer(Transfer),+ /// A device's media players changed (all of them; empty when nothing+ /// plays there, or the user turned media off for it). Artwork is+ /// attached when it's new for this session; keep it by `art_key`.+ MediaChanged {+ device: DeviceId,+ players: Vec<MediaPlayer>,+ },}diff --git a/core/nectarlink-core/src/features.rs b/core/nectarlink-core/src/features.rsindex c46f6f3..ebba517 100644--- a/core/nectarlink-core/src/features.rs+++ b/core/nectarlink-core/src/features.rs@@ -330,6 +330,7 @@ pub const DEVICE_TOGGLES: &[(&str, bool)] = &[("calls", true),("clipboard", true),("files", true),+ ("media", true),("photos", true),("pc_actions", true),// Browsing the PC's files from anywhere is opt-in.@@ -656,13 +657,17 @@ pub const FEATURES: &[FeatureDef] = &[FeatureDef {id: "media.phone_control",group: FeatureGroup::Media,- requires: &[phone("media.control", Unlock::Permission(Permission::NotificationAccess))],+ requires: &[+ phone("media.control", Unlock::Permission(Permission::NotificationAccess)),+ desktop("media.remote", UPDATE),+ DeviceToggle("media"),+ ],partial: None,},FeatureDef {id: "media.pc_control",group: FeatureGroup::Media,- requires: &[desktop("media.control", UPDATE)],+ requires: &[desktop("media.control", UPDATE), phone("media.remote", UPDATE), DeviceToggle("media")],partial: None,},// Camera@@ -732,10 +737,18 @@ mod tests {"mirror.capture","mirror.audio.playback","media.control",+ "media.remote","camera.stream",];- const PC: &[&str] =- &["pc.power", "files.transfer", "files.browse", "media.control", "input.inject", "deck.actions"];+ const PC: &[&str] = &[+ "pc.power",+ "files.transfer",+ "files.browse",+ "media.control",+ "media.remote",+ "input.inject",+ "deck.actions",+ ];fn pc() -> DeviceFacts {facts(DeviceKind::Laptop, "windows", "10.0.26200", PowerLevel::NotApplicable, PC)diff --git a/core/nectarlink-core/src/lib.rs b/core/nectarlink-core/src/lib.rsindex 21f2998..c7b647c 100644--- a/core/nectarlink-core/src/lib.rs+++ b/core/nectarlink-core/src/lib.rs@@ -12,6 +12,7 @@ mod error;mod events;pub mod features;mod identity;+mod media;mod node;mod notifications;mod pairing;@@ -26,13 +27,15 @@ pub use events::{};pub use features::{CapabilityMatrix, FeatureState};pub use identity::{KeyProtector, PlainKeyProtector, default_protector};+pub use media::{MediaAction, MediaError};pub use nectarlink_protocol::messages::clip::{IMAGE_TYPES as CLIP_IMAGE_TYPES, MAX_IMAGE_BYTES as CLIP_MAX_IMAGE_BYTES,};pub use nectarlink_protocol::{DeviceId,messages::{- Battery, CLIP_MAX_BYTES, DeviceInfo, DeviceKind, Notification, NotificationAction, PowerLevel,+ Battery, CLIP_MAX_BYTES, DeviceInfo, DeviceKind, MediaPlayer, Notification, NotificationAction,+ PowerLevel,},pairing::PairingUri,};diff --git a/core/nectarlink-core/src/media.rs b/core/nectarlink-core/src/media.rsnew file mode 100644index 0000000..66e12fb--- /dev/null+++ b/core/nectarlink-core/src/media.rs@@ -0,0 +1,257 @@+// SPDX-License-Identifier: MPL-2.0+//! Media control between devices (docs/protocol/media.md).+//!+//! A device that shares its players (`media.control`) keeps every allowed+//! device that shows them (`media.remote`) up to date: the full list when a+//! session starts or the user changes what that device may see, then after+//! each change. Artwork goes once per key and session. The other side turns+//! that into [`NodeEvent::MediaChanged`] and sends the user's commands back.++use std::{+ collections::HashMap,+ sync::{Arc, Mutex},+};++use nectarlink_protocol::{+ DeviceId, Envelope, ErrorCode,+ messages::{MediaCommand, MediaPlayer, MediaState, media_actions, media_limits::PLAYERS, types},+};++use crate::{Result, events::NodeEvent, node::Shared, session::Session};++/// The device toggle that allows media for a device.+pub(crate) const TOGGLE: &str = "media";+/// Offered by devices that show and control other devices' players.+const REMOTE: &str = "media.remote";+/// Offered by devices that share their own players.+const SHARES: &str = "media.control";++/// A command for a player.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum MediaAction {+ Play,+ Pause,+ Next,+ Previous,+ /// To the position given with it.+ Seek,+}++impl MediaAction {+ pub fn as_str(self) -> &'static str {+ match self {+ MediaAction::Play => media_actions::PLAY,+ MediaAction::Pause => media_actions::PAUSE,+ MediaAction::Next => media_actions::NEXT,+ MediaAction::Previous => media_actions::PREVIOUS,+ MediaAction::Seek => media_actions::SEEK,+ }+ }++ pub fn parse(action: &str) -> Option<MediaAction> {+ Some(match action {+ media_actions::PLAY => MediaAction::Play,+ media_actions::PAUSE => MediaAction::Pause,+ media_actions::NEXT => MediaAction::Next,+ media_actions::PREVIOUS => MediaAction::Previous,+ media_actions::SEEK => MediaAction::Seek,+ _ => return None,+ })+ }+}++/// Why a media command didn't run.+#[derive(Debug, Clone, PartialEq, Eq)]+pub enum MediaError {+ /// The player is gone.+ NotFound,+ /// The player (or this device) can't do that.+ Unsupported,+ /// Anything else; the string is for logs.+ Failed(String),+}++/// This device's players, as the app last reported them, with their+/// artwork kept apart for sending once per session.+#[derive(Default)]+pub(crate) struct Players {+ inner: Mutex<PlayersState>,+}++#[derive(Default)]+struct PlayersState {+ /// Set once the app reports players: only then does this device send.+ active: bool,+ players: Vec<MediaPlayer>,+ art: HashMap<String, Vec<u8>>,+}++impl Players {+ fn lock(&self) -> std::sync::MutexGuard<'_, PlayersState> {+ self.inner.lock().unwrap_or_else(|e| e.into_inner())+ }++ fn set(&self, players: Vec<MediaPlayer>) {+ let mut state = self.lock();+ state.active = true;+ state.players.clear();+ let mut art = HashMap::new();+ for mut player in players.into_iter().filter_map(MediaPlayer::sanitized).take(PLAYERS) {+ if let (Some(key), Some(bytes)) = (&player.art_key, player.art.take()) {+ art.insert(key.clone(), bytes);+ } else if let Some(key) = &player.art_key+ && let Some(bytes) = state.art.remove(key)+ {+ // Artwork is sent with every report; keep what's still in use.+ art.insert(key.clone(), bytes);+ }+ state.players.push(player);+ }+ state.art = art;+ }++ /// The players for one session, artwork attached where it hasn't had it.+ fn for_session(&self, session: &Session) -> MediaState {+ let state = self.lock();+ let mut sent = session.sent_art();+ let players = state+ .players+ .iter()+ .map(|player| {+ let mut player = player.clone();+ if let Some(key) = &player.art_key+ && let Some(art) = state.art.get(key)+ && sent.insert(key.clone())+ {+ player.art = Some(art.clone());+ }+ player+ })+ .collect();+ MediaState { players }+ }+}++impl Shared {+ fn shows_media(&self, peer: &DeviceId) -> bool {+ self.store.get_peer(peer).ok().flatten().is_some_and(|p| p.caps.contains(REMOTE))+ }++ /// Sends this device's players to one device: all of them when the+ /// user allows it, none (which clears that device) when not.+ pub(crate) async fn send_media_state(&self, session: &Arc<Session>) {+ if !self.players.lock().active || !self.shows_media(&session.peer) {+ return;+ }+ let state = if self.toggle_on(&session.peer, TOGGLE) {+ self.players.for_session(session)+ } else {+ MediaState { players: Vec::new() }+ };+ if let Ok(env) = Envelope::new(types::MEDIA_STATE, &state) {+ let _ = session.send(env).await;+ }+ }++ pub(crate) async fn media_changed(&self, players: Vec<MediaPlayer>) {+ self.players.set(players);+ for session in self.live_sessions() {+ if self.toggle_on(&session.peer, TOGGLE) {+ self.send_media_state(&session).await;+ }+ }+ }++ /// Brings both sides in line after the user allowed or stopped media+ /// for a device.+ pub(crate) fn media_toggled(self: &Arc<Self>, peer: DeviceId, enabled: bool) {+ let Some(session) = self.session(&peer) else {+ if !enabled {+ self.emit(NodeEvent::MediaChanged { device: peer, players: Vec::new() });+ }+ return;+ };+ let shared = self.clone();+ self.runtime.spawn(async move {+ // What this device shares: everything, or nothing.+ shared.send_media_state(&session).await;+ if enabled {+ // What the other one shares: ask for it.+ let _ = session.send(Envelope::empty(types::MEDIA_SYNC)).await;+ } else {+ shared.emit(NodeEvent::MediaChanged { device: peer, players: Vec::new() });+ }+ });+ }+}++/// Handles a `media.*` message on a session. Returns false for other types.+pub(crate) async fn handle(shared: &Arc<Shared>, session: &Arc<Session>, env: &Envelope) -> Result<bool> {+ let peer = session.peer;+ match env.t.as_str() {+ types::MEDIA_STATE => {+ if shared.toggle_on(&peer, TOGGLE) {+ let state: MediaState = env.body()?;+ let players =+ state.players.into_iter().filter_map(MediaPlayer::sanitized).take(PLAYERS).collect();+ shared.emit(NodeEvent::MediaChanged { device: peer, players });+ }+ }+ types::MEDIA_SYNC => shared.send_media_state(session).await,+ types::MEDIA_COMMAND => {+ let command: MediaCommand = env.body()?;+ let reply = run_command(shared, &peer, command).await;+ session.send(reply.reply_to(env.id)).await?;+ }+ _ => return Ok(false),+ }+ Ok(true)+}++async fn run_command(shared: &Arc<Shared>, peer: &DeviceId, command: MediaCommand) -> Envelope {+ if !shared.local_capabilities().iter().any(|c| c == SHARES) {+ return Envelope::error(ErrorCode::Unsupported, "media control is not available");+ }+ if !shared.toggle_on(peer, TOGGLE) {+ return Envelope::error(ErrorCode::Denied, "media is off for this device");+ }+ let Some(action) = MediaAction::parse(&command.action) else {+ return Envelope::error(ErrorCode::Unsupported, "unknown media action");+ };+ if action == MediaAction::Seek && command.position.is_none() {+ return Envelope::error(ErrorCode::BadMessage, "seek needs a position");+ }+ let platform = shared.platform.clone();+ let MediaCommand { player, position, .. } = command;+ let ran = tokio::task::spawn_blocking(move || platform.media_command(&player, action, position))+ .await+ .unwrap_or_else(|e| Err(MediaError::Failed(e.to_string())));+ match ran {+ Ok(()) => Envelope::empty(types::OK),+ Err(MediaError::NotFound) => Envelope::error(ErrorCode::NotFound, "no such player"),+ Err(MediaError::Unsupported) => Envelope::error(ErrorCode::Unsupported, "the player can't do that"),+ Err(MediaError::Failed(reason)) => {+ tracing::warn!(reason, "media command failed");+ Envelope::error(ErrorCode::Internal, "failed")+ }+ }+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn actions_round_trip() {+ for action in [+ MediaAction::Play,+ MediaAction::Pause,+ MediaAction::Next,+ MediaAction::Previous,+ MediaAction::Seek,+ ] {+ assert_eq!(MediaAction::parse(action.as_str()), Some(action));+ }+ assert_eq!(MediaAction::parse("rewind"), None);+ }+}diff --git a/core/nectarlink-core/src/node.rs b/core/nectarlink-core/src/node.rsindex 0018be1..3d439a3 100644--- a/core/nectarlink-core/src/node.rs+++ b/core/nectarlink-core/src/node.rs@@ -20,8 +20,8 @@ use iroh_mdns_address_lookup::{DiscoveryEvent, MdnsAddressLookup};use nectarlink_protocol::{ALPN_PAIR, ALPN_SESSION, DeviceId, Envelope, ErrorCode,messages::{- Battery, ClipSet, DeviceInfo, HelloUpdate, Notification, NotifyAction, NotifyKey, PowerLevel, Ring,- types,+ Battery, ClipSet, DeviceInfo, HelloUpdate, MediaCommand, MediaPlayer, Notification, NotifyAction,+ NotifyKey, PowerLevel, Ring, types,},pairing::PairingUri,};@@ -54,7 +54,7 @@ const IDLE_TIMEOUT: Duration = Duration::from_secs(8);/// Capabilities every build offers.const BASE_CAPABILITIES: &[&str] =- &["core.ping", "device.battery", "device.ring", "files.transfer", "clip.image"];+ &["core.ping", "device.battery", "device.ring", "files.transfer", "clip.image", "media.remote"];/// This device's mutable description, sent to peers.#[derive(Debug, Clone)]@@ -103,13 +103,15 @@ pub(crate) struct Shared {memory: MemoryLookup,/// This device's notifications, when it mirrors them (phones).pub notifications: Feed,+ /// This device's media players (docs/protocol/media.md).+ pub players: crate::media::Players,pub data_dir: std::path::PathBuf,/// Where received files go.pub downloads_dir: std::path::PathBuf,/// Running transfers, to cancel them.transfers: Mutex<HashMap<String, CancellationToken>>,/// The node's runtime, for work started from non-async callers.- runtime: tokio::runtime::Handle,+ pub(crate) runtime: tokio::runtime::Handle,}impl std::fmt::Debug for Shared {@@ -337,6 +339,7 @@ impl Shared {let _ = s.send(env).await;}shared.send_notification_snapshot(&s).await;+ shared.send_media_state(&s).await;});Some(session)}@@ -575,7 +578,7 @@ impl Node {app_version: config.app_version.clone(),device: config.device.clone(),power: config.power,- extra_capabilities: Vec::new(),+ extra_capabilities: config.capabilities.clone(),battery: None,}),pairing: PairingState::default(),@@ -589,6 +592,7 @@ impl Node {away_mode: config.away_mode,memory,notifications: Feed::default(),+ players: Default::default(),data_dir: config.data_dir.clone(),downloads_dir: config.downloads_dir.clone().unwrap_or_else(|| config.data_dir.join("received")),transfers: Mutex::new(HashMap::new()),@@ -797,6 +801,9 @@ impl Node {if toggle == notifications::TOGGLE {self.notifications_toggled(peer, enabled);}+ if toggle == crate::media::TOGGLE {+ self.shared.media_toggled(peer, enabled);+ }Ok(())}@@ -869,6 +876,32 @@ impl Node {crate::clipboard::send_image(&self.shared, &session, mime, bytes).await}+ // ---- Media (docs/protocol/media.md) ----++ /// This device's media players changed (most relevant first); sent to+ /// every connected device the user allows. Include each player's+ /// artwork every time: the core sends it once per device and session.+ pub async fn media_changed(&self, players: Vec<MediaPlayer>) {+ self.shared.media_changed(players).await;+ }++ /// Runs a command on a paired device's media player.+ pub async fn media_command(+ &self,+ peer: DeviceId,+ player: String,+ action: crate::MediaAction,+ position: Option<u64>,+ ) -> Result<()> {+ if !self.shared.toggle_on(&peer, crate::media::TOGGLE) {+ return Err(Error::Denied);+ }+ let command = MediaCommand { player, action: action.as_str().into(), position };+ let env = Envelope::new(types::MEDIA_COMMAND, &command)?;+ self.request(peer, env).await?.expect(types::OK)?;+ Ok(())+ }+// ---- Notifications (docs/protocol/notifications.md) ----/// A notification appeared or changed on this phone; sent to everydiff --git a/core/nectarlink-core/src/session.rs b/core/nectarlink-core/src/session.rsindex d2107e1..26af288 100644--- a/core/nectarlink-core/src/session.rs+++ b/core/nectarlink-core/src/session.rs@@ -114,6 +114,8 @@ pub(crate) struct Session {rtt_ms: AtomicU32,/// Apps whose notification icon this peer already got.sent_icons: crate::notifications::SentIcons,+ /// Artwork keys sent to this device (docs/protocol/media.md).+ sent_art: crate::notifications::SentIcons,pub(crate) cancel: CancellationToken,}@@ -154,6 +156,7 @@ impl Session {next_id: AtomicU64::new(1),rtt_ms: AtomicU32::new(initial_rtt),sent_icons: Default::default(),+ sent_art: Default::default(),cancel: shared.cancel.child_token(),});@@ -204,6 +207,10 @@ impl Session {self.sent_icons.lock().unwrap_or_else(|e| e.into_inner())}+ pub(crate) fn sent_art(&self) -> std::sync::MutexGuard<'_, std::collections::HashSet<String>> {+ self.sent_art.lock().unwrap_or_else(|e| e.into_inner())+ }+pub fn is_alive(&self) -> bool {!self.cancel.is_cancelled() && self.conn.close_reason().is_none()}@@ -339,6 +346,7 @@ async fn handle(shared: &Arc<Shared>, session: &Arc<Session>, env: Envelope) ->shared.forget_peer(&peer).await?;}t if t.starts_with("notify.") && crate::notifications::handle(shared, session, &env).await? => {}+ t if t.starts_with("media.") && crate::media::handle(shared, session, &env).await? => {}t if t.starts_with("clip.") && crate::clipboard::handle(shared, session, &env).await? => {}other => {if env.id.is_some() {diff --git a/core/nectarlink-core/tests/two_devices.rs b/core/nectarlink-core/tests/two_devices.rsindex 6dc8935..b02e0b3 100644--- a/core/nectarlink-core/tests/two_devices.rs+++ b/core/nectarlink-core/tests/two_devices.rs@@ -9,10 +9,10 @@ use std::{};use nectarlink_core::{- Battery, DeviceInfo, DeviceKind, Direction, Error, FeatureState, FileSource, LinkState, Node, NodeConfig,- NodeEvent, Notification, NotificationAction, NotificationError, OutgoingFile, PairingEvent,- PairingFailure, PairingUri, PlainKeyProtector, Platform, PowerLevel, Transfer, TransferFailure,- TransferState,+ Battery, DeviceInfo, DeviceKind, Direction, Error, FeatureState, FileSource, LinkState, MediaAction,+ MediaError, MediaPlayer, Node, NodeConfig, NodeEvent, Notification, NotificationAction,+ NotificationError, OutgoingFile, PairingEvent, PairingFailure, PairingUri, PlainKeyProtector, Platform,+ PowerLevel, Transfer, TransferFailure, TransferState,features::{Effort, Upgrade, UpgradeAction},};use tempfile::TempDir;@@ -27,6 +27,7 @@ struct RecordingPlatform {actions: Mutex<Vec<(String, String, Option<String>)>>,clipboard: Mutex<Vec<String>>,images: Mutex<Vec<(String, Vec<u8>)>>,+ media: Mutex<Vec<(String, MediaAction, Option<u64>)>>,}impl Platform for RecordingPlatform {@@ -60,6 +61,18 @@ impl Platform for RecordingPlatform {self.images.lock().unwrap().push((mime.to_owned(), bytes.to_vec()));Ok(())}+ fn media_command(+ &self,+ player: &str,+ action: MediaAction,+ position: Option<u64>,+ ) -> Result<(), MediaError> {+ if player == "gone" {+ return Err(MediaError::NotFound);+ }+ self.media.lock().unwrap().push((player.to_owned(), action, position));+ Ok(())+ }}struct TestDevice {@@ -95,6 +108,7 @@ async fn start_in(dir: TempDir, name: &'static str, kind: DeviceKind) -> TestDevconfig.lan_discovery = false;config.away_mode = false;config.key_protector = Some(Arc::new(PlainKeyProtector));+ config.capabilities = vec!["media.control".into()];let platform = Arc::new(RecordingPlatform::default());let node = Node::start(config, platform.clone()).await.expect("node starts");let events = node.events();@@ -640,6 +654,114 @@ async fn clipboard_images_go_both_ways_with_consent() {.expect_err("still off");}+fn player(id: &str, title: &str, art: Option<(&str, Vec<u8>)>) -> MediaPlayer {+ MediaPlayer {+ id: id.into(),+ app: "Music".into(),+ title: Some(title.into()),+ artist: Some("Artist".into()),+ album: None,+ playing: true,+ duration: Some(200_000),+ position: Some(12_000),+ actions: vec!["play".into(), "pause".into(), "next".into(), "seek".into(), "bogus".into()],+ art_key: art.as_ref().map(|(key, _)| (*key).into()),+ art: art.map(|(_, bytes)| bytes),+ }+}++fn media_of(device: nectarlink_core::DeviceId) -> impl FnMut(&NodeEvent) -> Option<Vec<MediaPlayer>> {+ move |e| match e {+ NodeEvent::MediaChanged { device: d, players } if *d == device => Some(players.clone()),+ _ => None,+ }+}++#[tokio::test(flavor = "multi_thread")]+async fn media_is_shared_and_controlled_with_consent() {+ let mut pc = device("Desktop", DeviceKind::Desktop).await;+ let mut phone = device("Pixel", DeviceKind::Phone).await;+ pair_qr(&mut pc, &mut phone).await;+ let (pc_id, phone_id) = (pc.node.device_id(), phone.node.device_id());++ // The phone plays something; the PC gets it with its artwork.+ let art = data(30_000, 7);+ phone.node.media_changed(vec![player("com.music", "Song", Some(("a1", art.clone())))]).await;+ let players = wait_for(&mut pc, "media", media_of(phone_id)).await;+ assert_eq!(players.len(), 1);+ assert_eq!(players[0].title.as_deref(), Some("Song"));+ assert_eq!(players[0].art.as_deref(), Some(&art[..]));+ assert_eq!(players[0].actions, ["play", "pause", "next", "seek"], "unknown actions are dropped");++ // Artwork goes once per session; a new picture goes again.+ phone.node.media_changed(vec![player("com.music", "Song", Some(("a1", art.clone())))]).await;+ let players = wait_for(&mut pc, "same art", media_of(phone_id)).await;+ assert_eq!((players[0].art_key.as_deref(), players[0].art.as_ref()), (Some("a1"), None));+ phone.node.media_changed(vec![player("com.music", "Next song", Some(("a2", vec![9; 10])))]).await;+ let players = wait_for(&mut pc, "new art", media_of(phone_id)).await;+ assert_eq!(players[0].art.as_deref(), Some(&[9; 10][..]));++ // The PC controls it.+ with_timeout("pause", pc.node.media_command(phone_id, "com.music".into(), MediaAction::Pause, None))+ .await+ .expect("paused");+ with_timeout(+ "seek",+ pc.node.media_command(phone_id, "com.music".into(), MediaAction::Seek, Some(60_000)),+ )+ .await+ .expect("sought");+ assert_eq!(+ *phone.platform.media.lock().unwrap(),+ [+ ("com.music".to_owned(), MediaAction::Pause, None),+ ("com.music".to_owned(), MediaAction::Seek, Some(60_000))+ ]+ );+ let gone = pc.node.media_command(phone_id, "gone".into(), MediaAction::Play, None).await;+ assert!(matches!(gone, Err(Error::NotFound)), "{gone:?}");+ let no_position = pc.node.media_command(phone_id, "com.music".into(), MediaAction::Seek, None).await;+ assert!(no_position.is_err(), "{no_position:?}");++ // The PC turns media off for the phone: it's cleared there at once,+ // and the phone stops taking its commands only if the phone turns it off.+ pc.node.set_device_toggle(phone_id, "media", false).unwrap();+ assert!(wait_for(&mut pc, "cleared", media_of(phone_id)).await.is_empty());+ let off = pc.node.media_command(phone_id, "com.music".into(), MediaAction::Play, None).await;+ assert!(matches!(off, Err(Error::Denied)), "{off:?}");+ // On again: the PC asks for what plays now.+ pc.node.set_device_toggle(phone_id, "media", true).unwrap();+ let players = wait_for(&mut pc, "synced", media_of(phone_id)).await;+ assert_eq!(players[0].title.as_deref(), Some("Next song"));++ // The phone turns it off for the PC: the PC is told nothing plays, and+ // commands are refused.+ phone.node.set_device_toggle(pc_id, "media", false).unwrap();+ assert!(wait_for(&mut pc, "cleared by the phone", media_of(phone_id)).await.is_empty());+ let denied = pc.node.media_command(phone_id, "com.music".into(), MediaAction::Play, None).await;+ assert!(matches!(denied, Err(Error::Denied)), "{denied:?}");++ // And the other way: the PC's own players reach the phone once the+ // phone allows media again.+ phone.node.set_device_toggle(pc_id, "media", true).unwrap();+ pc.node.media_changed(vec![player("Spotify.exe", "On the PC", None)]).await;+ let players = wait_for(&mut phone, "pc media", |e| match e {+ NodeEvent::MediaChanged { device, players } if *device == pc_id && !players.is_empty() => {+ Some(players.clone())+ }+ _ => None,+ })+ .await;+ assert_eq!(players[0].title.as_deref(), Some("On the PC"));+ with_timeout(+ "phone controls the pc",+ phone.node.media_command(pc_id, "Spotify.exe".into(), MediaAction::Next, None),+ )+ .await+ .expect("next");+ assert_eq!(pc.platform.media.lock().unwrap()[0], ("Spotify.exe".to_owned(), MediaAction::Next, None));+}+/// Deterministic, incompressible-looking test data.fn data(len: usize, seed: u64) -> Vec<u8> {let mut x = seed.wrapping_mul(0x9E37_79B9_7F4A_7C15) | 1;diff --git a/core/nectarlink-ffi/src/lib.rs b/core/nectarlink-ffi/src/lib.rsindex 9ec5c5a..3facb82 100644--- a/core/nectarlink-ffi/src/lib.rs+++ b/core/nectarlink-ffi/src/lib.rs@@ -165,6 +165,105 @@ pub struct Notification {pub icon: Option<Vec<u8>>,}+/// Something playing (or paused) on a device (docs/protocol/media.md).+#[derive(Clone, PartialEq, Eq, uniffi::Record)]+pub struct MediaPlayer {+ /// Stable while the player exists (Android: the app's package name).+ pub id: String,+ /// User-visible app name.+ pub app: String,+ pub title: Option<String>,+ pub artist: Option<String>,+ pub album: Option<String>,+ pub playing: bool,+ /// Milliseconds.+ pub duration: Option<u64>,+ /// Milliseconds, now; it moves on in real time while `playing`.+ pub position: Option<u64>,+ pub actions: Vec<MediaAction>,+ /// Identifies the artwork: the same key, the same picture.+ pub art_key: Option<String>,+ /// JPEG or PNG. Pass it every time; the core sends it once per device+ /// and connection. Arrives the same way: keep it by `art_key`.+ pub art: Option<Vec<u8>>,+}++/// Never prints what's playing (protocol v0 §11).+impl std::fmt::Debug for MediaPlayer {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.debug_struct("MediaPlayer").field("id", &self.id).finish_non_exhaustive()+ }+}++#[derive(Debug, Clone, Copy, PartialEq, Eq, uniffi::Enum)]+pub enum MediaAction {+ Play,+ Pause,+ Next,+ Previous,+ Seek,+}++impl From<MediaAction> for core::MediaAction {+ fn from(a: MediaAction) -> Self {+ match a {+ MediaAction::Play => core::MediaAction::Play,+ MediaAction::Pause => core::MediaAction::Pause,+ MediaAction::Next => core::MediaAction::Next,+ MediaAction::Previous => core::MediaAction::Previous,+ MediaAction::Seek => core::MediaAction::Seek,+ }+ }+}++impl From<core::MediaAction> for MediaAction {+ fn from(a: core::MediaAction) -> Self {+ match a {+ core::MediaAction::Play => MediaAction::Play,+ core::MediaAction::Pause => MediaAction::Pause,+ core::MediaAction::Next => MediaAction::Next,+ core::MediaAction::Previous => MediaAction::Previous,+ core::MediaAction::Seek => MediaAction::Seek,+ }+ }+}++impl From<MediaPlayer> for core::MediaPlayer {+ fn from(p: MediaPlayer) -> Self {+ core::MediaPlayer {+ id: p.id,+ app: p.app,+ title: p.title,+ artist: p.artist,+ album: p.album,+ playing: p.playing,+ duration: p.duration,+ position: p.position,+ actions: p.actions.into_iter().map(|a| core::MediaAction::from(a).as_str().to_owned()).collect(),+ art_key: p.art_key,+ art: p.art,+ }+ }+}++impl From<core::MediaPlayer> for MediaPlayer {+ fn from(p: core::MediaPlayer) -> Self {+ MediaPlayer {+ id: p.id,+ app: p.app,+ title: p.title,+ artist: p.artist,+ album: p.album,+ playing: p.playing,+ duration: p.duration,+ position: p.position,+ actions: p.actions.iter().filter_map(|a| core::MediaAction::parse(a)).map(Into::into).collect(),+ art_key: p.art_key,+ art: p.art,+ }+ }+}+/// Never prints content (protocol v0 §11).impl std::fmt::Debug for Notification {fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {@@ -308,6 +407,12 @@ pub enum Event {Transfer {transfer: Transfer,},+ /// A device's media players (all of them; empty when nothing plays+ /// there, or media is off for it).+ MediaChanged {+ id: String,+ players: Vec<MediaPlayer>,+ },}#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error, uniffi::Error)]@@ -667,6 +772,10 @@ impl From<NodeEvent> for Event {}NodeEvent::ClipboardReceived { device } => Event::ClipboardReceived { id: device.to_string() },NodeEvent::Transfer(t) => Event::Transfer { transfer: t.into() },+ NodeEvent::MediaChanged { device, players } => Event::MediaChanged {+ id: device.to_string(),+ players: players.into_iter().map(Into::into).collect(),+ },}}}@@ -707,6 +816,36 @@ impl From<NotificationFailure> for core::NotificationError {}}+/// Why a media command didn't run.+#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error, uniffi::Error)]+pub enum MediaFailure {+ /// The player is gone.+ #[error("no such player")]+ NotFound,+ /// The player can't do that (or media access was revoked).+ #[error("not available")]+ Unsupported,+ /// `reason` is for logs.+ #[error("failed: {reason}")]+ Failed { reason: String },+}++impl From<uniffi::UnexpectedUniFFICallbackError> for MediaFailure {+ fn from(e: uniffi::UnexpectedUniFFICallbackError) -> Self {+ MediaFailure::Failed { reason: e.reason }+ }+}++impl From<MediaFailure> for core::MediaError {+ fn from(f: MediaFailure) -> Self {+ match f {+ MediaFailure::NotFound => core::MediaError::NotFound,+ MediaFailure::Unsupported => core::MediaError::Unsupported,+ MediaFailure::Failed { reason } => core::MediaError::Failed(reason),+ }+ }+}+/// What the app does for the core: ring the phone, act on its/// notifications.#[uniffi::export(with_foreign)]@@ -729,6 +868,14 @@ pub trait Platform: Send + Sync {/// A PC sent this image (`image/png` or `image/jpeg`): put it on the/// clipboard. False if that failed.fn set_clipboard_image(&self, mime: String, bytes: Vec<u8>) -> bool;+ /// A paired device asked one of this phone's media players to do+ /// something; `position` (milliseconds) is set for seeking.+ fn media_command(+ &self,+ player: String,+ action: MediaAction,+ position: Option<u64>,+ ) -> Result<(), MediaFailure>;}/// Encrypts the device key at rest (Android: a Keystore key).@@ -773,6 +920,14 @@ impl core::Platform for PlatformAdapter {fn set_clipboard(&self, text: &str) -> Result<(), String> {if self.0.set_clipboard(text.to_owned()) { Ok(()) } else { Err("the clipboard rejected it".into()) }}+ fn media_command(+ &self,+ player: &str,+ action: core::MediaAction,+ position: Option<u64>,+ ) -> Result<(), core::MediaError> {+ Ok(self.0.media_command(player.to_owned(), action.into(), position)?)+ }fn set_clipboard_image(&self, mime: &str, bytes: &[u8]) -> Result<(), String> {if self.0.set_clipboard_image(mime.to_owned(), bytes.to_vec()) {Ok(())@@ -997,6 +1152,29 @@ impl NectarlinkNode {self.run(async move { Ok(node.send_clipboard_image(id, mime, bytes).await?) }).await}+ // ---- Media ----++ /// This phone's media players changed (most relevant first).+ pub async fn media_changed(&self, players: Vec<MediaPlayer>) {+ let node = self.node.clone();+ self.run(async move { node.media_changed(players.into_iter().map(Into::into).collect()).await })+ .await;+ }++ /// Runs a command on a paired device's media player; `position`+ /// (milliseconds) for seeking.+ pub async fn media_command(+ &self,+ id: String,+ player: String,+ action: MediaAction,+ position: Option<u64>,+ ) -> Result<()> {+ let id = parse_id(&id)?;+ let node = self.node.clone();+ self.run(async move { Ok(node.media_command(id, player, action.into(), position).await?) }).await+ }+// ---- Notifications ----/// A notification appeared or changed on this phone.diff --git a/core/nectarlink-protocol/src/messages.rs b/core/nectarlink-protocol/src/messages.rsindex 748ca30..41ad778 100644--- a/core/nectarlink-protocol/src/messages.rs+++ b/core/nectarlink-protocol/src/messages.rs@@ -35,6 +35,9 @@ pub mod types {pub const CLIP_SET: &str = "clip.set";pub const NOTIFY_DISMISS: &str = "notify.dismiss";pub const NOTIFY_ACTION: &str = "notify.action";+ pub const MEDIA_STATE: &str = "media.state";+ pub const MEDIA_SYNC: &str = "media.sync";+ pub const MEDIA_COMMAND: &str = "media.command";}/// What kind of device this is.@@ -317,6 +320,119 @@ impl std::fmt::Debug for NotifyAction {}}+// ---- Media (docs/protocol/media.md) ----++/// Limits from docs/protocol/media.md.+pub mod media_limits {+ pub const PLAYERS: usize = 8;+ pub const ID_BYTES: usize = 256;+ pub const TEXT_CHARS: usize = 256;+ pub const ART_KEY_BYTES: usize = 64;+ pub const ART_BYTES: usize = 256 * 1024;+}++/// What a player can be asked to do (`MediaPlayer::actions`,+/// `MediaCommand::action`).+pub mod media_actions {+ pub const PLAY: &str = "play";+ pub const PAUSE: &str = "pause";+ pub const NEXT: &str = "next";+ pub const PREVIOUS: &str = "previous";+ pub const SEEK: &str = "seek";+ pub const ALL: &[&str] = &[PLAY, PAUSE, NEXT, PREVIOUS, SEEK];+}++/// Something playing (or paused) on a device.+#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]+pub struct MediaPlayer {+ /// The device's ID for the player, stable while it exists (e.g. the+ /// app's package name).+ pub id: String,+ /// User-visible app name.+ pub app: String,+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub title: Option<String>,+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub artist: Option<String>,+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub album: Option<String>,+ #[serde(default)]+ pub playing: bool,+ /// Length, milliseconds.+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub duration: Option<u64>,+ /// Position when this was sent, milliseconds; it moves on in real time+ /// while `playing`.+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub position: Option<u64>,+ /// Commands it takes, from [`media_actions`].+ #[serde(default)]+ pub actions: Vec<String>,+ /// Identifies the artwork; the same key means the same picture.+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub art_key: Option<String>,+ /// The artwork (JPEG or PNG), sent once per key and session.+ #[serde(default, skip_serializing_if = "Option::is_none", with = "serde_bytes")]+ pub art: Option<Vec<u8>>,+}++/// Never prints what's playing (protocol v0 §11).+impl std::fmt::Debug for MediaPlayer {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.debug_struct("MediaPlayer")+ .field("id", &self.id)+ .field("playing", &self.playing)+ .field("actions", &self.actions)+ .finish_non_exhaustive()+ }+}++impl MediaPlayer {+ /// Applies the spec's limits. `None` if it can't be shown (no ID, or+ /// nothing to show it by).+ pub fn sanitized(mut self) -> Option<MediaPlayer> {+ use media_limits::*;+ if self.id.is_empty() || self.id.len() > ID_BYTES {+ return None;+ }+ truncate_chars(&mut self.app, TEXT_CHARS);+ self.title = clean_text(self.title, TEXT_CHARS);+ self.artist = clean_text(self.artist, TEXT_CHARS);+ self.album = clean_text(self.album, TEXT_CHARS);+ if self.title.is_none() && self.app.trim().is_empty() {+ return None;+ }+ self.actions.retain(|a| media_actions::ALL.contains(&a.as_str()));+ self.actions.dedup();+ if let (Some(position), Some(duration)) = (self.position, self.duration) {+ self.position = Some(position.min(duration));+ }+ if self.art_key.as_ref().is_some_and(|k| k.is_empty() || k.len() > ART_KEY_BYTES) {+ self.art_key = None;+ }+ if self.art_key.is_none() || self.art.as_ref().is_some_and(|a| a.is_empty() || a.len() > ART_BYTES) {+ self.art = None;+ }+ Some(self)+ }+}++/// Body of `media.state`: every player, most relevant first.+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]+pub struct MediaState {+ pub players: Vec<MediaPlayer>,+}++/// Body of `media.command`.+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]+pub struct MediaCommand {+ pub player: String,+ pub action: String,+ /// For `seek`: where to, milliseconds.+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub position: Option<u64>,+}+// ---- Files (docs/protocol/files.md) ----/// The files service and its one operation, for stream headers.diff --git a/desktop/app/Cargo.toml b/desktop/app/Cargo.tomlindex 8f24735..a83fe5d 100644--- a/desktop/app/Cargo.toml+++ b/desktop/app/Cargo.toml@@ -35,6 +35,10 @@ windows = { workspace = true, features = ["Data_Xml_Dom","Foundation","Foundation_Collections",+ "Media",+ "Media_Control",+ "Storage_Streams",+ "ApplicationModel","UI_Notifications","Win32_Foundation","Win32_Graphics_Dwm",@@ -49,6 +53,7 @@ windows = { workspace = true, features = ["Win32_System_Power","Win32_System_Registry","Win32_System_Threading",+ "Win32_System_WinRT","Win32_UI_Shell","Win32_UI_WindowsAndMessaging","Win32_NetworkManagement_IpHelper",diff --git a/desktop/app/build.rs b/desktop/app/build.rsindex 8bed864..0f0f261 100644--- a/desktop/app/build.rs+++ b/desktop/app/build.rs@@ -31,6 +31,7 @@ const QML: &[&str] = &["qml/components/LockChip.qml","qml/components/NavItem.qml","qml/components/NotificationItem.qml",+ "qml/components/NowPlaying.qml","qml/components/QrCode.qml","qml/components/Segmented.qml","qml/components/Sheet.qml",@@ -186,6 +187,7 @@ fn main() {"src/bridge/native.rs","src/bridge/app.rs","src/bridge/devices.rs",+ "src/bridge/media.rs","src/bridge/notifications.rs","src/bridge/pairing.rs","src/bridge/prefs.rs",diff --git a/desktop/app/qml/Icons.qml b/desktop/app/qml/Icons.qmlindex d062619..2da8666 100644--- a/desktop/app/qml/Icons.qml+++ b/desktop/app/qml/Icons.qml@@ -15,6 +15,11 @@ QtObject {readonly property string clipboard: "M9 3h6v4H9zM16 5h2a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V7a2 2 0 0 1 2-2h2"readonly property string mirror: "M4 4h10a2 2 0 0 1 2 2v6a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2zM18.5 8h2a1.5 1.5 0 0 1 1.5 1.5v7a1.5 1.5 0 0 1-1.5 1.5h-2A1.5 1.5 0 0 1 17 16.5v-7A1.5 1.5 0 0 1 18.5 8zM6 18h6"readonly property string send: "M22 2L11 13M22 2l-7 20-4-9-9-4z"+ readonly property string play: "M8 5.5v13a1 1 0 0 0 1.5.86l10.5-6.5a1 1 0 0 0 0-1.72L9.5 4.64A1 1 0 0 0 8 5.5z"+ readonly property string pause: "M9 5v14M15 5v14"+ readonly property string skipNext: "M5 6.2v11.6a1 1 0 0 0 1.6.8l7.4-5.8a1 1 0 0 0 0-1.6L6.6 5.4a1 1 0 0 0-1.6.8zM19 5v14"+ readonly property string skipPrevious: "M19 6.2v11.6a1 1 0 0 1-1.6.8L10 12.8a1 1 0 0 1 0-1.6l7.4-5.8a1 1 0 0 1 1.6.8zM5 5v14"+ readonly property string music: "M9 18V5l12-2v13M9 18a3 3 0 1 1-6 0 3 3 0 0 1 6 0zM21 16a3 3 0 1 1-6 0 3 3 0 0 1 6 0z"readonly property string ring: "M6 8a6 6 0 1 1 12 0c0 7 3 8 3 8H3s3-1 3-8M10 20a2 2 0 0 0 4 0M2 8c0-2 1-4 2-5M22 8c0-2-1-4-2-5"readonly property string mic: "M12 2a3 3 0 0 1 3 3v6a3 3 0 0 1-6 0V5a3 3 0 0 1 3-3zM5 11a7 7 0 0 0 14 0M12 18v4"readonly property string lock: "M7 11h10a2 2 0 0 1 2 2v6a2 2 0 0 1-2 2H7a2 2 0 0 1-2-2v-6a2 2 0 0 1 2-2zM8 11V8a4 4 0 0 1 8 0v3"diff --git a/desktop/app/qml/components/NowPlaying.qml b/desktop/app/qml/components/NowPlaying.qmlnew file mode 100644index 0000000..9f0e27d--- /dev/null+++ b/desktop/app/qml/components/NowPlaying.qml@@ -0,0 +1,201 @@+// SPDX-License-Identifier: GPL-3.0-or-later+import QtQuick+import app.nectarlink++// What plays on a phone: artwork, title and artist, a seek bar that moves+// with the music, and the player's controls. Commands go to the phone;+// what it plays next comes back as new state.+Card {+ id: card+ required property string deviceId+ required property string playerId+ required property string app+ required property string title+ required property string artist+ required property bool playing+ required property real duration+ required property real position+ required property real positionAt+ required property string art+ required property bool canPlay+ required property bool canPause+ required property bool canNext+ required property bool canPrevious+ required property bool canSeek++ // Time, ticking while it plays so the bar and clock move.+ property real now: Date.now()+ Timer {+ interval: 250+ repeat: true+ running: card.visible && card.playing && card.duration > 0+ onTriggered: card.now = Date.now()+ onRunningChanged: card.now = Date.now()+ }+ onPositionAtChanged: now = Date.now()++ readonly property real elapsed: position < 0 ? 0+ : Math.min(duration > 0 ? duration : Infinity, position + (playing ? Math.max(0, now - positionAt) : 0))+ // While dragging, the bar shows where the user is pointing.+ property real dragTo: -1+ readonly property real shown: dragTo >= 0 ? dragTo : elapsed++ function send(action, to) { MediaList.command(deviceId, playerId, action, to === undefined ? 0 : to) }+ function clock(ms) {+ const s = Math.floor(ms / 1000)+ const h = Math.floor(s / 3600)+ const m = Math.floor((s % 3600) / 60)+ const ss = String(s % 60).padStart(2, "0")+ return h > 0 ? h + ":" + String(m).padStart(2, "0") + ":" + ss : m + ":" + ss+ }++ Column {+ width: parent.width+ spacing: 14++ Txt { text: qsTr("Now playing"); role: "label"; muted: true }++ Row {+ width: parent.width+ spacing: 14+ Rectangle {+ id: cover+ width: 72; height: 72+ radius: Theme.radiusMd+ color: Theme.secondaryContainer+ clip: true+ Icon {+ anchors.centerIn: parent+ width: 28; height: 28+ visible: artwork.status !== Image.Ready+ path: Icons.music+ color: Theme.secondaryContainerContent+ }+ Image {+ id: artwork+ anchors.fill: parent+ source: card.art+ fillMode: Image.PreserveAspectCrop+ asynchronous: true+ sourceSize: Qt.size(144, 144)+ opacity: status === Image.Ready ? 1 : 0+ Behavior on opacity { NumberAnimation { duration: Theme.fadeNormal } }+ }+ }+ Column {+ width: parent.width - cover.width - parent.spacing+ anchors.verticalCenter: parent.verticalCenter+ spacing: 3+ Txt {+ width: parent.width+ role: "title"+ text: card.title.length > 0 ? card.title : card.app+ elide: Text.ElideRight+ maximumLineCount: 2+ wrapMode: Text.WordWrap+ }+ Txt {+ width: parent.width+ visible: card.artist.length > 0+ role: "bodySmall"+ muted: true+ text: card.artist+ elide: Text.ElideRight+ }+ Txt {+ width: parent.width+ role: "caption"+ muted: true+ text: card.app+ elide: Text.ElideRight+ }+ }+ }++ // ---- Seek bar ----+ Column {+ width: parent.width+ spacing: 4+ visible: card.duration > 0+ Item {+ id: bar+ width: parent.width+ height: 16+ readonly property real fraction: card.duration > 0 ? Math.max(0, Math.min(1, card.shown / card.duration)) : 0+ Rectangle {+ id: track+ anchors.verticalCenter: parent.verticalCenter+ width: parent.width+ height: 4+ radius: 2+ color: Theme.surfaceContainerHighest+ Rectangle {+ width: parent.width * bar.fraction+ height: parent.height+ radius: parent.radius+ color: Theme.primary+ }+ }+ Rectangle {+ visible: card.canSeek && (seek.containsMouse || seek.pressed)+ x: bar.width * bar.fraction - width / 2+ anchors.verticalCenter: parent.verticalCenter+ width: 12; height: 12; radius: 6+ color: Theme.primary+ }+ MouseArea {+ id: seek+ anchors.fill: parent+ anchors.margins: -4+ enabled: card.canSeek+ hoverEnabled: true+ cursorShape: card.canSeek ? Qt.PointingHandCursor : Qt.ArrowCursor+ function at(x) { return Math.max(0, Math.min(1, (x - 4) / bar.width)) * card.duration }+ onPressed: (mouse) => card.dragTo = at(mouse.x)+ onPositionChanged: (mouse) => { if (pressed) card.dragTo = at(mouse.x) }+ onReleased: (mouse) => {+ card.send("seek", Math.round(at(mouse.x)))+ card.dragTo = -1+ }+ onCanceled: card.dragTo = -1+ }+ }+ Item {+ width: parent.width+ height: elapsedText.height+ Txt { id: elapsedText; role: "caption"; muted: true; text: card.clock(card.shown) }+ Txt { anchors.right: parent.right; role: "caption"; muted: true; text: card.clock(card.duration) }+ }+ }++ // ---- Controls ----+ Row {+ anchors.horizontalCenter: parent.horizontalCenter+ spacing: 18+ IconButton {+ anchors.verticalCenter: parent.verticalCenter+ iconPath: Icons.skipPrevious+ label: qsTr("Previous")+ enabled: card.canPrevious+ onClicked: card.send("previous")+ }+ IconButton {+ id: playPause+ anchors.verticalCenter: parent.verticalCenter+ width: 48; height: 48+ tonal: true+ iconPath: card.playing ? Icons.pause : Icons.play+ label: card.playing ? qsTr("Pause") : qsTr("Play")+ enabled: card.playing ? card.canPause : card.canPlay+ onClicked: card.send(card.playing ? "pause" : "play")+ }+ IconButton {+ anchors.verticalCenter: parent.verticalCenter+ iconPath: Icons.skipNext+ label: qsTr("Next")+ enabled: card.canNext+ onClicked: card.send("next")+ }+ }+ }+}diff --git a/desktop/app/qml/pages/HomePage.qml b/desktop/app/qml/pages/HomePage.qmlindex 662ba05..cc3ca72 100644--- a/desktop/app/qml/pages/HomePage.qml+++ b/desktop/app/qml/pages/HomePage.qml@@ -480,6 +480,17 @@ Item {width: home.sideWidthspacing: Theme.gutter+ // What plays on the phone (its first player).+ Repeater {+ model: MediaList+ delegate: NowPlaying {+ required property int index+ width: home.sideWidth+ visible: deviceId === home.deviceId+ && MediaList.revision >= 0 && index === MediaList.firstFor(home.deviceId)+ }+ }+Card {width: parent.widthColumn {diff --git a/desktop/app/qml/pages/SettingsPage.qml b/desktop/app/qml/pages/SettingsPage.qmlindex 30fa963..579f337 100644--- a/desktop/app/qml/pages/SettingsPage.qml+++ b/desktop/app/qml/pages/SettingsPage.qml@@ -22,6 +22,8 @@ Item {messages: qsTr("Messages"),calls: qsTr("Calls"),clipboard: qsTr("Clipboard"),+ files: qsTr("Files"),+ media: qsTr("Media playing on either device"),photos: qsTr("Photos"),pc_actions: qsTr("Lock and sleep this PC"),remote_files: qsTr("Browse this PC's files while away")diff --git a/desktop/app/src/bridge/media.rs b/desktop/app/src/bridge/media.rsnew file mode 100644index 0000000..f2d260e--- /dev/null+++ b/desktop/app/src/bridge/media.rs@@ -0,0 +1,285 @@+// SPDX-License-Identifier: GPL-3.0-or-later+//! `MediaList`: what plays on paired devices as a Qt list model, plus the+//! commands QML sends to those players.++use std::{pin::Pin, time::UNIX_EPOCH};++use cxx_qt::{CxxQtType, Threading};+use cxx_qt_lib::{QByteArray, QHash, QHashPair_i32_QByteArray, QList, QModelIndex, QString, QUrl, QVariant};+use nectarlink_core::MediaAction;++use super::{Edit, diff};+use crate::{+ core_host, media,+ state::{Changes, PlayerView},+};++#[cxx_qt::bridge]+pub mod qobject {+ unsafe extern "C++" {+ include!(<QtCore/QAbstractListModel>);+ type QAbstractListModel;++ include!("cxx-qt-lib/qmodelindex.h");+ type QModelIndex = cxx_qt_lib::QModelIndex;+ include!("cxx-qt-lib/qvariant.h");+ type QVariant = cxx_qt_lib::QVariant;+ include!("cxx-qt-lib/qhash.h");+ type QHash_i32_QByteArray = cxx_qt_lib::QHash<cxx_qt_lib::QHashPair_i32_QByteArray>;+ include!("cxx-qt-lib/qlist.h");+ type QList_i32 = cxx_qt_lib::QList<i32>;+ include!("cxx-qt-lib/qstring.h");+ type QString = cxx_qt_lib::QString;+ }++ unsafe extern "RustQt" {+ #[qobject]+ #[base = QAbstractListModel]+ #[qml_element]+ #[qml_singleton]+ #[qproperty(i32, count)]+ /// Bumped when players come or go, for bindings on [`first_for`].+ #[qproperty(i32, revision)]+ type MediaList = super::MediaListRust;+ }++ // Inherited from QAbstractListModel.+ unsafe extern "RustQt" {+ #[inherit]+ #[cxx_name = "beginInsertRows"]+ unsafe fn begin_insert_rows(self: Pin<&mut MediaList>, parent: &QModelIndex, first: i32, last: i32);+ #[inherit]+ #[cxx_name = "endInsertRows"]+ unsafe fn end_insert_rows(self: Pin<&mut MediaList>);+ #[inherit]+ #[cxx_name = "beginRemoveRows"]+ unsafe fn begin_remove_rows(self: Pin<&mut MediaList>, parent: &QModelIndex, first: i32, last: i32);+ #[inherit]+ #[cxx_name = "endRemoveRows"]+ unsafe fn end_remove_rows(self: Pin<&mut MediaList>);+ #[inherit]+ fn index(self: &MediaList, row: i32, column: i32, parent: &QModelIndex) -> QModelIndex;++ #[inherit]+ #[qsignal]+ #[cxx_name = "dataChanged"]+ fn data_changed(+ self: Pin<&mut MediaList>,+ top_left: &QModelIndex,+ bottom_right: &QModelIndex,+ roles: &QList_i32,+ );+ }++ unsafe extern "RustQt" {+ #[qinvokable]+ #[cxx_override]+ fn data(self: &MediaList, index: &QModelIndex, role: i32) -> QVariant;+ #[qinvokable]+ #[cxx_override]+ #[cxx_name = "roleNames"]+ fn role_names(self: &MediaList) -> QHash_i32_QByteArray;+ #[qinvokable]+ #[cxx_override]+ #[cxx_name = "rowCount"]+ fn row_count(self: &MediaList, parent: &QModelIndex) -> i32;++ /// Runs "play", "pause", "next", "previous" or "seek" (to+ /// `position_ms`) on a device's player.+ #[qinvokable]+ fn command(self: &MediaList, device: &QString, player: &QString, action: &QString, position_ms: f64);+ /// The row of a device's first player (the one to show), or -1.+ #[qinvokable]+ #[cxx_name = "firstFor"]+ fn first_for(self: &MediaList, device: &QString) -> i32;+ }++ impl cxx_qt::Threading for MediaList {}+ impl cxx_qt::Initialize for MediaList {}+}++#[derive(Default)]+pub struct MediaListRust {+ count: i32,+ revision: i32,+ rows: Vec<PlayerView>,+}++/// Model roles (Qt::UserRole + n).+const ROLES: &[&str] = &[+ "deviceId",+ "playerId",+ "app",+ "title",+ "artist",+ "album",+ "playing",+ "duration",+ "position",+ "positionAt",+ "art",+ "canPlay",+ "canPause",+ "canNext",+ "canPrevious",+ "canSeek",+];+const USER_ROLE: i32 = 0x0100;++fn role_value(row: &PlayerView, role: &str) -> QVariant {+ let text = |s: &str| QVariant::from(&QString::from(s));+ let p = &row.player;+ let can = |action: &str| QVariant::from(&p.actions.iter().any(|a| a == action));+ match role {+ "deviceId" => text(&row.device.to_string()),+ "playerId" => text(&p.id),+ "app" => text(&p.app),+ "title" => text(p.title.as_deref().unwrap_or_default()),+ "artist" => text(p.artist.as_deref().unwrap_or_default()),+ "album" => text(p.album.as_deref().unwrap_or_default()),+ "playing" => QVariant::from(&p.playing),+ // Milliseconds; -1 when the player doesn't say.+ "duration" => QVariant::from(&p.duration.map_or(-1.0, |d| d as f64)),+ "position" => QVariant::from(&p.position.map_or(-1.0, |d| d as f64)),+ // When `position` was current, as JavaScript time.+ "positionAt" => {+ QVariant::from(&row.at.duration_since(UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64))+ }+ "art" => text(+ &row.art+ .as_deref()+ .map(|path| {+ String::from(+ &QUrl::from_local_file(&QString::from(&*path.to_string_lossy())).to_qstring(),+ )+ })+ .unwrap_or_default(),+ ),+ "canPlay" => can("play"),+ "canPause" => can("pause"),+ "canNext" => can("next"),+ "canPrevious" => can("previous"),+ "canSeek" => can("seek"),+ _ => QVariant::default(),+ }+}++impl cxx_qt::Initialize for qobject::MediaList {+ fn initialize(self: Pin<&mut Self>) {+ super::subscribe(self.qt_thread(), Changes::MEDIA, Self::refresh);+ }+}++impl qobject::MediaList {+ fn refresh(mut self: Pin<&mut Self>) {+ let new: Vec<PlayerView> = core_host::host().hub.read(|s| s.media.clone());+ let root = QModelIndex::default();+ for edit in diff(&self.rows, &new, |r| (r.device, r.player.id.clone())) {+ match edit {+ Edit::Remove(row) => {+ let r = row as i32;+ // SAFETY: row exists; begin/end are balanced.+ unsafe {+ self.as_mut().begin_remove_rows(&root, r, r);+ self.as_mut().rust_mut().rows.remove(row);+ self.as_mut().end_remove_rows();+ }+ }+ Edit::Insert(row) => {+ let r = row as i32;+ // SAFETY: row <= len; begin/end are balanced.+ unsafe {+ self.as_mut().begin_insert_rows(&root, r, r);+ self.as_mut().rust_mut().rows.insert(row, new[row].clone());+ self.as_mut().end_insert_rows();+ }+ }+ Edit::Change(row) => {+ self.as_mut().rust_mut().rows[row] = new[row].clone();+ let index = self.index(row as i32, 0, &root);+ self.as_mut().data_changed(&index, &index, &QList::default());+ }+ }+ }+ let count = self.rows.len() as i32;+ self.as_mut().set_count(count);+ let revision = self.revision.wrapping_add(1);+ self.set_revision(revision);+ }++ pub fn first_for(&self, device: &QString) -> i32 {+ let device = String::from(device);+ self.rows.iter().position(|r| r.device.to_string() == device).map_or(-1, |row| row as i32)+ }++ pub fn data(&self, index: &QModelIndex, role: i32) -> QVariant {+ let row = usize::try_from(index.row()).ok();+ let role = usize::try_from(role - USER_ROLE).ok().and_then(|r| ROLES.get(r));+ match (row.and_then(|r| self.rows.get(r)), role) {+ (Some(row), Some(role)) => role_value(row, role),+ _ => QVariant::default(),+ }+ }++ pub fn role_names(&self) -> QHash<QHashPair_i32_QByteArray> {+ let mut roles = QHash::<QHashPair_i32_QByteArray>::default();+ for (i, name) in ROLES.iter().enumerate() {+ roles.insert(USER_ROLE + i as i32, QByteArray::from(*name));+ }+ roles+ }++ pub fn row_count(&self, _parent: &QModelIndex) -> i32 {+ self.rows.len() as i32+ }++ pub fn command(&self, device: &QString, player: &QString, action: &QString, position_ms: f64) {+ let Some(device) = super::parse_device(device) else { return };+ let Some(action) = MediaAction::parse(&String::from(action)) else { return };+ let position = (action == MediaAction::Seek).then(|| position_ms.max(0.0) as u64);+ media::command(device, String::from(player), action, position);+ }+}++#[cfg(test)]+mod tests {+ use std::path::PathBuf;++ use nectarlink_core::{DeviceId, MediaPlayer};++ use super::*;++ #[test]+ fn roles_describe_the_player() {+ let row = PlayerView {+ device: DeviceId([4; 32]),+ player: MediaPlayer {+ id: "com.music".into(),+ app: "Music".into(),+ title: Some("Song".into()),+ artist: None,+ album: None,+ playing: true,+ duration: Some(90_000),+ position: None,+ actions: vec!["pause".into(), "seek".into()],+ art_key: None,+ art: None,+ },+ art: Some(PathBuf::from(r"C:\data\cache\art\a.jpg")),+ at: UNIX_EPOCH + std::time::Duration::from_millis(1_500),+ };+ let get = |role| role_value(&row, role);+ assert_eq!(get("title").value::<QString>().map(|s| String::from(&s)), Some("Song".into()));+ assert_eq!(get("artist").value::<QString>().map(|s| String::from(&s)), Some(String::new()));+ assert_eq!(get("duration").value::<f64>(), Some(90_000.0));+ assert_eq!(get("position").value::<f64>(), Some(-1.0));+ assert_eq!(get("positionAt").value::<f64>(), Some(1_500.0));+ assert_eq!(get("canPause").value::<bool>(), Some(true));+ assert_eq!(get("canPlay").value::<bool>(), Some(false));+ assert_eq!(+ get("art").value::<QString>().map(|s| String::from(&s)),+ Some("file:///C:/data/cache/art/a.jpg".into())+ );+ }+}diff --git a/desktop/app/src/bridge/mod.rs b/desktop/app/src/bridge/mod.rsindex f167d52..66d9e24 100644--- a/desktop/app/src/bridge/mod.rs+++ b/desktop/app/src/bridge/mod.rs@@ -10,6 +10,8 @@ pub mod app;#[allow(unsafe_code)]pub mod devices;#[allow(unsafe_code)]+pub mod media;+#[allow(unsafe_code)]pub mod native;#[allow(unsafe_code)]pub mod notifications;diff --git a/desktop/app/src/core_host.rs b/desktop/app/src/core_host.rsindex 4e4c7f7..736533a 100644--- a/desktop/app/src/core_host.rs+++ b/desktop/app/src/core_host.rs@@ -94,6 +94,8 @@ async fn run(data_dir: PathBuf, platform: Arc<dyn Platform>) {let host = host();let mut config = NodeConfig::new(&data_dir, this_device(), env!("CARGO_PKG_VERSION"));config.downloads_dir = Some(downloads_dir());+ // This PC's own players, for phones (crate::win::media_sessions).+ config.capabilities = vec!["media.control".into()];let node = match Node::start(config, platform).await {Ok(node) => node,Err(e) => {@@ -136,6 +138,7 @@ async fn run(data_dir: PathBuf, platform: Arc<dyn Platform>) {match events.recv().await {Ok(event) => {crate::notifications::apply(&event);+ crate::media::on_event(&event);crate::notifications::update_toasts(&event);crate::clipboard::on_event(&event);crate::transfers::on_event(&event);diff --git a/desktop/app/src/main.rs b/desktop/app/src/main.rsindex ba07113..37f2fb5 100644--- a/desktop/app/src/main.rs+++ b/desktop/app/src/main.rs@@ -15,6 +15,7 @@ mod icons;mod launch;mod logging;mod mark;+mod media;mod notifications;mod palette;mod qr;@@ -35,7 +36,7 @@ use std::{};use cxx_qt_lib::{QGuiApplication, QQmlApplicationEngine, QString, QUrl};-use nectarlink_core::Platform;+use nectarlink_core::{MediaAction, MediaError, Platform};use crate::{launch::Request,@@ -62,6 +63,14 @@ impl Platform for DesktopPlatform {fn set_clipboard_image(&self, mime: &str, bytes: &[u8]) -> Result<(), String> {win::clipboard::write_image(mime, bytes)}+ fn media_command(+ &self,+ player: &str,+ action: MediaAction,+ position: Option<u64>,+ ) -> Result<(), MediaError> {+ win::media_sessions::command(player, action, position)+ }}/// Command-line options (also used by shortcuts and autostart).@@ -202,6 +211,8 @@ fn main() -> ExitCode {watch_network();start_toasts();clipboard::start();+ win::smtc::start(media::on_flyout);+ win::media_sessions::start(media::local_changed);send_to::set_enabled(settings::Settings::load(&data_dir).send_to_menu);// A test instance (own data folder) leaves the user's menu alone.if options.data_dir.is_none() {diff --git a/desktop/app/src/media.rs b/desktop/app/src/media.rsnew file mode 100644index 0000000..08d9292--- /dev/null+++ b/desktop/app/src/media.rs@@ -0,0 +1,156 @@+// SPDX-License-Identifier: GPL-3.0-or-later+//! Media between this PC and paired phones (docs/protocol/media.md): what+//! plays on a phone shows in the app and in Windows' media flyout, where+//! it can be controlled; what plays on this PC goes to the phones (see+//! `win::media_sessions`).++use std::{+ collections::HashMap,+ path::PathBuf,+ sync::{Mutex, MutexGuard},+};++use nectarlink_core::{DeviceId, Error, MediaAction, NodeEvent};++use crate::{+ bridge::app::{describe, show_message},+ core_host,+ state::PlayerView,+ win::smtc,+};++fn lock<T>(m: &Mutex<T>) -> MutexGuard<'_, T> {+ m.lock().unwrap_or_else(|e| e.into_inner())+}++/// Artwork files by (device, art key), for players that only send a key.+static ART: Mutex<Option<HashMap<(DeviceId, String), PathBuf>>> = Mutex::new(None);+/// Artwork files kept per device (older ones are deleted).+const KEPT_ART: usize = 12;++fn art_dir() -> PathBuf {+ core_host::host().data_dir.join("cache").join("art")+}++/// Saves the artwork in a media event; returns every player's artwork file+/// by player ID.+pub fn save_art(event: &NodeEvent) -> HashMap<String, PathBuf> {+ let NodeEvent::MediaChanged { device, players } = event else { return HashMap::new() };+ let mut art = lock(&ART);+ let art = art.get_or_insert_default();+ let mut found = HashMap::new();+ for player in players {+ let Some(key) = &player.art_key else { continue };+ let slot = (*device, key.clone());+ if let Some(bytes) = &player.art {+ let ext = if bytes.starts_with(b"\x89PNG") { "png" } else { "jpg" };+ // Keys are the sender's; the file name is ours.+ let name = format!("{}-{:016x}.{ext}", device.short(), fnv(key));+ let path = art_dir().join(name);+ let written = std::fs::create_dir_all(art_dir()).and_then(|()| std::fs::write(&path, bytes));+ match written {+ Ok(()) => {+ art.insert(slot.clone(), path);+ }+ Err(e) => tracing::warn!(error = %e, "can't save artwork"),+ }+ }+ if let Some(path) = art.get(&slot) {+ found.insert(player.id.clone(), path.clone());+ }+ }+ prune(art, *device, players.iter().filter_map(|p| p.art_key.clone()).collect());+ found+}++/// Deletes a device's artwork beyond the newest few, never what's in use.+fn prune(art: &mut HashMap<(DeviceId, String), PathBuf>, device: DeviceId, in_use: Vec<String>) {+ let mut theirs: Vec<(String, PathBuf)> =+ art.iter().filter(|((d, _), _)| *d == device).map(|((_, k), p)| (k.clone(), p.clone())).collect();+ if theirs.len() <= KEPT_ART {+ return;+ }+ theirs.sort_by_key(|(_, path)| std::fs::metadata(path).and_then(|m| m.modified()).ok());+ let excess = theirs.len() - KEPT_ART;+ for (key, path) in theirs.into_iter().filter(|(k, _)| !in_use.contains(k)).take(excess) {+ let _ = std::fs::remove_file(&path);+ art.remove(&(device, key));+ }+}++fn fnv(text: &str) -> u64 {+ text.bytes().fold(0xcbf2_9ce4_8422_2325_u64, |h, b| (h ^ u64::from(b)).wrapping_mul(0x0100_0000_01b3))+}++/// Keeps Windows' media flyout showing what plays on the phones.+pub fn update_flyout() {+ let current = core_host::host().hub.read(|s| {+ // A player that's playing wins; then the first one shown.+ let pick = s.media.iter().find(|p| p.player.playing).or_else(|| s.media.first())?;+ let device = s.name_of(&pick.device).unwrap_or_default();+ Some(smtc::Shown { device: pick.device, device_name: device, view: pick.clone() })+ });+ smtc::show(current);+}++/// What the flyout or the keyboard's media keys asked for.+pub fn on_flyout(device: DeviceId, player: String, action: MediaAction, position: Option<u64>) {+ send(device, player, action, position, false);+}++/// Runs a command on a phone's player; says so if it didn't work.+pub fn command(device: DeviceId, player: String, action: MediaAction, position: Option<u64>) {+ send(device, player, action, position, true);+}++fn send(device: DeviceId, player: String, action: MediaAction, position: Option<u64>, tell: bool) {+ let Some(node) = core_host::node() else { return };+ core_host::spawn(async move {+ let result = node.media_command(device, player, action, position).await;+ let message = match result {+ Ok(()) => return,+ Err(Error::NotFound) => "That player isn't playing anymore.".to_owned(),+ Err(Error::Unsupported) => "That player can't do that.".to_owned(),+ Err(Error::Denied) => "Media is turned off for that device.".to_owned(),+ Err(e) => describe(&e),+ };+ tracing::debug!(message, "media command failed");+ if tell {+ show_message(message);+ }+ });+}++/// The hub's view of a player, with when its position was measured.+pub fn view(device: DeviceId, mut player: nectarlink_core::MediaPlayer, art: Option<PathBuf>) -> PlayerView {+ player.art = None;+ PlayerView { device, player, art, at: std::time::SystemTime::now() }+}++/// Folds a media event into the hub (artwork saved first) and updates the+/// flyout.+pub fn on_event(event: &NodeEvent) {+ let changed = match event {+ NodeEvent::MediaChanged { device, players } => {+ let art = save_art(event);+ let views = players.iter().map(|p| view(*device, p.clone(), art.get(&p.id).cloned())).collect();+ core_host::host().hub.update(|s| s.set_media(*device, views));+ true+ }+ // Offline or unpaired devices' players are cleared by the hub.+ NodeEvent::LinkChanged { .. } | NodeEvent::DeviceRemoved(_) => true,+ _ => false,+ };+ if changed {+ update_flyout();+ }+}++/// What plays on this PC changed: tell the phones.+pub fn local_changed(players: Vec<nectarlink_core::MediaPlayer>) {+ core_host::spawn(async move {+ if let Some(node) = core_host::wait_for_node().await {+ node.media_changed(players).await;+ }+ });+}diff --git a/desktop/app/src/state.rs b/desktop/app/src/state.rsindex 506f4e3..8ca8e60 100644--- a/desktop/app/src/state.rs+++ b/desktop/app/src/state.rs@@ -11,13 +11,13 @@ use std::{};use nectarlink_core::{- Battery, CapabilityMatrix, DeviceId, DeviceInfo, DiscoveredDevice, LinkState, NodeEvent, Notification,- PairedDevice, PairingEvent, PairingFailure, PowerLevel, Transfer, TransferState,+ Battery, CapabilityMatrix, DeviceId, DeviceInfo, DiscoveredDevice, LinkState, MediaPlayer, NodeEvent,+ Notification, PairedDevice, PairingEvent, PairingFailure, PowerLevel, Transfer, TransferState,};/// Which parts of the state changed, so listeners refresh only what they show.#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]-pub struct Changes(u8);+pub struct Changes(u16);impl Changes {pub const NONE: Changes = Changes(0);@@ -29,6 +29,7 @@ impl Changes {pub const RINGING: Changes = Changes(1 << 5);pub const NOTIFICATIONS: Changes = Changes(1 << 6);pub const TRANSFERS: Changes = Changes(1 << 7);+ pub const MEDIA: Changes = Changes(1 << 8);pub fn is_empty(self) -> bool {self.0 == 0@@ -84,6 +85,17 @@ impl From<PairedDevice> for DeviceView {}}+/// A player on a paired device (docs/protocol/media.md).+#[derive(Debug, Clone, PartialEq, Eq)]+pub struct PlayerView {+ pub device: DeviceId,+ /// Without its artwork, which is in `art`.+ pub player: MediaPlayer,+ pub art: Option<PathBuf>,+ /// When `player.position` was current.+ pub at: SystemTime,+}+/// The pairing screen's state.#[derive(Debug, Clone, PartialEq, Eq, Default)]pub enum PairingView {@@ -169,6 +181,8 @@ pub struct AppState {pub replies: HashMap<(DeviceId, String), Vec<SentReply>>,/// File transfers, newest first.pub transfers: Vec<TransferView>,+ /// Media players on paired devices, each device's in its order.+ pub media: Vec<PlayerView>,}impl AppState {@@ -242,9 +256,16 @@ impl AppState {if self.notifications.len() != notifications {changes |= Changes::NOTIFICATIONS;}+ changes | self.set_media(*id, Vec::new())+ }+ NodeEvent::LinkChanged { device, link } => {+ let mut changes = self.update_device(device, |d| d.link = link.clone());+ // A device that comes back sends what plays then.+ if matches!(link, LinkState::Offline { .. }) {+ changes |= self.set_media(*device, Vec::new());+ }changes}- NodeEvent::LinkChanged { device, link } => self.update_device(device, |d| d.link = link.clone()),NodeEvent::PeerInfoChanged { device, info } => {self.update_device(device, |d| d.info = info.clone())}@@ -305,7 +326,22 @@ impl AppState {// Feedback only (see crate::clipboard); nothing to keep.NodeEvent::ClipboardReceived { .. } => Changes::NONE,NodeEvent::Transfer(transfer) => self.update_transfer(transfer.clone(), Instant::now()),+ // With artwork saved first (see crate::media).+ NodeEvent::MediaChanged { .. } => Changes::NONE,+ }+ }++ /// Replaces a device's players.+ pub fn set_media(&mut self, device: DeviceId, players: Vec<PlayerView>) -> Changes {+ let at = self.media.iter().position(|p| p.device == device).unwrap_or(self.media.len());+ let before: Vec<PlayerView> = self.media.iter().filter(|p| p.device == device).cloned().collect();+ if before.is_empty() && players.is_empty() {+ return Changes::NONE;}+ self.media.retain(|p| p.device != device);+ let at = at.min(self.media.len());+ self.media.splice(at..at, players);+ Changes::MEDIA}/// Adds or updates a transfer, keeping a smoothed speed.diff --git a/desktop/app/src/win/image.rs b/desktop/app/src/win/image.rsindex 86b9331..6c8f676 100644--- a/desktop/app/src/win/image.rs+++ b/desktop/app/src/win/image.rs@@ -108,6 +108,43 @@ pub fn encode_png(bitmap: &Bitmap) -> Result<Vec<u8>> {})}+/// Scaled down (never up) so neither side is longer than `max`, averaging+/// the pixels each new one covers.+pub fn scale_to(bitmap: &Bitmap, max: u32) -> Bitmap {+ let longest = bitmap.width.max(bitmap.height);+ if longest <= max || longest == 0 {+ return bitmap.clone();+ }+ let scale = f64::from(max) / f64::from(longest);+ let width = ((f64::from(bitmap.width) * scale).round() as u32).max(1);+ let height = ((f64::from(bitmap.height) * scale).round() as u32).max(1);+ let mut bgra = Vec::with_capacity((width * height * 4) as usize);+ for y in 0..height {+ let (y0, y1) = (+ y * bitmap.height / height,+ ((y + 1) * bitmap.height / height).max(y * bitmap.height / height + 1),+ );+ for x in 0..width {+ let (x0, x1) = (+ x * bitmap.width / width,+ ((x + 1) * bitmap.width / width).max(x * bitmap.width / width + 1),+ );+ let mut sum = [0u32; 4];+ for sy in y0..y1 {+ for sx in x0..x1 {+ let i = ((sy * bitmap.width + sx) * 4) as usize;+ for (total, &value) in sum.iter_mut().zip(&bitmap.bgra[i..i + 4]) {+ *total += u32::from(value);+ }+ }+ }+ let n = (y1 - y0) * (x1 - x0);+ bgra.extend(sum.map(|s| (s / n) as u8));+ }+ }+ Bitmap { width, height, bgra }+}+/// A device-independent bitmap (`CF_DIB` / `CF_DIBV5`) as a BMP file, which/// WIC can decode: the DIB with a file header in front.pub fn bmp_from_dib(dib: &[u8]) -> Option<Vec<u8>> {@@ -195,6 +232,19 @@ mod tests {assert_eq!(decode(&png).unwrap(), bitmap);}+ #[test]+ fn large_pictures_scale_down_evenly() {+ let big = Bitmap { width: 400, height: 200, bgra: [10, 20, 30, 255].repeat(400 * 200) };+ let small = scale_to(&big, 100);+ assert_eq!((small.width, small.height), (100, 50));+ assert_eq!(&small.bgra[..4], &[10, 20, 30, 255]);+ assert_eq!(scale_to(&small, 100), small, "never scaled up");+ // A checkerboard averages to grey.+ let checker = checker(4, 4);+ let half = scale_to(&checker, 2);+ assert_eq!(half.bgra.len(), 2 * 2 * 4);+ }+#[test]fn copied_bitmaps_decode() {let bitmap = checker(7, 4);diff --git a/desktop/app/src/win/media_sessions.rs b/desktop/app/src/win/media_sessions.rsnew file mode 100644index 0000000..5a34306--- /dev/null+++ b/desktop/app/src/win/media_sessions.rs@@ -0,0 +1,344 @@+// SPDX-License-Identifier: GPL-3.0-or-later+//! What plays on this PC, for paired phones (docs/protocol/media.md): the+//! sessions apps report to Windows (Global System Media Transport+//! Controls), and commands from a phone sent to them.+//!+//! Nectarlink's own session (a phone's media in the flyout) is left out, so+//! a phone never gets its own music back.++use std::{+ collections::HashMap,+ sync::{OnceLock, mpsc},+ time::{Duration, SystemTime, UNIX_EPOCH},+};++use nectarlink_core::{MediaAction, MediaError, MediaPlayer};+use windows::{+ ApplicationModel::AppInfo,+ Foundation::TypedEventHandler,+ Media::Control::{+ GlobalSystemMediaTransportControlsSession, GlobalSystemMediaTransportControlsSessionManager,+ GlobalSystemMediaTransportControlsSessionPlaybackStatus,+ },+ Storage::Streams::DataReader,+ Win32::System::Com::{COINIT_MULTITHREADED, CoInitializeEx},+ core::HSTRING,+};++use super::toast::AUMID;++/// The largest artwork sent as is; larger pictures are scaled down.+const ART_BYTES: u64 = 256 * 1024;+/// Changes come in bursts (title, then artwork, then position).+const SETTLE: Duration = Duration::from_millis(300);+/// Windows' clock starts in 1601; Unix time in 1970.+const WINDOWS_TO_UNIX_SECS: i64 = 11_644_473_600;++type OnChange = Box<dyn Fn(Vec<MediaPlayer>) + Send + Sync>;++static MANAGER: OnceLock<GlobalSystemMediaTransportControlsSessionManager> = OnceLock::new();++/// Watches this PC's media; `on_change` gets every player after each+/// change (on the watcher's thread).+pub fn start(on_change: impl Fn(Vec<MediaPlayer>) + Send + Sync + 'static) {+ let on_change: OnChange = Box::new(on_change);+ let spawned = std::thread::Builder::new().name("media-sessions".into()).spawn(move || {+ if let Err(e) = run(&on_change) {+ tracing::warn!(error = %e, "can't watch this PC's media");+ }+ });+ if let Err(e) = spawned {+ tracing::warn!(error = %e, "can't watch this PC's media");+ }+}++fn run(on_change: &OnChange) -> windows::core::Result<()> {+ // SAFETY: COM for this thread, which lives as long as the app.+ unsafe { CoInitializeEx(None, COINIT_MULTITHREADED) }.ok()?;+ let manager = GlobalSystemMediaTransportControlsSessionManager::RequestAsync()?.join()?;+ let (changed, changes) = mpsc::channel::<()>();+ let signal = changed.clone();+ manager.SessionsChanged(&TypedEventHandler::new(move |_, _| {+ let _ = signal.send(());+ Ok(())+ }))?;+ let _ = MANAGER.set(manager.clone());++ let mut watched: Vec<String> = Vec::new();+ let mut art = ArtCache::default();+ let mut last: Option<Vec<MediaPlayer>> = None;+ loop {+ // Hook up sessions that are new since last time.+ let mut present = Vec::new();+ for session in manager.GetSessions()? {+ let id = session.SourceAppUserModelId()?.to_string();+ if !watched.contains(&id) {+ watch(&session, &changed)?;+ watched.push(id.clone());+ }+ present.push(id);+ }+ // A session that ends and comes back is hooked up again.+ watched.retain(|id| present.contains(id));+ let players = snapshot(&manager, &mut art);+ if last.as_ref() != Some(&players) {+ on_change(players.clone());+ last = Some(players);+ }+ // Wait for a change, then let the burst settle.+ if changes.recv().is_err() {+ return Ok(());+ }+ while changes.recv_timeout(SETTLE).is_ok() {}+ }+}++fn watch(+ session: &GlobalSystemMediaTransportControlsSession,+ changed: &mpsc::Sender<()>,+) -> windows::core::Result<()> {+ session.MediaPropertiesChanged(&TypedEventHandler::new({+ let changed = changed.clone();+ move |_, _| {+ let _ = changed.send(());+ Ok(())+ }+ }))?;+ session.PlaybackInfoChanged(&TypedEventHandler::new({+ let changed = changed.clone();+ move |_, _| {+ let _ = changed.send(());+ Ok(())+ }+ }))?;+ session.TimelinePropertiesChanged(&TypedEventHandler::new({+ let changed = changed.clone();+ move |_, _| {+ let _ = changed.send(());+ Ok(())+ }+ }))?;+ Ok(())+}++/// Every session but Nectarlink's own, the one Windows considers current+/// first.+fn snapshot(+ manager: &GlobalSystemMediaTransportControlsSessionManager,+ art: &mut ArtCache,+) -> Vec<MediaPlayer> {+ let current =+ manager.GetCurrentSession().ok().and_then(|s| s.SourceAppUserModelId().ok()).map(|id| id.to_string());+ let Ok(sessions) = manager.GetSessions() else { return Vec::new() };+ let mut players: Vec<MediaPlayer> = sessions+ .into_iter()+ .filter_map(|session| match player(&session, art) {+ Ok(player) => player,+ Err(e) => {+ tracing::debug!(error = %e, "can't read a media session");+ None+ }+ })+ .collect();+ players.sort_by_key(|p| (Some(&p.id) != current.as_ref(), !p.playing));+ players+}++fn player(+ session: &GlobalSystemMediaTransportControlsSession,+ art: &mut ArtCache,+) -> windows::core::Result<Option<MediaPlayer>> {+ let id = session.SourceAppUserModelId()?.to_string();+ if id == AUMID {+ return Ok(None);+ }+ let properties = session.TryGetMediaPropertiesAsync()?.join()?;+ let text =+ |s: windows::core::Result<HSTRING>| s.ok().map(|s| s.to_string()).filter(|s| !s.trim().is_empty());+ let title = text(properties.Title());+ let artist = text(properties.Artist());+ let album = text(properties.AlbumTitle());+ if title.is_none() {+ return Ok(None);+ }+ let info = session.GetPlaybackInfo()?;+ let playing = info.PlaybackStatus()? == GlobalSystemMediaTransportControlsSessionPlaybackStatus::Playing;+ let controls = info.Controls()?;+ let mut actions = Vec::new();+ for (on, action) in [+ (controls.IsPlayEnabled()?, "play"),+ (controls.IsPauseEnabled()?, "pause"),+ (controls.IsNextEnabled()?, "next"),+ (controls.IsPreviousEnabled()?, "previous"),+ (controls.IsPlaybackPositionEnabled()?, "seek"),+ ] {+ if on {+ actions.push(action.to_owned());+ }+ }++ let timeline = session.GetTimelineProperties()?;+ let ms = |ticks: i64| u64::try_from(ticks / 10_000).unwrap_or(0);+ let duration = ms(timeline.EndTime()?.Duration - timeline.StartTime()?.Duration);+ let (duration, position) = if duration > 0 {+ let mut position = ms(timeline.Position()?.Duration);+ // The position is as of the last update; it has moved on since.+ if playing {+ let updated = timeline.LastUpdatedTime()?.UniversalTime;+ let updated_ms = updated / 10_000 - WINDOWS_TO_UNIX_SECS * 1000;+ let now_ms = SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_millis() as i64);+ position = position.saturating_add(u64::try_from(now_ms - updated_ms).unwrap_or(0));+ }+ (Some(duration), Some(position.min(duration)))+ } else {+ (None, None)+ };++ let (art_key, art_bytes) = art.get(&id, &title, &artist, &album, || thumbnail(&properties)).unzip();+ Ok(Some(MediaPlayer {+ app: app_name(&id),+ id,+ title,+ artist,+ album,+ playing,+ duration,+ position,+ actions,+ art_key,+ art: art_bytes,+ }))+}++/// Artwork per track, read once: reading it takes a moment.+#[derive(Default)]+struct ArtCache {+ by_track: HashMap<String, Option<(String, Vec<u8>)>>,+}++impl ArtCache {+ fn get(+ &mut self,+ id: &str,+ title: &Option<String>,+ artist: &Option<String>,+ album: &Option<String>,+ read: impl FnOnce() -> Option<Vec<u8>>,+ ) -> Option<(String, Vec<u8>)> {+ let track = format!("{id}\u{1}{title:?}\u{1}{artist:?}\u{1}{album:?}");+ if !self.by_track.contains_key(&track) {+ if self.by_track.len() > 32 {+ self.by_track.clear();+ }+ let art = read().map(|bytes| (format!("{:016x}", fnv(&bytes)), bytes));+ self.by_track.insert(track.clone(), art);+ }+ self.by_track.get(&track).cloned().flatten()+ }+}++fn fnv(bytes: &[u8]) -> u64 {+ bytes.iter().fold(0xcbf2_9ce4_8422_2325_u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x0100_0000_01b3))+}++fn thumbnail(+ properties: &windows::Media::Control::GlobalSystemMediaTransportControlsSessionMediaProperties,+) -> Option<Vec<u8>> {+ let read = || -> windows::core::Result<Vec<u8>> {+ let stream = properties.Thumbnail()?.OpenReadAsync()?.join()?;+ let size = stream.Size()?;+ let reader = DataReader::CreateDataReader(&stream)?;+ let loaded = reader.LoadAsync(u32::try_from(size).unwrap_or(u32::MAX))?.join()?;+ let mut bytes = vec![0u8; loaded as usize];+ reader.ReadBytes(&mut bytes)?;+ Ok(bytes)+ };+ let bytes = read().ok().filter(|b| !b.is_empty())?;+ if bytes.len() as u64 <= ART_BYTES {+ return Some(bytes);+ }+ // Too large to send: a smaller PNG.+ let bitmap = super::image::decode(&bytes).ok()?;+ let small = super::image::encode_png(&super::image::scale_to(&bitmap, 360)).ok()?;+ (small.len() as u64 <= ART_BYTES).then_some(small)+}++/// The app's name as Windows shows it, or a readable form of its ID.+fn app_name(id: &str) -> String {+ let display =+ AppInfo::GetFromAppUserModelId(&HSTRING::from(id)).and_then(|info| info.DisplayInfo()?.DisplayName());+ if let Ok(name) = display {+ let name = name.to_string();+ if !name.trim().is_empty() {+ return name;+ }+ }+ readable_name(id)+}++/// "Spotify.exe" → "Spotify", `C:\…\vlc.exe` → "vlc", "Microsoft.ZuneMusic_8wekyb3d8bbwe!App" → "Microsoft.ZuneMusic".+fn readable_name(id: &str) -> String {+ let id = id.rsplit(['\\', '/']).next().unwrap_or(id);+ let id = id.split('!').next().unwrap_or(id);+ let id = id.split('_').next().unwrap_or(id);+ let id = id.strip_suffix(".exe").or_else(|| id.strip_suffix(".EXE")).unwrap_or(id);+ if id.is_empty() { "Media".into() } else { id.to_owned() }+}++/// Runs a phone's command on one of this PC's sessions.+pub fn command(player: &str, action: MediaAction, position: Option<u64>) -> Result<(), MediaError> {+ let manager = MANAGER.get().ok_or(MediaError::Unsupported)?;+ let failed = |e: windows::core::Error| MediaError::Failed(e.to_string());+ let session = manager+ .GetSessions()+ .map_err(failed)?+ .into_iter()+ .find(|s| s.SourceAppUserModelId().is_ok_and(|id| id == player))+ .ok_or(MediaError::NotFound)?;+ let done = match action {+ MediaAction::Play => session.TryPlayAsync(),+ MediaAction::Pause => session.TryPauseAsync(),+ MediaAction::Next => session.TrySkipNextAsync(),+ MediaAction::Previous => session.TrySkipPreviousAsync(),+ MediaAction::Seek => {+ let ticks = i64::try_from(position.unwrap_or(0)).unwrap_or(0).saturating_mul(10_000);+ session.TryChangePlaybackPositionAsync(ticks)+ }+ }+ .and_then(|op| op.join())+ .map_err(failed)?;+ if done { Ok(()) } else { Err(MediaError::Unsupported) }+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn app_ids_read_as_names() {+ assert_eq!(readable_name("Spotify.exe"), "Spotify");+ assert_eq!(readable_name(r"C:\Program Files\VideoLAN\vlc.exe"), "vlc");+ assert_eq!(+ readable_name("Microsoft.ZuneMusic_8wekyb3d8bbwe!Microsoft.ZuneMusic"),+ "Microsoft.ZuneMusic"+ );+ assert_eq!(readable_name("Chrome"), "Chrome");+ assert_eq!(readable_name(""), "Media");+ }++ #[test]+ fn artwork_is_read_once_per_track() {+ let mut cache = ArtCache::default();+ let mut reads = 0;+ let title = Some("Song".to_owned());+ for _ in 0..3 {+ let art = cache.get("app", &title, &None, &None, || {+ reads += 1;+ Some(vec![1, 2, 3])+ });+ assert_eq!(art.map(|(_, b)| b), Some(vec![1, 2, 3]));+ }+ assert_eq!(reads, 1);+ assert!(cache.get("app", &Some("Other".into()), &None, &None, || None).is_none());+ }+}diff --git a/desktop/app/src/win/mod.rs b/desktop/app/src/win/mod.rsindex e4b5430..7f437fb 100644--- a/desktop/app/src/win/mod.rs+++ b/desktop/app/src/win/mod.rs@@ -8,9 +8,11 @@pub mod clipboard;pub mod icon;pub mod image;+pub mod media_sessions;pub mod net;pub mod shortcut;pub mod single_instance;+pub mod smtc;pub mod sound;pub mod toast;pub mod tray;diff --git a/desktop/app/src/win/smtc.rs b/desktop/app/src/win/smtc.rsnew file mode 100644index 0000000..125ca57--- /dev/null+++ b/desktop/app/src/win/smtc.rs@@ -0,0 +1,288 @@+// SPDX-License-Identifier: GPL-3.0-or-later+//! What plays on a phone, in Windows' own media controls: the flyout by the+//! volume and Quick Settings, the lock screen, and the keyboard's media+//! keys (System Media Transport Controls).+//!+//! A desktop app gets these controls for one of its windows, so a hidden+//! window on its own thread owns them. Updates reach that thread through a+//! queue and a window message; button presses come back on Windows' threads.++use std::{+ path::PathBuf,+ sync::{+ Mutex, MutexGuard, OnceLock,+ atomic::{AtomicIsize, Ordering},+ mpsc,+ },+ time::SystemTime,+};++use nectarlink_core::{DeviceId, MediaAction};+use windows::{+ Foundation::{TimeSpan, TypedEventHandler},+ Media::{+ MediaPlaybackStatus, MediaPlaybackType, PlaybackPositionChangeRequestedEventArgs,+ SystemMediaTransportControls, SystemMediaTransportControlsButton,+ SystemMediaTransportControlsButtonPressedEventArgs, SystemMediaTransportControlsTimelineProperties,+ },+ Storage::Streams::{DataWriter, InMemoryRandomAccessStream, RandomAccessStreamReference},+ Win32::{+ Foundation::{HWND, LPARAM, LRESULT, WPARAM},+ System::{+ Com::{COINIT_MULTITHREADED, CoInitializeEx},+ LibraryLoader::GetModuleHandleW,+ WinRT::ISystemMediaTransportControlsInterop,+ },+ UI::WindowsAndMessaging::{+ CreateWindowExW, DefWindowProcW, DispatchMessageW, GetMessageW, MSG, PostMessageW,+ RegisterClassExW, WINDOW_EX_STYLE, WM_APP, WNDCLASSEXW, WS_OVERLAPPED,+ },+ },+ core::{HSTRING, w},+};++use crate::state::PlayerView;++/// What the controls show: one player on one device.+#[derive(Debug, Clone)]+pub struct Shown {+ pub device: DeviceId,+ pub device_name: String,+ pub view: PlayerView,+}++type OnCommand = Box<dyn Fn(DeviceId, String, MediaAction, Option<u64>) + Send + Sync>;++static WINDOW: AtomicIsize = AtomicIsize::new(0);+static QUEUE: OnceLock<Mutex<mpsc::Sender<Option<Shown>>>> = OnceLock::new();+static ON_COMMAND: OnceLock<OnCommand> = OnceLock::new();+/// The player the controls stand for, for button presses.+static CURRENT: Mutex<Option<(DeviceId, String)>> = Mutex::new(None);++const WM_UPDATE: u32 = WM_APP + 1;++fn lock<T>(m: &Mutex<T>) -> MutexGuard<'_, T> {+ m.lock().unwrap_or_else(|e| e.into_inner())+}++/// Starts the controls (hidden until something plays); `on_command` gets+/// what the user pressed, on a Windows thread.+pub fn start(on_command: impl Fn(DeviceId, String, MediaAction, Option<u64>) + Send + Sync + 'static) {+ if ON_COMMAND.set(Box::new(on_command)).is_err() {+ return;+ }+ let (sender, updates) = mpsc::channel();+ let _ = QUEUE.set(Mutex::new(sender));+ let spawned = std::thread::Builder::new().name("media-controls".into()).spawn(move || {+ if let Err(e) = run(updates) {+ tracing::warn!(error = %e, "Windows media controls are unavailable");+ }+ });+ if let Err(e) = spawned {+ tracing::warn!(error = %e, "Windows media controls are unavailable");+ }+}++/// Shows a player in the controls, or hides them.+pub fn show(current: Option<Shown>) {+ let Some(queue) = QUEUE.get() else { return };+ if lock(queue).send(current).is_err() {+ return;+ }+ let window = WINDOW.load(Ordering::Acquire);+ if window != 0 {+ // SAFETY: posting to our own window; it drains the queue.+ unsafe {+ let _ = PostMessageW(Some(HWND(window as *mut _)), WM_UPDATE, WPARAM(0), LPARAM(0));+ }+ }+}++struct Controls {+ smtc: SystemMediaTransportControls,+ /// The artwork shown, so it's only loaded when it changes.+ art: Option<PathBuf>,+}++static CONTROLS: OnceLock<Mutex<Option<Controls>>> = OnceLock::new();+static UPDATES: OnceLock<Mutex<mpsc::Receiver<Option<Shown>>>> = OnceLock::new();++fn run(updates: mpsc::Receiver<Option<Shown>>) -> windows::core::Result<()> {+ // SAFETY: COM for this thread, which lives as long as the app.+ unsafe { CoInitializeEx(None, COINIT_MULTITHREADED) }.ok()?;+ // SAFETY: class registration and a window that's never shown; the+ // window procedure only touches statics.+ let hwnd = unsafe {+ let instance = GetModuleHandleW(None)?;+ let class = WNDCLASSEXW {+ cbSize: std::mem::size_of::<WNDCLASSEXW>() as u32,+ lpfnWndProc: Some(window_proc),+ hInstance: instance.into(),+ lpszClassName: w!("Nectarlink.Media"),+ ..Default::default()+ };+ RegisterClassExW(&class);+ CreateWindowExW(+ WINDOW_EX_STYLE(0),+ w!("Nectarlink.Media"),+ w!("Nectarlink media"),+ WS_OVERLAPPED,+ 0,+ 0,+ 0,+ 0,+ None,+ None,+ Some(instance.into()),+ None,+ )?+ };+ let interop =+ windows::core::factory::<SystemMediaTransportControls, ISystemMediaTransportControlsInterop>()?;+ // SAFETY: the window is ours and alive for the app's lifetime.+ let smtc: SystemMediaTransportControls = unsafe { interop.GetForWindow(hwnd)? };+ smtc.SetIsEnabled(false)?;+ smtc.ButtonPressed(&TypedEventHandler::<+ SystemMediaTransportControls,+ SystemMediaTransportControlsButtonPressedEventArgs,+ >::new(|_, args| {+ let Some(args) = args.as_ref() else { return Ok(()) };+ let action = match args.Button()? {+ SystemMediaTransportControlsButton::Play => MediaAction::Play,+ SystemMediaTransportControlsButton::Pause => MediaAction::Pause,+ SystemMediaTransportControlsButton::Next => MediaAction::Next,+ SystemMediaTransportControlsButton::Previous => MediaAction::Previous,+ _ => return Ok(()),+ };+ command(action, None);+ Ok(())+ }))?;+ smtc.PlaybackPositionChangeRequested(&TypedEventHandler::<+ SystemMediaTransportControls,+ PlaybackPositionChangeRequestedEventArgs,+ >::new(|_, args| {+ if let Some(args) = args.as_ref() {+ let ticks = args.RequestedPlaybackPosition()?.Duration.max(0);+ command(MediaAction::Seek, Some(u64::try_from(ticks / 10_000).unwrap_or(0)));+ }+ Ok(())+ }))?;+ let _ = CONTROLS.set(Mutex::new(Some(Controls { smtc, art: None })));+ let _ = UPDATES.set(Mutex::new(updates));+ WINDOW.store(hwnd.0 as isize, Ordering::Release);+ // Anything queued before the window existed.+ apply_updates();++ let mut msg = MSG::default();+ // SAFETY: a plain message loop for this thread's window.+ unsafe {+ while GetMessageW(&mut msg, None, 0, 0).as_bool() {+ DispatchMessageW(&msg);+ }+ }+ Ok(())+}++fn command(action: MediaAction, position: Option<u64>) {+ let Some((device, player)) = lock(&CURRENT).clone() else { return };+ if let Some(on_command) = ON_COMMAND.get() {+ on_command(device, player, action, position);+ }+}++unsafe extern "system" fn window_proc(hwnd: HWND, msg: u32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {+ if msg == WM_UPDATE {+ apply_updates();+ return LRESULT(0);+ }+ // SAFETY: default handling for everything else.+ unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) }+}++/// Applies the newest queued update (older ones are already out of date).+fn apply_updates() {+ let Some(updates) = UPDATES.get() else { return };+ let Some(latest) = lock(updates).try_iter().last() else { return };+ let Some(controls) = CONTROLS.get() else { return };+ let mut controls = lock(controls);+ let Some(controls) = controls.as_mut() else { return };+ if let Err(e) = apply(controls, latest) {+ tracing::debug!(error = %e, "can't update the media controls");+ }+}++fn apply(controls: &mut Controls, shown: Option<Shown>) -> windows::core::Result<()> {+ let smtc = &controls.smtc;+ let Some(shown) = shown else {+ *lock(&CURRENT) = None;+ smtc.SetIsEnabled(false)?;+ smtc.DisplayUpdater()?.ClearAll()?;+ controls.art = None;+ return Ok(());+ };+ let player = &shown.view.player;+ *lock(&CURRENT) = Some((shown.device, player.id.clone()));+ let can = |action: &str| player.actions.iter().any(|a| a == action);++ smtc.SetIsEnabled(true)?;+ smtc.SetIsPlayEnabled(can("play"))?;+ smtc.SetIsPauseEnabled(can("pause"))?;+ smtc.SetIsNextEnabled(can("next"))?;+ smtc.SetIsPreviousEnabled(can("previous"))?;+ smtc.SetPlaybackStatus(if player.playing {+ MediaPlaybackStatus::Playing+ } else {+ MediaPlaybackStatus::Paused+ })?;++ let updater = smtc.DisplayUpdater()?;+ updater.SetType(MediaPlaybackType::Music)?;+ let music = updater.MusicProperties()?;+ music.SetTitle(&HSTRING::from(player.title.as_deref().unwrap_or(&player.app)))?;+ // The flyout shows two lines; the second says where it plays.+ let source = format!("{} on {}", player.app, shown.device_name);+ let artist = match &player.artist {+ Some(artist) => format!("{artist} · {source}"),+ None => source,+ };+ music.SetArtist(&HSTRING::from(artist))?;+ music.SetAlbumTitle(&HSTRING::from(player.album.as_deref().unwrap_or_default()))?;+ if controls.art != shown.view.art {+ match shown.view.art.as_deref().map(std::fs::read) {+ Some(Ok(bytes)) => updater.SetThumbnail(&thumbnail(&bytes)?)?,+ _ => updater.SetThumbnail(None)?,+ }+ controls.art = shown.view.art.clone();+ }+ updater.Update()?;++ if let Some(duration) = player.duration {+ let elapsed = if player.playing {+ SystemTime::now().duration_since(shown.view.at).map_or(0, |d| d.as_millis() as u64)+ } else {+ 0+ };+ let position = player.position.unwrap_or(0).saturating_add(elapsed).min(duration);+ let span = |ms: u64| TimeSpan { Duration: i64::try_from(ms).unwrap_or(i64::MAX / 10_000) * 10_000 };+ let timeline = SystemMediaTransportControlsTimelineProperties::new()?;+ timeline.SetStartTime(span(0))?;+ timeline.SetEndTime(span(duration))?;+ timeline.SetMinSeekTime(span(0))?;+ timeline.SetMaxSeekTime(span(if can("seek") { duration } else { 0 }))?;+ timeline.SetPosition(span(position))?;+ smtc.UpdateTimelineProperties(&timeline)?;+ }+ Ok(())+}++/// Artwork as a stream reference, which the controls take for thumbnails.+fn thumbnail(bytes: &[u8]) -> windows::core::Result<RandomAccessStreamReference> {+ let stream = InMemoryRandomAccessStream::new()?;+ let writer = DataWriter::CreateDataWriter(&stream)?;+ writer.WriteBytes(bytes)?;+ writer.StoreAsync()?.join()?;+ writer.FlushAsync()?.join()?;+ writer.DetachStream()?;+ stream.Seek(0)?;+ RandomAccessStreamReference::CreateFromStream(&stream)+}diff --git a/docs/protocol/capabilities.md b/docs/protocol/capabilities.mdindex f988939..541f13e 100644--- a/docs/protocol/capabilities.md+++ b/docs/protocol/capabilities.md@@ -23,7 +23,8 @@ capability is missing.| `device.ring` | Rings on `device.ring`, even on silent | always offered || `files.transfer` | Sends and receives files ([files service](files.md)) | app update || `clip.image` | Accepts images on its clipboard ([clipboard service](clipboard.md)) | app update |-| `media.control` | Exposes its media sessions for remote control (phone: needs notification access) | phone: notification access · desktop: app update |+| `media.control` | Shares its media players for remote control ([media service](media.md); phone: needs notification access) | phone: notification access · desktop: app update |+| `media.remote` | Shows and controls other devices' media players | app update |## Phonediff --git a/docs/protocol/media.md b/docs/protocol/media.mdnew file mode 100644index 0000000..cf2a752--- /dev/null+++ b/docs/protocol/media.md@@ -0,0 +1,78 @@+# Media service (protocol v0)++> Status: **draft.** Builds on [protocol v0](v0.md); breaking changes are+> allowed until v1. License: CC BY 4.0.++What plays on one device shows on the other, which can control it: a+phone's music in the PC's app and in Windows' media flyout, a PC's in the+phone's media controls. Both directions use the same messages.++## 1. Capabilities++| ID | Offered by | Meaning |+|---|---|---|+| `media.control` | both | Shares its players and takes commands for them (a phone needs notification access to read other apps' players) |+| `media.remote` | both | Shows other devices' players and sends commands to them |++A device sends its players only to devices that offer `media.remote`, and+only while the user allows media for that device (the `media` device+toggle, on by default). It takes commands only while it offers+`media.control` and allows the sender.++## 2. Messages++All on the control stream.++```+t = "media.state" b = { players: [Player] } // no reply+t = "media.sync" b = {} // no reply+t = "media.command" id = n b = { player: text, action: text, position?: uint }+t = "ok" re = n+```++**`media.state`** carries every player the sender shares, the most+relevant first (what Windows calls the current session, or what's playing,+then the most recently active). It's sent when a session starts, after+every change, and again when the user changes what the other device may+see. An empty list means nothing plays (or the user stopped sharing); the+receiver clears what it showed. A receiver also clears a device's players+when the connection ends.++**`media.sync`** asks for a `media.state` now (the user allowed media again+on the receiving side).++**`media.command`** asks a player to `play`, `pause`, go to the `next` or+`previous` item, or `seek` to `position` (milliseconds). Errors:+`NOT_FOUND` when the player is gone, `UNSUPPORTED` when it can't do that,+`DENIED` when the user turned media off for the sender, `BAD_MESSAGE` for a+seek without a position.++```+Player = {+ id: text, // 1–256 bytes, stable while the player exists+ app: text, // user-visible app name+ title?: text, // each text up to 256 characters+ artist?: text,+ album?: text,+ playing: bool,+ duration?: uint, // milliseconds+ position?: uint, // milliseconds, when sent+ actions: [text], // what it takes: play, pause, next, previous, seek+ art_key?: text, // 1–64 bytes; the same key means the same picture+ art?: bytes, // JPEG or PNG, up to 256 KiB+}+```++While `playing`, the position moves on in real time from when the state+arrived; senders don't send updates just because time passes.++Artwork goes once per `art_key` and session: later states carry the key+alone, and receivers keep pictures by key. A device sends at most 8+players. Receivers drop actions they don't know.++## 3. Rules++- **No echoes.** A device leaves its own stand-in for another device's+ player (Windows' flyout session, Android's media session) out of what it+ shares.+- Implementations **MUST NOT** log titles, artists or albums (v0 §11).diff --git a/docs/protocol/v0.md b/docs/protocol/v0.mdindex 08a766b..02bdf8f 100644--- a/docs/protocol/v0.md+++ b/docs/protocol/v0.md@@ -8,7 +8,8 @@ connect and exchange messages. Version 0 covers the Phase 0 scope: identity,pairing, the session handshake, capabilities, heartbeat and the first events(device info, battery, ping). Feature services are specified in separatedocuments that build on this one: [notifications](notifications.md),-[clipboard](clipboard.md) and [files](files.md) so far; mirroring follows.+[clipboard](clipboard.md), [files](files.md) and [media](media.md) so far;+mirroring follows.Key words **MUST**, **SHOULD** and **MAY** are used as in RFC 2119.