Skip to content

Your Android phone and your Windows PC, working as one. Local-first, end-to-end encrypted, no account.

feat(android): the phone goes quiet while its sound plays on the PC

Bee0cb3e14ce8b1
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c82ccef..3331519 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,8 +14,8 @@ change things that need a step from you, and its notes say so.
- **Your phone's sound on the PC, without mirroring.** **Phone sound on this
PC** on Home plays what the phone plays through the PC's speakers or
headphones, with no mirror window. With Elevated it starts right away;
- otherwise the phone asks once, as for mirroring. The phone's own speaker
- keeps playing for now
+ otherwise the phone asks once, as for mirroring. The phone goes quiet
+ meanwhile and its media volume comes back after
([#94](https://github.com/thtbee/nectarlink/issues/94)).
- **Sharing your PC's screen with the phone uses H.265** when both sides
can, and H.264 otherwise. Thanks to [@AS8148](https://github.com/AS8148)
diff --git a/android/app/src/main/kotlin/app/nectarlink/android/NectarlinkApplication.kt b/android/app/src/main/kotlin/app/nectarlink/android/NectarlinkApplication.kt
index a753b3b..483a8af 100644
--- a/android/app/src/main/kotlin/app/nectarlink/android/NectarlinkApplication.kt
+++ b/android/app/src/main/kotlin/app/nectarlink/android/NectarlinkApplication.kt
@@ -53,6 +53,8 @@ class NectarlinkApplication : Application(), DefaultLifecycleObserver {
// Before Elevated: it waits while paused for a banking app.
BankingPause.init(this)
BankingPause.restoreIfStranded(this)
+ // The media volume, if the app stopped while the phone's sound played on the PC.
+ app.nectarlink.android.mirror.PhoneQuiet.end(this)
Elevated.init(this)
// What the phone offers PCs follows Elevated starting and stopping.
Elevated.onChange = { core.refreshNotificationAccess() }
diff --git a/android/app/src/main/kotlin/app/nectarlink/android/mirror/MirrorService.kt b/android/app/src/main/kotlin/app/nectarlink/android/mirror/MirrorService.kt
index d274edc..9b4c63d 100644
--- a/android/app/src/main/kotlin/app/nectarlink/android/mirror/MirrorService.kt
+++ b/android/app/src/main/kotlin/app/nectarlink/android/mirror/MirrorService.kt
@@ -340,6 +340,8 @@ class MirrorService : Service() {
return@post
}
sound = SoundCapture(running, stream) { endSharing() }.also { it.start() }
+ // Only the sound, so it's heard on the PC instead of the phone (#94).
+ if (soundOnly && sound?.usesPlaybackCapture == true) PhoneQuiet.start(this)
}
}
@@ -379,6 +381,7 @@ class MirrorService : Service() {
encoder = null
sound?.stop()
sound = null
+ PhoneQuiet.end(this)
projection?.let { runCatching { it.stop() } }
projection = null
pcId = null
diff --git a/android/app/src/main/kotlin/app/nectarlink/android/mirror/PhoneQuiet.kt b/android/app/src/main/kotlin/app/nectarlink/android/mirror/PhoneQuiet.kt
new file mode 100644
index 0000000..8ea116e
--- /dev/null
+++ b/android/app/src/main/kotlin/app/nectarlink/android/mirror/PhoneQuiet.kt
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: GPL-3.0-or-later
+package app.nectarlink.android.mirror
+
+import android.content.Context
+import android.media.AudioManager
+import android.util.Log
+import androidx.core.content.edit
+
+/**
+ * Keeps the phone quiet while its sound plays on the PC on its own (#94).
+ *
+ * Android's playback capture runs at full volume whatever the media volume is
+ * (a loopback mix is fixed volume, `SwAudioOutputDescriptor::isFixedVolume`),
+ * so turning the media volume to 0 silences the speaker and not the PC.
+ * Muting the stream instead would silence both. Elevated captures through the
+ * remote submix, which takes the sound off the speaker by itself, so it
+ * doesn't need this.
+ *
+ * The volume to give back is kept on disk, so it comes back even if the app
+ * stopped while the phone was quiet ([end] at startup).
+ */
+object PhoneQuiet {
+ private const val TAG = "PhoneQuiet"
+ private const val PREFS = "phone_quiet"
+ private const val KEY_VOLUME = "volume"
+
+ /** Turns the media volume to 0, remembering what it was. */
+ fun start(context: Context) {
+ val audio = context.getSystemService(AudioManager::class.java) ?: return
+ if (audio.isVolumeFixed) return
+ val prefs = prefs(context)
+ if (prefs.contains(KEY_VOLUME)) return
+ val volume = audio.getStreamVolume(AudioManager.STREAM_MUSIC)
+ if (volume == 0) return
+ prefs.edit(commit = true) { putInt(KEY_VOLUME, volume) }
+ runCatching { audio.setStreamVolume(AudioManager.STREAM_MUSIC, 0, 0) }.onFailure {
+ Log.w(TAG, "can't turn the media volume down", it)
+ prefs.edit(commit = true) { remove(KEY_VOLUME) }
+ }
+ }
+
+ /**
+ * Gives the media volume back, unless the user turned it up meanwhile
+ * (then theirs stays). Nothing to do if [start] didn't turn it down.
+ */
+ fun end(context: Context) {
+ val prefs = prefs(context)
+ val volume = prefs.getInt(KEY_VOLUME, -1)
+ if (volume < 0) return
+ prefs.edit(commit = true) { remove(KEY_VOLUME) }
+ val audio = context.getSystemService(AudioManager::class.java) ?: return
+ if (audio.getStreamVolume(AudioManager.STREAM_MUSIC) == 0) {
+ runCatching { audio.setStreamVolume(AudioManager.STREAM_MUSIC, volume, 0) }
+ .onFailure { Log.w(TAG, "can't give the media volume back", it) }
+ }
+ }
+
+ private fun prefs(context: Context) = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
+}
diff --git a/android/app/src/main/kotlin/app/nectarlink/android/mirror/SoundCapture.kt b/android/app/src/main/kotlin/app/nectarlink/android/mirror/SoundCapture.kt
index 5fab58a..cfd6371 100644
--- a/android/app/src/main/kotlin/app/nectarlink/android/mirror/SoundCapture.kt
+++ b/android/app/src/main/kotlin/app/nectarlink/android/mirror/SoundCapture.kt
@@ -37,6 +37,14 @@ class SoundCapture(
@Volatile private var elevatedSocket: Socket? = null
private var thread: Thread? = null
+ /**
+ * After [start]: whether the sound comes through Android's playback
+ * capture, which leaves the phone's speaker playing (the Elevated helper's
+ * remote submix takes the sound off it).
+ */
+ var usesPlaybackCapture = false
+ private set
+
@SuppressLint("MissingPermission")
fun start() {
val proj = projection
@@ -62,6 +70,7 @@ class SoundCapture(
if (record != null && record.state == AudioRecord.STATE_INITIALIZED) {
running = true
+ usesPlaybackCapture = true
thread = Thread({ runRecord(record) }, "sound-capture").apply {
priority = Thread.MAX_PRIORITY
start()
diff --git a/docs/guide/cameras-and-audio.md b/docs/guide/cameras-and-audio.md
index bc8380d..7a5b0ac 100644
--- a/docs/guide/cameras-and-audio.md
+++ b/docs/guide/cameras-and-audio.md
@@ -80,7 +80,7 @@ To hear what your phone plays (music, videos, app sounds) through your PC's spea
- With `Elevated` it starts right away. Otherwise your phone asks first, as it does for mirroring: tap the notification, then allow it. Android's prompt talks about sharing your screen, but only the sound goes to the PC.
- It's the same sound mirroring sends: on Android 10 and newer, apps that allow their audio to be captured (most media and games). Calls aren't included.
-- Your phone's own speaker keeps playing too, for now.
+- Your phone goes quiet while its sound plays on the PC. Without `Elevated` it turns its media volume down to 0 and back up when you turn this off (if you turned the volume up yourself meanwhile, yours stays).
- Opening the phone's mirror window takes over, since that brings the sound with the picture. Turn it off on the PC, or tap **Stop** on the phone.
### 4. Voice Recorder: Record on Phone, Save on PC
diff --git a/docs/protocol/mirror.md b/docs/protocol/mirror.md
index 75336a3..e53519d 100644
--- a/docs/protocol/mirror.md
+++ b/docs/protocol/mirror.md
@@ -177,7 +177,12 @@ permission to record audio, which the phone asks for the first time a PC
wants the sound. Apps that don't allow playback capture, and calls, aren't
heard. Sharing all sound (`mirror.audio`) is planned for Elevated.
-The phone's own speaker keeps playing; the PC can mute its copy.
+While mirroring, the phone's own speaker keeps playing with playback
+capture, and the PC can mute its copy. Through Elevated the sound comes
+from the remote submix, which takes it off the speaker. With only the sound
+(`sound_only`), the phone turns its media volume to 0 for playback capture
+and gives it back afterwards: a loopback capture is fixed volume, so the PC
+still gets it at full level.
## 6. App windows