Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

fix(media): stabilize sharing recovery and remote input (#513)

AAA815f1a07c49f
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index b9e15cce..1a2cd93e 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -120,6 +120,40 @@ protocol identifiers; use normal language-specific identifier conventions.
their own licenses and notices. GPL/AGPL implementation sources remain
study-only unless a separate distribution decision accepts their obligations.
+## Temporary Dependency Patches
+
+Distinguish an upstream defect from a missing capability or a Piik integration
+error. Reproduce defects through the dependency's public API without Piik;
+requiring a new API demonstrates a capability gap, not a broken existing contract.
+For capability additions, prefer a Piik adapter through existing public extension
+points. Where those are insufficient, expose a narrow reusable hook and keep
+application policy outside the dependency. Compare total ownership: moving or
+duplicating private internals into an adapter does not remove maintenance cost.
+Evaluate larger feature additions as dependency choices, rather than treating
+them as small fixes.
+
+The [dependency provenance](./internal/thirdparty/README.md) owns official
+baselines, retained patches and removal criteria. Review upstream when changing
+a patched dependency and before phase acceptance. Do not create a second patch
+inventory or edit shared package caches.
+
+Prepare an update from the recorded base and patch head in a new directory:
+
+```sh
+node scripts/sync-upstream.mjs --source <fork-url-or-local-clone> --upstream <official-url-or-local-clone> --base <official-base> --head <patch-head> --out build/upstream-candidate
+```
+
+The tool fetches official `HEAD` (or `--onto <ref>`) and replays the patch range.
+It records exact revisions, upstream-equivalent patches and conflicts in
+`.git/piik-upstream-sync.json`, with the successful range comparison in
+`.git/piik-upstream-range-diff.txt`. Existing clones, local edits and Piik pins
+remain unchanged. Conflicts stay in the candidate for review.
+
+Inspect that comparison, then run the dependency's relevant checks and Piik's
+existing reproduction or acceptance gate before changing a pin. When upstream
+covers the retained behavior, remove the replacement. The tool prepares source;
+it does not publish changes or approve a dependency update.
+
## Verification Entrypoints
Every retained test must be collected by a documented verification entry point.
diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md
index 52087b02..dfd556c7 100644
--- a/docs/adr/0005-automatic-hybrid-media-routing.md
+++ b/docs/adr/0005-automatic-hybrid-media-routing.md
@@ -148,6 +148,10 @@ and pending route revisions, and one candidate connection identity. SFU routes
also carry a publication generation. Duplicate current messages are idempotent;
stale asynchronous results fail closed.
+Revision allocation belongs to the room's lifetime, including stopped shares and
+replacement controllers. Releasing a controller does not make its revisions
+available again; ending room authority releases the sequence.
+
The old committed route remains authoritative while the candidate prepares.
Parent and child prepare the same server-issued connection identity. Standard
ICE connected is progress only. For availability work, the exact candidate child
diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md
index 617d6d1f..95b3336c 100644
--- a/docs/guide/getting-started.md
+++ b/docs/guide/getting-started.md
@@ -26,6 +26,10 @@ Only have a four-digit code? Open the same Piik site and select **Join a room**.
Enter the site passphrase or room password if asked. If code entry is unavailable
or an old link fails, ask the Host for their current invitation.
+Each App's Local room or Public invite creates a separate site, so your friend's
+code does not work on your own App page. Open their full invitation instead;
+you do not need to create a room in your App to watch them.
+
## Use Piik online
1. Open [demo.piik.tv](https://demo.piik.tv) in your browser.
diff --git a/docs/guide/getting-started.zh-CN.md b/docs/guide/getting-started.zh-CN.md
index 4a7211d5..b3152082 100644
--- a/docs/guide/getting-started.zh-CN.md
+++ b/docs/guide/getting-started.zh-CN.md
@@ -25,6 +25,9 @@ Piik 支持一位房主分享屏幕,最多 20 位受邀观众观看。先选
只有四位房间号时,打开同一个 Piik 站点,选择 **加入房间**,按需输入站点口令或房间密码。
无法通过房间号加入,或旧链接失效时,请房主发送当前邀请。
+各自用 App 创建的本地房间/公网邀请属于不同站点,不能在自己的 App 页面输入对方的房间号。
+加入朋友时,直接打开对方的完整邀请链接即可,不必在自己的 App 中创建房间。
+
## 使用在线版
1. 在浏览器中打开 [demo.piik.tv](https://demo.piik.tv)。
diff --git a/docs/operations/self-hosting.md b/docs/operations/self-hosting.md
index 4379dec8..994bb055 100644
--- a/docs/operations/self-hosting.md
+++ b/docs/operations/self-hosting.md
@@ -190,8 +190,11 @@ Add `SFU_UDP_PORT=7882` to `.env`, allow UDP 7882, and restart Piik to enable
automatic SFU fallback. When the server is behind NAT, also set `SFU_PUBLIC_IP`
to its reachable public IPv4 address or DDNS hostname and forward the same external
and internal UDP port as `SFU_UDP_PORT`. Use a hostname pointing directly to that
-IPv4, with the CDN proxy disabled. Piik resolves it at startup; after the public
-IP changes and DDNS updates, restart Piik. The same binary provides the fallback.
+IPv4, with the CDN proxy disabled. Piik resolves it at startup and every minute.
+After DDNS updates, new media connections use the updated address without a
+service restart. If the address change interrupted an ongoing share, the Host
+can restart sharing and Viewers can rejoin the room.
+The same binary provides the fallback.
Allow server forwarding capacity and outbound bandwidth for Viewers actually
using the SFU.
The Host must turn off **Privacy mode** before sharing to allow this route.
diff --git a/docs/operations/self-hosting.zh-CN.md b/docs/operations/self-hosting.zh-CN.md
index aa839316..0484fdbe 100644
--- a/docs/operations/self-hosting.zh-CN.md
+++ b/docs/operations/self-hosting.zh-CN.md
@@ -168,7 +168,8 @@ TCP 8787 仅供本机反向代理访问。STUN 域名需要直接解析到服务
在 `.env` 中添加 `SFU_UDP_PORT=7882`,放行 UDP 7882,再重启 Piik,即可启用自动 SFU 兜底。
如果服务器处于 NAT 后方,还需将 `SFU_PUBLIC_IP` 设置为外部可达的公网 IPv4 地址或 DDNS 域名。
-域名应直接指向该 IPv4,关闭 CDN 代理。Piik 在启动时解析域名;公网 IP 变化、DDNS 更新后需重启 Piik。
+域名应直接指向该 IPv4,关闭 CDN 代理。Piik 启动时解析域名,之后每分钟更新一次。
+DDNS 更新后,新媒体连接使用最新地址,无需重启服务。若地址变化打断了当前分享,房主重新开播,观众重新进入房间。
公网和内部映射的 UDP 端口都应与 `SFU_UDP_PORT` 一致。
这项功能由同一个服务端程序提供。
服务器需要为实际经由 SFU 观看的观众承担转发负载和出口带宽。
diff --git a/docs/research/native-client-media.md b/docs/research/native-client-media.md
index a0c77f91..f36ce749 100644
--- a/docs/research/native-client-media.md
+++ b/docs/research/native-client-media.md
@@ -623,6 +623,14 @@ a source. Capture and receive admission share the RFC profile predicate. This
repairs the encoded-source contract without changing routing or adding encoding
paths; it is not evidence for the reported D3D device loss.
+An October 29 mixed-codec control exposed the remaining equivalent-profile
+case: `42c01f` plus VP8 still selected VP8 against the local `42e0` registration.
+The fixed Pion dependency now compares RFC-equivalent subprofiles at the matcher,
+preserving H.264 preference without rewriting remote SDP. Actual receiver
+negotiation, retained-receiver renegotiation and encoded forwarding pass with
+the repair. [Dependency provenance](../../internal/thirdparty/README.md#pion-h264-profile-matching)
+owns its upstream submission and removal boundary.
+
Firefox-generated SDP also reproduced audio omission for a valid `bundle-only`
section with port zero. The receiver had treated every zero port as rejection;
it now follows [RFC 9143](https://www.rfc-editor.org/rfc/rfc9143.html#section-6),
diff --git a/docs/research/room-interactions.md b/docs/research/room-interactions.md
index 8007a310..3d3dc79f 100644
--- a/docs/research/room-interactions.md
+++ b/docs/research/room-interactions.md
@@ -478,9 +478,11 @@ publishers with one selected primary picture is a smaller first UI than a live
grid; the data model should represent subscriptions rather than hard-code one
room-wide current publisher.
-The proposed starting point is one authority per room, many rooms per instance,
-and existing Go/media components. Automatic room migration, replicated authority
-and a language rewrite are separate decisions, not prerequisites.
+The proposed starting point is one authority per room and many rooms per
+instance. The owner selected Rust as the 3.0 implementation target on 2026-10-26;
+the existing Go/media components remain the current 2.x implementation and the
+reference for required behavior. Automatic room migration and replicated
+authority remain separate decisions, not prerequisites.
## Attended Remote-Control Prototype
@@ -863,12 +865,16 @@ accidental input after switching away; it is not an OS window sandbox.
`SendInput` has no target-window parameter, and shortcuts can change focus.
Window selection must never imply that remote input is confined to that window.
-Input execution and authorization have separate lifetimes. A failed system input
-call drops that operation, while the existing grant accepts subsequent input.
-Failed releases remain with the input owner until cleanup succeeds; actions and
-partially inserted text are never replayed. This follows the failure scope used by
+Input execution and authorization have separate lifetimes. An unrecovered system
+input failure drops that operation, while the existing grant accepts subsequent
+input. Failed releases remain with the input owner until cleanup succeeds;
+already accepted input is never replayed. This follows the failure scope used by
[Sunshine's Windows input sink](https://github.com/LizardByte/Sunshine/blob/v2025.923.33222/src/platform/windows/input.cpp#L435),
where an unsuccessful system call does not revoke the user's control session.
+On Windows, a rejected call may recover a thread bound to the wrong desktop:
+the sink pins the OS thread, binds it to its original desktop only while that
+desktop is active, retries the unaccepted input once, then restores the thread.
+This does not authorize another desktop or elevate the App.
The Windows sink uses physical-key mapping through
[scan codes](https://learn.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-keybdinput),
diff --git a/docs/standards/configuration.md b/docs/standards/configuration.md
index 4acf1b72..107ed846 100644
--- a/docs/standards/configuration.md
+++ b/docs/standards/configuration.md
@@ -67,7 +67,7 @@ Automatic SFU fallback runs inside the Hosted process when `SFU_UDP_PORT` is set
| --- | --- |
| `SFU_UDP_PORT` | Optional UDP media port `1..65535`; unset or blank disables SFU and fixes Privacy mode on for every room. Set `7882` for the standard public listener and allow the Host to choose Privacy mode. |
| `SFU_LISTEN_HOST` | IPv4 bind address, default `0.0.0.0`; independent of HTTP `LISTEN_HOST`. Read only when SFU is enabled. |
-| `SFU_PUBLIC_IP` | Optional advertised IPv4 address or DNS hostname for a host behind NAT. A hostname resolves to IPv4 at startup; restart Piik after its address changes. Forward the same UDP port as `SFU_UDP_PORT`. Read only when SFU is enabled. |
+| `SFU_PUBLIC_IP` | Optional advertised IPv4 address or DNS hostname for a host behind NAT. A hostname resolves at startup and every minute; new media connections use the latest successful IPv4 result without restarting Piik. Forward the same UDP port as `SFU_UDP_PORT`. Read only when SFU is enabled. |
| `SFU_ONLY` | `false` by default. `true` requires `SFU_UDP_PORT` and sends every room through SFU, without Peer candidates or background P2P convergence. Applies after Server restart. |
SFU control uses the application's authenticated signaling connection. No
diff --git a/docs/standards/media-quality.md b/docs/standards/media-quality.md
index 3eb623d6..7b9a1fea 100644
--- a/docs/standards/media-quality.md
+++ b/docs/standards/media-quality.md
@@ -247,8 +247,9 @@ fixed for the current share and is reconsidered for a new share.
ADR-0007 owns probe and negotiation mechanics.
Codec selection preserves the negotiated repair stream associated with the
-selected primary codec. Native and SFU receivers use standard Pion RTX/NACK;
-their shared outgoing transport also offers RTX, with LiveKit owning repair.
+selected primary codec. Native and SFU receivers use Pion for RTX deencapsulation
+and other feedback, with their LiveKit input buffers as the sole video NACK owners.
+Their shared outgoing transport also offers RTX, with LiveKit owning repair.
Codec restrictions must not strip recovery or disable SRTP replay protection.
The Browser chooses the concrete encoder implementation. A reported H.264 codec
diff --git a/docs/standards/routing-transport.md b/docs/standards/routing-transport.md
index e7041c28..853932ce 100644
--- a/docs/standards/routing-transport.md
+++ b/docs/standards/routing-transport.md
@@ -132,6 +132,11 @@ branch. Voice uses a companion WebRTC audio connection on each selected edge;
picture retains its existing Browser/Native sender and adaptation owners.
There is no all-pairs mesh or second route planner. SFU voice reuses its existing
UDP listener and admission, including on SFU-only sites.
+An audio input's lifetime is separate from its reusable listening output.
+Changing publishers preserves the negotiated output while the mature forwarding
+component maintains its RTP clock and feedback. Revoking a source stops both
+new and already-buffered delivery; stale source callbacks cannot affect its
+replacement.
A candidate carrying picture and voice commits only after both decoded-picture
proof and the required voice transports connect. A silent voice-only candidate
@@ -156,8 +161,10 @@ voice readiness never becomes evidence that a video frame has arrived.
One room controller owns the committed graph and one bounded child transition.
Healthy current edges remain sticky. When admitted overlap exists, a usable old
route stays active until the exact candidate child decodes a first new frame;
-ICE connected alone is not commit proof. Availability repair may instead use the
-explicit bounded-gap plan defined by ADR-0005 when no overlap slot exists.
+the candidate must still be connected when that proof is accepted and when it
+replaces playback. ICE connected alone is not commit proof. Availability repair
+may instead use the explicit bounded-gap plan defined by ADR-0005 when no overlap
+slot exists.
Failure or stale authority releases the candidate and preserves unaffected
branches. An operation's purpose is immutable; availability work preempts a
background operation by replacing it rather than changing its meaning. An
diff --git a/docs/standards/versioning.md b/docs/standards/versioning.md
index 54e3dffe..ef8fe2d3 100644
--- a/docs/standards/versioning.md
+++ b/docs/standards/versioning.md
@@ -126,8 +126,11 @@ release descriptor. Fixed names allow GitHub's native latest-asset links;
and verification of the matching Gitee mirror.
Android uses the installable `piik-app-android-universal.apk` for both ARM64 and
-x64. It follows the same release identity, checksum, complete-target and
-mirror rules. [Android build instructions](../../mobile/README.md#signed-release-apk)
+x64. The same build also produces `piik-app-android-universal.zip`, containing
+only that unchanged signed APK. GitHub keeps both originals; Gitee distributes
+the ZIP, and its download links explain extraction before installation.
+Both follow the same release identity, checksum and complete-target rules.
+[Android build instructions](../../mobile/README.md#signed-release-apk)
own signing input and installation details.
Public Release attachments contain the App ZIPs/APK and Server runtime archives
@@ -140,7 +143,7 @@ archives are separate from Piik's uploaded packages. Retain original build
metadata with operator deployment records as described in [deployment](../deployment.md).
Update checks compare actual release precedence. The App reports its compiled
-OS/architecture and links to that platform's ZIP in the selected release metadata;
+OS/architecture and links to that platform's package in the selected release metadata;
the release page remains the fallback when no matching attachment is available.
Fixed archive names and published names with that release's SHA suffix are
accepted only with the provider's exact release download URL. Downloads start on
@@ -344,8 +347,12 @@ GitHub Releases is the primary publication; Gitee is an optional official mirror
for domestic access, with a minimal README pointing development and documentation
back to GitHub. The Gitee repository need not copy the source tree. Its README/tag
commit is not the Piik source revision; release metadata retains the original
-GitHub SHA. Build each target once and copy the same archives, notices,
-version, full source SHA and artifact hashes to both. A Git repository mirror
+GitHub SHA. Build each target once and copy the selected original archives,
+notices, version, full source SHA and artifact hashes to both. Android's ZIP
+wrapper is a build artifact, not a mirror-side rebuild. Gitee's
+[official response](https://gitee.com/oschina/git-osc/issues/IB93ZJ) states that it
+cannot distribute APK downloads; do not work around its response type in the Browser.
+A Git repository mirror
alone does not synchronize Release attachments. Keep one release description.
Update detection reads each provider's API metadata; it does not parse the public
changelog or download build descriptors. The existing pipeline owns local
diff --git a/docs/status.md b/docs/status.md
index a421c923..4e22e64a 100644
--- a/docs/status.md
+++ b/docs/status.md
@@ -1,6 +1,6 @@
# Current Status
-Last updated: 2026-10-26
+Last updated: 2026-10-27
This is the compact execution/deployment index. Product modules own behavior,
[verification status](./verification-status.md) summarizes material evidence limits,
@@ -37,8 +37,12 @@ Release artifacts and operator records own delivery completion.
## Current Work
-The maintenance line preserves published protocols, compatibility and route
-ownership. [TODO](./todo.md) is the only unfinished-work ledger; Git and PRs own
+The maintenance candidate preserves published protocols, compatibility and route
+ownership. Its routing, media recovery and input-lifecycle audit is complete;
+confirmed defects are repaired and cross-reviewed. Publication awaits the owner's
+instruction. Isolated 3.0 work proceeds from the accepted semantic model rather
+than a mechanical translation of the current implementation.
+[TODO](./todo.md) is the only unfinished-work ledger; Git and PRs own
completed fixes. [Verification status](./verification-status.md) owns material
acceptance and platform boundaries.
diff --git a/docs/todo.md b/docs/todo.md
index c63bc484..82cd6550 100644
--- a/docs/todo.md
+++ b/docs/todo.md
@@ -1,6 +1,6 @@
# Current TODO Ledger
-Last reviewed: 2026-10-26
+Last reviewed: 2026-10-29
This ledger contains unfinished work and explicit owner holds. Product modules
own behavior; research and Git/PRs own evidence and completed history.
@@ -19,22 +19,36 @@ own behavior; research and Git/PRs own evidence and completed history.
[#499](https://github.com/TNTcraftHIM/Piik/issues/499) on niri/Wayland with
GStreamer NVENC. The Browser report records capture ending before connection
cleanup; obtain the native capture failure to distinguish the encoder and Portal.
-- [ ] **macOS window capture with sound (#500).** Follow
- [#500](https://github.com/TNTcraftHIM/Piik/issues/500). Compare the same window
- with source sound on and off in the App that waits for video `active`
- before starting source audio. Obtain the App diagnostic if the window stays black.
+- [ ] **macOS source-picker thumbnails (#500).** The reporter confirmed normal
+ sharing on 2026-10-28; the remaining [#500](https://github.com/TNTcraftHIM/Piik/issues/500)
+ report concerns missing screen thumbnails in the source picker. The macOS
+ helper now implements the existing preview command; verify its build and
+ display/window thumbnails on a physical Mac before closing the issue.
- [ ] **Control loss when the Host browser is minimized (#502).** Follow
- [#502](https://github.com/TNTcraftHIM/Piik/issues/502) with the corrected
- Windows App. If minimizing still prevents input, obtain its Host App
- diagnostic ZIP to identify the rejected system operation.
+ [#502](https://github.com/TNTcraftHIM/Piik/issues/502) on the original Windows
+ host with the Windows input-recovery candidate. The v2.5.0 report shows
+ rejected input followed by failed release cleanup. The candidate repairs a
+ reproduced thread-desktop mismatch and records the native rejection context;
+ confirm the original host's trigger before closing the issue.
- [ ] **Return LiveKit to an upstream pin.** Follow
[upstream #5016](https://github.com/livekit/livekit/pull/5016). Once upstream
supplies the allocator input Piik needs, verify recovery and remove the
temporary replacement. [Dependency provenance](../internal/thirdparty/README.md#local-encoder-probe-demand)
owns the patch and removal boundary.
+- [ ] **Return Pion to an upstream pin.** Follow
+ [upstream #3668](https://github.com/pion/webrtc/pull/3668). Once upstream
+ includes equivalent H.264 profile matching, verify the receiver and forwarding
+ controls and remove the temporary replacement. [Dependency provenance](../internal/thirdparty/README.md#pion-h264-profile-matching)
+ owns the patch and removal boundary.
## Next
+- [ ] **3.0 semantic reconstruction.** Continue the isolated `feat/channel-prototype`
+ from the accepted domain/room/participant model. Map existing behavior and
+ ownership, remove accidental constraints, then implement cohesive Rust runtimes
+ with mature components. Preserve the reviewed recovery semantics, not Go
+ function or module parity. The prototype's design owners and ledger govern
+ its implementation; keep it outside the maintenance candidate.
- [ ] **Release headlines in App and Server update checks.** Keep the prepared
implementation on `archive/deferred-prototypes` for a later feature release. It consumes
the existing [headline contract](./standards/versioning.md#release-notes).
@@ -64,11 +78,6 @@ own behavior; research and Git/PRs own evidence and completed history.
## Parked Product Work
-- **Piik 3.0 scope.** Explore independent room, chat, voice and screen-share
- lifecycles, with optional image/file exchange, retaining the current topology
- and route controller. Compare continued Go with scoped Rust components before
- choosing implementation scope. The [hosting and subscription assessment](./research/room-interactions.md#multi-room-and-distributed-hosting-assessment)
- distinguishes per-room hosting, federation and replicated room authority.
- **Virtual gamepads beyond Windows.** Linux uinput passed its isolated kernel
gate; desktop permissions and game recognition need physical checks. macOS
virtual HID needs an entitled App and macOS SDK/runtime checks. The
diff --git a/go.mod b/go.mod
index c2ee12e0..c58a9881 100644
--- a/go.mod
+++ b/go.mod
@@ -23,15 +23,15 @@ require (
github.com/livekit/server-sdk-go/v2 v2.14.0
github.com/netbirdio/go-nat v0.0.0-20260821095157-6b2c8c5c74e8
github.com/pion/ice/v4 v4.4.8
- github.com/pion/interceptor v0.1.49
+ github.com/pion/interceptor v0.1.51
github.com/pion/logging v0.2.4
- github.com/pion/rtcp v1.2.19
- github.com/pion/rtp v1.10.5
+ github.com/pion/rtcp v1.2.20
+ github.com/pion/rtp v1.10.6
github.com/pion/sdp/v3 v3.0.20
github.com/pion/stun/v4 v4.0.1
- github.com/pion/transport/v5 v5.1.1
- github.com/pion/turn/v5 v5.1.2
- github.com/pion/webrtc/v4 v4.2.24
+ github.com/pion/transport/v5 v5.1.4
+ github.com/pion/turn/v5 v5.1.3
+ github.com/pion/webrtc/v4 v4.2.26
github.com/stretchr/testify v1.12.1
github.com/thesyncim/gopus v0.2.2
github.com/wlynxg/anet v0.0.5
@@ -98,7 +98,7 @@ require (
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/petermattis/goid v0.0.0-20260918085751-abfca077860b // indirect
github.com/pion/datachannel v1.6.3 // indirect
- github.com/pion/dtls/v3 v3.1.13 // indirect
+ github.com/pion/dtls/v3 v3.1.15 // indirect
github.com/pion/mdns/v2 v2.2.2 // indirect
github.com/pion/randutil v0.1.0 // indirect
github.com/pion/sctp v1.12.0 // indirect
@@ -137,3 +137,6 @@ require (
// Local-encoder demand hook pending upstream #5016; see internal/thirdparty/README.md.
replace github.com/livekit/livekit-server => github.com/TNTcraftHIM/livekit v1.13.11-0.20261018043008-c7f56b389713
+
+// Equivalent H.264 profiles pending upstream #3668; see internal/thirdparty/README.md.
+replace github.com/pion/webrtc/v4 => github.com/TNTcraftHIM/pion-webrtc/v4 v4.0.0-20261029063205-6ed9e55c5935
diff --git a/go.sum b/go.sum
index a2b63d13..648102d2 100644
--- a/go.sum
+++ b/go.sum
@@ -16,6 +16,8 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERo
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/TNTcraftHIM/livekit v1.13.11-0.20261018043008-c7f56b389713 h1:RrcmStTuxqbsgEai2S+wN5vBYkrMyx9GZIWKBwJgIBA=
github.com/TNTcraftHIM/livekit v1.13.11-0.20261018043008-c7f56b389713/go.mod h1:VisDyFQKxQr0ncrV3558MBUKjlrTUqXMBQwsxEHMRgA=
+github.com/TNTcraftHIM/pion-webrtc/v4 v4.0.0-20261029063205-6ed9e55c5935 h1:RyZPG4+9aG+rzoDsbeOjS/hOAdvsQXEXRXumY0zBIgg=
+github.com/TNTcraftHIM/pion-webrtc/v4 v4.0.0-20261029063205-6ed9e55c5935/go.mod h1:vH5Fn0zOZ8LuLhIPEtsLkoYkwnvubV4TX+ZtjV5Jqgk=
github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
github.com/aymanbagabas/go-udiff v0.4.1 h1:OEIrQ8maEeDBXQDoGCbbTTXYJMYRCRO1fnodZ12Gv5o=
@@ -235,10 +237,14 @@ github.com/pion/datachannel v1.6.3 h1:HZAYLpunVI0xi057ZiANl558d0uSpomcWw4UEPz3qX
github.com/pion/datachannel v1.6.3/go.mod h1:oRmkXVVXxkUbqa8MatgdNHW29kvtQY3HUdjJQsfv2GE=
github.com/pion/dtls/v3 v3.1.13 h1:oWNjsoF3xUPDhzOuE8U42ql1BtVzWc8X4Asiz9BKwYk=
github.com/pion/dtls/v3 v3.1.13/go.mod h1:iKFQNYrjsN2TiA2YKKMqB9MOZaFpjFULBI/A4sW0eyc=
+github.com/pion/dtls/v3 v3.1.15 h1:okWk5LNwU21DfhvG8z4wm45knHVSPEdkihwp6BxfQcc=
+github.com/pion/dtls/v3 v3.1.15/go.mod h1:iKFQNYrjsN2TiA2YKKMqB9MOZaFpjFULBI/A4sW0eyc=
github.com/pion/ice/v4 v4.4.8 h1:NB22Ze5WrgMjcf9L+O+1B29bl6anq+sbyvddImDLdhc=
github.com/pion/ice/v4 v4.4.8/go.mod h1:zCRRg4Mkpdbx9B+75fd4z2APvIXeIraeUOB7IMPt8mU=
github.com/pion/interceptor v0.1.49 h1:iyBsNHRoLNNXZiJA/DdGvJgdyfS8YWDCzxIAlVjt5nY=
github.com/pion/interceptor v0.1.49/go.mod h1:MZ6PJkja/TCo350HAnBrs/rUIyad9mWjpcvytrf3ViQ=
+github.com/pion/interceptor v0.1.51 h1:PU09y6RghjWXR4AYJ+AVNyDX7EmgIZuDFMGDl6smKQA=
+github.com/pion/interceptor v0.1.51/go.mod h1:nVgAA8pNNhaglm6NSe9lMuFhmi8kwWbf7Whw1ukt9VU=
github.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=
github.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=
github.com/pion/mdns/v2 v2.2.2 h1:qq9Kj3PnjF0mfSwd8Icu0CXy5aoYdQKnNUIsGPeUpgk=
@@ -247,8 +253,12 @@ github.com/pion/randutil v0.1.0 h1:CFG1UdESneORglEsnimhUjf33Rwjubwj6xfiOXBa3mA=
github.com/pion/randutil v0.1.0/go.mod h1:XcJrSMMbbMRhASFVOlj/5hQial/Y8oH/HVo7TBZq+j8=
github.com/pion/rtcp v1.2.19 h1:T79lR2Tfd/7zj+41nsUrftrUEee+Z03NQr9GSPSQ/WU=
github.com/pion/rtcp v1.2.19/go.mod h1:vX7Es6skWYm4oVvOSC5+4WHLYUTZHcbGNmk5wjzMGH4=
+github.com/pion/rtcp v1.2.20 h1:ZxQpb1Z3st9W5Rts4PjGoC3ynDhUtQ9B/itKJho8h2I=
+github.com/pion/rtcp v1.2.20/go.mod h1:vX7Es6skWYm4oVvOSC5+4WHLYUTZHcbGNmk5wjzMGH4=
github.com/pion/rtp v1.10.5 h1:ip0HhO/wYZqQ4bKS+R99KnZh/GRCmIT0jDXikub7vlE=
github.com/pion/rtp v1.10.5/go.mod h1:Au8fc6cEByy8RLTwKTQTEeQqDB/SJDxwL4mZuxYA5Pk=
+github.com/pion/rtp v1.10.6 h1:1qs356bmniSc89gTF1KFdI0kej7qTdIUSnHOXu3P2EY=
+github.com/pion/rtp v1.10.6/go.mod h1:qvV617FVkp5DBUfoHGxP8uOnTI4UaNDJK0z7Sildylk=
github.com/pion/sctp v1.12.0 h1:AzUdVkVBNyGY1W4crWO1/aOWzMaW3zsVHGrNuAfFQiI=
github.com/pion/sctp v1.12.0/go.mod h1:6k1zrpkrxSwbisitn/AlzDQaa15DR49MFsrMochG/yc=
github.com/pion/sdp/v3 v3.0.20 h1:TS6DViqcmp+49f0+mjw9anbr9xY3vJtsZewxAvlMCRQ=
@@ -261,8 +271,12 @@ github.com/pion/transport/v4 v4.1.1 h1:kEBtbp1sRad0tnkREDNJ3fmop/CR8p/YxXnpFbn9t
github.com/pion/transport/v4 v4.1.1/go.mod h1:Oxo5oODQvjFF7NkHBBz3f4fZXLB7x03uLVuzfYeA5pE=
github.com/pion/transport/v5 v5.1.1 h1:Sauq8YJqRopgRfIgnpfKpdpKG14g0N4jWlRgpWsEfN8=
github.com/pion/transport/v5 v5.1.1/go.mod h1:Qxw6fCEjFWQkRDZOhS4Vf+neJBcihauvA3uyEa1J1F0=
+github.com/pion/transport/v5 v5.1.4 h1:j0E5QIPyv28YnWxa8yE9AyDmrxVnwhZ109c7Dpt48K4=
+github.com/pion/transport/v5 v5.1.4/go.mod h1:Qxw6fCEjFWQkRDZOhS4Vf+neJBcihauvA3uyEa1J1F0=
github.com/pion/turn/v5 v5.1.2 h1:acEOO+D5txJ52vKfcilmSMItfP23Q9K6pJ1LM5P2VV0=
github.com/pion/turn/v5 v5.1.2/go.mod h1:57Oadc8fHnac15VFyCd6nEaBTVZBxQe2POUMD4lsHRA=
+github.com/pion/turn/v5 v5.1.3 h1:dxBq+M6o5DCxIjtiBQNvtHkKshS+xlJbXwgc1MKzBH8=
+github.com/pion/turn/v5 v5.1.3/go.mod h1:ocKg/4W5d4N9hvpADSwoc5HdU53/4i966FIU1qJL0tY=
github.com/pion/webrtc/v4 v4.2.24 h1:Ove9nRpDAfK39+QSw3/Ru17VDiECjbiLaM73mRG4Yms=
github.com/pion/webrtc/v4 v4.2.24/go.mod h1:JxD/FgjMDKCWFukjknTdG5a9rR4PUxkg0yBX1/JBInM=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
diff --git a/internal/app/mediaedge/edge.go b/internal/app/mediaedge/edge.go
index 4483c803..40051526 100644
--- a/internal/app/mediaedge/edge.go
+++ b/internal/app/mediaedge/edge.go
@@ -13,6 +13,7 @@ import (
"github.com/TNTcraftHIM/Piik/internal/app/nativeaudio"
"github.com/TNTcraftHIM/Piik/internal/diagnostics"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/TNTcraftHIM/Piik/internal/media/inputroute"
"github.com/pion/ice/v4"
"github.com/pion/webrtc/v4"
)
@@ -21,8 +22,7 @@ const maxPendingCandidates = 64
type EdgeEvents struct {
LocalCandidate func(*webrtc.ICECandidateInit)
- // Historical loss still retires input permission. Media projections must
- // check isCurrentState again after any event queues.
+ // Media projections must check isCurrentState again after event queues.
ConnectionState func(state webrtc.PeerConnectionState, pair *SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool)
}
@@ -268,6 +268,16 @@ func (edge *Edge) OpenDataChannel(label string) (*webrtc.DataChannel, error) {
return edge.connection.CreateDataChannel(label, nil)
}
+// OnInputTransportLost binds the input leg before the initial offer. Pion's
+// ordered ICE observer is separate from its asynchronous PC state projection.
+func (edge *Edge) OnInputTransportLost(lost func()) {
+ inputroute.ObserveTransport(edge.connection.SCTP().Transport().ICETransport(), func(available bool) {
+ if !available {
+ lost()
+ }
+ })
+}
+
// Local bridges have no STUN or gateway discovery. Complete SDP keeps their
// candidates and answer in one bounded negotiation, including ICE restarts.
func (edge *Edge) CreateLocalOffer(ctx context.Context, offerID string, restart bool) (webrtc.SessionDescription, error) {
diff --git a/internal/app/mediaedge/engine.go b/internal/app/mediaedge/engine.go
index c404a8b0..e6fd3840 100644
--- a/internal/app/mediaedge/engine.go
+++ b/internal/app/mediaedge/engine.go
@@ -107,7 +107,7 @@ func NewEngine(options EngineOptions) (*Engine, error) {
return nil, err
}
registry := &interceptor.Registry{}
- if err := webrtc.RegisterDefaultInterceptors(mediaEngine, registry); err != nil {
+ if err := forwarding.RegisterIngressInterceptors(mediaEngine, registry); err != nil {
return nil, err
}
ctx, cancel := context.WithCancel(context.Background())
@@ -200,9 +200,13 @@ func (engine *Engine) newSource(codec string, capacity, layers int, relay *Relay
Formats: make([]forwarding.LayerFormat, layers), MaxPackets: encoded.MaxPacketWindow,
OnRTCP: func(layer int, packets []rtcp.Packet) {
for _, packet := range packets {
- switch packet.(type) {
+ switch packet := packet.(type) {
case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest:
source.requestLayerKeyFrame(layer)
+ case *rtcp.TransportLayerNack:
+ if layer == layers-1 && source.writeNACK != nil {
+ source.writeNACK(packet)
+ }
}
}
},
diff --git a/internal/app/mediaedge/publication.go b/internal/app/mediaedge/publication.go
index e3525b9e..889bd1ce 100644
--- a/internal/app/mediaedge/publication.go
+++ b/internal/app/mediaedge/publication.go
@@ -41,6 +41,10 @@ func (publication *Publication) OpenDataChannel(label string) (*webrtc.DataChann
return publication.signaling.OpenDataChannel(label)
}
+func (publication *Publication) OnInputTransportLost(lost func()) {
+ publication.signaling.OnInputTransportLost(lost)
+}
+
func (engine *Engine) NewPublication(source *Source, options EdgeOptions) (*Publication, error) {
if source == nil || source.engine != engine || options.Local || options.ConnectionID == "" ||
len(options.ConnectionID) > 256 || options.Audio != nil && options.Audio.engine != engine {
diff --git a/internal/app/mediaedge/receiver.go b/internal/app/mediaedge/receiver.go
index 174c543b..c63f5040 100644
--- a/internal/app/mediaedge/receiver.go
+++ b/internal/app/mediaedge/receiver.go
@@ -11,6 +11,7 @@ import (
"github.com/TNTcraftHIM/Piik/internal/app/nativeaudio"
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/TNTcraftHIM/Piik/internal/media/inputroute"
"github.com/pion/rtcp"
"github.com/pion/sdp/v3"
"github.com/pion/webrtc/v4"
@@ -21,6 +22,8 @@ type ReceiverEvents struct {
LocalCandidate func(candidate *webrtc.ICECandidateInit, current func() bool)
ConnectionState func(state webrtc.PeerConnectionState, pair *SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool)
DataChannel func(*webrtc.DataChannel)
+ // Bound before SDP starts the transport; runs in Pion's ordered ICE observer.
+ InputTransportAvailable func(bool)
}
type ReceiverOptions struct {
@@ -90,6 +93,9 @@ func (engine *Engine) NewReceiver(options ReceiverOptions) (_ *Receiver, _ webrt
if err != nil {
return nil, webrtc.SessionDescription{}, err
}
+ if options.Events.InputTransportAvailable != nil {
+ inputroute.ObserveTransport(connection.SCTP().Transport().ICETransport(), options.Events.InputTransportAvailable)
+ }
if err = connection.SetRemoteDescription(options.Offer); err != nil {
_ = connection.Close()
return nil, webrtc.SessionDescription{}, err
@@ -124,6 +130,17 @@ func (engine *Engine) NewReceiver(options ReceiverOptions) (_ *Receiver, _ webrt
_ = connection.Close()
return nil, webrtc.SessionDescription{}, err
}
+ // Install before OnTrack can feed the source. Only the received original
+ // requests upstream repair; derived outputs keep their existing recovery.
+ receiver.source.writeNACK = func(nack *rtcp.TransportLayerNack) {
+ receiver.mu.Lock()
+ ssrc, closed := receiver.videoSSRC, receiver.closed
+ receiver.mu.Unlock()
+ if !closed && ssrc != 0 {
+ nack.MediaSSRC = uint32(ssrc)
+ _ = connection.WriteRTCP([]rtcp.Packet{nack})
+ }
+ }
if hasAudio {
receiver.audioSource, err = engine.NewRelayedAudioSource(options.EdgeCapacity)
if err != nil {
@@ -147,6 +164,12 @@ func (engine *Engine) NewReceiver(options ReceiverOptions) (_ *Receiver, _ webrt
if !receiver.active() {
return
}
+ if state == webrtc.PeerConnectionStateConnected && isCurrentState(state) {
+ // A retained source can have lost a frame's dependencies while ICE
+ // restarted. Refresh it when transport resumes, before downstream
+ // decoders have to exhaust NACK repair and request their own key.
+ receiver.RequestKeyFrame()
+ }
debug := slog.Default().Enabled(engine.ctx, slog.LevelDebug)
if debug {
slog.Debug("piik-client", "event", "media-connection-state", "direction", "inbound", "state", state.String())
diff --git a/internal/app/mediaedge/receiver_test.go b/internal/app/mediaedge/receiver_test.go
index fa4bd09f..ea9c8408 100644
--- a/internal/app/mediaedge/receiver_test.go
+++ b/internal/app/mediaedge/receiver_test.go
@@ -10,13 +10,32 @@ import (
"testing"
"time"
+ "github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/TNTcraftHIM/Piik/internal/media/forwarding/forwardingtest"
"github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/pion/rtcp"
"github.com/pion/rtp"
"github.com/pion/sdp/v3"
"github.com/pion/stun/v4"
"github.com/pion/webrtc/v4"
)
+func TestReceiverStopsNACKAfterRTXRepair(t *testing.T) {
+ forwardingtest.IngressRepair(t, func(offer webrtc.SessionDescription, _ webrtc.SettingEngine) (*webrtc.PeerConnection, func() *forwarding.Source) {
+ engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = engine.Close() })
+ receiver, _, err := engine.NewReceiver(ReceiverOptions{Offer: offer, EdgeCapacity: 1})
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = receiver.Close() })
+ return receiver.connection, func() *forwarding.Source { return receiver.source.media.Source }
+ })
+}
+
func TestOfferSendsCodecDistinguishesBundledAndRejectedMedia(t *testing.T) {
for _, test := range []struct {
name string
@@ -288,6 +307,7 @@ func TestReceiverNegotiatesForwardableH264(t *testing.T) {
{"baseline", []string{"42001f"}, ""},
{"missing-profile", []string{""}, ""},
{"equivalent-baseline", []string{"42c01f"}, "42c01f"},
+ {"equivalent-baseline-with-vp8", []string{"42c01f", "vp8"}, "42c01f"},
{"small-browser-output", []string{"42e00b"}, "42e00b"},
{"level-5.1", []string{"42e033"}, "42e033"},
{"level-ceiling", []string{"42e034"}, "42e034"},
@@ -401,9 +421,27 @@ func testReceiverForwarding(t *testing.T, codec string) {
if err != nil {
t.Fatal(err)
}
- if _, err = upstream.AddTrack(track); err != nil {
+ videoSender, err := upstream.AddTrack(track)
+ if err != nil {
t.Fatal(err)
}
+ recoveryRequests := make(chan struct{}, 16)
+ go func() {
+ for {
+ packets, _, readErr := videoSender.ReadRTCP()
+ if readErr != nil {
+ return
+ }
+ for _, packet := range packets {
+ if _, ok := packet.(*rtcp.PictureLossIndication); ok {
+ select {
+ case recoveryRequests <- struct{}{}:
+ default:
+ }
+ }
+ }
+ }
+ }()
for _, transceiver := range upstream.GetTransceivers() {
if transceiver.Kind() == webrtc.RTPCodecTypeVideo {
if err = transceiver.SetCodecPreferences([]webrtc.RTPCodecParameters{videoCodecs[codec]}); err != nil {
@@ -627,6 +665,11 @@ func testReceiverForwarding(t *testing.T, codec string) {
source, audio := nativeReceiver.Source(), nativeReceiver.AudioSource()
socket := nativeReceiver.socket
for _, restart := range []bool{false, true} {
+ if restart {
+ for len(recoveryRequests) > 0 {
+ <-recoveryRequests
+ }
+ }
previousGathering := nativeReceiver.localCandidates
candidateMu.Lock()
remoteReady = false
@@ -660,6 +703,10 @@ func testReceiverForwarding(t *testing.T, codec string) {
_ = upstream.AddICECandidate(*candidate)
}
waitConnected(t, upstream, "renegotiated receiver")
+ if restart {
+ waitConnected(t, nativeReceiver.connection, "receiver after ICE restart")
+ waitSignal(t, recoveryRequests, "recovery frame request after Native transport resumes")
+ }
if (previousGathering != nativeReceiver.localCandidates) != restart {
t.Fatalf("collector replaced=%v, ICE restart=%v", previousGathering != nativeReceiver.localCandidates, restart)
}
diff --git a/internal/app/mediaedge/relay_derivation.go b/internal/app/mediaedge/relay_derivation.go
index 3bcb3075..828cf47c 100644
--- a/internal/app/mediaedge/relay_derivation.go
+++ b/internal/app/mediaedge/relay_derivation.go
@@ -289,14 +289,12 @@ func (run *relayRun) write(stream *nativecapture.Stream) error {
if frame.Width == 0 || frame.Height == 0 {
continue
}
- waitingRecovery = false
- stamp := clock.Update(timestamp)
- if stamp.IsUnhandled || stamp.ExtendedVal < clock.GetExtendedStart() {
+ var accepted bool
+ frame.Timestamp, frame.Duration, accepted = relayFrameTiming(clock, timestamp)
+ if !accepted {
continue
}
- ticks := stamp.ExtendedVal - clock.GetExtendedStart()
- frame.Timestamp = time.Duration(ticks/videoClockRate)*time.Second + time.Duration(ticks%videoClockRate)*time.Second/videoClockRate
- frame.Duration = sample.Duration
+ waitingRecovery = false
if frame.Duration <= 0 {
frame.Duration = time.Second / time.Duration(run.plan.profile.Framerate)
}
@@ -312,6 +310,21 @@ func (run *relayRun) write(stream *nativecapture.Stream) error {
}
}
+// The received RTP clock owns both decoder presentation and frame intervals.
+func relayFrameTiming(clock *utils.WrapAround[uint32, uint64], timestamp uint32) (time.Duration, time.Duration, bool) {
+ stamp := clock.Update(timestamp)
+ if stamp.IsUnhandled || stamp.ExtendedVal < clock.GetExtendedStart() {
+ return 0, 0, false
+ }
+ ticks := stamp.ExtendedVal - clock.GetExtendedStart()
+ pts := time.Duration(ticks/videoClockRate)*time.Second + time.Duration(ticks%videoClockRate)*time.Second/videoClockRate
+ var duration time.Duration
+ if stamp.PreExtendedHighest != clock.GetExtendedStart() && stamp.ExtendedVal > stamp.PreExtendedHighest {
+ duration = time.Duration(stamp.ExtendedVal-stamp.PreExtendedHighest) * time.Second / videoClockRate
+ }
+ return pts, duration, true
+}
+
func (run *relayRun) read(stream *nativecapture.Stream) {
for {
frame, err := stream.Read()
diff --git a/internal/app/mediaedge/relay_input.go b/internal/app/mediaedge/relay_input.go
index b1b73124..ffd39f38 100644
--- a/internal/app/mediaedge/relay_input.go
+++ b/internal/app/mediaedge/relay_input.go
@@ -8,6 +8,7 @@ import (
"io"
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
+ "github.com/TNTcraftHIM/Piik/internal/media/encoded"
"github.com/bluenviron/mediacommon/v2/pkg/codecs/h264"
mediacodec "github.com/livekit/mediatransportutil/pkg/codec"
"github.com/livekit/server-sdk-go/v2/pkg/samplebuilder"
@@ -32,19 +33,39 @@ type relayVideoInput struct {
pending bool
pendingTimestamp uint32
latestTimestamp uint32
+ h264AU *relayH264AU
+}
+
+// Use the SDK's sequence reorder without its inferred NAL/frame boundaries.
+// The AU owner below consumes the original timestamp, marker and FU chain.
+type relayH264Packets struct{ codecs.H264Packet }
+
+func (*relayH264Packets) IsPartitionHead([]byte) bool { return true }
+func (*relayH264Packets) IsPartitionTail(bool, []byte) bool { return true }
+
+type relayH264AU struct {
+ packets []*rtp.Packet
+ bytes int
+ failed bool
}
func (input *relayVideoInput) push(packet *rtp.Packet) error {
- if input.hasRecovery && int32(packet.Timestamp-input.recoveryTimestamp) < 0 {
+ if len(packet.Payload) > 0 && input.hasRecovery && int32(packet.Timestamp-input.recoveryTimestamp) < 0 {
return nil
}
recovery := mediacodec.IsKeyFrame(input.codec, packet.Payload)
if recovery && input.hasRecovery && packet.Timestamp == input.recoveryTimestamp && packet.SequenceNumber == input.recoverySequence {
return nil
}
- if input.builder == nil || recovery && (!input.hasRecovery || packet.Timestamp != input.recoveryTimestamp) {
+ sameH264AU := input.codec == "h264" && input.pending && packet.Timestamp == input.pendingTimestamp
+ if input.builder == nil || recovery && !sameH264AU && (!input.hasRecovery || packet.Timestamp != input.recoveryTimestamp) {
previous := input.builder
- var depacketizer rtp.Depacketizer = &codecs.H264Packet{}
+ var retired []*rtp.Packet
+ if input.h264AU != nil && !input.h264AU.failed {
+ retired = input.h264AU.packets
+ }
+ input.h264AU = nil
+ var depacketizer rtp.Depacketizer = &relayH264Packets{}
if input.codec == "vp8" {
depacketizer = &codecs.VP8Packet{}
}
@@ -53,26 +74,24 @@ func (input *relayVideoInput) push(packet *rtp.Packet) error {
samplebuilder.WithPacketDroppedHandler(func() {
// Old gaps cannot request another keyframe after its replacement
// has arrived. Loss in the current assembly still owns recovery.
- if input.builder == builder && input.onPacketDropped != nil {
- input.onPacketDropped()
+ if input.builder == builder {
+ if input.codec == "h264" {
+ input.discardH264(input.pendingTimestamp)
+ } else if input.onPacketDropped != nil {
+ input.onPacketDropped()
+ }
}
}))
input.builder = builder
input.pending = false
- if previous != nil {
- for {
- sample, _ := previous.ForcePopWithTimestamp()
- if sample == nil {
- break
- }
- // Keep completed H264 configuration, never replay expired video.
- if input.codec == "h264" {
- if _, err := input.frame(sample.Data); err != nil {
- return err
- }
- }
+ if previous != nil && input.codec == "h264" {
+ for packets := previous.ForcePopPackets(); len(packets) > 0; packets = previous.ForcePopPackets() {
+ retired = append(retired, packets...)
}
}
+ if err := input.cacheH264(retired); err != nil {
+ return err
+ }
}
if recovery {
input.recoveryTimestamp, input.hasRecovery = packet.Timestamp, true
@@ -90,6 +109,9 @@ func (input *relayVideoInput) push(packet *rtp.Packet) error {
}
func (input *relayVideoInput) pop() (*media.Sample, uint32) {
+ if input.codec == "h264" {
+ return input.popH264()
+ }
sample, timestamp := input.builder.PopWithTimestamp()
// The packet window bounds storage for large access units, not playout
// delay. After a second of advancing media without a complete sample, let
@@ -108,6 +130,154 @@ func (input *relayVideoInput) pop() (*media.Sample, uint32) {
return sample, timestamp
}
+// Only a whole, marked AU is decoder input. The SDK retains sequence reordering;
+// neither a new timestamp nor a padding packet proves a complete picture.
+func (input *relayVideoInput) popH264() (*media.Sample, uint32) {
+ for {
+ packets := input.builder.PopPackets()
+ if len(packets) == 0 && input.pending && int32(input.latestTimestamp-input.pendingTimestamp) > videoClockRate {
+ input.discardH264(input.pendingTimestamp)
+ packets = input.builder.ForcePopPackets()
+ }
+ if len(packets) == 0 {
+ return nil, 0
+ }
+ // Every SDK partition is one ordered packet. Keep padding's sequence
+ // within an existing AU, but never use it as source time or a boundary.
+ packet := packets[0]
+ if len(packet.Payload) == 0 {
+ if input.h264AU == nil || input.h264AU.failed {
+ continue
+ }
+ } else {
+ if input.h264AU != nil && packet.Timestamp != input.pendingTimestamp {
+ if !input.h264AU.failed {
+ input.discardH264(input.pendingTimestamp)
+ }
+ input.h264AU = nil
+ }
+ if input.h264AU != nil && input.h264AU.failed {
+ continue
+ }
+ if input.h264AU == nil {
+ input.h264AU = &relayH264AU{}
+ input.pendingTimestamp = packet.Timestamp
+ }
+ input.pending = true
+ }
+ timestamp := input.pendingTimestamp
+ input.h264AU.bytes += len(packet.Payload)
+ if len(input.h264AU.packets) >= relayPacketLimit ||
+ input.h264AU.bytes > encoded.MaxAccessUnitBytes+2*relayPacketLimit {
+ input.discardH264(timestamp)
+ continue
+ }
+ input.h264AU.packets = append(input.h264AU.packets, packet)
+ if len(packet.Payload) == 0 || !packet.Marker {
+ continue
+ }
+ data, complete := relayH264AccessUnit(input.h264AU.packets)
+ if !complete {
+ input.discardH264(timestamp)
+ continue
+ }
+ input.h264AU, input.pending = nil, false
+ return &media.Sample{Data: data}, timestamp
+ }
+}
+
+func (input *relayVideoInput) discardH264(timestamp uint32) {
+ if input.h264AU != nil && input.h264AU.failed && input.pendingTimestamp == timestamp {
+ return
+ }
+ input.h264AU = &relayH264AU{failed: true}
+ input.pending, input.pendingTimestamp = true, timestamp
+ if input.onPacketDropped != nil {
+ input.onPacketDropped()
+ }
+}
+
+// Depacketize a complete ordered AU once; FU continuity spans padding and
+// belongs to the AU, independently of the SDK's packet partitions.
+func relayH264AccessUnit(packets []*rtp.Packet) ([]byte, bool) {
+ depacketizer := &codecs.H264Packet{}
+ var data []byte
+ fragmented := false
+ for _, packet := range packets {
+ payload := packet.Payload
+ if len(payload) == 0 {
+ continue
+ }
+ if payload[0]&0x1f == 28 {
+ if len(payload) < 2 {
+ return nil, false
+ }
+ if payload[1]&0x80 != 0 {
+ if fragmented {
+ return nil, false
+ }
+ fragmented = true
+ } else if !fragmented {
+ return nil, false
+ }
+ if payload[1]&0x40 != 0 {
+ fragmented = false
+ }
+ } else if fragmented {
+ return nil, false
+ }
+ decoded, err := depacketizer.Unmarshal(payload)
+ if err != nil || len(data)+len(decoded) > encoded.MaxAccessUnitBytes {
+ return nil, false
+ }
+ data = append(data, decoded...)
+ }
+ return data, !fragmented && len(data) > 0
+}
+
+// Retirement keeps complete configuration NALs only. Never decode expired
+// pictures, or let an abandoned FU contaminate the next independent NAL.
+func (input *relayVideoInput) cacheH264(packets []*rtp.Packet) error {
+ var depacketizer *codecs.H264Packet
+ var previous uint16
+ var timestamp uint32
+ bytes := 0
+ for _, packet := range packets {
+ if depacketizer != nil && packet.SequenceNumber != previous+1 {
+ depacketizer = nil
+ }
+ previous = packet.SequenceNumber
+ if len(packet.Payload) == 0 {
+ continue
+ }
+ if (&codecs.H264Packet{}).IsPartitionHead(packet.Payload) {
+ depacketizer = &codecs.H264Packet{}
+ timestamp, bytes = packet.Timestamp, 0
+ }
+ if depacketizer == nil || packet.Timestamp != timestamp {
+ depacketizer = nil
+ continue
+ }
+ bytes += len(packet.Payload)
+ if bytes > encoded.MaxAccessUnitBytes+2*relayPacketLimit {
+ depacketizer = nil
+ continue
+ }
+ data, err := depacketizer.Unmarshal(packet.Payload)
+ if err != nil {
+ depacketizer = nil
+ continue
+ }
+ if len(data) > 0 {
+ depacketizer = nil
+ if _, err = input.frame(data); err != nil {
+ return err
+ }
+ }
+ }
+ return nil
+}
+
func (input *relayVideoInput) frame(data []byte) (nativecapture.Frame, error) {
frame := nativecapture.Frame{Data: data}
if input.codec == "vp8" {
diff --git a/internal/app/mediaedge/relay_input_test.go b/internal/app/mediaedge/relay_input_test.go
index f573c887..30acfdc5 100644
--- a/internal/app/mediaedge/relay_input_test.go
+++ b/internal/app/mediaedge/relay_input_test.go
@@ -10,6 +10,7 @@ import (
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
"github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/livekit/mediatransportutil/pkg/utils"
"github.com/pion/ice/v4"
"github.com/pion/interceptor"
"github.com/pion/logging"
@@ -490,3 +491,180 @@ func TestRelayAssemblyRecoversAtNewIndependentFrame(t *testing.T) {
})
}
}
+
+func TestRelayInputRejectsTruncatedH264BeforeIndependentRecovery(t *testing.T) {
+ for _, leadingNAL := range []bool{false, true} {
+ t.Run(map[bool]string{false: "fragment-only", true: "complete-NAL-before-fragment"}[leadingNAL], func(t *testing.T) {
+ requests := 0
+ input := relayVideoInput{codec: "h264", onPacketDropped: func() { requests++ }}
+ payloader := &codecs.H264Payloader{}
+ var key []byte
+ for _, nalu := range sfu.H264KeyFrame2x2 {
+ key = append(key, 0, 0, 0, 1)
+ key = append(key, nalu...)
+ }
+ sequence := uint16(10)
+ push := func(timestamp uint32, marker bool, payload []byte) {
+ t.Helper()
+ if err := input.push(&rtp.Packet{Header: rtp.Header{Version: 2, SequenceNumber: sequence, Timestamp: timestamp, Marker: marker}, Payload: payload}); err != nil {
+ t.Fatal(err)
+ }
+ sequence++
+ }
+ payloads := payloader.Payload(1200, key)
+ for i, payload := range payloads {
+ push(0, i == len(payloads)-1, payload)
+ }
+ recovered := false
+ assembled := 0
+ for sample, _ := input.pop(); sample != nil; sample, _ = input.pop() {
+ assembled++
+ frame, err := input.frame(sample.Data)
+ if err != nil {
+ t.Fatal(err)
+ }
+ recovered = recovered || frame.KeyFrame
+ }
+ if !recovered {
+ t.Fatal("initial IDR did not establish recovery")
+ }
+ if assembled != 1 {
+ t.Fatal("same-timestamp SPS/PPS/IDR was split before its marker")
+ }
+ partial := []byte{0, 0, 0, 1}
+ if leadingNAL {
+ partial = append(partial, 0x61, 0x80, 0, 0, 0, 1)
+ }
+ partial = append(partial, 0x61)
+ partial = append(partial, bytes.Repeat([]byte{0x55}, 70)...)
+ payloads = payloader.Payload(16, partial)
+ if len(payloads) < 3 || payloads[len(payloads)-1][1]&0x40 == 0 {
+ t.Fatal("fixture is not a real fragmented NAL")
+ }
+ // Chrome can abandon a FU without reserving its unsent tail sequence.
+ for _, payload := range payloads[:len(payloads)-1] {
+ push(3000, false, payload)
+ if sample, _ := input.pop(); sample != nil {
+ t.Fatal("unmarked FU escaped during incremental input")
+ }
+ }
+ push(6000, true, []byte{0x61, 0x80})
+ for sample, timestamp := input.pop(); sample != nil; sample, timestamp = input.pop() {
+ if timestamp == 3000 {
+ t.Fatalf("truncated AU escaped: bytes=%d", len(sample.Data))
+ }
+ }
+ if requests == 0 {
+ t.Fatal("truncated AU did not request independent recovery")
+ }
+ // An IDR NAL in the same damaged AU is not independent recovery.
+ payloads = payloader.Payload(16, partial)
+ for _, payload := range payloads[:len(payloads)-1] {
+ push(7000, false, payload)
+ if sample, _ := input.pop(); sample != nil {
+ t.Fatal("unmarked FU escaped during incremental input")
+ }
+ }
+ push(7000, true, sfu.H264KeyFrame2x2IDR)
+ if sample, _ := input.pop(); sample != nil {
+ t.Fatal("same-timestamp IDR revived a truncated AU")
+ }
+
+ payloads = payloader.Payload(1200, key)
+ for i, payload := range payloads {
+ push(9000, i == len(payloads)-1, payload)
+ }
+ recovered = false
+ for sample, timestamp := input.pop(); sample != nil; sample, timestamp = input.pop() {
+ frame, err := input.frame(sample.Data)
+ if err != nil || timestamp != 9000 {
+ t.Fatalf("recovery lost original RTP time: %d, %v", timestamp, err)
+ }
+ recovered = recovered || frame.KeyFrame && frame.Width > 0 && frame.Height > 0
+ }
+ if !recovered {
+ t.Fatal("fresh IDR did not recover immediately")
+ }
+ push(12000, false, []byte{0x61, 0x81})
+ if sample, _ := input.pop(); sample != nil {
+ t.Fatal("multi-NAL AU escaped before its marker")
+ }
+ push(12000, true, []byte{0x61, 0x82})
+ sample, timestamp := input.pop()
+ if sample == nil || timestamp != 12000 ||
+ !bytes.Equal(sample.Data, []byte{0, 0, 0, 1, 0x61, 0x81, 0, 0, 0, 1, 0x61, 0x82}) {
+ t.Fatal("quiet multi-NAL frame did not resume on the original RTP timeline")
+ }
+
+ // Drain after every arrival, as run.write does. Padding occupies a
+ // sequence between real FU fragments but has no media timestamp.
+ requestsBefore := requests
+ completeNAL := append([]byte{0x61}, bytes.Repeat([]byte{0x55}, 70)...)
+ payloads = payloader.Payload(16, completeNAL)
+ packets := make([]*rtp.Packet, 0, len(payloads)+1)
+ for i, payload := range payloads {
+ packets = append(packets, &rtp.Packet{Header: rtp.Header{Version: 2,
+ SequenceNumber: sequence, Timestamp: 15000, Marker: i == len(payloads)-1}, Payload: payload})
+ sequence++
+ if i == 0 {
+ padding := &rtp.Packet{Header: rtp.Header{Version: 2, Padding: true,
+ SequenceNumber: sequence}, PaddingSize: 4}
+ wire, err := padding.Marshal()
+ if err != nil {
+ t.Fatal(err)
+ }
+ padding = &rtp.Packet{}
+ if err = padding.Unmarshal(wire); err != nil {
+ t.Fatal(err)
+ }
+ if len(padding.Payload) != 0 {
+ t.Fatal("fixture is not padding-only RTP")
+ }
+ packets = append(packets, padding)
+ sequence++
+ }
+ }
+ // The SDK must also keep a middle fragment behind its missing
+ // predecessor after earlier packets have already been consumed.
+ packets[2], packets[3] = packets[3], packets[2]
+ assembled = 0
+ for _, packet := range packets {
+ if err := input.push(packet); err != nil {
+ t.Fatal(err)
+ }
+ for sample, timestamp := input.pop(); sample != nil; sample, timestamp = input.pop() {
+ assembled++
+ if timestamp != 15000 || !bytes.Equal(sample.Data, append([]byte{0, 0, 0, 1}, completeNAL...)) {
+ t.Fatal("padding or reordering changed the complete FU access unit")
+ }
+ }
+ }
+ if assembled != 1 || requests != requestsBefore {
+ t.Fatalf("FU with padding/reordering: complete=%d recovery requests=%d", assembled, requests-requestsBefore)
+ }
+ })
+ }
+}
+
+func TestRelayFrameTimingUsesOneSourceClockAcrossWrap(t *testing.T) {
+ clock := utils.NewWrapAround[uint32, uint64](utils.WrapAroundParams{})
+ anchor := uint32(0xfffff000)
+ clock.Update(anchor)
+ for _, sample := range []struct {
+ ticks uint32
+ pts, duration time.Duration
+ }{
+ {3000, time.Second / 30, 0}, // First frame retains the caller's nominal fallback.
+ {6000, time.Second / 15, time.Second / 30},
+ {3000, time.Second / 30, 0}, // Reordering cannot create a negative interval.
+ {9000, 100 * time.Millisecond, time.Second / 30},
+ } {
+ pts, duration, ok := relayFrameTiming(clock, anchor+sample.ticks)
+ if !ok || pts != sample.pts || duration != sample.duration {
+ t.Fatalf("source ticks %d: pts=%v duration=%v accepted=%v", sample.ticks, pts, duration, ok)
+ }
+ }
+ if _, _, ok := relayFrameTiming(clock, anchor-1); ok {
+ t.Fatal("frame before the source anchor was accepted")
+ }
+}
diff --git a/internal/app/mediaedge/source.go b/internal/app/mediaedge/source.go
index 6e0b6f0e..6f8ced7d 100644
--- a/internal/app/mediaedge/source.go
+++ b/internal/app/mediaedge/source.go
@@ -41,6 +41,7 @@ type Source struct {
outputBitrates []uint32
capacity int
requestKeyFrame func([]int)
+ writeNACK func(*rtcp.TransportLayerNack)
relayProfile *nativecapture.VideoProfile
relay *relayDerivation
groups map[int]*outputGroup
diff --git a/internal/app/nativehost/input_transport_windows_test.go b/internal/app/nativehost/input_transport_windows_test.go
new file mode 100644
index 00000000..cbc98ec1
--- /dev/null
+++ b/internal/app/nativehost/input_transport_windows_test.go
@@ -0,0 +1,277 @@
+//go:build windows
+
+package nativehost
+
+import (
+ "context"
+ "crypto/aes"
+ "crypto/cipher"
+ "crypto/rand"
+ "encoding/base64"
+ "encoding/binary"
+ "encoding/hex"
+ "encoding/json"
+ "net"
+ "strings"
+ "sync/atomic"
+ "testing"
+ "time"
+
+ "github.com/TNTcraftHIM/Piik/internal/app/mediaedge"
+ "github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
+ "github.com/TNTcraftHIM/Piik/internal/app/remoteinput"
+ "github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/TNTcraftHIM/Piik/internal/media/inputroute"
+ "github.com/TNTcraftHIM/Piik/internal/server/protocol"
+ "github.com/pion/ice/v4"
+ "github.com/pion/webrtc/v4"
+)
+
+type inputLossSocket struct {
+ net.PacketConn
+ drop atomic.Bool
+}
+
+func (s *inputLossSocket) WriteTo(p []byte, addr net.Addr) (int, error) {
+ if s.drop.Load() {
+ return len(p), nil
+ }
+ return s.PacketConn.WriteTo(p, addr)
+}
+func (s *inputLossSocket) ReadFrom(p []byte) (int, net.Addr, error) {
+ for {
+ n, addr, err := s.PacketConn.ReadFrom(p)
+ if err != nil || !s.drop.Load() {
+ return n, addr, err
+ }
+ }
+}
+
+// Real ICE/DTLS/SCTP and the production permission owner. Approving control
+// does not inject input; no key, button or pointer event is sent by this fixture.
+func TestInputTransportLossRetiresOnlyItsPreviousApproval(t *testing.T) {
+ for _, kind := range []string{"edge", "publication"} {
+ t.Run(kind, func(t *testing.T) { checkInputTransportOwner(t, kind) })
+ }
+}
+func checkInputTransportOwner(t *testing.T, kind string) {
+ ctx, cancel := context.WithTimeout(t.Context(), 30*time.Second)
+ defer cancel()
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ wait := func(check func() bool, what string) {
+ t.Helper()
+ for !check() {
+ select {
+ case <-ctx.Done():
+ t.Fatal("timeout: " + what)
+ case <-time.After(10 * time.Millisecond):
+ }
+ }
+ }
+ engine, err := mediaedge.NewEngine(mediaedge.EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
+ check(err)
+ defer engine.Close()
+ source, err := engine.NewSource("vp8", 1, 1, nil)
+ check(err)
+ defer source.Close()
+ check(source.SetFormat(0, 320, 180))
+ check(source.ConfigureOutputs([]uint32{300000}))
+ manager, err := remoteinput.New(ctx, false)
+ check(err)
+ defer manager.Close()
+ input := manager.NewSession()
+ input.SetTarget(nativecapture.CaptureTarget{Kind: "display", SourceID: "1"}, nil)
+ policy, err := input.BeginPolicy(ctx, "ask", "", false)
+ check(err)
+ check(policy.Apply())
+ binding := protocol.RemoteInputPeer{PeerID: "test-viewer", EndpointID: strings.Repeat("a", 32), Secret: strings.Repeat("b", 64)}
+ check(input.SetPeers([]protocol.RemoteInputPeer{binding}))
+ events := make(chan Event, 64)
+ session := &Session{shareID: "input-loss", engine: engine, source: source, remoteInput: input, ctx: ctx, edges: map[string]*mediaedge.Edge{}, events: func(_ context.Context, event Event) { events <- event }}
+ defer session.CloseEdge("input-edge")
+ defer session.ClosePublication("input-generation", "input-edge")
+ packet, err := net.ListenPacket("udp4", "127.0.0.1:0")
+ check(err)
+ socket := &inputLossSocket{PacketConn: packet}
+ defer socket.Close()
+ mux := forwarding.NewUDPMux(ice.UDPMuxParams{UDPConn: socket})
+ defer mux.Close()
+ settings := webrtc.SettingEngine{}
+ settings.SetICEUDPMux(mux)
+ settings.SetIncludeLoopbackCandidate(true)
+ settings.SetICETimeouts(time.Second, 15*time.Second, 100*time.Millisecond)
+ viewer, err := webrtc.NewAPI(webrtc.WithSettingEngine(settings)).NewPeerConnection(webrtc.Configuration{})
+ check(err)
+ defer viewer.Close()
+ viewer.OnICECandidate(func(candidate *webrtc.ICECandidate) {
+ if candidate != nil {
+ value := candidate.ToJSON()
+ if kind == "publication" {
+ publication, e := session.Publication("input-generation", "input-edge")
+ if e == nil {
+ _ = publication.AddRemoteCandidate(&value)
+ }
+ } else {
+ _ = session.AddCandidate("input-edge", &value)
+ }
+ }
+ })
+ channel := make(chan *webrtc.DataChannel, 1)
+ viewer.OnDataChannel(func(dc *webrtc.DataChannel) { dc.OnOpen(func() { channel <- dc }) })
+ var offer webrtc.SessionDescription
+ var setAnswer func(webrtc.SessionDescription) error
+ var localState func() webrtc.PeerConnectionState
+ if kind == "publication" {
+ offer, _, err = session.PreparePublication("input-generation", "input-edge", nil)
+ check(err)
+ publication, e := session.Publication("input-generation", "input-edge")
+ check(e)
+ setAnswer = publication.SetAnswer
+ } else {
+ offer, err = session.PrepareEdge("input-edge", nil, false)
+ check(err)
+ localState = session.edge("input-edge").State
+ setAnswer = func(answer webrtc.SessionDescription) error { return session.SetAnswer("input-edge", answer, "") }
+ }
+ check(viewer.SetRemoteDescription(offer))
+ answer, err := viewer.CreateAnswer(nil)
+ check(err)
+ check(viewer.SetLocalDescription(answer))
+ check(setAnswer(answer))
+ // Native trickle candidates can precede the remote description, so drain them only now.
+ connected := false
+ for !connected {
+ select {
+ case event := <-events:
+ if event.Candidate != nil {
+ check(viewer.AddICECandidate(*event.Candidate))
+ }
+ connected = event.State == "connected"
+ case <-ctx.Done():
+ t.Fatal("edge never connected")
+ }
+ }
+ var dc *webrtc.DataChannel
+ select {
+ case dc = <-channel:
+ case <-ctx.Done():
+ t.Fatal("input channel did not open")
+ }
+ key, _ := hex.DecodeString(binding.Secret)
+ block, _ := aes.NewCipher(key)
+ aead, _ := cipher.NewGCM(block)
+ epoch := strings.Repeat("c", 32)
+ type inputReply struct {
+ Grant string `json:"grant"`
+ CoPlay *bool `json:"coPlay"`
+ }
+ replies := make(chan inputReply, 64)
+ dc.OnMessage(func(message webrtc.DataChannelMessage) {
+ var packet inputroute.Packet
+ if json.Unmarshal(message.Data, &packet) != nil {
+ return
+ }
+ data, e := base64.StdEncoding.DecodeString(packet.Data)
+ if e != nil || len(data) < 36 {
+ return
+ }
+ aad := append(append([]byte{1}, data[:8]...), []byte(packet.Epoch)...)
+ plain, e := aead.Open(nil, data[8:20], data[20:], aad)
+ if e != nil {
+ return
+ }
+ var reply inputReply
+ if json.Unmarshal(plain, &reply) == nil {
+ replies <- reply
+ }
+ })
+ var sequence uint64
+ send := func(plain []byte, epoch string) {
+ t.Helper()
+ sequence++
+ header := make([]byte, 20)
+ binary.BigEndian.PutUint64(header, sequence)
+ _, err := rand.Read(header[8:20])
+ check(err)
+ aad := append(append([]byte{0}, header[:8]...), []byte(epoch)...)
+ encrypted := aead.Seal(header, header[8:20], plain, aad)
+ data, _ := json.Marshal(inputroute.Packet{ID: binding.EndpointID, Epoch: epoch, Data: base64.StdEncoding.EncodeToString(encrypted)})
+ check(dc.SendText(string(data)))
+ }
+ send([]byte(`{"type":"request","intent":"dddddddddddddddddddddddddddddddd"}`), epoch)
+ wait(func() bool { return len(input.State().Requests) == 1 }, "pending approval")
+ oldRequest := input.State().Requests[0].ID
+
+ session.mu.Lock()
+ locked := true
+ defer func() {
+ if locked {
+ session.mu.Unlock()
+ }
+ }()
+ socket.drop.Store(true)
+ // Pion default disconnect timeout is five seconds. Keep the fixture outage
+ // beyond it while the Host lock delays asynchronous media projections.
+ select {
+ case <-time.After(9 * time.Second):
+ case <-ctx.Done():
+ t.Fatal("outage deadline")
+ }
+ if requests := input.State().Requests; len(requests) != 0 {
+ t.Errorf("transport loss kept its previous approval while Host projection was queued: %+v", requests)
+ }
+ socket.drop.Store(false)
+ wait(func() bool { return viewer.ConnectionState() == webrtc.PeerConnectionStateConnected }, "remote recovered while native callback delayed")
+ if localState != nil {
+ wait(func() bool { return localState() == webrtc.PeerConnectionStateConnected }, "native recovered while callback delayed")
+ }
+ freshEpoch := strings.Repeat("e", 32)
+ send([]byte(`{"type":"request","intent":"ffffffffffffffffffffffffffffffff"}`), freshEpoch)
+ wait(func() bool {
+ requests := input.State().Requests
+ return len(requests) == 1 && requests[0].ID != oldRequest
+ }, "new epoch request before native loss cleanup")
+ freshRequest := input.State().Requests[0]
+ check(input.Decide(freshRequest.ID, true))
+ session.mu.Unlock()
+ locked = false
+ for {
+ select {
+ case event := <-events:
+ if event.State == "disconnected" {
+ goto observed
+ }
+ case <-ctx.Done():
+ t.Fatal("missing loss callback")
+ }
+ }
+observed:
+ if requests := input.State().Requests; len(requests) != 1 {
+ t.Fatalf("delayed old loss revoked new post-recovery grant: %+v", requests)
+ }
+ // The new grant must remain usable over the same physical SCTP channel.
+ // A capability reply has an explicit coPlay field, unlike earlier replies.
+ send([]byte(`{"type":"capabilities"}`), freshEpoch)
+ for {
+ select {
+ case reply := <-replies:
+ if reply.CoPlay != nil && reply.Grant != "" {
+ goto acknowledged
+ }
+ case <-ctx.Done():
+ t.Fatal("new grant did not survive capability round trip")
+ }
+ }
+acknowledged:
+ if kind == "publication" {
+ session.ClosePublication("input-generation", "input-edge")
+ } else {
+ session.CloseEdge("input-edge")
+ }
+ wait(func() bool { return len(input.State().Requests) == 0 }, "closed transport permission release")
+}
diff --git a/internal/app/nativehost/publication.go b/internal/app/nativehost/publication.go
index 32a5ce3b..3833817f 100644
--- a/internal/app/nativehost/publication.go
+++ b/internal/app/nativehost/publication.go
@@ -36,9 +36,6 @@ func (session *Session) PreparePublication(generation, connectionID string, serv
PublicationGeneration: generation, ConnectionID: connectionID, Candidate: candidate})
},
ConnectionState: func(state webrtc.PeerConnectionState, _ *mediaedge.SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool) {
- if current() && isCurrentState(state) && (state == webrtc.PeerConnectionStateDisconnected || state == webrtc.PeerConnectionStateFailed || state == webrtc.PeerConnectionStateClosed) {
- session.remoteInput.RevokeEdge(connectionID)
- }
session.emit(Event{Type: "publication-state", ShareID: session.shareID,
Current: func() bool { return current() && isCurrentState(state) },
PublicationGeneration: generation, ConnectionID: connectionID, State: state.String()})
@@ -63,7 +60,7 @@ func (session *Session) PreparePublication(generation, connectionID string, serv
if session.remoteInput != nil {
channel, channelErr := publication.OpenDataChannel(remoteinput.ChannelLabel)
if channelErr == nil {
- session.remoteInput.Attach(connectionID, channel)
+ publication.OnInputTransportLost(session.remoteInput.Attach(connectionID, channel))
}
}
offer, err := publication.CreateOffer()
diff --git a/internal/app/nativehost/session.go b/internal/app/nativehost/session.go
index e7ce8502..906407a8 100644
--- a/internal/app/nativehost/session.go
+++ b/internal/app/nativehost/session.go
@@ -367,9 +367,6 @@ func (session *Session) prepareEdge(
})
},
ConnectionState: func(state webrtc.PeerConnectionState, pair *mediaedge.SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool) {
- if current() && isCurrentState(state) && (state == webrtc.PeerConnectionStateDisconnected || state == webrtc.PeerConnectionStateFailed || state == webrtc.PeerConnectionStateClosed) {
- session.remoteInput.RevokeEdge(connectionID)
- }
session.emit(Event{
Current: func() bool { return current() && isCurrentState(state) },
Type: "edge-state", ShareID: session.shareID,
@@ -404,7 +401,7 @@ func (session *Session) prepareEdge(
if session.remoteInput != nil {
channel, channelErr := edge.OpenDataChannel(remoteinput.ChannelLabel)
if channelErr == nil {
- session.remoteInput.Attach(connectionID, channel)
+ edge.OnInputTransportLost(session.remoteInput.Attach(connectionID, channel))
}
}
if localOfferID != "" {
diff --git a/internal/app/nativeviewer/input_bridge_test.go b/internal/app/nativeviewer/input_bridge_test.go
index f700bc33..59905aa0 100644
--- a/internal/app/nativeviewer/input_bridge_test.go
+++ b/internal/app/nativeviewer/input_bridge_test.go
@@ -89,6 +89,7 @@ func TestInputChannelUsesPlaybackAndRelay(t *testing.T) {
}
offer = localDescription(upstream, offer)
events := make(chan Event, 32)
+ observed := make(chan Event, 32)
session, err := Start(t.Context(), Options{ShareID: "input-viewer", EdgeCapacity: 2, Events: events})
if err != nil {
t.Fatal(err)
@@ -107,6 +108,8 @@ func TestInputChannelUsesPlaybackAndRelay(t *testing.T) {
case event := <-events:
if event.Type == "edge-candidate" && event.ConnectionID == "upstream" && event.Candidate != nil {
_ = upstream.AddICECandidate(*event.Candidate)
+ } else {
+ observed <- event
}
case <-t.Context().Done():
return
@@ -187,6 +190,9 @@ func TestInputChannelUsesPlaybackAndRelay(t *testing.T) {
case <-time.After(5 * time.Second):
t.Fatal("Native connecting callback was not delayed")
}
+ // ICE restart retires the selected pair and enters Checking directly. Input
+ // must release on that ordered event, before this queued media projection.
+ waitMessage(states, `{"ready":false}`)
waitMessage(requests, `{"ready":true}`)
if err = remote.SendText(`{"ready":true}`); err != nil {
t.Fatal(err)
@@ -197,12 +203,22 @@ func TestInputChannelUsesPlaybackAndRelay(t *testing.T) {
}
waitMessage(requests, packet)
resume()
- waitMessage(states, `{"ready":false}`)
- waitMessage(requests, `{"ready":true}`)
- if err = remote.SendText(`{"ready":true}`); err != nil {
- t.Fatal(err)
+ // Wait until the delayed real callback has delivered its projection. Input
+ // already recovered on the ordered transport observer; this older callback
+ // must not demand another handshake or retire the fresh return path.
+ completed := time.NewTimer(5 * time.Second)
+ defer completed.Stop()
+ for {
+ select {
+ case event := <-observed:
+ if event.ConnectionID == "upstream" && event.State == "connecting" {
+ goto resumed
+ }
+ case <-completed.C:
+ t.Fatal("delayed Native projection did not finish")
+ }
}
- waitMessage(states, `{"ready":true}`)
+resumed:
if err = local.SendText(packet); err != nil {
t.Fatal(err)
}
@@ -218,12 +234,6 @@ func TestInputChannelUsesPlaybackAndRelay(t *testing.T) {
if !strings.Contains(relay.SDP, "\nm=application ") {
t.Fatal("relay child has no input return path")
}
- session.receiver("upstream").input.LoseDown("playback", nil)
- waitMessage(requests, closed)
- if err = local.SendText(packet); err != nil {
- t.Fatal(err)
- }
- waitMessage(requests, packet)
session.CloseEdge("playback")
waitMessage(requests, closed)
}
diff --git a/internal/app/nativeviewer/session.go b/internal/app/nativeviewer/session.go
index 33384b8c..adfe6112 100644
--- a/internal/app/nativeviewer/session.go
+++ b/internal/app/nativeviewer/session.go
@@ -169,9 +169,11 @@ func (session *Session) AcceptOffer(
session.CloseReceiver(connectionID)
var input *inputroute.Relay
var onDataChannel func(*webrtc.DataChannel)
+ var inputTransportAvailable func(bool)
if strings.Contains(offer.SDP, "\nm=application ") {
input = inputroute.New(nil)
onDataChannel = input.AttachUp
+ inputTransportAvailable = input.SetUpAvailable
}
mediaReceiver, answer, err := session.engine.NewReceiver(mediaedge.ReceiverOptions{
Offer: offer,
@@ -180,7 +182,8 @@ func (session *Session) AcceptOffer(
Relay: session.relay,
PunchedSocket: punchedSocket,
Events: mediaedge.ReceiverEvents{
- DataChannel: onDataChannel,
+ DataChannel: onDataChannel,
+ InputTransportAvailable: inputTransportAvailable,
LocalCandidate: func(candidate *webrtc.ICECandidateInit, current func() bool) {
session.emit(Event{
Current: current,
@@ -189,16 +192,6 @@ func (session *Session) AcceptOffer(
})
},
ConnectionState: func(state webrtc.PeerConnectionState, pair *mediaedge.SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool) {
- if input != nil {
- // Historical loss still releases grants; a recovered transport
- // then requests fresh readiness rather than restoring permission.
- if state != webrtc.PeerConnectionStateConnected {
- input.SetUpAvailable(false)
- }
- if isCurrentState(webrtc.PeerConnectionStateConnected) {
- input.SetUpAvailable(true)
- }
- }
session.emit(Event{
Current: func() bool { return isCurrentState(state) },
Type: "edge-state", ShareID: session.shareID,
@@ -316,9 +309,6 @@ func (session *Session) prepareEdge(
})
},
ConnectionState: func(state webrtc.PeerConnectionState, pair *mediaedge.SelectedPair, isCurrentState func(webrtc.PeerConnectionState) bool) {
- if receiver.input != nil && (state == webrtc.PeerConnectionStateDisconnected || state == webrtc.PeerConnectionStateFailed || state == webrtc.PeerConnectionStateClosed) {
- receiver.input.LoseDown(connectionID, nil)
- }
session.emit(Event{
Current: func() bool { return isCurrentState(state) },
Type: "edge-state", ShareID: session.shareID,
@@ -358,7 +348,7 @@ func (session *Session) prepareEdge(
session.CloseEdge(connectionID)
return webrtc.SessionDescription{}, channelErr
}
- receiver.input.AttachDown(connectionID, channel)
+ edge.OnInputTransportLost(receiver.input.AttachDown(connectionID, channel))
}
if local {
offer, err = edge.CreateLocalOffer(session.ctx, localOfferID, false)
diff --git a/internal/app/remoteinput/held.go b/internal/app/remoteinput/held.go
index fcca2283..7bcba947 100644
--- a/internal/app/remoteinput/held.go
+++ b/internal/app/remoteinput/held.go
@@ -1,6 +1,9 @@
package remoteinput
-import "errors"
+import (
+ "errors"
+ "fmt"
+)
type controllerTextSink struct {
manager *Manager
@@ -69,13 +72,19 @@ func (m *Manager) releaseButton(button int) error {
return err
}
-func (m *Manager) retryReleases() {
+func (m *Manager) retryReleases() error {
+ var result error
for code := range m.pendingKeys {
- _ = m.releaseKey(code)
+ if err := m.releaseKey(code); err != nil {
+ result = errors.Join(result, fmt.Errorf("key release retry: %w", err))
+ }
}
for button := range m.pendingButtons {
- _ = m.releaseButton(button)
+ if err := m.releaseButton(button); err != nil {
+ result = errors.Join(result, fmt.Errorf("button release retry: %w", err))
+ }
}
+ return result
}
func (m *Manager) noteInputResult(c *connection, err error) {
@@ -87,10 +96,10 @@ func (m *Manager) noteInputResult(c *connection, err error) {
}
func (m *Manager) applyHeld(c *connection, b bounds, event inputEvent) error {
- m.retryReleases()
+ releaseErr := m.retryReleases()
releasing := (event.Type == "key" || event.Type == "button") && !event.Down
if !releasing && (len(m.pendingKeys) != 0 || len(m.pendingButtons) != 0) {
- return errors.New("held input cleanup is unavailable")
+ return fmt.Errorf("held input cleanup is unavailable (%d key, %d button releases pending): %w", len(m.pendingKeys), len(m.pendingButtons), releaseErr)
}
switch event.Type {
case "key":
diff --git a/internal/app/remoteinput/held_test.go b/internal/app/remoteinput/held_test.go
index 81db8c7a..5d407610 100644
--- a/internal/app/remoteinput/held_test.go
+++ b/internal/app/remoteinput/held_test.go
@@ -2,6 +2,7 @@ package remoteinput
import (
"errors"
+ "strings"
"testing"
)
@@ -74,7 +75,9 @@ func TestRejectedInputKeepsGrantAndDoesNotReplay(t *testing.T) {
if c.grant != grant || !m.pendingKeys["KeyB"] || c.keys["KeyB"] {
t.Fatal("failed release lost cleanup debt or grant")
}
- sendTestInput(m, c, inputEvent{Type: "move", X: .7, Y: .4})
+ if err := m.applyHeld(c, bounds{}, inputEvent{Type: "move", X: .7, Y: .4}); !errors.Is(err, sink.applyErr) || !strings.Contains(err.Error(), "key release retry") {
+ t.Fatalf("cleanup hid the rejected release: %v", err)
+ }
if len(sink.events) != 0 {
t.Fatal("new action ran before cleanup")
}
@@ -116,6 +119,9 @@ func TestPlayerCleanupFailurePreservesSiblingGrant(t *testing.T) {
if b.grant != grant || m.owner != s {
t.Fatal("disconnect cleanup revoked sibling")
}
+ if err := m.applyHeld(b, bounds{}, inputEvent{Type: "move", X: .5, Y: .5}); !errors.Is(err, sink.applyErr) || !strings.Contains(err.Error(), "button release retry") {
+ t.Fatalf("shared cleanup hid the rejected button release: %v", err)
+ }
sink.applyErr = nil
m.retryReleases()
if event := <-sink.events; event.Type != "button" || event.Down {
diff --git a/internal/app/remoteinput/manager.go b/internal/app/remoteinput/manager.go
index 6a7de321..231f8784 100644
--- a/internal/app/remoteinput/manager.go
+++ b/internal/app/remoteinput/manager.go
@@ -241,8 +241,18 @@ func (s *Session) setTarget(target nativecapture.CaptureTarget, capture *nativec
// The Linux Portal owns input and screen selection in the same OS session.
func (s *Session) UsesCaptureInput() bool { return s != nil && runtime.GOOS == "linux" }
-func (s *Session) Attach(edgeID string, channel *webrtc.DataChannel) {
+// Attach returns the transport-loss action for this exact input channel. It
+// invalidates queued deliveries and grants together, retaining SCTP for recovery.
+func (s *Session) Attach(edgeID string, channel *webrtc.DataChannel) func() {
s.route.AttachDown(edgeID, channel)
+ return func() {
+ m := s.manager
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ if !s.closed && !m.closed && s.route.ResetDown(edgeID, channel) {
+ s.revokeEdgeLocked(edgeID)
+ }
+ }
}
// Bindings come only from the owning Host's authenticated room connection.
@@ -290,25 +300,28 @@ func (s *Session) SetPeers(peers []protocol.RemoteInputPeer) error {
return nil
}
-func (s *Session) receivePacket(edgeID string, packet inputroute.Packet) {
+func (s *Session) receivePacket(edgeID string, packet inputroute.Packet, current func() bool) {
m := s.manager
m.mu.Lock()
+ defer m.mu.Unlock()
c := m.connections[endpointKey{s, packet.ID}]
if c == nil || m.closed || s.closed {
- m.mu.Unlock()
return
}
+ // Retirement may already have removed its relay record before the channel
+ // closes. It still ends the matching epoch; only new input needs a live binding.
if packet.Closed {
if c.edgeID == edgeID && c.epoch == packet.Epoch {
m.revoke(c)
c.edgeID = ""
}
- m.mu.Unlock()
+ return
+ }
+ if current != nil && !current() {
return
}
plain, err := c.cipher.open(packet.Data, packet.Epoch)
if err != nil {
- m.mu.Unlock()
return
}
if c.edgeID != "" && (c.edgeID != edgeID || c.epoch != packet.Epoch) {
@@ -319,8 +332,7 @@ func (s *Session) receivePacket(edgeID string, packet inputroute.Packet) {
c.inputFeedback = false
}
c.edgeID, c.epoch = edgeID, packet.Epoch
- m.mu.Unlock()
- m.receive(c, webrtc.DataChannelMessage{IsString: true, Data: plain}, packet.Epoch)
+ m.receiveLocked(c, plain)
}
func (m *Manager) send(c *connection, state string) {
@@ -532,21 +544,17 @@ func (m *Manager) releaseConnection(c *connection) {
m.owner = nil
}
-func (m *Manager) receive(c *connection, message webrtc.DataChannelMessage, epoch string) {
- if !message.IsString || len(message.Data) > maxInputBytes {
+// Caller holds mu from delivery validation through the input action.
+func (m *Manager) receiveLocked(c *connection, plain []byte) {
+ if len(plain) > maxInputBytes {
return
}
var event inputEvent
- decoder := json.NewDecoder(bytes.NewReader(message.Data))
+ decoder := json.NewDecoder(bytes.NewReader(plain))
decoder.DisallowUnknownFields()
if decoder.Decode(&event) != nil || decoder.Decode(&struct{}{}) != io.EOF {
return
}
- m.mu.Lock()
- defer m.mu.Unlock()
- if m.closed || c.owner.closed || m.connections[endpointKey{c.owner, c.id}] != c || c.epoch != epoch {
- return
- }
switch event.Type {
case "sync":
m.send(c, m.state(c))
@@ -620,7 +628,8 @@ func (m *Manager) receive(c *connection, message webrtc.DataChannelMessage, epoc
err = m.applyHeld(c, b, event)
}
if err != nil {
- slog.Debug("remote-input", "event", "apply-failed", "connectionId", diagnostics.ID(c.edgeID), "inputGeneration", diagnostics.ID(c.epoch), diagnostics.Error(err))
+ slog.Debug("remote-input", "event", "apply-failed", "connectionId", diagnostics.ID(c.edgeID), "inputGeneration", diagnostics.ID(c.epoch),
+ "operation", event.Type, "sourceKind", c.owner.target.Kind, "targetReady", focused && b.width > 0 && b.height > 0, diagnostics.Error(err))
}
m.noteInputResult(c, err)
}
@@ -658,15 +667,8 @@ func (s *Session) Pause(paused bool) {
s.notify()
}
-// Transport loss retires permission, not the media-owned channel. A recovered
-// ICE connection can carry a new request, but never restore the previous grant.
-func (s *Session) RevokeEdge(edgeID string) {
- if s == nil {
- return
- }
+func (s *Session) revokeEdgeLocked(edgeID string) {
m := s.manager
- m.mu.Lock()
- defer m.mu.Unlock()
for _, c := range m.connections {
if c.owner == s && c.edgeID == edgeID {
m.revoke(c)
diff --git a/internal/app/remoteinput/manager_test.go b/internal/app/remoteinput/manager_test.go
index be967154..ae040386 100644
--- a/internal/app/remoteinput/manager_test.go
+++ b/internal/app/remoteinput/manager_test.go
@@ -34,6 +34,18 @@ type recordedInput struct {
resetErr error
}
+func receiveTestInput(m *Manager, c *connection, plain []byte) {
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ m.receiveLocked(c, plain)
+}
+
+func revokeTestEdge(s *Session, edgeID string) {
+ s.manager.mu.Lock()
+ defer s.manager.mu.Unlock()
+ s.revokeEdgeLocked(edgeID)
+}
+
func setTestPolicy(s *Session, mode, password string) error {
change, err := s.BeginPolicy(context.Background(), mode, password, s.coPlay)
if err != nil {
@@ -63,7 +75,11 @@ func TestSourceAuthorizationPolicy(t *testing.T) {
request := func(c *connection, password string) {
t.Helper()
data, _ := json.Marshal(inputEvent{Type: "request", Intent: strings.Repeat("f", 32), Password: password})
- m.receive(c, webrtc.DataChannelMessage{IsString: true, Data: data}, c.epoch)
+ header := make([]byte, 20)
+ binary.BigEndian.PutUint64(header, c.cipher.received+1)
+ _, _ = rand.Read(header[8:])
+ data = c.cipher.aead.Seal(header, header[8:], data, inputAAD(0, header[:8], c.epoch))
+ s.receivePacket(c.edgeID, inputroute.Packet{ID: c.id, Epoch: c.epoch, Data: base64.StdEncoding.EncodeToString(data)}, nil)
}
request(first, "")
if s.State().Mode != "off" || first.requested || m.owner != nil {
@@ -128,7 +144,7 @@ func TestSourceAuthorizationPolicy(t *testing.T) {
t.Fatal(err)
}
request(first, "")
- s.RevokeEdge("root-edge")
+ revokeTestEdge(s, "root-edge")
if m.owner != nil {
t.Fatal("route loss retained input")
}
@@ -142,7 +158,7 @@ func TestSourceAuthorizationPolicy(t *testing.T) {
t.Fatal(err)
}
s.Pause(false)
- m.receive(first, webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"key","code":"KeyA","down":true,"grant":"` + oldGrant + `"}`)}, first.epoch)
+ receiveTestInput(m, first, []byte(`{"type":"key","code":"KeyA","down":true,"grant":"`+oldGrant+`"}`))
request(second, "")
if m.owner != nil || second.requested || len(sink.events) != 0 {
t.Fatal("disabled remote control retained input or accepted a new request")
@@ -170,7 +186,7 @@ func TestSourceAuthorizationPolicy(t *testing.T) {
if m.owner != nil || first.grant != "" || m.connections[endpointKey{s, first.id}] != nil {
t.Fatal("room membership removal retained the active input grant")
}
- m.receive(first, webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"key","code":"KeyA","down":true,"grant":"` + revokedGrant + `"}`)}, first.epoch)
+ receiveTestInput(m, first, []byte(`{"type":"key","code":"KeyA","down":true,"grant":"`+revokedGrant+`"}`))
request(first, "")
if m.owner != nil || first.requested || len(sink.events) != 0 {
t.Fatal("removed endpoint regained input with its old grant or key")
@@ -203,7 +219,7 @@ func TestRouteReplacementRequiresApprovalAndIgnoresOldClosure(t *testing.T) {
_, _ = rand.Read(header[8:])
plain := []byte(`{"type":"request","intent":"` + epoch + `"}`)
data := cipher.aead.Seal(header, header[8:], plain, inputAAD(0, header[:8], epoch))
- s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: epoch, Data: base64.StdEncoding.EncodeToString(data)})
+ s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: epoch, Data: base64.StdEncoding.EncodeToString(data)}, nil)
views := s.State().Requests
if len(views) != 1 || views[0].State != "pending" {
t.Fatal("new channel did not require fresh approval")
@@ -219,12 +235,11 @@ func TestRouteReplacementRequiresApprovalAndIgnoresOldClosure(t *testing.T) {
if oldGrant == newGrant || len(sink.resets) != 1 {
t.Fatal("replacement retained old input")
}
- s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: oldEpoch, Closed: true})
- m.receive(m.connections[endpointKey{s, peer.EndpointID}], webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"release","intent":"` + newEpoch + `"}`)}, oldEpoch)
+ s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: oldEpoch, Closed: true}, nil)
if m.owner == nil || m.connections[endpointKey{s, peer.EndpointID}].grant != newGrant {
t.Fatal("old descendant closure retired the new grant")
}
- s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: newEpoch, Closed: true})
+ s.receivePacket("same-root-relay", inputroute.Packet{ID: peer.EndpointID, Epoch: newEpoch, Closed: true}, nil)
if m.owner != nil || len(sink.resets) != 2 {
t.Fatal("current route loss did not release input")
}
@@ -369,6 +384,14 @@ func TestInputFailureDiagnosticsPreserveExecutionAuthority(t *testing.T) {
if record["event"] != stage+"-failed" || record["connectionId"] != diagnostics.ID(c.edgeID) || record["inputGeneration"] != diagnostics.ID(c.epoch) {
t.Fatalf("failure stage or generation missing: %v", record)
}
+ if stage == "apply" {
+ if record["operation"] != "text" || record["sourceKind"] != c.owner.target.Kind {
+ t.Fatal("failure lost its operation or target kind")
+ }
+ if _, ok := record["targetReady"].(bool); !ok {
+ t.Fatal("failure lost its target observation")
+ }
+ }
for _, private := range []string{grant, c.intent, c.edgeID, c.epoch, "KeyA", "private committed text"} {
if strings.Contains(logs.String(), private) {
t.Fatal("failure diagnostic recorded private input or authority")
@@ -572,7 +595,7 @@ func TestCoPlayGamepadGrantIsolationAndRetirement(t *testing.T) {
}
sendTestInput(m, keyboard, inputEvent{Type: "key", Code: "KeyB", Down: true})
<-sink.events
- s.receivePacket(a.edgeID, inputroute.Packet{ID: a.id, Epoch: a.epoch, Closed: true})
+ s.receivePacket(a.edgeID, inputroute.Packet{ID: a.id, Epoch: a.epoch, Closed: true}, nil)
if !first.closed || a.grant != "" || b.grant == "" || !keyboard.keys["KeyB"] {
t.Fatal("one route loss affected another participant")
}
@@ -848,7 +871,7 @@ func sendTestInput(m *Manager, c *connection, event inputEvent) {
event.Grant = c.grant
}
data, _ := json.Marshal(event)
- m.receive(c, webrtc.DataChannelMessage{IsString: true, Data: data}, c.epoch)
+ receiveTestInput(m, c, data)
if event.Type == "gamepad" {
// Unit reports model the next periodic full report after lazy creation;
// production deliberately drops the report that started enumeration.
@@ -865,7 +888,7 @@ func sendTestInput(m *Manager, c *connection, event inputEvent) {
}
time.Sleep(time.Millisecond)
}
- m.receive(c, webrtc.DataChannelMessage{IsString: true, Data: data}, c.epoch)
+ receiveTestInput(m, c, data)
}
}
@@ -1092,7 +1115,7 @@ func TestCoPlayRetirementAndLegacyAdmission(t *testing.T) {
sendTestInput(m, c, inputEvent{Type: "key", Code: "KeyA", Down: true})
}
<-sink.events
- s.RevokeEdge(a.edgeID)
+ revokeTestEdge(s, a.edgeID)
if a.grant != "" || b.grant == "" || len(sink.events) != 0 || m.owner != s {
t.Fatal("one route loss released another player's held key")
}
@@ -1201,12 +1224,13 @@ func TestAttendedGrantLifecycle(t *testing.T) {
})
c.OnOpen(func() { channels <- c })
})
+ losses := map[string]func(){}
for _, id := range []string{"first", "second"} {
c, err := host.CreateDataChannel(id, nil)
if err != nil {
t.Fatal(err)
}
- s.Attach(id, c)
+ losses[id] = s.Attach(id, c)
}
offer, err := host.CreateOffer(nil)
if err != nil {
@@ -1305,7 +1329,7 @@ func TestAttendedGrantLifecycle(t *testing.T) {
return c.grant
}
key := func(c *connection, grant string) {
- m.receive(c, webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"key","code":"KeyA","down":true,"grant":"` + grant + `"}`)}, c.epoch)
+ receiveTestInput(m, c, []byte(`{"type":"key","code":"KeyA","down":true,"grant":"`+grant+`"}`))
}
<-s.Changes() // Drain display setup; the next notification must be a request.
first := request("first")
@@ -1374,7 +1398,7 @@ func TestAttendedGrantLifecycle(t *testing.T) {
waitState("ready")
second = request("second")
grant2 = decide(second, true)
- s.RevokeEdge("second")
+ losses["second"]()
key(second, grant2)
if len(sink.events) != 2 || len(sink.resets) != 3 {
t.Fatal("disconnect kept a grant")
@@ -1383,7 +1407,7 @@ func TestAttendedGrantLifecycle(t *testing.T) {
// without reviving the old grant or a pending approval from before loss.
second = request("second")
staleRequest := second.requestID
- s.RevokeEdge("second")
+ losses["second"]()
if s.Decide(staleRequest, true) == nil || len(s.State().Requests) != 0 {
t.Fatal("transport loss kept a pending approval")
}
@@ -1415,13 +1439,13 @@ func TestAttendedGrantLifecycle(t *testing.T) {
}
first = request("first")
oldRequest, oldIntent := first.requestID, first.intent
- m.receive(first, webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"release","intent":"` + oldIntent + `"}`)}, first.epoch)
+ receiveTestInput(m, first, []byte(`{"type":"release","intent":"`+oldIntent+`"}`))
first = request("first")
if s.Decide(oldRequest, true) == nil {
t.Fatal("late approval authorized a replacement request")
}
nextGrant := decide(first, true)
- m.receive(first, webrtc.DataChannelMessage{IsString: true, Data: []byte(`{"type":"release","intent":"` + oldIntent + `"}`)}, first.epoch)
+ receiveTestInput(m, first, []byte(`{"type":"release","intent":"`+oldIntent+`"}`))
key(first, grant)
key(first, nextGrant)
if len(sink.events) != 3 || nextGrant == grant || sink.target != nextDisplay {
@@ -1525,3 +1549,310 @@ func TestAttendedGrantLifecycle(t *testing.T) {
t.Fatal("source retirement kept a grant")
}
}
+
+func TestLateOldEpochCannotHideRelayedGrantFromBranchLoss(t *testing.T) {
+ sink := &recordedInput{events: make(chan inputEvent, 8), resets: make(chan struct{}, 8), focused: true}
+ m := &Manager{sink: sink, connections: map[endpointKey]*connection{}}
+ s := m.NewSession()
+ t.Cleanup(s.Close)
+ s.SetTarget(nativecapture.CaptureTarget{Kind: "display", SourceID: "1"}, nil)
+ if err := setTestPolicy(s, "ask", ""); err != nil {
+ t.Fatal(err)
+ }
+ peer := protocol.RemoteInputPeer{PeerID: "viewer-route-test", EndpointID: strings.Repeat("a", 32), Secret: strings.Repeat("b", 64)}
+ if err := s.SetPeers([]protocol.RemoteInputPeer{peer}); err != nil {
+ t.Fatal(err)
+ }
+ observed := make(chan inputroute.Packet, 10)
+ s.route.Close()
+ s.route = inputroute.New(func(edge string, p inputroute.Packet, current func() bool) {
+ s.receivePacket(edge, p, current)
+ observed <- p
+ })
+ rootChannel, relayUp := inputChannelPair(t)
+ s.Attach("sfu-root", rootChannel)
+ relay := inputroute.New(nil)
+ t.Cleanup(relay.Close)
+ relay.AttachUp(relayUp)
+ viewer, branch := inputChannelPair(t)
+ ready := make(chan struct{}, 10)
+ viewer.OnMessage(func(message webrtc.DataChannelMessage) {
+ var p inputroute.Packet
+ if json.Unmarshal(message.Data, &p) == nil && p.Ready != nil && *p.Ready {
+ ready <- struct{}{}
+ }
+ })
+ loseBranch := relay.AttachDown("shared-branch", branch)
+ relay.SetUpAvailable(true)
+ select {
+ case <-ready:
+ case <-time.After(3 * time.Second):
+ t.Fatal("relay not ready")
+ }
+ cipher, _ := newInputCipher(peer.Secret)
+ encode := func(seq uint64, epoch string, event inputEvent) inputroute.Packet {
+ header := make([]byte, 20)
+ binary.BigEndian.PutUint64(header, seq)
+ _, _ = rand.Read(header[8:])
+ plain, _ := json.Marshal(event)
+ data := cipher.aead.Seal(header, header[8:], plain, inputAAD(0, header[:8], epoch))
+ return inputroute.Packet{ID: peer.EndpointID, Epoch: epoch, Data: base64.StdEncoding.EncodeToString(data)}
+ }
+ sendPacket := func(p inputroute.Packet) {
+ t.Helper()
+ data, _ := json.Marshal(p)
+ if err := viewer.SendText(string(data)); err != nil {
+ t.Fatal(err)
+ }
+ deadline := time.After(3 * time.Second)
+ for {
+ select {
+ case got := <-observed:
+ if got == p {
+ return
+ }
+ case <-deadline:
+ t.Fatal("packet did not reach actual manager")
+ }
+ }
+ }
+ approve := func() string {
+ t.Helper()
+ views := s.State().Requests
+ if len(views) != 1 || views[0].State != "pending" {
+ t.Fatalf("request state: %+v", views)
+ }
+ if err := s.Decide(views[0].ID, true); err != nil {
+ t.Fatal(err)
+ }
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ return m.connections[endpointKey{s, peer.EndpointID}].grant
+ }
+ oldEpoch, newEpoch := strings.Repeat("c", 32), strings.Repeat("d", 32)
+ sendPacket(encode(1, oldEpoch, inputEvent{Type: "request", Intent: oldEpoch}))
+ oldGrant := approve()
+ // Packet 2 was sent on the old longer branch before the route migrated.
+ // The new shorter path overtakes it. Its lower sequence must be rejected.
+ late := encode(2, oldEpoch, inputEvent{Type: "key", Code: "KeyA", Down: true, Grant: oldGrant})
+ sendPacket(encode(3, newEpoch, inputEvent{Type: "request", Intent: newEpoch}))
+ newGrant := approve()
+ sendPacket(encode(4, newEpoch, inputEvent{Type: "key", Code: "KeyW", Down: true, Grant: newGrant}))
+ sendPacket(late)
+ m.mu.Lock()
+ current := m.connections[endpointKey{s, peer.EndpointID}]
+ if current.grant != newGrant || current.keys["KeyA"] || !current.keys["KeyW"] {
+ m.mu.Unlock()
+ t.Fatal("late old packet changed authority")
+ }
+ m.mu.Unlock()
+ // The shared upstream branch then disconnects while the root SFU leg stays up.
+ loseBranch()
+ // On recovery, a sync is ordered behind all branch retirement packets. It
+ // may report current availability but cannot restore the previous grant.
+ sendPacket(encode(5, newEpoch, inputEvent{Type: "sync"}))
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ if current.grant != "" || current.keys["KeyW"] {
+ t.Fatal("branch loss retained current grant and held key after a late old epoch")
+ }
+}
+
+func waitInputChannel(t *testing.T, label string, ready func() bool) {
+ t.Helper()
+ deadline := time.Now().Add(5 * time.Second)
+ for time.Now().Before(deadline) {
+ if ready() {
+ return
+ }
+ time.Sleep(5 * time.Millisecond)
+ }
+ t.Fatal(label)
+}
+
+func inputChannelPair(t *testing.T) (*webrtc.DataChannel, *webrtc.DataChannel) {
+ t.Helper()
+ settings := webrtc.SettingEngine{}
+ settings.SetIncludeLoopbackCandidate(true)
+ settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
+ api := webrtc.NewAPI(webrtc.WithSettingEngine(settings))
+ a, err := api.NewPeerConnection(webrtc.Configuration{})
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = a.Close() })
+ b, err := api.NewPeerConnection(webrtc.Configuration{})
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = b.Close() })
+ remote := make(chan *webrtc.DataChannel, 1)
+ b.OnDataChannel(func(c *webrtc.DataChannel) { remote <- c })
+ local, err := a.CreateDataChannel(inputroute.Label, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ offer, err := a.CreateOffer(nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ gather := webrtc.GatheringCompletePromise(a)
+ if err = a.SetLocalDescription(offer); err != nil {
+ t.Fatal(err)
+ }
+ select {
+ case <-gather:
+ case <-time.After(5 * time.Second):
+ t.Fatal("offer gather")
+ }
+ if err = b.SetRemoteDescription(*a.LocalDescription()); err != nil {
+ t.Fatal(err)
+ }
+ answer, err := b.CreateAnswer(nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ gather = webrtc.GatheringCompletePromise(b)
+ if err = b.SetLocalDescription(answer); err != nil {
+ t.Fatal(err)
+ }
+ select {
+ case <-gather:
+ case <-time.After(5 * time.Second):
+ t.Fatal("answer gather")
+ }
+ if err = a.SetRemoteDescription(*b.LocalDescription()); err != nil {
+ t.Fatal(err)
+ }
+ var other *webrtc.DataChannel
+ select {
+ case other = <-remote:
+ case <-time.After(5 * time.Second):
+ t.Fatal("remote data channel")
+ }
+ waitInputChannel(t, "SCTP opens", func() bool {
+ return local.ReadyState() == webrtc.DataChannelStateOpen && other.ReadyState() == webrtc.DataChannelStateOpen
+ })
+ return local, other
+}
+
+func TestInputRetirementOrdersAlreadyReadPackets(t *testing.T) {
+ for _, retired := range []string{"channel", "ice", "explicit-close"} {
+ t.Run(retired, func(t *testing.T) {
+ sink := &recordedInput{events: make(chan inputEvent, 8), resets: make(chan struct{}, 8), focused: true}
+ m := &Manager{sink: sink, connections: map[endpointKey]*connection{}}
+ s := m.NewSession()
+ t.Cleanup(s.Close)
+ s.SetTarget(nativecapture.CaptureTarget{Kind: "display", SourceID: "1"}, nil)
+ if err := setTestPolicy(s, "room", ""); err != nil {
+ t.Fatal(err)
+ }
+ peer := protocol.RemoteInputPeer{PeerID: "viewer-route-test", EndpointID: strings.Repeat("a", 32), Secret: strings.Repeat("b", 64)}
+ if err := s.SetPeers([]protocol.RemoteInputPeer{peer}); err != nil {
+ t.Fatal(err)
+ }
+ var arm atomic.Bool
+ entered, resume := make(chan struct{}), make(chan struct{})
+ resumed := false
+ defer func() {
+ if !resumed {
+ close(resume)
+ }
+ }()
+ observed := make(chan inputroute.Packet, 8)
+ s.route.Close()
+ s.route = inputroute.New(func(edge string, p inputroute.Packet, current func() bool) {
+ // This is the production handoff after Relay unlocks. Retirement
+ // must invalidate the captured delivery before Manager accepts it.
+ if (p.Data != "" || p.Closed) && arm.Swap(false) {
+ close(entered)
+ <-resume
+ }
+ s.receivePacket(edge, p, current)
+ observed <- p
+ })
+ viewer, root := inputChannelPair(t)
+ loseInput := s.Attach("root-edge", root)
+ cipher, _ := newInputCipher(peer.Secret)
+ epoch := strings.Repeat("c", 32)
+ send := func(seq uint64, event inputEvent) {
+ t.Helper()
+ header := make([]byte, 20)
+ binary.BigEndian.PutUint64(header, seq)
+ _, _ = rand.Read(header[8:])
+ plain, _ := json.Marshal(event)
+ data := cipher.aead.Seal(header, header[8:], plain, inputAAD(0, header[:8], epoch))
+ packet, _ := json.Marshal(inputroute.Packet{ID: peer.EndpointID, Epoch: epoch, Data: base64.StdEncoding.EncodeToString(data)})
+ if err := viewer.SendText(string(packet)); err != nil {
+ t.Fatal(err)
+ }
+ }
+ waitData := func() {
+ t.Helper()
+ deadline := time.After(3 * time.Second)
+ for {
+ select {
+ case p := <-observed:
+ if p.Data != "" {
+ return
+ }
+ case <-deadline:
+ t.Fatal("input did not reach Manager")
+ }
+ }
+ }
+ initial := inputEvent{Type: "sync"}
+ if retired == "explicit-close" {
+ initial = inputEvent{Type: "request", Intent: strings.Repeat("d", 32)}
+ }
+ send(1, initial)
+ waitData()
+ arm.Store(true)
+ if retired == "explicit-close" {
+ if len(s.State().Requests) != 1 {
+ t.Fatal("request was not approved")
+ }
+ data, _ := json.Marshal(inputroute.Packet{ID: peer.EndpointID, Epoch: epoch, Closed: true})
+ if err := viewer.SendText(string(data)); err != nil {
+ t.Fatal(err)
+ }
+ } else {
+ send(2, inputEvent{Type: "request", Intent: strings.Repeat("d", 32)})
+ }
+ select {
+ case <-entered:
+ case <-time.After(3 * time.Second):
+ t.Fatal("request was not read")
+ }
+ if retired != "ice" {
+ if err := root.Close(); err != nil {
+ t.Fatal(err)
+ }
+ } else {
+ loseInput()
+ }
+ close(resume)
+ resumed = true
+ if retired == "explicit-close" {
+ select {
+ case <-observed:
+ case <-time.After(3 * time.Second):
+ t.Fatal("explicit retirement did not reach Manager")
+ }
+ } else {
+ waitData()
+ }
+ if len(s.State().Requests) != 0 {
+ t.Fatal("already-read packet retained or recreated authorization after retirement")
+ }
+ if retired == "ice" {
+ send(3, inputEvent{Type: "request", Intent: strings.Repeat("e", 32)})
+ waitData()
+ views := s.State().Requests
+ if len(views) != 1 || views[0].State != "active" {
+ t.Fatalf("fresh request on recovered channel: %+v", views)
+ }
+ }
+ })
+ }
+}
diff --git a/internal/app/remoteinput/platform_windows.go b/internal/app/remoteinput/platform_windows.go
index e307ab91..60fcf3f8 100644
--- a/internal/app/remoteinput/platform_windows.go
+++ b/internal/app/remoteinput/platform_windows.go
@@ -8,6 +8,7 @@ import (
"fmt"
"math"
"runtime"
+ "syscall"
"unicode/utf16"
"unsafe"
@@ -33,6 +34,7 @@ type winInput struct {
Mouse mouseInput
}
type windowsSink struct {
+ desktop string
keys map[uint16]bool
buttons map[int]bool
}
@@ -44,20 +46,150 @@ func newPlatformSink(bool) (inputSink, error) {
// The capture helper reports physical pixels. Match that coordinate space
// before reading monitor bounds in this Go process (including mixed DPI).
user32.NewProc("SetProcessDpiAwarenessContext").Call(^uintptr(3))
- return &windowsSink{keys: map[uint16]bool{}, buttons: map[int]bool{}}, nil
+ runtime.LockOSThread()
+ desktop, _, _ := user32.NewProc("GetThreadDesktop").Call(uintptr(windows.GetCurrentThreadId()))
+ name, ok := desktopName(desktop)
+ runtime.UnlockOSThread()
+ if !ok {
+ return nil, errors.New("input backend desktop is unavailable")
+ }
+ return &windowsSink{desktop: name, keys: map[uint16]bool{}, buttons: map[int]bool{}}, nil
}
func (*windowsSink) Authorize() error { return nil }
-func inject(inputs ...winInput) error {
+type windowsInputError struct {
+ accepted, requested uintptr
+ code syscall.Errno
+ desktopAccessible bool
+ desktopMatchesThread string
+ foregroundPresent bool
+ higherIntegrity string
+}
+
+func (e *windowsInputError) Error() string {
+ return fmt.Sprintf("Windows accepted %d of %d input events (system code %d; input desktop accessible=%t; input desktop matches thread=%s; foreground present=%t; foreground higher integrity=%s)",
+ e.accepted, e.requested, e.code, e.desktopAccessible, e.desktopMatchesThread, e.foregroundPresent, e.higherIntegrity)
+}
+
+func (e *windowsInputError) Unwrap() error { return e.code }
+
+// Observe permission boundaries only after a rejected operation. These facts do
+// not identify UIPI as the cause, and contain no window, process or desktop names.
+func rejectedWindowsInput(accepted, requested uintptr, code syscall.Errno) error {
+ result := &windowsInputError{accepted: accepted, requested: requested, code: code, higherIntegrity: "unknown", desktopMatchesThread: "unknown"}
+ desktop, _, _ := user32.NewProc("OpenInputDesktop").Call(0, 0, 1) // DESKTOP_READOBJECTS
+ result.desktopAccessible = desktop != 0
+ if desktop != 0 {
+ threadDesktop, _, _ := user32.NewProc("GetThreadDesktop").Call(uintptr(windows.GetCurrentThreadId()))
+ inputName, inputOK := desktopName(desktop)
+ threadName, threadOK := desktopName(threadDesktop)
+ if inputOK && threadOK {
+ result.desktopMatchesThread = fmt.Sprint(inputName == threadName)
+ }
+ user32.NewProc("CloseDesktop").Call(desktop)
+ }
+ foreground, _, _ := user32.NewProc("GetForegroundWindow").Call()
+ result.foregroundPresent = foreground != 0
+ if foreground == 0 {
+ return result
+ }
+ var pid uint32
+ user32.NewProc("GetWindowThreadProcessId").Call(foreground, uintptr(unsafe.Pointer(&pid)))
+ process, err := windows.OpenProcess(windows.PROCESS_QUERY_LIMITED_INFORMATION, false, pid)
+ if err != nil {
+ return result
+ }
+ defer windows.CloseHandle(process)
+ var token windows.Token
+ if windows.OpenProcessToken(process, windows.TOKEN_QUERY, &token) != nil {
+ return result
+ }
+ defer token.Close()
+ own, ownOK := tokenIntegrity(windows.GetCurrentProcessToken())
+ other, otherOK := tokenIntegrity(token)
+ if ownOK && otherOK {
+ result.higherIntegrity = fmt.Sprint(other > own)
+ }
+ return result
+}
+
+func desktopName(desktop uintptr) (string, bool) {
+ if desktop == 0 {
+ return "", false
+ }
+ query := user32.NewProc("GetUserObjectInformationW")
+ var size uint32
+ query.Call(desktop, 2, 0, 0, uintptr(unsafe.Pointer(&size))) // UOI_NAME
+ if size == 0 {
+ return "", false
+ }
+ buffer := make([]uint16, (size+1)/2)
+ ok, _, _ := query.Call(desktop, 2, uintptr(unsafe.Pointer(&buffer[0])), uintptr(len(buffer)*2), uintptr(unsafe.Pointer(&size)))
+ return windows.UTF16ToString(buffer), ok != 0
+}
+
+func tokenIntegrity(token windows.Token) (uint32, bool) {
+ var size uint32
+ if err := windows.GetTokenInformation(token, windows.TokenIntegrityLevel, nil, 0, &size); err != windows.ERROR_INSUFFICIENT_BUFFER {
+ return 0, false
+ }
+ buffer := make([]byte, size)
+ if windows.GetTokenInformation(token, windows.TokenIntegrityLevel, &buffer[0], size, &size) != nil {
+ return 0, false
+ }
+ label := (*windows.Tokenmandatorylabel)(unsafe.Pointer(&buffer[0]))
+ sid := label.Label.Sid
+ if sid == nil || sid.SubAuthorityCount() == 0 {
+ return 0, false
+ }
+ return sid.SubAuthority(uint32(sid.SubAuthorityCount() - 1)), true
+}
+
+func (s *windowsSink) inject(inputs ...winInput) error {
+ if len(inputs) == 0 {
+ return nil
+ }
+ // Keep failure observations on the OS thread that attempted the operation.
+ runtime.LockOSThread()
+ defer runtime.UnlockOSThread()
n, _, err := sendInput.Call(uintptr(len(inputs)), uintptr(unsafe.Pointer(&inputs[0])), unsafe.Sizeof(inputs[0]))
if n != uintptr(len(inputs)) {
// Windows does not identify UIPI blocking through GetLastError. Preserve
// the accepted count and any reported code without inventing that cause.
- if err != windows.ERROR_SUCCESS {
- return fmt.Errorf("Windows accepted %d of %d input events: %w", n, len(inputs), err)
+ var code syscall.Errno
+ errors.As(err, &code)
+ original := rejectedWindowsInput(n, uintptr(len(inputs)), code)
+ // A changed active desktop grants no new input authority. Only repair a
+ // thread that drifted away from this backend's original desktop.
+ active, _, _ := user32.NewProc("OpenInputDesktop").Call(0, 0, 0x10000000) // GENERIC_ALL: input injection needs more than object observation.
+ if active == 0 {
+ return original
+ }
+ defer user32.NewProc("CloseDesktop").Call(active)
+ activeName, activeOK := desktopName(active)
+ thread, _, _ := user32.NewProc("GetThreadDesktop").Call(uintptr(windows.GetCurrentThreadId()))
+ threadName, threadOK := desktopName(thread)
+ if !activeOK || !threadOK || activeName != s.desktop || activeName == threadName {
+ return original
+ }
+ setDesktop := user32.NewProc("SetThreadDesktop")
+ if ok, _, _ := setDesktop.Call(active); ok == 0 {
+ return original
+ }
+ // Never replay the accepted prefix of a partially accepted operation.
+ remaining := inputs[n:]
+ accepted, _, retryErr := sendInput.Call(uintptr(len(remaining)), uintptr(unsafe.Pointer(&remaining[0])), unsafe.Sizeof(remaining[0]))
+ var result error
+ if accepted != uintptr(len(remaining)) {
+ var retryCode syscall.Errno
+ errors.As(retryErr, &retryCode)
+ result = rejectedWindowsInput(n+accepted, uintptr(len(inputs)), retryCode)
+ }
+ if ok, _, restoreErr := setDesktop.Call(thread); ok == 0 {
+ result = errors.Join(result, fmt.Errorf("restore input thread desktop: %w", restoreErr))
}
- return fmt.Errorf("Windows accepted %d of %d input events", n, len(inputs))
+ return result
}
return nil
}
@@ -120,12 +252,12 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
}
px := float64(b.left) + e.X*float64(b.width-1) - float64(int32(x))
py := float64(b.top) + e.Y*float64(b.height-1) - float64(int32(y))
- return inject(winInput{Mouse: mouseInput{X: int32(px * 65535 / float64(int32(w)-1)), Y: int32(py * 65535 / float64(int32(h)-1)), Flags: 0xc001}})
+ return s.inject(winInput{Mouse: mouseInput{X: int32(px * 65535 / float64(int32(w)-1)), Y: int32(py * 65535 / float64(int32(h)-1)), Flags: 0xc001}})
case "relative":
if e.DX < -32767 || e.DX > 32767 || e.DY < -32767 || e.DY > 32767 {
return errors.New("invalid relative pointer")
}
- return inject(winInput{Mouse: mouseInput{X: e.DX, Y: e.DY, Flags: 1}})
+ return s.inject(winInput{Mouse: mouseInput{X: e.DX, Y: e.DY, Flags: 1}})
case "button":
input, err := buttonInput(e.Button, e.Down)
if err != nil {
@@ -134,7 +266,7 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
if !e.Down && !s.buttons[e.Button] {
return nil
}
- if err := inject(input); err != nil {
+ if err := s.inject(input); err != nil {
return err
}
if e.Down {
@@ -146,7 +278,7 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
if e.Delta < -1200 || e.Delta > 1200 {
return errors.New("invalid wheel")
}
- return inject(winInput{Mouse: mouseInput{Data: uint32(int32(e.Delta)), Flags: 0x800}})
+ return s.inject(winInput{Mouse: mouseInput{Data: uint32(int32(e.Delta)), Flags: 0x800}})
case "key":
key := scanCodes[e.Code]
if key == 0 {
@@ -155,7 +287,7 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
if !e.Down && !s.keys[key] {
return nil
}
- if err := inject(keyInput(key, e.Down)); err != nil {
+ if err := s.inject(keyInput(key, e.Down)); err != nil {
return err
}
if e.Down {
@@ -164,7 +296,7 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
delete(s.keys, key)
}
case "text":
- return inject(textInputs(e.Text)...)
+ return s.inject(textInputs(e.Text)...)
default:
return errors.New("unsupported input")
}
@@ -174,7 +306,7 @@ func (s *windowsSink) Apply(b bounds, e inputEvent) error {
func (s *windowsSink) Reset() error {
var result error
for key := range s.keys {
- if err := inject(keyInput(key, false)); err != nil {
+ if err := s.inject(keyInput(key, false)); err != nil {
result = errors.Join(result, err)
} else {
delete(s.keys, key)
diff --git a/internal/app/remoteinput/platform_windows_test.go b/internal/app/remoteinput/platform_windows_test.go
index 619ef3ff..aeef75f0 100644
--- a/internal/app/remoteinput/platform_windows_test.go
+++ b/internal/app/remoteinput/platform_windows_test.go
@@ -4,10 +4,95 @@ package remoteinput
import (
"context"
+ "errors"
+ "runtime"
+ "strings"
+ "syscall"
"testing"
"unsafe"
+
+ "golang.org/x/sys/windows"
)
+func TestWindowsInputRecoversItsDesktopAndPendingRelease(t *testing.T) {
+ runtime.LockOSThread()
+ defer runtime.UnlockOSThread()
+ backend, err := newPlatformSink(false)
+ if err != nil {
+ t.Fatal(err)
+ }
+ sink := backend.(*windowsSink)
+ active, _, _ := user32.NewProc("OpenInputDesktop").Call(0, 0, 0x81)
+ if active == 0 {
+ t.Skip("no accessible input desktop")
+ }
+ defer user32.NewProc("CloseDesktop").Call(active)
+ activeName, ok := desktopName(active)
+ if !ok || activeName != sink.desktop {
+ t.Skip("test is not on the input desktop")
+ }
+ down, _, _ := user32.NewProc("GetAsyncKeyState").Call(1)
+ if down&0x8000 != 0 {
+ t.Skip("local mouse button is held")
+ }
+ thread, _, _ := user32.NewProc("GetThreadDesktop").Call(uintptr(windows.GetCurrentThreadId()))
+ name, _ := windows.UTF16PtrFromString(t.Name())
+ other, _, err := user32.NewProc("CreateDesktopW").Call(uintptr(unsafe.Pointer(name)), 0, 0, 0, 0x83, 0)
+ if other == 0 {
+ t.Fatalf("create isolated thread desktop: %v", err)
+ }
+ defer func() {
+ if restored, _, err := user32.NewProc("SetThreadDesktop").Call(thread); restored == 0 {
+ t.Errorf("restore test thread desktop: %v", err)
+ }
+ user32.NewProc("CloseDesktop").Call(other)
+ }()
+ if bound, _, err := user32.NewProc("SetThreadDesktop").Call(other); bound == 0 {
+ t.Fatalf("bind isolated thread: %v", err)
+ }
+ // Never switch the visible desktop. Only this test thread leaves it.
+ probe := winInput{Mouse: mouseInput{Flags: 1}}
+ restricted := &windowsSink{desktop: windows.UTF16PtrToString(name)}
+ if err := restricted.inject(probe); err == nil {
+ t.Fatal("input escaped the backend's desktop")
+ }
+ if err := sink.inject(probe); err != nil {
+ t.Fatalf("input did not recover to its own active desktop: %v", err)
+ }
+ current, _, _ := user32.NewProc("GetThreadDesktop").Call(uintptr(windows.GetCurrentThreadId()))
+ currentName, _ := desktopName(current)
+ if currentName != restricted.desktop {
+ t.Fatal("injection changed the caller's desktop")
+ }
+ // A previously failed release must use the same recovery as fresh input.
+ sink.buttons[0] = true
+ m := &Manager{sink: sink, pendingButtons: map[int]bool{0: true}}
+ if err := m.retryReleases(); err != nil || len(m.pendingButtons) != 0 || len(sink.buttons) != 0 {
+ t.Fatalf("release debt did not recover: %v", err)
+ }
+ if err := sink.Apply(bounds{}, inputEvent{Type: "relative"}); err != nil {
+ t.Fatalf("input remained blocked after cleanup: %v", err)
+ }
+}
+
+func TestRejectedWindowsInputDiagnostics(t *testing.T) {
+ for _, want := range []syscall.Errno{0, windows.ERROR_ACCESS_DENIED} {
+ err := rejectedWindowsInput(0, 1, want)
+ var code syscall.Errno
+ if !errors.As(err, &code) || code != want {
+ t.Fatalf("Windows rejection lost its system code: %v", err)
+ }
+ for _, fact := range []string{"Windows accepted 0 of 1", "input desktop accessible=", "input desktop matches thread=", "foreground present=", "foreground higher integrity="} {
+ if !strings.Contains(err.Error(), fact) {
+ t.Fatalf("Windows rejection lost %q: %v", fact, err)
+ }
+ }
+ }
+ if level, ok := tokenIntegrity(windows.GetCurrentProcessToken()); !ok || level == 0 {
+ t.Fatalf("current process integrity was not observed: %d %v", level, ok)
+ }
+}
+
func TestNativeStopShortcutRegistrationLifetime(t *testing.T) {
// Reserve an available chord through the OS API without injecting input.
var reserved func()
diff --git a/internal/media/encoded/packetizer.go b/internal/media/encoded/packetizer.go
index dbc1d221..a525d44f 100644
--- a/internal/media/encoded/packetizer.go
+++ b/internal/media/encoded/packetizer.go
@@ -57,9 +57,5 @@ func (p *Packetizer) Packetize(data []byte, timestamp, duration time.Duration) (
p.hasTimestamp = true
}
p.lastTimestamp = timestamp
- packets := p.rtp.Packetize(data, 0)
- if len(packets) == 0 {
- return nil, errors.New("encoded video access unit produced no RTP packets")
- }
- return packets, nil
+ return p.rtp.Packetize(data, 0), nil
}
diff --git a/internal/media/forwarding/audio_output.go b/internal/media/forwarding/audio_output.go
new file mode 100644
index 00000000..89d70a0c
--- /dev/null
+++ b/internal/media/forwarding/audio_output.go
@@ -0,0 +1,207 @@
+package forwarding
+
+import (
+ "errors"
+ "io"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/TNTcraftHIM/Piik/internal/diagnostics"
+ "github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/livekit/livekit-server/pkg/sfu/buffer"
+ "github.com/livekit/livekit-server/pkg/sfu/pacer"
+ "github.com/livekit/protocol/livekit"
+ "github.com/pion/interceptor"
+ "github.com/pion/rtcp"
+ "github.com/pion/webrtc/v4"
+)
+
+// ConfigureAudioForwarding installs callback-driven RTCP only for DownTracks.
+// Incoming tracks retain ordinary Pion buffers and their ReadRTCP reader. Call
+// this instead of registering another sender/receiver report interceptor pair.
+func ConfigureAudioForwarding(settings *webrtc.SettingEngine, registry *interceptor.Registry,
+ lookup func(uint32) *sfu.DownTrack,
+) (*buffer.Factory, error) {
+ factory := buffer.NewFactoryOfBufferFactory(buffer.InitPacketBufferSizeVideo, buffer.InitPacketBufferSizeAudio).CreateBufferFactory()
+ settings.BufferFactory = packetBufferFactory(factory)
+ if err := configureRTCPReports(registry, lookup); err != nil {
+ return nil, err
+ }
+ return factory, nil
+}
+
+type AudioOutputOptions struct {
+ ID, StreamID string
+ Codec webrtc.RTPCodecParameters
+ BufferFactory *buffer.Factory
+ RTCPWriter func([]rtcp.Packet) error
+}
+
+// AudioOutput retains one negotiated sender's clock across input publications.
+// Its caller owns subscription authority; detaching synchronously fences input
+// callbacks without retiring the reusable sender or adding an SDP media section.
+type AudioOutput struct {
+ mu sync.Mutex
+ track *sfu.DownTrack
+ idle *audioReceiver
+ receiver *audioReceiver
+}
+
+func NewAudioOutput(options AudioOutputOptions) (*AudioOutput, error) {
+ if !strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeOpus) || options.BufferFactory == nil || options.RTCPWriter == nil {
+ return nil, errors.New("forwarded audio output is invalid")
+ }
+ idle, err := NewSource(SourceOptions{ID: options.ID, StreamID: options.StreamID, Codec: options.Codec})
+ if err != nil {
+ return nil, err
+ }
+ output := &AudioOutput{}
+ output.idle = &audioReceiver{Source: idle, output: output}
+ output.receiver = output.idle
+ log := diagnostics.MediaLogger("audio-output").WithValues("trackId", diagnostics.ID(options.ID))
+ output.track, err = sfu.NewDownTrack(sfu.DownTrackParams{
+ Codecs: []webrtc.RTPCodecParameters{options.Codec}, Receiver: output.idle,
+ Source: livekit.TrackSource_MICROPHONE, StreamID: options.StreamID,
+ SubID: livekit.ParticipantID(options.ID), BufferFactory: options.BufferFactory,
+ MaxTrack: buffer.InitPacketBufferSizeAudio, Pacer: pacer.NewPassThrough(log, nil),
+ Logger: log, Listener: output, RTCPWriter: options.RTCPWriter,
+ DisableSenderReportPassThrough: true,
+ })
+ if err != nil {
+ idle.Close()
+ return nil, err
+ }
+ return output, nil
+}
+
+func (output *AudioOutput) Track() webrtc.TrackLocal { return output.track }
+func (output *AudioOutput) DownTrack() *sfu.DownTrack { return output.track }
+func (output *AudioOutput) SetTransceiver(t *webrtc.RTPTransceiver) { output.track.SetTransceiver(t) }
+func (output *AudioOutput) SetConnected() { output.track.SetConnected() }
+
+func (output *AudioOutput) ReplaceSource(next *Source) error {
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ if output.track.IsClosed() {
+ return io.ErrClosedPipe
+ }
+ if next == nil {
+ return errors.New("replacement audio source is absent")
+ }
+ next.mu.Lock()
+ defer next.mu.Unlock()
+ if next.closed || next.IsClosed() {
+ return io.ErrClosedPipe
+ }
+ before, after := output.idle.Codec(), next.Codec()
+ if next.Kind() != webrtc.RTPCodecTypeAudio || before.MimeType != after.MimeType ||
+ before.ClockRate != after.ClockRate || before.Channels != after.Channels {
+ return errors.New("replacement audio source changed its codec")
+ }
+ if output.receiver.Source == next {
+ return nil
+ }
+ output.track.ReceiverRestart(output.receiver)
+ output.receiver = &audioReceiver{Source: next, output: output, attached: true}
+ output.track.SetReceiver(output.receiver)
+ // AddDownTrack's live reports take mu. Replay the current accepted clock
+ // here, outside its synchronous registration callback, with the same fence.
+ if report := next.currentSenderReport(0); report != nil {
+ _ = output.track.HandleRTCPSenderReportData(after.PayloadType, 0, report)
+ }
+ return nil
+}
+
+func (output *AudioOutput) DetachSource() {
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ output.detachLocked()
+}
+
+func (output *AudioOutput) detachLocked() {
+ if output.track.IsClosed() || output.receiver == output.idle {
+ return
+ }
+ output.receiver = output.idle
+ output.track.SetReceiver(output.idle)
+}
+
+func (output *AudioOutput) Close() {
+ output.mu.Lock()
+ output.detachLocked()
+ output.track.CloseWithFlush(false, true)
+ output.mu.Unlock()
+ output.idle.Source.Close()
+}
+
+// The empty receiver keeps an idle slot bound. Each real receiver registers a
+// source-specific sender so detached broadcasts cannot enter its replacement.
+type audioReceiver struct {
+ *Source
+ output *AudioOutput
+ attached bool
+}
+
+func (receiver *audioReceiver) AddDownTrack(sfu.TrackSender) error {
+ if !receiver.attached {
+ return nil
+ }
+ return receiver.Source.ReceiverBase.AddDownTrack(&audioSourceSender{
+ DownTrack: receiver.output.track, receiver: receiver,
+ })
+}
+
+type audioSourceSender struct {
+ *sfu.DownTrack
+ receiver *audioReceiver
+}
+
+func (sender *audioSourceSender) WriteRTP(packet *buffer.ExtPacket, layer int32) int32 {
+ output := sender.receiver.output
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ if output.receiver != sender.receiver {
+ return 0
+ }
+ return output.track.WriteRTP(packet, layer)
+}
+
+func (sender *audioSourceSender) Close() {
+ output := sender.receiver.output
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ if output.receiver == sender.receiver {
+ output.detachLocked()
+ }
+}
+
+func (sender *audioSourceSender) ReceiverRestart(receiver sfu.TrackReceiver) {
+ output := sender.receiver.output
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ if output.receiver == sender.receiver {
+ output.track.ReceiverRestart(receiver)
+ }
+}
+
+func (sender *audioSourceSender) HandleRTCPSenderReportData(payloadType webrtc.PayloadType, layer int32, _ *livekit.RTCPSenderReportState) error {
+ output := sender.receiver.output
+ output.mu.Lock()
+ defer output.mu.Unlock()
+ if output.receiver != sender.receiver {
+ return nil
+ }
+ if report := sender.receiver.currentSenderReport(layer); report != nil {
+ return output.track.HandleRTCPSenderReportData(payloadType, layer, report)
+ }
+ return nil
+}
+
+func (*AudioOutput) OnBindAndConnected() {}
+func (*AudioOutput) OnStatsUpdate(*livekit.AnalyticsStat) {}
+func (*AudioOutput) OnMaxSubscribedLayerChanged(int32) {}
+func (*AudioOutput) OnRttUpdate(uint32) {}
+func (*AudioOutput) OnCodecNegotiated(webrtc.RTPCodecCapability) {}
+func (*AudioOutput) OnDownTrackClose(bool) {}
+func (*AudioOutput) OnStreamStarted(time.Duration) {}
diff --git a/internal/media/forwarding/audio_output_test.go b/internal/media/forwarding/audio_output_test.go
new file mode 100644
index 00000000..503978d9
--- /dev/null
+++ b/internal/media/forwarding/audio_output_test.go
@@ -0,0 +1,156 @@
+package forwarding
+
+import (
+ "bytes"
+ "testing"
+ "time"
+
+ "github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/livekit/livekit-server/pkg/sfu/buffer"
+ "github.com/livekit/mediatransportutil"
+ "github.com/livekit/protocol/livekit"
+ "github.com/pion/interceptor"
+ "github.com/pion/rtcp"
+ "github.com/pion/rtp"
+ "github.com/pion/transport/v5/packetio"
+ "github.com/pion/webrtc/v4"
+)
+
+// Bind at Pion's public TrackLocal boundary. Source buffers and the LiveKit
+// forwarder run unchanged; the writer observes the exact RTP given to SRTP.
+type audioTrackContext struct {
+ webrtc.TrackLocalContext
+ codec webrtc.RTPCodecParameters
+ packets chan *rtp.Packet
+}
+
+func (context *audioTrackContext) CodecParameters() []webrtc.RTPCodecParameters {
+ return []webrtc.RTPCodecParameters{context.codec}
+}
+func (*audioTrackContext) SSRC() webrtc.SSRC { return 9000 }
+func (*audioTrackContext) SSRCRetransmission() webrtc.SSRC { return 0 }
+func (context *audioTrackContext) WriteStream() webrtc.TrackLocalWriter { return context }
+func (context *audioTrackContext) WriteRTP(header *rtp.Header, payload []byte) (int, error) {
+ context.packets <- &rtp.Packet{Header: header.Clone(), Payload: bytes.Clone(payload)}
+ return header.MarshalSize() + len(payload), nil
+}
+func (context *audioTrackContext) Write(data []byte) (int, error) {
+ packet := &rtp.Packet{}
+ if err := packet.Unmarshal(data); err != nil {
+ return 0, err
+ }
+ return context.WriteRTP(&packet.Header, packet.Payload)
+}
+
+func TestAudioOutputRetainsClockAcrossInputLifetimeAndPacketLoss(t *testing.T) {
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ codec := webrtc.RTPCodecParameters{PayloadType: 111, RTPCodecCapability: webrtc.RTPCodecCapability{
+ MimeType: webrtc.MimeTypeOpus, ClockRate: 48_000, Channels: 2, SDPFmtpLine: "minptime=10;useinbandfec=1",
+ }}
+ settings, registry := webrtc.SettingEngine{}, &interceptor.Registry{}
+ factory, err := ConfigureAudioForwarding(&settings, registry, func(uint32) *sfu.DownTrack { return nil })
+ check(err)
+ output, err := NewAudioOutput(AudioOutputOptions{ID: "slot", StreamID: "room", Codec: codec,
+ BufferFactory: factory, RTCPWriter: func([]rtcp.Packet) error { return nil },
+ })
+ check(err)
+ t.Cleanup(output.Close)
+ context := &audioTrackContext{codec: codec, packets: make(chan *rtp.Packet, 32)}
+ _, err = output.Track().Bind(context)
+ check(err)
+ output.SetConnected()
+ makeSource := func(id string, ssrc uint32) *Source {
+ t.Helper()
+ source, err := NewSource(SourceOptions{ID: id, StreamID: id, Codec: codec})
+ check(err)
+ t.Cleanup(source.Close)
+ check(source.BindLayer(0, ssrc, webrtc.RTPParameters{Codecs: []webrtc.RTPCodecParameters{codec}}))
+ if info := source.TrackInfo(); info.Type != livekit.TrackType_AUDIO || len(info.Layers) != 0 {
+ t.Fatalf("audio acquired video allocation metadata: %+v", info)
+ }
+ return source
+ }
+ write := func(source *Source, ssrc uint32, sequence uint16, timestamp uint32) *rtp.Packet {
+ t.Helper()
+ check(source.WriteRTP(0, &rtp.Packet{Header: rtp.Header{Version: 2, PayloadType: 111,
+ SSRC: ssrc, SequenceNumber: sequence, Timestamp: timestamp}, Payload: []byte{0xf8, 0xff, 0xfe}}))
+ select {
+ case packet := <-context.packets:
+ return packet
+ case <-time.After(time.Second):
+ t.Fatal("audio did not reach the bound output")
+ return nil
+ }
+ }
+ first := makeSource("first", 1000)
+ check(output.ReplaceSource(first))
+ stale := first.GetDownTracks()[0]
+ a := write(first, 1000, 20_000, 9_600_000)
+ b := write(first, 1000, 20_001, 9_600_960)
+ if b.SequenceNumber-a.SequenceNumber != 1 || b.Timestamp-a.Timestamp != 960 {
+ t.Fatal("ordinary audio timing changed")
+ }
+ // Preserve a lost packet's sequence/timestamp space and a later DTX gap.
+ c := write(first, 1000, 20_003, 9_602_880)
+ late := write(first, 1000, 20_002, 9_601_920)
+ d := write(first, 1000, 20_004, 9_607_680)
+ if c.SequenceNumber-b.SequenceNumber != 2 || c.Timestamp-b.Timestamp != 1920 ||
+ late.SequenceNumber-b.SequenceNumber != 1 || late.Timestamp-b.Timestamp != 960 ||
+ d.SequenceNumber-c.SequenceNumber != 1 || d.Timestamp-c.Timestamp != 4800 {
+ t.Fatal("loss or DTX was compressed out of the output timeline")
+ }
+ check(first.SenderReport(0, &rtcp.SenderReport{SSRC: 1000, RTPTime: 9_607_680,
+ NTPTime: uint64(mediatransportutil.ToNtpTime(time.Now())), PacketCount: 4, OctetCount: 12}))
+ if report := output.DownTrack().CreateSenderReport(); report == nil || report.SSRC != a.SSRC {
+ t.Fatal("output did not retain its own sender report identity")
+ }
+ next := makeSource("next", 2000)
+ check(output.ReplaceSource(next))
+ // A detached source's already queued close/report cannot retire or re-clock
+ // the reusable slot. The source itself may close after the new one attaches.
+ stale.Close()
+ check(stale.HandleRTCPSenderReportData(111, 0, nil))
+ if stale.WriteRTP(&buffer.ExtPacket{Packet: &rtp.Packet{}}, 0) != 0 {
+ t.Fatal("a detached broadcast wrote into the replacement")
+ }
+ first.Close()
+ e := write(next, 2000, 1000, 480_000)
+ if e.SSRC != d.SSRC || e.SequenceNumber-d.SequenceNumber != 1 || int32(e.Timestamp-d.Timestamp) <= 0 {
+ t.Fatalf("independent input clock escaped into the reused slot: before=%+v after=%+v", d.Header, e.Header)
+ }
+ // Closing the current input detaches it, while the same output remains usable
+ // for a later publication, including one which reuses its predecessor's SSRC.
+ next.Close()
+ if output.DownTrack().IsClosed() || output.receiver != output.idle {
+ t.Fatal("input retirement destroyed the reusable output slot")
+ }
+ last := makeSource("last", 2000)
+ check(output.ReplaceSource(last))
+ f := write(last, 2000, 100, 48_000)
+ if f.SSRC != e.SSRC || f.SequenceNumber-e.SequenceNumber != 1 || int32(f.Timestamp-e.Timestamp) <= 0 {
+ t.Fatal("a later input could not resume the existing output clock")
+ }
+ output.DetachSource()
+ if len(last.GetDownTracks()) != 0 || output.DownTrack().IsClosed() {
+ t.Fatal("subscription removal did not detach the live input")
+ }
+ // The shared factory claims only outgoing DownTrack RTCP; ingress reports
+ // must still be readable by the receiving connection's ordinary RTCP loop.
+ input := settings.BufferFactory(packetio.RTCPBufferPacket, 1234)
+ defer input.Close()
+ report, err := (&rtcp.SenderReport{SSRC: 1234}).Marshal()
+ check(err)
+ _, err = input.Write(report)
+ check(err)
+ read := make([]byte, 1500)
+ n, err := input.Read(read)
+ check(err)
+ if !bytes.Equal(read[:n], report) {
+ t.Fatal("incoming RTCP was consumed by an output callback")
+ }
+}
diff --git a/internal/media/forwarding/codecs.go b/internal/media/forwarding/codecs.go
index d185618e..86860496 100644
--- a/internal/media/forwarding/codecs.go
+++ b/internal/media/forwarding/codecs.go
@@ -5,10 +5,29 @@ import (
"strings"
"github.com/pion/interceptor"
+ "github.com/pion/interceptor/pkg/nack"
"github.com/pion/rtp"
"github.com/pion/webrtc/v4"
)
+// RegisterIngressInterceptors leaves video NACK generation to the forwarding
+// buffer, which sees primary sequence numbers after RTX repair. Pion's wire
+// reader cannot observe those repairs; its reports, TWCC and responder remain.
+func RegisterIngressInterceptors(media *webrtc.MediaEngine, registry *interceptor.Registry) error {
+ return webrtc.RegisterDefaultInterceptorsWithOptions(media, registry,
+ webrtc.WithNackGeneratorOptions(nack.GeneratorStreamsFilter(func(info *interceptor.StreamInfo) bool {
+ if strings.HasPrefix(strings.ToLower(info.MimeType), "video/") {
+ return false
+ }
+ for _, feedback := range info.RTCPFeedback {
+ if feedback.Type == "nack" && feedback.Parameter == "" {
+ return true
+ }
+ }
+ return false
+ })))
+}
+
// Each outgoing connection offers one primary video codec and Opus (111).
// RTX keeps late repair outside the primary stream's SRTP replay window.
func registerVideoCodec(media *webrtc.MediaEngine, codec webrtc.RTPCodecParameters) error {
diff --git a/internal/media/forwarding/encoded_source.go b/internal/media/forwarding/encoded_source.go
index e0b05af0..6b7c9f6e 100644
--- a/internal/media/forwarding/encoded_source.go
+++ b/internal/media/forwarding/encoded_source.go
@@ -104,6 +104,9 @@ func (input *EncodedSource) WriteFrame(layer int, frame encoded.Frame) error {
input.rtpBytes[layer] += uint64(packet.MarshalSize())
}
input.codecBytes[layer] += uint64(len(frame.Data))
+ if len(packets) == 0 {
+ return nil
+ }
// Retain the capture instant's Go monotonic component when converting to
// the media core's epoch-based monotonic clock, including after wall jumps.
localTime := mono.FromTime(sourceTime)
diff --git a/internal/media/forwarding/encoded_source_test.go b/internal/media/forwarding/encoded_source_test.go
index 088757a2..017e5f94 100644
--- a/internal/media/forwarding/encoded_source_test.go
+++ b/internal/media/forwarding/encoded_source_test.go
@@ -8,6 +8,7 @@ import (
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
"github.com/livekit/livekit-server/pkg/sfu"
"github.com/pion/webrtc/v4"
+ "google.golang.org/protobuf/proto"
)
func TestCodecBudgetUsesActualSourceFraming(t *testing.T) {
@@ -66,3 +67,66 @@ func TestCodecBudgetUsesActualSourceFraming(t *testing.T) {
t.Fatal("replacement source inherited stale framing ratios")
}
}
+
+func TestEncodedSourceH264MetadataPreservesCorrelation(t *testing.T) {
+ input, err := NewEncodedSource(SourceOptions{ID: "h264-metadata", StreamID: "h264-metadata",
+ Codec: webrtc.RTPCodecParameters{RTPCodecCapability: webrtc.RTPCodecCapability{
+ MimeType: webrtc.MimeTypeH264, ClockRate: 90_000,
+ SDPFmtpLine: "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42c01f",
+ }, PayloadType: 96}, Formats: []LayerFormat{{Width: 2, Height: 2, Bitrate: 100_000}}})
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer input.Close()
+ const period = 40 * time.Millisecond
+ origin := time.Now()
+ write := func(index int, data []byte) {
+ t.Helper()
+ pts := time.Duration(index) * period
+ if err := input.BeginFrame(pts, origin.Add(pts)); err != nil {
+ t.Fatal(err)
+ }
+ if err := input.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: period}); err != nil {
+ t.Fatal(err)
+ }
+ }
+ var keyframe []byte
+ for _, nalu := range sfu.H264KeyFrame2x2 {
+ keyframe = append(keyframe, 0, 0, 0, 1)
+ keyframe = append(keyframe, nalu...)
+ }
+ write(0, keyframe)
+ buffer := input.GetAllBuffers()[0]
+ first := buffer.GetSenderReportData()
+ if first == nil || input.packets[0] == 0 {
+ t.Fatal("first IDR did not establish RTP correlation")
+ }
+ correlation := proto.Clone(first)
+ packets, octets, rtpBytes, codecBytes := input.packets[0], input.octets[0], input.rtpBytes[0], input.codecBytes[0]
+ for index, metadata := range [][]byte{
+ sfu.H264KeyFrame2x2SPS, sfu.H264KeyFrame2x2PPS,
+ {0x09, 0xf0}, {0x0c, 0xff, 0x80}, // AUD and filler produce no RTP.
+ } {
+ write(index+1, metadata)
+ codecBytes += uint64(len(metadata))
+ if input.packets[0] != packets || input.octets[0] != octets || input.rtpBytes[0] != rtpBytes {
+ t.Fatal("metadata invented RTP packets or cost")
+ }
+ if input.codecBytes[0] != codecBytes || !proto.Equal(correlation, buffer.GetSenderReportData()) {
+ t.Fatal("accepted metadata lost codec bytes or advanced RTP correlation")
+ }
+ }
+ write(5, sfu.H264KeyFrame2x2IDR)
+ last := buffer.GetSenderReportData()
+ if input.packets[0] != packets+2 || last.RtpTimestamp-first.RtpTimestamp != 18_000 {
+ t.Fatal("cached SPS/PPS and the next IDR lost their source RTP timeline")
+ }
+ if captured, ok := input.ReferenceTime(0, last.RtpTimestamp); !ok || captured.Sub(origin.Add(5*period)).Abs() > time.Microsecond {
+ t.Fatal("next IDR correlation used metadata or arrival time")
+ }
+ packets = input.packets[0]
+ write(6, []byte{0x06, 0x05, 0x00, 0x80}) // SEI remains a normal RTP payload.
+ if input.packets[0] != packets+1 {
+ t.Fatal("SEI was incorrectly treated as packetless metadata")
+ }
+}
diff --git a/internal/media/forwarding/forwardingtest/ingress.go b/internal/media/forwarding/forwardingtest/ingress.go
new file mode 100644
index 00000000..d88894bb
--- /dev/null
+++ b/internal/media/forwarding/forwardingtest/ingress.go
@@ -0,0 +1,173 @@
+// Package forwardingtest exercises shared media boundaries through real Pion peers.
+package forwardingtest
+
+import (
+ "bytes"
+ "context"
+ "encoding/binary"
+ "net"
+ "sync/atomic"
+ "testing"
+ "time"
+
+ "github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/pion/interceptor"
+ "github.com/pion/rtcp"
+ "github.com/pion/rtp"
+ "github.com/pion/webrtc/v4"
+)
+
+type repairLoss struct {
+ interceptor.NoOp
+ allowRepair atomic.Bool
+ repairs atomic.Uint32
+}
+
+func (loss *repairLoss) NewInterceptor(string) (interceptor.Interceptor, error) { return loss, nil }
+
+func (loss *repairLoss) BindLocalStream(info *interceptor.StreamInfo, writer interceptor.RTPWriter) interceptor.RTPWriter {
+ return interceptor.RTPWriterFunc(func(header *rtp.Header, payload []byte, attributes interceptor.Attributes) (int, error) {
+ if header.SSRC == info.SSRC && header.SequenceNumber == 17 {
+ return header.MarshalSize() + len(payload), nil
+ }
+ if header.SSRC == info.SSRCRetransmission && len(payload) >= 2 && binary.BigEndian.Uint16(payload) == 17 {
+ if !loss.allowRepair.Load() {
+ return header.MarshalSize() + len(payload), nil
+ }
+ loss.repairs.Add(1)
+ }
+ return writer.Write(header, payload, attributes)
+ })
+}
+
+// IngressRepair checks the actual receiving owner, not a standalone NACK log.
+// The callback installs a normal Native receiver or SFU publication and returns
+// its physical connection and admitted primary source.
+func IngressRepair(t *testing.T, receive func(webrtc.SessionDescription, webrtc.SettingEngine) (*webrtc.PeerConnection, func() *forwarding.Source)) {
+ t.Helper()
+ ctx, cancel := context.WithTimeout(t.Context(), 7*time.Second)
+ defer cancel()
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ wait := func(done <-chan struct{}) {
+ t.Helper()
+ select {
+ case <-done:
+ case <-ctx.Done():
+ t.Fatal(ctx.Err())
+ }
+ }
+ settings := webrtc.SettingEngine{}
+ settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
+ settings.SetIncludeLoopbackCandidate(true)
+ settings.SetIPFilter(func(ip net.IP) bool { return ip.IsLoopback() })
+ media := &webrtc.MediaEngine{}
+ check(media.RegisterDefaultCodecs())
+ registry := &interceptor.Registry{}
+ loss := &repairLoss{}
+ // Drop after the stock responder has cached the original packet.
+ registry.Add(loss)
+ check(webrtc.RegisterDefaultInterceptors(media, registry))
+ check(webrtc.ConfigureTWCCHeaderExtensionSender(media, registry))
+ sender, err := webrtc.NewAPI(webrtc.WithMediaEngine(media), webrtc.WithInterceptorRegistry(registry),
+ webrtc.WithSettingEngine(settings)).NewPeerConnection(webrtc.Configuration{})
+ check(err)
+ defer sender.Close()
+ track, err := webrtc.NewTrackLocalStaticRTP(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeVP8, ClockRate: 90_000}, "screen", "repair")
+ check(err)
+ videoSender, err := sender.AddTrack(track)
+ check(err)
+ var nacks, reports, feedback atomic.Uint32
+ readDone := make(chan struct{})
+ go func() {
+ defer close(readDone)
+ for {
+ packets, _, err := videoSender.ReadRTCP()
+ if err != nil {
+ return
+ }
+ for _, packet := range packets {
+ switch packet := packet.(type) {
+ case *rtcp.TransportLayerNack:
+ for _, pair := range packet.Nacks {
+ for _, sequence := range pair.PacketList() {
+ if sequence == 17 {
+ nacks.Add(1)
+ }
+ }
+ }
+ case *rtcp.ReceiverReport:
+ reports.Add(1)
+ case *rtcp.TransportLayerCC:
+ feedback.Add(1)
+ }
+ }
+ }
+ }()
+ defer func() { _ = sender.Close(); <-readDone }()
+ offer, err := sender.CreateOffer(nil)
+ check(err)
+ gathered := webrtc.GatheringCompletePromise(sender)
+ check(sender.SetLocalDescription(offer))
+ wait(gathered)
+ receiver, source := receive(*sender.LocalDescription(), settings)
+ wait(webrtc.GatheringCompletePromise(receiver))
+ check(sender.SetRemoteDescription(*receiver.LocalDescription()))
+ ticker := time.NewTicker(20 * time.Millisecond)
+ defer ticker.Stop()
+ payload := append([]byte{0x10}, sfu.VP8KeyFrame8x8...)
+ sequence := uint16(1)
+ var repairedAt time.Time
+ var settledNACKs uint32
+ var settledAt time.Time
+ for {
+ select {
+ case <-ctx.Done():
+ t.Fatalf("repair did not settle: NACK=%d RTX=%d RR=%d TWCC=%d", nacks.Load(), loss.repairs.Load(), reports.Load(), feedback.Load())
+ case <-ticker.C:
+ }
+ check(track.WriteRTP(&rtp.Packet{Header: rtp.Header{Version: 2, Marker: true,
+ SequenceNumber: sequence, Timestamp: uint32(sequence) * 1800}, Payload: payload}))
+ sequence++
+ // Keep the real hole open through several feedback opportunities.
+ if nacks.Load() >= 3 {
+ loss.allowRepair.Store(true)
+ }
+ if repairedAt.IsZero() && loss.repairs.Load() > 0 && source() != nil {
+ raw := make([]byte, 1500)
+ n, err := source().ReadRTP(raw, 0, 17)
+ if err == nil {
+ packet := &rtp.Packet{}
+ check(packet.Unmarshal(raw[:n]))
+ primaryRaw := make([]byte, 1500)
+ n, err = source().ReadRTP(primaryRaw, 0, 16)
+ check(err)
+ primary := &rtp.Packet{}
+ check(primary.Unmarshal(primaryRaw[:n]))
+ if packet.SequenceNumber != 17 || packet.Timestamp != 17*1800 ||
+ packet.SSRC != primary.SSRC || packet.PayloadType != uint8(source().Codec().PayloadType) ||
+ !bytes.Equal(packet.Payload, payload) {
+ t.Fatal("repair did not restore the admitted primary packet")
+ }
+ repairedAt = time.Now()
+ }
+ }
+ if !repairedAt.IsZero() && time.Since(repairedAt) >= 150*time.Millisecond && settledAt.IsZero() {
+ settledNACKs, settledAt = nacks.Load(), time.Now()
+ }
+ if !settledAt.IsZero() && time.Since(settledAt) >= 700*time.Millisecond {
+ if got := nacks.Load(); got != settledNACKs {
+ t.Fatalf("repaired primary still NACKed: before=%d after=%d", settledNACKs, got)
+ }
+ if reports.Load() > 0 && feedback.Load() > 0 {
+ t.Logf("unrepaired hole retried; cached primary repaired; NACK=%d RTX=%d RR=%d TWCC=%d", nacks.Load(), loss.repairs.Load(), reports.Load(), feedback.Load())
+ return
+ }
+ }
+ }
+}
diff --git a/internal/media/forwarding/output.go b/internal/media/forwarding/output.go
index b1692077..21a11ab9 100644
--- a/internal/media/forwarding/output.go
+++ b/internal/media/forwarding/output.go
@@ -92,26 +92,36 @@ func (output *Output) reconcileLocked(budgetReduced bool) {
if output.congestionState != nil {
congestion = output.congestionState()
}
- stable := !budgetReduced && output.hasAllocation && !output.IsDeficient() && congestion != bwe.CongestionStateCongested
- if !measured || (output.feedbackObserved != nil && !output.feedbackObserved()) || stable {
+ if (!output.hasAllocation && !measured) || (output.feedbackObserved != nil && !output.feedbackObserved()) {
if len(available) > 0 {
// Source history is not this new connection's capacity. Bootstrap
// through the library until video can establish real feedback;
// pausing against the initial estimate would prevent that feedback.
- // Stable streams also retain the library's optimal allocation, including
- // while individual layers' bitrate windows are becoming available.
output.allocation = output.AllocateOptimal(false, congestion == bwe.CongestionStateEarlyWarning)
output.hasAllocation = true
}
return
}
+ maximum := output.MaxLayer()
+ target := output.allocation.TargetLayer
+ targetAllowed := target.IsValid() && target.Spatial <= maximum.Spatial && target.Temporal <= maximum.Temporal &&
+ slices.Contains(available, target.Spatial)
+ if !measured && targetAllowed {
+ return
+ }
+ if !budgetReduced && output.hasAllocation && congestion != bwe.CongestionStateCongested && targetAllowed &&
+ rates[target.Spatial][target.Temporal] == 0 {
+ // An active layer's missing rate window does not revoke its allocation.
+ // Retain that target rather than optimally allocating a different layer.
+ return
+ }
output.ProvisionalAllocatePrepare()
remaining := output.budget
retained := buffer.InvalidLayer
- if !budgetReduced && output.hasAllocation {
+ if !budgetReduced && output.hasAllocation && congestion != bwe.CongestionStateCongested {
// Match the library's cooperative allocation: changed source measurements
- // do not imply congestion. Retain a valid target; only a lower connection
- // budget constrains it again. Larger outputs still need available headroom.
+ // alone retain a valid target. A lower budget or confirmed congestion
+ // constrains it again. Larger outputs still need available headroom.
transition := output.ProvisionalAllocateGetCooperativeTransition(false)
if transition.From.IsValid() && transition.From == transition.To {
retained = transition.To
@@ -120,7 +130,6 @@ func (output *Output) reconcileLocked(budgetReduced bool) {
output.ProvisionalAllocateReset()
}
}
- maximum := output.MaxLayer()
controlled, ok := output.Receiver().(interface{ RateControlled(int32) bool })
keepLowest := ok && controlled.RateControlled(0) && slices.Contains(available, int32(0))
for spatial := int32(0); spatial <= maximum.Spatial; spatial++ {
diff --git a/internal/media/forwarding/output_test.go b/internal/media/forwarding/output_test.go
index 2509a3bf..77939ccc 100644
--- a/internal/media/forwarding/output_test.go
+++ b/internal/media/forwarding/output_test.go
@@ -6,6 +6,7 @@ import (
"github.com/livekit/livekit-server/pkg/sfu"
"github.com/livekit/livekit-server/pkg/sfu/buffer"
+ "github.com/livekit/livekit-server/pkg/sfu/bwe"
"github.com/livekit/livekit-server/pkg/sfu/pacer"
"github.com/livekit/protocol/livekit"
"github.com/livekit/protocol/logger"
@@ -221,13 +222,56 @@ func TestOutputUsesLibraryAllocationAndExactLifetime(t *testing.T) {
if state := sibling.State(); state.Target != 1 {
t.Fatalf("measured variation downgraded an already constrained output: %+v", state)
}
+ sibling.congestionState = func() bwe.CongestionState { return bwe.CongestionStateCongested }
+ sibling.Reconcile()
+ if state := sibling.State(); state.Target != 0 {
+ t.Errorf("confirmed congestion retained an output above the unchanged budget: %+v", state)
+ }
+ sibling.congestionState = nil
+
+ // A recovered low layer does not authorize an unmeasured high layer.
+ receiver.rates = sfu.Bitrates{{50_000}, {0}, {0}}
+ receiver.available = []int32{0}
+ sibling.SetBudget(60_000)
+ if state := sibling.State(); state.Target != 0 {
+ t.Fatalf("recovery did not select the measured affordable layer: %+v", state)
+ }
+ receiver.available = []int32{0, 1, 2}
+ sibling.Reconcile()
+ if state := sibling.State(); state.Target != 0 {
+ t.Errorf("a resumed unmeasured layer bypassed the connection budget: %+v", state)
+ }
receiver.rates[1][0] = 300_000
receiver.rates[2][0] = 900_000
sibling.SetBudget(1_000_000)
if state := sibling.State(); state.Target != 2 {
t.Fatalf("increased capacity did not restore the high output: %+v", state)
}
+ // Missing rate windows retain a legal target, but cannot override source
+ // availability or subscription limits. The connection budget stays unchanged.
+ receiver.rates = sfu.Bitrates{}
+ sibling.Reconcile()
+ if state := sibling.State(); state.Target != 2 || state.VideoBudget != 1_000_000 {
+ t.Fatalf("missing rate windows changed a legal allocation: %+v", state)
+ }
+ receiver.available = []int32{0}
+ sibling.UpTrackLayersChange()
+ if state := sibling.State(); state.Target > 0 || state.VideoBudget != 1_000_000 {
+ t.Fatalf("missing rate windows retained an unavailable target: %+v", state)
+ }
+ receiver.available = []int32{0, 1, 2}
+ receiver.rates = sfu.Bitrates{{50_000}, {300_000}, {900_000}}
+ sibling.Reconcile()
+ if state := sibling.State(); state.Target != 2 {
+ t.Fatalf("restored source did not resume before subscription change: %+v", state)
+ }
+ receiver.rates = sfu.Bitrates{}
sibling.SetMaxSpatialLayer(0)
+ if state := sibling.State(); state.Target > 0 || state.VideoBudget != 1_000_000 {
+ t.Fatalf("missing rate windows retained a target above the subscription: %+v", state)
+ }
+ receiver.rates = sfu.Bitrates{{50_000}, {300_000}, {900_000}}
+ sibling.Reconcile()
if state := sibling.State(); state.Target != 0 {
t.Fatalf("stable allocation ignored a changed subscription limit: %+v", state)
}
@@ -259,10 +303,27 @@ func TestOutputUsesLibraryAllocationAndExactLifetime(t *testing.T) {
if after.Current != -1 || after.Target != before.Target || after.Requested != before.Requested {
t.Fatalf("controlled frame fence replaced its allocation: before=%+v after=%+v", before, after)
}
+ output.SetBudget(200_000)
output.Resync()
if state := output.State(); state.Target != -1 || state.Current != -1 {
t.Fatalf("source resync retained old output state: %+v", state)
}
+ output.Reconcile()
+ if state := output.State(); state.Target != 0 || state.VideoBudget != 200_000 {
+ t.Fatalf("source resync discarded the established connection budget: %+v", state)
+ }
+ receiver.available = []int32{0, 1, 2}
+ receiver.rates = sfu.Bitrates{{100_000}, {300_000}, {900_000}}
+ sibling.SetMaxSpatialLayer(2)
+ if state := sibling.State(); state.Target != 2 {
+ t.Fatalf("high layer did not resume before source resync: %+v", state)
+ }
+ sibling.Resync()
+ receiver.rates = sfu.Bitrates{{100_000}}
+ sibling.Reconcile()
+ if state := sibling.State(); state.Target != 0 {
+ t.Fatalf("source resync reused an unmeasured retired allocation: %+v", state)
+ }
output.SetBudget(1_000_000)
output.ReceiverRestart(receiver)
if state := output.State(); state.Current != -1 {
diff --git a/internal/media/forwarding/packet_buffer.go b/internal/media/forwarding/packet_buffer.go
index 12345c06..cadab3da 100644
--- a/internal/media/forwarding/packet_buffer.go
+++ b/internal/media/forwarding/packet_buffer.go
@@ -7,8 +7,8 @@ import (
"github.com/pion/transport/v5/packetio"
)
-// packetBufferFactory uses LiveKit's claimed video RTCP reader and preserves
-// stock Pion SRTP/SRTCP bounds for all remaining streams, including audio.
+// packetBufferFactory uses LiveKit's claimed DownTrack RTCP reader and preserves
+// stock Pion SRTP/SRTCP bounds for all remaining streams.
func packetBufferFactory(factory *buffer.Factory) func(packetio.BufferPacketType, uint32) io.ReadWriteCloser {
return func(kind packetio.BufferPacketType, ssrc uint32) io.ReadWriteCloser {
if kind == packetio.RTCPBufferPacket {
diff --git a/internal/media/forwarding/rtcp_reports.go b/internal/media/forwarding/rtcp_reports.go
index d2ed2e24..89d9948d 100644
--- a/internal/media/forwarding/rtcp_reports.go
+++ b/internal/media/forwarding/rtcp_reports.go
@@ -49,8 +49,8 @@ func (sender *senderReports) BindRTCPWriter(writer interceptor.RTCPWriter) inter
for _, packet := range packets {
if report, ok := packet.(*rtcp.SenderReport); ok {
if track := sender.downTrack(report.SSRC); track != nil {
- // Pion owns cadence; DownTrack owns the projected video clock and
- // its matching RR/RTT state. Audio keeps the ordinary Pion report.
+ // Pion owns cadence; DownTrack owns its projected clock and the
+ // matching RR/RTT state. Other tracks keep the Pion report.
report = track.CreateSenderReport()
if report == nil {
continue
diff --git a/internal/media/forwarding/source.go b/internal/media/forwarding/source.go
index 97d44e7a..e3242e4b 100644
--- a/internal/media/forwarding/source.go
+++ b/internal/media/forwarding/source.go
@@ -74,33 +74,46 @@ func (source *Source) RateControlled(layer int32) bool {
}
func NewSource(options SourceOptions) (*Source, error) {
- if options.ID == "" || options.StreamID == "" ||
- options.Codec.ClockRate != 90_000 ||
- (!strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeH264) &&
- !strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeVP8)) {
- return nil, errors.New("forwarded video source is invalid")
+ audio := strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeOpus) && options.Codec.ClockRate == 48_000
+ video := options.Codec.ClockRate == 90_000 && (strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeH264) ||
+ strings.EqualFold(options.Codec.MimeType, webrtc.MimeTypeVP8))
+ if options.ID == "" || options.StreamID == "" || (!audio && !video) {
+ return nil, errors.New("forwarded source is invalid")
+ }
+ minimum, defaultPackets := buffer.InitPacketBufferSizeVideo, encoded.MaxPacketWindow
+ if audio {
+ minimum, defaultPackets = buffer.InitPacketBufferSizeAudio, buffer.InitPacketBufferSizeAudio
}
if options.MaxPackets == 0 {
- options.MaxPackets = encoded.MaxPacketWindow
+ options.MaxPackets = defaultPackets
}
- if options.MaxPackets < buffer.InitPacketBufferSizeVideo || options.MaxPackets > 1<<16 {
- return nil, errors.New("forwarded video packet bound is invalid")
+ if options.MaxPackets < minimum || options.MaxPackets > 1<<16 {
+ return nil, errors.New("forwarded packet bound is invalid")
}
info := &livekit.TrackInfo{
Sid: options.ID, Type: livekit.TrackType_VIDEO, Source: livekit.TrackSource_SCREEN_SHARE,
MimeType: options.Codec.MimeType,
}
- if err := setLayerFormats(info, options.Formats); err != nil {
- return nil, err
+ kind, layers := webrtc.RTPCodecTypeVideo, len(options.Formats)
+ if audio {
+ if len(options.Formats) != 0 {
+ return nil, errors.New("forwarded audio cannot declare video layers")
+ }
+ info.Type, info.Source = livekit.TrackType_AUDIO, livekit.TrackSource_MICROPHONE
+ kind, layers = webrtc.RTPCodecTypeAudio, 1
+ } else {
+ if err := setLayerFormats(info, options.Formats); err != nil {
+ return nil, err
+ }
}
source := &Source{
ReceiverBase: sfu.NewReceiverBase(sfu.ReceiverBaseParams{
TrackID: livekit.TrackID(options.ID), StreamID: options.StreamID,
- Kind: webrtc.RTPCodecTypeVideo, Codec: options.Codec,
+ Kind: kind, Codec: options.Codec,
HeaderExtensions: canonicalHeaderExtensions(options.HeaderExtensions), Logger: diagnostics.MediaLogger("source").WithValues("trackId", diagnostics.ID(options.ID)),
StreamTrackerManagerConfig: sfu.DefaultStreamTrackerManagerConfig,
}, info, sfu.ReceiverCodecStateNormal),
- buffers: make([]*sourceBuffer, len(options.Formats)),
+ buffers: make([]*sourceBuffer, layers),
maxPackets: options.MaxPackets, onRTCP: options.OnRTCP,
}
source.currentTrackInfo.Store(info)
@@ -119,9 +132,11 @@ func (source *Source) AddDownTrack(track sfu.TrackSender) error {
if err := source.ReceiverBase.AddDownTrack(track); err != nil {
return err
}
- tracker := source.StreamTrackerManager()
- track.UpTrackMaxPublishedLayerChange(tracker.GetMaxPublishedLayer())
- track.UpTrackMaxTemporalLayerSeenChange(tracker.GetMaxTemporalLayerSeen())
+ if source.Kind() == webrtc.RTPCodecTypeVideo {
+ tracker := source.StreamTrackerManager()
+ track.UpTrackMaxPublishedLayerChange(tracker.GetMaxPublishedLayer())
+ track.UpTrackMaxTemporalLayerSeenChange(tracker.GetMaxTemporalLayerSeen())
+ }
for layer, input := range source.ReceiverBase.GetAllBuffers() {
if input != nil {
if report := input.GetSenderReportData(); report != nil {
@@ -188,7 +203,7 @@ func (source *Source) UpdateFormats(formats []LayerFormat) error {
if closed {
return sfu.ErrReceiverClosed
}
- if len(formats) != count {
+ if source.Kind() != webrtc.RTPCodecTypeVideo || len(formats) != count {
return errors.New("forwarded video layer count changed")
}
info := source.TrackInfo()
@@ -209,7 +224,7 @@ func (source *Source) UpdateDimensions(layer int, width, height uint32) error {
if source.closed {
return sfu.ErrReceiverClosed
}
- if layer < 0 || layer >= len(source.buffers) || width == 0 || height == 0 {
+ if source.Kind() != webrtc.RTPCodecTypeVideo || layer < 0 || layer >= len(source.buffers) || width == 0 || height == 0 {
return errors.New("forwarded video layer dimensions are invalid")
}
current := source.currentTrackInfo.Load()
@@ -245,16 +260,22 @@ func (source *Source) BindLayer(layer int, ssrc uint32, parameters webrtc.RTPPar
return candidate.PayloadType == codec.PayloadType && candidate.ClockRate == codec.ClockRate &&
strings.EqualFold(candidate.MimeType, codec.MimeType) && candidate.SDPFmtpLine == codec.SDPFmtpLine
}) || !slices.Equal(canonicalHeaderExtensions(parameters.HeaderExtensions), source.HeaderExtensions()) {
- return errors.New("forwarded video layer codec does not match the source")
+ return errors.New("forwarded layer codec does not match the source")
}
for _, existing := range source.buffers {
if existing != nil && existing.SSRC() == ssrc {
return errors.New("forwarded video layers cannot share an SSRC")
}
}
- input := &sourceBuffer{Buffer: buffer.NewBuffer(ssrc, source.maxPackets, buffer.InitPacketBufferSizeAudio)}
+ audioPackets, bitrate := buffer.InitPacketBufferSizeAudio, 0
+ if source.Kind() == webrtc.RTPCodecTypeAudio {
+ audioPackets = source.maxPackets
+ } else {
+ bitrate = int(source.TrackInfo().Layers[layer].Bitrate)
+ }
+ input := &sourceBuffer{Buffer: buffer.NewBuffer(ssrc, source.maxPackets, audioPackets)}
input.SetLogger(diagnostics.MediaLogger("rtp-buffer").WithValues("trackId", diagnostics.ID(string(source.TrackID()))))
- if err := input.Bind(parameters, codec.RTPCodecCapability, int(source.TrackInfo().Layers[layer].Bitrate)); err != nil {
+ if err := input.Bind(parameters, codec.RTPCodecCapability, bitrate); err != nil {
_ = input.Close()
return err
}
diff --git a/internal/media/inputroute/relay.go b/internal/media/inputroute/relay.go
index 6c3fb0d0..d172fad1 100644
--- a/internal/media/inputroute/relay.go
+++ b/internal/media/inputroute/relay.go
@@ -14,6 +14,10 @@ const Label = "piik-remote-input-prototype-v1"
// 2 KiB of text, JSON escaping, authenticated encryption and the relay envelope.
const MaxPacket = 6144
+// Allow route overlap for the room's 20 endpoints while bounding retirement
+// metadata. Epochs remain opaque here; only the Host knows which is authorized.
+const maxTrackedEpochs = 128
+
// Packet data is authenticated and encrypted between the Viewer and Host App.
// Relays only learn endpoint IDs. Small Host state replies fan out along the
// existing tree; only the named endpoint can read them.
@@ -27,7 +31,7 @@ type Packet struct {
type leg struct {
channel *webrtc.DataChannel
- endpoints map[string]string
+ endpoints map[string]map[string]struct{}
}
type Relay struct {
@@ -36,11 +40,12 @@ type Relay struct {
down map[string]*leg
ready bool
closed bool
- // Root callbacks run outside mu. The root validates endpoint authority.
- receive func(edgeID string, packet Packet)
+ // Root callbacks run outside mu. The root checks the delivery binding while
+ // holding its permission lock, so retirement can invalidate an already-read packet.
+ receive func(edgeID string, packet Packet, current func() bool)
}
-func New(receive func(string, Packet)) *Relay {
+func New(receive func(string, Packet, func() bool)) *Relay {
return &Relay{down: make(map[string]*leg), receive: receive, ready: receive != nil}
}
@@ -108,20 +113,21 @@ func (r *Relay) SetUpAvailable(available bool) {
}
}
-func (r *Relay) AttachDown(edgeID string, channel *webrtc.DataChannel) {
+// AttachDown returns transient-loss cleanup tied to this exact physical channel.
+func (r *Relay) AttachDown(edgeID string, channel *webrtc.DataChannel) func() {
r.mu.Lock()
// SCTP can close before its OnClose callback retires the old binding.
if old := r.down[edgeID]; !r.closed && old != nil && old.channel.ReadyState() == webrtc.DataChannelStateClosed {
r.mu.Unlock()
- r.LoseDown(edgeID, old.channel)
+ r.loseDown(edgeID, old.channel, true)
r.mu.Lock()
}
if r.closed || r.down[edgeID] != nil {
r.mu.Unlock()
_ = channel.Close()
- return
+ return func() {}
}
- child := &leg{channel: channel, endpoints: make(map[string]string)}
+ child := &leg{channel: channel, endpoints: make(map[string]map[string]struct{})}
r.down[edgeID] = child
r.mu.Unlock()
channel.OnOpen(func() { r.mu.Lock(); defer r.mu.Unlock(); send(channel, Packet{Ready: &r.ready}) })
@@ -131,7 +137,8 @@ func (r *Relay) AttachDown(edgeID string, channel *webrtc.DataChannel) {
return
}
r.mu.Lock()
- if r.closed || r.down[edgeID] != child {
+ child := r.down[edgeID]
+ if r.closed || child == nil || child.channel != channel {
r.mu.Unlock()
return
}
@@ -145,15 +152,35 @@ func (r *Relay) AttachDown(edgeID string, channel *webrtc.DataChannel) {
return
}
if p.Closed {
- if child.endpoints[p.ID] == p.Epoch {
+ delete(child.endpoints[p.ID], p.Epoch)
+ if len(child.endpoints[p.ID]) == 0 {
delete(child.endpoints, p.ID)
}
} else {
- if child.endpoints[p.ID] == "" && len(child.endpoints) >= 20 {
+ epochs := child.endpoints[p.ID]
+ if epochs == nil && len(child.endpoints) >= 20 {
r.mu.Unlock()
return
}
- child.endpoints[p.ID] = p.Epoch
+ if _, known := epochs[p.Epoch]; !known {
+ count := 0
+ for _, tracked := range child.endpoints {
+ count += len(tracked)
+ }
+ if count >= maxTrackedEpochs {
+ r.mu.Unlock()
+ r.loseDown(edgeID, channel, true)
+ _ = channel.Close()
+ return
+ }
+ if epochs == nil {
+ epochs = make(map[string]struct{})
+ child.endpoints[p.ID] = epochs
+ }
+ // A late packet from an old branch must not erase another epoch
+ // whose grant the Host may still need to retire on branch loss.
+ epochs[p.Epoch] = struct{}{}
+ }
}
if r.receive == nil {
send(r.up, p)
@@ -161,27 +188,49 @@ func (r *Relay) AttachDown(edgeID string, channel *webrtc.DataChannel) {
return
}
r.mu.Unlock()
- r.receive(edgeID, p)
+ r.receive(edgeID, p, func() bool {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ return !r.closed && r.down[edgeID] == child && channel.ReadyState() == webrtc.DataChannelStateOpen
+ })
})
- channel.OnClose(func() { r.LoseDown(edgeID, channel) })
+ channel.OnClose(func() { r.loseDown(edgeID, channel, true) })
+ return func() { r.loseDown(edgeID, channel, false) }
}
func boolPointer(value bool) *bool { return &value }
-func (r *Relay) LoseDown(edgeID string, channel *webrtc.DataChannel) {
+// ResetDown invalidates in-flight deliveries from this exact input channel.
+// The permission owner calls this under its own lock and retires its grants in
+// the same transaction; this method must not call back into that owner.
+func (r *Relay) ResetDown(edgeID string, channel *webrtc.DataChannel) bool {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ child := r.down[edgeID]
+ if r.closed || child == nil || child.channel != channel {
+ return false
+ }
+ r.down[edgeID] = &leg{channel: channel, endpoints: make(map[string]map[string]struct{})}
+ return true
+}
+
+func (r *Relay) loseDown(edgeID string, channel *webrtc.DataChannel, remove bool) {
r.mu.Lock()
child := r.down[edgeID]
- if child == nil || channel != nil && child.channel != channel {
+ if child == nil || child.channel != channel {
r.mu.Unlock()
return
}
packets := make([]Packet, 0, len(child.endpoints))
- for id, epoch := range child.endpoints {
- packets = append(packets, Packet{ID: id, Epoch: epoch, Closed: true})
+ for id, epochs := range child.endpoints {
+ for epoch := range epochs {
+ packets = append(packets, Packet{ID: id, Epoch: epoch, Closed: true})
+ }
}
- clear(child.endpoints)
- if channel != nil {
+ if remove {
delete(r.down, edgeID)
+ } else {
+ r.down[edgeID] = &leg{channel: child.channel, endpoints: make(map[string]map[string]struct{})}
}
if r.receive == nil {
for _, p := range packets {
@@ -191,7 +240,7 @@ func (r *Relay) LoseDown(edgeID string, channel *webrtc.DataChannel) {
r.mu.Unlock()
if r.receive != nil {
for _, p := range packets {
- r.receive(edgeID, p)
+ r.receive(edgeID, p, nil)
}
}
}
diff --git a/internal/media/inputroute/relay_test.go b/internal/media/inputroute/relay_test.go
index f4ec6895..74076d0f 100644
--- a/internal/media/inputroute/relay_test.go
+++ b/internal/media/inputroute/relay_test.go
@@ -1,6 +1,7 @@
package inputroute
import (
+ "fmt"
"strings"
"sync"
"testing"
@@ -11,10 +12,10 @@ import (
func TestClosedDownstreamBindingAllowsReplacement(t *testing.T) {
received := make(chan Packet, 10)
- relay := New(func(_ string, p Packet) { received <- p })
+ relay := New(func(_ string, p Packet, _ func() bool) { received <- p })
defer relay.Close()
oldSend, oldChannel := inputChannelPair(t)
- relay.AttachDown("same-edge", oldChannel)
+ oldLoss := relay.AttachDown("same-edge", oldChannel)
oldPacket := Packet{ID: strings.Repeat("a", 32), Epoch: strings.Repeat("b", 32), Data: "old"}
send(oldSend, oldPacket)
if p := receiveInputPacket(t, received); p != oldPacket {
@@ -27,7 +28,7 @@ func TestClosedDownstreamBindingAllowsReplacement(t *testing.T) {
// Hold the actual SCTP callback after its channel has entered Closed.
oldChannel.OnClose(func() {
<-released
- relay.LoseDown("same-edge", oldChannel)
+ relay.loseDown("same-edge", oldChannel, true)
close(closed)
})
if err := oldSend.Close(); err != nil {
@@ -55,12 +56,135 @@ func TestClosedDownstreamBindingAllowsReplacement(t *testing.T) {
t.Fatal("old OnClose did not finish")
}
checkReplacement() // Late cleanup must not detach the replacement.
+ oldLoss()
+ checkReplacement() // A queued ICE loss also belongs only to the old channel.
_, duplicate := inputChannelPair(t)
relay.AttachDown("same-edge", duplicate)
waitInputChannel(t, "live duplicate rejected", func() bool { return duplicate.ReadyState() == webrtc.DataChannelStateClosed })
checkReplacement()
}
+func TestBranchLossRetiresEveryUnclosedEpoch(t *testing.T) {
+ for _, oldClosed := range []bool{false, true} {
+ t.Run(map[bool]string{false: "late-old-data", true: "late-old-closure"}[oldClosed], func(t *testing.T) {
+ received := make(chan Packet, 10)
+ relay := New(func(_ string, p Packet, _ func() bool) { received <- p })
+ defer relay.Close()
+ viewer, branch := inputChannelPair(t)
+ loseBranch := relay.AttachDown("shared-branch", branch)
+ old := Packet{ID: strings.Repeat("a", 32), Epoch: strings.Repeat("b", 32), Data: "old"}
+ current := Packet{ID: old.ID, Epoch: strings.Repeat("c", 32), Data: "current"}
+ // Two routes may converge on this leg. The old longer route's last
+ // packet can follow a newer epoch without becoming current at Host.
+ for _, p := range []Packet{old, current, old} {
+ send(viewer, p)
+ if got := receiveInputPacket(t, received); got != p {
+ t.Fatalf("relayed packet = %+v, want %+v", got, p)
+ }
+ }
+ if oldClosed {
+ p := Packet{ID: old.ID, Epoch: old.Epoch, Closed: true}
+ send(viewer, p)
+ if got := receiveInputPacket(t, received); got != p {
+ t.Fatalf("old closure = %+v", got)
+ }
+ }
+ loseBranch()
+ retired := map[string]bool{}
+ for len(received) > 0 {
+ p := <-received
+ if p.ID != current.ID || !p.Closed {
+ t.Fatalf("branch retirement = %+v", p)
+ }
+ retired[p.Epoch] = true
+ }
+ if !retired[current.Epoch] || retired[old.Epoch] == oldClosed {
+ t.Fatalf("retired epochs = %v, old already closed = %v", retired, oldClosed)
+ }
+ })
+ }
+}
+
+func TestEpochTrackingLimitRetiresInputWithoutClosingTransport(t *testing.T) {
+ received := make(chan Packet, maxTrackedEpochs+2)
+ relay := New(func(_ string, p Packet, _ func() bool) { received <- p })
+ defer relay.Close()
+ viewer, branch := inputChannelPair(t)
+ relay.AttachDown("branch", branch)
+ tracked := map[string]Packet{}
+ for i := range maxTrackedEpochs {
+ p := Packet{ID: fmt.Sprintf("%032x", i%20), Epoch: fmt.Sprintf("%032x", i), Data: "input"}
+ send(viewer, p)
+ if got := receiveInputPacket(t, received); got != p {
+ t.Fatalf("packet = %+v, want %+v", got, p)
+ }
+ tracked[p.Epoch] = p
+ }
+ // At capacity, an existing epoch still carries input. A Closed packet frees
+ // its exact record, without retiring another epoch of the same endpoint.
+ p := tracked[fmt.Sprintf("%032x", 0)]
+ send(viewer, p)
+ if got := receiveInputPacket(t, received); got != p {
+ t.Fatalf("existing epoch at capacity = %+v", got)
+ }
+ send(viewer, Packet{ID: p.ID, Epoch: p.Epoch, Closed: true})
+ if got := receiveInputPacket(t, received); got.Epoch != p.Epoch || !got.Closed {
+ t.Fatalf("closure at capacity = %+v", got)
+ }
+ delete(tracked, p.Epoch)
+ p.Epoch = fmt.Sprintf("%032x", maxTrackedEpochs)
+ send(viewer, p)
+ if got := receiveInputPacket(t, received); got != p {
+ t.Fatalf("replacement at capacity = %+v", got)
+ }
+ tracked[p.Epoch] = p
+ p.Epoch = fmt.Sprintf("%032x", maxTrackedEpochs+1)
+ send(viewer, p)
+ for range maxTrackedEpochs {
+ closed := receiveInputPacket(t, received)
+ known, ok := tracked[closed.Epoch]
+ if !ok || !closed.Closed || closed.ID != known.ID {
+ t.Fatalf("overflow forwarded new input or lost known retirement: %+v", closed)
+ }
+ delete(tracked, closed.Epoch)
+ }
+ waitInputChannel(t, "overflow closes input channel", func() bool { return branch.ReadyState() == webrtc.DataChannelStateClosed })
+ if branch.Transport().State() != webrtc.SCTPTransportStateConnected {
+ t.Fatal("input metadata limit closed the shared transport")
+ }
+}
+
+func TestResetDownInvalidatesReadBindingAcrossSameEpoch(t *testing.T) {
+ permits := make(chan func() bool, 4)
+ relay := New(func(_ string, _ Packet, current func() bool) { permits <- current })
+ defer relay.Close()
+ viewer, branch := inputChannelPair(t)
+ relay.AttachDown("branch", branch)
+ p := Packet{ID: strings.Repeat("a", 32), Epoch: strings.Repeat("b", 32), Data: "input"}
+ readPermit := func() func() bool {
+ t.Helper()
+ send(viewer, p)
+ select {
+ case current := <-permits:
+ return current
+ case <-time.After(2 * time.Second):
+ t.Fatal("missing root delivery")
+ return nil
+ }
+ }
+ old := readPermit()
+ if !old() || !relay.ResetDown("branch", branch) {
+ t.Fatal("current input binding was unavailable")
+ }
+ current := readPermit() // The same epoch can send a fresh request after ICE recovery.
+ if old() || !current() {
+ t.Fatal("re-registering the same epoch revived a pre-loss delivery")
+ }
+ if relay.ResetDown("branch", viewer) || !current() {
+ t.Fatal("another physical channel retired the current binding")
+ }
+}
+
func receiveInputPacket(t *testing.T, packets <-chan Packet) Packet {
t.Helper()
select {
diff --git a/internal/media/inputroute/transport.go b/internal/media/inputroute/transport.go
new file mode 100644
index 00000000..ad865d1c
--- /dev/null
+++ b/internal/media/inputroute/transport.go
@@ -0,0 +1,17 @@
+package inputroute
+
+import "github.com/pion/webrtc/v4"
+
+// ObserveTransport binds the input lifetime to Pion's ordered ICE notifications.
+// It owns the transport's public observer; Pion's internal PeerConnection observer
+// remains separate. Bind before negotiation, not from asynchronous PC projections.
+func ObserveTransport(transport *webrtc.ICETransport, changed func(bool)) {
+ transport.OnConnectionStateChange(func(state webrtc.ICETransportState) {
+ switch state {
+ case webrtc.ICETransportStateConnected, webrtc.ICETransportStateCompleted:
+ changed(true)
+ case webrtc.ICETransportStateChecking, webrtc.ICETransportStateDisconnected, webrtc.ICETransportStateFailed, webrtc.ICETransportStateClosed:
+ changed(false)
+ }
+ })
+}
diff --git a/internal/server/app/app.go b/internal/server/app/app.go
index bb38ad5c..e844cacf 100644
--- a/internal/server/app/app.go
+++ b/internal/server/app/app.go
@@ -84,6 +84,8 @@ type Server struct {
logger *slog.Logger
media *sfu.Media
mediaMux ice.UDPMux
+ sfuPublicIP atomic.Pointer[string]
+ stopSFUDNS func()
signalOptions signal.Options
httpServer *http.Server
// Listen binds unless New received owned listeners. Shutdown waits for
@@ -232,14 +234,13 @@ func (s *Server) Listen(ctx context.Context) (int, error) {
}
func (s *Server) start(ctx context.Context) (int, error) {
- var publicIP string
if s.config.SFU != nil {
- var err error
- publicIP, err = resolveSFUPublicIP(ctx, s.config.SFU.PublicIP)
+ publicIP, err := resolveSFUPublicIP(ctx, s.config.SFU.PublicIP)
if err != nil {
_ = s.stopServing(ctx)
return 0, err
}
+ s.sfuPublicIP.Store(&publicIP)
}
// Bind first. A listener owned by another process must fail before
// the room database is opened.
@@ -276,10 +277,14 @@ func (s *Server) start(ctx context.Context) (int, error) {
settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
settings.SetICEUDPMux(s.mediaMux)
settings.SetIncludeLoopbackCandidate(net.ParseIP(configuration.ListenHost).IsLoopback())
- if publicIP != "" {
- settings.SetNAT1To1IPs([]string{publicIP}, webrtc.ICECandidateTypeHost)
- }
- s.media = sfu.NewMedia(sfu.MediaOptions{Settings: settings, Events: func(event sfu.MediaEvent) {
+ s.media = sfu.NewMedia(sfu.MediaOptions{Settings: func() webrtc.SettingEngine {
+ // A Pion API owns its settings snapshot for its entire lifetime.
+ snapshot := settings
+ if address := *s.sfuPublicIP.Load(); address != "" {
+ snapshot.SetNAT1To1IPs([]string{address}, webrtc.ICECandidateTypeHost)
+ }
+ return snapshot
+ }, Events: func(event sfu.MediaEvent) {
if signaling := s.signaling.Load(); signaling != nil {
signaling.HandleSfuMediaEvent(event)
}
@@ -287,6 +292,7 @@ func (s *Server) start(ctx context.Context) (int, error) {
s.signalOptions.SfuFallback = &signal.SfuFallback{Media: s.media, Admission: sfu.NewAdmission(sfu.AdmissionOptions{
IngressCapacity: room.Capacity, EgressCapacity: room.Capacity * s.config.MaxViewersPerRoom,
})}
+ s.startSFUDNSRefresh(configuration.PublicIP)
}
for _, listener := range s.listeners {
go func() {
@@ -332,6 +338,10 @@ func (s *Server) start(ctx context.Context) (int, error) {
// listener afterwards makes the port release synchronous. A second close of an
// already closed HTTP listener is harmless.
func (s *Server) stopServing(ctx context.Context) error {
+ if s.stopSFUDNS != nil {
+ s.stopSFUDNS()
+ s.stopSFUDNS = nil
+ }
err := s.httpServer.Shutdown(ctx)
for _, listener := range s.listeners {
_ = listener.Close()
diff --git a/internal/server/app/sfu_address.go b/internal/server/app/sfu_address.go
index d39efa66..2f105849 100644
--- a/internal/server/app/sfu_address.go
+++ b/internal/server/app/sfu_address.go
@@ -6,10 +6,11 @@ import (
"net"
"net/netip"
"time"
+
+ "github.com/TNTcraftHIM/Piik/internal/diagnostics"
)
-// Pion advertises literal candidates; a configured DDNS name is resolved once
-// for this server lifetime, just like an explicitly configured public IPv4.
+// Pion advertises literal candidates, so DNS names resolve before creating peers.
func resolveSFUPublicIP(ctx context.Context, host string) (string, error) {
if host == "" {
return "", nil
@@ -28,3 +29,39 @@ func resolveSFUPublicIP(ctx context.Context, host string) (string, error) {
}
return addresses[0].Unmap().String(), nil
}
+
+func (s *Server) startSFUDNSRefresh(host string) {
+ if host == "" {
+ return
+ }
+ if _, err := netip.ParseAddr(host); err == nil {
+ return
+ }
+ ctx, cancel := context.WithCancel(context.Background())
+ done := make(chan struct{})
+ s.stopSFUDNS = func() { cancel(); <-done }
+ go func() {
+ defer close(done)
+ ticker := time.NewTicker(time.Minute)
+ defer ticker.Stop()
+ for {
+ select {
+ case <-ctx.Done():
+ return
+ case <-ticker.C:
+ s.refreshSFUPublicIP(ctx, host)
+ }
+ }
+ }()
+}
+
+func (s *Server) refreshSFUPublicIP(ctx context.Context, host string) {
+ address, err := resolveSFUPublicIP(ctx, host)
+ if err != nil {
+ if ctx.Err() == nil {
+ s.logger.Debug("sfu-dns-refresh-failed", diagnostics.Error(err))
+ }
+ return
+ }
+ s.sfuPublicIP.Store(&address)
+}
diff --git a/internal/server/app/sfu_address_test.go b/internal/server/app/sfu_address_test.go
index f0d91b2c..aae7f847 100644
--- a/internal/server/app/sfu_address_test.go
+++ b/internal/server/app/sfu_address_test.go
@@ -5,13 +5,123 @@ import (
"net"
"net/netip"
"strings"
+ "sync/atomic"
"testing"
+ "time"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
"github.com/TNTcraftHIM/Piik/internal/server/config"
"github.com/TNTcraftHIM/Piik/internal/server/sfu"
+ "github.com/pion/ice/v4"
+ "golang.org/x/net/dns/dnsmessage"
)
+func TestSFUPublicDNSRefreshChangesNewVoiceCandidates(t *testing.T) {
+ dns, err := net.ListenPacket("udp4", "127.0.0.1:0")
+ if err != nil {
+ t.Fatal(err)
+ }
+ var address atomic.Uint32
+ address.Store(5)
+ done := make(chan struct{})
+ go func() {
+ defer close(done)
+ var buffer [1500]byte
+ for {
+ n, peer, err := dns.ReadFrom(buffer[:])
+ if err != nil {
+ return
+ }
+ var query dnsmessage.Message
+ if query.Unpack(buffer[:n]) != nil {
+ continue
+ }
+ reply := dnsmessage.Message{Header: dnsmessage.Header{ID: query.ID, Response: true, RecursionDesired: query.RecursionDesired, RecursionAvailable: true}, Questions: query.Questions}
+ last := address.Load()
+ if last == 0 {
+ reply.RCode = dnsmessage.RCodeNameError
+ } else {
+ for _, question := range query.Questions {
+ if question.Type == dnsmessage.TypeA {
+ reply.Answers = append(reply.Answers, dnsmessage.Resource{
+ Header: dnsmessage.ResourceHeader{Name: question.Name, Type: dnsmessage.TypeA, Class: dnsmessage.ClassINET},
+ Body: &dnsmessage.AResource{A: [4]byte{198, 51, 100, byte(last)}},
+ })
+ }
+ }
+ }
+ if packet, err := reply.Pack(); err == nil {
+ _, _ = dns.WriteTo(packet, peer)
+ }
+ }
+ }()
+ t.Cleanup(func() { _ = dns.Close(); <-done })
+ resolver := net.DefaultResolver
+ net.DefaultResolver = &net.Resolver{PreferGo: true, Dial: func(ctx context.Context, _, _ string) (net.Conn, error) {
+ return (&net.Dialer{}).DialContext(ctx, "udp4", dns.LocalAddr().String())
+ }}
+ t.Cleanup(func() { net.DefaultResolver = resolver })
+
+ configuration := testConfig(t)
+ configuration.SFU = &config.SFUConfig{ListenHost: "127.0.0.1", Port: 0, PublicIP: "sfu-refresh.test."}
+ server := start(t, Options{Config: configuration})
+ candidate := func(connection string) ice.Candidate {
+ t.Helper()
+ candidates := make(chan string, 1)
+ plan := sfu.VoiceConnectionPlan{RoomID: "1234", PeerID: connection, SessionID: connection, ConnectionID: connection,
+ Publish: []sfu.VoicePublication{{ID: connection, PeerID: connection}}}
+ work, err := server.media.ConfigureVoice(plan, func(event sfu.VoiceMediaEvent) {
+ if event.Candidate != nil {
+ select {
+ case candidates <- event.Candidate.Candidate:
+ default:
+ }
+ }
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := work(); err != nil {
+ t.Fatal(err)
+ }
+ select {
+ case value := <-candidates:
+ parsed, err := ice.UnmarshalCandidate(value)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return parsed
+ case <-time.After(5 * time.Second):
+ t.Fatal("voice connection did not gather a real ICE candidate")
+ return nil
+ }
+ }
+ initial := candidate("initial")
+ if initial.Address() != "198.51.100.5" || initial.Port() == 0 {
+ t.Fatalf("initial DNS candidate: %s", initial)
+ }
+ address.Store(9)
+ server.refreshSFUPublicIP(t.Context(), configuration.SFU.PublicIP)
+ updated := candidate("updated")
+ if updated.Address() != "198.51.100.9" || updated.Port() != initial.Port() {
+ t.Fatalf("new connection must use refreshed IP on the same UDP listener: initial=%s updated=%s", initial, updated)
+ }
+ address.Store(0)
+ server.refreshSFUPublicIP(t.Context(), configuration.SFU.PublicIP)
+ retained := candidate("dns-failed")
+ if retained.Address() != updated.Address() || retained.Port() != updated.Port() {
+ t.Fatalf("DNS failure lost the last usable address: updated=%s retained=%s", updated, retained)
+ }
+ ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+ defer cancel()
+ if err := server.Close(ctx); err != nil {
+ t.Fatal(err)
+ }
+ if server.stopSFUDNS != nil {
+ t.Fatal("server shutdown did not retire its DNS refresh worker")
+ }
+}
+
func TestSFUPublicHostnameStartsTheExistingMediaRuntime(t *testing.T) {
configuration := testConfig(t)
configuration.SFU = &config.SFUConfig{ListenHost: "127.0.0.1", Port: 0, PublicIP: "localhost"}
diff --git a/internal/server/route/api.go b/internal/server/route/api.go
index 48e927eb..5e04315c 100644
--- a/internal/server/route/api.go
+++ b/internal/server/route/api.go
@@ -453,11 +453,20 @@ type SfuPublisherQualityEvidenceInput struct {
AcceptedAtMs int64
}
+// RevisionSequence belongs to the room, including across controller replacement.
+// Its zero value is ready to use; the caller serializes access with controller calls.
+type RevisionSequence struct {
+ latest int64
+}
+
// Options configures a Controller. DebugRoomID empty disables the sanitised
// route debug events.
type Options struct {
HostPeerID string
DebugRoomID string
+ // Revisions may be shared by successive controllers for the same room.
+ // nil gives this controller its own sequence.
+ Revisions *RevisionSequence
// DebugLog receives sanitized route events. nil disables them; the effect
// layer owns environment, logger and retention policy.
DebugLog func(string, ...any)
@@ -492,7 +501,7 @@ type Controller struct {
hostPublication *HostPublication
operation *operation
revision int64
- latestRevision int64
+ revisions *RevisionSequence
factVersion int64
nextJoinOrder int64
paused bool
diff --git a/internal/server/route/candidates.go b/internal/server/route/candidates.go
index 585d576c..df086750 100644
--- a/internal/server/route/candidates.go
+++ b/internal/server/route/candidates.go
@@ -568,7 +568,7 @@ func endpointTransitionEquals(left, right EndpointTransition) bool {
// sfu-reuse edge, which belongs to another holder), publication, overlap.
func reservationResources(reservation CandidateReservation) []*Resource {
resources := []*Resource{}
- if reservation.Kind != ReservationDirect && !(reservation.Kind == ReservationSfuReuse && reservation.Borrowed) {
+ if reservation.Kind == ReservationSfuCreate || reservation.Kind == ReservationSfuReuse && !reservation.Borrowed {
resources = append(resources, reservation.Edge)
}
if reservation.Kind == ReservationSfuCreate {
diff --git a/internal/server/route/controller.go b/internal/server/route/controller.go
index b1495ab6..53fefd3a 100644
--- a/internal/server/route/controller.go
+++ b/internal/server/route/controller.go
@@ -270,6 +270,9 @@ func New(options Options) *Controller {
if options.SfuOnly && !options.SfuEnabled {
panic(errors.New("SFU-only routing requires SFU availability"))
}
+ if options.Revisions == nil {
+ options.Revisions = &RevisionSequence{}
+ }
c := &Controller{
hostPeerID: options.HostPeerID,
debugRoomID: options.DebugRoomID,
@@ -280,6 +283,7 @@ func New(options Options) *Controller {
sfuOnly: options.SfuOnly,
qualityConvergenceEnabled: options.QualityConvergenceEnabled && !options.SfuOnly,
natPredictionEnabled: options.NatPredictionEnabled && !options.SfuOnly,
+ revisions: options.Revisions,
routeTimings: make(map[string]*routeTimingRecord),
qualityObservations: make(map[string]*routeQualityObservation),
senderQualityObservations: make(map[string]*senderQualityObservation),
@@ -894,7 +898,7 @@ func (c *Controller) Dispose() []*Resource {
c.sfuBootstrapIntent = nil
c.consumedSfuBootstrapOpportunities.Clear()
c.rootConvergenceRootPeerID = ""
- // nextJoinOrder, revision, latestRevision and factVersion survive.
+ // nextJoinOrder, revision, the room's revision sequence and factVersion survive.
c.paused = true
return resources.resources()
}
@@ -950,11 +954,11 @@ func (c *Controller) touchFacts() { c.factVersion++ }
// allocateRevision bounds revisions by MAX_SAFE_INTEGER.
func (c *Controller) allocateRevision() int64 {
- if c.latestRevision >= protocol.MaxMediaRouteRevision {
+ if c.revisions.latest >= protocol.MaxMediaRouteRevision {
panic(errors.New("Media route revision space exhausted"))
}
- c.latestRevision++
- return c.latestRevision
+ c.revisions.latest++
+ return c.revisions.latest
}
// isSafeInteger keeps values exactly representable by the Browser's number type.
diff --git a/internal/server/route/operation.go b/internal/server/route/operation.go
index 5fee4cbb..1c1f5ce8 100644
--- a/internal/server/route/operation.go
+++ b/internal/server/route/operation.go
@@ -984,7 +984,11 @@ func (c *Controller) commitAttempt(op *operation, att *attempt) []*Resource {
ConnectionID: att.connectionID,
Usable: true,
PhysicalActive: true,
- opportunityKey: c.candidateOpportunityBase(op.childPeerID, CandidatePlan{Tuple: att.tuple}),
+ // Regenerate distinguishes quality trials, not the acquired route's
+ // availability opportunity if this committed connection later fails.
+ opportunityKey: c.candidateOpportunityBase(op.childPeerID, CandidatePlan{Tuple: CandidateTuple{
+ Kind: UpstreamPeer, ParentPeerID: att.tuple.ParentPeerID,
+ }}),
})
c.directContinuations.Delete(op.childPeerID)
} else {
diff --git a/internal/server/route/publication_test.go b/internal/server/route/publication_test.go
index 54905416..d269bb4a 100644
--- a/internal/server/route/publication_test.go
+++ b/internal/server/route/publication_test.go
@@ -150,6 +150,38 @@ func TestStopCancelsAttachmentAndRejectsLateFrame(t *testing.T) {
eq(t, edgeOf(t, routes, B).ConnectionID, "voice-b")
}
+func TestCanceledPublicationPreparationReleasesNoRoomResources(t *testing.T) {
+ for _, change := range []string{"stop", "replacement", "pause", "child-session", "parent-session"} {
+ t.Run(change, func(t *testing.T) {
+ routes := voiceTree(t)
+ routes.SetVideoPublication("share", false, 10)
+ op := must(t, routes.Reconcile(11).Operation)
+ eq(t, op.Reason, DemandPublicationAttach)
+ // The effect layer prepares this optional facet without a reservation,
+ // then yields before handing the guarded result back to the controller.
+ prepared := BeginInput{Guard: cursorGuard(op), NowMs: 13, ConnectionID: "late-video"}
+ switch change {
+ case "stop":
+ routes.SetVideoPublication("", false, 12)
+ case "replacement":
+ routes.SetVideoPublication("replacement-share", false, 12)
+ case "pause":
+ routes.SetVideoPublication("share", true, 12)
+ case "child-session":
+ routes.UpsertParticipant(viewerInput(A, "replacement-child", 1), ms(12))
+ case "parent-session":
+ routes.UpsertParticipant(hostInput("replacement-parent", 1), ms(12))
+ }
+ result := routes.BeginCurrentCandidate(prepared)
+ eq(t, result.Accepted, false)
+ eq(t, len(result.Released), 0)
+ eq(t, edgeOf(t, routes, A).ConnectionID, "voice-a")
+ eq(t, edgeOf(t, routes, B).ConnectionID, "voice-b")
+ eq(t, edgeOf(t, routes, A).Usable && edgeOf(t, routes, B).Usable, true)
+ })
+ }
+}
+
func TestVoiceOptInRetainsAlreadyProvenVideo(t *testing.T) {
routes := newController(1, Options{})
addViewer(routes, A, 0, ms(1))
diff --git a/internal/server/route/session_opportunity_test.go b/internal/server/route/session_opportunity_test.go
index 599dc9fa..03cd4d86 100644
--- a/internal/server/route/session_opportunity_test.go
+++ b/internal/server/route/session_opportunity_test.go
@@ -120,6 +120,43 @@ func TestRebuiltDirectConnectionConsumesCurrentSessionOpportunity(t *testing.T)
eq(t, hasTuple(must(t, routes.Reconcile(4).Operation), isPeer), false)
}
+func TestRegeneratedEdgeFailureConsumesOnlyItsAcquiredSessionOpportunity(t *testing.T) {
+ for _, prediction := range []bool{false, true} {
+ for _, changed := range []string{"none", "child", "parent"} {
+ t.Run(fmt.Sprintf("nat=%t/session=%s", prediction, changed), func(t *testing.T) {
+ routes := newController(2, Options{SfuEnabled: true, QualityConvergenceEnabled: true, NatPredictionEnabled: prediction})
+ addViewer(routes, A, 0, nil)
+ routes.Reconcile(0)
+ commitCurrent(t, routes, 1, "original")
+ observePersistentDegraded(t, routes, A, "original", 10)
+ op := must(t, routes.Reconcile(14).Operation)
+ eq(t, op.Reason, DemandQualityConvergence)
+ eq(t, op.Candidates[0].Tuple, regenerateTuple(HOST))
+ prepared := beginOperation(t, routes, BeginInput{NowMs: 15, ConnectionID: "regenerated", Reservation: direct()})
+ guard := guardFor(A, prepared.Current.Revision, "regenerated")
+ eq(t, routes.CandidateReady(guard, 16, nil, CandidateProof{RelativeQualityApproved: true}).Committed, false)
+ eq(t, routes.ObserveSenderQualityEvidence(senderSample(HOST, A, prepared.Current.Revision,
+ "regenerated", "regenerated-sender", SenderQualityHealthy, 17), nil).Committed, true)
+
+ switch changed {
+ case "child":
+ routes.UpsertParticipant(viewerInput(A, "replacement-child", 0), ms(18))
+ case "parent":
+ routes.UpsertParticipant(hostInput("replacement-parent", 2), ms(18))
+ }
+ edge := edgeOf(t, routes, A)
+ eq(t, routes.InvalidateEdge(EdgeGuard{
+ ChildPeerID: A, ChildSessionID: edge.ChildSessionID, ParentSessionID: edge.ParentSessionID,
+ RouteRevision: routes.Revision(), ConnectionID: edge.ConnectionID,
+ }, ms(19)), true)
+ repair := must(t, routes.Reconcile(20).Operation)
+ eq(t, hasTuple(repair, isPeer), changed != "none")
+ eq(t, hasTuple(repair, isSfu), true)
+ })
+ }
+ }
+}
+
func TestFreshRelayOpportunityBehindHealthySfu(t *testing.T) {
for _, voiceOnly := range []bool{false, true} {
for _, nat := range []bool{false, true} {
diff --git a/internal/server/sfu/media.go b/internal/server/sfu/media.go
index 04d67baa..0cf5b91d 100644
--- a/internal/server/sfu/media.go
+++ b/internal/server/sfu/media.go
@@ -35,8 +35,9 @@ type MediaEvent struct {
}
type MediaOptions struct {
- // Settings can contain the application-owned UDP mux. Media never closes it.
- Settings webrtc.SettingEngine
+ // Settings returns a fresh, nonblocking snapshot for each peer. It may
+ // contain the application-owned UDP mux, which Media never closes.
+ Settings func() webrtc.SettingEngine
// Events run without media locks and hand exact handles back to room authority.
// Physical close must be scheduled asynchronously from these callbacks.
Events func(MediaEvent)
@@ -116,6 +117,9 @@ type mediaSubscription struct {
}
func NewMedia(options MediaOptions) *Media {
+ if options.Settings == nil {
+ options.Settings = func() webrtc.SettingEngine { return webrtc.SettingEngine{} }
+ }
return &Media{options: options, publications: make(map[ResourceFence]*mediaPublication), voice: newVoiceMedia(options.Settings)}
}
@@ -169,11 +173,11 @@ func (media *Media) PreparePublication(fence ResourceFence, connectionID string,
return err
}
registry := &interceptor.Registry{}
- if err := webrtc.RegisterDefaultInterceptors(engine, registry); err != nil {
+ if err := forwarding.RegisterIngressInterceptors(engine, registry); err != nil {
return err
}
pc, err := webrtc.NewAPI(webrtc.WithMediaEngine(engine), webrtc.WithInterceptorRegistry(registry),
- webrtc.WithSettingEngine(media.options.Settings)).NewPeerConnection(webrtc.Configuration{})
+ webrtc.WithSettingEngine(media.options.Settings())).NewPeerConnection(webrtc.Configuration{})
if err != nil {
return err
}
@@ -213,19 +217,6 @@ func (media *Media) PreparePublication(fence ResourceFence, connectionID string,
}
func (publication *mediaPublication) connectionStateChanged(state webrtc.PeerConnectionState) {
- publication.mu.Lock()
- input := publication.input
- publication.mu.Unlock()
- if input != nil {
- // Historical loss retires grants even if this PC has recovered. Recovery
- // requests a fresh handshake; it must never restore the previous grant.
- if state != webrtc.PeerConnectionStateConnected {
- input.SetUpAvailable(false)
- }
- if state != webrtc.PeerConnectionStateFailed && publication.pc.ConnectionState() == webrtc.PeerConnectionStateConnected {
- input.SetUpAvailable(true)
- }
- }
if state == webrtc.PeerConnectionStateFailed {
publication.fail(errors.New("SFU publisher transport failed"))
return
@@ -360,6 +351,7 @@ func (media *Media) AcceptPublisherOffer(fence ResourceFence, connectionID strin
}
if publication.input == nil && strings.Contains(offer.SDP, "\nm=application ") {
publication.input = inputroute.New(nil)
+ inputroute.ObserveTransport(publication.pc.SCTP().Transport().ICETransport(), publication.input.SetUpAvailable)
}
publication.mu.Unlock()
if err = publication.pc.SetRemoteDescription(offer); err != nil {
@@ -461,10 +453,11 @@ func (publication *mediaPublication) consumeTrack(track *webrtc.TrackRemote, rec
Codec: track.Codec(), Formats: publication.metadata.Formats,
HeaderExtensions: receiver.GetParameters().HeaderExtensions,
OnRTCP: func(_ int, packets []rtcp.Packet) {
- // Pion's ingress interceptors already own NACK and receiver reports.
+ // The buffer sees repaired primary sequence numbers and owns NACK;
+ // Pion retains receiver reports and transport-wide feedback.
for _, packet := range packets {
switch packet.(type) {
- case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest:
+ case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest, *rtcp.TransportLayerNack:
_ = publication.pc.WriteRTCP([]rtcp.Packet{packet})
}
}
@@ -606,7 +599,7 @@ func (media *Media) PrepareSubscriber(ctx context.Context, fence SubscriptionFen
audio = publication.audio
}
transport, err := forwarding.NewTransport(forwarding.TransportOptions{
- Source: source, Settings: media.options.Settings, ConnectionID: fence.ViewerPeerID + ":" + connectionID,
+ Source: source, Settings: media.options.Settings(), ConnectionID: fence.ViewerPeerID + ":" + connectionID,
InitialBitrate: int(metadata.Formats[len(metadata.Formats)-1].Bitrate), Audio: audio,
OnDemandChanged: publication.wakeDemand,
})
@@ -627,7 +620,12 @@ func (media *Media) PrepareSubscriber(ctx context.Context, fence SubscriptionFen
if channelErr != nil {
return webrtc.SessionDescription{}, channelErr
}
- input.AttachDown(fence.ViewerPeerID+":"+connectionID, channel)
+ lost := input.AttachDown(fence.ViewerPeerID+":"+connectionID, channel)
+ inputroute.ObserveTransport(transport.PC.SCTP().Transport().ICETransport(), func(available bool) {
+ if !available {
+ lost()
+ }
+ })
}
transport.PC.OnICECandidate(func(candidate *webrtc.ICECandidate) {
var value *webrtc.ICECandidateInit
@@ -653,14 +651,10 @@ func (media *Media) PrepareSubscriber(ctx context.Context, fence SubscriptionFen
func (publication *mediaPublication) subscriberStateChanged(subscriber *mediaSubscription, state webrtc.PeerConnectionState) {
publication.mu.Lock()
current := !publication.closed && publication.failure == nil && publication.subscribers[subscriber.key] == subscriber && subscriber.ctx.Err() == nil
- input := publication.input
publication.mu.Unlock()
if !current {
return
}
- if input != nil && (state == webrtc.PeerConnectionStateDisconnected || state == webrtc.PeerConnectionStateFailed || state == webrtc.PeerConnectionStateClosed) {
- input.LoseDown(subscriber.key.peerID+":"+subscriber.key.connectionID, nil)
- }
// Failed still belongs to the route's terminal failure owner. Other queued
// states describe current readiness, never permission to attach an old PC.
if state != webrtc.PeerConnectionStateFailed {
diff --git a/internal/server/sfu/media_test.go b/internal/server/sfu/media_test.go
index b1dc916c..d2d1a994 100644
--- a/internal/server/sfu/media_test.go
+++ b/internal/server/sfu/media_test.go
@@ -12,6 +12,7 @@ import (
"time"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ "github.com/TNTcraftHIM/Piik/internal/media/forwarding/forwardingtest"
"github.com/TNTcraftHIM/Piik/internal/media/inputroute"
livekitsfu "github.com/livekit/livekit-server/pkg/sfu"
"github.com/pion/interceptor"
@@ -22,6 +23,30 @@ import (
"github.com/pion/webrtc/v4/pkg/media"
)
+func TestPublicationStopsNACKAfterRTXRepair(t *testing.T) {
+ forwardingtest.IngressRepair(t, func(offer webrtc.SessionDescription, settings webrtc.SettingEngine) (*webrtc.PeerConnection, func() *forwarding.Source) {
+ service := NewMedia(MediaOptions{Settings: func() webrtc.SettingEngine { return settings }})
+ t.Cleanup(func() { _ = service.Close() })
+ fence := ResourceFence{RoomID: "1234", ShareGeneration: "share_12345678", PublicationGeneration: "publication_12345678"}
+ metadata := PublicationMedia{Codec: "vp8", RIDs: []string{""}, Formats: []forwarding.LayerFormat{{Width: 8, Height: 8, Bitrate: 1_000_000}}}
+ if err := service.PreparePublication(fence, "publisher_12345678", metadata); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := service.AcceptPublisherOffer(fence, "publisher_12345678", offer); err != nil {
+ t.Fatal(err)
+ }
+ publication, err := service.publication(fence, "publisher_12345678")
+ if err != nil {
+ t.Fatal(err)
+ }
+ return publication.pc, func() *forwarding.Source {
+ publication.mu.Lock()
+ defer publication.mu.Unlock()
+ return publication.source
+ }
+ })
+}
+
func TestMediaOnePublicationTwoSubscribersAndExactRetirement(t *testing.T) {
for _, codec := range []string{"vp8", "h264"} {
for _, layerCount := range []int{1, 2} {
@@ -172,7 +197,7 @@ func runMediaFanout(t *testing.T, codec string, layerCount int) {
return *pc.LocalDescription()
}
stateEvents := make(chan MediaEvent, 128)
- service := NewMedia(MediaOptions{Settings: settings, Events: func(event MediaEvent) {
+ service := NewMedia(MediaOptions{Settings: func() webrtc.SettingEngine { return settings }, Events: func(event MediaEvent) {
select {
case stateEvents <- event:
default:
@@ -337,6 +362,7 @@ func runMediaFanout(t *testing.T, codec string, layerCount int) {
}
var transports [2]*forwarding.Transport
inputStates := make(chan string, 32)
+ viewerInput := make(chan *webrtc.DataChannel, 1)
for index, subscription := range fences {
if !admission.ReserveSubscription(subscription) {
t.Fatal("subscription admission failed")
@@ -356,6 +382,7 @@ func runMediaFanout(t *testing.T, codec string, layerCount int) {
if index == 0 {
receiver.OnDataChannel(func(channel *webrtc.DataChannel) {
channel.OnMessage(func(message webrtc.DataChannelMessage) { inputStates <- string(message.Data) })
+ viewerInput <- channel
})
}
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
@@ -443,23 +470,55 @@ func runMediaFanout(t *testing.T, codec string, layerCount int) {
}
check(inputChannel.SendText(`{"ready":true}`))
waitInput(inputStates, `{"ready":true}`)
- // Pion dispatches state callbacks asynchronously. A Connecting callback
- // may resume after this same physical PC has already recovered.
+ var viewerChannel *webrtc.DataChannel
+ select {
+ case viewerChannel = <-viewerInput:
+ case <-ctx.Done():
+ t.Fatal("viewer input did not open")
+ }
for len(requests) > 0 {
<-requests
}
+ for len(inputStates) > 0 {
+ <-inputStates
+ }
+ // Pion dispatches PC projections asynchronously. Old projections must
+ // not retire input that belongs to this same transport's recovery.
publication.connectionStateChanged(webrtc.PeerConnectionStateConnecting)
- waitInput(inputStates, `{"ready":false}`)
- select {
- case message := <-requests:
- if message != `{"ready":true}` {
- t.Fatalf("fresh input handshake = %s", message)
+ const reply = `{"id":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","epoch":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","data":"after-publisher-projection"}`
+ check(inputChannel.SendText(reply))
+ for waiting := true; waiting; {
+ select {
+ case message := <-inputStates:
+ if message == `{"ready":false}` {
+ t.Error("late publisher projection retired recovered input")
+ }
+ waiting = message != reply
+ case <-ctx.Done():
+ t.Fatal("publisher projection marker did not reach viewer")
}
- case <-time.After(time.Second):
- t.Fatal("late loss left recovered SFU input unavailable")
}
check(inputChannel.SendText(`{"ready":true}`))
waitInput(inputStates, `{"ready":true}`)
+ const request = `{"id":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","epoch":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","data":"current-request"}`
+ check(viewerChannel.SendText(request))
+ waitInput(requests, request)
+ physical, getErr := service.subscriber(fences[0], "subscriber_123456")
+ check(getErr)
+ publication.subscriberStateChanged(physical, webrtc.PeerConnectionStateDisconnected)
+ const next = `{"id":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","epoch":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","data":"after-subscriber-projection"}`
+ check(viewerChannel.SendText(next))
+ for waiting := true; waiting; {
+ select {
+ case message := <-requests:
+ if strings.Contains(message, `"closed":true`) {
+ t.Error("late subscriber projection retired recovered input")
+ }
+ waiting = message != next
+ case <-ctx.Done():
+ t.Fatal("subscriber projection marker did not reach publisher")
+ }
+ }
}
publication.mu.Lock()
bound := 0
diff --git a/internal/server/sfu/voice.go b/internal/server/sfu/voice.go
index 7cf2e179..95eeb6a6 100644
--- a/internal/server/sfu/voice.go
+++ b/internal/server/sfu/voice.go
@@ -69,12 +69,12 @@ func (p VoiceConnectionPlan) key() voiceKey {
type voiceMedia struct {
mu sync.Mutex
- settings webrtc.SettingEngine
+ settings func() webrtc.SettingEngine
connections map[voiceKey]*voiceConnection
closed bool
}
-func newVoiceMedia(settings webrtc.SettingEngine) *voiceMedia {
+func newVoiceMedia(settings func() webrtc.SettingEngine) *voiceMedia {
return &voiceMedia{settings: settings, connections: make(map[voiceKey]*voiceConnection)}
}
@@ -158,6 +158,11 @@ func (media *Media) ConfigureVoice(plan VoiceConnectionPlan, emit func(VoiceMedi
delete(c.outgoing, id)
}
}
+ for _, input := range c.bindings {
+ if !slices.Contains(plan.Publish, input.publication) {
+ c.detachInputLocked(input.source)
+ }
+ }
v.mu.Unlock()
return func() error { return c.negotiate(false) }, nil
}
diff --git a/internal/server/sfu/voice_connection.go b/internal/server/sfu/voice_connection.go
index 7a15e66a..c2d23231 100644
--- a/internal/server/sfu/voice_connection.go
+++ b/internal/server/sfu/voice_connection.go
@@ -3,12 +3,15 @@ package sfu
import (
"errors"
"slices"
+ "strconv"
"strings"
"sync"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
+ livekitsfu "github.com/livekit/livekit-server/pkg/sfu"
+ "github.com/livekit/livekit-server/pkg/sfu/buffer"
"github.com/pion/interceptor"
- "github.com/pion/rtp"
+ "github.com/pion/rtcp"
"github.com/pion/webrtc/v4"
"github.com/pion/webrtc/v4/pkg/rtcerr"
)
@@ -21,20 +24,21 @@ var voiceCodec = webrtc.RTPCodecCapability{
type voiceReceiveSlot struct {
transceiver *webrtc.RTPTransceiver
publication VoicePublication
- rtcp bool
}
type voiceSendSlot struct {
mu sync.Mutex
publication VoicePublication
transceiver *webrtc.RTPTransceiver
- track *webrtc.TrackLocalStaticRTP
+ output *forwarding.AudioOutput
+ input *forwarding.Source
active bool
}
func (s *voiceSendSlot) disable() {
s.mu.Lock()
s.active = false
+ s.detachSource()
s.mu.Unlock()
}
@@ -44,15 +48,27 @@ func (s *voiceSendSlot) source() VoicePublication {
return s.publication
}
-func (s *voiceSendSlot) write(publication VoicePublication, packet *rtp.Packet) {
+func (s *voiceSendSlot) attach(publication VoicePublication, source *forwarding.Source) {
s.mu.Lock()
defer s.mu.Unlock()
- if s.active && s.publication == publication {
- // A failed subscriber write belongs to that connection, not the source.
- _ = s.track.WriteRTP(packet)
+ if s.active && s.publication == publication && s.input != source {
+ if err := s.output.ReplaceSource(source); err == nil {
+ s.input = source
+ }
}
}
+// Called under the slot mutex; the negotiated output survives a source's end.
+func (s *voiceSendSlot) detachSource() {
+ s.output.DetachSource()
+ s.input = nil
+}
+
+type voiceInput struct {
+ publication VoicePublication
+ source *forwarding.Source
+}
+
type voiceOffer struct {
description VoiceDescription
version uint64
@@ -60,16 +76,17 @@ type voiceOffer struct {
}
type voiceConnection struct {
- owner *voiceMedia
- key voiceKey
- pc *webrtc.PeerConnection
+ owner *voiceMedia
+ key voiceKey
+ pc *webrtc.PeerConnection
+ factory *buffer.Factory
// The owner mutex protects plans and the packet-forwarding view.
plan VoiceConnectionPlan
version uint64
emit func(VoiceMediaEvent)
outgoing map[string]*voiceSendSlot
remote map[string]VoicePublication
- bindings map[string]*webrtc.TrackRemote
+ bindings map[string]*voiceInput
closed bool
readers sync.WaitGroup
// SDP mutations are serialized independently of room signaling and RTP.
@@ -89,22 +106,34 @@ type voiceConnection struct {
}
func newVoiceConnection(owner *voiceMedia, plan VoiceConnectionPlan, emit func(VoiceMediaEvent)) (*voiceConnection, error) {
+ c := &voiceConnection{owner: owner, key: plan.key(), emit: emit,
+ outgoing: make(map[string]*voiceSendSlot), remote: make(map[string]VoicePublication), bindings: make(map[string]*voiceInput),
+ events: make(chan VoiceMediaEvent, 128), stop: make(chan struct{})}
engine := &webrtc.MediaEngine{}
if err := engine.RegisterCodec(webrtc.RTPCodecParameters{RTPCodecCapability: voiceCodec, PayloadType: 111}, webrtc.RTPCodecTypeAudio); err != nil {
return nil, err
}
registry := &interceptor.Registry{}
- if err := webrtc.RegisterDefaultInterceptors(engine, registry); err != nil {
+ settings := owner.settings()
+ factory, err := forwarding.ConfigureAudioForwarding(&settings, registry, func(ssrc uint32) *livekitsfu.DownTrack {
+ owner.mu.Lock()
+ defer owner.mu.Unlock()
+ for _, slot := range c.sendSlots {
+ if track := slot.output.DownTrack(); track.SSRC() == ssrc {
+ return track
+ }
+ }
+ return nil
+ })
+ if err != nil {
return nil, err
}
pc, err := webrtc.NewAPI(webrtc.WithMediaEngine(engine), webrtc.WithInterceptorRegistry(registry),
- webrtc.WithSettingEngine(owner.settings)).NewPeerConnection(webrtc.Configuration{})
+ webrtc.WithSettingEngine(settings)).NewPeerConnection(webrtc.Configuration{})
if err != nil {
return nil, err
}
- c := &voiceConnection{owner: owner, key: plan.key(), pc: pc, emit: emit,
- outgoing: make(map[string]*voiceSendSlot), remote: make(map[string]VoicePublication), bindings: make(map[string]*webrtc.TrackRemote),
- events: make(chan VoiceMediaEvent, 128), stop: make(chan struct{})}
+ c.pc, c.factory = pc, factory
// A silent room still negotiates its admitted transport. The empty receive
// slot is reused by a later authorized publication; it emits no audio.
idle, err := pc.AddTransceiverFromKind(webrtc.RTPCodecTypeAudio,
@@ -121,7 +150,16 @@ func newVoiceConnection(owner *voiceMedia, plan VoiceConnectionPlan, emit func(V
c.queue(VoiceMediaEvent{Candidate: &value})
}
})
- pc.OnConnectionStateChange(func(state webrtc.PeerConnectionState) { c.queue(VoiceMediaEvent{State: state}) })
+ pc.OnConnectionStateChange(func(state webrtc.PeerConnectionState) {
+ if state == webrtc.PeerConnectionStateConnected {
+ owner.mu.Lock()
+ for _, slot := range c.sendSlots {
+ slot.output.SetConnected()
+ }
+ owner.mu.Unlock()
+ }
+ c.queue(VoiceMediaEvent{State: state})
+ })
pc.OnTrack(c.consume)
go c.deliver()
return c, nil
@@ -259,29 +297,11 @@ func (c *voiceConnection) syncSlots(plan VoiceConnectionPlan) error {
}
c.receiveSlots[index].publication = pub
}
- for _, slot := range c.receiveSlots {
- if !slot.rtcp {
- slot.rtcp = true
- if c.startReader() {
- go func(receiver *webrtc.RTPReceiver) {
- defer c.readers.Done()
- for {
- if _, _, err := receiver.ReadRTCP(); err != nil {
- return
- }
- }
- }(slot.transceiver.Receiver())
- }
- }
- }
for _, slot := range c.sendSlots {
if slot.publication.ID != "" && !slices.Contains(plan.Subscribe, slot.publication) {
slot.disable()
- if err := slot.transceiver.Sender().ReplaceTrack(nil); err != nil {
- return err
- }
slot.mu.Lock()
- slot.publication, slot.track = VoicePublication{}, nil
+ slot.publication = VoicePublication{}
slot.mu.Unlock()
}
}
@@ -290,25 +310,38 @@ func (c *voiceConnection) syncSlots(plan VoiceConnectionPlan) error {
c.authorizeOutput(c.sendSlots[index], pub)
continue
}
- track, err := webrtc.NewTrackLocalStaticRTP(voiceCodec, pub.ID, pub.PeerID)
- if err != nil {
- return err
- }
index := slices.IndexFunc(c.sendSlots, func(slot *voiceSendSlot) bool { return slot.publication.ID == "" })
if index < 0 {
- transceiver, err := c.pc.AddTransceiverFromTrack(track, webrtc.RTPTransceiverInit{Direction: webrtc.RTPTransceiverDirectionSendonly})
+ output, err := forwarding.NewAudioOutput(forwarding.AudioOutputOptions{
+ ID: c.key.connection + "_voice_" + strconv.Itoa(len(c.sendSlots)), StreamID: c.key.peer,
+ Codec: webrtc.RTPCodecParameters{RTPCodecCapability: voiceCodec, PayloadType: 111},
+ BufferFactory: c.factory, RTCPWriter: c.pc.WriteRTCP,
+ })
if err != nil {
return err
}
- c.sendSlots = append(c.sendSlots, &voiceSendSlot{transceiver: transceiver})
+ transceiver, err := c.pc.AddTransceiverFromTrack(output.Track(), webrtc.RTPTransceiverInit{Direction: webrtc.RTPTransceiverDirectionSendonly})
+ if err != nil {
+ output.Close()
+ return err
+ }
+ output.SetTransceiver(transceiver)
+ c.owner.mu.Lock()
+ if c.closed {
+ c.owner.mu.Unlock()
+ output.Close()
+ return errors.New("voice connection is retiring")
+ }
+ c.sendSlots = append(c.sendSlots, &voiceSendSlot{transceiver: transceiver, output: output})
index = len(c.sendSlots) - 1
- c.readRTCP(transceiver.Sender())
- } else if err = c.sendSlots[index].transceiver.Sender().ReplaceTrack(track); err != nil {
- return err
+ if c.pc.ConnectionState() == webrtc.PeerConnectionStateConnected {
+ output.SetConnected()
+ }
+ c.owner.mu.Unlock()
}
slot := c.sendSlots[index]
slot.mu.Lock()
- slot.publication, slot.track = pub, track
+ slot.publication = pub
slot.mu.Unlock()
c.authorizeOutput(slot, pub)
}
@@ -410,18 +443,24 @@ func (c *voiceConnection) startReader() bool {
return true
}
-func (c *voiceConnection) readRTCP(sender *webrtc.RTPSender) {
- if !c.startReader() {
- return
- }
- go func() {
- defer c.readers.Done()
- for {
- if _, _, err := sender.ReadRTCP(); err != nil {
- return
+func (c *voiceConnection) readInputRTCP(input *voiceInput, receiver *webrtc.RTPReceiver) {
+ defer c.readers.Done()
+ for {
+ packets, _, err := receiver.ReadRTCP()
+ if err != nil {
+ return
+ }
+ c.owner.mu.Lock()
+ source := input.source
+ c.owner.mu.Unlock()
+ if source != nil {
+ for _, packet := range packets {
+ if report, ok := packet.(*rtcp.SenderReport); ok {
+ _ = source.SenderReport(0, report)
+ }
}
}
- }()
+ }
}
func (c *voiceConnection) consume(track *webrtc.TrackRemote, receiver *webrtc.RTPReceiver) {
@@ -439,17 +478,30 @@ func (c *voiceConnection) consume(track *webrtc.TrackRemote, receiver *webrtc.RT
return
}
c.owner.mu.Lock()
- if c.closed || c.bindings[mid] != nil || len(c.bindings) >= maxVoicePublications {
+ if c.closed || (c.bindings[mid] == nil && len(c.bindings) >= maxVoicePublications) {
c.owner.mu.Unlock()
return
}
- c.bindings[mid] = track
+ if previous := c.bindings[mid]; previous != nil {
+ c.detachInputLocked(previous.source)
+ }
+ input := &voiceInput{}
+ c.bindings[mid] = input
c.readers.Add(1)
c.owner.mu.Unlock()
+ if c.startReader() {
+ go c.readInputRTCP(input, receiver)
+ }
defer func() {
c.owner.mu.Lock()
- delete(c.bindings, mid)
+ if c.bindings[mid] == input {
+ delete(c.bindings, mid)
+ }
+ c.detachInputLocked(input.source)
c.owner.mu.Unlock()
+ if input.source != nil {
+ input.source.Close()
+ }
c.readers.Done()
}()
for {
@@ -458,22 +510,67 @@ func (c *voiceConnection) consume(track *webrtc.TrackRemote, receiver *webrtc.RT
return
}
c.owner.mu.Lock()
+ if c.closed || c.bindings[mid] != input {
+ c.owner.mu.Unlock()
+ return
+ }
pub := c.remote[mid]
- var outputs []*voiceSendSlot
- if !c.closed && pub.ID != "" && slices.Contains(c.plan.Publish, pub) {
- for _, other := range c.owner.connections {
- if other != c && !other.closed && other.key.room == c.key.room {
- if out := other.outgoing[pub.ID]; out != nil && slices.Contains(other.plan.Subscribe, pub) {
- outputs = append(outputs, out)
- }
+ if pub.ID == "" || !slices.Contains(c.plan.Publish, pub) {
+ c.owner.mu.Unlock()
+ continue
+ }
+ previous := input.source
+ if previous == nil || input.publication != pub {
+ source, err := forwarding.NewSource(forwarding.SourceOptions{
+ ID: pub.ID, StreamID: pub.PeerID, Codec: track.Codec(),
+ })
+ if err == nil {
+ err = source.BindLayer(0, packet.SSRC, webrtc.RTPParameters{Codecs: []webrtc.RTPCodecParameters{track.Codec()}})
+ }
+ if err != nil {
+ c.owner.mu.Unlock()
+ if source != nil {
+ source.Close()
+ }
+ c.queue(VoiceMediaEvent{State: webrtc.PeerConnectionStateFailed})
+ return
+ }
+ c.detachInputLocked(previous)
+ input.publication, input.source = pub, source
+ }
+ for _, other := range c.owner.connections {
+ if other != c && !other.closed && other.key.room == c.key.room {
+ if out := other.outgoing[pub.ID]; out != nil && slices.Contains(other.plan.Subscribe, pub) {
+ out.attach(pub, input.source)
}
}
}
c.owner.mu.Unlock()
+ if previous != nil && previous != input.source {
+ previous.Close()
+ }
// Destination transports own their negotiated extensions and RTCP.
packet.Extension, packet.ExtensionProfile, packet.Extensions = false, 0, nil
- for _, out := range outputs {
- out.write(pub, packet)
+ _ = input.source.WriteRTP(0, packet)
+ }
+}
+
+// Authority revocation fences already-buffered input at every attached output.
+// Physical source closure remains with its reader, outside the authority lock.
+func (c *voiceConnection) detachInputLocked(source *forwarding.Source) {
+ if source == nil {
+ return
+ }
+ for _, other := range c.owner.connections {
+ if other.key.room != c.key.room {
+ continue
+ }
+ for _, out := range other.outgoing {
+ out.mu.Lock()
+ if out.input == source {
+ out.detachSource()
+ }
+ out.mu.Unlock()
}
}
}
@@ -481,6 +578,9 @@ func (c *voiceConnection) consume(track *webrtc.TrackRemote, receiver *webrtc.RT
// retireLocked revokes packet delivery before physical close can block.
func (c *voiceConnection) retireLocked() {
c.closed = true
+ for _, input := range c.bindings {
+ c.detachInputLocked(input.source)
+ }
for _, out := range c.outgoing {
out.disable()
}
@@ -494,6 +594,11 @@ func (c *voiceConnection) close() error {
close(c.stop)
c.closeErr = c.pc.Close()
c.readers.Wait()
+ c.signaling.Lock()
+ for _, slot := range c.sendSlots {
+ slot.output.Close()
+ }
+ c.signaling.Unlock()
c.owner.mu.Lock()
if c.owner.connections[c.key] == c {
delete(c.owner.connections, c.key)
diff --git a/internal/server/sfu/voice_test.go b/internal/server/sfu/voice_test.go
index 2efb3235..5b5e4e72 100644
--- a/internal/server/sfu/voice_test.go
+++ b/internal/server/sfu/voice_test.go
@@ -19,8 +19,9 @@ import (
)
type voiceFixtureSource struct {
- track *webrtc.TrackLocalStaticRTP
- active atomic.Bool
+ track *webrtc.TrackLocalStaticRTP
+ active atomic.Bool
+ sequence atomic.Uint32
}
type voiceFixturePeer struct {
@@ -179,14 +180,13 @@ func newVoiceFixturePeer(t *testing.T, ctx context.Context, service *Media, sett
go func() {
ticker := time.NewTicker(10 * time.Millisecond)
defer ticker.Stop()
- var sequence uint16
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if source.active.Load() {
- sequence++
+ sequence := uint16(source.sequence.Add(1))
_ = track.WriteRTP(&rtp.Packet{Header: rtp.Header{Version: 2, SequenceNumber: sequence, Timestamp: uint32(sequence) * 480}, Payload: []byte{marker, 0, 0}})
}
}
@@ -212,7 +212,7 @@ func TestVoiceJunctionDuplexAuthorizationAndExactRetirement(t *testing.T) {
defer mux.Close()
serverSettings := settings
serverSettings.SetICEUDPMux(mux)
- service := NewMedia(MediaOptions{Settings: serverSettings})
+ service := NewMedia(MediaOptions{Settings: func() webrtc.SettingEngine { return serverSettings }})
defer service.Close()
a1, a2, b1 := VoicePublication{"voice_a1", "speaker_a"}, VoicePublication{"voice_a2", "speaker_c"}, VoicePublication{"voice_b1", "speaker_b"}
aPlan := VoiceConnectionPlan{RoomID: "1234", PeerID: "relay_a", SessionID: "session_a", ConnectionID: "connection_a", Publish: []VoicePublication{a1, a2}, Subscribe: []VoicePublication{b1}}
@@ -389,6 +389,63 @@ func TestVoiceJunctionDuplexAuthorizationAndExactRetirement(t *testing.T) {
}
}
+func TestVoiceSubscriptionReusesSlotAcrossIndependentPacketClocks(t *testing.T) {
+ ctx, cancel := context.WithTimeout(t.Context(), 8*time.Second)
+ defer cancel()
+ settings := webrtc.SettingEngine{}
+ settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
+ settings.SetIncludeLoopbackCandidate(true)
+ settings.SetIPFilter(func(ip net.IP) bool { return ip.IsLoopback() })
+ service := NewMedia(MediaOptions{Settings: func() webrtc.SettingEngine { return settings }})
+ t.Cleanup(func() { _ = service.Close() })
+ first, next := VoicePublication{"voice_first", "speaker_first"}, VoicePublication{"voice_next", "speaker_next"}
+ send := VoiceConnectionPlan{RoomID: "1234", PeerID: "sender", SessionID: "sender_session", ConnectionID: "sender_connection", Publish: []VoicePublication{first, next}}
+ receive := VoiceConnectionPlan{RoomID: "1234", PeerID: "listener", SessionID: "listener_session", ConnectionID: "listener_connection", Subscribe: []VoicePublication{first}}
+ sender := newVoiceFixturePeer(t, ctx, service, settings, send, map[string]byte{first.ID: 0xa1, next.ID: 0xa2})
+ listener := newVoiceFixturePeer(t, ctx, service, settings, receive, nil)
+ // Different microphones have independent RTP clocks. Reusing an output
+ // section must not make the second source look like old SRTP packets.
+ sender.sources[first.ID].sequence.Store(20_000)
+ sender.sources[next.ID].sequence.Store(1_000)
+ for _, peer := range []*voiceFixturePeer{sender, listener} {
+ if err := configureVoiceForTest(service, peer.plan, peer.event); err != nil {
+ t.Fatal(err)
+ }
+ }
+ waitForSource := func(marker byte, target int) {
+ t.Helper()
+ ticker := time.NewTicker(5 * time.Millisecond)
+ defer ticker.Stop()
+ for listener.count(marker) < target {
+ select {
+ case err := <-sender.errors:
+ t.Fatal(err)
+ case err := <-listener.errors:
+ t.Fatal(err)
+ case <-ctx.Done():
+ t.Fatalf("source %x did not reach listener: %v", marker, ctx.Err())
+ case <-ticker.C:
+ }
+ }
+ }
+ waitForSource(0xa1, 3)
+ sections := len(listener.pc.GetTransceivers())
+ receive.Subscribe = []VoicePublication{next}
+ if err := configureVoiceForTest(service, receive, listener.event); err != nil {
+ t.Fatal(err)
+ }
+ waitForSource(0xa2, 3)
+ firstTarget := listener.count(0xa1) + 3
+ receive.Subscribe = []VoicePublication{first}
+ if err := configureVoiceForTest(service, receive, listener.event); err != nil {
+ t.Fatal(err)
+ }
+ waitForSource(0xa1, firstTarget)
+ if len(listener.pc.GetTransceivers()) != sections {
+ t.Fatal("source replacement grew the negotiated media sections")
+ }
+}
+
func TestVoiceFullRoomAdmitsBothPublicationReplacementFacets(t *testing.T) {
service := NewMedia(MediaOptions{})
t.Cleanup(func() {
@@ -542,7 +599,7 @@ func TestVoiceSilentTransportAcceptsLatePublicationOnSameConnection(t *testing.T
settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
settings.SetIncludeLoopbackCandidate(true)
settings.SetIPFilter(func(ip net.IP) bool { return ip.IsLoopback() })
- service := NewMedia(MediaOptions{Settings: settings})
+ service := NewMedia(MediaOptions{Settings: func() webrtc.SettingEngine { return settings }})
defer service.Close()
pub := VoicePublication{ID: "late_voice", PeerID: "speaker"}
aPlan := VoiceConnectionPlan{RoomID: "1234", PeerID: "a", SessionID: "session_a", ConnectionID: "connection_a"}
diff --git a/internal/server/signal/router.go b/internal/server/signal/router.go
index 98ffd74d..58b04445 100644
--- a/internal/server/signal/router.go
+++ b/internal/server/signal/router.go
@@ -152,6 +152,7 @@ type peerSignalDebugInput struct {
type roomRuntime struct {
hostPeerID string
controller *route.Controller
+ revisions *route.RevisionSequence
roomMedia bool
mediaGeneration string
pendingMedia *preparedCandidate
@@ -1069,25 +1070,21 @@ func (r *router) removeViewer(roomID, peerID string) {
}
if rm != nil && rm.controller != nil && rm.controller.ConfirmDeparture(peerID, r.nowPtr()) {
r.requestPump(roomID)
- } else if rm != nil && rm.controller == nil && rm.viewerSettings.Len() == 0 {
- r.rooms.Delete(roomID)
}
}
-// stopRoom preserves Viewer settings across shares.
+// stopRoom preserves Viewer settings and the room's revision sequence across shares.
func (r *router) stopRoom(roomID string) {
- if rm, _ := r.rooms.Get(roomID); rm != nil && rm.roomMedia && rm.controller != nil {
- r.setVideoPublication(roomID, "", false)
+ rm, _ := r.rooms.Get(roomID)
+ if rm == nil {
return
}
- var settings ordered.Map[string, viewerRouteSettings]
- if rm, _ := r.rooms.Get(roomID); rm != nil {
- settings = rm.viewerSettings
+ if rm.roomMedia && rm.controller != nil {
+ r.setVideoPublication(roomID, "", false)
+ return
}
r.clearRoom(roomID)
- if settings.Len() > 0 {
- r.room(roomID).viewerSettings = settings
- }
+ r.rooms.Set(roomID, &roomRuntime{revisions: rm.revisions, viewerSettings: rm.viewerSettings})
}
func (r *router) deleteRoom(roomID string) {
@@ -1097,7 +1094,7 @@ func (r *router) deleteRoom(roomID string) {
func (r *router) room(roomID string) *roomRuntime {
rm, ok := r.rooms.Get(roomID)
if !ok {
- rm = &roomRuntime{}
+ rm = &roomRuntime{revisions: &route.RevisionSequence{}}
r.rooms.Set(roomID, rm)
}
return rm
@@ -1112,6 +1109,7 @@ func (r *router) createController(roomID string, rm *roomRuntime, host authentic
}
rm.controller = route.New(route.Options{
HostPeerID: host.peerID,
+ Revisions: rm.revisions,
DebugRoomID: roomID,
DebugLog: routeDebugSink(),
EndpointMediaCopyCapacity: r.capacity,
diff --git a/internal/server/signal/router_failure_test.go b/internal/server/signal/router_failure_test.go
index b1b21d97..6dba5e58 100644
--- a/internal/server/signal/router_failure_test.go
+++ b/internal/server/signal/router_failure_test.go
@@ -9,6 +9,37 @@ import (
"github.com/TNTcraftHIM/Piik/internal/server/sfu"
)
+func TestCancelPreparedVideoAttachmentDoesNotFailRoom(t *testing.T) {
+ h := newRouterHarness(t, routerHarnessOptions{capacity: 2})
+ created := h.createRoom()
+ host := h.connectHost(created, nil, "", "")
+ viewer := h.connectViewer(created, "cancel-video-attachment")
+ h.locked(func() { h.doComplete(host); h.doComplete(viewer) })
+ prepared := h.waitPrepared(viewer.sessionID)
+ h.routeReady(viewer, int64(prepared.Revision))
+ h.settle()
+ h.locked(func() {
+ rm, _ := h.router.rooms.Get(created.RoomID)
+ rm.roomMedia, rm.mediaGeneration = true, "room_media_12345678"
+ rm.controller.SetVideoPublication(testShareGeneration, false, h.router.now())
+ rm.controller.SetVideoPublication("", false, h.router.now())
+ rm.controller.SetVideoPublication(testShareGeneration, false, h.router.now())
+ h.router.requestPump(created.RoomID)
+ op := rm.controller.Operation()
+ if op == nil || op.AttachConnectionID == "" || op.Current != nil {
+ t.Fatal("expected admitted attachment waiting for the pump continuation")
+ }
+ // A stop can overtake the asynchronous BeginCurrentCandidate step.
+ h.router.setVideoPublication(created.RoomID, "", false)
+ })
+ h.settle()
+ for _, message := range h.messages(host.sessionID) {
+ if failure, ok := message.(protocol.ErrorMessage); ok && failure.Code == "SERVER_ERROR" {
+ t.Fatal("canceling a video attachment surfaced a media routing error")
+ }
+ }
+}
+
func TestRoomVideoFailuresCannotInvalidateAnotherPhysicalFacet(t *testing.T) {
h := newRouterHarness(t, routerHarnessOptions{capacity: 2})
created := h.createRoom()
diff --git a/internal/server/signal/router_generation_test.go b/internal/server/signal/router_generation_test.go
new file mode 100644
index 00000000..1815d0cb
--- /dev/null
+++ b/internal/server/signal/router_generation_test.go
@@ -0,0 +1,34 @@
+package signal
+
+import "testing"
+
+func TestStoppedShareReadyCannotCommitReplacementShare(t *testing.T) {
+ h := newRouterHarness(t, routerHarnessOptions{capacity: 2, fakeTimers: true})
+ created := h.createRoom()
+ host := h.connectHost(created, nil, "", "")
+ viewer := h.connectViewer(created, "share-ready")
+ h.locked(func() {
+ h.doComplete(host)
+ h.doComplete(viewer)
+ })
+ old := h.waitPrepared(viewer.sessionID)
+ h.settle()
+ h.locked(func() {
+ h.router.stopRoom(created.RoomID)
+ h.sent[viewer.sessionID] = nil
+ h.router.hooks.shareGeneration = func(string) string { return "replacement_generation_12345678" }
+ h.doComplete(host)
+ })
+ current := h.waitPrepared(viewer.sessionID)
+ if current.Candidate.ConnectionID == old.Candidate.ConnectionID {
+ t.Fatal("replacement reused the physical connection")
+ }
+ h.routeReady(viewer, int64(old.Revision))
+ if _, committed := h.activeEdge(created.RoomID, viewer.peerID); committed {
+ t.Fatalf("stopped share ready at revision %d committed replacement revision %d", old.Revision, current.Revision)
+ }
+ h.routeReady(viewer, int64(current.Revision))
+ if _, committed := h.activeEdge(created.RoomID, viewer.peerID); !committed {
+ t.Fatal("replacement's own first-frame proof did not commit")
+ }
+}
diff --git a/internal/server/signal/router_sfu_signal_test.go b/internal/server/signal/router_sfu_signal_test.go
index 0505f77a..24217524 100644
--- a/internal/server/signal/router_sfu_signal_test.go
+++ b/internal/server/signal/router_sfu_signal_test.go
@@ -82,7 +82,7 @@ func TestEmbeddedSfuAuthenticatedWebSocketCarriesActualRTP(t *testing.T) {
settings.SetIncludeLoopbackCandidate(true)
settings.SetIPFilter(func(ip net.IP) bool { return ip.IsLoopback() })
var signaling atomic.Pointer[Server]
- service := sfu.NewMedia(sfu.MediaOptions{Settings: settings, Events: func(event sfu.MediaEvent) {
+ service := sfu.NewMedia(sfu.MediaOptions{Settings: func() webrtc.SettingEngine { return settings }, Events: func(event sfu.MediaEvent) {
if server := signaling.Load(); server != nil {
server.HandleSfuMediaEvent(event)
}
diff --git a/internal/server/signal/server.go b/internal/server/signal/server.go
index 4cb12fc4..2641acbc 100644
--- a/internal/server/signal/server.go
+++ b/internal/server/signal/server.go
@@ -670,7 +670,7 @@ func (s *Server) armHeartbeat() {
// handleMessage requires mu to be held.
func (s *Server) handleMessage(sess *session, encoded []byte) {
- if !s.sessions.Has(sess) || s.closing {
+ if !s.sessions.Has(sess) || s.closing || !sess.open() {
return
}
if sess.revoked {
diff --git a/internal/server/signal/server_test.go b/internal/server/signal/server_test.go
index 956fd0b6..00c0c239 100644
--- a/internal/server/signal/server_test.go
+++ b/internal/server/signal/server_test.go
@@ -2864,6 +2864,32 @@ func TestSignalRequiresAllowedOriginTimelyAuthenticationAndBoundedPayloads(t *te
expectCloseCode(t, oversized, 1009)
}
+func TestClosingSignalSessionDoesNotReserveLateViewerAdmission(t *testing.T) {
+ h := startHarness(t, harnessOptions{})
+ client := openClient(t, h)
+ if !waitFor(t, defaultWait, func() bool { return h.sessionCount() == 1 }) {
+ t.Fatal("signaling session was not registered")
+ }
+ message, err := json.Marshal(map[string]any{
+ "type": "authenticate", "protocol": protocol.SignalingProtocol,
+ "roomId": h.room.RoomID, "role": protocol.RoleViewer, "clientId": "late-viewer",
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ h.locked(func() {
+ sess := h.server.sessions.Keys()[0]
+ // An authentication deadline can queue close before a frame already
+ // read from the socket acquires the signaling lock.
+ sess.close(websocket.StatusCode(protocol.SignalCloseAuthenticationFailed), "Authentication required")
+ h.server.handleMessage(sess, message)
+ if peers := h.store.GetViewerPeerIDs(h.room.RoomID); len(peers) != 0 {
+ t.Fatalf("closing session left room capacity reserved: %v", peers)
+ }
+ })
+ expectCloseCode(t, client, int(protocol.SignalCloseAuthenticationFailed))
+}
+
func TestSignalTerminatesStaleV12BeforeAuthentication(t *testing.T) {
h := startHarness(t, harnessOptions{})
invalidClient := openClient(t, h)
diff --git a/internal/thirdparty/README.md b/internal/thirdparty/README.md
index 56413add..35b92354 100644
--- a/internal/thirdparty/README.md
+++ b/internal/thirdparty/README.md
@@ -1,4 +1,7 @@
-# Scoped Go Dependency Repairs
+# Scoped Go Dependency Changes
+
+Use the shared [upstream refresh procedure](../../CONTRIBUTING.md#temporary-dependency-patches)
+to compare retained patches with current official source.
The root `go.mod` replaces these two modules with local source. They retain
their original module paths and Apache-2.0 licenses. The checked-in source,
@@ -15,7 +18,9 @@ Modified upstream files carry a notice; formatting follows the project Go
toolchain. The package notice collector reads the replacement modules' license
files directly, so App archives retain their attribution.
-These repairs preserve the existing gateway selector and PCPv6 composition.
+Assigned-port and deletion handling repair NAT-PMP behavior. The Context methods
+extend cancellation support; the shorter optional PCPv6 waits are Piik's mapping
+budget policy. These changes preserve the gateway selector and PCPv6 composition.
The [NAT research owner](../../docs/research/nat-traversal.md#gateway-and-survey-limits)
describes the runtime limits. Do not add a second gateway client in Piik.
@@ -35,8 +40,6 @@ the temporary allocator input described below. No build-time patch is applied.
Measurement corrections are marked in the modified files: probe finalization
returns the current acknowledged capacity and rejects invalid/empty feedback;
-inconclusive probes use existing backoff from completion without growing the
-next probe or reporting measured congestion;
probe timing begins with its first acknowledged packet, excluding pre-probe idle
time; congestion statistics retain the packet that starts each new CTR group,
including its received/lost accounting; overlapping TWCC feedback changes each
@@ -47,10 +50,12 @@ advances in send order with a receive-time watermark, while older ACKs repair
counts without rewinding it. Reset reseeds that delay baseline without discarding
delivery history. Piik's shared
forwarding package uses this copy for both individual transports and publications.
-Delay inference tolerates ordinary clock-rate skew using RFC 6817 A.2's
+Two recovery policies accompany those corrections. Inconclusive probes use
+existing backoff from completion without growing the next probe or reporting
+measured congestion. Delay inference tolerates clock-rate skew using RFC 6817 A.2's
500 ppm virtual-sender-clock allowance; packet timestamps, acknowledged traffic
and loss remain unadjusted. Long, lossless TWCC controls and real queue/loss
-recovery controls fence this measurement repair.
+recovery controls exercise both policies.
Its regression checks use real TWCC feedback and retain the no-feedback and
idle-before-probe controls. The
[media research owner](../../docs/research/node-local-media-design.md#send-queue-recovery)
@@ -99,6 +104,23 @@ capacity. Once an upstream revision supports this input and passes those
checks, remove the replacement directive and its replacement checksums. The
separate local send-side estimator corrections retain their own removal gate.
+## Pion H.264 Profile Matching
+
+Until [upstream #3668](https://github.com/pion/webrtc/pull/3668) is available,
+the root module replaces Pion with the immutable
+[6ed9e55c revision](https://github.com/TNTcraftHIM/pion-webrtc/commit/6ed9e55c5935edf3b55d0528fdfb5cc85110a013),
+based on official v4.2.26. It matches equivalent H.264 subprofiles according to
+[RFC 6184 Table 5](https://www.rfc-editor.org/rfc/rfc6184.html#section-8.1).
+Without this repair, an offer containing Constrained Baseline `42c01f` and VP8
+can select VP8 even though the receiver prefers and supports that H.264 profile.
+The patch changes only profile matching and its regressions; level and
+packetization handling stay upstream-owned. Piik adds no SDP aliases or local
+Pion source. The package notice collector reads the replacement's MIT license.
+
+Once an upstream revision passes the actual receiver negotiation and encoded
+forwarding controls, remove this replacement and its checksums. Keep the mixed
+H.264/VP8 regression in the existing media-edge receiver test table.
+
## Pion Repaired-Packet Identity
The shared [RTP receive adapter](../media/forwarding/codecs.go) uses the negotiated
@@ -109,4 +131,14 @@ an earlier RTX packet can therefore be queued with payload type zero.
Ordinary packet validation remains unchanged. A real simulcast/NACK regression
fences that ordering without sleeps and fails when the correction is removed.
Remove the adapter when a maintained upstream version passes that same control
-without it; no Pion source fork or second retransmission handler is retained.
+without it; this repair adds no second retransmission handler.
+
+Pion's [wire-level NACK generator](https://github.com/pion/interceptor/blob/v0.1.51/pkg/nack/generator_interceptor.go)
+also does not observe repaired primary sequence numbers. Native receive and SFU
+publication use the public [interceptor options](https://github.com/pion/webrtc/blob/v4.2.26/interceptor.go)
+to leave video NACK generation to their existing LiveKit input buffer, after
+deencapsulation. That buffer retries missing packets and removes repaired holes;
+its existing RTCP callback returns NACKs to the actual upstream SSRC. Pion retains
+audio NACK behavior, receiver reports, TWCC and the retransmission responder.
+The shared real-peer regression holds a hole open, restores its exact primary
+packet through RTX, and verifies that retries stop on both receiving paths.
diff --git a/mobile/README.md b/mobile/README.md
index ae9a2e4a..2b9a125a 100644
--- a/mobile/README.md
+++ b/mobile/README.md
@@ -84,8 +84,10 @@ and verify that the same clean source revision still owns the final output.
The universal `piik-app-android-universal.apk` contains arm64 and x86-64.
The build verifies its signature, package identity, version and ABIs, then writes
-the existing schema-2 release descriptor and SHA-256 sidecar beside the APK.
-The existing CI App matrix publishes it with the matching desktop/Server
+the existing schema-2 release descriptor and SHA-256 sidecars. It also wraps
+the unchanged signed APK in `piik-app-android-universal.zip` for Gitee; extract
+that ZIP before installation. GitHub publishes both distributions.
+The existing CI App matrix publishes them with the matching desktop/Server
artifacts. Publication requires four repository secrets:
`PIIK_ANDROID_KEYSTORE_BASE64` (the keystore bytes in Base64),
`PIIK_ANDROID_STORE_PASSWORD`, `PIIK_ANDROID_KEY_ALIAS` and
diff --git a/native/capture/darwin/README.md b/native/capture/darwin/README.md
index 83b6af5b..ce14e7ac 100644
--- a/native/capture/darwin/README.md
+++ b/native/capture/darwin/README.md
@@ -9,6 +9,10 @@ audio is converted to stereo PCM16 for the existing Go Opus boundary. The Go
session applies live profile changes by replacing that capture generation behind
the same Pion source.
+Source-picker thumbnails use the same exact target filter and one complete
+ScreenCaptureKit frame, then stop capture before returning a bounded BMP.
+They do not start the video encoder or source audio.
+
Build on Apple silicon or Intel macOS 13 or newer; the helper uses the build
machine's architecture:
@@ -20,9 +24,10 @@ sh native/capture/darwin/build.sh /outside/repository/build
/outside/repository/build/piik-capture --probe
```
-`--geometry-self-test` checks content cropping, Retina scaling and rejection of
-invalid frame geometry, plus H.264 format/aperture admission, without requiring
-a hardware encoder. Go validates every raw SPS's coded size and reference budget
+`--geometry-self-test` checks content cropping, Retina scaling, the system BMP
+thumbnail conversion and rejection of invalid frame geometry, plus H.264
+format/aperture admission, without requiring a hardware encoder. Go validates
+every raw SPS's coded size and reference budget
before sending encoded input to the helper; CoreMedia can normalize SPS cropping
in its format description. `--self-test`
encodes successive landscape, portrait, ultrawide and small 420v buffers through
diff --git a/native/capture/darwin/build.sh b/native/capture/darwin/build.sh
index 91e6795f..2d4db3de 100644
--- a/native/capture/darwin/build.sh
+++ b/native/capture/darwin/build.sh
@@ -19,9 +19,11 @@ xcrun swiftc \
-swift-version 5 \
-target "$architecture-apple-macos13.0" \
-framework AVFoundation \
+ -framework AppKit \
-framework AudioToolbox \
-framework CoreAudio \
-framework CoreMedia \
+ -framework CoreImage \
-framework CoreVideo \
-framework ScreenCaptureKit \
-framework VideoToolbox \
diff --git a/native/capture/darwin/main.swift b/native/capture/darwin/main.swift
index 727f46ad..2f302762 100644
--- a/native/capture/darwin/main.swift
+++ b/native/capture/darwin/main.swift
@@ -1,7 +1,9 @@
import AVFoundation
+import AppKit
import AudioToolbox
import CoreAudio
import CoreGraphics
+import CoreImage
import CoreMedia
import CoreVideo
import Darwin
@@ -326,6 +328,89 @@ private func listSources() async throws {
try writeJSON(Array((displays + windows).prefix(1024)))
}
+private func bitmapPreview(_ image: VideoImage) throws -> Data {
+ // Core Video's clean rect and Core Image both use a lower-left origin.
+ let pixels = CIImage(cvPixelBuffer: image.buffer)
+ .cropped(to: CVImageBufferGetCleanRect(image.buffer))
+ guard let source = CIContext().createCGImage(pixels, from: pixels.extent) else {
+ throw CaptureFailure(description: "source preview image is unavailable")
+ }
+ let scale = min(1, 320 / CGFloat(source.width), 180 / CGFloat(source.height))
+ let width = max(1, Int(CGFloat(source.width) * scale))
+ let height = max(1, Int(CGFloat(source.height) * scale))
+ // Opaque RGB keeps the system BMP encoder within the shared preview budget.
+ guard let context = CGContext(data: nil, width: width, height: height,
+ bitsPerComponent: 8, bytesPerRow: width * 4, space: CGColorSpaceCreateDeviceRGB(),
+ bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue | CGBitmapInfo.byteOrder32Big.rawValue) else {
+ throw CaptureFailure(description: "source preview bitmap is unavailable")
+ }
+ let bounds = CGRect(x: 0, y: 0, width: width, height: height)
+ context.setFillColor(CGColor(gray: 0, alpha: 1))
+ context.fill(bounds)
+ context.interpolationQuality = .high
+ context.draw(source, in: bounds)
+ guard let thumbnail = context.makeImage(),
+ let data = NSBitmapImageRep(cgImage: thumbnail).representation(using: .bmp, properties: [:]) else {
+ throw CaptureFailure(description: "source preview encoding failed")
+ }
+ return data
+}
+
+private final class PreviewOutput: NSObject, SCStreamOutput, SCStreamDelegate {
+ let done = StopSignal()
+ private(set) var data: Data?
+
+ func stream(_ stream: SCStream, didOutputSampleBuffer sampleBuffer: CMSampleBuffer,
+ of outputType: SCStreamOutputType) {
+ guard !done.isStopped, outputType == .screen, sampleBuffer.isValid,
+ let attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer,
+ createIfNecessary: false) as? [[SCStreamFrameInfo: Any]],
+ let attachments = attachments.first,
+ let status = attachments[.status] as? Int, SCFrameStatus(rawValue: status) == .complete,
+ let buffer = CMSampleBufferGetImageBuffer(sampleBuffer),
+ let image = VideoImage(buffer, attachments: attachments) else { return }
+ do {
+ data = try bitmapPreview(image)
+ done.signal()
+ } catch { done.signal(error) }
+ }
+
+ func stream(_ stream: SCStream, didStopWithError error: Error) {
+ done.signal(error)
+ }
+}
+
+private func previewSource(_ arguments: [String]) async throws {
+ guard arguments.count == 6, let sourceID = UInt32(arguments[3]), sourceID > 0,
+ let pid = UInt32(arguments[4]) else {
+ throw CaptureFailure(description: "invalid source preview arguments")
+ }
+ let filter = try await captureFilter(kind: arguments[2], sourceID: sourceID,
+ pid: pid, creationTime: arguments[5])
+ let configuration = SCStreamConfiguration()
+ configuration.width = 320
+ configuration.height = 180
+ configuration.scalesToFit = true
+ if #available(macOS 14.0, *) { configuration.preservesAspectRatio = true }
+ configuration.minimumFrameInterval = CMTime(value: 1, timescale: 1)
+ configuration.queueDepth = 3
+ configuration.pixelFormat = kCVPixelFormatType_32BGRA
+ configuration.showsCursor = false
+ let output = PreviewOutput()
+ let stream = SCStream(filter: filter, configuration: configuration, delegate: output)
+ try stream.addStreamOutput(output, type: .screen,
+ sampleHandlerQueue: DispatchQueue(label: "piik.capture.preview"))
+ try await stream.startCapture()
+ do { try output.done.wait() }
+ catch {
+ try? await stream.stopCapture()
+ throw error
+ }
+ try await stream.stopCapture()
+ guard let data = output.data else { throw CaptureFailure(description: "source preview is empty") }
+ FileHandle.standardOutput.write(data)
+}
+
private final class StopSignal {
private let lock = NSLock()
private let semaphore = DispatchSemaphore(value: 0)
@@ -1169,12 +1254,30 @@ private func geometrySelfTest() throws {
}
}
let buffer = try syntheticPixelBuffer()
+ let fullPreview = try bitmapPreview(VideoImage(buffer))
+ guard fullPreview.starts(with: [0x42, 0x4d]), fullPreview.count <= 192 * 1024,
+ let fullThumbnail = NSBitmapImageRep(data: fullPreview),
+ fullThumbnail.pixelsWide == 320, fullThumbnail.pixelsHigh == 180 else {
+ throw CaptureFailure(description: "source preview bitmap self-test failed")
+ }
let ceiling = OutputProfile(width: 1280, height: 720, fps: frameRate, bitrate: bitrate)
let rect = CGRect(x: 50, y: 40, width: 200, height: 300)
let attachments: [SCStreamFrameInfo: Any] = [
.contentRect: rect.dictionaryRepresentation,
.scaleFactor: CGFloat(2), .contentScale: CGFloat(1),
]
+ // Leave unequal black padding above/below the crop, with dark content on
+ // top and light content below. Decode the actual BMP to detect a Y flip.
+ try require(CVPixelBufferLockBaseAddress(buffer, []), "preview-test-buffer-lock")
+ guard let luma = CVPixelBufferGetBaseAddressOfPlane(buffer, 0) else {
+ CVPixelBufferUnlockBaseAddress(buffer, [])
+ throw CaptureFailure(description: "preview test pixels are unavailable")
+ }
+ let stride = CVPixelBufferGetBytesPerRowOfPlane(buffer, 0)
+ for row in 80..<680 {
+ memset(luma.advanced(by: row * stride + 100), row < 380 ? 80 : 180, 400)
+ }
+ CVPixelBufferUnlockBaseAddress(buffer, [])
// Core Video returns a lower-left origin; the surface crop starts 80 px
// from the top, so its lower edge is 720 - 80 - 600 = 40 px from the bottom.
guard let image = VideoImage(buffer, attachments: attachments),
@@ -1183,6 +1286,16 @@ private func geometrySelfTest() throws {
CVImageBufferGetCleanRect(buffer) == CGRect(x: 100, y: 40, width: 400, height: 600) else {
throw CaptureFailure(description: "capture content geometry self-test failed")
}
+ let preview = try bitmapPreview(image)
+ guard preview.starts(with: [0x42, 0x4d]), preview.count <= 192 * 1024,
+ let thumbnail = NSBitmapImageRep(data: preview),
+ thumbnail.pixelsWide == 120, thumbnail.pixelsHigh == 180,
+ let top = thumbnail.colorAt(x: 5, y: 3)?.usingColorSpace(.deviceRGB),
+ let bottom = thumbnail.colorAt(x: 5, y: 176)?.usingColorSpace(.deviceRGB),
+ top.redComponent > 0.15, top.redComponent < 0.45,
+ bottom.redComponent > 0.6, bottom.redComponent < 0.9 else {
+ throw CaptureFailure(description: "source preview conversion self-test failed")
+ }
var upscaled = attachments
upscaled[.contentScale] = CGFloat(2)
guard let smaller = VideoImage(buffer, attachments: upscaled),
@@ -2191,6 +2304,8 @@ private struct PiikCapture {
try audioSelfTest()
} else if arguments.count == 2, arguments[1] == "--list" {
try await listSources()
+ } else if arguments.count > 1, arguments[1] == "--preview" {
+ try await previewSource(arguments)
} else if arguments.count == 2, arguments[1] == "--list-microphones" {
try listMicrophones()
} else if arguments.count == 2, arguments[1] == "--capture-microphone" {
diff --git a/scripts/app-package-targets.mjs b/scripts/app-package-targets.mjs
index 4c58292d..b9c9574b 100644
--- a/scripts/app-package-targets.mjs
+++ b/scripts/app-package-targets.mjs
@@ -5,6 +5,7 @@ export const CLOUDFLARED_SOURCE_SHA256 = "6c6901ae333508e1c529184bf90cb87ff59592
export const ANDROID_APP_PACKAGE_TARGET = {
id: "android-universal",
artifact: "piik-app-android-universal.apk",
+ mirrorArtifact: "piik-app-android-universal.zip",
};
export function androidVersionCode(version) {
diff --git a/scripts/build-android.mjs b/scripts/build-android.mjs
index 93c3ebfd..296b326a 100644
--- a/scripts/build-android.mjs
+++ b/scripts/build-android.mjs
@@ -9,6 +9,7 @@ import { writeAndroidAppIcon } from './app-icons.mjs';
import { CLOUDFLARED_VERSION, CLOUDFLARED_SOURCE_SHA256, ANDROID_APP_PACKAGE_TARGET, androidVersionCode } from './app-package-targets.mjs';
import { buildVersion } from './release-version.mjs';
import { assertCleanRevision } from './build-workspace.mjs';
+import { packageAndroidRelease } from './package-android-release.mjs';
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..');
const android = join(root, 'mobile', 'android');
@@ -164,14 +165,10 @@ if (!tunnelOnly) {
if (!badging.includes(`package: name='tv.piik.android' versionCode='${versionCode}' versionName='${versionName}'`) ||
/application-debuggable/.test(badging) || !badging.includes("native-code: 'arm64-v8a' 'x86_64'"))
throw new Error('Android release identity, ABIs or debuggable flag differs from the package contract.');
- const { id: target, artifact } = ANDROID_APP_PACKAGE_TARGET;
- const digest = createHash('sha256').update(readFileSync(apk)).digest('hex');
+ const { artifact } = ANDROID_APP_PACKAGE_TARGET;
+ assertCleanRevision(root, revision);
+ packageAndroidRelease(apk, output, version, revision);
assertCleanRevision(root, revision);
- mkdirSync(output, { recursive: true });
- copyFileSync(apk, join(output, artifact));
- writeFileSync(join(output, `${artifact}.sha256`), `${digest} ${artifact}\n`, 'ascii');
- writeFileSync(join(output, artifact.replace(/\.apk$/, '.release.json')),
- `${JSON.stringify({ schema: 2, version, revision, target, artifact, artifactSha256: digest }, null, 2)}\n`);
console.log(`Piik Android signed release APK: ${join(output, artifact)}`);
} else console.log(`Piik Android development APK: ${apk}`);
}
diff --git a/scripts/media-lifecycle-gate.ts b/scripts/media-lifecycle-gate.ts
index 1cd24696..3c1000aa 100644
--- a/scripts/media-lifecycle-gate.ts
+++ b/scripts/media-lifecycle-gate.ts
@@ -76,7 +76,7 @@ for (let iteration = 0; iteration < repeats && !signal.aborted; iteration++) for
let versions: Array<{ browser: string; app: boolean; appPort: number | null }> = [];
const result = { scenario: scenario.id, iteration: iteration + 1, passed: false, stage: "startup", error: "",
versions: [] as typeof versions,
- stages: [] as Array<{ name: string; firstPictureMs: number; route: string | null; peer: number }>,
+ stages: [] as Array<{ name: string; firstPictureMs: number; firstNewFrameMs?: number; recoveryMs?: number; route: string | null; peer: number }>,
samples: [] as Array<{ stage: string; host: MediaSample; viewer: MediaSample }>, cleanup: [] as unknown[] };
const stage = (name: string) => { result.stage = name; process.stdout.write(JSON.stringify({ case: scenario.id, stage: name }) + "\n"); };
const sample = async () => {
@@ -85,16 +85,31 @@ for (let iteration = 0; iteration < repeats && !signal.aborted; iteration++) for
result.samples.push({ stage: result.stage, host: host!, viewer: viewer! });
return { host: host!, viewer: viewer! };
};
- const media = async (name: string, previousPeer?: number, expectedRoute = scenario.sfu ? "sfu" : "peer") => {
+ const media = async (name: string, previousPeer?: number, expectedRoute = scenario.sfu ? "sfu" : "peer",
+ recovery?: { baseline: MediaSample; started: number }) => {
stage(name); const started = performance.now();
let first: MediaSample | undefined, firstAt = 0;
+ let previous = recovery?.baseline, firstNewFrameAt: number | undefined, recoveryError = "No new decoded frame";
while (performance.now() - started < 40_000) {
const { viewer } = await sample();
if (viewer.events.some(event => event.reason === "route-exhausted")) throw new Error("Route exhausted");
if (hasPicture(viewer) && viewer.route === expectedRoute && viewer.playbackPeerId !== previousPeer) {
- if (!first || first.playbackPeerId !== viewer.playbackPeerId) { first = viewer; firstAt = performance.now(); }
+ if (!first || first.playbackPeerId !== viewer.playbackPeerId) {
+ // Historical decoded counters survive an ICE restart. Recovery begins
+ // only after a new frame, then needs a complete continuity window.
+ if (!previous || viewer.frames > previous.frames) {
+ first = viewer; firstAt = performance.now(); firstNewFrameAt ??= firstAt;
+ }
+ }
else if (performance.now() - firstAt >= 3000) {
- checkContinuity(first, viewer);
+ try { checkContinuity(first, viewer); }
+ catch (error) {
+ if (!recovery) throw error;
+ recoveryError = error instanceof Error ? error.message : String(error);
+ first = undefined; previous = viewer;
+ await delay(250, undefined, { signal });
+ continue;
+ }
if (viewer.errors.length) throw new Error(`Viewer errors: ${viewer.errors.join("; ")}`);
const last = result.samples.at(-1)!;
for (const [index, page] of [last.host, viewer].entries()) {
@@ -103,13 +118,15 @@ for (let iteration = 0; iteration < repeats && !signal.aborted; iteration++) for
}
if (scenario.host && expectedRoute === "peer") checkAppPath(last.host, "prepare-edge", localAddresses);
if (scenario.viewer && expectedRoute === "peer") checkAppPath(viewer, "receive-offer", localAddresses);
- result.stages.push({ name, firstPictureMs: firstAt - started, route: viewer.route, peer: viewer.playbackPeerId });
+ result.stages.push({ name, firstPictureMs: firstAt - started,
+ ...(recovery ? { firstNewFrameMs: firstNewFrameAt! - recovery.started, recoveryMs: performance.now() - recovery.started } : {}),
+ route: viewer.route, peer: viewer.playbackPeerId });
return viewer;
}
} else first = undefined;
await delay(250, undefined, { signal });
}
- throw new Error(`No sustained ${expectedRoute} media during ${name}`);
+ throw new Error(`No sustained ${expectedRoute} media during ${name}${recovery ? `: ${recoveryError}` : ""}`);
};
try {
await waitForSample(deadline => {
@@ -150,12 +167,14 @@ for (let iteration = 0; iteration < repeats && !signal.aborted; iteration++) for
if (!scenario.host && !scenario.sfu) {
stage("ice-restart");
const before = await sample();
+ const restartAt = performance.now();
if (!await endpoints[1]!.call("restart-upstream")) throw new Error("No committed Peer route to restart");
await waitForSample(sample, row => row.host.events.filter(event => event.type === "restart-request").length >
before.host.events.filter(event => event.type === "restart-request").length && (!scenario.viewer ||
row.viewer.controls.filter(command => command.type === "receive-offer").length >
before.viewer.controls.filter(command => command.type === "receive-offer").length), 10_000);
- const restarted = await media("same-edge-recovered");
+ const restarted = await media("same-edge-recovered", undefined, undefined,
+ { baseline: result.samples.at(-1)!.viewer, started: restartAt });
if (restarted.playbackPeerId !== received.playbackPeerId) throw new Error("ICE restart replaced the playback connection");
received = restarted;
}
diff --git a/scripts/mirror-release.mjs b/scripts/mirror-release.mjs
index 903e866b..b33e21eb 100644
--- a/scripts/mirror-release.mjs
+++ b/scripts/mirror-release.mjs
@@ -8,7 +8,7 @@ const [directory, version, revision, option] = process.argv.slice(2);
if (option && option !== "--dry-run") {
throw new Error("Use mirror-release.mjs <artifacts> <version> <full-SHA> [--dry-run]");
}
-const { files } = readReleaseArtifacts(directory, version, revision);
+const { mirrorFiles: files } = readReleaseArtifacts(directory, version, revision);
if (files.some((file) => file.size > 100_000_000)) throw new Error("Gitee's 100 MB attachment limit is exceeded");
const repository = "TNTcraftHIM/Piik";
const api = `https://gitee.com/api/v5/repos/${repository}`;
diff --git a/scripts/package-android-release.mjs b/scripts/package-android-release.mjs
new file mode 100644
index 00000000..e1673a16
--- /dev/null
+++ b/scripts/package-android-release.mjs
@@ -0,0 +1,34 @@
+import { createHash } from "node:crypto";
+import { copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { createZip } from "./archive-tool.mjs";
+import { ANDROID_APP_PACKAGE_TARGET } from "./app-package-targets.mjs";
+
+// Wrap the already signed APK; neither distribution changes its installed bytes.
+export function packageAndroidRelease(apk, output, version, revision) {
+ const { id: target, artifact, mirrorArtifact } = ANDROID_APP_PACKAGE_TARGET;
+ mkdirSync(output, { recursive: true });
+ copyFileSync(apk, join(output, artifact));
+ const staging = mkdtempSync(join(tmpdir(), "piik-android-package-"));
+ try {
+ // zip updates existing archives, so each packaging starts with a fresh file.
+ const archive = join(staging, "distribution.zip");
+ const contents = join(staging, "contents");
+ mkdirSync(contents);
+ copyFileSync(apk, join(contents, artifact));
+ createZip(contents, archive);
+ copyFileSync(archive, join(output, mirrorArtifact));
+ } finally {
+ rmSync(staging, { recursive: true, force: true });
+ }
+ const hashes = [artifact, mirrorArtifact].map(name => {
+ const hash = createHash("sha256").update(readFileSync(join(output, name))).digest("hex");
+ writeFileSync(join(output, `${name}.sha256`), `${hash} ${name}\n`, "ascii");
+ return hash;
+ });
+ const descriptor = { schema: 2, version, revision, target, artifact, artifactSha256: hashes[0],
+ mirrorArtifact, mirrorArtifactSha256: hashes[1] };
+ writeFileSync(join(output, artifact.replace(/\.apk$/, ".release.json")), `${JSON.stringify(descriptor, null, 2)}\n`);
+ return descriptor;
+}
diff --git a/scripts/release-artifacts.mjs b/scripts/release-artifacts.mjs
index be40988e..f4e8d918 100644
--- a/scripts/release-artifacts.mjs
+++ b/scripts/release-artifacts.mjs
@@ -12,6 +12,7 @@ export function readReleaseArtifacts(directory, version, revision) {
const root = resolve(directory);
const files = new Map();
const packages = [];
+ const mirrorFiles = [];
const servers = [];
const targets = new Set();
function asset(name) {
@@ -38,6 +39,19 @@ export function readReleaseArtifacts(directory, version, revision) {
throw new Error(`Release artifact checksum mismatch: ${name}`);
}
packages.push(archive);
+ let mirror = archive;
+ if (descriptor.mirrorArtifact !== undefined || descriptor.mirrorArtifactSha256 !== undefined) {
+ if (target !== ANDROID_APP_PACKAGE_TARGET.id || descriptor.mirrorArtifact !== ANDROID_APP_PACKAGE_TARGET.mirrorArtifact) {
+ throw new Error(`Release mirror artifact is invalid: ${target}`);
+ }
+ mirror = asset(descriptor.mirrorArtifact);
+ if (mirror.sha256 !== descriptor.mirrorArtifactSha256 ||
+ readFileSync(asset(`${mirror.name}.sha256`).path, "utf8").trim() !== `${mirror.sha256} ${mirror.name}`) {
+ throw new Error(`Release mirror artifact checksum mismatch: ${target}`);
+ }
+ packages.push(mirror);
+ }
+ mirrorFiles.push(mirror);
const serverTarget = SERVER_PACKAGE_TARGETS.find(candidate => candidate.id === target);
if (serverTarget) {
if (asset(descriptor.manifest).sha256 !== descriptor.manifestSha256) {
@@ -53,6 +67,6 @@ export function readReleaseArtifacts(directory, version, revision) {
if (targets.size !== expected.length || expected.some((target) => !targets.has(target))) {
throw new Error("Publication requires every registered Server and App target");
}
- // Sidecars prove the local build; only installable packages are public assets.
- return { files: packages, servers, targets: [...targets] };
+ // Sidecars prove the local build; only distribution packages are public assets.
+ return { files: packages, mirrorFiles, servers, targets: [...targets] };
}
diff --git a/scripts/room-voice-gate.ts b/scripts/room-voice-gate.ts
index 3c0e9e0c..c369aa46 100644
--- a/scripts/room-voice-gate.ts
+++ b/scripts/room-voice-gate.ts
@@ -425,7 +425,7 @@ async function main(): Promise<void> {
check("source commentary defaults off, sends shared audio alongside room voice, and can stop independently");
await cdp.call("Page.bringToFront", {}, host.sessionId, Date.now() + 3000);
// A stationary hover does not wake idle playback controls.
- await ui(host, "hover", ".lr-top-right");
+ await ui(host, "hover", ".lr-header-controls");
await ui(host, "hover", ".lr-tv video");
await waitForSample(() => evaluate<boolean>(cdp!, host, "!!document.querySelector('.lr-tv video')&&!document.querySelector('.lr-tv video').paused", Date.now() + 3000), Boolean, 5000);
if (!await evaluate<boolean>(cdp, host, "document.querySelector('.lr-tv video').muted", Date.now() + 3000)) throw new Error("Host preview started with sound");
diff --git a/scripts/sync-upstream.mjs b/scripts/sync-upstream.mjs
new file mode 100644
index 00000000..843fc9d6
--- /dev/null
+++ b/scripts/sync-upstream.mjs
@@ -0,0 +1,117 @@
+import { spawnSync } from "node:child_process";
+import { existsSync, mkdirSync, writeFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+import { parseArgs } from "node:util";
+
+const { values } = parseArgs({
+ options: {
+ source: { type: "string" },
+ upstream: { type: "string" },
+ base: { type: "string" },
+ head: { type: "string" },
+ onto: { type: "string", default: "HEAD" },
+ out: { type: "string" },
+ help: { type: "boolean" },
+ },
+});
+
+if (values.help) {
+ console.log(`Prepare an isolated dependency patch stack on current upstream.
+
+node scripts/sync-upstream.mjs --source <fork URL or local clone> \
+ --upstream <official URL or local clone> --base <official base> \
+ --head <patch commit> --out <new directory> [--onto <upstream ref>]
+
+Only the new directory is written. Existing clones, Piik pins and remotes stay
+unchanged. Git replays the patch range; conflicts remain in the candidate for
+review. No push, package installation, build or dependency update is performed.
+The report and range-diff are saved inside the candidate's .git directory.`);
+} else {
+ try {
+ prepare(values);
+ } catch (error) {
+ console.error(error.message);
+ process.exitCode = 1;
+ }
+}
+
+function prepare(options) {
+ for (const key of ["source", "upstream", "base", "head", "out"]) {
+ if (!options[key]) throw new Error(`Missing --${key}; use --help.`);
+ }
+ const source = repository(options.source);
+ const upstream = repository(options.upstream);
+ const output = resolve(options.out);
+ // Creating a fresh repository leaves all existing worktrees and local edits alone.
+ mkdirSync(dirname(output), { recursive: true });
+ mkdirSync(output);
+ const git = (...args) => command(output, args);
+ git("init", "--quiet");
+ git("remote", "add", "patch-source", source);
+ git("remote", "add", "upstream", upstream);
+ console.log("Fetching the recorded patch range and upstream reference…");
+ git("fetch", "--no-tags", "--no-recurse-submodules", "--", "patch-source",
+ `${options.base}:refs/piik/base`, `${options.head}:refs/piik/patch`);
+ git("fetch", "--no-tags", "--no-recurse-submodules", "--", "upstream",
+ `${options.onto}:refs/piik/upstream`);
+ git("merge-base", "--is-ancestor", "refs/piik/base", "refs/piik/patch");
+ if (git("rev-list", "--merges", "refs/piik/base..refs/piik/patch")) {
+ throw new Error("The patch range contains merge commits; select a focused linear patch stack.");
+ }
+ const report = {
+ source,
+ upstream,
+ upstreamRef: options.onto,
+ base: git("rev-parse", "refs/piik/base^{commit}"),
+ patch: git("rev-parse", "refs/piik/patch^{commit}"),
+ upstreamHead: git("rev-parse", "refs/piik/upstream^{commit}"),
+ patchEquivalents: git("cherry", "refs/piik/upstream", "refs/piik/patch", "refs/piik/base"),
+ };
+ const patches = git("rev-list", "--reverse", "refs/piik/base..refs/piik/patch").split("\n").filter(Boolean);
+ const unapplied = new Set(report.patchEquivalents.split("\n")
+ .filter(line => line.startsWith("+ ")).map(line => line.slice(2)));
+ report.alreadyApplied = patches.filter(commit => !unapplied.has(commit));
+ git("checkout", "--quiet", "-b", "candidate", "refs/piik/patch");
+ const replay = spawnSync("git", ["rebase", "--onto", "refs/piik/upstream", "refs/piik/base"], {
+ cwd: output, encoding: "utf8", windowsHide: true,
+ env: { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_EDITOR: "true" },
+ });
+ if (replay.error) throw replay.error;
+ report.conflicts = git("diff", "--name-only", "--diff-filter=U");
+ report.status = replay.status === 0 ? "ready" : report.conflicts ? "conflict" : "failed";
+ report.replayLog = `${replay.stdout}${replay.stderr}`.trim();
+ if (replay.status === 0) {
+ report.candidate = git("rev-parse", "HEAD");
+ report.remaining = Number(git("rev-list", "--count", "refs/piik/upstream..HEAD"));
+ writeFileSync(join(output, ".git", "piik-upstream-range-diff.txt"),
+ report.remaining
+ ? git("range-diff", "--no-color", "refs/piik/base..refs/piik/patch", "refs/piik/upstream..HEAD") + "\n"
+ : "No local patches remain.\n");
+ }
+ const path = join(output, ".git", "piik-upstream-sync.json");
+ writeFileSync(path, JSON.stringify(report, null, 2) + "\n");
+ console.log(`${report.status}: ${output}\nReport: ${path}`);
+ if (replay.status !== 0) {
+ console.error(report.replayLog);
+ process.exitCode = 1;
+ }
+}
+
+function repository(value) {
+ if (existsSync(value)) return resolve(value);
+ const url = new URL(value);
+ if (url.protocol !== "https:" || url.username || url.password || url.search || url.hash) {
+ throw new Error("Use a local clone or an HTTPS repository URL without embedded credentials.");
+ }
+ return url.href;
+}
+
+function command(cwd, args) {
+ const result = spawnSync("git", args, {
+ cwd, encoding: "utf8", windowsHide: true, maxBuffer: 16 * 1024 * 1024,
+ env: { ...process.env, GIT_TERMINAL_PROMPT: "0" },
+ });
+ if (result.error) throw result.error;
+ if (result.status !== 0) throw new Error(result.stderr.trim() || `git ${args[0]} failed`);
+ return result.stdout.trim();
+}
diff --git a/src/client/components/living/AppReleaseNotice.tsx b/src/client/components/living/AppReleaseNotice.tsx
index a6cb965a..d3ac8c1b 100644
--- a/src/client/components/living/AppReleaseNotice.tsx
+++ b/src/client/components/living/AppReleaseNotice.tsx
@@ -15,9 +15,11 @@ export function AppReleaseNotice({ update }: { update: ReleaseUpdateNotice }) {
const { vis, t } = useCopy();
const warningId = useId();
const major = update.kind === "major-update";
- const label = `${t(labels[update.kind])} · ${update.version}`;
+ const label = `${t(labels[update.kind])} · ${update.version}${update.extractAPK ? " (ZIP)" : ""}`;
+ const updateHint = major ? t("client.update.majorHint") : vis ? update.version : label;
+ const hint = update.extractAPK ? `${updateHint} ${t("client.update.extractAPK")}` : updateHint;
return <div className="lr-client-release">
- <Tooltip kind="update-available" text={major ? t("client.update.majorHint") : vis ? update.version : label}>
+ <Tooltip kind="update-available" text={hint}>
<a className="lr-btn lr-client-update" href={update.url} target="_blank" rel="noopener noreferrer"
aria-label={label} aria-describedby={major ? warningId : undefined}>
<Glyph name={major ? "alert" : "arrowUp"} size={17} />
diff --git a/src/client/components/living/Header.tsx b/src/client/components/living/Header.tsx
index 0867644e..c4d947c0 100644
--- a/src/client/components/living/Header.tsx
+++ b/src/client/components/living/Header.tsx
@@ -147,7 +147,7 @@ export function HeaderControls({ diagnosticControl, appControl }: { diagnosticCo
</button>
);
return (
- <span className="lr-top-right lr-header-controls">
+ <span className="lr-header-controls">
{appControl}
<LanguageControl />
<Tooltip kind={theme === "dark" ? "hint-theme-light" : "hint-theme-dark"} text={vis ? undefined : themeTitle} place="below" align="end">
@@ -174,20 +174,20 @@ export function AppHeader({
const { t } = useCopy();
return (
<header className="lr-top">
- {homeHref !== "/client" && <a
- className="lr-brand"
- href={withBrowserDebug(homeHref)}
- aria-label={t("brand.home")}
- aria-current={atHome ? "page" : undefined}
- >
- {!atHome && <Glyph name="chevron" size={14} className="lr-home-arrow" />}
- <BrandMark size={34} motion="once" />
- </a>}
- <span className="lr-top-right">
+ <span className="lr-header-context">
+ {homeHref !== "/client" && <a
+ className="lr-brand"
+ href={withBrowserDebug(homeHref)}
+ aria-label={t("brand.home")}
+ aria-current={atHome ? "page" : undefined}
+ >
+ {!atHome && <Glyph name="chevron" size={14} className="lr-home-arrow" />}
+ <BrandMark size={34} motion="once" />
+ </a>}
{led}
- <HeaderControls diagnosticControl={diagnosticControl}
- appControl={homeHref !== "/client" ? <AppConnectionButton onConnected={onAppConnected} /> : undefined} />
</span>
+ <HeaderControls diagnosticControl={diagnosticControl}
+ appControl={homeHref !== "/client" ? <AppConnectionButton onConnected={onAppConnected} /> : undefined} />
</header>
);
}
diff --git a/src/client/components/living/segmented-control.css b/src/client/components/living/segmented-control.css
index 62de60b5..2ac055c1 100644
--- a/src/client/components/living/segmented-control.css
+++ b/src/client/components/living/segmented-control.css
@@ -58,6 +58,7 @@
.lr-lang > button { min-width: 0; padding: 0; }
.lr-toggle button, .lr-lang > button { cursor: pointer; }
.lr-toggle button:disabled, .lr-lang > button:disabled { cursor: default; }
+.lr-toggle button > .lr-glyph { flex: none; }
.lr-toggle > label { position: relative; min-width: 0; }
.lr-toggle > label > input { position: absolute; inset: 0; width: 100%; height: 100%; margin: 0; opacity: 0; cursor: pointer; }
.lr-toggle > label > input:disabled { cursor: default; }
diff --git a/src/client/components/living/use-anchored-popover.ts b/src/client/components/living/use-anchored-popover.ts
index 13ab355a..7e6ea99d 100644
--- a/src/client/components/living/use-anchored-popover.ts
+++ b/src/client/components/living/use-anchored-popover.ts
@@ -1,4 +1,4 @@
-import { useLayoutEffect, type RefObject } from "react";
+import { useLayoutEffect, useRef, type RefObject } from "react";
import { useSurfaceDocument } from "../../ui/panel-window";
// Native popovers own visibility, dismissal and focus. This only places a
@@ -11,6 +11,11 @@ export function useAnchoredPopover(
} = {},
) {
const ownerDocument = useSurfaceDocument();
+ const reposition = useRef<(() => void) | null>(null);
+ // A React layout change can move the anchor without resizing it.
+ useLayoutEffect(() => {
+ if (panel.current?.matches(":popover-open")) reposition.current?.();
+ });
useLayoutEffect(() => {
const menu = panel.current;
if (!enabled || !menu) return;
@@ -31,17 +36,21 @@ export function useAnchoredPopover(
return;
}
const above = Math.max(0, rect.top - top - 8), below = Math.max(0, bottom - rect.bottom - 8);
+ // Measure against the viewport, not the previous side's constrained size.
+ menu.style.maxWidth = `${right - left}px`;
+ menu.style.maxHeight = `${Math.min(maxHeight, bottom - top)}px`;
const height = Math.min(menu.scrollHeight, maxHeight);
// Scrollable palettes can prefer their requested side without requiring
// every row to fit; compact menus retain the full-content threshold.
const preferredHeight = Math.min(height, minVisibleHeight);
const openAbove = side === "above" ? above >= preferredHeight || above > below : below < preferredHeight && above > below;
- menu.style.maxWidth = `${right - left}px`;
- menu.style.maxHeight = `${Math.min(maxHeight, openAbove ? above : below)}px`;
+ const beside = openAbove ? above : below;
+ menu.style.maxHeight = `${Math.min(maxHeight, beside >= preferredHeight ? beside : bottom - top)}px`;
menu.classList.toggle("is-above", openAbove);
const x = align === "center" ? rect.left + (rect.width - menu.offsetWidth) / 2 : rect.right - menu.offsetWidth;
menu.style.left = `${Math.max(left, Math.min(x, right - menu.offsetWidth))}px`;
- menu.style.top = `${Math.max(top, openAbove ? rect.top - menu.offsetHeight - 8 : rect.bottom + 8)}px`;
+ const y = openAbove ? rect.top - menu.offsetHeight - 8 : rect.bottom + 8;
+ menu.style.top = `${Math.max(top, Math.min(y, bottom - menu.offsetHeight))}px`;
};
// The containing panel may reposition itself in the same resize. Measure
// after its layout owner has finished, including content/translation changes.
@@ -50,6 +59,7 @@ export function useAnchoredPopover(
view.cancelAnimationFrame(frame);
frame = view.requestAnimationFrame(place);
};
+ reposition.current = schedule;
const observer = new ResizeObserver(schedule);
const toggled = () => {
observer.disconnect();
@@ -70,6 +80,7 @@ export function useAnchoredPopover(
viewport?.addEventListener("scroll", schedule);
toggled();
return () => {
+ reposition.current = null;
observer.disconnect();
view.cancelAnimationFrame(frame);
menu.removeEventListener("beforetoggle", schedule);
diff --git a/src/client/components/living/use-host-preview-playback.ts b/src/client/components/living/use-host-preview-playback.ts
index bf1f61d1..980ae5a1 100644
--- a/src/client/components/living/use-host-preview-playback.ts
+++ b/src/client/components/living/use-host-preview-playback.ts
@@ -26,6 +26,11 @@ export function useHostPreviewPlayback(
setSuspended(false);
return;
}
+ // Native PiP/fullscreen sound controls must not turn shared audio into
+ // local monitoring. The Host's volume control owns outgoing sound.
+ const keepMuted = () => { video.muted = true; };
+ keepMuted();
+ video.addEventListener("volumechange", keepMuted);
const syncPlayback = () => {
const shouldPause = shouldPauseLocalPreview(document.visibilityState, document.hasFocus(), isPictureInPicture(video), autoPause);
setSuspended(shouldPause);
@@ -39,6 +44,7 @@ export function useHostPreviewPlayback(
pictureEvents.forEach(event => video.addEventListener(event, syncPlayback));
syncPlayback();
return () => {
+ video.removeEventListener("volumechange", keepMuted);
document.removeEventListener("visibilitychange", syncPlayback);
window.removeEventListener("blur", syncPlayback);
window.removeEventListener("focus", syncPlayback);
diff --git a/src/client/lib/release-update.ts b/src/client/lib/release-update.ts
index f0cd569f..3d5e1534 100644
--- a/src/client/lib/release-update.ts
+++ b/src/client/lib/release-update.ts
@@ -16,6 +16,7 @@ interface PublishedRelease {
url: string;
releaseURL: string;
publishedAt?: string;
+ extractAPK?: boolean;
}
export interface ReleaseUpdateNotice extends PublishedRelease {
kind: "update-available" | "major-update" | "different-build" | "official-release";
@@ -31,21 +32,25 @@ function normalizeVersion(value: unknown): string | null {
return version ? `v${version}` : null;
}
-function packageDownloadURL(release: PublishedRelease, assets: unknown, target?: string): string {
- if (!target || !/^[a-z0-9]+-[a-z0-9]+$/.test(target) || !Array.isArray(assets)) return release.url;
- const extension = target === "android-universal" ? "apk" : "zip";
+function packageDownload(release: PublishedRelease, assets: unknown, target?: string): Pick<PublishedRelease, "url" | "extractAPK"> {
+ if (!target || !/^[a-z0-9]+-[a-z0-9]+$/.test(target) || !Array.isArray(assets)) return { url: release.url };
+ const android = target === "android-universal";
+ const mirror = release.url.startsWith(MIRROR_RELEASE_PAGE);
+ const extension = android && !mirror ? "apk" : "zip";
const names = [`piik-app-${target}.${extension}`];
- if (release.revision && extension === "zip") names.push(`piik-app-${target}-${release.revision.slice(0, 7)}.zip`);
+ if (release.revision && !android) names.push(`piik-app-${target}-${release.revision.slice(0, 7)}.zip`);
for (const name of names) {
const matches = assets.filter(asset => asset?.name === name);
if (!matches.length) continue;
const asset = matches[0];
const expectedURL = `${release.url.replace("/releases/tag/", "/releases/download/")}/${name}`;
- return matches.length === 1 && (release.url.startsWith(MIRROR_RELEASE_PAGE) ||
+ return matches.length === 1 && (mirror ||
(Number.isSafeInteger(asset.size) && asset.size > 0 && asset.state === "uploaded")) &&
- asset.browser_download_url === expectedURL ? asset.browser_download_url : release.url;
+ asset.browser_download_url === expectedURL
+ ? { url: asset.browser_download_url, ...(android && mirror ? { extractAPK: true } : {}) }
+ : { url: release.url };
}
- return release.url;
+ return { url: release.url };
}
export function parseReleaseMetadata(value: unknown, packageTarget?: string): PublishedRelease | null {
@@ -66,7 +71,7 @@ export function parseReleaseMetadata(value: unknown, packageTarget?: string): Pu
releaseURL: RELEASE_PAGE + version,
...(Number.isFinite(publishedTime) ? { publishedAt: new Date(publishedTime).toISOString() } : {}),
};
- return { ...normalized, url: packageDownloadURL(normalized, release.assets, packageTarget) };
+ return { ...normalized, ...packageDownload(normalized, release.assets, packageTarget) };
}
export function parseMirrorReleaseMetadata(value: unknown, packageTarget?: string): PublishedRelease | null {
@@ -83,7 +88,7 @@ export function parseMirrorReleaseMetadata(value: unknown, packageTarget?: strin
if (!normalized) return null;
const mirror = { ...normalized, url: MIRROR_RELEASE_PAGE + normalized.version,
releaseURL: MIRROR_RELEASE_PAGE + normalized.version };
- return { ...mirror, url: packageDownloadURL(mirror, release.assets, packageTarget) };
+ return { ...mirror, ...packageDownload(mirror, release.assets, packageTarget) };
}
export function releaseUpdateNotice(
diff --git a/src/client/lib/room-interactions.ts b/src/client/lib/room-interactions.ts
index 69b54130..f5d2e437 100644
--- a/src/client/lib/room-interactions.ts
+++ b/src/client/lib/room-interactions.ts
@@ -66,6 +66,11 @@ export class RoomInteractionSession {
this.state = { ...this.state, ...patch };
this.listeners.forEach(listener => listener());
}
+ private startCooldown() {
+ clearTimeout(this.cooldownTimer);
+ this.update({ coolingDown: true });
+ this.cooldownTimer = setTimeout(() => this.update({ coolingDown: false }), INTERACTION_INTERVAL_MS);
+ }
authenticated(peerId: string) {
if (this.closed) return;
@@ -105,7 +110,11 @@ export class RoomInteractionSession {
const beganAt = Math.min(now, message.occurredAt - this.serverOffset);
const own = message.sender.peerId === this.state.peerId;
const confirmsPending = own && message.requestId === this.requestId;
- if (confirmsPending) this.requestId = null;
+ if (confirmsPending) {
+ this.requestId = null;
+ // The Host's interval starts at acceptance, after outbound queueing.
+ this.startCooldown();
+ }
const confirmed = confirmsPending ? message : this.state.confirmed;
const pending = confirmsPending ? null : this.state.pending;
if (!pending) clearTimeout(this.pendingTimer);
@@ -150,9 +159,9 @@ export class RoomInteractionSession {
// The timeout releases the send control, not this request's identity.
// A late result may settle it until another submission or connection replaces it.
this.requestId = requestId;
- this.update({ pending: { requestId, payload }, coolingDown: true, error: null });
+ this.update({ pending: { requestId, payload }, error: null });
this.pendingTimer = setTimeout(() => this.update({ pending: null, error: "unconfirmed" }), 5000);
- this.cooldownTimer = setTimeout(() => this.update({ coolingDown: false }), INTERACTION_INTERVAL_MS);
+ this.startCooldown();
return true;
}
diff --git a/src/client/locales/en.ts b/src/client/locales/en.ts
index fb8e8017..80e134b3 100644
--- a/src/client/locales/en.ts
+++ b/src/client/locales/en.ts
@@ -297,6 +297,7 @@ export const en: Record<CopyKey, string> = {
"client.update.major": "New major Piik release",
"client.update.majorHint": "This update includes breaking changes. Read the release notes before upgrading.",
"client.update.downloads": "Gitee mirror",
+ "client.update.extractAPK": "Extract the ZIP, then install the APK.",
"client.update.differentBuild": "This build differs from the official release",
"client.update.official": "An official Piik release is available",
diff --git a/src/client/locales/zh.ts b/src/client/locales/zh.ts
index bc739f95..0210ada6 100644
--- a/src/client/locales/zh.ts
+++ b/src/client/locales/zh.ts
@@ -297,6 +297,7 @@ export const zh = {
"client.update.major": "发现新的 Piik 大版本",
"client.update.majorHint": "此更新含不兼容改动。升级前请查看版本说明。",
"client.update.downloads": "Gitee 备用下载",
+ "client.update.extractAPK": "解压 ZIP 后安装 APK。",
"client.update.differentBuild": "当前构建与官方发行版不同",
"client.update.official": "可获取 Piik 正式发行版",
diff --git a/src/client/media/candidate-quality-probe.ts b/src/client/media/candidate-quality-probe.ts
index 819bee98..ff515a4f 100644
--- a/src/client/media/candidate-quality-probe.ts
+++ b/src/client/media/candidate-quality-probe.ts
@@ -41,7 +41,6 @@ function deliveredVideoSample(
sampleWindowMs,
frameWidth,
frameHeight,
- framesPerSecond,
bitrateKbps,
intervalFramesDecoded,
} = metrics;
@@ -50,18 +49,16 @@ function deliveredVideoSample(
sampleWindowMs === null ||
frameWidth === null ||
frameHeight === null ||
- framesPerSecond === null ||
intervalFramesDecoded === null ||
!Number.isFinite(sampleTimestampMs) ||
!Number.isFinite(sampleWindowMs) ||
!Number.isFinite(frameWidth) ||
!Number.isFinite(frameHeight) ||
- !Number.isFinite(framesPerSecond) ||
+ !Number.isFinite(intervalFramesDecoded) ||
frameWidth <= 0 ||
frameHeight <= 0 ||
sampleWindowMs < 1_000 ||
sampleWindowMs > 5_000 ||
- framesPerSecond <= 0 ||
intervalFramesDecoded <= 0
) {
return null;
@@ -70,7 +67,8 @@ function deliveredVideoSample(
timestampMs: sampleTimestampMs,
windowMs: sampleWindowMs,
pixels: frameWidth * frameHeight,
- framesPerSecond: Math.round(framesPerSecond),
+ // Native framesPerSecond covers the last second, not this comparison window.
+ framesPerSecond: Math.round((intervalFramesDecoded * 1_000) / sampleWindowMs),
bitrateKbps:
typeof bitrateKbps === "number" &&
Number.isFinite(bitrateKbps) &&
diff --git a/src/client/media/viewer-audio.ts b/src/client/media/viewer-audio.ts
index d740c940..94022a7b 100644
--- a/src/client/media/viewer-audio.ts
+++ b/src/client/media/viewer-audio.ts
@@ -13,6 +13,8 @@ export class ViewerAudio {
private source: MediaStreamAudioSourceNode | null = null;
private track: MediaStreamTrack | null = null;
private disposed = false;
+ private nativeVolume: number;
+ private nativeMuted: boolean;
constructor(
private readonly video: HTMLMediaElement,
@@ -23,6 +25,8 @@ export class ViewerAudio {
) {
this.level = video.volume;
this.muted = video.muted;
+ this.nativeVolume = video.volume;
+ this.nativeMuted = video.muted;
video.addEventListener("play", this.onPlay);
video.addEventListener("volumechange", this.onNativeVolume);
for (const event of ["pause", "ended"]) {
@@ -139,6 +143,10 @@ export class ViewerAudio {
}
this.video.volume = Math.min(this.level, 1);
this.video.muted = this.muted || this.level === 0 || boosted;
+ // iPhone may ignore volume writes; echoes must match actual native state,
+ // while the requested listening level stays owned by the gain graph.
+ this.nativeVolume = this.video.volume;
+ this.nativeMuted = this.video.muted;
};
private readonly onPlay = (): void => {
@@ -147,13 +155,15 @@ export class ViewerAudio {
};
private readonly onNativeVolume = (): void => {
- if (this.disposed || this.context) return;
+ if (this.disposed) return;
const { volume, muted } = this.video;
- // sync() also mutes zero volume. Its queued event is not a user mute.
- if (this.level === volume && (this.muted || this.level === 0) === muted) return;
+ // Ignore our queued writes, including the native mute while WebAudio plays.
+ if (volume === this.nativeVolume && muted === this.nativeMuted) return;
this.level = volume;
this.muted = muted;
- this.onChange(this.snapshot);
+ // PiP/native controls now own listening intent. Retire the parallel graph
+ // before honoring an unmute; later explicit gain adjustment can restore it.
+ this.useNativeControls();
};
private disableBoost(): void {
diff --git a/src/client/media/viewer-sfu-route.ts b/src/client/media/viewer-sfu-route.ts
index a0d4ef22..7f84f852 100644
--- a/src/client/media/viewer-sfu-route.ts
+++ b/src/client/media/viewer-sfu-route.ts
@@ -25,6 +25,7 @@ interface ViewerSubscriberTransport {
getRemoteInput?(): RTCDataChannel | null;
connect(config: SfuConnectionConfig): Promise<boolean>;
updateConfig(config: SfuConnectionConfig): void;
+ isConnected(): boolean;
acceptSignal(message: SfuSignalMessage): Promise<void>;
reconnect(): boolean;
resync(): void;
@@ -56,7 +57,7 @@ interface ViewerSfuRouteEvents {
activatePeer: (
assignment: ParticipantRouteAssignment,
revision?: number,
- ) => boolean | void | Promise<boolean | void>;
+ ) => boolean | "waiting" | void | Promise<boolean | "waiting" | void>;
preparePeer?: (
assignment: ParticipantRouteAssignment | null,
revision?: number,
@@ -446,6 +447,8 @@ export class ViewerSfuRoute {
} else if (
this.pending === slot &&
!slot.failed &&
+ !slot.readySent &&
+ this.route.getPhase() === "prepare" &&
slot.qualityProbe
) {
if (this.events.qualityProbeEligible?.() === false) {
@@ -471,6 +474,9 @@ export class ViewerSfuRoute {
onState: (state: "connected" | "reconnecting") => {
if (this.active === slot && !slot.failed) {
this.events.onSfuState?.(state, slot.revision);
+ } else if (this.pending === slot && !slot.failed && state === "connected") {
+ this.sendPendingReady(slot);
+ void this.queueTransition(() => this.promotePendingSfu(slot));
}
},
onDisconnected: (failure?: "VIDEO_REJECTED") => this.failSubscriberSlot(slot, failure),
@@ -551,6 +557,14 @@ export class ViewerSfuRoute {
);
}
+ retryPeerActivation(connectionId: string): boolean {
+ const token = this.route.token();
+ if (this.closed || this.resyncing || !token || !this.route.owns(token, "active") ||
+ this.pendingPeer?.revision !== token.revision || this.pendingPeer.connectionId !== connectionId) return false;
+ void this.queueActiveRoute(token, false);
+ return true;
+ }
+
async acceptSignal(message: SfuSignalMessage): Promise<void> {
const slot = [this.pending, this.active].find((slot) =>
slot && !slot.failed && slot.connectionId === message.connectionId &&
@@ -663,6 +677,7 @@ export class ViewerSfuRoute {
if (!this.route.owns(token, "active")) {
return;
}
+ if (promoted === "waiting") return;
if (pendingPeer?.revision === token.revision && promoted !== true) {
// Readiness may reach the server before a candidate's local failure.
// Once committed, report against the exact active connection for recovery.
@@ -785,6 +800,7 @@ export class ViewerSfuRoute {
this.pending !== slot ||
slot.failed ||
slot.readySent ||
+ !slot.subscriber.isConnected() ||
!slot.decodedFrame ||
(slot.qualityProbe !== null && slot.qualityResult !== "approved") ||
(slot.qualityProbe !== null &&
@@ -806,6 +822,7 @@ export class ViewerSfuRoute {
const assignment = this.route.getActiveAssignment();
if (
this.pending !== slot ||
+ !slot.subscriber.isConnected() ||
!slot.stream ||
!slot.decodedFrame ||
!token ||
diff --git a/src/client/native/native-viewer-peer.ts b/src/client/native/native-viewer-peer.ts
index b03b5c5d..bf4f0208 100644
--- a/src/client/native/native-viewer-peer.ts
+++ b/src/client/native/native-viewer-peer.ts
@@ -612,7 +612,8 @@ class NativeViewerPeer implements ViewerMediaPeer {
readFramesDecoded: () => bridge.decodedVideoFrames(),
owns: () =>
!this.disposed && this.bridge === bridge && this.connectionId === connectionId,
- onProof: () => this.events.onFirstDecodedFrame?.(connectionId) ?? true,
+ onProof: () => this.isConnected() &&
+ (this.events.onFirstDecodedFrame?.(connectionId) ?? true),
});
this.statsTimer = window.setInterval(() => {
void this.updateStats(bridge, connectionId);
diff --git a/src/client/pages/ControlsPreview.tsx b/src/client/pages/ControlsPreview.tsx
index 122b533b..a0a740f8 100644
--- a/src/client/pages/ControlsPreview.tsx
+++ b/src/client/pages/ControlsPreview.tsx
@@ -74,6 +74,7 @@ export function ControlsPreview() {
const [launchLan, setLaunchLan] = useState("192.0.2.10");
const [policy, setPolicy] = useState<"open" | "private">("open");
const [roomPassword, setRoomPassword] = useState(false);
+ const [inviteAvailable, setInviteAvailable] = useState(true);
const [includeCredential, setIncludeCredential] = useState(true);
const roomLink = `https://piik.example/r/9527${includeCredential ? "#v=preview" : ""}`;
const [paused, setPaused] = useState(false);
@@ -204,7 +205,7 @@ export function ControlsPreview() {
<SwitchItem checked disabled locked onChange={() => undefined} label={t("host.advanced.route.sfuOnly")}
note={t("host.advanced.route.sfuOnlyHint")} hint="hint-route-sfu" />
</div>
- <div className="cp-tools" id="admission-preview">
+ <div className="cp-tools lr-row-group" id="admission-preview">
<span className="lr-toggle" role="group" aria-label={t("host.policy")}>
{(["open", "private"] as const).map(value => <Tooltip key={value} kind={value === "open" ? "hint-admission-code" : "hint-policy-private"}
text={vis ? undefined : `${t(`host.policy.${value}`)}\n${t(value === "private" && roomPassword ? "host.policy.privatePasswordHint" : `host.policy.${value}Hint`)}`}>
@@ -213,14 +214,19 @@ export function ControlsPreview() {
<Glyph name={value === "open" ? "globe" : "lock"} size={19} />{vis ? null : <span className="lr-cap">{t(`host.policy.${value}`)}</span>}
</button></Tooltip>)}
</span>
+ {!inviteAvailable && <Pill icon="link" tone="off" comic="hint-invite-link"
+ label={t(policy === "open" ? "host.invite.emptyOpen" : roomPassword ? "host.invite.emptyPassword" : "host.invite.emptyPrivate")} />}
+ </div>
+ <div className="cp-tools">
<SwitchItem checked={roomPassword} onChange={setRoomPassword} label={t("host.password.set")} />
+ <SwitchItem checked={inviteAvailable} onChange={setInviteAvailable} label={en ? "Active invitation" : "邀请链接有效"} />
</div>
<div className="cp-tools">
<RoomAdmissionBadge policy="open" passwordEnabled={false} />
<RoomAdmissionBadge policy="private" passwordEnabled />
<RoomAdmissionBadge policy="private" passwordEnabled={false} />
</div>
- <div className="lr-invite-field">
+ {inviteAvailable && <div className="lr-invite-field">
<Tooltip kind="hint-invite-link" text={roomLink} className="lr-invite-hint">
<input className="lr-invite-url" readOnly dir="ltr" value={roomLink}
aria-label={t(includeCredential ? "host.invite" : "host.invite.address")} />
@@ -229,8 +235,8 @@ export function ControlsPreview() {
<SwitchItem checked={includeCredential} onChange={setIncludeCredential}
label={t("host.invite.includeCredential")} note={t("host.invite.credentialHint")} hint="hint-invite-link" />
</span>
- </div>
- {!includeCredential && policy === "private" && !roomPassword ? <Pill icon="lock" tone="warn"
+ </div>}
+ {inviteAvailable && !includeCredential && policy === "private" && !roomPassword ? <Pill icon="lock" tone="warn"
comic="hint-policy-private" label={t("host.invite.credentialRequired")} /> : null}
<p>{en ? "The website's architecture picker uses the same sliding style with native radio buttons. Arrow keys change the choice."
: "官网的架构选择也复用滑动样式,使用原生单选项;聚焦后可用方向键切换。"}</p>
diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx
index f2f7cf43..0fdd3d69 100644
--- a/src/client/pages/HostPage.tsx
+++ b/src/client/pages/HostPage.tsx
@@ -2911,11 +2911,14 @@ export function HostPage({
if (!isCurrentShare(generation, shareGeneration)) {
audio.sourceStream.getTracks().forEach(track => track.stop()); audio.dispose(); return;
}
+ if (!captured.getVideoTracks().some(track => track.readyState === "live")) {
+ throw new DOMException("Browser capture ended before sharing started", "NotReadableError");
+ }
} catch (error) {
audio.sourceStream.getTracks().forEach(track => track.stop()); audio.dispose();
if (isCurrentShare(generation, shareGeneration)) {
activeGenerationRef.current = null;
- disposeResources(false); setNoticeError(error, "sourceAudio"); setPhase("error");
+ disposeResources(false); setCaptureError(error, selection.source, "capture"); setPhase("error");
}
return;
}
@@ -3256,6 +3259,9 @@ export function HostPage({
audio.dispose();
return;
}
+ if (!captured.getVideoTracks().some(track => track.readyState === "live")) {
+ throw new DOMException("Browser capture ended before source replacement", "NotReadableError");
+ }
} catch (error) {
audio.dispose();
throw error;
diff --git a/src/client/pages/ViewerPage.tsx b/src/client/pages/ViewerPage.tsx
index 9652f10c..5dc3bb49 100644
--- a/src/client/pages/ViewerPage.tsx
+++ b/src/client/pages/ViewerPage.tsx
@@ -958,6 +958,7 @@ export function ViewerPage({
snapshot !== null &&
snapshot.peerId === probe.parentPeerId &&
probe.peer?.hasConnectionId(snapshot.connectionId) === true &&
+ probe.peer.isConnected() &&
probe.decodedFrame
);
}
@@ -1019,6 +1020,7 @@ export function ViewerPage({
discardPendingPeer();
return false;
}
+ if (!probe.peer.isConnected()) return "waiting";
const previousPeer = peerRef.current;
const activatedPeer = probe.peer;
activatedPeer.stopDecodedFrameProof();
@@ -1358,7 +1360,7 @@ export function ViewerPage({
if (snapshot.connectionId !== probe.candidateConnectionId) return;
probe.snapshot = snapshot;
noteReceiveFailure(snapshot, probe.revision);
- if (probe.qualityProbe) {
+ if (probe.qualityProbe && !probe.readySent) {
if (
pageSuspended ||
currentHostPaused ||
@@ -1388,6 +1390,7 @@ export function ViewerPage({
}
}
provePendingPeer();
+ if (peer.isConnected()) viewerSfuRoute?.retryPeerActivation(snapshot.connectionId);
} else if (active && peerRef.current === peer) {
noteReceiveFailure(snapshot, currentRouteRevision);
activePeerMetrics = snapshot.metrics;
@@ -1415,12 +1418,6 @@ export function ViewerPage({
},
onRecoveryExhausted: (parentPeerId, connectionId): boolean => {
if (pendingPeer === probe && peer.hasConnectionId(connectionId)) {
- signal.send({
- type: "route-failed",
- revision: probe.revision,
- phase: "prepare",
- connectionId,
- });
probe.peer = null;
probe.stream = null;
probe.snapshot = null;
@@ -1428,6 +1425,14 @@ export function ViewerPage({
probe.connectedSent = false;
probe.readySent = false;
peer.dispose();
+ if (!viewerSfuRoute?.retryPeerActivation(connectionId)) {
+ signal.send({
+ type: "route-failed",
+ revision: probe.revision,
+ phase: "prepare",
+ connectionId,
+ });
+ }
return true;
}
return peerRef.current === peer
diff --git a/src/client/remote-control/input-route.ts b/src/client/remote-control/input-route.ts
index 33136ac5..b27751df 100644
--- a/src/client/remote-control/input-route.ts
+++ b/src/client/remote-control/input-route.ts
@@ -2,6 +2,8 @@ export const INPUT_CHANNEL = "piik-remote-input-prototype-v1";
export type InputIdentity = { endpointId: string; secret: string };
type Packet = { id?: string; epoch?: string; data?: string; closed?: boolean; ready?: boolean };
export type InputTransport = Pick<RTCDataChannel, "readyState" | "bufferedAmount" | "send" | "close" | "addEventListener" | "removeEventListener">;
+// Local relay metadata bound, independent of ciphertext authorization or route age.
+const MAX_UNCLOSED_EPOCHS_PER_CHILD = 128;
function packet(data: unknown): Packet | null {
if (typeof data !== "string" || data.length > 6144) return null;
@@ -26,7 +28,7 @@ export class InputRoute {
private identity: InputIdentity | null = null;
private counter = { sent: 0n };
private register: ((identity: InputIdentity) => boolean) | null = null;
- private children = new Map<RTCDataChannel, Map<string, string>>();
+ private children = new Map<RTCDataChannel, Map<string, Set<string>>>();
constructor(private readonly changed: () => void) {}
get current(): InputTransport | null { return this.endpoint; }
@@ -46,10 +48,7 @@ export class InputRoute {
this.up.removeEventListener("message", this.received);
this.up.removeEventListener("close", this.lost);
this.up.removeEventListener("open", this.opened);
- for (const ids of this.children.values()) {
- for (const [id, epoch] of ids) send(this.up, { id, epoch, closed: true });
- ids.clear();
- }
+ for (const child of this.children.keys()) this.loseChild(child);
}
this.retireEndpoint();
this.ready = false;
@@ -89,14 +88,20 @@ export class InputRoute {
this.endpoint = null;
this.changed();
}
+ private loseChild(channel: RTCDataChannel, detach = false): void {
+ const ids = this.children.get(channel);
+ if (!ids) return;
+ if (detach) this.children.delete(channel);
+ for (const [id, epochs] of ids) for (const epoch of epochs) send(this.up, { id, epoch, closed: true });
+ ids.clear();
+ }
attachChild(channel: RTCDataChannel, connection: RTCPeerConnection): void {
- const ids = new Map<string, string>();
+ const ids = new Map<string, Set<string>>();
this.children.set(channel, ids);
connection.addEventListener("connectionstatechange", () => {
if (!this.children.has(channel)) return;
if (connection.connectionState === "disconnected" || connection.connectionState === "failed" || connection.connectionState === "closed") {
- for (const [id, epoch] of ids) send(this.up, { id, epoch, closed: true });
- ids.clear();
+ this.loseChild(channel);
}
});
channel.addEventListener("open", () => send(channel, { ready: this.ready }));
@@ -104,14 +109,22 @@ export class InputRoute {
const p = packet(event.data); if (!p || !this.children.has(channel)) return;
if (p.ready !== undefined) { send(channel, { ready: this.ready }); return; }
if (!this.ready || !p.id || !p.epoch) return;
- if (p.closed) { if (ids.get(p.id) === p.epoch) ids.delete(p.id); }
- else { if (!ids.has(p.id) && ids.size >= 20) return; ids.set(p.id, p.epoch); }
+ const epochs = ids.get(p.id);
+ if (p.closed) {
+ epochs?.delete(p.epoch);
+ if (epochs?.size === 0) ids.delete(p.id);
+ } else if (!epochs?.has(p.epoch)) {
+ if (!epochs && ids.size >= 20) return;
+ if ([...ids.values()].reduce((total, epochs) => total + epochs.size, 0) >= MAX_UNCLOSED_EPOCHS_PER_CHILD) {
+ this.loseChild(channel, true);
+ channel.close();
+ return;
+ }
+ if (epochs) epochs.add(p.epoch); else ids.set(p.id, new Set([p.epoch]));
+ }
send(this.up, p);
});
- channel.addEventListener("close", () => {
- if (!this.children.delete(channel)) return;
- for (const [id, epoch] of ids) send(this.up, { id, epoch, closed: true });
- });
+ channel.addEventListener("close", () => this.loseChild(channel, true));
if (channel.readyState === "open") send(channel, { ready: this.ready });
}
dispose(): void {
diff --git a/src/client/room-voice/browser-endpoint.ts b/src/client/room-voice/browser-endpoint.ts
index ce85673b..be696c38 100644
--- a/src/client/room-voice/browser-endpoint.ts
+++ b/src/client/room-voice/browser-endpoint.ts
@@ -76,10 +76,11 @@ export class BrowserVoiceEndpoint {
if (localOnly) return levels;
const connections = [...this.connections.values()].filter(connection => !connection.config.pending && connection.pc.connectionState === "connected");
await Promise.all(connections.map(async connection => {
+ const received = [...this.received].filter(([, binding]) => binding.connectionId === connection.config.connectionId);
const stats = await connection.pc.getStats();
- if (!this.current(connection)) return;
- for (const binding of this.received.values()) {
- if (binding.connectionId !== connection.config.connectionId) continue;
+ if (!this.current(connection) || connection.config.pending || connection.pc.connectionState !== "connected") return;
+ for (const [key, binding] of received) {
+ if (this.received.get(key) !== binding) continue;
const publisher = this.state?.publications.find(pub => pub.id === binding.publicationId)?.peerId;
if (!publisher) continue;
for (const row of stats.values()) if (row.type === "inbound-rtp" && row.kind === "audio" && row.trackIdentifier === binding.track.id) {
@@ -92,16 +93,18 @@ export class BrowserVoiceEndpoint {
async connectionMetrics(): Promise<readonly VoiceConnectionMetric[]> {
const connections = [...this.connections.values()].filter(connection => !connection.config.pending && connection.pc.connectionState === "connected");
const samples = await Promise.all(connections.map(async connection => {
+ const received = [...this.received].filter(([, binding]) => binding.connectionId === connection.config.connectionId);
+ const senders = [...connection.senders].map(([publicationId, sender]) => ({ publicationId, sender, track: sender.track }));
const report = await connection.pc.getStats();
if (!this.current(connection) || connection.config.pending || connection.pc.connectionState !== "connected") return [];
- const samples: VoiceConnectionMetric[] = [], retained = new Set<string>();
const { config } = connection;
+ const samples: VoiceConnectionMetric[] = [];
+ const retained = new Set([...config.send.map(id => `send/${id}`), ...config.receive.map(id => `receive/${id}`)]);
const viaPeerId = config.transport === "sfu" ? null : config.parentPeerId === this.peerId ? config.childPeerId : config.parentPeerId ?? null;
const sample = (publicationId: string, track: MediaStreamTrack, direction: "send" | "receive") => {
const publication = this.state?.publications.find(publication => publication.id === publicationId);
if (!publication || track.readyState === "ended") return;
const key = `${direction}/${publicationId}`;
- retained.add(key);
const previous = connection.metrics.get(key) ?? createStatsAccumulator();
connection.metrics.set(key, previous);
samples.push({ connectionId: config.connectionId, peerId: publication.peerId, viaPeerId,
@@ -109,11 +112,11 @@ export class BrowserVoiceEndpoint {
metrics: collectConnectionMetricsFromReport(report, direction, previous,
{ kind: "audio", trackIdentifier: track.id, audioTrackIdentifier: track.id }) });
};
- for (const binding of this.received.values()) {
- if (binding.connectionId === config.connectionId && config.receive.includes(binding.publicationId)) sample(binding.publicationId, binding.track, "receive");
+ for (const [key, binding] of received) {
+ if (this.received.get(key) === binding && config.receive.includes(binding.publicationId)) sample(binding.publicationId, binding.track, "receive");
}
- for (const [publicationId, sender] of connection.senders) {
- if (config.send.includes(publicationId) && sender.track) sample(publicationId, sender.track, "send");
+ for (const { publicationId, sender, track } of senders) {
+ if (config.send.includes(publicationId) && connection.senders.get(publicationId) === sender && sender.track === track && track) sample(publicationId, track, "send");
}
for (const key of connection.metrics.keys()) if (!retained.has(key)) connection.metrics.delete(key);
return samples;
@@ -331,10 +334,16 @@ export class BrowserVoiceEndpoint {
private async configureSenders(connection: Connection): Promise<void> {
const bitrate = this.state?.bitrate ?? 32_000;
for (const sender of connection.senders.values()) {
- const parameters = sender.getParameters();
- if (!parameters.encodings.length || !parameters.codecs?.length) continue;
- if (parameters.encodings[0]?.maxBitrate !== audioSenderCeiling(parameters, bitrate)) {
- await this.wait(connection, () => configureAudioSender(sender, bitrate));
+ try {
+ const parameters = sender.getParameters();
+ if (!parameters.encodings.length || !parameters.codecs?.length) continue;
+ if (parameters.encodings[0]?.maxBitrate !== audioSenderCeiling(parameters, bitrate)) {
+ await this.wait(connection, () => configureAudioSender(sender, bitrate));
+ }
+ } catch (error) {
+ if (!this.current(connection)) throw error;
+ // A rejected tuning request leaves the negotiated transport usable.
+ // Readback on the next explicit reconcile retries an unapplied ceiling.
}
}
}
@@ -345,8 +354,10 @@ export class BrowserVoiceEndpoint {
async reconcile(): Promise<void> {
const generation = this.generation;
- for (const config of this.state?.connections ?? []) {
+ for (const planned of this.state?.connections ?? []) {
if (generation !== this.generation) return;
+ const config = this.state?.connections.find(current => current.connectionId === planned.connectionId);
+ if (!config) continue;
const connection = this.connections.get(config.connectionId) ?? this.create(config);
try {
await this.syncSenders(connection);
diff --git a/src/client/sfu/subscriber.ts b/src/client/sfu/subscriber.ts
index e777ef39..e48af459 100644
--- a/src/client/sfu/subscriber.ts
+++ b/src/client/sfu/subscriber.ts
@@ -71,6 +71,8 @@ export class SfuSubscriber {
this.peer?.updateConfig(config);
}
+ isConnected(): boolean { return this.peer?.pc.connectionState === "connected"; }
+
// Reauthentication cannot establish whether a lost offer/answer was applied.
// Retire that exact subscription through the route owner; do not replay SDP.
resync(): void {
@@ -117,9 +119,14 @@ export class SfuSubscriber {
this.restartPending || !this.peer
)
return false;
- this.restartPending = true;
- this.events.onState?.("reconnecting");
- return this.peer.send({ kind: "subscribe" });
+ const sent = this.peer.send({ kind: "subscribe" });
+ // An unsent manual request cannot retire media that is still connected.
+ // A disconnected transport still needs reconciliation after signaling returns.
+ if (sent || this.peer.pc.connectionState !== "connected") {
+ this.restartPending = true;
+ this.events.onState?.("reconnecting");
+ }
+ return sent;
}
armDecodedFrameProof(requireProgress = false): void {
@@ -224,6 +231,7 @@ export class SfuSubscriber {
this.proofMode === mode,
requireProgress: mode === "progress",
onProof: () => {
+ if (!this.isConnected()) return false;
const accepted = this.events.onFirstDecodedFrame?.() ?? true;
if (accepted) {
this.proofMode = null;
diff --git a/src/client/styles.css b/src/client/styles.css
index 5fa38940..a0c300dd 100644
--- a/src/client/styles.css
+++ b/src/client/styles.css
@@ -91,9 +91,10 @@ ul { margin: 0; padding: 0; list-style: none; }
.lr-top {
display: flex;
+ flex-wrap: wrap;
align-items: center;
justify-content: space-between;
- gap: 12px;
+ gap: 8px 12px;
padding: 14px clamp(16px, 4vw, 44px) 6px;
}
@@ -114,8 +115,9 @@ ul { margin: 0; padding: 0; list-style: none; }
.lr-brand:active { transform: scale(.92); }
.lr-home-arrow { transform: rotate(90deg); color: var(--muted); }
-.lr-top-right { display: inline-flex; gap: 10px; align-items: center; }
-.lr-top > .lr-top-right { margin-left: auto; min-width: 0; flex-wrap: wrap; justify-content: flex-end; }
+.lr-header-context, .lr-header-controls { display: inline-flex; flex: none; gap: 10px; align-items: center; }
+.lr-header-context:empty { display: none; }
+.lr-header-controls { margin-inline-start: auto; }
/* Discrete signaling state, independent of picture quality. */
.lr-leds {
@@ -1780,7 +1782,7 @@ body.lr-theater-open .lr-top { visibility: hidden; }
.lr-meter-cell { justify-content: flex-start; }
.lr-group-name .lr-name-tag { flex: 1; }
.lr-viewer-self-slot .lr-name-tag { flex: 0 1 auto; }
- .lr-row-group > .lr-toggle { flex: 1; }
+ .lr-row-group > .lr-toggle { flex: none; }
.lr-invite-hint, .lr-invite-url { flex: 0 1 auto; max-width: none; min-height: 44px; }
.lr-invite-field { flex: none; width: 100%; }
.lr-door-body { padding: 12px; }
@@ -1795,8 +1797,7 @@ body.lr-theater-open .lr-top { visibility: hidden; }
.lr-source-option { min-height: 58px; padding: 6px 8px; }
.lr-source-option-preview { height: 90px; }
.lr-source-option-icon { width: 34px; height: 34px; }
- .lr-leds-label { display: none; }
- .lr-top-right { gap: 8px; }
+ .lr-header-context, .lr-header-controls { gap: 8px; }
.lr-leds { gap: 4px; padding: 6px; }
.lr-top .lr-btn.is-compact { width: var(--control-compact-size); padding: 0; }
.lr-top .lr-btn.is-compact .lr-cap { display: none; }
@@ -1942,17 +1943,11 @@ details[open] > summary > .lr-sharing-chevron { transform: rotate(180deg); }
/* Theatre keeps ordinary feedback with the surrounding room controls.
Missing pictures and required playback actions retain their stage overlay. */
.lr-room.is-theater .lr-stage-notices { display: none; }
-@media (max-width: 480px) {
- .lr-top:has(.lr-leds) { display: grid; grid-template-columns: auto minmax(0, 1fr); row-gap: 8px; }
- .lr-top:has(.lr-leds) > .lr-top-right { display: contents; }
- .lr-top:has(.lr-leds) > .lr-top-right > [role="status"] { justify-self: end; }
- .lr-top:has(.lr-leds) .lr-header-controls { grid-column: 1 / -1; justify-self: end; }
-}
@media (max-width: 360px) {
.lr-client-modes { grid-template-columns: 1fr; }
.lr-lcd { gap: 4px; padding-inline: 6px; font-size: 18px; }
.lr-top { column-gap: 4px; padding-inline: 10px; }
- .lr-top-right { gap: 4px; }
+ .lr-header-context, .lr-header-controls { gap: 4px; }
.lr-brand { gap: 0; padding-inline: 0; }
.lr-home-arrow { width: 10px; }
.lr-tiles .lr-tile { padding-inline: 4px; }
diff --git a/src/client/webrtc/viewer-peer.ts b/src/client/webrtc/viewer-peer.ts
index 5d68193a..4c434036 100644
--- a/src/client/webrtc/viewer-peer.ts
+++ b/src/client/webrtc/viewer-peer.ts
@@ -369,8 +369,8 @@ export class ViewerPeer implements ViewerMediaPeer {
);
},
owns: () => this.isCurrentConnection(connection, connectionId),
- onProof: () =>
- this.events.onFirstDecodedFrame?.(connectionId) ?? true,
+ onProof: () => connection.connectionState === "connected" &&
+ (this.events.onFirstDecodedFrame?.(connectionId) ?? true),
});
}
this.emit();
diff --git a/tests/android-release-package.test.ts b/tests/android-release-package.test.ts
new file mode 100644
index 00000000..d4e47582
--- /dev/null
+++ b/tests/android-release-package.test.ts
@@ -0,0 +1,56 @@
+import { createHash } from "node:crypto";
+import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { describe, expect, it } from "vitest";
+import { packageAndroidRelease } from "../scripts/package-android-release.mjs";
+import { extractZip } from "../scripts/archive-tool.mjs";
+import { readReleaseArtifacts } from "../scripts/release-artifacts.mjs";
+import { APP_PACKAGE_TARGETS, ANDROID_APP_PACKAGE_TARGET } from "../scripts/app-package-targets.mjs";
+import { SERVER_PACKAGE_TARGETS } from "../scripts/server-package-targets.mjs";
+
+const hash = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex");
+
+describe("Android signed package distributions", () => {
+ it("wraps only the unchanged APK, validates both distributions and retains old descriptors", () => {
+ const root = mkdtempSync(join(tmpdir(), "piik-android-distribution-"));
+ const output = join(root, "output"), extracted = join(root, "extracted");
+ const version = "v2.5.1", revision = "a".repeat(40);
+ try {
+ const apk = join(root, "signed.apk");
+ writeFileSync(apk, "signed APK fixture bytes");
+ const descriptor = packageAndroidRelease(apk, output, version, revision);
+ mkdirSync(extracted);
+ extractZip(join(output, descriptor.mirrorArtifact), extracted);
+ expect(readdirSync(extracted)).toEqual([descriptor.artifact]);
+ expect(hash(readFileSync(join(extracted, descriptor.artifact)))).toBe(hash(readFileSync(apk)));
+ expect(descriptor.artifactSha256).toBe(hash(readFileSync(apk)));
+ for (const target of [...SERVER_PACKAGE_TARGETS, ...APP_PACKAGE_TARGETS]) {
+ const server = SERVER_PACKAGE_TARGETS.some(value => value.id === target.id);
+ const artifact = `${target.id}.zip`;
+ writeFileSync(join(output, artifact), target.id);
+ writeFileSync(join(output, `${artifact}.sha256`), `${hash(target.id)} ${artifact}\n`);
+ writeFileSync(join(output, `${target.id}.manifest.tsv`), "manifest");
+ writeFileSync(join(output, `${target.id}.release.json`), JSON.stringify({ schema: 2, version, revision,
+ target: target.id, artifact, artifactSha256: hash(target.id),
+ ...(server ? { manifest: `${target.id}.manifest.tsv`, manifestSha256: hash("manifest") } : {}) }));
+ }
+ const current = readReleaseArtifacts(output, version, revision);
+ expect(current.targets).toHaveLength(9);
+ expect(current.files).toHaveLength(10);
+ expect(current.mirrorFiles).toHaveLength(9);
+ expect(current.mirrorFiles.some(file => file.name.endsWith(".apk"))).toBe(false);
+ const sidecar = join(output, `${descriptor.mirrorArtifact}.sha256`);
+ writeFileSync(sidecar, "wrong");
+ expect(() => readReleaseArtifacts(output, version, revision)).toThrow(/mirror artifact checksum/);
+ writeFileSync(sidecar, `${descriptor.mirrorArtifactSha256} ${descriptor.mirrorArtifact}\n`);
+ writeFileSync(join(output, descriptor.mirrorArtifact), "corrupt ZIP");
+ expect(() => readReleaseArtifacts(output, version, revision)).toThrow(/mirror artifact checksum/);
+ const { mirrorArtifact, mirrorArtifactSha256, ...legacy } = descriptor;
+ writeFileSync(join(output, ANDROID_APP_PACKAGE_TARGET.artifact.replace(/\.apk$/, ".release.json")), JSON.stringify(legacy));
+ const old = readReleaseArtifacts(output, version, revision);
+ expect(old.files).toHaveLength(9);
+ expect(old.mirrorFiles).toEqual(old.files);
+ } finally { rmSync(root, { recursive: true, force: true }); }
+ }, 30_000);
+});
diff --git a/tests/candidate-quality-probe.test.ts b/tests/candidate-quality-probe.test.ts
index 9aab8957..46d6f823 100644
--- a/tests/candidate-quality-probe.test.ts
+++ b/tests/candidate-quality-probe.test.ts
@@ -33,6 +33,25 @@ function metrics(
}
describe("SFU quality probe", () => {
+ it.each([sfuCandidateDoesNotRegress, p2pCandidateDoesNotRegress])(
+ "compares frame delivery over the sampled window rather than the last second",
+ (compare) => {
+ const current = metrics(3_000);
+ expect(compare(current, metrics(3_100, { framesPerSecond: 30 }))).toBe(true);
+ expect(compare(current, metrics(3_100, { framesPerSecond: null }))).toBe(true);
+ expect(
+ compare(current, metrics(3_100, { intervalFramesDecoded: 60 })),
+ ).toBe(false);
+ expect(
+ compare(current, metrics(3_100, {
+ sampleWindowMs: 3_000,
+ intervalFramesDecoded: 180,
+ framesPerSecond: 30,
+ })),
+ ).toBe(true);
+ },
+ );
+
it("accepts only complete same-Viewer non-regressing windows", () => {
const current = metrics(2_000);
expect(sfuCandidateDoesNotRegress(current, metrics(2_100))).toBe(true);
@@ -73,6 +92,7 @@ describe("SFU quality probe", () => {
frameWidth: 320,
frameHeight: 180,
framesPerSecond: 10,
+ intervalFramesDecoded: 20,
bitrateKbps: 200,
});
expect(
@@ -82,6 +102,7 @@ describe("SFU quality probe", () => {
frameWidth: 1_280,
frameHeight: 720,
framesPerSecond: 30,
+ intervalFramesDecoded: 60,
bitrateKbps: 150,
}),
),
@@ -93,6 +114,7 @@ describe("SFU quality probe", () => {
frameWidth: 1_280,
frameHeight: 720,
framesPerSecond: 30,
+ intervalFramesDecoded: 60,
bitrateKbps: null,
}),
),
@@ -105,6 +127,7 @@ describe("SFU quality probe", () => {
frameWidth: 1_280,
frameHeight: 720,
framesPerSecond: 5,
+ intervalFramesDecoded: 10,
}),
),
).toBe(false);
@@ -134,6 +157,7 @@ describe("SFU quality probe", () => {
frameWidth: 320,
frameHeight: 180,
framesPerSecond: 10,
+ intervalFramesDecoded: 10,
});
const candidate = (timestampMs: number) =>
metrics(timestampMs, {
@@ -141,6 +165,7 @@ describe("SFU quality probe", () => {
frameWidth: 1_280,
frameHeight: 720,
framesPerSecond: 30,
+ intervalFramesDecoded: 30,
});
expect(
@@ -159,12 +184,14 @@ describe("SFU quality probe", () => {
frameWidth: 320,
frameHeight: 180,
framesPerSecond: 10,
+ intervalFramesDecoded: 10,
});
const candidate = metrics(10_900, {
sampleWindowMs: 1_000,
frameWidth: 1_280,
frameHeight: 720,
framesPerSecond: 30,
+ intervalFramesDecoded: 30,
});
expect(p2pCandidateDoesNotRegress(current, candidate)).toBe(true);
@@ -182,7 +209,7 @@ describe("SFU quality probe", () => {
it("rejects after three comparable regressing P2P windows", () => {
const probe = new P2pQualityProbe();
const candidate = (timestampMs: number) =>
- metrics(timestampMs, { framesPerSecond: 30 });
+ metrics(timestampMs, { framesPerSecond: 30, intervalFramesDecoded: 60 });
expect(probe.observe(metrics(1_000), candidate(1_100))).toBe("pending");
expect(probe.observe(metrics(3_000), candidate(3_100))).toBe("pending");
expect(probe.observe(metrics(5_000), candidate(5_100))).toBe("rejected");
@@ -206,7 +233,10 @@ describe("SFU quality probe", () => {
expect(probe.observe(metrics(1_000), metrics(1_100))).toBe("pending");
expect(probe.observe(metrics(3_000), metrics(3_100))).toBe("pending");
expect(
- probe.observe(metrics(5_000), metrics(5_100, { framesPerSecond: 30 })),
+ probe.observe(
+ metrics(5_000),
+ metrics(5_100, { framesPerSecond: 30, intervalFramesDecoded: 60 }),
+ ),
).toBe("pending");
expect(probe.observe(metrics(7_000), metrics(7_100))).toBe("pending");
expect(probe.observe(metrics(9_000), metrics(9_100))).toBe("pending");
@@ -242,6 +272,7 @@ describe("zero-frame current route", () => {
frameWidth: 320,
frameHeight: 180,
framesPerSecond: 10,
+ intervalFramesDecoded: 20,
});
it("approves clean delivery on both candidate route types", () => {
diff --git a/tests/host-quality-ownership.test.ts b/tests/host-quality-ownership.test.ts
index 59ee463c..85992b34 100644
--- a/tests/host-quality-ownership.test.ts
+++ b/tests/host-quality-ownership.test.ts
@@ -329,6 +329,57 @@ describe("Camera replacement ownership", () => {
return { ...current, nativeControl, remoteSnapshot, picture, previous, camera, nativeStream, oldBridge, peer, nativePeer, startedPeerStreams, replacePublication };
}
+ it.each(["initial", "replacement"])("rejects Browser capture that ends during %s audio preparation", async entry => {
+ const current = sourceFixture(entry === "replacement");
+ const pending = deferred<void>();
+ const outputAudio = { readyState: "live", stop: vi.fn() };
+ const resume = vi.fn(() => pending.promise), close = vi.fn(async () => {});
+ const resolveStreamVideoCodec = vi.fn(async () => ({ primary: "h264" }));
+ Object.assign(current.context, { resolveStreamVideoCodec, startBrowserNativeIngress: vi.fn(async () => {}) });
+ if (entry === "initial") {
+ current.context.phase = "idle";
+ current.activeGenerationRef.current = null;
+ current.nativeShareGenerationRef.current = null;
+ current.context.streamRef.current = null;
+ current.context.hostAudioRef.current = null;
+ }
+ vi.stubGlobal("AudioContext", class {
+ resume = resume;
+ close = close;
+ createMediaStreamDestination() { return { stream: { getAudioTracks: () => [outputAudio], getTracks: () => [outputAudio] } }; }
+ createGain() { return { gain: { value: 1 }, connect() {}, disconnect() {} }; }
+ });
+ vi.stubGlobal("MediaStream", class {
+ constructor(private tracks: unknown[]) {}
+ getTracks() { return this.tracks; }
+ getVideoTracks() { return this.tracks.filter(track => track === current.camera.video); }
+ getAudioTracks() { return this.tracks.filter(track => track === outputAudio); }
+ });
+ try {
+ const starting = entry === "initial" ? current.startBrowser() : current.context.switchSource("browser");
+ await vi.waitFor(() => expect(resume).toHaveBeenCalledOnce());
+ expect(current.oldBridge.dispose).not.toHaveBeenCalled();
+ current.camera.video.readyState = "ended";
+ current.camera.video.dispatchEvent(new Event("ended"));
+ pending.resolve();
+ await starting;
+ expect(current.context.streamRef.current).toBe(entry === "initial" ? null : current.previous);
+ expect(current.oldBridge.dispose).not.toHaveBeenCalled();
+ expect(current.client.stopShare).not.toHaveBeenCalled();
+ expect(current.previous.video.stop).not.toHaveBeenCalled();
+ expect(current.camera.video.stop).toHaveBeenCalled();
+ expect(outputAudio.stop).toHaveBeenCalledOnce();
+ expect(close).toHaveBeenCalledOnce();
+ expect(resolveStreamVideoCodec).not.toHaveBeenCalled();
+ expect(current.createRoom).not.toHaveBeenCalled();
+ expect(current.replacePublication).not.toHaveBeenCalled();
+ expect(current.sourceSwitchRef.current).toBeNull();
+ expect(current.setNoticeValue).toHaveBeenLastCalledWith(expect.objectContaining({
+ target: entry === "initial" ? "television" : "operation", tone: "bad", key: "host.capture.readFailed",
+ }));
+ } finally { vi.unstubAllGlobals(); }
+ });
+
it.each(["NotAllowedError", "NotReadableError"])("keeps native capture and its room when Camera fails with %s", async name => {
const current = sourceFixture(true);
const permission = deferred<never>();
@@ -1563,7 +1614,8 @@ describe("Host quality ownership", () => {
current.nativeShareGenerationRef.current = null;
const probe = deferred<{ primary: string }>();
const ingress = deferred<void>();
- const captured = { getTracks: () => [current.track] };
+ Object.assign(current.track, { readyState: "live" });
+ const captured = { getTracks: () => [current.track], getVideoTracks: () => [current.track] };
const resolveStreamVideoCodec = vi.fn(() => probe.promise);
const startBrowserNativeIngress = vi.fn(() => ingress.promise);
const setResolvedVideoCodec = vi.fn();
diff --git a/tests/native-runtime-affinity.test.ts b/tests/native-runtime-affinity.test.ts
index 878a3689..d644a17a 100644
--- a/tests/native-runtime-affinity.test.ts
+++ b/tests/native-runtime-affinity.test.ts
@@ -43,6 +43,14 @@ async function runtime(delay: number) {
it("selects the launched runtime over a faster resident App and never substitutes it after exit", async () => {
const old = await runtime(0);
const selected = await runtime(30);
+ const fetchHTTP = globalThis.fetch;
+ // Keep the real HTTP fixtures independent of Apps already running locally.
+ vi.stubGlobal("fetch", ((input, init) => {
+ const port = Number(new URL(String(input)).port);
+ return port === old.port || port === selected.port
+ ? fetchHTTP(input, init)
+ : Promise.reject(new TypeError("Not this test's runtime"));
+ }) as typeof fetch);
const saved = new Map<string, string>();
const tabSaved = new Map<string, string>();
const page = {
diff --git a/tests/native-viewer-peer.test.ts b/tests/native-viewer-peer.test.ts
index bd922b0a..d25f8304 100644
--- a/tests/native-viewer-peer.test.ts
+++ b/tests/native-viewer-peer.test.ts
@@ -691,6 +691,45 @@ it.each([
}
});
+it("defers local bridge decoded proof while its prepared Native network edge is disconnected", async () => {
+ vi.useFakeTimers();
+ vi.stubGlobal("window", globalThis);
+ vi.stubGlobal("MediaStream", class { getTracks() { return []; } });
+ vi.stubGlobal("RTCPeerConnection", class { close() {} });
+ vi.spyOn(NativeMediaBridge.prototype, "start").mockResolvedValue({} as MediaStream);
+ vi.spyOn(NativeMediaBridge.prototype, "collectMetrics").mockResolvedValue({ ...EMPTY_METRICS });
+ let finishStats!: (frames: number) => void;
+ const stats = new Promise<number>(resolve => { finishStats = resolve; });
+ vi.spyOn(NativeMediaBridge.prototype, "decodedVideoFrames").mockReturnValue(stats);
+ let event!: (event: NativeClientEvent) => void;
+ const client = {
+ receiveOffer: vi.fn(async () => ({ answer: { type: "answer", sdp: "answer" }, audio: false, codec: "vp8" })),
+ onEvent: (listener: typeof event) => { event = listener; return () => {}; }, onClose: () => () => {},
+ closeReceiver: vi.fn(async () => {}), closeEdge: vi.fn(async () => {}),
+ } as unknown as NativeClient;
+ const decoded = vi.fn(() => true), restart = vi.fn(() => true), unavailable = vi.fn();
+ const peer = new NativeCapableViewerPeer({ iceServers: [] }, {
+ sendSignal: () => true, sendRestartRequest: restart, onStream: () => {}, onUpdate: () => {}, onFirstDecodedFrame: decoded,
+ }, { recoveryOwner: "route" }, async () => client, unavailable, "session", 2);
+ const state = (value: "connected" | "disconnected") => event({
+ version: 10, shareId: "session", connectionId: "prepared", type: "edge-state", state: value,
+ });
+ try {
+ await peer.acceptSignal("host", { kind: "description", connectionId: "prepared", description: { type: "offer",
+ sdp: "v=0\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\na=sendonly\r\na=rtpmap:96 VP8/90000\r\n" } });
+ state("connected"); state("disconnected");
+ finishStats(1);
+ await vi.advanceTimersByTimeAsync(100);
+ expect(decoded).not.toHaveBeenCalled();
+ state("connected");
+ await vi.advanceTimersByTimeAsync(100);
+ expect(decoded).toHaveBeenCalledExactlyOnceWith("prepared");
+ expect(client.receiveOffer).toHaveBeenCalledOnce();
+ expect(restart).not.toHaveBeenCalled();
+ expect(unavailable).not.toHaveBeenCalled();
+ } finally { peer.dispose(); }
+});
+
it.each(["first", "replacement", "failed-offer"])("keeps committed route recovery and its budget across backend replacement (%s)", async (scenario) => {
vi.useFakeTimers();
vi.stubGlobal("window", globalThis);
diff --git a/tests/publish-release.test.ts b/tests/publish-release.test.ts
index 1b6cf6dd..ac5a0bc9 100644
--- a/tests/publish-release.test.ts
+++ b/tests/publish-release.test.ts
@@ -19,10 +19,16 @@ for (const target of ["server", "server-linux-arm64", "server-windows-amd64", "w
// The Windows Server package ships a ZIP; the Linux Server packages keep tar.gz.
const archive = target === "android-universal" ? "apk" : target === "server-windows-amd64" ? "zip" : server ? "tar.gz" : "zip";
const artifact = `${target}.${archive}`;
+ const mirrorArtifact = target === "android-universal" ? "piik-app-android-universal.zip" : undefined;
+ if (mirrorArtifact) {
+ writeFileSync(join(directory, mirrorArtifact), "wrapped APK");
+ writeFileSync(join(directory, `${mirrorArtifact}.sha256`), `${hash("wrapped APK")} ${mirrorArtifact}\n`);
+ }
writeFileSync(join(directory, artifact), target);
writeFileSync(join(directory, `${target}.release.json`), JSON.stringify({
schema: 2, version, revision, artifact, artifactSha256: hash(target),
target, ...(server ? { manifest: `${target}.manifest.tsv`, manifestSha256: hash("manifest") } : {}),
+ ...(mirrorArtifact ? { mirrorArtifact, mirrorArtifactSha256: hash("wrapped APK") } : {}),
}));
if (server) writeFileSync(join(directory, `${target}.manifest.tsv`), "manifest");
else writeFileSync(join(directory, `${artifact}.sha256`), `${hash(target)} ${artifact}\n`);
@@ -58,7 +64,8 @@ describe("Gitee mirror publication", () => {
} }));
};
const mutations: string[] = [];
- const attachments = sourceAssets.map(({ name, size }) => ({ name, size, browser_download_url: download + name }));
+ const attachments = sourceAssets.filter(asset => !asset.name.endsWith(".apk"))
+ .map(({ name, size }) => ({ name, size, browser_download_url: download + name }));
const present = options.existing === "pending" ? attachments.slice(0, 1) : options.missing ? [] : attachments;
const source = {
...release(false), html_url: `https://github.com/TNTcraftHIM/Piik/releases/tag/${version}`,
@@ -79,6 +86,7 @@ describe("Gitee mirror publication", () => {
vi.stubGlobal("fetch", vi.fn<typeof fetch>(async (input, init) => {
const url = String(input);
if (url.startsWith(download)) {
+ expect(url).not.toMatch(/\.apk$/);
expect(init?.headers).toBeUndefined();
const file = url.slice(download.length);
if (options.downloadFault && options.downloadFault.remaining > 0) {
@@ -148,13 +156,13 @@ describe("Gitee mirror publication", () => {
expect(result.failure).toBeUndefined();
expect(result.mutations[0]).toBe("POST /releases");
expect(result.mutations.at(-1)).toBe("PATCH /releases/42");
- expect(result.mutations.filter((value) => value.endsWith("/attach_files"))).toHaveLength(sourceAssets.length);
+ expect(result.mutations.filter((value) => value.endsWith("/attach_files"))).toHaveLength(sourceAssets.length - 1);
});
it("resumes pending uploads and leaves complete stable releases unchanged", async () => {
const pending = await mirror({ existing: "pending" });
expect(pending.failure).toBeUndefined();
- expect(pending.mutations.filter((value) => value.endsWith("/attach_files"))).toHaveLength(sourceAssets.length - 1);
+ expect(pending.mutations.filter((value) => value.endsWith("/attach_files"))).toHaveLength(sourceAssets.length - 2);
expect(await mirror({ existing: "published" })).toEqual({ mutations: [], failure: undefined });
});
diff --git a/tests/release-update.test.ts b/tests/release-update.test.ts
index c69fa59c..7638e774 100644
--- a/tests/release-update.test.ts
+++ b/tests/release-update.test.ts
@@ -49,17 +49,31 @@ describe("App release update notice", () => {
.toBe(current.browser_download_url);
});
- it("selects the Android APK from either release provider", () => {
+ it("selects the GitHub Android APK and the mirror ZIP with installation guidance", async () => {
const name = "piik-app-android-universal.apk";
const asset = { name, size: 1024, state: "uploaded",
browser_download_url: `https://github.com/TNTcraftHIM/Piik/releases/download/${release.tag_name}/${name}` };
- expect(parseReleaseMetadata({ ...release, assets: [asset] }, "android-universal")?.url)
- .toBe(asset.browser_download_url);
- const mirrorAsset = { name,
- browser_download_url: `https://gitee.com/TNTcraftHIM/Piik/releases/download/${release.tag_name}/${name}` };
- expect(parseMirrorReleaseMetadata({ tag_name: release.tag_name, prerelease: false,
- body: `<!-- piik-source: ${latestRevision} -->`, assets: [mirrorAsset] }, "android-universal")?.url)
- .toBe(mirrorAsset.browser_download_url);
+ const zipName = name.replace(".apk", ".zip");
+ const zip = { ...asset, name: zipName, browser_download_url: asset.browser_download_url.replace(".apk", ".zip") };
+ const primary = parseReleaseMetadata({ ...release, assets: [zip, asset] }, "android-universal");
+ expect(primary?.url).toBe(asset.browser_download_url);
+ expect(primary?.extractAPK).toBeUndefined();
+ const mirrorAsset = { name: zipName,
+ browser_download_url: `https://gitee.com/TNTcraftHIM/Piik/releases/download/${release.tag_name}/${zipName}` };
+ const mirror = { tag_name: release.tag_name, prerelease: false,
+ body: `<!-- piik-source: ${latestRevision} -->`, assets: [mirrorAsset] };
+ const fetchImpl = vi.fn<typeof fetch>(async input => String(input) === DEFAULT_RELEASE_API_URL
+ ? new Response(null, { status: 503 }) : Response.json([mirror]));
+ await expect(checkReleaseUpdate({ version: "v1.1.0", revision: currentRevision },
+ { fetchImpl, packageTarget: "android-universal" })).resolves.toMatchObject({
+ url: mirrorAsset.browser_download_url, extractAPK: true,
+ });
+ for (const assets of [[], [{ ...mirrorAsset, name }], [mirrorAsset, mirrorAsset],
+ [{ ...mirrorAsset, browser_download_url: mirrorAsset.browser_download_url.replace("gitee.com", "evil.example") }]]) {
+ const legacy = parseMirrorReleaseMetadata({ ...mirror, assets }, "android-universal");
+ expect(legacy?.url).toBe(`https://gitee.com/TNTcraftHIM/Piik/releases/tag/${release.tag_name}`);
+ expect(legacy?.extractAPK).toBeUndefined();
+ }
});
it("keeps the release page when a matching safe ZIP is missing, without trying another provider", async () => {
diff --git a/tests/remote-input.test.ts b/tests/remote-input.test.ts
index 5914f760..7b9fd0db 100644
--- a/tests/remote-input.test.ts
+++ b/tests/remote-input.test.ts
@@ -219,6 +219,78 @@ it("reuses the authenticated identity across media routes and syncs after delaye
route.dispose();
});
+it.each([false, true])("retires the current input epoch after late old-branch packets (closed=%s)", async closed => {
+ const controller = new InputRoute(() => {}), relay = new InputRoute(() => {});
+ controller.enable(() => true);
+ const oldUpstream = new Channel(), nextUpstream = new Channel(), upstream = new Channel(), sharedChild = new Channel();
+ const connection = Object.assign(new EventTarget(), { connectionState: "connected" });
+ relay.setUpstream(upstream as unknown as RTCDataChannel);
+ upstream.receive({ ready: true });
+ relay.attachChild(sharedChild as unknown as RTCDataChannel, connection as unknown as RTCPeerConnection);
+ try {
+ controller.setUpstream(oldUpstream as unknown as RTCDataChannel);
+ oldUpstream.receive({ ready: true });
+ controller.current!.send(JSON.stringify({ type: "request" }));
+ controller.current!.send(JSON.stringify({ type: "key", code: "KeyA", down: true }));
+ await vi.waitFor(() => expect(oldUpstream.sent.filter(packet => packet.data)).toHaveLength(2));
+ const [oldRequest, lateOldInput] = oldUpstream.sent.filter(packet => packet.data);
+ controller.setUpstream(nextUpstream as unknown as RTCDataChannel);
+ nextUpstream.receive({ ready: true });
+ controller.current!.send(JSON.stringify({ type: "request" }));
+ controller.current!.send(JSON.stringify({ type: "key", code: "KeyW", down: true }));
+ await vi.waitFor(() => expect(nextUpstream.sent.filter(packet => packet.data)).toHaveLength(2));
+ const currentPackets = nextUpstream.sent.filter(packet => packet.data);
+ expect(currentPackets[0].id).toBe(oldRequest.id);
+ expect(currentPackets[0].epoch).not.toBe(oldRequest.epoch);
+
+ sharedChild.receive(oldRequest);
+ currentPackets.forEach(packet => sharedChild.receive(packet));
+ // A longer retired route can reach this shared leg after the current route.
+ sharedChild.receive(lateOldInput);
+ if (closed) sharedChild.receive({ id: oldRequest.id, epoch: oldRequest.epoch, closed: true });
+ connection.connectionState = "disconnected";
+ connection.dispatchEvent(new Event("connectionstatechange"));
+ expect(upstream.sent.filter(packet => packet.closed)).toContainEqual({
+ id: currentPackets[0].id, epoch: currentPackets[0].epoch, closed: true,
+ });
+ } finally {
+ controller.dispose(); relay.dispose();
+ }
+});
+
+it("bounds unclosed child epochs and retires them before input-channel overflow closes", () => {
+ const route = new InputRoute(() => {}), upstream = new Channel(), child = new Channel();
+ const connection = Object.assign(new EventTarget(), { connectionState: "connected" });
+ const envelope = (index: number) => ({ id: (index % 20).toString(16).padStart(32, "0"),
+ epoch: index.toString(16).padStart(32, "0"), data: "opaque" });
+ route.setUpstream(upstream as unknown as RTCDataChannel);
+ upstream.receive({ ready: true });
+ route.attachChild(child as unknown as RTCDataChannel, connection as unknown as RTCPeerConnection);
+ // Real close is asynchronous; retirement cannot wait for its close event.
+ vi.spyOn(child, "close").mockImplementation(() => { child.readyState = "closing"; });
+ try {
+ for (let index = 0; index < 128; index++) child.receive(envelope(index));
+ child.receive(envelope(0)); // Existing tuples still forward at capacity.
+ child.receive({ ...envelope(0), id: "f".repeat(32) }); // Keep the existing 20-identity bound.
+ expect(upstream.sent.filter(packet => packet.data)).toHaveLength(129);
+ expect(child.close).not.toHaveBeenCalled();
+ child.receive(envelope(128));
+ expect(child.close).toHaveBeenCalledOnce();
+ expect(upstream.sent.filter(packet => packet.data)).toHaveLength(129);
+ expect(upstream.sent.filter(packet => packet.closed)).toHaveLength(128);
+ for (let index = 0; index < 128; index++) expect(upstream.sent).toContainEqual({
+ id: envelope(index).id, epoch: envelope(index).epoch, closed: true,
+ });
+ const retiredCount = upstream.sent.length;
+ child.receive(envelope(129));
+ child.dispatchEvent(new Event("close"));
+ expect(upstream.sent).toHaveLength(retiredCount);
+ expect(connection.connectionState).toBe("connected");
+ } finally {
+ route.dispose();
+ }
+});
+
it("binds input to the current request and retires cancelled or replaced grants", () => {
const channel = new Channel();
const session = new RemoteInputSession(channel as unknown as RTCDataChannel, () => {});
diff --git a/tests/room-interactions.test.ts b/tests/room-interactions.test.ts
index c6d08a66..8f38224e 100644
--- a/tests/room-interactions.test.ts
+++ b/tests/room-interactions.test.ts
@@ -133,6 +133,30 @@ test("data needs explicit readiness, waits for server echo, and has no offline r
expect(vi.getTimerCount()).toBe(0);
});
+test("a delayed own confirmation restarts cooldown before the next interaction", () => {
+ const { session } = setup();
+ session.receive({ type: "room-interactions-ready", serverTime: Date.now() });
+ const payload = { kind: "reaction", reaction: "poop", targetPeerId: "other_peer_1234" } as const;
+ expect(session.send(payload)).toBe(true);
+ const first = session.getSnapshot().pending!;
+ vi.advanceTimersByTime(120);
+ const acceptedAt = Date.now();
+ vi.advanceTimersByTime(60);
+ session.receive({ ...message("delayed_reaction"), occurredAt: acceptedAt, requestId: first.requestId, payload,
+ sender: { peerId: "self_peer_1234", role: "viewer", displayName: "Me" } });
+ vi.advanceTimersByTime(620); // The send-time 800ms has elapsed, but acceptance was later.
+ expect(session.getSnapshot()).toMatchObject({ pending: null, coolingDown: true });
+ expect(session.send(payload)).toBe(false);
+ vi.advanceTimersByTime(179);
+ expect(session.send(payload)).toBe(false);
+ vi.advanceTimersByTime(1);
+ expect(session.send(payload)).toBe(true);
+ // Even a second request with no outbound delay now respects the Host's interval.
+ expect(Date.now() - acceptedAt).toBeGreaterThanOrEqual(800);
+ session.close();
+ expect(vi.getTimerCount()).toBe(0);
+});
+
test("history and animations are bounded, deduplicated and retired with the room", () => {
const { session } = setup();
session.receive({ type: "room-interactions-ready", serverTime: Date.now() });
@@ -170,8 +194,10 @@ test("rejection and timeout leave delivery unconfirmed without automatic retries
const echoed = { ...message("late_confirmed"), requestId: pending.requestId, payload: pending.payload,
sender: { peerId: "self_peer_1234", role: "viewer" as const, displayName: "Me" } };
session.receive(echoed);
- expect(session.getSnapshot()).toMatchObject({ error: null, pending: null, confirmed: { id: "late_confirmed" } });
- session.send({ kind: "chat", text: "Next attempt" });
+ expect(session.getSnapshot()).toMatchObject({ error: null, pending: null, coolingDown: true, confirmed: { id: "late_confirmed" } });
+ expect(session.send({ kind: "chat", text: "Next attempt" })).toBe(false);
+ vi.advanceTimersByTime(800);
+ expect(session.send({ kind: "chat", text: "Next attempt" })).toBe(true);
const rejected = session.getSnapshot().pending!;
vi.advanceTimersByTime(5000);
session.receive({ type: "room-interaction-rejected", requestId: rejected.requestId, reason: "busy" });
@@ -224,8 +250,9 @@ test("late echoes do not confirm a newer same-text request or erase its rejectio
expect(session.getSnapshot().confirmed).toBeNull();
expect(session.getSnapshot().pending?.requestId).toBe(next.requestId);
session.receive({ type: "room-interaction-rejected", requestId: next.requestId, reason: "busy" });
+ vi.advanceTimersByTime(800);
session.receive({ ...echo, id: "late_event_5678" });
- expect(session.getSnapshot()).toMatchObject({ confirmed: null, error: "busy" });
+ expect(session.getSnapshot()).toMatchObject({ confirmed: null, error: "busy", coolingDown: false });
expect(session.getSnapshot().messages).toHaveLength(2);
session.close();
});
diff --git a/tests/room-voice-browser-endpoint.test.ts b/tests/room-voice-browser-endpoint.test.ts
index 36a923f9..f4c2bd38 100644
--- a/tests/room-voice-browser-endpoint.test.ts
+++ b/tests/room-voice-browser-endpoint.test.ts
@@ -1,12 +1,53 @@
import { afterEach, expect, it, vi } from "vitest";
import { BrowserVoiceEndpoint } from "../src/client/room-voice/browser-endpoint";
import type { ClientMessage, RoomVoiceState } from "../src/shared/protocol";
+import type { MicrophoneInput } from "../src/client/media/microphone-input";
const candidate = (index: number) => ({
candidate: `candidate:${index} 1 UDP 100 192.0.2.1 ${40000 + index} typ host`,
sdpMid: "0", sdpMLineIndex: 0,
});
+it("keeps connected voice on sender parameter rejection and applies a later explicit update", async () => {
+ vi.stubGlobal("MediaStream", class { constructor(public tracks: unknown[]) {} });
+ const messages: ClientMessage[] = [];
+ const endpoint = new BrowserVoiceEndpoint({ iceServers: [] }, message => { messages.push(message); return true; }, () => {});
+ const authority: RoomVoiceState = { type: "room-voice-state", revision: 1, enabled: true, hostPeerId: "self", bitrate: 32_000,
+ members: [{ peerId: "self", connected: true, joined: true, upstream: { kind: "none" } }],
+ publications: [{ id: "mic", peerId: "self" }], unavailable: [],
+ connections: [{ connectionId: "voice", transport: "peer", parentPeerId: "self", childPeerId: "other",
+ revision: 1, pending: false, send: ["mic"], receive: [] }] };
+ const track = { kind: "audio" };
+ let parameters = { encodings: [{ maxBitrate: 32_000 }], codecs: [{ mimeType: "audio/opus", sdpFmtpLine: "maxaveragebitrate=510000" }] };
+ const sender = { track, getParameters: () => parameters, setStreams: vi.fn(),
+ setParameters: vi.fn(async (next: typeof parameters) => { parameters = next; }) };
+ const input = { stream: { getAudioTracks: () => [track] }, close: vi.fn() };
+ // The real endpoint owns sender creation, negotiation, failure reporting and retirement.
+ vi.stubGlobal("RTCPeerConnection", class extends Peer { addTrack = vi.fn(() => sender); });
+ try {
+ endpoint.setAuthority(authority, "self");
+ endpoint.attach("mic", input as unknown as MicrophoneInput);
+ await endpoint.reconcile();
+ const pc = Peer.instances[0]!;
+ expect(endpoint.connected).toBe(true);
+ sender.setParameters.mockRejectedValueOnce(new DOMException("temporarily unavailable", "InvalidStateError"));
+ endpoint.setAuthority({ ...authority, revision: 2, bitrate: 48_000 }, "self");
+ await expect(endpoint.reconcile()).resolves.toBeUndefined();
+ expect(parameters.encodings[0]!.maxBitrate).toBe(32_000);
+ expect(messages.filter(message => message.type === "room-voice-transport" && message.state === "failed")).toEqual([]);
+ expect(pc.close).not.toHaveBeenCalled();
+ await endpoint.reconcile();
+ expect(parameters.encodings[0]!.maxBitrate).toBe(48_000);
+ expect(sender.setParameters).toHaveBeenCalledTimes(2);
+ pc.setRemoteDescription = async () => { throw new Error("invalid SDP"); };
+ await expect(endpoint.receive({ type: "room-voice-signal", connectionId: "voice",
+ description: { revision: 1, sdp: { type: "answer", sdp: "invalid" }, tracks: [] } })).rejects.toThrow("invalid SDP");
+ expect(messages).toContainEqual({ type: "room-voice-transport", connectionId: "voice", state: "failed" });
+ endpoint.setAuthority({ ...authority, revision: 3, connections: [] }, "self");
+ expect(pc.close).toHaveBeenCalledOnce();
+ } finally { endpoint.close(); }
+});
+
class Peer {
static instances: Peer[] = [];
connectionState: RTCPeerConnectionState = "connected";
@@ -14,7 +55,7 @@ class Peer {
localDescription: RTCSessionDescriptionInit | null = null;
remoteDescription: RTCSessionDescriptionInit | null = null;
onicecandidate: ((event: { candidate: RTCIceCandidateInit | null }) => void) | null = null;
- getTransceivers = () => [];
+ getTransceivers = (): unknown[] => [];
addTransceiver = () => {};
createDataChannel = () => {};
createOffer = async () => ({ type: "offer" as const, sdp: "offer" });
@@ -32,6 +73,106 @@ class Peer {
afterEach(() => { vi.unstubAllGlobals(); Peer.instances = []; });
+it.each(["levels", "metrics"].flatMap(observation => [false, true].map(replaced => ({ observation, replaced }))))(
+"keeps in-flight $observation tied to its receiver publication (replaced=$replaced)", async ({ observation, replaced }) => {
+ class AudioElement extends EventTarget {
+ volume = 1; muted = false; paused = true; ended = false;
+ srcObject: unknown = null;
+ play = async () => { this.paused = false; };
+ pause = () => { this.paused = true; };
+ remove = () => {};
+ }
+ vi.stubGlobal("Audio", AudioElement);
+ vi.stubGlobal("document", { body: { append: () => {} } });
+ vi.stubGlobal("MediaStream", class {
+ constructor(private tracks: MediaStreamTrack[]) {}
+ getAudioTracks() { return this.tracks; }
+ });
+ const track = { id: "stable-receiver", kind: "audio", readyState: "live" };
+ let finishStats!: (report: RTCStatsReport) => void;
+ const reading = new Promise<RTCStatsReport>(resolve => { finishStats = resolve; });
+ const getStats = vi.fn(() => reading);
+ vi.stubGlobal("RTCPeerConnection", class extends Peer {
+ getStats = getStats;
+ getTransceivers = () => [{ mid: "0", direction: "recvonly", receiver: { track }, sender: { track: null } }];
+ });
+ const endpoint = new BrowserVoiceEndpoint({ iceServers: [] }, () => true, () => {});
+ const authority = (publicationId: string, peerId: string, revision: number): RoomVoiceState => ({
+ type: "room-voice-state", revision, enabled: true, hostPeerId: "host", unavailable: [],
+ members: [{ peerId: "self", connected: true, joined: true, upstream: { kind: "sfu" } }],
+ publications: [{ id: publicationId, peerId }],
+ connections: [{ connectionId: "stable-connection", childPeerId: "self", transport: "sfu", revision, pending: false, send: [], receive: [publicationId] }],
+ });
+ const manifest = async (publicationId: string, revision: number) => endpoint.receive({
+ type: "room-voice-signal", connectionId: "stable-connection", description: {
+ revision, sdp: { type: "offer", sdp: "offer" }, tracks: [{ mid: "0", publicationId }],
+ },
+ });
+ const report = (ssrc: number, audioLevel: number) => new Map([["audio", {
+ id: "audio", type: "inbound-rtp", kind: "audio", ssrc, timestamp: ssrc * 1000,
+ trackIdentifier: track.id, audioLevel, bytesReceived: ssrc * 100,
+ }]]) as unknown as RTCStatsReport;
+ try {
+ endpoint.setAuthority(authority("old-publication", "old-publisher", 1), "self");
+ await endpoint.reconcile(); await manifest("old-publication", 1);
+ const pending = observation === "levels" ? endpoint.levels() : endpoint.connectionMetrics();
+ expect(getStats).toHaveBeenCalledOnce();
+ const publication = replaced ? "new-publication" : "old-publication";
+ const publisher = replaced ? "new-publisher" : "old-publisher";
+ endpoint.setAuthority(authority(publication, publisher, 2), "self");
+ await manifest(publication, 2);
+ finishStats(report(1, .75));
+ if (replaced) expect([...(await pending)]).toEqual([]);
+ else if (observation === "levels") expect([...(await pending)]).toEqual([[publisher, .75]]);
+ else expect(await pending).toEqual([expect.objectContaining({ peerId: publisher, direction: "receive" })]);
+
+ getStats.mockResolvedValue(report(2, .25));
+ if (observation === "levels") expect([...(await endpoint.levels())]).toEqual([[publisher, .25]]);
+ else expect(await endpoint.connectionMetrics()).toEqual([
+ expect.objectContaining({ peerId: publisher, direction: "receive", connectionId: "stable-connection" }),
+ ]);
+ expect(Peer.instances).toHaveLength(1);
+ expect(Peer.instances[0]!.close).not.toHaveBeenCalled();
+ } finally { endpoint.close(); }
+});
+
+it("drops an in-flight sender report after microphone replacement on the same publication", async () => {
+ vi.stubGlobal("MediaStream", class { constructor(public tracks: unknown[]) {} });
+ const oldTrack = { id: "old-input", kind: "audio", readyState: "live" };
+ const newTrack = { id: "new-input", kind: "audio", readyState: "live" };
+ const sender = { track: oldTrack, setStreams: () => {}, getParameters: () => ({ encodings: [], codecs: [] }),
+ replaceTrack: async (track: typeof oldTrack) => { sender.track = track; } };
+ let finishStats!: (report: RTCStatsReport) => void;
+ const reading = new Promise<RTCStatsReport>(resolve => { finishStats = resolve; });
+ const getStats = vi.fn(() => reading);
+ vi.stubGlobal("RTCPeerConnection", class extends Peer { getStats = getStats; addTrack = () => sender; });
+ const endpoint = new BrowserVoiceEndpoint({ iceServers: [] }, () => true, () => {});
+ endpoint.setAuthority({ type: "room-voice-state", revision: 1, enabled: true, hostPeerId: "self", unavailable: [],
+ members: [{ peerId: "self", connected: true, joined: true, upstream: { kind: "none" } }],
+ publications: [{ id: "mic", peerId: "self" }],
+ connections: [{ connectionId: "stable", transport: "peer", parentPeerId: "self", childPeerId: "other", revision: 1, pending: false, send: ["mic"], receive: [] }],
+ }, "self");
+ const input = (track: typeof oldTrack) => ({ stream: { getAudioTracks: () => [track] },
+ close: () => { track.readyState = "ended"; } }) as unknown as MicrophoneInput;
+ const report = (track: typeof oldTrack) => new Map([
+ ["sender", { id: "sender", type: "outbound-rtp", kind: "audio", mediaSourceId: "source", timestamp: 1000, ssrc: 1 }],
+ ["source", { id: "source", type: "media-source", kind: "audio", trackIdentifier: track.id }],
+ ]) as unknown as RTCStatsReport;
+ try {
+ endpoint.attach("mic", input(oldTrack)); await endpoint.reconcile();
+ const pending = endpoint.connectionMetrics();
+ endpoint.attach("mic", input(newTrack)); await endpoint.reconcile();
+ finishStats(report(oldTrack));
+ expect(await pending).toEqual([]);
+ getStats.mockResolvedValue(report(newTrack));
+ expect(await endpoint.connectionMetrics()).toEqual([
+ expect.objectContaining({ peerId: "self", direction: "send", metrics: expect.objectContaining({ trackIdentifier: newTrack.id }) }),
+ ]);
+ expect(Peer.instances).toHaveLength(1);
+ expect(Peer.instances[0]!.close).not.toHaveBeenCalled();
+ } finally { endpoint.close(); }
+});
+
it.each(["local", "remote"])("retains and flushes the first 64 %s voice candidates without failing the transport", async phase => {
vi.stubGlobal("RTCPeerConnection", Peer);
const messages: ClientMessage[] = [];
@@ -75,3 +216,45 @@ it.each(["local", "remote"])("retains and flushes the first 64 %s voice candidat
expect(messages).toHaveLength(count);
} finally { endpoint.close(); }
});
+
+it("does not recreate a retired voice connection from an in-flight reconciliation", async () => {
+ let started!: () => void;
+ const preparing = new Promise<void>(resolve => { started = resolve; });
+ type Offer = { type: "offer"; sdp: string };
+ let finish!: (offer: Offer) => void;
+ const offer = new Promise<Offer>(resolve => { finish = resolve; });
+ vi.stubGlobal("RTCPeerConnection", class extends Peer {
+ createOffer = async () => {
+ if (this === Peer.instances[0]) { started(); return offer; }
+ return { type: "offer" as const, sdp: "offer" };
+ };
+ });
+ const messages: ClientMessage[] = [];
+ const endpoint = new BrowserVoiceEndpoint({ iceServers: [] }, message => { messages.push(message); return true; }, () => {});
+ const connection = (connectionId: string) => ({ connectionId, transport: "peer" as const,
+ parentPeerId: "self", childPeerId: connectionId, revision: 1, pending: false, send: [], receive: [] });
+ const retained = connection("retained"), retired = connection("retired"), replacement = connection("replacement");
+ const state: RoomVoiceState = { type: "room-voice-state", revision: 1, enabled: true, hostPeerId: "self",
+ members: [{ peerId: "self", connected: true, joined: true, upstream: { kind: "none" } }],
+ publications: [], unavailable: [], connections: [retained, retired] };
+ try {
+ endpoint.setAuthority(state, "self");
+ const pending = endpoint.reconcile();
+ await preparing;
+ const pc = Peer.instances[0]!;
+ // RoomVoiceSession applies this authority synchronously, even while SDP
+ // is awaiting the Browser. Its normal queue schedules the next reconcile.
+ endpoint.setAuthority({ ...state, revision: 2, connections: [retained, replacement] }, "self");
+ finish({ type: "offer", sdp: "offer" });
+ await pending;
+ expect(Peer.instances).toHaveLength(1);
+ expect(messages.some(message => "connectionId" in message && message.connectionId === retired.connectionId)).toBe(false);
+ await endpoint.reconcile();
+ expect(Peer.instances).toHaveLength(2);
+ expect(endpoint.connected).toBe(true);
+ expect(pc.close).not.toHaveBeenCalled();
+ expect(messages).toContainEqual(expect.objectContaining({ type: "room-voice-signal", connectionId: replacement.connectionId }));
+ endpoint.close();
+ expect(Peer.instances.every(peer => peer.close.mock.calls.length === 1)).toBe(true);
+ } finally { endpoint.close(); }
+});
diff --git a/tests/route-transition.test.ts b/tests/route-transition.test.ts
index 48042487..15581c24 100644
--- a/tests/route-transition.test.ts
+++ b/tests/route-transition.test.ts
@@ -111,6 +111,7 @@ const candidate = (
function createFakeSubscriber(events: SubscriberEvents, log: string[], label: string) {
return {
+ isConnected: vi.fn(() => true),
resync: vi.fn(),
updateConfig: vi.fn(),
acceptSignal: vi.fn(async () => undefined),
@@ -174,6 +175,55 @@ function receiveMetrics(
}
describe("minimal route transition contracts", () => {
+ it.each(["connected", "failed", "superseded"] as const)("retains the old media while a committed Peer waits, then handles %s", async outcome => {
+ const messages: ClientMessage[] = [];
+ let status: "disconnected" | "connected" | "failed" = "disconnected";
+ const activatePeer = vi.fn((_assignment: ParticipantRouteAssignment, revision?: number): boolean | "waiting" =>
+ revision === undefined || status === "connected" ? true : status === "disconnected" ? "waiting" : false);
+ let subscriber!: ReturnType<typeof createFakeSubscriber>;
+ const onSfuStream = vi.fn();
+ const route = new ViewerSfuRoute("viewer_12345678", {
+ activatePeer, onSfuStream,
+ send: message => { messages.push(message); return true; },
+ createSubscriber: events => (subscriber = createFakeSubscriber(events, [], "retained")),
+ });
+ await prepareSfu(route, 1);
+ subscriber.events.onStream({} as MediaStream);
+ subscriber.events.onFirstDecodedFrame();
+ await vi.waitFor(() => expect(onSfuStream).toHaveBeenCalledOnce());
+ const assignment = peerAssignment("candidate_parent");
+ route.accept({ revision: 2, phase: "prepare", assignment, candidate: candidate(2) });
+ route.accept({ revision: 2, phase: "active", assignment });
+ await vi.waitFor(() => expect(activatePeer).toHaveBeenCalledWith(assignment, 2));
+ expect(messages.filter(message => message.type === "route-failed")).toEqual([]);
+ expect(subscriber.disconnect).not.toHaveBeenCalled();
+ expect(route.retryPeerActivation("retired_connection")).toBe(false);
+
+ if (outcome === "superseded") {
+ const replacement = peerAssignment("replacement_parent");
+ route.accept({ revision: 3, phase: "prepare", assignment: replacement, candidate: candidate(3) });
+ status = "connected";
+ expect(route.retryPeerActivation(candidate(2).connectionId)).toBe(false);
+ expect(route.retryPeerActivation(candidate(3).connectionId)).toBe(false);
+ expect(subscriber.disconnect).not.toHaveBeenCalled();
+ route.accept({ revision: 3, phase: "active", assignment: replacement });
+ await vi.waitFor(() => expect(subscriber.disconnect).toHaveBeenCalledOnce());
+ expect(activatePeer).toHaveBeenCalledWith(replacement, 3);
+ } else {
+ status = outcome;
+ expect(route.retryPeerActivation(candidate(2).connectionId)).toBe(true);
+ if (outcome === "connected") await vi.waitFor(() => expect(subscriber.disconnect).toHaveBeenCalledOnce());
+ else {
+ await vi.waitFor(() => expect(messages).toContainEqual({
+ type: "route-failed", revision: 2, phase: "active", connectionId: candidate(2).connectionId,
+ }));
+ expect(subscriber.disconnect).not.toHaveBeenCalled();
+ }
+ }
+ expect(messages.filter(message => message.type === "route-failed")).toHaveLength(outcome === "failed" ? 1 : 0);
+ await route.disconnect();
+ });
+
it("reports active failure when a prepared Peer disappears before commit delivery", async () => {
const messages: ClientMessage[] = [];
const onSfuStream = vi.fn();
@@ -1403,6 +1453,17 @@ describe("minimal route transition contracts", () => {
expect(messages.filter((message) => message.type === "route-ready")).toEqual([
{ type: "route-ready", revision: 2, phase: "prepare" },
]);
+ subscriber.isConnected.mockReturnValue(false);
+ route.accept({ revision: 2, phase: "active", assignment: viewerSfuAssignment() });
+ for (const timestampMs of [17_100, 19_100, 21_100]) {
+ currentMetrics = receiveMetrics(timestampMs - 100);
+ subscriber.events.onStats(receiveMetrics(timestampMs, { framesPerSecond: 1 }));
+ }
+ expect(messages.filter(message => message.type === "route-failed")).toEqual([]);
+ expect(subscriber.disconnect).not.toHaveBeenCalled();
+ subscriber.isConnected.mockReturnValue(true);
+ subscriber.events.onState("connected");
+ await route.disconnect();
});
it("rejects a quality SFU candidate after three comparable regressions", async () => {
@@ -1451,7 +1512,10 @@ describe("minimal route transition contracts", () => {
for (const timestampMs of [1_100, 3_100, 5_100]) {
currentMetrics = receiveMetrics(timestampMs - 100);
subscriber.events.onStats(
- receiveMetrics(timestampMs, { framesPerSecond: 30 }),
+ receiveMetrics(timestampMs, {
+ framesPerSecond: 30,
+ intervalFramesDecoded: 60,
+ }),
);
}
diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts
index 7105b5f5..c69b482a 100644
--- a/tests/sfu-client.test.ts
+++ b/tests/sfu-client.test.ts
@@ -2,9 +2,11 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { QUALITY_PROFILES } from "../src/client/media/quality.ts";
import { SfuPublisher } from "../src/client/sfu/publisher.ts";
import { SfuSubscriber } from "../src/client/sfu/subscriber.ts";
+import { ViewerSfuRoute } from "../src/client/media/viewer-sfu-route.ts";
import * as senderTracks from "../src/client/media/sender-video-track";
import {
clientMessageSchema,
+ type ClientMessage,
type SfuSignalMessage,
} from "../src/shared/protocol.ts";
@@ -650,6 +652,74 @@ describe("embedded SFU browser transport", () => {
expect(onStream).toHaveBeenLastCalledWith(null);
});
+ it("waits for the exact SFU transport when decoded proof or commit arrives during disconnection", async () => {
+ const messages: ClientMessage[] = [];
+ const onSfuStream = vi.fn(), activatePeer = vi.fn(() => true);
+ const route = new ViewerSfuRoute("viewer_12345678", {
+ activatePeer, onSfuStream,
+ send: message => { messages.push(message); return true; },
+ });
+ cleanups.push(() => route.disconnect());
+ route.accept({ revision: 3, phase: "active", assignment: {
+ upstream: { kind: "peer", peerId: "healthy_parent" }, childPeerIds: [], sfuPublicationGeneration: null,
+ } });
+ await vi.advanceTimersByTimeAsync(0);
+ const assignment = { upstream: { kind: "sfu" as const }, childPeerIds: [], sfuPublicationGeneration: config.publicationGeneration };
+ route.accept({ revision: config.revision, phase: "prepare", assignment, candidate: {
+ childPeerId: "viewer_12345678", connectionId: config.connectionId, transport: "sfu", qualityProbe: false,
+ } });
+ await route.acceptConfig({ type: "sfu-config", ...config });
+ const pc = FakePc.instances[0]!;
+ await route.acceptSignal(signal({ description: { type: "offer", sdp: "offer" } }));
+ let completeStats!: (report: Map<string, object>) => void;
+ const report = new Promise<Map<string, object>>(resolve => { completeStats = resolve; });
+ pc.ontrack?.({ track: new FakeTrack("video"), receiver: { getStats: () => report } });
+ pc.state("connected");
+ pc.state("disconnected");
+ completeStats(new Map([["video", { id: "video", type: "inbound-rtp", kind: "video", framesDecoded: 1 }]]));
+ await vi.advanceTimersByTimeAsync(0);
+ expect(messages.filter(message => message.type === "route-ready")).toEqual([]);
+ expect(onSfuStream).not.toHaveBeenCalled();
+
+ pc.state("connected");
+ await vi.advanceTimersByTimeAsync(100);
+ expect(messages.filter(message => message.type === "route-ready")).toEqual([
+ { type: "route-ready", revision: config.revision, phase: "prepare" },
+ ]);
+ pc.state("disconnected");
+ route.accept({ revision: config.revision, phase: "active", assignment });
+ await vi.advanceTimersByTimeAsync(0);
+ expect(onSfuStream).not.toHaveBeenCalled();
+ pc.state("connected");
+ await vi.advanceTimersByTimeAsync(0);
+ expect(onSfuStream).toHaveBeenCalledOnce();
+ expect(FakePc.instances).toHaveLength(1);
+ expect(pc.close).not.toHaveBeenCalled();
+ });
+
+ it("keeps a healthy subscription when a manual restart cannot be sent", async () => {
+ const send = vi.fn(() => true);
+ const onState = vi.fn(), onDisconnected = vi.fn();
+ const subscriber = new SfuSubscriber({ send, onStream: vi.fn(), onState, onDisconnected });
+ cleanups.push(() => subscriber.disconnect());
+ await subscriber.connect(config);
+ subscriber.activate();
+ const pc = FakePc.instances[0]!;
+ pc.state("connected");
+ onState.mockClear();
+ send.mockReturnValue(false);
+
+ expect(subscriber.reconnect()).toBe(false);
+ subscriber.resync();
+ expect(pc.close).not.toHaveBeenCalled();
+ expect(onDisconnected).not.toHaveBeenCalled();
+ expect(onState).not.toHaveBeenCalled();
+
+ send.mockReturnValue(true);
+ expect(subscriber.reconnect()).toBe(true);
+ expect(onState).toHaveBeenCalledExactlyOnceWith("reconnecting");
+ });
+
it("reports Browser video rejection through the existing SFU failure owner", async () => {
const send = vi.fn(() => true), onDisconnected = vi.fn();
const subscriber = new SfuSubscriber({ send, onStream: vi.fn(), onDisconnected });
diff --git a/tests/upstream-sync.test.ts b/tests/upstream-sync.test.ts
new file mode 100644
index 00000000..9b0d12c3
--- /dev/null
+++ b/tests/upstream-sync.test.ts
@@ -0,0 +1,85 @@
+import { execFileSync, spawnSync } from "node:child_process";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join, resolve } from "node:path";
+import { expect, it } from "vitest";
+
+it("replays a real patch stack, drops upstream equivalents and preserves conflicts and existing work", () => {
+ const root = mkdtempSync(join(tmpdir(), "piik-upstream-"));
+ const upstream = join(root, "upstream"), fork = join(root, "fork");
+ const git = (cwd: string, ...args: string[]) => execFileSync("git", args, {
+ cwd, encoding: "utf8", windowsHide: true, stdio: ["ignore", "pipe", "pipe"],
+ }).trim();
+ const identity = (cwd: string) => {
+ git(cwd, "config", "user.name", "Piik fixture");
+ git(cwd, "config", "user.email", "[email protected]");
+ git(cwd, "config", "commit.gpgsign", "false");
+ };
+ const commit = (cwd: string, file: string, contents: string, title: string) => {
+ writeFileSync(join(cwd, file), contents);
+ git(cwd, "add", file);
+ git(cwd, "commit", "-m", title);
+ return git(cwd, "rev-parse", "HEAD");
+ };
+ try {
+ mkdirSync(upstream);
+ git(upstream, "init", "-b", "main");
+ identity(upstream);
+ const base = commit(upstream, "api.txt", "original\n", "original API");
+ git(root, "clone", "--no-hardlinks", upstream, fork);
+ identity(fork);
+ const accepted = commit(fork, "profile.txt", "equivalent profiles\n", "fix profile matching");
+ const head = commit(fork, "api.txt", "patched\n", "add required input");
+ commit(upstream, "notes.txt", "upstream change\n", "advance upstream");
+ git(upstream, "fetch", fork, accepted);
+ git(upstream, "cherry-pick", accepted);
+ const updated = git(upstream, "rev-parse", "HEAD");
+ expect(updated).not.toBe(accepted);
+ writeFileSync(join(fork, "api.txt"), "uncommitted user work\n");
+ const run = (out: string, onto = "main") => spawnSync(process.execPath, [
+ resolve("scripts/sync-upstream.mjs"), "--source", fork, "--upstream", upstream,
+ "--base", base, "--head", head, "--onto", onto, "--out", out,
+ ], {
+ encoding: "utf8", windowsHide: true,
+ env: { ...process.env, GIT_COMMITTER_NAME: "Piik fixture", GIT_COMMITTER_EMAIL: "[email protected]" },
+ });
+ const candidate = join(root, "candidate");
+ const ready = run(candidate);
+ expect(ready.status, ready.stdout + ready.stderr).toBe(0);
+ const report = JSON.parse(readFileSync(join(candidate, ".git/piik-upstream-sync.json"), "utf8"));
+ expect(report.status).toBe("ready");
+ expect(report.remaining).toBe(1);
+ expect(report.upstreamHead).toBe(updated);
+ expect(report.alreadyApplied).toContain(accepted);
+ expect(git(candidate, "show", "HEAD:api.txt")).toBe("patched");
+ expect(git(candidate, "show", "HEAD:notes.txt")).toBe("upstream change");
+ expect(git(candidate, "status", "--porcelain")).toBe("");
+ expect(git(fork, "rev-parse", "HEAD")).toBe(head);
+ expect(readFileSync(join(fork, "api.txt"), "utf8")).toBe("uncommitted user work\n");
+ expect(git(upstream, "rev-parse", "HEAD")).toBe(updated);
+
+ git(upstream, "checkout", "-b", "conflicting");
+ commit(upstream, "api.txt", "upstream API\n", "change the same API");
+ const conflicted = join(root, "conflicted");
+ expect(run(conflicted, "conflicting").status).toBe(1);
+ const conflict = JSON.parse(readFileSync(join(conflicted, ".git/piik-upstream-sync.json"), "utf8"));
+ expect(conflict.status).toBe("conflict");
+ expect(conflict.conflicts).toBe("api.txt");
+ expect(git(conflicted, "diff", "--name-only", "--diff-filter=U")).toBe("api.txt");
+ expect(run(candidate).status).toBe(1);
+ expect(git(candidate, "rev-parse", "HEAD")).toBe(report.candidate);
+ expect(readFileSync(join(fork, "api.txt"), "utf8")).toBe("uncommitted user work\n");
+
+ git(upstream, "checkout", "main");
+ git(upstream, "fetch", fork, head);
+ git(upstream, "cherry-pick", head);
+ const absorbed = join(root, "absorbed");
+ const complete = run(absorbed);
+ expect(complete.status, complete.stdout + complete.stderr).toBe(0);
+ const final = JSON.parse(readFileSync(join(absorbed, ".git/piik-upstream-sync.json"), "utf8"));
+ expect(final.remaining).toBe(0);
+ expect(final.candidate).toBe(final.upstreamHead);
+ } finally {
+ rmSync(root, { recursive: true, force: true });
+ }
+}, 30_000);
diff --git a/tests/viewer-audio.test.ts b/tests/viewer-audio.test.ts
index b88d0311..e8285039 100644
--- a/tests/viewer-audio.test.ts
+++ b/tests/viewer-audio.test.ts
@@ -63,6 +63,63 @@ describe("Viewer local audio", () => {
});
afterEach(() => vi.unstubAllGlobals());
+ it("retains gain on its mute echo when the system ignores volume writes", () => {
+ const video = new PlaybackVideo();
+ Object.defineProperty(video, "volume", { get: () => 1, set: () => {} });
+ const audio = new ViewerAudio(video as unknown as HTMLVideoElement, vi.fn());
+ audio.bind(new AudioStream([audioTrack()]) as unknown as MediaStream);
+ audio.setLevel(0.5);
+ const context = PlaybackContext.instances[0];
+ context.start();
+ expect(video.volume).toBe(1);
+ expect(video.muted).toBe(true);
+ video.dispatchEvent(new Event("volumechange"));
+ expect(context.close).not.toHaveBeenCalled();
+ expect(context.gain.gain.value).toBe(0.5);
+ expect(audio.snapshot).toEqual({ level: 0.5, muted: false, boostAvailable: true });
+
+ video.muted = false;
+ video.dispatchEvent(new Event("volumechange"));
+ expect(context.sources[0].disconnect).toHaveBeenCalledOnce();
+ expect(context.close).toHaveBeenCalledOnce();
+ expect(audio.snapshot).toEqual({ level: 1, muted: false, boostAvailable: true });
+ expect(video.muted).toBe(false);
+ audio.dispose();
+ });
+
+ it("hands boosted playback to native volume controls without leaving two audio outputs", () => {
+ const video = new PlaybackVideo();
+ const audio = new ViewerAudio(video as unknown as HTMLVideoElement, vi.fn());
+ audio.bind(new AudioStream([audioTrack()]) as unknown as MediaStream);
+ audio.setLevel(1.5);
+ const context = PlaybackContext.instances[0];
+ context.start();
+ video.dispatchEvent(new Event("volumechange")); // The owner's queued native mute write.
+ expect(context.close).not.toHaveBeenCalled();
+ expect(context.gain.gain.value).toBe(1.5);
+
+ // Chromium's PiP mute control writes the media element's muted property.
+ video.muted = false;
+ video.dispatchEvent(new Event("volumechange"));
+ expect(context.sources[0].disconnect).toHaveBeenCalledOnce();
+ expect(context.gain.disconnect).toHaveBeenCalledOnce();
+ expect(context.close).toHaveBeenCalledOnce();
+ expect(video.muted).toBe(false);
+ expect(audio.snapshot).toEqual({ level: 1, muted: false, boostAvailable: true });
+ video.muted = true;
+ video.dispatchEvent(new Event("volumechange"));
+ expect(audio.snapshot.muted).toBe(true);
+ video.muted = false;
+ video.dispatchEvent(new Event("volumechange"));
+ expect(audio.snapshot.muted).toBe(false);
+
+ audio.setLevel(1.5);
+ PlaybackContext.instances[1].start();
+ expect(video.muted).toBe(true);
+ expect(PlaybackContext.instances[1].gain.gain.value).toBe(1.5);
+ audio.dispose();
+ });
+
it("lazily adjusts current audio while pause, mute and track changes remain local", () => {
const video = new PlaybackVideo();
const srcObject = video.srcObject;
diff --git a/tests/viewer-peer-generation.test.ts b/tests/viewer-peer-generation.test.ts
index d140988a..42d624fc 100644
--- a/tests/viewer-peer-generation.test.ts
+++ b/tests/viewer-peer-generation.test.ts
@@ -423,7 +423,9 @@ describe("ViewerPeer connection generations", () => {
},
);
- await peer.acceptSignal("host", offer("fresh-exact"));
+ const accepting = peer.acceptSignal("host", offer("fresh-exact"));
+ FakePeerConnection.instances[0]!.connectionState = "connected";
+ await accepting;
await flushAsyncWork();
expect(decoded).toHaveBeenCalledOnce();
@@ -431,6 +433,35 @@ describe("ViewerPeer connection generations", () => {
peer.dispose();
});
+ it("defers an in-flight decoded proof until the same prepared transport reconnects", async () => {
+ vi.useFakeTimers();
+ const stats = createDeferred<RTCStatsReport>();
+ FakePeerConnection.plans.push({ statsGate: stats.promise });
+ const decoded = vi.fn(() => true), restart = vi.fn(() => true);
+ const peer = new ViewerPeer({ iceServers: [] }, {
+ sendSignal: () => true, sendRestartRequest: restart,
+ onStream: () => undefined, onUpdate: () => undefined, onFirstDecodedFrame: decoded,
+ }, { recoveryOwner: "route" });
+ try {
+ await peer.acceptSignal("host", offer("prepared"));
+ const connection = FakePeerConnection.instances[0]!;
+ connection.connectionState = "connected";
+ connection.dispatchEvent(new Event("connectionstatechange"));
+ connection.connectionState = "disconnected";
+ connection.dispatchEvent(new Event("connectionstatechange"));
+ stats.resolve(decodedReport(1));
+ await vi.advanceTimersByTimeAsync(100);
+ expect(decoded).not.toHaveBeenCalled();
+ connection.connectionState = "connected";
+ connection.dispatchEvent(new Event("connectionstatechange"));
+ await vi.advanceTimersByTimeAsync(100);
+ expect(decoded).toHaveBeenCalledExactlyOnceWith("prepared");
+ expect(FakePeerConnection.instances).toHaveLength(1);
+ expect(restart).not.toHaveBeenCalled();
+ expect(timeoutCallbacks.size).toBe(0);
+ } finally { peer.dispose(); vi.useRealTimers(); }
+ });
+
it("does not publish an in-flight decoded proof from a replaced connection", async () => {
const oldStats = createDeferred<RTCStatsReport>();
FakePeerConnection.plans.push(