Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

feat: prefer proven H264 browser senders

AAA6b3c3de8a03b
diff --git a/docs/adr/0007-path-isolated-representation-quality.md b/docs/adr/0007-path-isolated-representation-quality.md
index aac3c2dd..aa4a2092 100644
--- a/docs/adr/0007-path-isolated-representation-quality.md
+++ b/docs/adr/0007-path-isolated-representation-quality.md
@@ -23,7 +23,8 @@ it does not need a second bitrate, resolution, FPS, or layer-control system.
## Decision
-1. Browser video makes one internal, sender-scoped codec decision. A bounded
+1. Browser video makes one internal, share-generation-scoped codec decision per
+ sending endpoint. A bounded
actual-`RTCPeerConnection` preflight measures H.264 encoded progress against
a deterministic moving probe track at the current share's effective target
dimensions and frame rate. The real capture supplies those target settings,
@@ -43,7 +44,7 @@ it does not need a second bitrate, resolution, FPS, or layer-control system.
After accepting an answer, the Host reapplies the current video profile to
the negotiated sender because the browser may replace or rewrite encoding
parameters during negotiation.
-5. Direct and Browser-relay offers with a proved source order H.264 before VP8
+5. Direct and Browser-relay offers with a proved share source order H.264 before VP8
in one standard codec-preference list; both endpoints select their first
common codec without a parallel media connection. A failed source gate
offers VP8 only. Existing edges are not renegotiated when a later relay gate
diff --git a/docs/project-memory.md b/docs/project-memory.md
index a01ec539..e56fdd26 100644
--- a/docs/project-memory.md
+++ b/docs/project-memory.md
@@ -87,7 +87,7 @@ Last updated: 2026-08-26
- WebRTC owns direct/peer media adaptation. The Host reapplies the selected video
profile after answer negotiation; no periodic application controller exists.
- The production SFU publisher uses the Host's H.264/VP8 decision and selected
- ceiling but no custom simulcast layers. Pinned LiveKit defaults own
+ ceiling but no backup codec or custom simulcast layers. Pinned LiveKit defaults own
representations, Dynacast owns aggregate
demand, and server send-side BWE owns subscriber forwarding. AdaptiveStream
stays disabled because any Viewer may relay its received track.
@@ -118,8 +118,10 @@ Last updated: 2026-08-26
## Current Source And Production
-- Production and the matching canonical application/runtime tree use strict
- `screener-v12`; exact identity is indexed by [status](./status.md).
+- Production and canonical source use strict `screener-v12`, but the pending
+ application candidate adds the internal H.264/VP8 decision while production
+ remains on its prior fixed-VP8 artifact. Exact identities are indexed by
+ [status](./status.md).
- Production uses random memory rooms, 20-Viewer admission, endpoint cap `2`,
fixed 9,000-publication ingress and `9000 * 20` subscription egress admission,
STUN UDP 3478, LiveKit media UDP 7882, and Web TCP 80/443. Node 8787 and
diff --git "a/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md" "b/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
index e7f7396c..c2e35428 100644
--- "a/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
+++ "b/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
@@ -2,7 +2,7 @@
## 2026-08-18 14:45
-状态:production 与 canonical application/runtime tree 使用严格 `screener-v12`。Browser 视频固定 VP8 并使用 `contentHint = "motion"`,音频使用 `music`;codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。Viewer 页面恢复、原生控件和当前线路重连已进入 source 与 production,精确执行状态由 `docs/status.md` 持有。
+状态:production 与 canonical source 使用严格 `screener-v12`。Canonical source 已实现每分享代次的 H.264 实际 sender gate 与 VP8 fallback,production 仍是待切换的固定 VP8 artifact;两者都使用 video `contentHint = "motion"` 和 audio `music`,且 codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。精确执行状态由 `docs/status.md` 持有。
## 目标与边界
@@ -162,7 +162,7 @@ accepted 的唯一 wire 是 `screener-v12`,不做版本协商、旧 parser、
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级,实际结果由 stats 观测;拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。
-Accepted `screener-v12` Web 视频使用页面内 sender-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 收到新 upstream stream 后在后台运行同一 gate,完成前建立的 child 使用 VP8,完成后只影响未来 child。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host source 当时的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。
+Accepted `screener-v12` Web 视频使用页面内 share-generation-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 在该分享代首次收到 upstream video 且没有 active/prepared child 时运行同一 gate;若 child prepare 先到,则等待已经在运行的 bounded gate 结束后只创建一份真实 child sender。质量设置、换源和换父不重复 gate。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host share generation 的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。
Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`autoGainControl` 和 `voiceIsolation`,并以 `channelCount: { ideal: 2 }` 偏好双声道;这是当前 Chrome/Edge Host 基线的实现契约,用于避开网页 `getDisplayMedia()` 默认的语音处理和单声道 APM 候选,不宣称跨浏览器保证,也不固定采样率。所有值都是非强制 post-selection preference;不使用 `exact`、`min` 或 `advanced`。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`:窗口选项请求该窗口/应用音频,整个屏幕选项允许用户勾选系统音频;标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些约束或 picker hints,且没有对应的标准来源类型读回,因此 UI 不额外推断或标注来源。系统音频由用户显式选择,可能包含语音或通知;video-only 始终可用。
@@ -172,7 +172,7 @@ Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`
直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`,不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流;成功后为每个现有 sender 调用 `replaceTrack()`,健康连接不重新协商。初始 offer 预留 send-only audio transceiver,使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败,只重建该 peer;新流提交后才停止旧流,并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置,避免把不同 track 的计数器混算。
-直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前 source 的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。
+直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前分享代次的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。
Host 本地预览直接绑定现有 capture stream,不创建 viewer、`RTCPeerConnection` 或额外媒体边,也不产生 SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本;在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。后台性能诊断必须把 Host 页面前台/遮挡/最小化、被捕获 source 前台/遮挡/最小化以及 preview 播放/暂停分成独立变量,并关联 document visibility/focus、track mute/readyState 与 capture/outbound/inbound stats。页面 timer、伪活跃和 Screen Wake Lock 都不构成屏幕捕获保活方案;若标准浏览器无法维持目标行为,应记录平台能力边界而不是制造第二套 lifecycle。只有稳定收益且捕获、编码和远端画面不变时,才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开;隐藏预览绝不停止 capture track 或 sender encode。
@@ -363,7 +363,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio
## 当前交付边界
-- Production 与 canonical application/runtime tree 使用严格单一 `screener-v12` wire。当前实现包含短 room-lived grant、明确 missing-room 结果、产品诊断下载移除、完整短 URL 邀请 UI、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性,固定 Browser VP8、video `contentHint = "motion"`、audio `music`,且不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list或分享生命周期 codec 分支。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。
+- Production 与 canonical source 使用严格单一 `screener-v12` wire。Canonical source 的每分享代次 actual-sender gate 在证据通过时让未来 direct/relay 和 Host 单一 SFU publication 使用 H.264,否则使用 VP8;production 仍是固定 VP8 artifact。两者都使用 video `contentHint = "motion"`、audio `music`,不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list、backup publication或健康 edge codec 切换。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。
- Pause/Resume 由当前 Host session 和 `shareGeneration` 约束。Host 后台性能和 VP8 hardware acceleration 分别按受控根因证据诊断;标准 ICE/SFU 按真实异构网络验收。
- 当前发布面只包含 Web Host、Viewer 和 browser relay。Native sender、capture helper、shared-encode executable 与两个分发包仍由各自后续决策和物理证据驱动,不属于当前 Browser release。
diff --git "a/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md" "b/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
index 1096f58e..0bbb194b 100644
--- "a/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
+++ "b/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
@@ -27,7 +27,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
- 端点发送容量:所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity,由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置,默认 `2` 且只接受 `1`、`2` 或 `3`;不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份,上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量,角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`;普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child,超额 Viewer 保持入房等待并在槽位释放后提升,信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment;并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`,active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare,绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级;缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。
- P2P 优先:每条普通逻辑媒体边先尝试 direct/peer UDP,服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布;每名 Viewer 同时至多一个 active upstream,失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate;Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction,并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时,同一 controller 用有限 carrier cursor 建立唯一 publication;carrier 必须能在既有 overlap 上限内完成,不能先切断健康分支。终态归原始等待需求,publication 成功后该需求先取得 SFU 画面,剩余 direct candidate 再在后台收敛;完整事务由 ADR-0005 定义。
- NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`,保留 LiveKit signaling 交付的授权 SFU UDP candidates,并由标准 ICE 选择 nominated non-relay pair;不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC,也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。
-- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser 对每个新分享 source 运行一次有界实际 sender 预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。
+- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser Host 在每个新分享代次运行一次有界实际 sender 预检,Relay 在该代首次取得 upstream source 且尚无 child 时运行同一预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不因质量设置、换源或换父重复探测或切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。
- 路径隔离表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 不设置 `screenShareSimulcastLayers`,由锁定的 LiveKit client 默认行为拥有当前 H.264 或 VP8 source decision 对应的 screen-share representations、source replacement 与 republish;LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 负责需求聚合与每个 subscriber 的实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children,因此 subscriber 保持 `adaptiveStream: false`,不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。
- 路由失效输入:应用不推断 parent-wide 质量,也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`,或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时,使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`,route/connection变化或新帧重置、暂停抑制,同一 edge只报告一次,不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时,现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline;它不得停表、伪造连接指标或立即判定 edge 失效,只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 视为已尝试;foreground direct window结束后用SFU取得可用画面,剩余direct candidate只在画面后方收敛,新的外部事实才允许本轮失败tuple重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时,它作为child换父并保留subtree;disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。
- 断线恢复:临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile;任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting,healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束,未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall,并在resume后重新reconcile。
@@ -161,7 +161,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
- Given 同一时刻采集、host outbound 与 viewer inbound 证据,When 进行根因分类,Then 关联派生后的实际协商视频 codec/profile/parameters、适用时的 `scalabilityMode` 与真实解码表现,依次区分 Capture 低、Capture 高但 outbound 因 CPU 低、Capture 高但 outbound 因 bandwidth 低、outbound 正常但 inbound 低、以及 inbound 正常但肉眼模糊五类;不用 UA 或设备型号代替运行时证据,也不上传 raw SDP。
- Given Viewer 上报当前路径质量证据,When 报告未鉴权、过频、重复、过期或没有匹配的 sender/viewer 统计,Then 报告被丢弃;有效报告也只能用于诊断,不能改变 Host 表示或路径状态。
- Given 受控 RTT/丢包测试条件,When 连续分享高动态游戏画面 30 分钟,Then 延迟、帧率与丢帧达到上述目标,且没有不可恢复的冻结。
-- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 新 source 的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host source 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。
+- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 当前分享代次的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host share generation 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。
- Given 分享过程中切换 Wi-Fi/有线网络,When ICE 连接失败,Then 客户端触发 ICE restart,并在成功或超时后给出明确状态。
- Given 当前路径仍可播放且控制器准备迁移 logical parent 或替换 transport,When 候选路径拥有模型授权的 overlap 资源,Then 旧路径保持权威,直到 exact candidate child 解码出第一帧新画面后才原子提交;ICE `connected` 单独不能触发切换。
- Given 候选没有准入资源,或 provisional 路径失败、超时、代次过期、参与者变化、房间停止、权限撤销或 session 被替换,When 本次机会结束,Then 原路径和未受影响 branches 继续工作,临时资源被销毁,重试遵守模型定义的有界策略。
@@ -171,7 +171,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
- Given Web Host 或浏览器 relay 建立或替换分享音轨,When 对实际音频 sender 应用发送参数,Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 192000`、调用 `setParameters()` 并立即读回,档位缺失时按 128000 处理;Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer,Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=192000`,并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用;空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续,不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增,其他 fmtp/FEC/session/media 字段逐项保留。
- Given 通过 SFU 发布分享音轨,When 构建 LiveKit 2.22.0 publish options,Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、192 kbps 使用同等结构的显式 sender ceiling,三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`;验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率,不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。
- Given 分享正在直播或暂停且 Host 选择新的 64/128/192 kbps 音频档位,When 当前 profile mutation 串行执行,Then 服务端以 last-wins 保存并广播最新 desired profile,每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback,过期 completion 不得覆盖新值;以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试,不得谎称全房已收敛。当前 applied readback 保持 endpoint-local;没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation,也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge;其他浏览器只有实际 sender readback 成功时才算支持。
-- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。source-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。
+- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。share-generation-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。
- Given 浏览器返回屏幕音轨,When 产品展示音频状态或未来连接详情,Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实;字段缺失必须显示为未知,且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。
- Given Web 或后续 native sender 分享电影/视频音频,When 进行产品验收,Then 必须用独立 A/B/C 证据关联 capture audio track/settings、实际 negotiated codec/fmtp、outbound/inbound 实际码率、loss、jitter、concealment、jitter-buffer 与 A/V sync;P2P、浏览器 relay 与 SFU route switch 保持同一持久 Viewer `MediaStream`/媒体元素和 stereo 协商,不新增独立 audio element 或 Web Audio 时钟边界。native sender 必须通过该音质和同步矩阵,且不得把 audio `maxBitrate` ceiling 当作提质证明。
- Given 分享者同时运行语音软件,When 选择仅分享游戏声音,Then supported Windows 11 native sender 只保留目标游戏及其子进程的渲染音频,观众、通知和其他应用声音不得进入分享;浏览器 system audio 无法满足时必须允许 video-only 或明确告知其可能包含语音/通知,不能用 AEC、降噪或来源标签伪称已隔离。
diff --git a/src/client/components/StatsGrid.tsx b/src/client/components/StatsGrid.tsx
index 74bf6eb1..a9013a36 100644
--- a/src/client/components/StatsGrid.tsx
+++ b/src/client/components/StatsGrid.tsx
@@ -48,8 +48,13 @@ function codecName(codec: string): string {
export function codecContractWarnings(metrics: ConnectionMetrics): string[] {
const warnings: string[] = [];
- if (metrics.codec && metrics.codec.toLowerCase() !== "video/vp8") {
- warnings.push(`视频编码为 ${codecName(metrics.codec)},预期 VP8`);
+ if (
+ metrics.codec &&
+ !["video/h264", "video/vp8"].includes(metrics.codec.toLowerCase())
+ ) {
+ warnings.push(
+ `视频编码为 ${codecName(metrics.codec)},预期 H264 或 VP8`,
+ );
}
if (metrics.audioCodec && metrics.audioCodec.toLowerCase() !== "audio/opus") {
warnings.push(`音频编码为 ${codecName(metrics.audioCodec)},预期 Opus`);
diff --git a/src/client/media/host-provisional-child.ts b/src/client/media/host-provisional-child.ts
index 7f6246b2..81c8d991 100644
--- a/src/client/media/host-provisional-child.ts
+++ b/src/client/media/host-provisional-child.ts
@@ -6,6 +6,7 @@ import type {
} from "../../shared/protocol";
import { countEndpointMediaCopies } from "../../shared/media-copy-accounting";
import { HostPeer } from "../webrtc/host-peer";
+import type { BrowserVideoCodec } from "../webrtc/video-codec";
import type { PeerSnapshot } from "../types";
import type { QualityProfile } from "./quality";
@@ -21,6 +22,7 @@ interface HostProvisionalPrepareInput extends HostProvisionalInput {
iceConfig: IceConfig;
stream: MediaStream;
profile: QualityProfile;
+ videoCodec: BrowserVideoCodec;
}
interface HostProvisionalChildEvents {
@@ -86,7 +88,10 @@ export class HostProvisionalChild {
private startPrepared(
revision: number,
candidate: PreparedRouteCandidate,
- input: Pick<HostProvisionalPrepareInput, "iceConfig" | "stream" | "profile">,
+ input: Pick<
+ HostProvisionalPrepareInput,
+ "iceConfig" | "stream" | "profile" | "videoCodec"
+ >,
): void {
let peer: HostPeer;
peer = new HostPeer(
@@ -113,6 +118,7 @@ export class HostProvisionalChild {
}
},
},
+ input.videoCodec,
candidate.connectionId,
);
this.signalingPeer = peer;
diff --git a/src/client/media/host-sfu-route.ts b/src/client/media/host-sfu-route.ts
index 5f38d628..d6583803 100644
--- a/src/client/media/host-sfu-route.ts
+++ b/src/client/media/host-sfu-route.ts
@@ -10,6 +10,7 @@ import {
type SfuPublisherFailureStage,
} from "../sfu/publisher";
import type { ConnectionMetrics } from "../types";
+import type { BrowserVideoCodec } from "../webrtc/video-codec";
import type {
AudioSenderParameterReadback,
QualityProfile,
@@ -25,7 +26,11 @@ import {
interface HostPublisherTransport {
connect(config: SfuConnectionConfig): Promise<boolean>;
- activate(stream: MediaStream, profile: QualityProfile): Promise<boolean>;
+ activate(
+ stream: MediaStream,
+ profile: QualityProfile,
+ videoCodec?: BrowserVideoCodec,
+ ): Promise<boolean>;
deactivate(): Promise<boolean>;
replaceStream(stream: MediaStream): Promise<boolean>;
updateProfile(profile: QualityProfile): Promise<boolean>;
@@ -54,6 +59,7 @@ export interface HostSfuPublisherSnapshot {
interface HostSfuRouteEvents {
getStream: () => MediaStream | null;
getProfile: () => QualityProfile;
+ getVideoCodec: () => BrowserVideoCodec;
reconcileChildren: (childPeerIds: string[]) => void;
send: (message: ClientMessage) => boolean;
onPublisherUpdate?: (snapshot: HostSfuPublisherSnapshot | null) => void;
@@ -481,12 +487,13 @@ export class HostSfuRoute {
private async preparePublisher(slot: HostPublisherSlot): Promise<boolean> {
const stream = this.events.getStream();
const profile = this.events.getProfile();
+ const videoCodec = this.events.getVideoCodec();
if (!stream) {
this.handleFailure(slot);
return false;
}
try {
- if (!(await slot.publisher.activate(stream, profile))) {
+ if (!(await slot.publisher.activate(stream, profile, videoCodec))) {
this.handleFailure(slot);
await disconnectPublisher(slot.publisher);
return false;
diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx
index 54c1103e..ec85d18d 100644
--- a/src/client/pages/HostPage.tsx
+++ b/src/client/pages/HostPage.tsx
@@ -120,6 +120,8 @@ import {
MAX_ENDPOINT_MEDIA_CHILDREN,
reconcileBoundedMediaChildren,
} from "../webrtc/media-assignment";
+import type { BrowserVideoCodec } from "../webrtc/video-codec";
+import { preferredVideoCodecForTrack } from "../webrtc/video-codec-preflight";
import {
hostActionErrorNotice,
hostServerErrorNotice,
@@ -321,6 +323,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
const pendingQualityChangeRef = useRef<QualitySettings | null>(null);
const qualitySettingsRef = useRef<QualitySettings>(DEFAULT_QUALITY_SETTINGS);
const advancedQualityRef = useRef<QualitySettings>(advancedQuality);
+ const videoCodecRef = useRef<BrowserVideoCodec>("vp8");
+ const codecProbeAbortRef = useRef<AbortController | null>(null);
const sharingPausedRef = useRef(false);
const retiringStreamRef = useRef<MediaStream | null>(null);
const hostSfuRouteRef = useRef<HostSfuRoute | null>(null);
@@ -398,6 +402,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
sourceSwitchRef.current = null;
qualityChangeRef.current = null;
pendingQualityChangeRef.current = null;
+ codecProbeAbortRef.current?.abort();
+ codecProbeAbortRef.current = null;
stopPreferredRoomRenewal();
signalRef.current?.stop();
peersRef.current.forEach((peer) => peer.dispose());
@@ -432,6 +438,28 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
);
}
+ async function probeStreamVideoCodec(
+ stream: MediaStream,
+ ): Promise<BrowserVideoCodec> {
+ codecProbeAbortRef.current?.abort();
+ const controller = new AbortController();
+ codecProbeAbortRef.current = controller;
+ const track = stream.getVideoTracks()[0];
+ try {
+ return track
+ ? await preferredVideoCodecForTrack(
+ track,
+ qualitySettingsRef.current,
+ controller.signal,
+ )
+ : "vp8";
+ } finally {
+ if (codecProbeAbortRef.current === controller) {
+ codecProbeAbortRef.current = null;
+ }
+ }
+ }
+
function stopPreferredRoomRenewal(): void {
if (preferredRoomRenewalTimerRef.current !== null) {
window.clearInterval(preferredRoomRenewalTimerRef.current);
@@ -469,6 +497,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
route = new HostSfuRoute({
getStream: () => streamRef.current,
getProfile: () => qualitySettingsRef.current,
+ getVideoCodec: () => videoCodecRef.current,
reconcileChildren: (childPeerIds) => {
if (
isCurrentGeneration(generation) &&
@@ -522,6 +551,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
sourceSwitchRef.current = null;
qualityChangeRef.current = null;
pendingQualityChangeRef.current = null;
+ codecProbeAbortRef.current?.abort();
+ codecProbeAbortRef.current = null;
+ videoCodecRef.current = "vp8";
const signal = signalRef.current;
if (signal) {
if (notifyServer) {
@@ -1002,6 +1034,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
iceConfig,
stream,
profile: qualitySettingsRef.current,
+ videoCodec: videoCodecRef.current,
});
}
@@ -1074,6 +1107,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
}
},
},
+ videoCodecRef.current,
);
peersRef.current.set(peerId, peer);
let started: boolean;
@@ -1448,6 +1482,12 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
setDetails(captureDetails(captured));
watchCaptureEnd(captured, generation);
+ videoCodecRef.current = await probeStreamVideoCodec(captured);
+ if (!isCurrentGeneration(generation)) {
+ captured.getTracks().forEach((track) => track.stop());
+ return;
+ }
+
let createdRoom: HostRoomState | null = null;
let claimedRoom = false;
try {
diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts
index 1a42dda9..51bd2dcd 100644
--- a/src/client/sfu/publisher.ts
+++ b/src/client/sfu/publisher.ts
@@ -23,6 +23,7 @@ import {
type VideoSenderParameterReadback,
} from "../media/quality";
import type { ConnectionMetrics } from "../types";
+import type { BrowserVideoCodec } from "../webrtc/video-codec";
import {
captureMetrics,
collectConnectionMetricsFromReport,
@@ -169,7 +170,11 @@ export class SfuPublisher {
}
}
- activate(stream: MediaStream, profile: QualityProfile): Promise<boolean> {
+ activate(
+ stream: MediaStream,
+ profile: QualityProfile,
+ videoCodec: BrowserVideoCodec = "vp8",
+ ): Promise<boolean> {
this.desiredProfile = profile;
this.startupVideoProfilePending = needsStartupVideoProfile(profile);
const effectiveVideoProfile = startupVideoProfile(profile);
@@ -195,7 +200,7 @@ export class SfuPublisher {
room,
videoTrack,
sdk.Track.Source.ScreenShare,
- videoPublishOptions(effectiveVideoProfile),
+ videoPublishOptions(effectiveVideoProfile, videoCodec),
);
if (!this.owns(room, generation)) {
return false;
@@ -1165,7 +1170,7 @@ async function configurePublishedVideo(
const retainedPublishOptions = {
...published.publication.options,
...videoTrack.publishOptions,
- ...videoPublishOptions(profile),
+ ...videoPublishOptions(profile, publishedVideoCodec(published)),
};
published.publication.options = retainedPublishOptions;
videoTrack.publishOptions = retainedPublishOptions;
@@ -1248,10 +1253,19 @@ function mergeQualityWarnings(...warnings: Array<string | null>): string | null
return present.length > 0 ? present.join(";") : null;
}
-function videoPublishOptions(profile: QualityProfile): TrackPublishOptions {
+function publishedVideoCodec(published: PublishedTrack): BrowserVideoCodec {
+ return published.publication.options?.videoCodec === "h264"
+ ? "h264"
+ : "vp8";
+}
+
+function videoPublishOptions(
+ profile: QualityProfile,
+ videoCodec: BrowserVideoCodec,
+): TrackPublishOptions {
return {
backupCodec: false,
- videoCodec: "vp8",
+ videoCodec,
screenShareEncoding: {
maxBitrate: profile.maxBitrate,
maxFramerate: profile.maxFramerate,
diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts
index be2bad5b..3d2e45fa 100644
--- a/src/client/webrtc/host-peer.ts
+++ b/src/client/webrtc/host-peer.ts
@@ -24,7 +24,10 @@ import {
collectConnectionMetrics,
createStatsAccumulator,
} from "./stats";
-import { applyVp8Codec } from "./vp8-codec";
+import {
+ applyVideoCodecPreference,
+ type BrowserVideoCodec,
+} from "./video-codec";
const MAX_PENDING_CANDIDATES = 64;
type PeerIceConfig = Pick<RTCConfiguration, "iceServers">;
@@ -73,6 +76,7 @@ export class HostPeer {
private stream: MediaStream,
private desiredProfile: QualityProfile,
private readonly events: HostPeerEvents,
+ private readonly videoCodec: BrowserVideoCodec = "vp8",
connectionId = createOpaqueId(),
) {
this.connectionId = connectionId;
@@ -105,8 +109,8 @@ export class HostPeer {
direction: "sendonly",
streams: [this.stream],
});
- if (!applyVp8Codec(videoTransceiver)) {
- this.setError(null, "当前浏览器无法使用 VP8 视频编码");
+ if (!applyVideoCodecPreference(videoTransceiver, this.videoCodec)) {
+ this.setError(null, "当前浏览器无法使用支持的视频编码");
return false;
}
this.videoSender = videoTransceiver.sender;
diff --git a/src/client/webrtc/video-codec-preflight.ts b/src/client/webrtc/video-codec-preflight.ts
new file mode 100644
index 00000000..3ef5989a
--- /dev/null
+++ b/src/client/webrtc/video-codec-preflight.ts
@@ -0,0 +1,276 @@
+import {
+ configureVideoSender,
+ startupVideoProfile,
+ type QualityProfile,
+} from "../media/quality";
+import {
+ applyH264ProbeCodec,
+ type BrowserVideoCodec,
+} from "./video-codec";
+
+const PREFLIGHT_DEADLINE_MS = 4_000;
+const PREFLIGHT_POLL_MS = 100;
+const PREFLIGHT_MIN_SOURCE_FRAMES = 15;
+const PREFLIGHT_MAX_FRAME_DEFICIT = 2;
+const PREFLIGHT_FPS_WINDOW_MS = 1_000;
+
+export interface H264ProbeSample {
+ outboundId: string;
+ sourceId: string;
+ timestamp: number;
+ codec: string;
+ framesEncoded: number | null;
+ sourceFrames: number | null;
+ encodedFramesPerSecond: number | null;
+ sourceFramesPerSecond: number | null;
+ qualityLimitationReason: string | null;
+}
+
+type StatsRecord = RTCStats & Record<string, unknown>;
+
+function finiteNumber(value: unknown): number | null {
+ return typeof value === "number" && Number.isFinite(value) ? value : null;
+}
+
+function stringValue(value: unknown): string | null {
+ return typeof value === "string" ? value : null;
+}
+
+function recordById(
+ report: RTCStatsReport,
+ id: unknown,
+): StatsRecord | null {
+ if (typeof id !== "string") {
+ return null;
+ }
+ return (report.get(id) as StatsRecord | undefined) ?? null;
+}
+
+function readH264ProbeSample(report: RTCStatsReport): H264ProbeSample | null {
+ const candidates: StatsRecord[] = [];
+ report.forEach((raw) => {
+ const record = raw as StatsRecord;
+ if (
+ record.type === "outbound-rtp" &&
+ record.kind === "video" &&
+ record.isRemote !== true &&
+ finiteNumber(record.framesEncoded) !== null
+ ) {
+ candidates.push(record);
+ }
+ });
+ if (candidates.length !== 1) {
+ return null;
+ }
+ const outbound = candidates[0]!;
+ const codec = recordById(report, outbound.codecId);
+ const source = recordById(report, outbound.mediaSourceId);
+ const mimeType = stringValue(codec?.mimeType);
+ if (
+ codec?.type !== "codec" ||
+ source?.type !== "media-source" ||
+ mimeType?.toLowerCase() !== "video/h264"
+ ) {
+ return null;
+ }
+ return {
+ outboundId: outbound.id,
+ sourceId: source.id,
+ timestamp: finiteNumber(outbound.timestamp) ?? 0,
+ codec: mimeType,
+ framesEncoded: finiteNumber(outbound.framesEncoded),
+ sourceFrames: finiteNumber(source.frames),
+ encodedFramesPerSecond: finiteNumber(outbound.framesPerSecond),
+ sourceFramesPerSecond: finiteNumber(source.framesPerSecond),
+ qualityLimitationReason: stringValue(outbound.qualityLimitationReason),
+ };
+}
+
+export function h264ProbeSustainsSource(
+ baseline: H264ProbeSample,
+ current: H264ProbeSample,
+): boolean | null {
+ if (
+ baseline.outboundId !== current.outboundId ||
+ baseline.sourceId !== current.sourceId ||
+ current.timestamp <= baseline.timestamp ||
+ current.codec.toLowerCase() !== "video/h264" ||
+ current.qualityLimitationReason === "cpu"
+ ) {
+ return false;
+ }
+ if (
+ baseline.framesEncoded !== null &&
+ current.framesEncoded !== null &&
+ baseline.sourceFrames !== null &&
+ current.sourceFrames !== null &&
+ current.framesEncoded >= baseline.framesEncoded &&
+ current.sourceFrames >= baseline.sourceFrames
+ ) {
+ const encoded = current.framesEncoded - baseline.framesEncoded;
+ const source = current.sourceFrames - baseline.sourceFrames;
+ if (source < PREFLIGHT_MIN_SOURCE_FRAMES) {
+ return null;
+ }
+ return encoded + PREFLIGHT_MAX_FRAME_DEFICIT >= source;
+ }
+ if (
+ current.timestamp - baseline.timestamp < PREFLIGHT_FPS_WINDOW_MS ||
+ current.encodedFramesPerSecond === null ||
+ current.sourceFramesPerSecond === null ||
+ current.sourceFramesPerSecond <= 0
+ ) {
+ return null;
+ }
+ return (
+ current.encodedFramesPerSecond + PREFLIGHT_MAX_FRAME_DEFICIT >=
+ current.sourceFramesPerSecond
+ );
+}
+
+function hasProbeWindow(
+ baseline: H264ProbeSample,
+ current: H264ProbeSample,
+): boolean {
+ if (
+ baseline.outboundId !== current.outboundId ||
+ baseline.sourceId !== current.sourceId ||
+ current.timestamp <= baseline.timestamp
+ ) {
+ return false;
+ }
+ if (
+ baseline.sourceFrames !== null &&
+ current.sourceFrames !== null &&
+ current.sourceFrames >= baseline.sourceFrames
+ ) {
+ return (
+ current.sourceFrames - baseline.sourceFrames >=
+ PREFLIGHT_MIN_SOURCE_FRAMES
+ );
+ }
+ return current.timestamp - baseline.timestamp >= PREFLIGHT_FPS_WINDOW_MS;
+}
+
+function delay(milliseconds: number, signal?: AbortSignal): Promise<void> {
+ return new Promise((resolve, reject) => {
+ if (signal?.aborted) {
+ reject(signal.reason);
+ return;
+ }
+ const onAbort = () => {
+ window.clearTimeout(timer);
+ reject(signal?.reason);
+ };
+ const timer = window.setTimeout(() => {
+ signal?.removeEventListener("abort", onAbort);
+ resolve();
+ }, milliseconds);
+ signal?.addEventListener("abort", onAbort, { once: true });
+ });
+}
+
+async function waitForIceGathering(
+ connection: RTCPeerConnection,
+ deadline: number,
+ signal?: AbortSignal,
+): Promise<void> {
+ while (connection.iceGatheringState !== "complete") {
+ if (Date.now() >= deadline) {
+ throw new Error("H.264 preflight timed out");
+ }
+ await delay(PREFLIGHT_POLL_MS, signal);
+ }
+}
+
+async function runH264Probe(
+ track: MediaStreamTrack,
+ profile: QualityProfile,
+ signal?: AbortSignal,
+): Promise<boolean> {
+ const senderConnection = new RTCPeerConnection({ iceServers: [] });
+ const receiverConnection = new RTCPeerConnection({ iceServers: [] });
+ const abort = () => {
+ senderConnection.close();
+ receiverConnection.close();
+ };
+ signal?.addEventListener("abort", abort, { once: true });
+ try {
+ const transceiver = senderConnection.addTransceiver(track, {
+ direction: "sendonly",
+ });
+ if (!applyH264ProbeCodec(transceiver)) {
+ return false;
+ }
+ await configureVideoSender(
+ transceiver.sender,
+ startupVideoProfile(profile),
+ );
+ const deadline = Date.now() + PREFLIGHT_DEADLINE_MS;
+ await senderConnection.setLocalDescription(
+ await senderConnection.createOffer(),
+ );
+ await waitForIceGathering(senderConnection, deadline, signal);
+ await receiverConnection.setRemoteDescription(
+ senderConnection.localDescription!,
+ );
+ await receiverConnection.setLocalDescription(
+ await receiverConnection.createAnswer(),
+ );
+ await waitForIceGathering(receiverConnection, deadline, signal);
+ await senderConnection.setRemoteDescription(
+ receiverConnection.localDescription!,
+ );
+
+ let warmupBaseline: H264ProbeSample | null = null;
+ let measurementBaseline: H264ProbeSample | null = null;
+ while (Date.now() < deadline) {
+ if (signal?.aborted || track.readyState === "ended") {
+ return false;
+ }
+ const sample = readH264ProbeSample(
+ await transceiver.sender.getStats(),
+ );
+ if (sample) {
+ warmupBaseline ??= sample;
+ if (
+ !measurementBaseline &&
+ sample !== warmupBaseline &&
+ hasProbeWindow(warmupBaseline, sample)
+ ) {
+ measurementBaseline = sample;
+ } else if (measurementBaseline && sample !== measurementBaseline) {
+ const result = h264ProbeSustainsSource(measurementBaseline, sample);
+ if (result !== null) {
+ return result;
+ }
+ }
+ }
+ await delay(PREFLIGHT_POLL_MS, signal);
+ }
+ return false;
+ } finally {
+ signal?.removeEventListener("abort", abort);
+ senderConnection.close();
+ receiverConnection.close();
+ }
+}
+
+export async function preferredVideoCodecForTrack(
+ track: MediaStreamTrack,
+ profile: QualityProfile,
+ signal?: AbortSignal,
+): Promise<BrowserVideoCodec> {
+ let codec: BrowserVideoCodec = "vp8";
+ if (track.kind !== "video" || track.readyState === "ended") {
+ console.debug("[Screener] video codec preflight", codec);
+ return codec;
+ }
+ try {
+ codec = (await runH264Probe(track, profile, signal)) ? "h264" : "vp8";
+ } catch {}
+ if (!signal?.aborted) {
+ console.debug("[Screener] video codec preflight", codec);
+ }
+ return codec;
+}
diff --git a/src/client/webrtc/video-codec.ts b/src/client/webrtc/video-codec.ts
new file mode 100644
index 00000000..c370b3ff
--- /dev/null
+++ b/src/client/webrtc/video-codec.ts
@@ -0,0 +1,83 @@
+export type BrowserVideoCodec = "h264" | "vp8";
+
+const REPAIR_CODEC_MIME_TYPES = new Set([
+ "video/rtx",
+ "video/red",
+ "video/ulpfec",
+ "video/flexfec-03",
+]);
+
+function videoCapabilities(): RTCRtpCodec[] {
+ return RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
+}
+
+function vp8Codecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
+ return codecs.filter(
+ ({ mimeType }) => mimeType.toLowerCase() === "video/vp8",
+ );
+}
+
+function h264ModeOneCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
+ return codecs.filter(({ mimeType, sdpFmtpLine }) => {
+ if (mimeType.toLowerCase() !== "video/h264") {
+ return false;
+ }
+ return (
+ !sdpFmtpLine ||
+ /(?:^|;)\s*packetization-mode=1(?:;|$)/i.test(sdpFmtpLine)
+ );
+ });
+}
+
+function repairCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
+ return codecs.filter(({ mimeType }) =>
+ REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()),
+ );
+}
+
+function applyCodecs(
+ transceiver: RTCRtpTransceiver,
+ codecs: RTCRtpCodec[],
+): boolean {
+ if (typeof transceiver.setCodecPreferences !== "function") {
+ return false;
+ }
+ try {
+ transceiver.setCodecPreferences(codecs);
+ return true;
+ } catch {
+ return false;
+ }
+}
+
+export function applyVideoCodecPreference(
+ transceiver: RTCRtpTransceiver,
+ preference: BrowserVideoCodec,
+): boolean {
+ if (typeof RTCRtpSender === "undefined") {
+ return false;
+ }
+ const codecs = videoCapabilities();
+ const vp8 = vp8Codecs(codecs);
+ if (vp8.length === 0) {
+ return false;
+ }
+ const h264 = preference === "h264" ? h264ModeOneCodecs(codecs) : [];
+ return applyCodecs(transceiver, [
+ ...h264,
+ ...vp8,
+ ...repairCodecs(codecs),
+ ]);
+}
+
+export function applyH264ProbeCodec(
+ transceiver: RTCRtpTransceiver,
+): boolean {
+ if (typeof RTCRtpSender === "undefined") {
+ return false;
+ }
+ const codecs = videoCapabilities();
+ const h264 = h264ModeOneCodecs(codecs);
+ return h264.length > 0 &&
+ applyCodecs(transceiver, [...h264, ...repairCodecs(codecs)]);
+}
diff --git a/src/client/webrtc/viewer-relay.ts b/src/client/webrtc/viewer-relay.ts
index 5b89c4c4..38ddd5d6 100644
--- a/src/client/webrtc/viewer-relay.ts
+++ b/src/client/webrtc/viewer-relay.ts
@@ -7,6 +7,8 @@ import type { QualityProfile } from "../media/quality";
import type { PeerSnapshot } from "../types";
import { HostPeer } from "./host-peer";
import { MAX_ENDPOINT_MEDIA_CHILDREN } from "./media-assignment";
+import type { BrowserVideoCodec } from "./video-codec";
+import { preferredVideoCodecForTrack } from "./video-codec-preflight";
interface ViewerRelayEvents {
sendSignal: (peerId: string, payload: SignalPayload) => boolean;
@@ -31,6 +33,11 @@ export class ViewerRelay {
private preparedCandidate: PreparedRouteCandidate | null = null;
private plannedChildPeerIds: string[] = [];
private syncQueue = Promise.resolve();
+ private videoCodec: BrowserVideoCodec = "vp8";
+ private codecProbeTrack: MediaStreamTrack | null = null;
+ private codecProbeAbort: AbortController | null = null;
+ private codecProbePromise: Promise<void> | null = null;
+ private codecProbeSettled = false;
private disposed = false;
constructor(
@@ -84,7 +91,14 @@ export class ViewerRelay {
this.preparedCandidate = { ...candidate };
this.plannedChildPeerIds = planned;
if (candidate.transport === "direct") {
- this.startPreparedChild(revision, candidate, stream);
+ const pendingProbe = this.codecProbePromise;
+ if (pendingProbe) {
+ void pendingProbe.then(() => {
+ this.startPreparedChildIfCurrent(revision, candidate, stream);
+ });
+ } else {
+ this.startPreparedChildIfCurrent(revision, candidate, stream);
+ }
}
return true;
}
@@ -126,6 +140,7 @@ export class ViewerRelay {
this.preparedCandidate = null;
this.plannedChildPeerIds = [];
this.discardPreparedPeer();
+ this.startCodecProbe();
}
private discardPreparedPeer(): void {
@@ -155,6 +170,9 @@ export class ViewerRelay {
}
}
this.childPeerIds = nextChildPeerIds;
+ if (this.childPeerIds.length > 0) {
+ this.cancelCodecProbe();
+ }
for (const childPeerId of this.childPeerIds) {
const peer = this.peers.get(childPeerId);
if (
@@ -166,17 +184,27 @@ export class ViewerRelay {
this.startPeer(childPeerId, this.stream);
}
}
+ this.startCodecProbe();
}
setStream(stream: MediaStream): void {
if (this.disposed) {
return;
}
+ const nextVideoTrack = stream.getVideoTracks()[0] ?? null;
+ if (nextVideoTrack !== this.codecProbeTrack) {
+ this.cancelCodecProbe();
+ this.codecProbeTrack = nextVideoTrack;
+ if (!this.codecProbeSettled) {
+ this.videoCodec = "vp8";
+ }
+ }
const preparedPeer = this.preparedChild?.peer ?? null;
this.stream = stream;
this.syncQueue = this.syncQueue
.then(() => this.syncStream(stream, preparedPeer))
.catch(() => undefined);
+ this.startCodecProbe();
}
updateProfile(profile: QualityProfile): Promise<boolean> {
@@ -271,6 +299,10 @@ export class ViewerRelay {
}
stop(): void {
+ this.cancelCodecProbe();
+ this.codecProbeTrack = null;
+ this.codecProbeSettled = false;
+ this.videoCodec = "vp8";
this.stream = null;
this.discardPreparedChild();
this.disposePeers();
@@ -281,6 +313,8 @@ export class ViewerRelay {
return;
}
this.disposed = true;
+ this.cancelCodecProbe();
+ this.codecProbeTrack = null;
this.stream = null;
this.discardPreparedChild();
this.childPeerIds = [];
@@ -316,6 +350,23 @@ export class ViewerRelay {
});
}
+ private startPreparedChildIfCurrent(
+ revision: number,
+ candidate: PreparedRouteCandidate,
+ stream: MediaStream,
+ ): void {
+ if (
+ this.disposed ||
+ this.stream !== stream ||
+ this.preparedChild ||
+ this.preparedRevision !== revision ||
+ !sameCandidate(this.preparedCandidate, candidate)
+ ) {
+ return;
+ }
+ this.startPreparedChild(revision, candidate, stream);
+ }
+
private async syncStream(
stream: MediaStream,
preparedPeer: HostPeer | null,
@@ -475,11 +526,61 @@ export class ViewerRelay {
}
},
},
+ this.videoCodec,
connectionId,
);
return peer;
}
+ private startCodecProbe(): void {
+ const track = this.codecProbeTrack;
+ if (
+ this.disposed ||
+ this.codecProbeSettled ||
+ this.codecProbeAbort ||
+ this.codecProbePromise ||
+ !track ||
+ track.readyState === "ended" ||
+ this.childPeerIds.length > 0 ||
+ this.preparedChild
+ ) {
+ return;
+ }
+ const controller = new AbortController();
+ this.codecProbeAbort = controller;
+ const promise = preferredVideoCodecForTrack(
+ track,
+ this.desiredProfile,
+ controller.signal,
+ )
+ .then((codec) => {
+ if (
+ this.codecProbeAbort !== controller ||
+ controller.signal.aborted ||
+ this.codecProbeTrack !== track ||
+ this.disposed
+ ) {
+ return;
+ }
+ this.videoCodec = codec;
+ this.codecProbeSettled = true;
+ this.codecProbeAbort = null;
+ })
+ .finally(() => {
+ if (this.codecProbePromise === promise) {
+ this.codecProbePromise = null;
+ this.startCodecProbe();
+ }
+ });
+ this.codecProbePromise = promise;
+ }
+
+ private cancelCodecProbe(): void {
+ const controller = this.codecProbeAbort;
+ this.codecProbeAbort = null;
+ controller?.abort();
+ }
+
private failPreparedChild(peer: HostPeer): void {
const prepared = this.preparedChild;
if (prepared?.peer !== peer) {
diff --git a/src/client/webrtc/vp8-codec.ts b/src/client/webrtc/vp8-codec.ts
deleted file mode 100644
index 6d0956a1..00000000
--- a/src/client/webrtc/vp8-codec.ts
+++ /dev/null
@@ -1,35 +0,0 @@
-const REPAIR_CODEC_MIME_TYPES = new Set([
- "video/rtx",
- "video/red",
- "video/ulpfec",
- "video/flexfec-03",
-]);
-
-export function applyVp8Codec(transceiver: RTCRtpTransceiver): boolean {
- if (
- typeof RTCRtpSender === "undefined" ||
- typeof transceiver.setCodecPreferences !== "function"
- ) {
- return false;
- }
-
- try {
- const codecs = RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
- const vp8Codecs = codecs.filter(
- ({ mimeType }) => mimeType.toLowerCase() === "video/vp8",
- );
- if (vp8Codecs.length === 0) {
- return false;
- }
-
- transceiver.setCodecPreferences([
- ...vp8Codecs,
- ...codecs.filter(({ mimeType }) =>
- REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()),
- ),
- ]);
- return true;
- } catch {
- return false;
- }
-}
diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts
index 2b8515da..efd8424e 100644
--- a/tests/host-peer-tracks.test.ts
+++ b/tests/host-peer-tracks.test.ts
@@ -7,12 +7,21 @@ import {
import { HostProvisionalChild } from "../src/client/media/host-provisional-child.ts";
import type { PeerSnapshot } from "../src/client/types.ts";
import { HostPeer } from "../src/client/webrtc/host-peer.ts";
+import type { BrowserVideoCodec } from "../src/client/webrtc/video-codec.ts";
import { ViewerRelay } from "../src/client/webrtc/viewer-relay.ts";
import type {
IceConfig,
ParticipantRouteAssignment,
} from "../src/shared/protocol.ts";
+const codecPreflight = vi.hoisted(() => ({
+ probe: vi.fn(async (): Promise<"h264" | "vp8"> => "vp8"),
+}));
+
+vi.mock("../src/client/webrtc/video-codec-preflight.ts", () => ({
+ preferredVideoCodecForTrack: codecPreflight.probe,
+}));
+
const statsCallbacks: Array<() => void> = [];
class FakeSender {
@@ -387,6 +396,7 @@ function createPeer(
onUpdate: (snapshot: PeerSnapshot) => void = () => undefined,
iceConfig: IceConfig = { iceServers: [] },
profile: QualityProfile = QUALITY_PROFILES["720p30"],
+ videoCodec: BrowserVideoCodec = "vp8",
): HostPeer {
return new HostPeer(
"viewer-peer",
@@ -397,6 +407,7 @@ function createPeer(
sendSignal: () => true,
onUpdate,
},
+ videoCodec,
);
}
@@ -434,6 +445,8 @@ async function completeVideoStartup(
}
beforeEach(() => {
+ codecPreflight.probe.mockReset();
+ codecPreflight.probe.mockResolvedValue("vp8");
FakePeerConnection.latest = null;
FakePeerConnection.instances = [];
FakePeerConnection.activeCount = 0;
@@ -519,6 +532,50 @@ describe("HostPeer source replacement", () => {
expect(connection.createOfferCallCount).toBe(1);
});
+ it("prefers native H264 mode 1 and retains VP8 fallback", async () => {
+ vi.stubGlobal("RTCRtpSender", {
+ getCapabilities: () => ({
+ codecs: [
+ {
+ mimeType: "video/H264",
+ clockRate: 90_000,
+ sdpFmtpLine: "packetization-mode=0;profile-level-id=42001f",
+ },
+ {
+ mimeType: "video/H264",
+ clockRate: 90_000,
+ sdpFmtpLine: "packetization-mode=1;profile-level-id=42001f",
+ },
+ { mimeType: "video/VP8", clockRate: 90_000 },
+ { mimeType: "video/rtx", clockRate: 90_000 },
+ { mimeType: "video/RED", clockRate: 90_000 },
+ ],
+ headerExtensions: [],
+ }),
+ });
+ const peer = createPeer(
+ createStream(createTrack("video", "video"), null),
+ () => undefined,
+ { iceServers: [] },
+ QUALITY_PROFILES["720p30"],
+ "h264",
+ );
+
+ await expect(peer.start()).resolves.toBe(true);
+
+ expect(
+ FakePeerConnection.latest!.codecPreferenceCalls[0]?.map((codec) => [
+ codec.mimeType.toLowerCase(),
+ codec.sdpFmtpLine ?? null,
+ ]),
+ ).toEqual([
+ ["video/h264", "packetization-mode=1;profile-level-id=42001f"],
+ ["video/vp8", null],
+ ["video/rtx", null],
+ ["video/red", null],
+ ]);
+ });
+
it("fails before creating an offer when codec preferences are unavailable", async () => {
FakePeerConnection.omitCodecPreferenceSetter = true;
const updates: PeerSnapshot[] = [];
@@ -531,7 +588,7 @@ describe("HostPeer source replacement", () => {
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
expect(FakePeerConnection.latest!.localDescription).toBeNull();
- expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
+ expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
});
it("fails before creating an offer when VP8 is unavailable", async () => {
@@ -554,7 +611,7 @@ describe("HostPeer source replacement", () => {
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
expect(FakePeerConnection.latest!.localDescription).toBeNull();
- expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
+ expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
});
it("fails before creating an offer when codec preference setup fails", async () => {
@@ -569,7 +626,7 @@ describe("HostPeer source replacement", () => {
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
expect(FakePeerConnection.latest!.localDescription).toBeNull();
- expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
+ expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
});
it("applies STUN-only ICE configuration at creation and update", () => {
@@ -1384,6 +1441,7 @@ function hostProvisionalInput(
iceConfig: { iceServers: [] },
stream,
profile: QUALITY_PROFILES["720p30"],
+ videoCodec: "vp8" as const,
};
}
@@ -1516,6 +1574,29 @@ describe("ViewerRelay downstream ownership", () => {
transport: "direct" as const,
});
+ it("uses a completed H264 probe only for a future child", async () => {
+ codecPreflight.probe.mockResolvedValueOnce("h264");
+ const relay = new ViewerRelay(
+ { iceServers: [] },
+ QUALITY_PROFILES["720p30"],
+ { sendSignal: () => true },
+ );
+ relay.setStream(createStream(createTrack("video", "h264-source"), null));
+ await vi.waitFor(() => expect(codecPreflight.probe).toHaveBeenCalledOnce());
+ await Promise.resolve();
+ relay.setChildren(["h264-child"]);
+ await vi.waitFor(() =>
+ expect(FakePeerConnection.latest?.codecPreferenceCalls).toHaveLength(1),
+ );
+
+ expect(
+ FakePeerConnection.latest!.codecPreferenceCalls[0]?.map(({ mimeType }) =>
+ mimeType.toLowerCase(),
+ ),
+ ).toEqual(["video/h264", "video/vp8"]);
+ relay.dispose();
+ });
+
it("promotes the exact prepared child connection within the current Viewer cap", async () => {
const signals: Array<{ peerId: string; connectionId: string }> = [];
const relay = new ViewerRelay(
@@ -1593,6 +1674,7 @@ describe("ViewerRelay downstream ownership", () => {
);
relay.setStream(createStream(createTrack("video", "rollback-video"), null));
expect(relay.prepareChild(7, routeCandidate(7, "first-probe"), ["first-probe"])).toBe(true);
+ await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
const firstProbe = FakePeerConnection.latest!;
expect(relay.prepareChild(8, routeCandidate(8, "second-probe"), ["second-probe"])).toBe(true);
const secondProbe = FakePeerConnection.latest!;
@@ -1675,6 +1757,7 @@ describe("ViewerRelay downstream ownership", () => {
["prepared-child"],
),
).toBe(true);
+ await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
const preparedConnection = FakePeerConnection.latest!;
expect(preparedConnection.senders[1]?.track).toBeNull();
@@ -1701,6 +1784,7 @@ describe("ViewerRelay downstream ownership", () => {
);
relay.setStream(createStream(createTrack("video", "cap-video"), null));
expect(relay.prepareChild(1, routeCandidate(1, "child-0"), ["child-0"])).toBe(true);
+ await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
const preparedConnection = FakePeerConnection.latest!;
preparedConnection.connectionState = "connected";
relay.activateChildren(1, ["child-0"]);
diff --git a/tests/route-transition.test.ts b/tests/route-transition.test.ts
index 31afb92d..ff44c4fe 100644
--- a/tests/route-transition.test.ts
+++ b/tests/route-transition.test.ts
@@ -259,6 +259,7 @@ describe("minimal route transition contracts", () => {
const route = new HostSfuRoute({
getStream: () => ({}) as MediaStream,
getProfile: () => QUALITY_PROFILES["720p30"],
+ getVideoCodec: () => "h264",
reconcileChildren: () => undefined,
send: (message) => {
messages.push(message);
@@ -281,6 +282,11 @@ describe("minimal route transition contracts", () => {
});
await route.acceptConfig(sfuConfig(1));
expect(publishers[0]?.activate).toHaveBeenCalledOnce();
+ expect(publishers[0]?.activate).toHaveBeenCalledWith(
+ expect.anything(),
+ QUALITY_PROFILES["720p30"],
+ "h264",
+ );
route.accept({
revision: 2,
@@ -311,6 +317,7 @@ describe("minimal route transition contracts", () => {
const route = new HostSfuRoute({
getStream: () => ({}) as MediaStream,
getProfile: () => QUALITY_PROFILES["720p30"],
+ getVideoCodec: () => "vp8",
reconcileChildren: (children) => reconciledChildren.push(children),
send: () => true,
createPublisher: () => {
@@ -365,6 +372,7 @@ describe("minimal route transition contracts", () => {
const route = new HostSfuRoute({
getStream: () => ({}) as MediaStream,
getProfile: () => QUALITY_PROFILES["720p30"],
+ getVideoCodec: () => "vp8",
reconcileChildren: () => undefined,
send: () => true,
createPublisher: () => {
diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts
index 9636888f..6d5bcb14 100644
--- a/tests/sfu-client.test.ts
+++ b/tests/sfu-client.test.ts
@@ -1,5 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { QUALITY_PROFILES } from "../src/client/media/quality.ts";
import { SfuPublisher } from "../src/client/sfu/publisher.ts";
import { SfuSubscriber } from "../src/client/sfu/subscriber.ts";
import type { ConnectionMetrics } from "../src/client/types.ts";
@@ -878,6 +879,27 @@ describe("SfuPublisher", () => {
});
});
+ it("publishes one H264 source without a backup codec", async () => {
+ const publisher = new SfuPublisher();
+ const video = track("video", "video-h264");
+ await publisher.connect(connection);
+
+ await expect(
+ publisher.activate(stream(video), qualityProfile, "h264"),
+ ).resolves.toBe(true);
+
+ const publication = livekit.state.rooms[0].localParticipant.publications[0];
+ expect(publication.options).toMatchObject({
+ source: Track.Source.ScreenShare,
+ backupCodec: false,
+ videoCodec: "h264",
+ });
+ await expect(
+ publisher.updateProfile(QUALITY_PROFILES["720p30"]),
+ ).resolves.toBe(true);
+ expect(publication.options.videoCodec).toBe("h264");
+ });
+
it("keeps media and prior audio readback when a live update fails", async () => {
const publisher = new SfuPublisher();
await publisher.connect(connection);
diff --git a/tests/stats-grid.test.ts b/tests/stats-grid.test.ts
index ab0f3003..59ec1917 100644
--- a/tests/stats-grid.test.ts
+++ b/tests/stats-grid.test.ts
@@ -107,14 +107,14 @@ describe("StatsGrid progressive disclosure", () => {
createElement(StatsGrid, {
metrics: {
...metrics,
- codec: "video/H264",
+ codec: "video/VP9",
audioCodec: "audio/PCMU",
},
direction: "receive",
}),
);
- expect(html).toContain("视频编码为 H264,预期 VP8");
+ expect(html).toContain("视频编码为 VP9,预期 H264 或 VP8");
expect(html).toContain("音频编码为 PCMU,预期 Opus");
expect(html).not.toContain("视频 Codec");
expect(html).not.toContain("音频 Codec");
diff --git a/tests/video-codec-preflight.test.ts b/tests/video-codec-preflight.test.ts
new file mode 100644
index 00000000..9c677ab9
--- /dev/null
+++ b/tests/video-codec-preflight.test.ts
@@ -0,0 +1,128 @@
+import { describe, expect, it } from "vitest";
+
+import {
+ h264ProbeSustainsSource,
+ type H264ProbeSample,
+} from "../src/client/webrtc/video-codec-preflight.ts";
+
+function sample(
+ overrides: Partial<H264ProbeSample> = {},
+): H264ProbeSample {
+ return {
+ outboundId: "outbound",
+ sourceId: "source",
+ timestamp: 1_000,
+ codec: "video/H264",
+ framesEncoded: 10,
+ sourceFrames: 10,
+ encodedFramesPerSecond: 30,
+ sourceFramesPerSecond: 30,
+ qualityLimitationReason: "none",
+ ...overrides,
+ };
+}
+
+describe("H264 sender preflight", () => {
+ it("accepts bounded pipeline lag against the same source", () => {
+ const baseline = sample();
+ const current = sample({
+ timestamp: 2_000,
+ framesEncoded: 39,
+ sourceFrames: 40,
+ });
+
+ expect(h264ProbeSustainsSource(baseline, current)).toBe(true);
+ });
+
+ it("rejects sustained encoder frame dropping", () => {
+ const baseline = sample();
+ const current = sample({
+ timestamp: 2_000,
+ framesEncoded: 22,
+ sourceFrames: 40,
+ encodedFramesPerSecond: 12,
+ sourceFramesPerSecond: 30,
+ });
+
+ expect(h264ProbeSustainsSource(baseline, current)).toBe(false);
+ });
+
+ it("waits for enough source progress before deciding", () => {
+ const baseline = sample();
+ const current = sample({
+ timestamp: 1_400,
+ framesEncoded: 20,
+ sourceFrames: 20,
+ });
+
+ expect(h264ProbeSustainsSource(baseline, current)).toBeNull();
+ });
+
+ it("rejects CPU-limited or identity-mismatched samples", () => {
+ const baseline = sample();
+
+ expect(
+ h264ProbeSustainsSource(
+ baseline,
+ sample({
+ timestamp: 2_000,
+ framesEncoded: 40,
+ sourceFrames: 40,
+ qualityLimitationReason: "cpu",
+ }),
+ ),
+ ).toBe(false);
+ expect(
+ h264ProbeSustainsSource(
+ baseline,
+ sample({
+ outboundId: "replacement",
+ timestamp: 2_000,
+ framesEncoded: 40,
+ sourceFrames: 40,
+ }),
+ ),
+ ).toBe(false);
+ });
+
+ it("uses reported cadence when cumulative source frames are unavailable", () => {
+ const baseline = sample({ framesEncoded: null, sourceFrames: null });
+
+ expect(
+ h264ProbeSustainsSource(
+ baseline,
+ sample({
+ timestamp: 2_000,
+ framesEncoded: null,
+ sourceFrames: null,
+ encodedFramesPerSecond: 29,
+ sourceFramesPerSecond: 30,
+ }),
+ ),
+ ).toBe(true);
+ expect(
+ h264ProbeSustainsSource(
+ baseline,
+ sample({
+ timestamp: 2_000,
+ framesEncoded: null,
+ sourceFrames: null,
+ encodedFramesPerSecond: 12,
+ sourceFramesPerSecond: 30,
+ }),
+ ),
+ ).toBe(false);
+ expect(
+ h264ProbeSustainsSource(
+ baseline,
+ sample({
+ timestamp: 1_500,
+ framesEncoded: null,
+ sourceFrames: null,
+ encodedFramesPerSecond: 30,
+ sourceFramesPerSecond: 30,
+ }),
+ ),
+ ).toBeNull();
+ });
+});