Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

feat(web): prefer H.264 media [skip ci]

AAA74102782199c
diff --git a/docs/project-memory.md b/docs/project-memory.md
index e055c7f3..42c06e03 100644
--- a/docs/project-memory.md
+++ b/docs/project-memory.md
@@ -13,7 +13,7 @@ Last updated: 2026-08-21
- Direct/peer keeps per-PC stock GCC. Active SFU publishes exactly `q,h` at a `HIGH` ceiling; built-in BWE may forward `LOW`. The zero-descendant quality/resource gate is next; if it passes, no app media selector is built. Explicit quality, manual activation, and custom/native remain fallbacks.
- A later explicit quality fallback evacuates children under a generation guard first. Autonomous BWE enters `suspect`; confirmation evacuates children, and no confirmed `FALLBACK` parent remains. Root-with-children impact is a default-on gate; capacity returns after longer recovery plus cooldown. Self-report alone never triggers it.
- Site access protects room creation, Host publication, and every code-only Viewer entry. A valid room-scoped fragment grant bypasses that site gate; otherwise site access is checked before public-watch or a private room password. Persist neither raw credential; keep accounts, ACLs, users, and session tables out.
-- Keep decisions, snapshots, research, code, `AGENTS.md`, and `.codex/` in Git; rewrite memory/status in place. Research current primary sources before material work and reject speculative machinery.
+- Prefer primary sources, maintained implementations, target-device evidence and sanitized production stats. Ordinary-PC synthetic runs prove only correctness/interoperability; never use them to calibrate performance or block a reversible standard API. Keep accepted truth in Git and replace stale facts in place.
- Migrate client, server, and deployment atomically. After a canary, delete superseded config/wire/parsers/tests; do not retain compatibility layers, dual writes, or a second architecture without a current consumer. Git history owns the old implementation.
- Deploy after narrow tests, independent review, one local full gate, and rollback preflight. Build/test off the 960 MiB production host; prepare audited dist with one bounded production install. Actions remain for a necessary main/release gate only.
@@ -24,7 +24,7 @@ Last updated: 2026-08-21
- Use direct host P2P for one or two viewers, then the bounded controller for every room when `PEER_ASSISTED_MEDIA=true`. Room `1` is historical smoke, not a runtime gate; resource, quality, recovery, SFU/UDP, bounded-failure, and browser/mobile evidence remain separate.
- Browser relays resend remote `MediaStreamTrack` values and re-encode at each hop; WebRTC does not guarantee a shared encoder across peer connections, so measure the cost.
- Keep experiments bounded: the standard representation sequence is below; native RTP relay, encoded-object striping, and FEC remain separate.
-- ADR-0006's historical browser-bridge canary remains no-go; VP8 stays default. Browser H.264 rendered 298/299; Native WGC/MF rendered 203 with PID/LUID-correlated `VideoEncode`. Downloaded use, multi-viewer/endurance/public proof remain open.
+- Current Web source prefers H.264 through standard ordering, keeps the browser fallback list, and requests SFU H.264 with `backupCodec=false`; VP8 is used only when H.264 is unavailable. This is not a second encoder stack or shared-encode proof. ADR-0006's browser bridge remains no-go.
- Mobile endpoints are Web Viewer-only. Current Android/iOS browsers do not expose Web Host capture; AirPlay/system mirroring is Viewer-local output.
- ADR-0005: direct/peer UDP -> bounded SFU roots -> selected-edge TURN. The deployed controller caps pending/answered `peer-selected` at one per room; Host ingress is independent. A pre-`ecc794d` exact-source/production-media canary proves active Host ingress only. Ordinary peers stay STUN-only.
- The controller is process-enabled, not room-allowlisted: `PEER_ASSISTED_MEDIA=true` gives every normal room the same direct/peer -> SFU -> selected-edge path with per-room state. Room `1` is historical smoke. Preserve sticky P2P, one active upstream and break-before-make; recovery gets one attempt per layer (ICE restart, same-parent rebuild, alternate peer, then SFU), never three identical retries.
@@ -38,7 +38,7 @@ Last updated: 2026-08-21
## Current Execution Principle
-- Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/logs/comments=current; history/migration=`historical`; ship reverse-scan visible copy->source; drop no-consumer layer; copy masks no wrong model.
+- Parallel flagship work; minimum relevant checks and rollback proof. Research plus target-device/production evidence drives product choices; synthetic runs cover only correctness/interoperability. Active text is current, Git owns history, and no-consumer layers are deleted.
## Current Implementation
@@ -51,7 +51,6 @@ Last updated: 2026-08-21
- Production requires the independent site-access secret through `SITE_ACCESS_PASSWORD`. Its stateless cookie authorizes creation/Host and permits code-only Viewer attempts; valid private fragment grants remain direct. Public-watch accepts site access plus code, while private code-only entry additionally requires the room password. The env, endpoint, cookie, and nginx limiter naming migrated atomically; anonymous failures stay neutral, and there are no accounts/JWT/session rows.
- Production uses nginx, Node.js 24.19.0, coturn 4.17.2, and LiveKit 1.13.5 on UDP 7882 (TCP fallback off). Ordinary peer ICE receives no TURN; the selected-edge tuple is configured only for the current controller edge, with no credential pre-advertised to ordinary peers. Coturn retains the old authenticated-relay ports. Services are healthy with zero restarts.
- ADR-0005 rejects `PEER_ASSISTED_ROOM_IDS`; `PEER_ASSISTED_MEDIA=true` enables every room and ordinary peers stay STUN-only. Initial or active Host ingress may consume one bound relay-only grant; ready/abort clears it and failure restores peer baseline. Stale room-ID config fails startup; `screener-v2` is the only deployed wire.
-- Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence.
- Web SFU uses LiveKit 2.22.0 `q,h`; local SFU/UDP and selected-TURN ingress work, while BWE remains open. P2P now reapplies its selected profile after each answer on the same sender. A local A/B supports the lifecycle candidate, but natural ramp, real capture, production, and SFU validation remain open.
- Chrome 151 five-Viewer 720p30 cap2 evidence: Host/relay fanout two; both relay children +80 decoded frames/~1.1 MB; every Viewer decoded; clean stop. Functional only; resource and heterogeneous-network behavior remain open.
@@ -73,7 +72,7 @@ These are measurement gates, not performance claims.
- Project license and distribution model, which determines whether GPL/AGPL sources can move beyond study-only use.
- Initial deployment regions and expected mainland China, Hong Kong, and overseas network mix.
- Whether voice chat ever enters scope or Screener stays complementary to an existing voice application.
-- Production calibration of local-reparent thresholds, physical H.264 hardware attribution, and the ADR-0007 native matrix; #28's minimum-of-two rule is not a candidate.
+- Production calibration of reparent thresholds and any adaptive codec policy. Hardware attribution gates claims, not H.264 preference; #28's minimum-of-two rule is not a candidate.
## Source Of Truth
diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md
index 06b40e82..854a35ae 100644
--- a/docs/research/realtime-quality-adaptation.md
+++ b/docs/research/realtime-quality-adaptation.md
@@ -142,28 +142,42 @@ server traffic but may still consume compositor/GPU work. Compare the exact
release and current `main` under one fixture, with preview on/off as a separate
binary intervention.
-## Codec And Hardware Gate
-
-Do not default to AV1 from compression efficiency alone. The deployed release
-does not call `setCodecPreferences()`, and a browser may expose a negotiable
-codec without a power-efficient WebRTC encoder. `getCapabilities()` establishes
-only an optimistic negotiation set; Media Capabilities supplies
-supported/smooth/power-efficient candidate evidence; outbound `codecId`,
-`encoderImplementation`, and `powerEfficientEncoder` describe the stream only
-when the browser exposes them. Final acceptance still requires interval encode
-cost, game FPS, CPU/GPU video-encode activity, and active sender count. Chromium
-`main` on the access date gates WebRTC AV1 hardware encoding off by default on
-Windows even when a platform accelerator exists; this is an implementation
-snapshot, not a permanent browser contract.
-
-First diagnose the Host-refresh case using the browser-selected codec. Only if
-encode cost or `cpu` limitation is abnormal, hold scene, resolution/FPS/bitrate,
-network, and viewer constant while comparing browser default, H.264, and VP8.
-VP9 or AV1 enters that spike only when both endpoints can negotiate it and
-Media Capabilities reports the exact configuration supported, smooth, and
-power-efficient. A preference change must retain negotiated repair codecs and
-is successful only when outbound stats prove the codec actually in use; silent
-software fallback fails the gate.
+## Codec Preference And Evidence Boundary
+
+RFC 7742 requires WebRTC browsers to implement both VP8 and H.264 Constrained
+Baseline, while W3C `setCodecPreferences()` lets the application reorder the
+browser's negotiated codec set. The smallest Web policy is therefore to place
+all advertised H.264 entries first and retain the complete repair and fallback
+list. If H.264 capability is absent or the API rejects the preference, leave the
+browser default unchanged. LiveKit publication requests H.264 only when the
+sender advertises it, falls back to VP8 otherwise, and keeps `backupCodec=false`;
+there is no parallel backup-codec publication.
+
+H.264 is the accepted product preference because its mature cross-device hardware
+encode/decode paths address the observed VP8 software-path cost and are more
+valuable here than VP8's royalty-free implementation model. This does not claim
+an intrinsic bitrate win for every implementation or scene; actual outbound
+codec/profile and decoded stats remain the result truth. The choice also does not
+prove shared encode: separate browser PeerConnections may construct separate
+encoders, and the SFU intentionally publishes exactly the existing `q,h` H.264
+representations.
+
+Open-source distribution is not itself a patent-license exemption. This Web
+change only requests a codec already implemented by the browser/LiveKit path and
+ships no H.264 codec binary or new codec dependency, so licensing uncertainty is
+not a runtime blocker for the preference. Bundling a codec implementation or
+changing the distribution/service model still requires a separate license review.
+
+This reversible standard preference is gated by focused ordering, fallback and
+interoperability tests, not by an exhaustive CPU/GPU/game matrix on one ordinary
+PC. Primary specifications, maintained implementation behavior, representative
+target-device observations and sanitized production stats drive product choices.
+Synthetic local runs may verify negotiation, decode, cleanup and edge bounds;
+they must not calibrate capacity, claim performance, or block a standard feature.
+Hardware attribution and controlled performance evidence remain required before
+making efficiency claims or introducing dynamic codec selection. AV1 remains
+outside the default because compression efficiency alone does not establish a
+power-efficient WebRTC encoder on the target cohort.
Discord's published Go Live material is a useful architecture comparison, not
a preset to copy. It describes native OS/driver-integrated capture and encoding,
@@ -628,6 +642,8 @@ is a separate optimization.
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/)
- [W3C WebRTC](https://www.w3.org/TR/webrtc/)
- [W3C WebRTC codec preferences](https://www.w3.org/TR/webrtc/#dom-rtcrtptransceiver-setcodecpreferences)
+- [RFC 7742 WebRTC video codec requirements](https://www.rfc-editor.org/rfc/rfc7742.html)
+- [Via LA AVC/H.264 licensing program](https://via-la.com/licensing-programs/avc-h-264/)
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
diff --git a/docs/status.md b/docs/status.md
index a6ef4e74..926eaed6 100644
--- a/docs/status.md
+++ b/docs/status.md
@@ -7,11 +7,11 @@ Last updated: 2026-08-21
Production at `https://share.bonfire.icu` is exact `261e980c3a9ff2d1a6b54ce18cf3daedb491b777`; immutable source/dist artifacts are 604,336/354,230 bytes, with hashes in deployment.
Cutover: 2,276.366 ms lock/703.625 ms stop-health/562.538 ms switch-health. SQLite v3/five rooms has SHA-256 `aef724ae52c4107be8ec8791e72f8dcaa11b0ec849fb432d022e2cfb9cfe0974`, owner/mode `screener:screener`/0600. Four services are active/running at `NRestarts=0`; rollbacks are `691863e`, then `6634cb9` and `ecc794d`.
Ordinary ICE is STUN-only; limits are SFU roots <=2, one `peer-selected` attempt excluding Host ingress, and selected UDP TTL 120. This rollout ran no browser/media/SFU/TURN/performance canary; earlier exact-`16f6eab` Host-ingress proof remains separate.
-Stable SFU state, evidence-backed route/transport labels, Host preview notice, entry/nickname UI and Web relay cap2 are live. Native is source-only; Web defaults VP8.
+Stable SFU, route/transport truth, preview notice, entry/nickname UI and Web relay cap2 are live. Native is source-only. Production `261e980c` uses VP8; current source prefers H.264 with negotiated VP8 fallback.
## Execution Principle
-Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/logs/comments=current; history/migration=`historical`; ship reverse-scan visible copy->source; drop no-consumer layer; copy masks no wrong model.
+Parallel flagship work; minimum relevant checks and rollback proof. Research plus target-device/production evidence drives performance decisions; ordinary-PC synthetic runs prove only correctness/interoperability. Active text is current; Git owns history; delete no-consumer layers.
## Current Snapshot
@@ -29,7 +29,7 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
- Chrome 151 + local LiveKit 1.13.5/client 2.22.0 SFU/UDP passed after `q,f` -> `q,h`: one root, decoded 3 -> 23, rendered 26, Host edge one, clean leaves, no TURN. Functional only.
- Host A+B and authenticated P2P Viewer C are sanitized, generation-bound, read-only, and fail closed for stale, ambiguous, or SFU-fed evidence; no raw media metadata is retained.
- The `a11a73d` built-in TURN canary is rejected: local allocation passed, but direct Host/Pion Viewer failed before forced relay. Full rollback restored STUN-only client ICE and zero allocations.
-- Chrome 151 browser loopbacks decoded VP8 299 and H.264 opt-in 299/298 rendered at 1280x720 without fatal/encoder errors. Browser hardware attribution, multi-viewer/endurance and public proof remain open; production stays VP8.
+- Chrome 151 loopback proves H.264 negotiation/decode interoperability, not performance. Source orders H.264 first and keeps browser repair/fallback codecs; production remains VP8 until deployment.
- Native WGC/MF used no browser capture/encoder. One Chrome Viewer rendered 203 1280x720 frames and received 500 Opus packets; process/LUID-correlated `VideoEncode` was nonzero, with no fatal/encoder errors.
- Native Win11 audio is source-only/default-off: target isolation was 4018x and one Viewer got 495 Opus packets. Download, game sync, Win10, and other routes remain open.
- Access protocol/config/HTTP/storage/SQLite/signaling focused tests pass, including commit-first teardown and v1 rollback.
@@ -53,7 +53,7 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
Next: representative production-room UDP/frame/cap/stop canary; room `1` is history. TURN performance follows later.
Recovery uses one ICE restart, one same-parent rebuild, one alternate peer, then SFU. Next gate exactly-two/Dynacast-off BWE on a zero-child SFU leaf, then root evacuation.
-Native WGC/MF passed one-Viewer hardware; package download, Viewer2/FIFO and game A/V remain. Benchmark later.
+Native WGC/MF passed one-Viewer hardware; package download, Viewer2/FIFO and game A/V remain. Use representative target-device or production evidence for performance, not an ordinary-PC synthetic gate.
Ordinary peers stay STUN-only; initial ingress and `peer-selected` still need forced-relay proof.
ADR-0004 still needs its resource/quality matrix; the 20-viewer gate precedes a default change. Mobile uses the same capacity and is a compatibility observation.
diff --git "a/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md" "b/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
index 28b29978..f4f5b6b7 100644
--- "a/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
+++ "b/docs/\346\226\271\346\241\210\350\256\276\350\256\241.md"
@@ -156,13 +156,13 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级,实际结果由 stats 观测;拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。
-PoC 不强制 codec,也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`,再依据目标机器测量决定是否调整 H.264、VP8、VP9 或 AV1 优先级。`getCapabilities()` 只证明可协商,Media Capabilities 的 supported/smooth/power-efficient 只提供候选证据;最终必须同时检查实际 outbound codec、区间 encode time、游戏 FPS、CPU/GPU encode 引擎和 sender 数量。AV1 压缩效率不能证明目标机器具有 WebRTC 硬件编码,强制后可能落入更重的软件路径。
+Web 端不做 SDP munging。P2P 与浏览器 relay 在首个 offer 前读取标准 sender capabilities;存在 H.264 时用 `setCodecPreferences()` 将全部 H.264 条目置前,同时保留完整的 browser codec/repair 列表,能力缺失或 API 拒绝时维持浏览器默认。SFU publication 同样在 H.264 能力存在时请求 `videoCodec: "h264"`,否则请求 VP8,并始终保持 `backupCodec: false`,因此不会为兼容回退同时维护第二份 codec publication。实际 outbound codec/profile 与 stats 是协商结果真相。这个可逆偏好只要求定向互通和回退验证;硬件、CPU/GPU、游戏 FPS 和目标网络证据用于性能结论及未来动态选择,不作为上线前置。AV1 仍不得仅凭压缩率推断默认启用。
`audio: true` 只是请求。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`:窗口选项请求该窗口/应用音频,整个屏幕选项允许用户勾选系统音频;标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些 picker hints,且没有对应的标准来源类型读回,因此 UI 不额外推断或标注来源。系统音频由用户显式选择,可能包含语音或通知;video-only 始终可用。
当前 Windows Native sender 用本地 opaque ID 保存所选窗口的 HWND、PID 与进程创建时间;helper 在使用前重新验证三者绑定,并以 WASAPI process loopback 的 include-target-process-tree 捕获游戏及其子进程。目标进程无渲染流、退出或捕获失败时显示无声/未知,不回退为整机音频,也不上传 PID、路径、窗口标题、设备标识或原始音频。该路径只支持 Windows 11;Windows 10 game-only 音频保持明确不支持。
-同一 helper 的显式 `native-window-h264` source 用 `CreateForWindow` 建立 WGC item,在所选 adapter 的 free-threaded D3D11 frame pool 接收 BGRA surface;D3D11 VideoProcessor 做固定 1280x720/30 aspect-fit NV12 转换,随后进入 adapter-LUID-bound、hardware-only asynchronous Media Foundation H.264 MFT。输出必须是 Annex-B `42c01f`,首次、周期和反馈恢复帧携带 SPS、PPS 与 IDR。PCM、H.264 access unit 和 local-only hardware status 通过一个有界 helper protocol 进入 generation-bound Go session;H.264 直接写现有 shared fanout,UI 只收一份单向预览副本,PCM 保持现有本地 WebCodecs Opus bridge。任一 Native capture/encode 失败都终止该 source,不静默改用软件 MFT、其他 codec、browser capture、monitor capture 或 system mix;browser VP8 保持默认,browser H.264 保持独立 opt-in。一名普通 Chrome Viewer 已 decoded/rendered 203 帧并收到 500 Opus packets;同一 helper PID 与 adapter LUID 的 `VideoEncode` 观测匹配。
+同一 helper 的显式 `native-window-h264` source 用 `CreateForWindow` 建立 WGC item,在所选 adapter 的 free-threaded D3D11 frame pool 接收 BGRA surface;D3D11 VideoProcessor 做固定 1280x720/30 aspect-fit NV12 转换,随后进入 adapter-LUID-bound、hardware-only asynchronous Media Foundation H.264 MFT。输出必须是 Annex-B `42c01f`,首次、周期和反馈恢复帧携带 SPS、PPS 与 IDR。PCM、H.264 access unit 和 local-only hardware status 通过一个有界 helper protocol 进入 generation-bound Go session;H.264 直接写现有 shared fanout,UI 只收一份单向预览副本,PCM 保持现有本地 WebCodecs Opus bridge。任一 Native capture/encode 失败都终止该 source,不静默改用软件 MFT、其他 codec、browser capture、monitor capture 或 system mix;Web 端的 H.264-first 协商与 VP8 fallback 是独立标准 WebRTC 路径。一名普通 Chrome Viewer 已 decoded/rendered 203 帧并收到 500 Opus packets;同一 helper PID 与 adapter LUID 的 `VideoEncode` 观测匹配。
直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`,不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流;成功后为每个现有 sender 调用 `replaceTrack()`,健康连接不重新协商。初始 offer 预留 send-only audio transceiver,使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败,只重建该 peer;新流提交后才停止旧流,并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置,避免把不同 track 的计数器混算。
diff --git "a/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md" "b/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
index c78067bc..8eb96985 100644
--- "a/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
+++ "b/docs/\351\234\200\346\261\202\347\220\206\350\247\243.md"
@@ -54,13 +54,13 @@
- [ ] Windows 应用音频:Windows 11 Native sender 使用本地 opaque window target 绑定 HWND、PID 与进程创建时间,只捕获所选游戏及其子进程的音频,避免把语音聊天或通知声一并分享;“全部系统声音”只能由分享者显式选择。普通 Windows 10 不具备当前 process-loopback API 门槛,必须明确标为未解决/不支持而不是伪兼容。浏览器无法证明或隔离任意进程音频时必须如实提示,原生捕获失败、无声或进程退出不得静默扩大为整机捕获。
- [ ] 连接自检:在开始分享前验证信令、STUN 和 SFU ICE/UDP 可达性,并给出不暴露地址或凭据的诊断结果。启用 selected-edge TURN canary 时只报告脱敏的授权阶段与 selected candidate type,不输出 URL、username、credential、地址或原始 candidate;自检只覆盖当前 UDP 媒体梯级。
- [ ] 受限网络增强:短期鉴权 TURN 只能由完整、默认关闭的部署 tuple 与当前 room controller 启用,并且只能在 direct/peer UDP 与 SFU/UDP 均已按有界梯级失败后,向 controller 选中的当前异常 edge 一次性授权。普通 peer PC 从创建起保持 STUN-only;不得向整个 participant、房间或所有 peer connections 预发 TURN candidates。凭据必须绑定当前授权 session 与 route/connection generation、短期有效、内存态、一次性消费且不进入 URL、日志、浏览器持久化或 SQLite。媒体保持 UDP-only,全部路径耗尽后明确失败;TURN 不得被描述为更高清的传输方式。
-- [ ] 运行时编解码选择:先保留浏览器默认协商;只有端点能力相交、Media Capabilities 对目标分辨率/FPS/码率报告 supported/smooth/power-efficient,且实际 outbound codec、区间 encode time、游戏 FPS、CPU/GPU 与 sender 数量共同通过时,才调整 H.264、VP8、VP9 或 AV1 优先级。AV1 不得因压缩率推断而默认启用,任何候选都不得静默落入高负载软件编码。
+- [x] Web H.264 优先与协商回退:P2P/浏览器 relay 在首个 offer 前用标准 `setCodecPreferences()` 把浏览器发送能力中的全部 H.264 条目置前,同时保留完整的 repair 与其他 codec 列表;SFU 在端点支持 H.264 时请求 H.264 并保持 `backupCodec = false`。能力缺失或服务端不能采用时回退 VP8,不维护第二套应用编码链。实际协商 codec/profile 和 stats 是结果真相;目标设备/生产证据约束性能宣传与未来自适应选择,但完整 CPU/GPU/游戏矩阵不再阻塞这个可逆标准偏好。AV1 仍不得仅凭压缩率推断默认启用。
- [ ] 诊断页面:用户可导出不含敏感地址和凭据的连接诊断信息。
### P2 - 可选功能
- [ ] 证据驱动的统一 relay capacity:当前上线档固定为 `2`,不是永久产品上限。后续只在独立验收后允许所有 Web relay 使用同一套离散 `0/1/2/3` 容量状态,不按设备或 UA 分类;持续 sender bitrate、`availableOutgoingBitrate`、区间 encode time、`qualityLimitationReason`、loss 与 RTT 健康窗共同决定升降,服务端 deployment hard ceiling 与 cooldown 继续限界。H.264、hardware encode 或 shared encode 可以降低编码成本或同观感码率,但每个 child 仍消耗一份网络副本。本次 slice 不实现动态控制。
-- [x] 原生 Windows 捕获辅助程序:显式 `native-window-h264` 模式使用同一 bound window target 的 WGC/D3D11 画面与 WASAPI process-tree PCM,并用 adapter-bound、hardware-only Media Foundation H.264 `42c01f` 直接进入现有 Go fanout;浏览器 VP8/H.264 是显式替代选项,VP8 默认,不允许运行时静默改用软件编码、显示器画面或系统混音。一名普通 Chrome Viewer 已 decoded/rendered 203 帧并收到 500 Opus packets,同一 helper PID/adapter 的 `VideoEncode` 与 LUID 匹配。
+- [x] 原生 Windows 捕获辅助程序:显式 `native-window-h264` 模式使用同一 bound window target 的 WGC/D3D11 画面与 WASAPI process-tree PCM,并用 adapter-bound、hardware-only Media Foundation H.264 `42c01f` 直接进入现有 Go fanout;Web 端独立使用标准 H.264-first 协商并保留 VP8 回退,不允许运行时静默改用软件编码、显示器画面或系统混音。一名普通 Chrome Viewer 已 decoded/rendered 203 帧并收到 500 Opus packets,同一 helper PID/adapter 的 `VideoEncode` 与 LUID 匹配。
- [ ] Viewer 向电视输出:系统级屏幕镜像现在可直接使用;产品只复用平台 AirPlay/Cast,不自造投屏协议。截至 2026-08-21,Chromium 的 Remote Playback URL 路径和 WebKit 明确拒绝 wireless target 的 live `MediaStream` player 均未给当前 WebRTC `srcObject` 提供可用契约,因此不显示伪可用入口。只在浏览器实现明确支持并由实体 Apple TV 或 Chromecast/Google TV 证明当前 live 画面与音频后,才按通过的平台加入 Viewer 本地图标按钮。投屏不能改变 room、TURN/SFU/peer route、Host fanout 或服务端媒体分发;Google Cast receiver 需要可获取的媒体 URL,当前 P2P `MediaStream` 不满足该输入,因此不加入 Cast SDK/receiver。
- [ ] 多地区 SFU-root 调度,以及可选 TURN 兼容层的按地区、运营商可达率。
diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts
index 9ce7cce3..8f92837c 100644
--- a/src/client/sfu/publisher.ts
+++ b/src/client/sfu/publisher.ts
@@ -15,6 +15,7 @@ import {
type QualityProfile,
type VideoSenderParameterReadback,
} from "../media/quality";
+import { preferredVideoCodec } from "../webrtc/video-codec-preference";
export interface SfuConnectionConfig {
url: string;
@@ -588,6 +589,7 @@ function videoPublishOptions(
const highResolution = QUALITY_RESOLUTIONS[profile.resolution];
return {
backupCodec: false,
+ videoCodec: preferredVideoCodec(),
simulcast: true,
screenShareEncoding: {
maxBitrate: profile.maxBitrate,
diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts
index bc6cf433..473b15a5 100644
--- a/src/client/webrtc/host-peer.ts
+++ b/src/client/webrtc/host-peer.ts
@@ -15,6 +15,7 @@ import {
collectConnectionMetrics,
createStatsAccumulator,
} from "./stats";
+import { preferH264 } from "./video-codec-preference";
const MAX_PENDING_CANDIDATES = 64;
type PeerIceConfig = Pick<RTCConfiguration, "iceServers">;
@@ -113,10 +114,12 @@ export class HostPeer {
return false;
}
const audioTrack = this.stream.getAudioTracks()[0] ?? null;
- this.videoSender = this.connection.addTransceiver(videoTrack, {
+ const videoTransceiver = this.connection.addTransceiver(videoTrack, {
direction: "sendonly",
streams: [this.stream],
- }).sender;
+ });
+ preferH264(videoTransceiver);
+ this.videoSender = videoTransceiver.sender;
this.audioSender = this.connection.addTransceiver(audioTrack ?? "audio", {
direction: "sendonly",
streams: [this.stream],
diff --git a/src/client/webrtc/video-codec-preference.ts b/src/client/webrtc/video-codec-preference.ts
new file mode 100644
index 00000000..37a2355b
--- /dev/null
+++ b/src/client/webrtc/video-codec-preference.ts
@@ -0,0 +1,36 @@
+export type PreferredVideoCodec = "h264" | "vp8";
+
+function videoCodecs(): RTCRtpCodec[] {
+ if (typeof RTCRtpSender === "undefined") {
+ return [];
+ }
+ return RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
+}
+
+function isH264(codec: RTCRtpCodec): boolean {
+ return codec.mimeType.toLowerCase() === "video/h264";
+}
+
+export function preferredVideoCodec(): PreferredVideoCodec {
+ return videoCodecs().some(isH264) ? "h264" : "vp8";
+}
+
+export function preferH264(transceiver: RTCRtpTransceiver): boolean {
+ if (typeof transceiver.setCodecPreferences !== "function") {
+ return false;
+ }
+ const codecs = videoCodecs();
+ const h264 = codecs.filter(isH264);
+ if (h264.length === 0) {
+ return false;
+ }
+ try {
+ transceiver.setCodecPreferences([
+ ...h264,
+ ...codecs.filter((codec) => !isH264(codec)),
+ ]);
+ return true;
+ } catch {
+ return false;
+ }
+}
diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts
index 9501b274..d5870855 100644
--- a/tests/host-peer-tracks.test.ts
+++ b/tests/host-peer-tracks.test.ts
@@ -79,6 +79,7 @@ class FakePeerConnection {
trackOrKind: MediaStreamTrack | string;
init?: RTCRtpTransceiverInit;
}> = [];
+ readonly codecPreferenceCalls: RTCRtpCodec[][] = [];
connectionState: RTCPeerConnectionState = "new";
iceConnectionState: RTCIceConnectionState = "new";
signalingState: RTCSignalingState = "stable";
@@ -113,7 +114,12 @@ class FakePeerConnection {
);
this.senders.push(sender);
this.transceiverInputs.push({ trackOrKind, init });
- return { sender } as unknown as RTCRtpTransceiver;
+ return {
+ sender,
+ setCodecPreferences: vi.fn((codecs: RTCRtpCodec[]) => {
+ this.codecPreferenceCalls.push([...codecs]);
+ }),
+ } as unknown as RTCRtpTransceiver;
}
addEventListener(): void {}
@@ -339,6 +345,21 @@ beforeEach(() => {
FakePeerConnection.offersFailing = 0;
statsCallbacks.length = 0;
vi.stubGlobal("RTCPeerConnection", FakePeerConnection);
+ vi.stubGlobal("RTCRtpSender", {
+ getCapabilities: vi.fn(() => ({
+ codecs: [
+ { mimeType: "video/VP8", clockRate: 90_000 },
+ { mimeType: "video/rtx", clockRate: 90_000, sdpFmtpLine: "apt=96" },
+ {
+ mimeType: "video/H264",
+ clockRate: 90_000,
+ sdpFmtpLine: "packetization-mode=1;profile-level-id=42e01f",
+ },
+ { mimeType: "video/rtx", clockRate: 90_000, sdpFmtpLine: "apt=102" },
+ ],
+ headerExtensions: [],
+ })),
+ });
vi.stubGlobal("window", {
setInterval: vi.fn((callback: () => void) => {
statsCallbacks.push(callback);
@@ -357,6 +378,34 @@ afterEach(() => {
});
describe("HostPeer source replacement", () => {
+ it("prefers H.264 before the first offer while retaining codec fallback", async () => {
+ const peer = createPeer(createStream(createTrack("video", "video"), null));
+
+ await expect(peer.start()).resolves.toBe(true);
+
+ const preferences = FakePeerConnection.latest!.codecPreferenceCalls[0]!;
+ expect(preferences).toHaveLength(4);
+ expect(preferences[0]?.mimeType).toBe("video/H264");
+ expect(preferences.map(({ mimeType }) => mimeType)).toContain("video/VP8");
+ expect(
+ preferences.filter(({ mimeType }) => mimeType === "video/rtx"),
+ ).toHaveLength(2);
+ });
+
+ it("keeps browser defaults when H.264 send capability is absent", async () => {
+ vi.stubGlobal("RTCRtpSender", {
+ getCapabilities: vi.fn(() => ({
+ codecs: [{ mimeType: "video/VP8", clockRate: 90_000 }],
+ headerExtensions: [],
+ })),
+ });
+ const peer = createPeer(createStream(createTrack("video", "video"), null));
+
+ await expect(peer.start()).resolves.toBe(true);
+
+ expect(FakePeerConnection.latest!.codecPreferenceCalls).toEqual([]);
+ });
+
it("applies STUN-only ICE configuration at creation and update", () => {
const peer = createPeer(
createStream(createTrack("video", "video"), null),
diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts
index 2a108689..2b8fa7b6 100644
--- a/tests/sfu-client.test.ts
+++ b/tests/sfu-client.test.ts
@@ -333,6 +333,15 @@ beforeEach(() => {
livekit.state.connectGate = null;
livekit.state.nextSenderParameterError = null;
vi.stubGlobal("MediaStream", FakeMediaStream);
+ vi.stubGlobal("RTCRtpSender", {
+ getCapabilities: vi.fn(() => ({
+ codecs: [
+ { mimeType: "video/VP8", clockRate: 90_000 },
+ { mimeType: "video/H264", clockRate: 90_000 },
+ ],
+ headerExtensions: [],
+ })),
+ });
});
afterEach(() => {
@@ -414,6 +423,7 @@ describe("SfuPublisher", () => {
expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(1, video, {
source: Track.Source.ScreenShare,
backupCodec: false,
+ videoCodec: "h264",
simulcast: true,
screenShareEncoding: {
maxBitrate: 8_000_000,
@@ -474,6 +484,25 @@ describe("SfuPublisher", () => {
expect(room.disconnect).not.toHaveBeenCalled();
});
+ it("keeps VP8 for SFU publication when H.264 send capability is absent", async () => {
+ vi.stubGlobal("RTCRtpSender", {
+ getCapabilities: vi.fn(() => ({
+ codecs: [{ mimeType: "video/VP8", clockRate: 90_000 }],
+ headerExtensions: [],
+ })),
+ });
+ const publisher = new SfuPublisher();
+ await publisher.connect(connection);
+
+ await expect(
+ publisher.activate(stream(track("video", "video-1")), qualityProfile),
+ ).resolves.toBe(true);
+
+ expect(
+ livekit.state.rooms[0].localParticipant.publications[0].options,
+ ).toMatchObject({ videoCodec: "vp8", backupCodec: false });
+ });
+
it("fails closed when initial sender configuration is rejected", async () => {
const disconnected = vi.fn();
const publisher = new SfuPublisher({ onDisconnected: disconnected });