Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

test(media): validate codec and recovery boundaries

AAA2d3e82e1f1d3
diff --git a/docs/README.md b/docs/README.md
index f0c146b2..f8c11904 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -17,7 +17,8 @@
- [Native shared-encode sender](./research/native-shared-encode-sender.md): consolidated Draft #16/#18/#22/#23/#25/#28 ladder, unclassified product-gate failure, staged revalidation, and stop line.
- [Native H.264 hardware decision](./research/native-h264-hardware-decision.md): bounded WebCodecs no-go, proved Media Foundation/NVIDIA hardware encode, the default-fmtp `42c01f` stop, and the retained interop boundary.
- [Native H.264 opt-in path](./research/native-h264-opt-in-path.md): explicit H.264 sender mode, exact Pion profile registration, and one bounded Host/Viewer loopback with hardware caveats.
-- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): fixed Browser VP8/no-video-hint evidence, manual quality ceilings, diagnostic-only capture/send/receive correlation, and LiveKit-owned SFU dual-representation adaptation evidence.
+- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): Browser codec and `motion` evidence, manual quality ceilings, path-quality authority, and LiveKit-owned SFU representation adaptation.
+- [Cross-restart room recovery](./research/cross-restart-room-recovery.md): pending lineage, authority-epoch, invitation-replay, and bounded media-rebuild evidence without reopening durable room storage.
- [Browser background capture diagnostics](./research/browser-background-capture.md): browser lifecycle evidence, a privacy-bounded capture/send/receive experiment, interpretation gates, and non-goal keepalive mechanisms.
- [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, live sender-ceiling readback, Opus negotiation boundaries, and remaining audible/device evidence.
- [Display name and presence boundaries](./research/display-name-presence.md): browser-local storage failure behavior, Unicode normalization, control-character policy, and the Web-only presence capability boundary.
diff --git a/docs/research/cross-restart-room-recovery.md b/docs/research/cross-restart-room-recovery.md
new file mode 100644
index 00000000..5531ab10
--- /dev/null
+++ b/docs/research/cross-restart-room-recovery.md
@@ -0,0 +1,100 @@
+# Cross-Restart Room Recovery
+
+Status: Research evidence for an unresolved product decision. ADR-0002 continues
+to own the accepted behavior: application restart loses every room, credential,
+lease, invitation, route, and media authority.
+
+## Current Boundary
+
+The application process owns the complete `RoomStore`, while the Host browser
+keeps its raw Host token, creation profile, and best-effort preferred room code.
+After restart the old token and Viewer grant cannot authenticate; replaying the
+preferred code and profile creates a new room incarnation. LiveKit rooms and the
+Screener SFU resource ledger are also generation-scoped and process-resident.
+
+LiveKit resume covers a transient signaling or network interruption while the
+same server-side session exists. A full reconnect rebuilds participants, tracks,
+publications, and subscriptions. The current deployment additionally deletes
+managed LiveKit rooms before admitting a new application generation. Neither
+WebRTC nor LiveKit can restore application authorization or route ownership from
+surviving media packets.
+
+## Minimal Recoverable Model
+
+The smallest candidate is Host-led lineage recovery, not persistent room-state
+reconstruction:
+
+- a random stable `lineageId` identifies the Host-owned room across application
+ generations;
+- every successful creation or recovery allocates a fresh random
+ `authorityEpoch`;
+- the server signs a bounded Host recovery capsule containing the lineage,
+ Host-token digest, preferred code hint, server-owned expiry, and purpose;
+- the Host retains the capsule and raw Host token locally; a Viewer can never
+ create, recover, reserve, or renew a room;
+- recovery atomically verifies site access, capsule, Host token, expiry, code
+ availability, and replayed policy/password before publishing the new room;
+- session, share, connection, route, authorization, and SFU generations are
+ fenced by the fresh authority epoch. Old asynchronous results fail closed.
+
+The room code remains a best-effort locator. If another lineage already owns it,
+recovery must not evict that Host; the recovering lineage receives another free
+code or fails according to the future accepted contract. Client clocks, local
+lease extensions, roster, graph, first-frame proof, and resource counts never
+become server truth.
+
+## Invitation And Revocation Boundary
+
+A signature proves that the server issued a capsule; it does not prove that the
+capsule is the newest one. Restoring the old Viewer-grant digest and authorization
+generation from a still-valid capsule can replay a grant that the Host rotated or
+revoked before the crash. Rejecting that stale checkpoint requires a server-side
+latest-generation high-water mark or accepting replay until the capsule expires.
+There is no stateless third option.
+
+An old invitation must always bind to its lineage and authorization generation.
+It cannot fall through to code-only admission, enter a different lineage that
+reused the same four-digit code, or create a missing room. A private room whose
+local password material is unavailable must fail closed rather than become open.
+
+The lowest-risk first phase would recover only Host lineage, control, lease
+ceiling, and a free preferred code, then issue a new invitation. Stable old
+invitations are a separate decision that must settle revocation high-water and
+code relocation first.
+
+## Media Boundary
+
+Control recovery can be automatic while the Host tab and capture track remain
+alive, but a page or browser restart still requires a new user gesture for screen
+capture. Every recovered route must close or fence old P2P/SFU senders, rebuild
+the graph and resource ledger, and recommit on a newly decoded frame. Media may
+briefly survive a process crash, but it has no current authority and cannot be
+advertised as uninterrupted service.
+
+True multi-node LiveKit continuity requires its distributed Redis deployment and
+draining model. That is a different infrastructure and availability decision,
+not a consequence of Host lineage recovery.
+
+## Decision Needed
+
+1. Is Host lineage/control recovery sufficient, with new invitations and a
+ bounded media gap?
+2. If old invitations must survive, does the product accept bounded stale replay
+ or one small durable latest-authorization-generation owner?
+3. May a recovered lineage move to another room code when its hint is occupied,
+ and should a lineage-bound invitation resolve that new code?
+4. Is a same-browser bearer recovery capsule an acceptable ownership boundary?
+5. Are application and LiveKit restart targets both bounded rebuild, or is
+ LiveKit high availability a separately funded requirement?
+
+## Primary Sources
+
+- [W3C Capability URLs](https://www.w3.org/TR/capability-urls/)
+- [RFC 6750 bearer-token threats](https://www.rfc-editor.org/rfc/rfc6750.html#section-5.2)
+- [RFC 7519 expiration and token IDs](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1)
+- [RFC 9246 signed-URI replay boundary](https://www.rfc-editor.org/rfc/rfc9246.html)
+- [Kubernetes names and UIDs](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/)
+- [The Chubby lock service](https://storage.googleapis.com/gweb-research2023-media/pubtools/4444.pdf)
+- [WebRTC ICE restart](https://www.w3.org/TR/webrtc/#dom-rtcpeerconnection-restartice)
+- [LiveKit reconnect behavior](https://docs.livekit.io/intro/basics/connect/#network-changes-and-reconnection)
+- [LiveKit distributed deployment and draining](https://docs.livekit.io/transport/self-hosting/distributed/)
diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md
index 14e693b3..86208920 100644
--- a/docs/research/realtime-quality-adaptation.md
+++ b/docs/research/realtime-quality-adaptation.md
@@ -241,27 +241,45 @@ The Browser candidates therefore have narrower potential than their names imply:
| AV1 | Modern hardware can provide the strongest compression candidate. | Chrome 151 and 153 keep Windows WebRTC AV1 hardware encode disabled by default, so codec support does not provide a usable hardware contract. |
| H.265 | Supported hardware can provide efficient H.264-class low-CPU encode at lower bitrate. | It is an advanced codec not supported by every Browser client; LiveKit compatibility requires regression or a backup publication, and the current cross-Browser relay matrix is absent. |
-No candidate currently provides a product-wide Browser improvement with VP8's
-coverage. Browser media therefore remains VP8 while one bounded preflight is
-open. It reproduces the accepted `motion` capture semantics and compares VP8
-with H.264 constrained baseline on exact Chrome 153. The default browser path is
-the product evidence; forcing AMD software BRC is a diagnostic arm only.
-
-The gate covers Host-to-Viewer and Browser-relay-to-child paths with one and two
-outbound senders. Codec capability is insufficient: each endpoint must prove the
-negotiated codec/profile, encoder and decoder implementation when exposed,
-power-efficiency evidence, encoded and decoded cadence, bitrate, resolution,
-encode/decode time, quality-limitation reason, and process CPU/GPU video-engine
-attribution. This determines whether a P2P connection can select H.264 for an
-efficient sender and receiver and otherwise retain VP8, including the relay's
-fresh encode cost.
-
-A single SFU publication cannot supply subscriber-specific H.264 and VP8 without
-another encoded publication. A VP8-first viewing path followed by background
-H.264 confirmation must therefore separately prove its transition behavior and
-the temporary dual-encode or republish cost before it can be accepted as
-seamless fallback. The spike adds no VP9, AV1, or H.265 runtime probing, codec
-selector, product integration, or deployment.
+No candidate yet provides a product-wide Browser improvement with VP8's
+coverage, so Browser media remains VP8. A bounded preflight nevertheless proved
+that an actual sender can distinguish the previously bad and a newly good H.264
+path without naming a GPU or trusting capability advertisement. All samples used
+one real tab `getDisplayMedia()` source at 1904x928@30, `contentHint = "motion"`,
+a 5 Mbps ceiling, five seconds of settling, and a ten-second measurement window.
+
+| Browser and path | Actual implementation | Source / encoded / decoded cadence | Result |
+| --- | --- | --- | --- |
+| Chrome 151 direct H.264 | AMD Media Foundation hardware encoder; D3D11 decoder | 30.0 / 12.5 / 12.4 fps | Failed despite `powerEfficientEncoder = true`; encoded/source ratio 0.42. |
+| Chrome 153 direct VP8 | `libvpx` software encoder and decoder | about 29.9 / 29.9 / 29.9 fps | Control sustained source cadence. |
+| Chrome 153 direct H.264 | NVIDIA Media Foundation hardware encoder; D3D11 decoder | 29.9 / 29.9 / 30.0 fps | Full-cadence constrained baseline, ratio 1.00. |
+| Chrome 153 LiveKit 1.13.5 H.264 | Simulcast adapter with two NVIDIA MFT encoders; D3D11 decoder | 29.5 / 29.1 / 29.1 fps | One `backupCodec: false` HIGH+LOW publication delivered the HIGH stream, ratio 0.99. |
+
+Exact Chrome 153 also sustained about 30 fps on two simultaneous direct H.264
+senders. A Host-to-Browser-relay-to-child chain sustained 29.6-29.7 fps at full
+source resolution, and both encode stages remained NVIDIA MFT while both decode
+stages remained D3D11. The dual-direct sample was still inside native bandwidth
+ramp-up and therefore proves encoder cadence, not steady resolution quality.
+Process identities changed during startup, so those whole-browser CPU deltas are
+not retained as a codec cost claim.
+
+`powerEfficientEncoder` alone cannot be the gate because the cadence-failing AMD
+path also reported true. The portable evidence is the actual negotiated H.264
+profile and implementation, plus identity-stable source-frame and encoded-frame
+progress; the successful cohorts kept their encoded/source deficit bounded while
+the bad path dropped frames continuously. The measurement window and acceptable
+deficit still require multi-device and real-game calibration before product use.
+
+For direct peers, native SDP negotiation can order H.264 before VP8 in one offer
+and choose the first codec supported by both endpoints; it does not require a
+VP8 connection and a parallel H.264 connection. A research candidate is one
+share-scoped Host sender preflight and one received-source-scoped Browser-relay
+preflight: a proved sender prefers H.264 for future edges, otherwise it retains
+VP8, and already active edges are not churned. A single SFU publication still
+cannot provide per-subscriber H.264 and VP8 without a second encoded publication.
+LiveKit backup codec is therefore not accepted, and an H.264 SFU publication
+requires a separate decision on the supported Browser decode floor. No VP9,
+AV1, H.265, product codec selector, or deployment is accepted by this evidence.
HEVC, AV1, custom WebCodecs pipelines, and application packetization do not
replace the browser WebRTC sender without a new capture, RTP/RTCP, feedback,
@@ -352,6 +370,25 @@ distort comparison. It necessarily permits a temporarily worse candidate and a
second transition on restore; that behavior, the freeze SLO, inconclusive result,
and permission to create or prioritize SFU resources remain product decisions.
+The current quality-evidence wire is not yet eligible for that shadow authority.
+WebRTC defines freeze and pause from rendered frames, while the current payload
+has decoded progress but no rendered-frame or standard pause evidence. It also
+caps one recovered `freezeDuration` delta to one report window even though the
+cumulative counter can legitimately jump by several windows after rendering
+resumes. Presentation eligibility must bind visibility, page lifecycle, local
+playback/autoplay, Host pause, current frame proof, and an identity epoch; the
+first sample after any epoch change establishes a baseline only.
+
+The minimal active state machine remains `observe -> qualified -> queued ->
+trial prepare -> first-frame commit -> probation -> keep | restore once ->
+cooldown`. It uses the existing reservation and room-serial child operation,
+permits no bounded-gap quality trial, and yields to availability work. Relay
+ingress evidence may reparent that relay while retaining its subtree, but one or
+several child reports cannot infer that the relay is globally bad. Phase one has
+no quality-driven relay-wide abdication. Freeze entry/exit, probation evidence,
+improvement margin, cooldown, and room disruption budget must be calibrated from
+real annotated sessions; paper and library defaults are not Screener thresholds.
+
## Current Verification Gaps
- Real games under competing CPU/GPU load at 720p30, 1080p30, and 1080p60.
diff --git a/docs/todo.md b/docs/todo.md
index 5b360298..5686b8a5 100644
--- a/docs/todo.md
+++ b/docs/todo.md
@@ -7,21 +7,12 @@ later topic is not implementation authority by itself.
## Now
-1. **Browser codec preflight.** Keep the product on VP8 while an isolated Chrome
- 153 experiment compares VP8 with H.264 constrained baseline under the current
- `motion` capture semantics. Measure Host send, Viewer decode, Browser relay,
- one/two outbound senders, and a VP8-first background transition using actual
- codec/profile, implementation and power-efficiency evidence when exposed,
- cadence, encode/decode time, and CPU/GPU attribution. The default browser path
- is product evidence; an AMD forced-SW-BRC arm is diagnostic only. Do not add
- product codec selection, VP9/AV1/H.265 probing, a custom encoder, or deploy a
- codec change before this gate is accepted.
-2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices for
+1. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices for
autoplay gesture, foreground/background audio, foreground video recovery,
lock/page reclamation, rotation, Wi-Fi/cellular migration, and assigned-relay
survival or controller recovery. Web does not promise background video or
relay execution after OS suspension.
-3. **Finish real-network route and media acceptance.** Exercise direct peer,
+2. **Finish real-network route and media acceptance.** Exercise direct peer,
Browser relay, SFU, relay-ingress recovery with subtree retention,
disconnect/capacity drain, Pause/Resume, source replacement, screen-audio
continuity, and real-game A/V sync across representative IPv4/IPv6,
@@ -66,8 +57,9 @@ later topic is not implementation authority by itself.
| Item | Decision |
| --- | --- |
| Repository and distribution license | Decide before public release or package distribution. GPL/AGPL implementations remain research-only until then. |
-| Cross-restart room identity and media continuity | Decide whether stable invitations and uninterrupted control justify durable or signed room lineage plus exact route/resource recovery. Preferred code/password replay alone remains a new room incarnation. |
-| Quality-driven route exploration | Confirm the trigger SLO, whether one post-commit trial plus one restore may temporarily worsen playback, how inconclusive probation settles, and whether quality work may prioritize an existing SFU publication or create a new one. No active route change is authorized until then. |
+| Adaptive Browser codec gate | Decide whether to accept one share/source-scoped H.264 sender preflight plus native H.264-to-VP8 negotiation. Chrome 153 passed direct, dual-sender, Browser-relay, and LiveKit paths on the measured NVIDIA/AMD machine while Chrome 151's selected AMD MFT failed. Probe calibration, real-game/60 fps, heterogeneous endpoints, and the single-SFU-publication codec policy remain unaccepted. |
+| Cross-restart room identity and media continuity | Decide whether the target stops at Host lineage/control recovery with a fresh authority epoch or also preserves old invitations. A signed Host capsule can restore lineage but cannot combine stable old grants with strong cross-restart rotate/revoke without a small durable latest-generation truth. Media can be rebuilt with a bounded gap, not promised uninterrupted. |
+| Quality-driven route exploration | First repair rendered/pause evidence, long recovered-freeze deltas, and presentation eligibility. Then confirm the freeze SLO, whether one post-commit trial plus one best-effort restore may temporarily worsen playback, how inconclusive probation settles, and whether quality work may prioritize an existing SFU publication. No active route change or relay-wide quality abdication is authorized. |
## Evidence Boundaries
diff --git a/scripts/browser-codec-preflight.ts b/scripts/browser-codec-preflight.ts
new file mode 100644
index 00000000..6cc2c342
--- /dev/null
+++ b/scripts/browser-codec-preflight.ts
@@ -0,0 +1,384 @@
+import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
+import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
+import { createServer, type Server } from "node:http";
+import { tmpdir } from "node:os";
+import { join, resolve } from "node:path";
+
+import { AccessToken, TrackSource } from "livekit-server-sdk";
+
+import {
+ CdpConnection,
+ cleanupRun,
+ createPage,
+ reservePort,
+ waitForVersion,
+ type PageHandle,
+} from "./native-one-viewer-gate";
+
+const WIDTH = 1_920;
+const HEIGHT = 1_080;
+const FRAME_RATE = 30;
+const MAX_BITRATE = 5_000_000;
+const SETTLE_MS = 5_000;
+const SAMPLE_MS = 10_000;
+
+interface RuntimeResult<T> {
+ result: { value?: T };
+ exceptionDetails?: {
+ text?: string;
+ exception?: { description?: string };
+ };
+}
+
+interface ProcessInfo {
+ type: string;
+ id: number;
+ cpuTime: number;
+}
+
+interface ProcessSnapshot {
+ processInfo: ProcessInfo[];
+}
+
+async function evaluate<T>(
+ cdp: CdpConnection,
+ page: PageHandle,
+ expression: string,
+ timeoutMs: number,
+): Promise<T> {
+ const value = await cdp.call<RuntimeResult<T>>(
+ "Runtime.evaluate",
+ {
+ expression,
+ awaitPromise: true,
+ returnByValue: true,
+ userGesture: true,
+ },
+ page.sessionId,
+ Date.now() + timeoutMs,
+ );
+ if (value.exceptionDetails) {
+ throw new Error(
+ value.exceptionDetails.exception?.description ??
+ value.exceptionDetails.text ??
+ "Browser evaluation failed",
+ );
+ }
+ return value.result.value as T;
+}
+
+function sourceHtml(): string {
+ return `<!doctype html><title>Codec Preflight Source</title>
+<style>html,body{margin:0;overflow:hidden;background:#000}canvas{display:block;width:100vw;height:100vh}</style>
+<canvas width="${WIDTH}" height="${HEIGHT}"></canvas><script>
+const c=document.querySelector('canvas'),x=c.getContext('2d',{alpha:false});let f=0;
+function draw(){f+=1;for(let r=0;r<14;r+=1)for(let q=0;q<24;q+=1){
+x.fillStyle='hsl('+((f*29+r*71+q*43)%360)+' 80% '+(35+(f+r+q)%30)+'%)';
+x.fillRect(q*80,r*78,80,78)}x.fillStyle='#fff';x.fillRect(f*13%1920,0,12,1080);
+x.fillStyle='#000';x.font='60px monospace';x.fillText(String(f).padStart(8,'0'),24,72)}
+draw();setInterval(draw,1000/${FRAME_RATE});</script>`;
+}
+
+async function startServer(port: number): Promise<Server> {
+ const livekitBundle = await readFile(
+ resolve("node_modules/livekit-client/dist/livekit-client.umd.js"),
+ );
+ const server = createServer((request, response) => {
+ const path = new URL(request.url ?? "/", `http://127.0.0.1:${port}`).pathname;
+ if (path === "/livekit.js") {
+ response.setHeader("content-type", "text/javascript; charset=utf-8");
+ response.end(livekitBundle);
+ return;
+ }
+ response.setHeader("content-type", "text/html; charset=utf-8");
+ response.end(path === "/source"
+ ? sourceHtml()
+ : "<!doctype html><title>Codec Preflight</title><script src='/livekit.js'></script><video autoplay muted playsinline style='width:960px'></video>");
+ });
+ await new Promise<void>((resolveListen, rejectListen) => {
+ server.once("error", rejectListen);
+ server.listen(port, "127.0.0.1", resolveListen);
+ });
+ return server;
+}
+
+function browserProbe(): string {
+ return `(() => {
+ const WIDTH=${WIDTH},HEIGHT=${HEIGHT},FRAME_RATE=${FRAME_RATE},MAX_BITRATE=${MAX_BITRATE};
+ const SETTLE_MS=${SETTLE_MS},SAMPLE_MS=${SAMPLE_MS};let stream=null,track=null;
+ const delay=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));
+ async function waitFor(predicate,timeout,label){const deadline=performance.now()+timeout;
+ while(performance.now()<deadline){if(await predicate())return;await delay(50)}
+ throw new Error('Timed out waiting for '+label)}
+ function preferences(codec){const all=RTCRtpSender.getCapabilities('video')?.codecs||[];
+ const mime=codec==='h264'?'video/h264':'video/vp8';const primary=all.filter((item)=>item.mimeType.toLowerCase()===mime);
+ if(!primary.length)throw new Error(mime+' unavailable');
+ const repair=new Set(['video/rtx','video/red','video/ulpfec','video/flexfec-03']);
+ return [...primary,...all.filter((item)=>repair.has(item.mimeType.toLowerCase()))]}
+ async function gathered(pc){if(pc.iceGatheringState==='complete')return;
+ await new Promise((resolve)=>{const changed=()=>{if(pc.iceGatheringState!=='complete')return;
+ pc.removeEventListener('icegatheringstatechange',changed);resolve()};pc.addEventListener('icegatheringstatechange',changed)})}
+ async function negotiate(a,b){await a.setLocalDescription(await a.createOffer());await gathered(a);
+ await b.setRemoteDescription(a.localDescription);await b.setLocalDescription(await b.createAnswer());
+ await gathered(b);await a.setRemoteDescription(b.localDescription)}
+ function statsValue(item){if(!item)return null;const keys=['id','timestamp','bytesSent','bytesReceived',
+ 'framesEncoded','framesDecoded','framesDropped','framesPerSecond','frameWidth','frameHeight','totalEncodeTime',
+ 'totalDecodeTime','qualityLimitationReason','qualityLimitationDurations','encoderImplementation',
+ 'decoderImplementation','powerEfficientEncoder','powerEfficientDecoder','frames'];const value={};
+ for(const key of keys)value[key]=item[key]??null;return value}
+ async function snapshot(sender,receiver,rendered){const [sendReport,receiveReport]=await Promise.all([sender.getStats(),receiver.getStats()]);
+ let outbound=null,inbound=null;sendReport.forEach((item)=>{if(item.type==='outbound-rtp'&&item.kind==='video'&&!item.isRemote)outbound=item});
+ receiveReport.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video'&&!item.isRemote)inbound=item});
+ const sendCodec=outbound?.codecId?sendReport.get(outbound.codecId):null;
+ const receiveCodec=inbound?.codecId?receiveReport.get(inbound.codecId):null;
+ const mediaSource=outbound?.mediaSourceId?sendReport.get(outbound.mediaSourceId):null;
+ const codec=(item)=>item?{mimeType:item.mimeType||null,sdpFmtpLine:item.sdpFmtpLine||null}:null;
+ return {rendered:rendered(),sender:statsValue(outbound),receiver:statsValue(inbound),
+ mediaSource:statsValue(mediaSource),senderCodec:codec(sendCodec),receiverCodec:codec(receiveCodec)}}
+ function delta(after,before,key){const a=after?.[key],b=before?.[key];return Number.isFinite(a)&&Number.isFinite(b)&&a>=b?a-b:null}
+ function summarize(before,after,sampleMs){const encoded=delta(after.sender,before.sender,'framesEncoded');
+ const decoded=delta(after.receiver,before.receiver,'framesDecoded');const bytes=delta(after.sender,before.sender,'bytesSent');
+ const sourceFrames=delta(after.mediaSource,before.mediaSource,'frames');
+ const encodeTime=delta(after.sender,before.sender,'totalEncodeTime');const decodeTime=delta(after.receiver,before.receiver,'totalDecodeTime');
+ return {senderCodec:after.senderCodec,receiverCodec:after.receiverCodec,sender:after.sender,receiver:after.receiver,
+ measured:{sampleMs,encodedFps:encoded===null?null:encoded*1000/sampleMs,
+ decodedFps:decoded===null?null:decoded*1000/sampleMs,bitrateKbps:bytes===null?null:bytes*8/sampleMs,
+ encodeMsPerFrame:encodeTime!==null&&encoded>0?encodeTime*1000/encoded:null,
+ decodeMsPerFrame:decodeTime!==null&&decoded>0?decodeTime*1000/decoded:null,
+ sourceFps:sourceFrames===null?after.mediaSource?.framesPerSecond:sourceFrames*1000/sampleMs,
+ encodedSourceRatio:sourceFrames>0&&encoded!==null?encoded/sourceFrames:null,
+ renderedFrames:delta(after,before,'rendered')}}}
+ async function setup(){stream=await navigator.mediaDevices.getDisplayMedia({video:{
+ width:{ideal:WIDTH,max:WIDTH},height:{ideal:HEIGHT,max:HEIGHT},frameRate:{ideal:FRAME_RATE,max:FRAME_RATE}},audio:false});
+ track=stream.getVideoTracks()[0];if(!track)throw new Error('No display video track');track.contentHint='motion';
+ return {settings:track.getSettings(),constraints:track.getConstraints(),contentHint:track.contentHint,label:track.label}}
+ function capabilities(){const simple=(codec)=>({mimeType:codec.mimeType,sdpFmtpLine:codec.sdpFmtpLine||null});
+ return {sender:(RTCRtpSender.getCapabilities('video')?.codecs||[]).map(simple),
+ receiver:(RTCRtpReceiver.getCapabilities('video')?.codecs||[]).map(simple)}}
+ async function edge(codec,sourceTrack){
+ const sendPc=new RTCPeerConnection({iceServers:[]}),receivePc=new RTCPeerConnection({iceServers:[]});
+ const transceiver=sendPc.addTransceiver(sourceTrack,{direction:'sendonly'});transceiver.setCodecPreferences(preferences(codec));
+ let receiver=null,remoteTrack=null;receivePc.addEventListener('track',(event)=>{receiver=event.receiver;remoteTrack=event.track});
+ await negotiate(sendPc,receivePc);await waitFor(()=>sendPc.connectionState==='connected'&&receivePc.connectionState==='connected',10000,'connection');
+ await waitFor(()=>receiver&&remoteTrack,5000,'remote track');const video=document.createElement('video');let rendered=0;
+ video.autoplay=true;video.muted=true;video.playsInline=true;document.body.append(video);
+ video.srcObject=new MediaStream([remoteTrack]);if(video.requestVideoFrameCallback){const frame=()=>{rendered+=1;video.requestVideoFrameCallback(frame)};video.requestVideoFrameCallback(frame)}
+ await video.play();const parameters=transceiver.sender.getParameters();if(!parameters.encodings.length)parameters.encodings=[{}];
+ parameters.encodings[0].maxBitrate=MAX_BITRATE;parameters.encodings[0].maxFramerate=FRAME_RATE;
+ parameters.encodings[0].scaleResolutionDownBy=1;parameters.degradationPreference='balanced';await transceiver.sender.setParameters(parameters);
+ await waitFor(async()=>{const report=await receiver.getStats();let frames=0;report.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video')frames=Math.max(frames,item.framesDecoded||0)});return frames>0&&rendered>0},10000,'decoded frame');
+ return {sendPc,receivePc,sender:transceiver.sender,receiver,remoteTrack,rendered:()=>rendered}}
+ async function run(codec,count,topology){if(!track)throw new Error('Capture not initialized');const edges=[];
+ try{edges.push(await edge(codec,track));
+ if(topology==='relay')edges.push(await edge(codec,edges[0].remoteTrack));
+ else for(let index=1;index<count;index+=1)edges.push(await edge(codec,track));await delay(SETTLE_MS);
+ const before=await Promise.all(edges.map((item)=>snapshot(item.sender,item.receiver,item.rendered))),started=performance.now();
+ await delay(SAMPLE_MS);const sampleMs=performance.now()-started;
+ const after=await Promise.all(edges.map((item)=>snapshot(item.sender,item.receiver,item.rendered)));
+ return edges.map((_,index)=>summarize(before[index],after[index],sampleMs))}
+ finally{for(const item of edges){item.sendPc.close();item.receivePc.close()}await delay(300)}}
+ async function runLivekit(codec,url,publisherToken,subscriberToken){if(!track)throw new Error('Capture not initialized');
+ const sdk=globalThis.LivekitClient,publisher=new sdk.Room({dynacast:true}),subscriber=new sdk.Room();let remoteTrack=null,rendered=0;
+ subscriber.on(sdk.RoomEvent.TrackSubscribed,(nextTrack)=>{if(nextTrack.kind==='video')remoteTrack=nextTrack});
+ try{await publisher.connect(url,publisherToken,{autoSubscribe:false,rtcConfig:{iceServers:[]}});
+ await subscriber.connect(url,subscriberToken,{rtcConfig:{iceServers:[]}});
+ const publication=await publisher.localParticipant.publishTrack(track,{source:sdk.Track.Source.ScreenShare,
+ videoCodec:codec,backupCodec:false,screenShareEncoding:{maxBitrate:MAX_BITRATE,maxFramerate:FRAME_RATE},
+ degradationPreference:'balanced'});
+ await waitFor(()=>publication.videoTrack?.sender&&remoteTrack,15000,'LiveKit publication');
+ const video=document.createElement('video');video.autoplay=true;video.muted=true;video.playsInline=true;document.body.append(video);
+ video.srcObject=new MediaStream([remoteTrack.mediaStreamTrack]);if(video.requestVideoFrameCallback){const frame=()=>{rendered+=1;video.requestVideoFrameCallback(frame)};video.requestVideoFrameCallback(frame)}
+ await video.play();const receiver={getStats:async()=>{const report=await remoteTrack.getRTCStatsReport();if(!report)throw new Error('LiveKit receiver stats unavailable');return report}};
+ await waitFor(async()=>{const report=await receiver.getStats();let frames=0;report?.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video')frames=Math.max(frames,item.framesDecoded||0)});return frames>0&&rendered>0},15000,'LiveKit decoded frame');
+ await delay(SETTLE_MS);const sender=publication.videoTrack.sender;
+ const before=await snapshot(sender,receiver,()=>rendered),started=performance.now();await delay(SAMPLE_MS);
+ const sampleMs=performance.now()-started,after=await snapshot(sender,receiver,()=>rendered);
+ return {...summarize(before,after,sampleMs),senderParameters:sender.getParameters()}}
+ finally{await Promise.allSettled([publisher.disconnect(false),subscriber.disconnect(false)]);await delay(300)}}
+ function stop(){stream?.getTracks().forEach((item)=>item.stop())}
+ Object.defineProperty(globalThis,'__CODEC_PREFLIGHT__',{value:{setup,capabilities,run,runLivekit,stop}});
+ })()`;
+}
+
+async function processSnapshot(cdp: CdpConnection): Promise<ProcessSnapshot> {
+ return cdp.call<ProcessSnapshot>(
+ "SystemInfo.getProcessInfo",
+ {},
+ undefined,
+ Date.now() + 5_000,
+ );
+}
+
+function processCpu(
+ before: ProcessSnapshot,
+ after: ProcessSnapshot,
+ wallMs: number,
+): Record<string, unknown> {
+ const baseline = new Map(
+ before.processInfo.map((item) => [`${item.type}:${item.id}`, item.cpuTime]),
+ );
+ let cpuSeconds = 0;
+ let stableProcesses = 0;
+ for (const item of after.processInfo) {
+ const previous = baseline.get(`${item.type}:${item.id}`);
+ if (previous === undefined || item.cpuTime < previous) continue;
+ cpuSeconds += item.cpuTime - previous;
+ stableProcesses += 1;
+ }
+ const wallSeconds = wallMs / 1_000;
+ return {
+ wallSeconds,
+ cpuSeconds,
+ logicalCoreEquivalent: cpuSeconds / wallSeconds,
+ stableProcesses,
+ processIdentityChanged:
+ stableProcesses !== before.processInfo.length ||
+ stableProcesses !== after.processInfo.length,
+ };
+}
+
+async function main(): Promise<void> {
+ const chromePath = process.env.CHROME_PATH?.trim();
+ if (!chromePath) throw new Error("CHROME_PATH must point to Chrome");
+ const codec = process.env.CODEC_PREFLIGHT_CODEC?.trim().toLowerCase();
+ if (codec !== "vp8" && codec !== "h264") {
+ throw new Error("CODEC_PREFLIGHT_CODEC must be vp8 or h264");
+ }
+ const edges = Number(process.env.CODEC_PREFLIGHT_EDGES ?? "1");
+ if (edges !== 1 && edges !== 2) {
+ throw new Error("CODEC_PREFLIGHT_EDGES must be 1 or 2");
+ }
+ const topology = process.env.CODEC_PREFLIGHT_TOPOLOGY?.trim() || "direct";
+ if (topology !== "direct" && topology !== "relay" && topology !== "livekit") {
+ throw new Error("CODEC_PREFLIGHT_TOPOLOGY must be direct, relay, or livekit");
+ }
+ const [debugPort, pagePort] = await Promise.all([reservePort(), reservePort()]);
+ const profile = await mkdtemp(join(tmpdir(), "screener-codec-preflight-"));
+ let chrome: ChildProcessWithoutNullStreams | null = null;
+ let cdp: CdpConnection | null = null;
+ let server: Server | null = null;
+ try {
+ server = await startServer(pagePort);
+ chrome = spawn(chromePath, [
+ `--remote-debugging-port=${debugPort}`,
+ `--user-data-dir=${profile}`,
+ "--no-first-run",
+ "--no-default-browser-check",
+ "--disable-extensions",
+ "--disable-background-timer-throttling",
+ "--disable-backgrounding-occluded-windows",
+ "--disable-renderer-backgrounding",
+ "--autoplay-policy=no-user-gesture-required",
+ "--auto-select-tab-capture-source-by-title=Codec Preflight Source",
+ "--window-size=1920,1080",
+ "about:blank",
+ ], { stdio: "pipe", windowsHide: true });
+ chrome.stdout.resume();
+ chrome.stderr.resume();
+ const version = await waitForVersion(debugPort, chrome);
+ cdp = await CdpConnection.connect(version.webSocketDebuggerUrl, Date.now() + 5_000);
+ const gpuInfo = await cdp.call<unknown>("SystemInfo.getInfo", {}, undefined, Date.now() + 5_000);
+ await createPage(cdp, `http://127.0.0.1:${pagePort}/source`, undefined, true);
+ const page = await createPage(cdp, `http://127.0.0.1:${pagePort}/test`, undefined, true);
+ await evaluate<void>(cdp, page, browserProbe(), 5_000);
+ const capabilities = await evaluate<unknown>(
+ cdp,
+ page,
+ "globalThis.__CODEC_PREFLIGHT__.capabilities()",
+ 5_000,
+ );
+ const capture = await evaluate<unknown>(
+ cdp,
+ page,
+ "globalThis.__CODEC_PREFLIGHT__.setup()",
+ 15_000,
+ );
+ const before = await processSnapshot(cdp);
+ const started = performance.now();
+ let media: unknown;
+ if (topology === "livekit") {
+ const livekitUrl = process.env.CODEC_PREFLIGHT_LIVEKIT_URL?.trim();
+ const apiKey = process.env.CODEC_PREFLIGHT_LIVEKIT_KEY?.trim();
+ const apiSecret = process.env.CODEC_PREFLIGHT_LIVEKIT_SECRET?.trim();
+ if (!livekitUrl || !apiKey || !apiSecret) {
+ throw new Error("LiveKit preflight credentials are missing");
+ }
+ const room = `codec-preflight-${Date.now()}`;
+ const token = async (identity: string, publish: boolean) => {
+ const access = new AccessToken(apiKey, apiSecret, { identity, ttl: 60 });
+ access.addGrant({
+ roomJoin: true,
+ room,
+ canPublish: publish,
+ canPublishSources: publish ? [TrackSource.SCREEN_SHARE] : undefined,
+ canSubscribe: !publish,
+ canPublishData: false,
+ });
+ return access.toJwt();
+ };
+ const [publisherToken, subscriberToken] = await Promise.all([
+ token("host", true),
+ token("viewer", false),
+ ]);
+ media = await evaluate<unknown>(
+ cdp,
+ page,
+ `globalThis.__CODEC_PREFLIGHT__.runLivekit(${JSON.stringify(codec)}, ${JSON.stringify(livekitUrl)}, ${JSON.stringify(publisherToken)}, ${JSON.stringify(subscriberToken)})`,
+ 60_000,
+ );
+ } else {
+ media = await evaluate<unknown>(
+ cdp,
+ page,
+ `globalThis.__CODEC_PREFLIGHT__.run(${JSON.stringify(codec)}, ${edges}, ${JSON.stringify(topology)})`,
+ 45_000,
+ );
+ }
+ const wallMs = performance.now() - started;
+ const after = await processSnapshot(cdp);
+ await evaluate<void>(cdp, page, "globalThis.__CODEC_PREFLIGHT__.stop()", 5_000);
+ const report = {
+ schemaVersion: 1,
+ browser: version.Browser,
+ configuration: {
+ width: WIDTH,
+ height: HEIGHT,
+ frameRate: FRAME_RATE,
+ maxBitrate: MAX_BITRATE,
+ contentHint: "motion",
+ settleMs: SETTLE_MS,
+ sampleMs: SAMPLE_MS,
+ edges,
+ topology,
+ },
+ gpuInfo,
+ capabilities,
+ capture,
+ codec,
+ media,
+ cpu: processCpu(before, after, wallMs),
+ };
+ const outputDirectory = resolve("benchmark-results");
+ await mkdir(outputDirectory, { recursive: true });
+ const output = join(
+ outputDirectory,
+ `browser-codec-preflight-${codec}-${topology}-${edges}.json`,
+ );
+ await writeFile(output, `${JSON.stringify(report, null, 2)}\n`, "utf8");
+ console.log(output);
+ } finally {
+ await cleanupRun({
+ cdp,
+ native: null,
+ chrome,
+ server: null,
+ profile,
+ ports: [debugPort],
+ });
+ if (server) {
+ await new Promise<void>((resolveClose) => server!.close(() => resolveClose()));
+ }
+ }
+}
+
+await main();
diff --git a/tsconfig.tools.json b/tsconfig.tools.json
index 5f0a3b5a..7c85324c 100644
--- a/tsconfig.tools.json
+++ b/tsconfig.tools.json
@@ -21,6 +21,7 @@
"vitest.config.ts",
"scripts/native-one-viewer-gate.ts",
"scripts/native-one-viewer-ledger.ts",
+ "scripts/browser-codec-preflight.ts",
"tests"
]
}