Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。
test(media): validate codec and recovery boundaries
AAA2d3e82e1f1d3
diff --git a/docs/README.md b/docs/README.mdindex f0c146b2..f8c11904 100644--- a/docs/README.md+++ b/docs/README.md@@ -17,7 +17,8 @@- [Native shared-encode sender](./research/native-shared-encode-sender.md): consolidated Draft #16/#18/#22/#23/#25/#28 ladder, unclassified product-gate failure, staged revalidation, and stop line.- [Native H.264 hardware decision](./research/native-h264-hardware-decision.md): bounded WebCodecs no-go, proved Media Foundation/NVIDIA hardware encode, the default-fmtp `42c01f` stop, and the retained interop boundary.- [Native H.264 opt-in path](./research/native-h264-opt-in-path.md): explicit H.264 sender mode, exact Pion profile registration, and one bounded Host/Viewer loopback with hardware caveats.-- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): fixed Browser VP8/no-video-hint evidence, manual quality ceilings, diagnostic-only capture/send/receive correlation, and LiveKit-owned SFU dual-representation adaptation evidence.+- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): Browser codec and `motion` evidence, manual quality ceilings, path-quality authority, and LiveKit-owned SFU representation adaptation.+- [Cross-restart room recovery](./research/cross-restart-room-recovery.md): pending lineage, authority-epoch, invitation-replay, and bounded media-rebuild evidence without reopening durable room storage.- [Browser background capture diagnostics](./research/browser-background-capture.md): browser lifecycle evidence, a privacy-bounded capture/send/receive experiment, interpretation gates, and non-goal keepalive mechanisms.- [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, live sender-ceiling readback, Opus negotiation boundaries, and remaining audible/device evidence.- [Display name and presence boundaries](./research/display-name-presence.md): browser-local storage failure behavior, Unicode normalization, control-character policy, and the Web-only presence capability boundary.diff --git a/docs/research/cross-restart-room-recovery.md b/docs/research/cross-restart-room-recovery.mdnew file mode 100644index 00000000..5531ab10--- /dev/null+++ b/docs/research/cross-restart-room-recovery.md@@ -0,0 +1,100 @@+# Cross-Restart Room Recovery++Status: Research evidence for an unresolved product decision. ADR-0002 continues+to own the accepted behavior: application restart loses every room, credential,+lease, invitation, route, and media authority.++## Current Boundary++The application process owns the complete `RoomStore`, while the Host browser+keeps its raw Host token, creation profile, and best-effort preferred room code.+After restart the old token and Viewer grant cannot authenticate; replaying the+preferred code and profile creates a new room incarnation. LiveKit rooms and the+Screener SFU resource ledger are also generation-scoped and process-resident.++LiveKit resume covers a transient signaling or network interruption while the+same server-side session exists. A full reconnect rebuilds participants, tracks,+publications, and subscriptions. The current deployment additionally deletes+managed LiveKit rooms before admitting a new application generation. Neither+WebRTC nor LiveKit can restore application authorization or route ownership from+surviving media packets.++## Minimal Recoverable Model++The smallest candidate is Host-led lineage recovery, not persistent room-state+reconstruction:++- a random stable `lineageId` identifies the Host-owned room across application+ generations;+- every successful creation or recovery allocates a fresh random+ `authorityEpoch`;+- the server signs a bounded Host recovery capsule containing the lineage,+ Host-token digest, preferred code hint, server-owned expiry, and purpose;+- the Host retains the capsule and raw Host token locally; a Viewer can never+ create, recover, reserve, or renew a room;+- recovery atomically verifies site access, capsule, Host token, expiry, code+ availability, and replayed policy/password before publishing the new room;+- session, share, connection, route, authorization, and SFU generations are+ fenced by the fresh authority epoch. Old asynchronous results fail closed.++The room code remains a best-effort locator. If another lineage already owns it,+recovery must not evict that Host; the recovering lineage receives another free+code or fails according to the future accepted contract. Client clocks, local+lease extensions, roster, graph, first-frame proof, and resource counts never+become server truth.++## Invitation And Revocation Boundary++A signature proves that the server issued a capsule; it does not prove that the+capsule is the newest one. Restoring the old Viewer-grant digest and authorization+generation from a still-valid capsule can replay a grant that the Host rotated or+revoked before the crash. Rejecting that stale checkpoint requires a server-side+latest-generation high-water mark or accepting replay until the capsule expires.+There is no stateless third option.++An old invitation must always bind to its lineage and authorization generation.+It cannot fall through to code-only admission, enter a different lineage that+reused the same four-digit code, or create a missing room. A private room whose+local password material is unavailable must fail closed rather than become open.++The lowest-risk first phase would recover only Host lineage, control, lease+ceiling, and a free preferred code, then issue a new invitation. Stable old+invitations are a separate decision that must settle revocation high-water and+code relocation first.++## Media Boundary++Control recovery can be automatic while the Host tab and capture track remain+alive, but a page or browser restart still requires a new user gesture for screen+capture. Every recovered route must close or fence old P2P/SFU senders, rebuild+the graph and resource ledger, and recommit on a newly decoded frame. Media may+briefly survive a process crash, but it has no current authority and cannot be+advertised as uninterrupted service.++True multi-node LiveKit continuity requires its distributed Redis deployment and+draining model. That is a different infrastructure and availability decision,+not a consequence of Host lineage recovery.++## Decision Needed++1. Is Host lineage/control recovery sufficient, with new invitations and a+ bounded media gap?+2. If old invitations must survive, does the product accept bounded stale replay+ or one small durable latest-authorization-generation owner?+3. May a recovered lineage move to another room code when its hint is occupied,+ and should a lineage-bound invitation resolve that new code?+4. Is a same-browser bearer recovery capsule an acceptable ownership boundary?+5. Are application and LiveKit restart targets both bounded rebuild, or is+ LiveKit high availability a separately funded requirement?++## Primary Sources++- [W3C Capability URLs](https://www.w3.org/TR/capability-urls/)+- [RFC 6750 bearer-token threats](https://www.rfc-editor.org/rfc/rfc6750.html#section-5.2)+- [RFC 7519 expiration and token IDs](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1)+- [RFC 9246 signed-URI replay boundary](https://www.rfc-editor.org/rfc/rfc9246.html)+- [Kubernetes names and UIDs](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/)+- [The Chubby lock service](https://storage.googleapis.com/gweb-research2023-media/pubtools/4444.pdf)+- [WebRTC ICE restart](https://www.w3.org/TR/webrtc/#dom-rtcpeerconnection-restartice)+- [LiveKit reconnect behavior](https://docs.livekit.io/intro/basics/connect/#network-changes-and-reconnection)+- [LiveKit distributed deployment and draining](https://docs.livekit.io/transport/self-hosting/distributed/)diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.mdindex 14e693b3..86208920 100644--- a/docs/research/realtime-quality-adaptation.md+++ b/docs/research/realtime-quality-adaptation.md@@ -241,27 +241,45 @@ The Browser candidates therefore have narrower potential than their names imply:| AV1 | Modern hardware can provide the strongest compression candidate. | Chrome 151 and 153 keep Windows WebRTC AV1 hardware encode disabled by default, so codec support does not provide a usable hardware contract. || H.265 | Supported hardware can provide efficient H.264-class low-CPU encode at lower bitrate. | It is an advanced codec not supported by every Browser client; LiveKit compatibility requires regression or a backup publication, and the current cross-Browser relay matrix is absent. |-No candidate currently provides a product-wide Browser improvement with VP8's-coverage. Browser media therefore remains VP8 while one bounded preflight is-open. It reproduces the accepted `motion` capture semantics and compares VP8-with H.264 constrained baseline on exact Chrome 153. The default browser path is-the product evidence; forcing AMD software BRC is a diagnostic arm only.--The gate covers Host-to-Viewer and Browser-relay-to-child paths with one and two-outbound senders. Codec capability is insufficient: each endpoint must prove the-negotiated codec/profile, encoder and decoder implementation when exposed,-power-efficiency evidence, encoded and decoded cadence, bitrate, resolution,-encode/decode time, quality-limitation reason, and process CPU/GPU video-engine-attribution. This determines whether a P2P connection can select H.264 for an-efficient sender and receiver and otherwise retain VP8, including the relay's-fresh encode cost.--A single SFU publication cannot supply subscriber-specific H.264 and VP8 without-another encoded publication. A VP8-first viewing path followed by background-H.264 confirmation must therefore separately prove its transition behavior and-the temporary dual-encode or republish cost before it can be accepted as-seamless fallback. The spike adds no VP9, AV1, or H.265 runtime probing, codec-selector, product integration, or deployment.+No candidate yet provides a product-wide Browser improvement with VP8's+coverage, so Browser media remains VP8. A bounded preflight nevertheless proved+that an actual sender can distinguish the previously bad and a newly good H.264+path without naming a GPU or trusting capability advertisement. All samples used+one real tab `getDisplayMedia()` source at 1904x928@30, `contentHint = "motion"`,+a 5 Mbps ceiling, five seconds of settling, and a ten-second measurement window.++| Browser and path | Actual implementation | Source / encoded / decoded cadence | Result |+| --- | --- | --- | --- |+| Chrome 151 direct H.264 | AMD Media Foundation hardware encoder; D3D11 decoder | 30.0 / 12.5 / 12.4 fps | Failed despite `powerEfficientEncoder = true`; encoded/source ratio 0.42. |+| Chrome 153 direct VP8 | `libvpx` software encoder and decoder | about 29.9 / 29.9 / 29.9 fps | Control sustained source cadence. |+| Chrome 153 direct H.264 | NVIDIA Media Foundation hardware encoder; D3D11 decoder | 29.9 / 29.9 / 30.0 fps | Full-cadence constrained baseline, ratio 1.00. |+| Chrome 153 LiveKit 1.13.5 H.264 | Simulcast adapter with two NVIDIA MFT encoders; D3D11 decoder | 29.5 / 29.1 / 29.1 fps | One `backupCodec: false` HIGH+LOW publication delivered the HIGH stream, ratio 0.99. |++Exact Chrome 153 also sustained about 30 fps on two simultaneous direct H.264+senders. A Host-to-Browser-relay-to-child chain sustained 29.6-29.7 fps at full+source resolution, and both encode stages remained NVIDIA MFT while both decode+stages remained D3D11. The dual-direct sample was still inside native bandwidth+ramp-up and therefore proves encoder cadence, not steady resolution quality.+Process identities changed during startup, so those whole-browser CPU deltas are+not retained as a codec cost claim.++`powerEfficientEncoder` alone cannot be the gate because the cadence-failing AMD+path also reported true. The portable evidence is the actual negotiated H.264+profile and implementation, plus identity-stable source-frame and encoded-frame+progress; the successful cohorts kept their encoded/source deficit bounded while+the bad path dropped frames continuously. The measurement window and acceptable+deficit still require multi-device and real-game calibration before product use.++For direct peers, native SDP negotiation can order H.264 before VP8 in one offer+and choose the first codec supported by both endpoints; it does not require a+VP8 connection and a parallel H.264 connection. A research candidate is one+share-scoped Host sender preflight and one received-source-scoped Browser-relay+preflight: a proved sender prefers H.264 for future edges, otherwise it retains+VP8, and already active edges are not churned. A single SFU publication still+cannot provide per-subscriber H.264 and VP8 without a second encoded publication.+LiveKit backup codec is therefore not accepted, and an H.264 SFU publication+requires a separate decision on the supported Browser decode floor. No VP9,+AV1, H.265, product codec selector, or deployment is accepted by this evidence.HEVC, AV1, custom WebCodecs pipelines, and application packetization do notreplace the browser WebRTC sender without a new capture, RTP/RTCP, feedback,@@ -352,6 +370,25 @@ distort comparison. It necessarily permits a temporarily worse candidate and asecond transition on restore; that behavior, the freeze SLO, inconclusive result,and permission to create or prioritize SFU resources remain product decisions.+The current quality-evidence wire is not yet eligible for that shadow authority.+WebRTC defines freeze and pause from rendered frames, while the current payload+has decoded progress but no rendered-frame or standard pause evidence. It also+caps one recovered `freezeDuration` delta to one report window even though the+cumulative counter can legitimately jump by several windows after rendering+resumes. Presentation eligibility must bind visibility, page lifecycle, local+playback/autoplay, Host pause, current frame proof, and an identity epoch; the+first sample after any epoch change establishes a baseline only.++The minimal active state machine remains `observe -> qualified -> queued ->+trial prepare -> first-frame commit -> probation -> keep | restore once ->+cooldown`. It uses the existing reservation and room-serial child operation,+permits no bounded-gap quality trial, and yields to availability work. Relay+ingress evidence may reparent that relay while retaining its subtree, but one or+several child reports cannot infer that the relay is globally bad. Phase one has+no quality-driven relay-wide abdication. Freeze entry/exit, probation evidence,+improvement margin, cooldown, and room disruption budget must be calibrated from+real annotated sessions; paper and library defaults are not Screener thresholds.+## Current Verification Gaps- Real games under competing CPU/GPU load at 720p30, 1080p30, and 1080p60.diff --git a/docs/todo.md b/docs/todo.mdindex 5b360298..5686b8a5 100644--- a/docs/todo.md+++ b/docs/todo.md@@ -7,21 +7,12 @@ later topic is not implementation authority by itself.## Now-1. **Browser codec preflight.** Keep the product on VP8 while an isolated Chrome- 153 experiment compares VP8 with H.264 constrained baseline under the current- `motion` capture semantics. Measure Host send, Viewer decode, Browser relay,- one/two outbound senders, and a VP8-first background transition using actual- codec/profile, implementation and power-efficiency evidence when exposed,- cadence, encode/decode time, and CPU/GPU attribution. The default browser path- is product evidence; an AMD forced-SW-BRC arm is diagnostic only. Do not add- product codec selection, VP9/AV1/H.265 probing, a custom encoder, or deploy a- codec change before this gate is accepted.-2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices for+1. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices forautoplay gesture, foreground/background audio, foreground video recovery,lock/page reclamation, rotation, Wi-Fi/cellular migration, and assigned-relaysurvival or controller recovery. Web does not promise background video orrelay execution after OS suspension.-3. **Finish real-network route and media acceptance.** Exercise direct peer,+2. **Finish real-network route and media acceptance.** Exercise direct peer,Browser relay, SFU, relay-ingress recovery with subtree retention,disconnect/capacity drain, Pause/Resume, source replacement, screen-audiocontinuity, and real-game A/V sync across representative IPv4/IPv6,@@ -66,8 +57,9 @@ later topic is not implementation authority by itself.| Item | Decision || --- | --- || Repository and distribution license | Decide before public release or package distribution. GPL/AGPL implementations remain research-only until then. |-| Cross-restart room identity and media continuity | Decide whether stable invitations and uninterrupted control justify durable or signed room lineage plus exact route/resource recovery. Preferred code/password replay alone remains a new room incarnation. |-| Quality-driven route exploration | Confirm the trigger SLO, whether one post-commit trial plus one restore may temporarily worsen playback, how inconclusive probation settles, and whether quality work may prioritize an existing SFU publication or create a new one. No active route change is authorized until then. |+| Adaptive Browser codec gate | Decide whether to accept one share/source-scoped H.264 sender preflight plus native H.264-to-VP8 negotiation. Chrome 153 passed direct, dual-sender, Browser-relay, and LiveKit paths on the measured NVIDIA/AMD machine while Chrome 151's selected AMD MFT failed. Probe calibration, real-game/60 fps, heterogeneous endpoints, and the single-SFU-publication codec policy remain unaccepted. |+| Cross-restart room identity and media continuity | Decide whether the target stops at Host lineage/control recovery with a fresh authority epoch or also preserves old invitations. A signed Host capsule can restore lineage but cannot combine stable old grants with strong cross-restart rotate/revoke without a small durable latest-generation truth. Media can be rebuilt with a bounded gap, not promised uninterrupted. |+| Quality-driven route exploration | First repair rendered/pause evidence, long recovered-freeze deltas, and presentation eligibility. Then confirm the freeze SLO, whether one post-commit trial plus one best-effort restore may temporarily worsen playback, how inconclusive probation settles, and whether quality work may prioritize an existing SFU publication. No active route change or relay-wide quality abdication is authorized. |## Evidence Boundariesdiff --git a/scripts/browser-codec-preflight.ts b/scripts/browser-codec-preflight.tsnew file mode 100644index 00000000..6cc2c342--- /dev/null+++ b/scripts/browser-codec-preflight.ts@@ -0,0 +1,384 @@+import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";+import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";+import { createServer, type Server } from "node:http";+import { tmpdir } from "node:os";+import { join, resolve } from "node:path";++import { AccessToken, TrackSource } from "livekit-server-sdk";++import {+ CdpConnection,+ cleanupRun,+ createPage,+ reservePort,+ waitForVersion,+ type PageHandle,+} from "./native-one-viewer-gate";++const WIDTH = 1_920;+const HEIGHT = 1_080;+const FRAME_RATE = 30;+const MAX_BITRATE = 5_000_000;+const SETTLE_MS = 5_000;+const SAMPLE_MS = 10_000;++interface RuntimeResult<T> {+ result: { value?: T };+ exceptionDetails?: {+ text?: string;+ exception?: { description?: string };+ };+}++interface ProcessInfo {+ type: string;+ id: number;+ cpuTime: number;+}++interface ProcessSnapshot {+ processInfo: ProcessInfo[];+}++async function evaluate<T>(+ cdp: CdpConnection,+ page: PageHandle,+ expression: string,+ timeoutMs: number,+): Promise<T> {+ const value = await cdp.call<RuntimeResult<T>>(+ "Runtime.evaluate",+ {+ expression,+ awaitPromise: true,+ returnByValue: true,+ userGesture: true,+ },+ page.sessionId,+ Date.now() + timeoutMs,+ );+ if (value.exceptionDetails) {+ throw new Error(+ value.exceptionDetails.exception?.description ??+ value.exceptionDetails.text ??+ "Browser evaluation failed",+ );+ }+ return value.result.value as T;+}++function sourceHtml(): string {+ return `<!doctype html><title>Codec Preflight Source</title>+<style>html,body{margin:0;overflow:hidden;background:#000}canvas{display:block;width:100vw;height:100vh}</style>+<canvas width="${WIDTH}" height="${HEIGHT}"></canvas><script>+const c=document.querySelector('canvas'),x=c.getContext('2d',{alpha:false});let f=0;+function draw(){f+=1;for(let r=0;r<14;r+=1)for(let q=0;q<24;q+=1){+x.fillStyle='hsl('+((f*29+r*71+q*43)%360)+' 80% '+(35+(f+r+q)%30)+'%)';+x.fillRect(q*80,r*78,80,78)}x.fillStyle='#fff';x.fillRect(f*13%1920,0,12,1080);+x.fillStyle='#000';x.font='60px monospace';x.fillText(String(f).padStart(8,'0'),24,72)}+draw();setInterval(draw,1000/${FRAME_RATE});</script>`;+}++async function startServer(port: number): Promise<Server> {+ const livekitBundle = await readFile(+ resolve("node_modules/livekit-client/dist/livekit-client.umd.js"),+ );+ const server = createServer((request, response) => {+ const path = new URL(request.url ?? "/", `http://127.0.0.1:${port}`).pathname;+ if (path === "/livekit.js") {+ response.setHeader("content-type", "text/javascript; charset=utf-8");+ response.end(livekitBundle);+ return;+ }+ response.setHeader("content-type", "text/html; charset=utf-8");+ response.end(path === "/source"+ ? sourceHtml()+ : "<!doctype html><title>Codec Preflight</title><script src='/livekit.js'></script><video autoplay muted playsinline style='width:960px'></video>");+ });+ await new Promise<void>((resolveListen, rejectListen) => {+ server.once("error", rejectListen);+ server.listen(port, "127.0.0.1", resolveListen);+ });+ return server;+}++function browserProbe(): string {+ return `(() => {+ const WIDTH=${WIDTH},HEIGHT=${HEIGHT},FRAME_RATE=${FRAME_RATE},MAX_BITRATE=${MAX_BITRATE};+ const SETTLE_MS=${SETTLE_MS},SAMPLE_MS=${SAMPLE_MS};let stream=null,track=null;+ const delay=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));+ async function waitFor(predicate,timeout,label){const deadline=performance.now()+timeout;+ while(performance.now()<deadline){if(await predicate())return;await delay(50)}+ throw new Error('Timed out waiting for '+label)}+ function preferences(codec){const all=RTCRtpSender.getCapabilities('video')?.codecs||[];+ const mime=codec==='h264'?'video/h264':'video/vp8';const primary=all.filter((item)=>item.mimeType.toLowerCase()===mime);+ if(!primary.length)throw new Error(mime+' unavailable');+ const repair=new Set(['video/rtx','video/red','video/ulpfec','video/flexfec-03']);+ return [...primary,...all.filter((item)=>repair.has(item.mimeType.toLowerCase()))]}+ async function gathered(pc){if(pc.iceGatheringState==='complete')return;+ await new Promise((resolve)=>{const changed=()=>{if(pc.iceGatheringState!=='complete')return;+ pc.removeEventListener('icegatheringstatechange',changed);resolve()};pc.addEventListener('icegatheringstatechange',changed)})}+ async function negotiate(a,b){await a.setLocalDescription(await a.createOffer());await gathered(a);+ await b.setRemoteDescription(a.localDescription);await b.setLocalDescription(await b.createAnswer());+ await gathered(b);await a.setRemoteDescription(b.localDescription)}+ function statsValue(item){if(!item)return null;const keys=['id','timestamp','bytesSent','bytesReceived',+ 'framesEncoded','framesDecoded','framesDropped','framesPerSecond','frameWidth','frameHeight','totalEncodeTime',+ 'totalDecodeTime','qualityLimitationReason','qualityLimitationDurations','encoderImplementation',+ 'decoderImplementation','powerEfficientEncoder','powerEfficientDecoder','frames'];const value={};+ for(const key of keys)value[key]=item[key]??null;return value}+ async function snapshot(sender,receiver,rendered){const [sendReport,receiveReport]=await Promise.all([sender.getStats(),receiver.getStats()]);+ let outbound=null,inbound=null;sendReport.forEach((item)=>{if(item.type==='outbound-rtp'&&item.kind==='video'&&!item.isRemote)outbound=item});+ receiveReport.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video'&&!item.isRemote)inbound=item});+ const sendCodec=outbound?.codecId?sendReport.get(outbound.codecId):null;+ const receiveCodec=inbound?.codecId?receiveReport.get(inbound.codecId):null;+ const mediaSource=outbound?.mediaSourceId?sendReport.get(outbound.mediaSourceId):null;+ const codec=(item)=>item?{mimeType:item.mimeType||null,sdpFmtpLine:item.sdpFmtpLine||null}:null;+ return {rendered:rendered(),sender:statsValue(outbound),receiver:statsValue(inbound),+ mediaSource:statsValue(mediaSource),senderCodec:codec(sendCodec),receiverCodec:codec(receiveCodec)}}+ function delta(after,before,key){const a=after?.[key],b=before?.[key];return Number.isFinite(a)&&Number.isFinite(b)&&a>=b?a-b:null}+ function summarize(before,after,sampleMs){const encoded=delta(after.sender,before.sender,'framesEncoded');+ const decoded=delta(after.receiver,before.receiver,'framesDecoded');const bytes=delta(after.sender,before.sender,'bytesSent');+ const sourceFrames=delta(after.mediaSource,before.mediaSource,'frames');+ const encodeTime=delta(after.sender,before.sender,'totalEncodeTime');const decodeTime=delta(after.receiver,before.receiver,'totalDecodeTime');+ return {senderCodec:after.senderCodec,receiverCodec:after.receiverCodec,sender:after.sender,receiver:after.receiver,+ measured:{sampleMs,encodedFps:encoded===null?null:encoded*1000/sampleMs,+ decodedFps:decoded===null?null:decoded*1000/sampleMs,bitrateKbps:bytes===null?null:bytes*8/sampleMs,+ encodeMsPerFrame:encodeTime!==null&&encoded>0?encodeTime*1000/encoded:null,+ decodeMsPerFrame:decodeTime!==null&&decoded>0?decodeTime*1000/decoded:null,+ sourceFps:sourceFrames===null?after.mediaSource?.framesPerSecond:sourceFrames*1000/sampleMs,+ encodedSourceRatio:sourceFrames>0&&encoded!==null?encoded/sourceFrames:null,+ renderedFrames:delta(after,before,'rendered')}}}+ async function setup(){stream=await navigator.mediaDevices.getDisplayMedia({video:{+ width:{ideal:WIDTH,max:WIDTH},height:{ideal:HEIGHT,max:HEIGHT},frameRate:{ideal:FRAME_RATE,max:FRAME_RATE}},audio:false});+ track=stream.getVideoTracks()[0];if(!track)throw new Error('No display video track');track.contentHint='motion';+ return {settings:track.getSettings(),constraints:track.getConstraints(),contentHint:track.contentHint,label:track.label}}+ function capabilities(){const simple=(codec)=>({mimeType:codec.mimeType,sdpFmtpLine:codec.sdpFmtpLine||null});+ return {sender:(RTCRtpSender.getCapabilities('video')?.codecs||[]).map(simple),+ receiver:(RTCRtpReceiver.getCapabilities('video')?.codecs||[]).map(simple)}}+ async function edge(codec,sourceTrack){+ const sendPc=new RTCPeerConnection({iceServers:[]}),receivePc=new RTCPeerConnection({iceServers:[]});+ const transceiver=sendPc.addTransceiver(sourceTrack,{direction:'sendonly'});transceiver.setCodecPreferences(preferences(codec));+ let receiver=null,remoteTrack=null;receivePc.addEventListener('track',(event)=>{receiver=event.receiver;remoteTrack=event.track});+ await negotiate(sendPc,receivePc);await waitFor(()=>sendPc.connectionState==='connected'&&receivePc.connectionState==='connected',10000,'connection');+ await waitFor(()=>receiver&&remoteTrack,5000,'remote track');const video=document.createElement('video');let rendered=0;+ video.autoplay=true;video.muted=true;video.playsInline=true;document.body.append(video);+ video.srcObject=new MediaStream([remoteTrack]);if(video.requestVideoFrameCallback){const frame=()=>{rendered+=1;video.requestVideoFrameCallback(frame)};video.requestVideoFrameCallback(frame)}+ await video.play();const parameters=transceiver.sender.getParameters();if(!parameters.encodings.length)parameters.encodings=[{}];+ parameters.encodings[0].maxBitrate=MAX_BITRATE;parameters.encodings[0].maxFramerate=FRAME_RATE;+ parameters.encodings[0].scaleResolutionDownBy=1;parameters.degradationPreference='balanced';await transceiver.sender.setParameters(parameters);+ await waitFor(async()=>{const report=await receiver.getStats();let frames=0;report.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video')frames=Math.max(frames,item.framesDecoded||0)});return frames>0&&rendered>0},10000,'decoded frame');+ return {sendPc,receivePc,sender:transceiver.sender,receiver,remoteTrack,rendered:()=>rendered}}+ async function run(codec,count,topology){if(!track)throw new Error('Capture not initialized');const edges=[];+ try{edges.push(await edge(codec,track));+ if(topology==='relay')edges.push(await edge(codec,edges[0].remoteTrack));+ else for(let index=1;index<count;index+=1)edges.push(await edge(codec,track));await delay(SETTLE_MS);+ const before=await Promise.all(edges.map((item)=>snapshot(item.sender,item.receiver,item.rendered))),started=performance.now();+ await delay(SAMPLE_MS);const sampleMs=performance.now()-started;+ const after=await Promise.all(edges.map((item)=>snapshot(item.sender,item.receiver,item.rendered)));+ return edges.map((_,index)=>summarize(before[index],after[index],sampleMs))}+ finally{for(const item of edges){item.sendPc.close();item.receivePc.close()}await delay(300)}}+ async function runLivekit(codec,url,publisherToken,subscriberToken){if(!track)throw new Error('Capture not initialized');+ const sdk=globalThis.LivekitClient,publisher=new sdk.Room({dynacast:true}),subscriber=new sdk.Room();let remoteTrack=null,rendered=0;+ subscriber.on(sdk.RoomEvent.TrackSubscribed,(nextTrack)=>{if(nextTrack.kind==='video')remoteTrack=nextTrack});+ try{await publisher.connect(url,publisherToken,{autoSubscribe:false,rtcConfig:{iceServers:[]}});+ await subscriber.connect(url,subscriberToken,{rtcConfig:{iceServers:[]}});+ const publication=await publisher.localParticipant.publishTrack(track,{source:sdk.Track.Source.ScreenShare,+ videoCodec:codec,backupCodec:false,screenShareEncoding:{maxBitrate:MAX_BITRATE,maxFramerate:FRAME_RATE},+ degradationPreference:'balanced'});+ await waitFor(()=>publication.videoTrack?.sender&&remoteTrack,15000,'LiveKit publication');+ const video=document.createElement('video');video.autoplay=true;video.muted=true;video.playsInline=true;document.body.append(video);+ video.srcObject=new MediaStream([remoteTrack.mediaStreamTrack]);if(video.requestVideoFrameCallback){const frame=()=>{rendered+=1;video.requestVideoFrameCallback(frame)};video.requestVideoFrameCallback(frame)}+ await video.play();const receiver={getStats:async()=>{const report=await remoteTrack.getRTCStatsReport();if(!report)throw new Error('LiveKit receiver stats unavailable');return report}};+ await waitFor(async()=>{const report=await receiver.getStats();let frames=0;report?.forEach((item)=>{if(item.type==='inbound-rtp'&&item.kind==='video')frames=Math.max(frames,item.framesDecoded||0)});return frames>0&&rendered>0},15000,'LiveKit decoded frame');+ await delay(SETTLE_MS);const sender=publication.videoTrack.sender;+ const before=await snapshot(sender,receiver,()=>rendered),started=performance.now();await delay(SAMPLE_MS);+ const sampleMs=performance.now()-started,after=await snapshot(sender,receiver,()=>rendered);+ return {...summarize(before,after,sampleMs),senderParameters:sender.getParameters()}}+ finally{await Promise.allSettled([publisher.disconnect(false),subscriber.disconnect(false)]);await delay(300)}}+ function stop(){stream?.getTracks().forEach((item)=>item.stop())}+ Object.defineProperty(globalThis,'__CODEC_PREFLIGHT__',{value:{setup,capabilities,run,runLivekit,stop}});+ })()`;+}++async function processSnapshot(cdp: CdpConnection): Promise<ProcessSnapshot> {+ return cdp.call<ProcessSnapshot>(+ "SystemInfo.getProcessInfo",+ {},+ undefined,+ Date.now() + 5_000,+ );+}++function processCpu(+ before: ProcessSnapshot,+ after: ProcessSnapshot,+ wallMs: number,+): Record<string, unknown> {+ const baseline = new Map(+ before.processInfo.map((item) => [`${item.type}:${item.id}`, item.cpuTime]),+ );+ let cpuSeconds = 0;+ let stableProcesses = 0;+ for (const item of after.processInfo) {+ const previous = baseline.get(`${item.type}:${item.id}`);+ if (previous === undefined || item.cpuTime < previous) continue;+ cpuSeconds += item.cpuTime - previous;+ stableProcesses += 1;+ }+ const wallSeconds = wallMs / 1_000;+ return {+ wallSeconds,+ cpuSeconds,+ logicalCoreEquivalent: cpuSeconds / wallSeconds,+ stableProcesses,+ processIdentityChanged:+ stableProcesses !== before.processInfo.length ||+ stableProcesses !== after.processInfo.length,+ };+}++async function main(): Promise<void> {+ const chromePath = process.env.CHROME_PATH?.trim();+ if (!chromePath) throw new Error("CHROME_PATH must point to Chrome");+ const codec = process.env.CODEC_PREFLIGHT_CODEC?.trim().toLowerCase();+ if (codec !== "vp8" && codec !== "h264") {+ throw new Error("CODEC_PREFLIGHT_CODEC must be vp8 or h264");+ }+ const edges = Number(process.env.CODEC_PREFLIGHT_EDGES ?? "1");+ if (edges !== 1 && edges !== 2) {+ throw new Error("CODEC_PREFLIGHT_EDGES must be 1 or 2");+ }+ const topology = process.env.CODEC_PREFLIGHT_TOPOLOGY?.trim() || "direct";+ if (topology !== "direct" && topology !== "relay" && topology !== "livekit") {+ throw new Error("CODEC_PREFLIGHT_TOPOLOGY must be direct, relay, or livekit");+ }+ const [debugPort, pagePort] = await Promise.all([reservePort(), reservePort()]);+ const profile = await mkdtemp(join(tmpdir(), "screener-codec-preflight-"));+ let chrome: ChildProcessWithoutNullStreams | null = null;+ let cdp: CdpConnection | null = null;+ let server: Server | null = null;+ try {+ server = await startServer(pagePort);+ chrome = spawn(chromePath, [+ `--remote-debugging-port=${debugPort}`,+ `--user-data-dir=${profile}`,+ "--no-first-run",+ "--no-default-browser-check",+ "--disable-extensions",+ "--disable-background-timer-throttling",+ "--disable-backgrounding-occluded-windows",+ "--disable-renderer-backgrounding",+ "--autoplay-policy=no-user-gesture-required",+ "--auto-select-tab-capture-source-by-title=Codec Preflight Source",+ "--window-size=1920,1080",+ "about:blank",+ ], { stdio: "pipe", windowsHide: true });+ chrome.stdout.resume();+ chrome.stderr.resume();+ const version = await waitForVersion(debugPort, chrome);+ cdp = await CdpConnection.connect(version.webSocketDebuggerUrl, Date.now() + 5_000);+ const gpuInfo = await cdp.call<unknown>("SystemInfo.getInfo", {}, undefined, Date.now() + 5_000);+ await createPage(cdp, `http://127.0.0.1:${pagePort}/source`, undefined, true);+ const page = await createPage(cdp, `http://127.0.0.1:${pagePort}/test`, undefined, true);+ await evaluate<void>(cdp, page, browserProbe(), 5_000);+ const capabilities = await evaluate<unknown>(+ cdp,+ page,+ "globalThis.__CODEC_PREFLIGHT__.capabilities()",+ 5_000,+ );+ const capture = await evaluate<unknown>(+ cdp,+ page,+ "globalThis.__CODEC_PREFLIGHT__.setup()",+ 15_000,+ );+ const before = await processSnapshot(cdp);+ const started = performance.now();+ let media: unknown;+ if (topology === "livekit") {+ const livekitUrl = process.env.CODEC_PREFLIGHT_LIVEKIT_URL?.trim();+ const apiKey = process.env.CODEC_PREFLIGHT_LIVEKIT_KEY?.trim();+ const apiSecret = process.env.CODEC_PREFLIGHT_LIVEKIT_SECRET?.trim();+ if (!livekitUrl || !apiKey || !apiSecret) {+ throw new Error("LiveKit preflight credentials are missing");+ }+ const room = `codec-preflight-${Date.now()}`;+ const token = async (identity: string, publish: boolean) => {+ const access = new AccessToken(apiKey, apiSecret, { identity, ttl: 60 });+ access.addGrant({+ roomJoin: true,+ room,+ canPublish: publish,+ canPublishSources: publish ? [TrackSource.SCREEN_SHARE] : undefined,+ canSubscribe: !publish,+ canPublishData: false,+ });+ return access.toJwt();+ };+ const [publisherToken, subscriberToken] = await Promise.all([+ token("host", true),+ token("viewer", false),+ ]);+ media = await evaluate<unknown>(+ cdp,+ page,+ `globalThis.__CODEC_PREFLIGHT__.runLivekit(${JSON.stringify(codec)}, ${JSON.stringify(livekitUrl)}, ${JSON.stringify(publisherToken)}, ${JSON.stringify(subscriberToken)})`,+ 60_000,+ );+ } else {+ media = await evaluate<unknown>(+ cdp,+ page,+ `globalThis.__CODEC_PREFLIGHT__.run(${JSON.stringify(codec)}, ${edges}, ${JSON.stringify(topology)})`,+ 45_000,+ );+ }+ const wallMs = performance.now() - started;+ const after = await processSnapshot(cdp);+ await evaluate<void>(cdp, page, "globalThis.__CODEC_PREFLIGHT__.stop()", 5_000);+ const report = {+ schemaVersion: 1,+ browser: version.Browser,+ configuration: {+ width: WIDTH,+ height: HEIGHT,+ frameRate: FRAME_RATE,+ maxBitrate: MAX_BITRATE,+ contentHint: "motion",+ settleMs: SETTLE_MS,+ sampleMs: SAMPLE_MS,+ edges,+ topology,+ },+ gpuInfo,+ capabilities,+ capture,+ codec,+ media,+ cpu: processCpu(before, after, wallMs),+ };+ const outputDirectory = resolve("benchmark-results");+ await mkdir(outputDirectory, { recursive: true });+ const output = join(+ outputDirectory,+ `browser-codec-preflight-${codec}-${topology}-${edges}.json`,+ );+ await writeFile(output, `${JSON.stringify(report, null, 2)}\n`, "utf8");+ console.log(output);+ } finally {+ await cleanupRun({+ cdp,+ native: null,+ chrome,+ server: null,+ profile,+ ports: [debugPort],+ });+ if (server) {+ await new Promise<void>((resolveClose) => server!.close(() => resolveClose()));+ }+ }+}++await main();diff --git a/tsconfig.tools.json b/tsconfig.tools.jsonindex 5f0a3b5a..7c85324c 100644--- a/tsconfig.tools.json+++ b/tsconfig.tools.json@@ -21,6 +21,7 @@"vitest.config.ts","scripts/native-one-viewer-gate.ts","scripts/native-one-viewer-ledger.ts",+ "scripts/browser-codec-preflight.ts","tests"]}