A platform for community discussion. Free, open, simple.
Video & Screen Sharing
Revision note (2026-06-12): this doc supersedes the original draft. Two core decisions changed after a deep read of the WebRTC layer:
- Media strategy — the original plan relied on
addTrack()+onnegotiationneededrenegotiation. The codebase has no renegotiation path (noonnegotiationneededhandler; glare handling, the ufrag restart heuristic, and answer-state guards all assume one-shot negotiation at peer setup). The new design pre-allocates video transceivers at peer creation and toggles video withreplaceTrack()— zero renegotiation, matching the pattern noise suppression already uses.- UI — a dedicated full-page room view (sidebar stays) replaces the sliding video panel. Simpler, deep-linkable, and it doubles as the subscription signal for who should receive video at all.
Also new: a per-room
video_enabledtoggle and subscription-gated sending, which raises the practical mesh ceiling well above the original 3–5 stream estimate.
Overview
Add optional camera video and screen sharing to live rooms, staying pure full-mesh P2P — no SFU, no media through the server. Joining a room remains audio-first; video is something you opt into by opening the room's dedicated page and turning on your camera.
Bandwidth is explicitly not the constraint we design for (entry-level broadband upload is now in the hundreds of Mbps). The binding constraints are:
- CPU encoder sessions — in a mesh, each
RTCPeerConnectionruns its own independent video encoder. A participant sending video to 8 watchers runs 8 simultaneous encodes. - Decode + render cost on the receiving side, especially mobile.
Both are attacked with the mesh topology's unique lever: every receiver has a dedicated sender, so we can decide per peer whether to send video at all, and at what resolution/bitrate. An SFU can't do better than simulcast here; mesh gets per-link adaptation natively.
Goals / non-goals
Goals
- Camera video and screen share in Open rooms, full mesh.
- Dedicated room page with a video grid; Discourse sidebar (and the rooms section in it) stays visible.
- Audio experience is never degraded or destabilized by the video feature.
- Per-room and site-wide opt-in.
Non-goals (for now)
- SFU integration (future, separate effort).
- Video in Stage rooms (speakers-only video is a follow-on).
- Recording, virtual backgrounds, reactions overlay.
Facts about the current architecture that shape the design
From live-webrtc.js, peer-manager.js, signaling.js:
- Negotiation is one-shot. Offers/answers happen at peer setup. Glare is resolved by user-ID ordering only during setup; answers in unexpected signaling states are dropped; a changed ICE ufrag means "peer restarted — tear down and rebuild" rather than "renegotiate".
- Mid-call media changes never renegotiate. Noise suppression swaps the
outgoing audio track with
sender.replaceTrack()(#replaceTrackOnAllPeers); role changes rebuild peers wholesale (#reconnectAllPeers). - The signal relay is payload-opaque. SDP with video m-lines flows through
POST /live/rooms/:id/signalunchanged. No server work needed for media. - Presence metadata is a free-form per-participant hash in Redis
(
ParticipantTracker), already carryingis_muted/is_deafened/idle_state, broadcast viaRoomBroadcaster.publish_participants. Video flags slot straight in. - Call state lives in the
live-webrtcservice, independent of routes — which is why audio survives navigation today, and why a video page can be a pure view over service state.
Media plan: pre-allocated transceivers + replaceTrack
Peer setup
At PeerManager.create(), alongside the existing audio addTrack calls, both
sides add a video transceiver:
pc.addTransceiver("video", { direction: "sendrecv" });
The video m-line is negotiated once, in the same initial offer/answer that already works. An idle negotiated video sender transmits nothing, so audio-only rooms pay no bandwidth or CPU cost.
Answerer alignment (JSEP gotcha). Applying a remote offer only reuses
local transceivers created via addTrack — never ones from addTransceiver,
which are reserved for the local side's next offer. Left alone, the answerer
gets a fresh recvonly transceiver for the offered video m-line and its
pre-allocated transceiver is orphaned, so the answerer can receive video but
never send it (one-directional video, surfaced as "the offerer's camera works,
the answerer's doesn't"). Between setRemoteDescription(offer) and
createAnswer(), the answerer therefore flips the associated transceiver's
direction to sendrecv (the answer then carries it — still no renegotiation)
and migrates any camera track off the orphan. Audio is unaffected because
audio tracks go through addTrack and are reused per spec.
Camera on/off
- On: acquire a separate camera stream with its own
getUserMedia({ video: ... })call — never re-acquire or touch the mic stream, so the noise-suppression chain is untouched — thentransceiver.sender.replaceTrack(cameraTrack)on each peer that should receive video (see subscription gating below). - Off:
replaceTrack(null)everywhere, stop the camera track (releases the camera light), broadcast the state change.
No renegotiation, no new signaling messages, no glare risk, ever.
Required fixes this surfaces
- Per-user stream registry.
#registerRemoteStreamkeeps one stream per user and replaces it when a different stream arrives. WithaddTransceiver, the incoming video track has no associated stream (event.streamsis empty), so the currentontrackfallback would create a video-onlyMediaStreamthat clobbers the user's audio entry. Fix: maintain oneMediaStreamper remote user andaddTrack()incoming tracks into it (or store{ audioStream, videoStream }per user). - Restart path must restore video.
PeerManager.restart()rebuilds the peer fromgetLocalStream(). Centralize an "attach current local media" step (audio tracks + video transceiver + current camera track if the peer is a subscriber) used by bothcreate()and restart. - UI must not key off
ontrackfor video. The remote video track arrives muted at initial negotiation, long before any camera turns on. Drive tile visibility from theis_video_onpresence flag (authoritative) plus the track'smute/unmuteevents (frame-accurate).
Legacy peers
Connections negotiated before this ships have audio-only SDPs with no video
m-line. If a user enables camera against such a peer, fall back to the
existing sledgehammer: the #reconnectAllPeers-style rebuild that role
changes already use. Transitional code only.
Subscription-gated sending (the mesh superpower)
Senders only attach the camera track toward peers who are watching — i.e. currently on the room's video page. Everyone else's sender keeps a null track.
- Each non-watching peer skipped saves the sender an entire encoder session, not just bandwidth.
- The signal is free: entering/leaving the room page flips a
watching_videoflag in presence metadata, which already broadcasts to the room. - Sidebar-only listeners (today's default UX) cost video senders nothing.
This is what raises the practical ceiling from the old "3–5 video streams" to "~8 publishers, bounded by who is actually watching".
Quality budget
Capture once at 720p; downscale per sender with
sender.setParameters({ encodings: [{ maxBitrate, scaleResolutionDownBy, maxFramerate }] }),
scaled by watcher count:
| Watchers | Per-peer encoding | Upload @ 8 watchers | Encodes |
|---|---|---|---|
| ≤ 3 | 720p @ ~1.2 Mbps | — | ≤ 3 |
| 4–6 | 480p @ ~700 kbps | ~3.5 Mbps | 4–6 |
| 7+ | 360p @ ~400 kbps | ~2.8 Mbps | 7+ |
Plus:
track.contentHint = "motion"(camera) /"detail"(screen share);degradationPreference: "maintain-framerate"for camera.- Frame-rate caps: 24 fps, 15 fps in large rooms; screen share 15 fps.
- Mesh gives per-link adaptation for free: each
RTCPeerConnection's bandwidth estimator adapts to that peer's downlink independently — one weak peer never degrades what others receive. (This is the mesh-native equivalent of simulcast; simulcast itself is an SFU mechanism and does not apply.) - A user-selectable quality profile (High/Medium/Low, persisted in
localStorage) remains useful as a manual override for weak sending hardware, layered under the automatic count-based ladder. live_video_max_publishers(default 8) as the honest backstop: camera buttons disable past the cap, server rejects the flag. Audio mesh keeps scaling beyond it.
UX: the room page
Navigation
- Sidebar room rows gain an "open room" affordance (and/or clicking the room name while connected navigates; click-to-join on the join button is unchanged).
- Route:
/live/r/:slug. The engine already serves JSON at/live/rooms/:id, so the page uses a distinct path. Server side: a route that renders the Ember app shell (core chat's full-page route is the reference pattern); client side: a plugin route map entry. - The Discourse sidebar — including the Live rooms section — persists automatically: the page renders in the regular content outlet.
┌──────────┬─────────────────────────────────┐
│ Sidebar │ Room: Watercooler │
│ │ ┌─────────┐ ┌─────────┐ │
│ 🎙 Rooms │ │ Alice 🎥 │ │ Bob 🎥 │ │
│ Water.. │ ├─────────┤ ├─────────┤ │
│ 👤👤👤 │ │ Carol 👤 │ │ You 🎥 │ │
│ Lounge │ └─────────┘ └─────────┘ │
│ │ │
│ (rest of │ [🎤] [🎧] [📷] [🖥️] [☎ leave] │
│ sidebar) │ │
└──────────┴─────────────────────────────────┘
- The page works for every room, video-enabled or not. Participants
without video render as avatar tiles with the existing speaking ring
(reusing
AudioMonitorstate). This makes the page a general "room view", with video as an enhancement where allowed. - Being on the page =
watching_video: truein presence; navigating away flips it off and senders drop your video feed. Audio continues regardless (service state is route-independent). - Layout: CSS grid,
auto-fit/minmax, BEM classes (.live-room-page,.live-video-tile, modifiers like--speaking,--screen-share,--self). - Active-speaker spotlight and click-to-pin: follow-on, not v1.
Video tile anatomy (unchanged from original draft)
<video>withobject-fit: cover(camera) /contain(screen share).- Bottom overlay: username, mute icon, camera/screen badge.
- Speaking indicator: border glow driven by existing speaking detection.
- Self view mirrored with
transform: scaleX(-1)(camera only). - Camera-off: avatar centered on dark background.
- Right-click: existing participant context menu (volume, mute, kick).
Controls bar
Mic toggle, deafen, camera toggle, screen share, leave. Camera and screen share buttons render only when video is effectively enabled for the room (below) and disable with a tooltip when the publisher cap is reached.
Sidebar indicators
- Camera icon next to participants with
is_video_onin the room's expanded participant rows. - A video badge on the room link itself when any participant is publishing, so browsers of the sidebar can see "there's video happening here".
Per-room video toggle
Video has a different social temperature than audio; room owners should control it.
Schema:
video_enabledboolean onlive_rooms,null: false,default: true.Effective rule:
video allowed in room = SiteSetting.live_video_enabled && room.video_enabled && room.open? # stage rooms excluded in v1Editing: checkbox in the room create/edit form (FormKit), editable by whoever can manage the room (creator/managers — existing
can_manage_live_room?guardian); also exposed in the admin room UI. Permitted inrooms_controller#room_paramsand serialized inRoomSerializer.Enforcement: server-side in the state endpoint (below) — reject
is_video_on: true/ screen-share flags when video isn't allowed for the room, mirroring howtoggle_muterejects listener unmutes in stage rooms. Client hides the buttons; the server check is the real gate.Flipping it off live: broadcast the room update (directory broadcaster already pushes room CRUD); clients with active cameras stop sending (
replaceTrack(null)) and toast.
Server changes (small by design)
Migration: add
video_enabledtolive_rooms(see above).Presence metadata keys:
is_video_on,is_screen_sharing,watching_video— same mechanism asis_muted/is_deafened.Endpoint: generalize
toggle_muteinto (or add alongside it)POST /live/rooms/:id/stateaccepting the metadata booleans, with the per-room enforcement above and the publisher-cap check (count of participants withis_video_on≥live_video_max_publishers→ 422).Site settings (
config/settings.yml):live_video_enabled: default: false client: true live_video_max_publishers: default: 8 min: 2 max: 16 client: trueOff by default — admin opt-in, independent of live.
Page route rendering the app shell at
/live/r/:slug, guarded byensure_can_join_live_room!-equivalent visibility (the room must be visible to the user; joining still requires the explicit join action).
Nothing changes in the signal relay.
Screen sharing
Nearly free once camera works — it's the same transceiver machinery:
getDisplayMedia({ video: { frameRate: { max: 15 } } }), thenreplaceTrackon the same video transceiver.- Camera XOR screen per user (one video track each way), as in the original draft — keeps transceiver count and UI simple.
- Handle the browser's native "Stop sharing" via
track.onended. - Hide the button where
getDisplayMediais unavailable (iOS Safari, most Android browsers). - Screen tiles:
object-fit: contain, no mirror, "Screen" badge; a screen share auto-pins as the spotlight tile once spotlight exists.
Phasing
- Foundation (no UI to speak of). Video transceivers at peer setup,
separate camera stream,
replaceTrackcamera toggle (temporary debug button), per-user stream registry fix, restart-path media reattachment,is_video_onpresence flag, site settings. Behindlive_video_enabled; audio path untouched. - The page.
/live/r/:slugroute, tile grid (avatar tiles for all rooms, video tiles where allowed), controls bar, sidebar indicators, per-roomvideo_enabledtoggle + form + enforcement. - Boundary pushing.
watching_videosubscription gating, watcher-count bitrate/resolution ladder, publisher cap enforcement, quality profile override, mobile tile limits. - Screen share + spotlight.
getDisplayMedia, camera XOR screen, active-speaker/pin spotlight layout.
Edge cases
- Camera enable at publisher cap: button disabled with tooltip; server 422s as the real gate (client count can be stale).
- Two users enable camera simultaneously: no interaction at all —
replaceTrackinvolves no signaling, so there is nothing to glare. (The original draft's renegotiation-storm concern dissolves with this design.) - User joins mid-video: initial negotiation includes the video m-line;
watchers receive frames as soon as publishers' senders pick them up from
the
watching_videobroadcast. No renegotiation. - Peer connection restart while camera on: restart path reattaches the camera track for subscribed peers (required fix #2).
- Room's
video_enabledswitched off mid-call: publishers stop sending, toast shown, tiles fall back to avatars. - Mobile: camera works; cap rendered tiles on small viewports (decode
cost), prefer the spotlight layout; backgrounding the browser stops the
camera — broadcast
is_video_on: falseon track end. - PTT + video: independent — PTT drives audio track
enabled, video is a separate track. - Stage rooms: excluded in v1 (
room.open?in the effective rule). Follow-on: speakers may publish, listeners are watch-only — falls out of existing role logic since listeners already have no send path. - Deafened users: deafen is audio-only; they still see video. (Matches Discord semantics.)
Future enhancements (out of scope)
- Video in Stage rooms (speakers publish, listeners watch).
- SFU integration for rooms beyond mesh limits.
- Picture-in-picture via the native PiP API (supersedes the original draft's
window.openpopout idea — far less machinery for the same need). - Active-speaker auto-spotlight, click-to-pin (phase 4 starts this).
- Virtual backgrounds (segmentation model, WebGL) — note the DTLN worklet already establishes the WASM-in-media-pipeline pattern.
- Recording, reactions overlay.