Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

fix(media): improve sharing recovery and mobile picture controls

AAA72ca6f2bc97d
diff --git a/docs/project-memory.md b/docs/project-memory.md
index 881f230c..25416bc9 100644
--- a/docs/project-memory.md
+++ b/docs/project-memory.md
@@ -4,8 +4,8 @@ Last updated: 2026-10-26
Piik is private, low-latency screen sharing for one Host and up to 20
authenticated friends. The current product surface is Web Host, Web Viewer, and
-Browser relay; desktop and mobile Browsers are Viewer targets, and camera-capable
-mobile Browsers can also host. It is not a
+Browser relay; desktop and mobile Browsers are Viewer targets. Browser hosting
+uses the capture capabilities available on the device. It is not a
public broadcast service. Hosted and App run one shared Go core
([ADR-0012](./adr/0012-shared-go-backend-core.md)); Node and Vite build the
Browser UI and are not a runtime. Desktop and Android Apps run
diff --git a/docs/research/browser-local-encoding-pool.md b/docs/research/browser-local-encoding-pool.md
index 68dc4e68..86cdc4f1 100644
--- a/docs/research/browser-local-encoding-pool.md
+++ b/docs/research/browser-local-encoding-pool.md
@@ -595,8 +595,8 @@ Reapplying exact dimensions to the original track did not unblock it.
The Piik sender boundary now forwards bounded raw display frames using
[Chromium's track streams](https://developer.chrome.com/docs/capabilities/web-apis/mediastreamtrack-insertable-media-processing),
-without resizing, re-encoding or generated cadence. Camera/received tracks and
-unsupported Browsers keep ordinary clones. In the textured-tab comparison,
+without resizing, re-encoding or generated cadence. Ordinary camera/received
+senders and unsupported Browsers keep ordinary clones. In the textured-tab comparison,
all but the initial source frame stayed 1904x1048; output escaped 192p through
262/390p to 524p. It still remained 524p after 125 recovery seconds. This removes
the reproduced capture lock, not content-dependent encoder hysteresis or every
diff --git a/docs/research/nat-traversal.md b/docs/research/nat-traversal.md
index 4795576f..3744566d 100644
--- a/docs/research/nat-traversal.md
+++ b/docs/research/nat-traversal.md
@@ -557,8 +557,10 @@ than replacing it with a later, destination-dependent STUN mapping. Other
controls cover early/late cancellation, source/Engine retirement, premature
commit, resource limits, stale signaling and Browser decline. These establish
bounded composition, not a physical difficult-network success-rate improvement.
-The candidate's send window is 800 ms within a 1.1-second hard bound; the earlier
-isolated experiment's shorter window is not its product timing.
+The candidate's initial send window is 800 ms; readers serve the full existing
+1.1-second check bound. Initial fanout reserves capacity for authenticated
+peer-triggered checks within the same 64-request budget. The earlier isolated
+experiment's shorter window is not its product timing.
Stable observations now use the exact survey socket in that same preparation;
reopening a socket would invalidate the observed address. Mixed stable/sequential
diff --git a/docs/standards/media-quality.md b/docs/standards/media-quality.md
index 7b9a1fea..b73a1811 100644
--- a/docs/standards/media-quality.md
+++ b/docs/standards/media-quality.md
@@ -99,6 +99,9 @@ Piik does not automatically mute either microphone or switch between them.
default. Missing audio is reported clearly but does not block video-only
sharing. Native screen capture can include system playback audio and native
window capture can include selected-process audio when the platform exposes it.
+- Shared Browser encoders normalize capture orientation into pixels before
+ reusing encoded frames across receivers. Ordinary senders retain Browser-owned
+ orientation handling; missing normalization APIs select that existing path.
- Authoritative Browser pause disables the current source and sender-owned tracks
while retaining the room and established routes. The connection owns its
encoded-output pause even when shared encoding is unavailable; the pool pauses
@@ -122,7 +125,10 @@ Piik does not automatically mute either microphone or switch between them.
Applied settings change only after the replacement is ready and installed;
required-output failure preserves the previous capture and profile. Failed lower
outputs stay unavailable across preparation and installation without rejecting
- a healthy original output. Platform quality preference uses the hardware
+ a healthy original output. A reported Windows capture-device loss prepares and
+ installs a fresh video generation through that same owner, retaining the source,
+ media connections and source audio. Source closure and permission cancellation
+ still end capture. Platform quality preference uses the hardware
encoder's standard quality-versus-speed hint. Pion owns Native WebRTC
transport; the shared media adapter uses LiveKit media components for bandwidth
estimation, forwarding allocation, pacing and recovery.
diff --git a/docs/standards/presentation-lifecycle.md b/docs/standards/presentation-lifecycle.md
index eda7e310..238c2318 100644
--- a/docs/standards/presentation-lifecycle.md
+++ b/docs/standards/presentation-lifecycle.md
@@ -196,8 +196,9 @@ Viewer playback settings reuse shared controls in an anchored popover;
placement follows the [visual language](./visual-language.md) without resizing video. Local picture
fit defaults to contain; cover preserves proportions and may crop edges, while
stretch fills the player by changing proportions. These choices do not change
-capture or encoding. Manual quarter-turn rotation is another local picture
-preference; it does not depend on device auto-rotation and applies to page
+capture or encoding. Each rotation action advances the local picture 90° clockwise,
+returning to its original direction after four turns. This preference does not
+depend on device auto-rotation and applies to page
playback, including theater and DOM fullscreen. System video presentations
retain their browser-owned behavior. Remote input and magnification use the same rendered
picture geometry. Incoming-route choice follows the
diff --git a/docs/standards/visual-language.md b/docs/standards/visual-language.md
index 95403814..f355fd36 100644
--- a/docs/standards/visual-language.md
+++ b/docs/standards/visual-language.md
@@ -101,8 +101,9 @@ of a circle. Header utilities share compact alignment and icon scale with the
language track. Low-frequency help stays unframed and quiet; it need not carry
the outline or shadow of direct settings. Narrow layouts shorten captions or
spacing before changing that hierarchy.
-Keep App, language, theme and help together; narrow room headers place connection
-status beside the brand above this utility group instead of wrapping each action.
+Keep desktop App, language, theme and help together on the right. Narrow headers
+retain one row with the brand/back action, signaling dots, App and More; language,
+theme and help share that attached disclosure without duplicating their controls.
`segmented-control.css` owns the sliding treatment for language, mode selection,
tabs and native radio groups. Consumers supply selection and layout; they must
not duplicate the selected colours or press animation. Language choices keep
diff --git a/docs/status.md b/docs/status.md
index 4e22e64a..84750c70 100644
--- a/docs/status.md
+++ b/docs/status.md
@@ -38,10 +38,12 @@ Release artifacts and operator records own delivery completion.
## Current Work
The maintenance candidate preserves published protocols, compatibility and route
-ownership. Its routing, media recovery and input-lifecycle audit is complete;
-confirmed defects are repaired and cross-reviewed. Publication awaits the owner's
-instruction. Isolated 3.0 work proceeds from the accepted semantic model rather
-than a mechanical translation of the current implementation.
+ownership. New release feedback has reopened Windows capture termination and
+Browser motion-sharing cadence investigations. Device-loss recovery now retains
+the active media session; the original termination triggers remain under
+investigation. Isolated 3.0 work
+proceeds from the accepted semantic model rather than a mechanical translation
+of the current implementation.
[TODO](./todo.md) is the only unfinished-work ledger; Git and PRs own
completed fixes. [Verification status](./verification-status.md) owns material
acceptance and platform boundaries.
diff --git a/docs/todo.md b/docs/todo.md
index 82cd6550..bc8e8683 100644
--- a/docs/todo.md
+++ b/docs/todo.md
@@ -1,12 +1,24 @@
# Current TODO Ledger
-Last reviewed: 2026-10-29
+Last reviewed: 2026-11-01
This ledger contains unfinished work and explicit owner holds. Product modules
own behavior; research and Git/PRs own evidence and completed history.
## Now
+- [ ] **Browser sharing falls to low source FPS during motion.** Trace raw
+ capture, sender-owned track input and encoding on Windows Chrome with mixed
+ direct/SFU delivery. The observed source fell to roughly 2 fps during video
+ or game playback; reproduce with real window/display capture. Keep the
+ separate SFU packet-loss stage distinct from source-frame production.
+- [ ] **Windows App sharing ends unexpectedly.** Identify WGC/source, device or
+ encoder termination in the supplied release feedback using the native stage
+ and HRESULT, and reproduce the unexpected termination.
+- [ ] **Native screen sound recaptures room voice.** Exclude Piik voice while
+ retaining intended source audio. Verify Browser own-document exclusion and
+ Native process exclusion; keep acoustic echo and duplicate commentary with
+ their separate microphone owners.
- [ ] **Virtual-LAN page loading.** Investigate the UU report where only the tab
title appears and the page never loads. Static compression reduces transfer
size; capture the stalled document/script response to locate this failure.
@@ -24,12 +36,13 @@ own behavior; research and Git/PRs own evidence and completed history.
report concerns missing screen thumbnails in the source picker. The macOS
helper now implements the existing preview command; verify its build and
display/window thumbnails on a physical Mac before closing the issue.
-- [ ] **Control loss when the Host browser is minimized (#502).** Follow
+- [ ] **Mouse input stops when Host Chrome is minimized (#502).** Follow
[#502](https://github.com/TNTcraftHIM/Piik/issues/502) on the original Windows
- host with the Windows input-recovery candidate. The v2.5.0 report shows
- rejected input followed by failed release cleanup. The candidate repairs a
- reproduced thread-desktop mismatch and records the native rejection context;
- confirm the original host's trigger before closing the issue.
+ host: the v2.5.1 follow-up retains the grant, but mouse movement and left/right
+ clicks stop until Chrome is restored. The diagnostic candidate adds aggregate
+ input-stage and sink-API counts without recording input values. Obtain App ZIPs
+ during the minimized failure and after restoration; the local controlled
+ Chrome trial did not reproduce the original machine's trigger.
- [ ] **Return LiveKit to an upstream pin.** Follow
[upstream #5016](https://github.com/livekit/livekit/pull/5016). Once upstream
supplies the allocator input Piik needs, verify recovery and remove the
@@ -55,10 +68,9 @@ own behavior; research and Git/PRs own evidence and completed history.
## Held By Owner
-- [ ] **iOS App publication.** Held on 2026-10-25. Keep the system-browser
- implementation on `feat/ios-client`, outside the Android release candidate.
- Apple SDK builds pass; signed-device acceptance still needs continued-session
- admission/expiry, recording consent, background capture and hosted-site linkage.
+- [ ] **iOS native App.** Paused in favor of Browser use. Preserve the isolated
+ implementation on `feat/ios-client`; native development and publication stay
+ outside the current phase.
- [ ] **QR invitations.** Deferred from the maintenance release on 2026-10-24.
Keep the local QR implementation on `archive/deferred-prototypes` for later acceptance.
- [ ] **Additional collaboration cursors.** Deferred by the owner. Keep the
diff --git a/internal/app/app.go b/internal/app/app.go
index e12944f6..27c69682 100644
--- a/internal/app/app.go
+++ b/internal/app/app.go
@@ -158,6 +158,11 @@ func Run(ctx context.Context, options Options) (returnedErr error) {
"localAccessProtected": config.LocalAccessPassword != ""})
recorder.Context("capture", nativeMedia.capture)
recorder.Binary("captureExecutable", nativeMedia.captureProcess)
+ options.console.send(consoleDebug{logPath: recorder.LogPath(), export: func() (string, error) {
+ snapshot := input.Diagnostics()
+ recorder.Context("remoteInput", snapshot)
+ return recorder.Export()
+ }})
slog.Debug("piik-client", "event", "native-capabilities",
"video", nativeMedia.capabilities.Video, "processAudio", nativeMedia.capabilities.ProcessAudio,
"systemAudio", nativeMedia.capabilities.SystemAudio, "hardwareH264", nativeMedia.capabilities.HardwareH264,
@@ -709,6 +714,7 @@ func discoverNativeMedia(ctx context.Context, configuredPath string) nativeRunti
ShareVolume: true,
ProcessAudio: summary.ProcessAudio,
ProcessAudioExclusion: summary.ProcessAudioExclusion,
+ CaptureFailureDetails: true,
SystemAudio: summary.SystemAudio,
HardwareH264: summary.HardwareH264,
SoftwareVP8: summary.SoftwareVP8,
diff --git a/internal/app/loopback/server.go b/internal/app/loopback/server.go
index b64ebd3c..70e3ae4b 100644
--- a/internal/app/loopback/server.go
+++ b/internal/app/loopback/server.go
@@ -95,6 +95,7 @@ type NativeMediaCapabilities struct {
CaptureBorderControl bool `json:"captureBorderControl,omitempty"`
Microphone bool `json:"microphone,omitempty"`
SourceCancellation bool `json:"sourceCancellation,omitempty"`
+ CaptureFailureDetails bool `json:"captureFailureDetails,omitempty"`
ShareVolume bool `json:"shareVolume,omitempty"`
ProcessAudio bool `json:"processAudio"`
ProcessAudioExclusion bool `json:"processAudioExclusion,omitempty"`
diff --git a/internal/app/mediaedge/groups_test.go b/internal/app/mediaedge/groups_test.go
index 13fa120a..d99c1248 100644
--- a/internal/app/mediaedge/groups_test.go
+++ b/internal/app/mediaedge/groups_test.go
@@ -468,7 +468,7 @@ func TestLateOutputGroupKeepsSharedCaptureClock(t *testing.T) {
var timestamp uint32
for _, media := range []*forwarding.EncodedSource{source.media, group.media} {
if err := media.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8,
- PTS: pts, Duration: time.Second / 30}); err != nil {
+ PTS: pts, Duration: time.Second / 30, Recovery: true}); err != nil {
t.Fatal(err)
}
report := media.GetAllBuffers()[0].GetSenderReportData()
@@ -615,7 +615,7 @@ func TestRelayGroupSharesReceivedClockAcrossWrap(t *testing.T) {
}
write(3, 1544, 20*time.Millisecond)
check(source.WriteVideo(2, encoded.Frame{Data: sfu.VP8KeyFrame8x8,
- PTS: 20 * time.Millisecond, Duration: time.Second / 30}))
+ PTS: 20 * time.Millisecond, Duration: time.Second / 30, Recovery: true}))
// A delayed older wire report is rejected by the original owner before
// either internal copy can adopt it.
check(source.SenderReport(earlier))
diff --git a/internal/app/mediaedge/punch_sockets.go b/internal/app/mediaedge/punch_sockets.go
index 8d04ab5b..f09daa49 100644
--- a/internal/app/mediaedge/punch_sockets.go
+++ b/internal/app/mediaedge/punch_sockets.go
@@ -33,7 +33,8 @@ func punchSocketArray(ctx context.Context, listen func() (net.PacketConn, error)
if remote == nil || remote.IP.To4() == nil || remote.Port < 1 || remote.Port > 65535 || len(secret) != 32 {
return nil, errors.New("invalid peer or generation secret")
}
- ctx, cancel := context.WithTimeout(ctx, 1100*time.Millisecond)
+ parent := ctx
+ ctx, cancel := context.WithTimeout(parent, 1100*time.Millisecond)
defer cancel()
for len(sockets) < count {
if err := ctx.Err(); err != nil {
@@ -63,6 +64,21 @@ func punchSocketArray(ctx context.Context, listen func() (net.PacketConn, error)
ready := make(chan int, len(sockets))
var readers sync.WaitGroup
var responses atomic.Int32
+ var proved atomic.Bool
+ var requestsLeft atomic.Int32
+ requestsLeft.Store(64)
+ sendCheck := func(index int, reserve int32) {
+ for {
+ left := requestsLeft.Load()
+ if left <= reserve {
+ return
+ }
+ if requestsLeft.CompareAndSwap(left, left-1) {
+ _, _ = sockets[index].WriteTo(requests[index].Raw, remote)
+ return
+ }
+ }
+ }
for i, connection := range sockets {
requests[i] = stun.MustBuild(stun.TransactionID, stun.BindingRequest, integrity, stun.Fingerprint)
readers.Add(1)
@@ -86,8 +102,14 @@ func punchSocketArray(ctx context.Context, listen func() (net.PacketConn, error)
reply := stun.MustBuild(stun.BindingSuccess, stun.NewTransactionIDSetter(message.TransactionID),
&stun.XORMappedAddress{IP: remote.IP, Port: remote.Port}, integrity, stun.Fingerprint)
_, _ = connection.WriteTo(reply.Raw, remote)
+ if !proved.Load() {
+ // The peer may start after our initial fanout. Check the
+ // authenticated tuple through this same socket and budget.
+ sendCheck(i, 0)
+ }
case stun.BindingSuccess:
if message.TransactionID == requests[i].TransactionID {
+ proved.Store(true)
select {
case ready <- i:
default:
@@ -97,21 +119,23 @@ func punchSocketArray(ctx context.Context, listen func() (net.PacketConn, error)
}
}()
}
- // Keep servicing authenticated peer requests until the finite send window
- // ends, including after a local response arrives. One peer finishing first
- // must not strand its partner before socket adoption.
+ // Keep the existing eight-round send window, reserving one check per socket
+ // for authenticated peer-triggered work. Both share the same 64-request bound.
for round := 0; round < 8 && ctx.Err() == nil; round++ {
- for i, connection := range sockets {
+ for i := range sockets {
if ctx.Err() != nil {
break
}
- _, _ = connection.WriteTo(requests[i].Raw, remote)
+ sendCheck(i, int32(len(sockets)))
}
select {
case <-ctx.Done():
case <-time.After(100 * time.Millisecond):
}
}
+ // The 800 ms initial fanout is not the 1.1-second receive budget. Retain
+ // readers through that budget so delayed peer starts and replies can finish.
+ <-ctx.Done()
for _, connection := range sockets {
_ = connection.SetReadDeadline(time.Now())
}
@@ -121,7 +145,7 @@ func punchSocketArray(ctx context.Context, listen func() (net.PacketConn, error)
if !stopDeadline() {
<-deadlineDone
}
- if err := ctx.Err(); err != nil {
+ if err := parent.Err(); err != nil {
return nil, err
}
select {
diff --git a/internal/app/mediaedge/punch_test.go b/internal/app/mediaedge/punch_test.go
index 9810da6a..d4d97a7d 100644
--- a/internal/app/mediaedge/punch_test.go
+++ b/internal/app/mediaedge/punch_test.go
@@ -58,12 +58,15 @@ func TestPunchAdoptsProvedTupleIntoMediaAndFeedback(t *testing.T) {
name string
left, right vnet.NATType
skew time.Duration
+ dropLate bool
}{
- {"sequential", sequentialPunchNAT, sequentialPunchNAT, 0},
- {"sequential-delayed", sequentialPunchNAT, sequentialPunchNAT, 250 * time.Millisecond},
- {"stable-sender", stable, sequentialPunchNAT, 250 * time.Millisecond},
- {"stable-receiver", sequentialPunchNAT, stable, 250 * time.Millisecond},
- {"both-stable", stable, stable, 0},
+ {"sequential", sequentialPunchNAT, sequentialPunchNAT, 0, false},
+ {"sequential-delayed", sequentialPunchNAT, sequentialPunchNAT, 250 * time.Millisecond, false},
+ {"sequential-late", sequentialPunchNAT, sequentialPunchNAT, 900 * time.Millisecond, false},
+ {"stable-sender", stable, sequentialPunchNAT, 250 * time.Millisecond, false},
+ {"stable-sender-late-loss", stable, sequentialPunchNAT, 900 * time.Millisecond, true},
+ {"stable-receiver", sequentialPunchNAT, stable, 250 * time.Millisecond, false},
+ {"both-stable", stable, stable, 0, false},
} {
t.Run(scenario.name, func(t *testing.T) {
network := newPunchLab(t, scenario.left, scenario.right)
@@ -81,15 +84,56 @@ func TestPunchAdoptsProvedTupleIntoMediaAndFeedback(t *testing.T) {
t.Cleanup(func() { _ = source.Close() })
left, leftPrediction := preparePunch(t, engines[0], source, "connection", strings.Repeat("42", 32))
right, rightPrediction := preparePunch(t, engines[1], nil, "connection", strings.Repeat("42", 32))
+ var observing, late atomic.Bool
+ var checks, replies [2]atomic.Int32
+ var lateChecks, dropped atomic.Int32
+ observing.Store(true)
+ network.wan.AddChunkFilter(func(chunk vnet.Chunk) bool {
+ if !observing.Load() || chunk.Network() != "udp" {
+ return true
+ }
+ message := &stun.Message{Raw: chunk.UserData()}
+ if message.Decode() != nil {
+ return true
+ }
+ for side := range 2 {
+ if chunk.SourceAddr().(*net.UDPAddr).IP.String() != network.public[side] ||
+ chunk.DestinationAddr().(*net.UDPAddr).IP.String() != network.public[1-side] {
+ continue
+ }
+ switch message.Type {
+ case stun.BindingRequest:
+ checks[side].Add(1)
+ if side == 0 && late.Load() && lateChecks.Add(1) == 1 && scenario.dropLate {
+ dropped.Add(1)
+ return false
+ }
+ case stun.BindingSuccess:
+ replies[side].Add(1)
+ }
+ }
+ return true
+ })
results := make(chan bool, 2)
go func() { results <- left.Run(*rightPrediction) }()
time.Sleep(scenario.skew)
+ late.Store(true)
go func() { results <- right.Run(*leftPrediction) }()
for range 2 {
if !<-results {
t.Fatal("authenticated preparation failed")
}
}
+ observing.Store(false)
+ for side := range 2 {
+ if checks[side].Load() > 64 || replies[side].Load() > 64 {
+ t.Fatalf("preparation packet budget exceeded: requests=%d replies=%d", checks[side].Load(), replies[side].Load())
+ }
+ }
+ if scenario.dropLate && (dropped.Load() != 1 || lateChecks.Load() < 2) {
+ t.Fatalf("missing first triggered-check loss/retry: dropped=%d late=%d", dropped.Load(), lateChecks.Load())
+ }
+ t.Logf("preparation requests=%d/%d replies=%d/%d late-checks=%d dropped=%d", checks[0].Load(), checks[1].Load(), replies[0].Load(), replies[1].Load(), lateChecks.Load(), dropped.Load())
// No fresh STUN mapping can replace the endpoint that proved this path.
socket := engines[1].TakePunchedSocket("connection", nil)
if socket == nil {
diff --git a/internal/app/mediaedge/source_test.go b/internal/app/mediaedge/source_test.go
index 972ad77a..38484800 100644
--- a/internal/app/mediaedge/source_test.go
+++ b/internal/app/mediaedge/source_test.go
@@ -351,7 +351,7 @@ func TestSourceFormatAdaptationPreservesLiveLayers(t *testing.T) {
// codec activation through the separate demand planner.
check(source.beginGroupFrame(pts, time.Now()))
for layer := range 2 {
- check(source.WriteVideo(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: period}))
+ check(source.WriteVideo(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: period, Recovery: true}))
}
pts += period
}
diff --git a/internal/app/nativecapture/stream.go b/internal/app/nativecapture/stream.go
index 2e58523f..57d03f39 100644
--- a/internal/app/nativecapture/stream.go
+++ b/internal/app/nativecapture/stream.go
@@ -40,6 +40,33 @@ var (
captureHRESULT = regexp.MustCompile(`^0x[0-9a-fA-F]{8}$`)
)
+// Failure contains only validated helper codes, never stderr detail or identity.
+type Failure struct {
+ Stage string `json:"stage,omitempty"`
+ HRESULT string `json:"hresult,omitempty"`
+ ExitCode int `json:"exitCode"`
+}
+
+type processFailure struct {
+ details Failure
+ cause error
+}
+
+func (failure *processFailure) Error() string {
+ return fmt.Sprintf("native capture failed (stage=%s, hresult=%s): %v", failure.details.Stage, failure.details.HRESULT, failure.cause)
+}
+
+func (failure *processFailure) Unwrap() error { return failure.cause }
+
+func FailureDetails(err error) *Failure {
+ var failure *processFailure
+ if !errors.As(err, &failure) {
+ return nil
+ }
+ details := failure.details
+ return &details
+}
+
type audioCaptureState struct {
State string `json:"state"`
Audio bool `json:"audio"`
@@ -532,10 +559,10 @@ func startStreamWithEnvironment(
stopParent()
_ = trace.Close()
stage, hresult := captureFailureCodes(stderr.Bytes())
- if waitErr != nil && (stage != "" || hresult != "") {
+ if waitErr != nil && (stage != "" || hresult != "" || ctx.Err() == nil) {
// Preserve validated failure codes and the exit cause for diagnostics
// and errors.Is/As, never the helper's raw detail.
- stream.failure = fmt.Errorf("native capture failed (stage=%s, hresult=%s): %w", stage, hresult, waitErr)
+ stream.failure = &processFailure{details: Failure{Stage: stage, HRESULT: hresult, ExitCode: command.ProcessState.ExitCode()}, cause: waitErr}
}
logCaptureFailure(ctx, waitErr, command.ProcessState.ExitCode(), stderr.Bytes())
logger.DebugContext(ctx, "piik-client", "event", "capture-process-ended", "mode", mode,
diff --git a/internal/app/nativecapture/stream_test.go b/internal/app/nativecapture/stream_test.go
index 42ae41f2..98415087 100644
--- a/internal/app/nativecapture/stream_test.go
+++ b/internal/app/nativecapture/stream_test.go
@@ -84,6 +84,9 @@ func TestCaptureStopHelper(t *testing.T) {
if mode == "" {
return
}
+ if mode == "startup-exit" {
+ os.Exit(3)
+ }
if mode == "startup-stage" {
_, _ = io.WriteString(os.Stderr, "result=window-capture-failed\nstage=mft-output-timeout\nhresult=0x80004005\ndetail=private window title C:\\private\\capture.exe\n")
os.Exit(2)
@@ -128,6 +131,13 @@ func TestCaptureFailureSurvivesProcessRetirementWithoutDebugLogging(t *testing.T
t.Fatal("failed process produced a ready frame")
}
err = stream.Close()
+ details := FailureDetails(errors.Join(errors.New("capture read ended"), err))
+ if details == nil || *details != (Failure{Stage: "mft-output-timeout", HRESULT: "0x80004005", ExitCode: 2}) {
+ t.Fatalf("process failure lost structured codes: %#v", details)
+ }
+ if FailureDetails(errors.New("stage=forged hresult=0x80004005")) != nil || FailureDetails(nil) != nil {
+ t.Fatal("failure metadata was inferred from an untyped error")
+ }
if err == nil || !strings.Contains(err.Error(), "mft-output-timeout") || !strings.Contains(err.Error(), "0x80004005") {
t.Fatalf("process retirement lost native failure stage: %v", err)
}
@@ -138,6 +148,17 @@ func TestCaptureFailureSurvivesProcessRetirementWithoutDebugLogging(t *testing.T
if !errors.As(err, &exit) || exit.ExitCode() != 2 {
t.Fatalf("native failure lost its process exit cause: %v", err)
}
+ stream, err = startStreamWithEnvironment(t.Context(), executable,
+ []string{"-test.run=^TestCaptureStopHelper$"}, []string{"PIIK_CAPTURE_STOP_FIXTURE=startup-exit"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, err := stream.Read(); err == nil {
+ t.Fatal("uncoded process failure produced a frame")
+ }
+ if details = FailureDetails(stream.Close()); details == nil || *details != (Failure{ExitCode: 3}) {
+ t.Fatalf("uncoded process failure lost its exit status: %#v", details)
+ }
}
func TestCaptureStopUsesOneBoundedRetirementPath(t *testing.T) {
diff --git a/internal/app/nativecontrol/session.go b/internal/app/nativecontrol/session.go
index 0b33bf5a..a5c5d204 100644
--- a/internal/app/nativecontrol/session.go
+++ b/internal/app/nativecontrol/session.go
@@ -762,7 +762,7 @@ func (session *Session) startShare(
codec := host.Codec()
slog.Debug("piik-client", "event", "native-profile-applied", "requestId", envelope.ID,
"share", diagnostics.ID(request.ShareID), "profile", profile, "codec", codec, "audio", host.HasAudio())
- go func() { defer cancel(); session.watchHost(host) }()
+ go func() { defer cancel(); session.watchHost(host, request.ReportCaptureFailure) }()
return shareStartedResponse{
responseEnvelope: response(envelope, "share-started"),
ShareID: request.ShareID,
@@ -999,7 +999,7 @@ func (session *Session) stopViewer(shareID string) {
_ = viewer.Close()
}
-func (session *Session) watchHost(host *nativehost.Session) {
+func (session *Session) watchHost(host *nativehost.Session, reportFailure bool) {
for {
select {
case <-host.RemoteInputChanges():
@@ -1017,12 +1017,16 @@ func (session *Session) watchHost(host *nativehost.Session) {
}
session.mu.Unlock()
if open {
- session.emit(shareEndedEvent{
+ ended := shareEndedEvent{
Version: loopback.ProtocolVersion,
Type: "share-ended",
ShareID: host.ShareID(),
Failed: err != nil,
- })
+ }
+ if reportFailure && err != nil {
+ ended.Failure = nativecapture.FailureDetails(err)
+ }
+ session.emit(ended)
}
return
}
diff --git a/internal/app/nativecontrol/session_test.go b/internal/app/nativecontrol/session_test.go
index 700724b6..0e778cc5 100644
--- a/internal/app/nativecontrol/session_test.go
+++ b/internal/app/nativecontrol/session_test.go
@@ -275,9 +275,73 @@ func runQuietCaptureFixture(directory string) {
}
state.State = "active"
status()
+ if os.Getenv("PIIK_CAPTURE_LIVE_FAILURE") == "1" {
+ for {
+ select {
+ case <-closed:
+ return
+ case <-time.After(20 * time.Millisecond):
+ if _, err := os.Stat(filepath.Join(directory, "failure")); err == nil {
+ _, _ = io.WriteString(os.Stderr, "stage=windows-runtime\nhresult=0x80004005\ndetail=private fixture title and path\n")
+ os.Exit(2)
+ }
+ }
+ }
+ }
<-closed
}
+func TestLiveCaptureFailureDetailsRequireOptIn(t *testing.T) {
+ executable, err := os.Executable()
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, report := range []bool{false, true} {
+ t.Run(strconv.FormatBool(report), func(t *testing.T) {
+ directory := t.TempDir()
+ t.Setenv("PIIK_QUIET_CAPTURE_FIXTURE", directory)
+ t.Setenv("PIIK_CAPTURE_LIVE_FAILURE", "1")
+ session := New(executable, nativecapture.Capabilities{SoftwareVP8: true}, false, nil)
+ defer session.Close()
+ request := startShareRequest{Version: 10, ID: "request_start", Type: "start-share", ShareID: "share_123456",
+ Source: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"},
+ ReportCaptureFailure: report, EdgeCapacity: 1, Codec: "vp8",
+ Profile: qualitySettings{Resolution: protocol.QualityResolution{Width: 1920, Height: 1080}, MaxFramerate: 30, MaxBitrate: 5_000_000, DegradationPreference: "balanced"}}
+ payload, _ := json.Marshal(request)
+ if _, err := awaitControlResponse(t, session, payload); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(directory, "failure"), nil, 0600); err != nil {
+ t.Fatal(err)
+ }
+ deadline := time.NewTimer(5 * time.Second)
+ defer deadline.Stop()
+ for {
+ select {
+ case value := <-session.Events():
+ ended, ok := deliveredControlValue(value).(shareEndedEvent)
+ if !ok {
+ continue
+ }
+ if !ended.Failed || session.current(request.ShareID) != nil || (ended.Failure != nil) != report {
+ t.Fatalf("capture failure changed retirement or legacy payload: %#v", ended)
+ }
+ if report && *ended.Failure != (nativecapture.Failure{Stage: "windows-runtime", HRESULT: "0x80004005", ExitCode: 2}) {
+ t.Fatalf("capture failure lost structured codes: %#v", ended.Failure)
+ }
+ encoded, _ := json.Marshal(ended)
+ if strings.Contains(string(encoded), "private") || strings.Contains(string(encoded), "detail") {
+ t.Fatalf("capture failure exposed helper detail: %s", encoded)
+ }
+ return
+ case <-deadline.C:
+ t.Fatal("live capture failure lost its event")
+ }
+ }
+ })
+ }
+}
+
func TestMicrophoneOptInPreservesLegacyAndOrdersStateBeforeCompletion(t *testing.T) {
for _, capabilities := range []struct{ optIn, microphone bool }{{false, true}, {true, true}, {true, false}} {
optIn, microphone := capabilities.optIn, capabilities.microphone
diff --git a/internal/app/nativecontrol/wire.go b/internal/app/nativecontrol/wire.go
index 38837878..9a6c9e4b 100644
--- a/internal/app/nativecontrol/wire.go
+++ b/internal/app/nativecontrol/wire.go
@@ -40,6 +40,7 @@ type qualitySettings = protocol.QualitySettings
type startShareRequest struct {
ReportSourceCancellation bool `json:"reportSourceCancellation,omitempty"`
+ ReportCaptureFailure bool `json:"reportCaptureFailure,omitempty"`
Version int `json:"version"`
ID string `json:"id"`
Type string `json:"type"`
@@ -387,8 +388,9 @@ type edgeQualityEvent struct {
}
type shareEndedEvent struct {
- Version int `json:"version"`
- Type string `json:"type"`
- ShareID string `json:"shareId"`
- Failed bool `json:"failed"`
+ Version int `json:"version"`
+ Type string `json:"type"`
+ ShareID string `json:"shareId"`
+ Failed bool `json:"failed"`
+ Failure *nativecapture.Failure `json:"failure,omitempty"`
}
diff --git a/internal/app/nativehost/audio_test.go b/internal/app/nativehost/audio_test.go
index 40623420..0e01dfe3 100644
--- a/internal/app/nativehost/audio_test.go
+++ b/internal/app/nativehost/audio_test.go
@@ -59,6 +59,14 @@ func runAudioRecoveryCapture() {
}
}
audio := microphone || (len(os.Args) > 1 && os.Args[1] == "--capture-audio")
+ var deviceFailure string
+ if marker := os.Getenv("PIIK_VIDEO_DEVICE_FAILURE"); !audio && marker != "" {
+ // Only the first video process fails; a replacement uses the same fixture.
+ if owner, err := os.OpenFile(marker+".owner", os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600); err == nil {
+ _ = owner.Close()
+ deviceFailure = marker
+ }
+ }
startupGate := os.Getenv("PIIK_VIDEO_START_GATE")
if audio && !microphone && startupGate != "" {
if _, err := os.Stat(startupGate + ".active"); err != nil {
@@ -181,6 +189,17 @@ func runAudioRecoveryCapture() {
Duration: period, Data: make([]byte, nativeaudio.FrameBytes)})
continue
}
+ if deviceFailure != "" {
+ if _, err := os.Stat(deviceFailure); err == nil {
+ if os.Getenv("PIIK_VIDEO_DEVICE_CONTROL") == "1" {
+ _ = os.Stdin.Close()
+ write(nativecapture.Frame{Kind: nativecapture.FrameBegin, Timestamp: pts, Duration: period})
+ time.Sleep(50 * time.Millisecond)
+ }
+ _, _ = io.WriteString(os.Stderr, "stage=capture-device-lost\nhresult=0x887a0005\n")
+ os.Exit(2)
+ }
+ }
write(nativecapture.Frame{Kind: nativecapture.FrameBegin, Timestamp: pts, Duration: period})
for layer := range 2 {
write(nativecapture.Frame{Kind: nativecapture.FrameVP8, Layer: layer, Width: 8, Height: 8,
diff --git a/internal/app/nativehost/session.go b/internal/app/nativehost/session.go
index 906407a8..b9fa42d9 100644
--- a/internal/app/nativehost/session.go
+++ b/internal/app/nativehost/session.go
@@ -106,6 +106,7 @@ type Session struct {
publications map[publicationKey]*mediaedge.Publication
captureApplied chan struct{}
+ captureChanged chan struct{}
captureUnavailable []int
audioChanged chan struct{}
@@ -150,22 +151,23 @@ func Start(parent context.Context, options Options) (*Session, error) {
defer func() { capturePreparationDone(preparing) }()
ctx, cancel := context.WithCancel(parent)
session := &Session{
- shareID: options.ShareID,
- capture: capture,
- videoOptions: options.Video,
- profile: options.Profile,
- edgeCapacity: options.EdgeCapacity,
- stream: stream,
- engine: engine,
- events: options.Events,
- ctx: ctx,
- cancel: cancel,
- done: make(chan error, 1),
- ready: make(chan error, 1),
- videoActive: make(chan struct{}),
- edges: make(map[string]*mediaedge.Edge),
- audioChanged: make(chan struct{}, 1),
- remoteInput: options.RemoteInput.NewSession(),
+ shareID: options.ShareID,
+ capture: capture,
+ videoOptions: options.Video,
+ profile: options.Profile,
+ edgeCapacity: options.EdgeCapacity,
+ stream: stream,
+ engine: engine,
+ events: options.Events,
+ ctx: ctx,
+ cancel: cancel,
+ done: make(chan error, 1),
+ ready: make(chan error, 1),
+ videoActive: make(chan struct{}),
+ edges: make(map[string]*mediaedge.Edge),
+ audioChanged: make(chan struct{}, 1),
+ captureChanged: make(chan struct{}, 1),
+ remoteInput: options.RemoteInput.NewSession(),
}
session.syncRemoteInputSource(nil)
// Drain video while the optional audio process initializes. Otherwise a full
@@ -795,6 +797,10 @@ func (session *Session) commitCapture(
session.captureUnavailable = state.unavailableLayers
session.videoOptions = options
session.profile = profile
+ select {
+ case session.captureChanged <- struct{}{}:
+ default:
+ }
}
if replaceAudio {
session.audioStream = replacementAudio
@@ -837,6 +843,28 @@ func (session *Session) commitCapture(
return nil
}
+func (session *Session) prepareCaptureRecovery(failed CaptureStream) error {
+ session.updateMu.Lock()
+ defer session.updateMu.Unlock()
+ session.mu.Lock()
+ if session.closed || session.ctx.Err() != nil {
+ session.mu.Unlock()
+ return errors.New("native share is unavailable")
+ }
+ if session.stream != failed {
+ session.mu.Unlock()
+ return nil
+ }
+ options, profile := session.videoOptions, session.profile
+ session.mu.Unlock()
+ replacement, state, err := session.prepareVideo(session.ctx, options, false, false)
+ if err != nil {
+ return err
+ }
+ defer capturePreparationDone(replacement)
+ return session.commitCapture(options, profile, replacement, state, nil, false, nil)
+}
+
// Only the video reader installs source metadata, after leaving the old input.
// The update response waits for this exact stream's installation, not its process.
func (session *Session) installCapture(next CaptureStream) error {
@@ -1007,13 +1035,18 @@ func (session *Session) runVideo() error {
}
activeSeen = true
}
+ current := session.currentStream()
fail := func(err error) error {
+ // A final frame/control error can arrive before the process waiter joins.
+ // Preserve that read/validation error and the selected helper's exit cause.
+ if current != nil {
+ err = errors.Join(err, current.Close())
+ }
if !ready {
session.ready <- err
}
return err
}
- current := session.currentStream()
changeStream := func(next CaptureStream) error {
if err := session.installCapture(next); err != nil {
return err
@@ -1023,148 +1056,194 @@ func (session *Session) runVideo() error {
markActive()
return nil
}
- for current != nil {
- frame, err := current.Read()
- if err != nil {
- next := session.currentStream()
- if next != nil && next != current {
- if err = changeStream(next); err != nil {
- return fail(err)
+ readFrames := func() error {
+ for current != nil {
+ frame, err := current.Read()
+ if err != nil {
+ // Retirement joins the helper's exit and its sanitized failure codes.
+ // Recheck ownership afterward: a replacement can commit during Close.
+ _ = current.Close()
+ next := session.currentStream()
+ if next != nil && next != current {
+ if err = changeStream(next); err != nil {
+ return fail(err)
+ }
+ continue
}
- continue
- }
- slog.DebugContext(session.ctx, "piik-client", "event", "capture-video-read-ended",
- "share", diagnostics.ID(session.shareID), "canceled", session.ctx.Err() != nil, "eof", errors.Is(err, io.EOF), diagnostics.Error(err))
- if errors.Is(err, nativecapture.ErrSourceCanceled) {
- return fail(err)
- }
- return fail(errors.New("native capture stopped unexpectedly"))
- }
- if next := session.currentStream(); next != current {
- if next != nil {
- if err = changeStream(next); err != nil {
+ slog.DebugContext(session.ctx, "piik-client", "event", "capture-video-read-ended",
+ "share", diagnostics.ID(session.shareID), "canceled", session.ctx.Err() != nil, "eof", errors.Is(err, io.EOF), diagnostics.Error(err))
+ if errors.Is(err, nativecapture.ErrSourceCanceled) {
return fail(err)
}
+ return fail(errors.Join(errors.New("native capture stopped unexpectedly"), err))
}
- continue
- }
- switch frame.Kind {
- case nativecapture.FrameStatus:
- status, statusErr := decodeCaptureState(frame.Data)
- if statusErr != nil {
- return fail(statusErr)
- }
- if !slices.Equal(status.Outputs, current.Outputs()) {
- return fail(errors.New("native capture outputs were not applied"))
- }
- session.mu.Lock()
- if session.closed {
- session.mu.Unlock()
- return fail(errors.New("native share stopped"))
- }
- if session.stream != current {
- session.mu.Unlock()
+ if next := session.currentStream(); next != current {
+ if next != nil {
+ if err = changeStream(next); err != nil {
+ return fail(err)
+ }
+ }
continue
}
- if session.source == nil {
- if session.videoOptions.Codec != "auto" && session.videoOptions.Codec != status.Codec {
+ switch frame.Kind {
+ case nativecapture.FrameStatus:
+ status, statusErr := decodeCaptureState(frame.Data)
+ if statusErr != nil {
+ return fail(statusErr)
+ }
+ if !slices.Equal(status.Outputs, current.Outputs()) {
+ return fail(errors.New("native capture outputs were not applied"))
+ }
+ session.mu.Lock()
+ if session.closed {
session.mu.Unlock()
- return fail(errors.New("native capture codec was not applied"))
+ return fail(errors.New("native share stopped"))
}
- source, sourceErr := session.engine.NewSource(status.Codec, mediaedge.PhysicalSourceCapacity(session.edgeCapacity), min(2, len(status.Outputs)), func(layers []int) {
- if stream := session.currentStream(); stream != nil {
- for _, layer := range layers {
- _ = stream.RequestKeyFrame(layer)
+ if session.stream != current {
+ session.mu.Unlock()
+ continue
+ }
+ if session.source == nil {
+ if session.videoOptions.Codec != "auto" && session.videoOptions.Codec != status.Codec {
+ session.mu.Unlock()
+ return fail(errors.New("native capture codec was not applied"))
+ }
+ source, sourceErr := session.engine.NewSource(status.Codec, mediaedge.PhysicalSourceCapacity(session.edgeCapacity), min(2, len(status.Outputs)), func(layers []int) {
+ if stream := session.currentStream(); stream != nil {
+ for _, layer := range layers {
+ _ = stream.RequestKeyFrame(layer)
+ }
}
+ })
+ if sourceErr != nil {
+ session.mu.Unlock()
+ return fail(sourceErr)
}
- })
- if sourceErr != nil {
+ if sourceErr = configureCaptureOutputs(source, status.Outputs); sourceErr != nil {
+ session.mu.Unlock()
+ _ = source.Close()
+ return fail(sourceErr)
+ }
+ session.source = source
+ session.videoOptions.Codec = status.Codec
+ } else if session.source.Codec() != status.Codec {
session.mu.Unlock()
- return fail(sourceErr)
+ return fail(errors.New("native capture codec changed within a share"))
}
- if sourceErr = configureCaptureOutputs(source, status.Outputs); sourceErr != nil {
- session.mu.Unlock()
- _ = source.Close()
- return fail(sourceErr)
+ if status.State == "active" {
+ markActive()
+ if status.RestoreToken != "" {
+ session.videoOptions.RestoreToken = status.RestoreToken
+ }
}
- session.source = source
- session.videoOptions.Codec = status.Codec
- } else if session.source.Codec() != status.Codec {
+ status.applyBackend(&session.videoOptions)
session.mu.Unlock()
- return fail(errors.New("native capture codec changed within a share"))
- }
- if status.State == "active" {
- markActive()
- if status.RestoreToken != "" {
- session.videoOptions.RestoreToken = status.RestoreToken
+ slog.Debug("piik-client", "event", "capture-state", "state", status.State, "codec", status.Codec,
+ "share", diagnostics.ID(session.shareID), "width", status.Width, "height", status.Height, "fps", status.FPS,
+ "hardware", status.HardwareOnly, "adapterIndex", status.AdapterIndex, "encoderIndex", status.EncoderIndex,
+ "adapterName", diagnostics.SafeText(status.AdapterName), "encoderName", diagnostics.SafeText(status.EncoderName),
+ "profileLevelId", status.ProfileLevelID, "outputs", status.Outputs)
+ session.emit(Event{Type: "capture-state", ShareID: session.shareID, State: status.State})
+ if !ready {
+ ready = true
+ session.ready <- nil
}
- }
- status.applyBackend(&session.videoOptions)
- session.mu.Unlock()
- slog.Debug("piik-client", "event", "capture-state", "state", status.State, "codec", status.Codec,
- "share", diagnostics.ID(session.shareID), "width", status.Width, "height", status.Height, "fps", status.FPS,
- "hardware", status.HardwareOnly, "adapterIndex", status.AdapterIndex, "encoderIndex", status.EncoderIndex,
- "adapterName", diagnostics.SafeText(status.AdapterName), "encoderName", diagnostics.SafeText(status.EncoderName),
- "profileLevelId", status.ProfileLevelID, "outputs", status.Outputs)
- session.emit(Event{Type: "capture-state", ShareID: session.shareID, State: status.State})
- if !ready {
- ready = true
- session.ready <- nil
- }
- case nativecapture.FrameBegin:
- if session.source == nil {
- return fail(errors.New("native capture input arrived before its state"))
- }
- plan, beginErr := session.source.BeginFrame(frame.Timestamp)
- if beginErr != nil {
- return fail(beginErr)
- }
- session.mu.Lock()
- paused := session.paused
- session.mu.Unlock()
- if paused {
- clear(plan.Active)
- }
- controlErr := applyOutputPlan(current, plan, activeSeen)
- if controlErr != nil && session.currentStream() == current {
- return fail(controlErr)
- }
- case nativecapture.FrameH264, nativecapture.FrameVP8:
- if session.source == nil ||
- (frame.Kind == nativecapture.FrameH264) != (session.source.Codec() == "h264") {
- return fail(errors.New("native video frame does not match its source codec"))
- }
- if formatErr := session.source.SetFormat(frame.Layer, frame.Width, frame.Height); formatErr != nil {
- return fail(formatErr)
- }
- // Edge state owns write failures; one retired edge cannot stop siblings.
- session.mu.Lock()
- paused := session.paused
- session.mu.Unlock()
- if !paused {
- writeErr := session.source.WriteVideo(frame.Layer, encoded.Frame{
- Data: frame.Data, PTS: frame.Timestamp, Duration: frame.Duration, Recovery: frame.KeyFrame,
- })
- if errors.Is(writeErr, encoded.ErrInvalidTimestamp) {
- return fail(writeErr)
+ case nativecapture.FrameBegin:
+ if session.source == nil {
+ return fail(errors.New("native capture input arrived before its state"))
}
+ plan, beginErr := session.source.BeginFrame(frame.Timestamp)
+ if beginErr != nil {
+ return fail(beginErr)
+ }
+ session.mu.Lock()
+ paused := session.paused
+ session.mu.Unlock()
+ if paused {
+ clear(plan.Active)
+ }
+ controlErr := applyOutputPlan(current, plan, activeSeen)
+ if controlErr != nil && session.currentStream() == current {
+ return fail(controlErr)
+ }
+ case nativecapture.FrameH264, nativecapture.FrameVP8:
+ if session.source == nil ||
+ (frame.Kind == nativecapture.FrameH264) != (session.source.Codec() == "h264") {
+ return fail(errors.New("native video frame does not match its source codec"))
+ }
+ if formatErr := session.source.SetFormat(frame.Layer, frame.Width, frame.Height); formatErr != nil {
+ return fail(formatErr)
+ }
+ // Edge state owns write failures; one retired edge cannot stop siblings.
+ session.mu.Lock()
+ paused := session.paused
+ session.mu.Unlock()
+ if !paused {
+ writeErr := session.source.WriteVideo(frame.Layer, encoded.Frame{
+ Data: frame.Data, PTS: frame.Timestamp, Duration: frame.Duration, Recovery: frame.KeyFrame,
+ })
+ if errors.Is(writeErr, encoded.ErrInvalidTimestamp) {
+ return fail(writeErr)
+ }
+ }
+ case nativecapture.FrameLayerUnavailable:
+ slog.DebugContext(session.ctx, "piik-client", "event", "capture-output-unavailable",
+ "share", diagnostics.ID(session.shareID), "layer", frame.Layer, "detail", diagnostics.SafeText(string(frame.Data)))
+ if session.source == nil {
+ return fail(errors.New("native output ended before its source state"))
+ }
+ if err = session.disableCaptureLayer(current, frame.Layer); err != nil {
+ return fail(err)
+ }
+ case nativecapture.FramePCM:
+ return fail(errors.New("native video process emitted audio"))
+ default:
+ return fail(errors.New("native video process emitted an unknown frame"))
}
- case nativecapture.FrameLayerUnavailable:
- slog.DebugContext(session.ctx, "piik-client", "event", "capture-output-unavailable",
- "share", diagnostics.ID(session.shareID), "layer", frame.Layer, "detail", diagnostics.SafeText(string(frame.Data)))
- if session.source == nil {
- return fail(errors.New("native output ended before its source state"))
- }
- if err = session.disableCaptureLayer(current, frame.Layer); err != nil {
- return fail(err)
- }
- case nativecapture.FramePCM:
- return fail(errors.New("native video process emitted audio"))
+ }
+ return fail(errors.New("native capture stopped unexpectedly"))
+ }
+ for {
+ err := readFrames()
+ failure := nativecapture.FailureDetails(err)
+ if !activeSeen || session.ctx.Err() != nil || errors.Is(err, nativecapture.ErrSourceCanceled) || failure == nil {
+ return err
+ }
+ switch failure.HRESULT {
+ case "0x887a0005", "0x887a0006", "0x887a0007", "0x887a0020": // DXGI device removed, hung, reset, driver internal error.
default:
- return fail(errors.New("native video process emitted an unknown frame"))
+ return err
+ }
+ slog.DebugContext(session.ctx, "piik-client", "event", "capture-device-recovering",
+ "share", diagnostics.ID(session.shareID), "stage", failure.Stage, "hresult", failure.HRESULT, diagnostics.Error(err))
+ recovered := make(chan error, 1)
+ go func(failed CaptureStream) { recovered <- session.prepareCaptureRecovery(failed) }(current)
+ // Updates retain the operation lock until this reader installs their
+ // metadata. Keep serving that handoff while recovery waits for the lock.
+ var recoveryErr error
+ waiting:
+ for {
+ select {
+ case recoveryErr = <-recovered:
+ break waiting
+ case <-session.captureChanged:
+ if next := session.currentStream(); next != nil && next != current {
+ if recoveryErr = changeStream(next); recoveryErr != nil {
+ session.cancel()
+ <-recovered
+ break waiting
+ }
+ }
+ case <-session.ctx.Done():
+ recoveryErr = session.ctx.Err()
+ <-recovered
+ break waiting
+ }
+ }
+ if recoveryErr != nil {
+ return errors.Join(err, recoveryErr)
}
}
- return fail(errors.New("native capture stopped unexpectedly"))
}
func configureCaptureOutputs(source *mediaedge.Source, outputs []nativecapture.OutputProfile) error {
diff --git a/internal/app/nativehost/session_test.go b/internal/app/nativehost/session_test.go
index ce34e6c7..0acf1a8d 100644
--- a/internal/app/nativehost/session_test.go
+++ b/internal/app/nativehost/session_test.go
@@ -8,16 +8,19 @@ import (
"io"
"log/slog"
"os"
+ "os/exec"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
+ "sync/atomic"
"testing"
"time"
"github.com/TNTcraftHIM/Piik/internal/app/mediaedge"
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
+ "github.com/pion/webrtc/v4"
)
func TestMain(tests *testing.M) {
@@ -29,6 +32,16 @@ func TestMain(tests *testing.M) {
writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameLayerUnavailable, Layer: value, Data: []byte("fixture")})
}
}
+ if failure := os.Getenv("PIIK_CAPTURE_LIVE_FAILURE"); failure != "" {
+ if failure == "control" {
+ _ = os.Stdin.Close()
+ writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameBegin, Timestamp: time.Second, Duration: time.Second / 30})
+ // A final buffered frame can reach the reader before process exit is joined.
+ time.Sleep(50 * time.Millisecond)
+ }
+ _, _ = io.WriteString(os.Stderr, "stage=windows-runtime\nhresult=0x887a0005\ndetail=private capture detail\n")
+ os.Exit(2)
+ }
_, _ = io.Copy(io.Discard, os.Stdin)
os.Exit(0)
}
@@ -52,6 +65,15 @@ type retiringCapture struct {
entered, release chan struct{}
}
+type videoCaptureFunc struct {
+ processCapture
+ video func(context.Context, nativecapture.VideoOptions) (CaptureStream, error)
+}
+
+func (capture videoCaptureFunc) Video(ctx context.Context, options nativecapture.VideoOptions) (CaptureStream, error) {
+ return capture.video(ctx, options)
+}
+
func (capture retiringCapture) Video(ctx context.Context, options nativecapture.VideoOptions) (CaptureStream, error) {
stream, err := capture.processCapture.Video(ctx, options)
if err != nil {
@@ -331,6 +353,304 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
}
}
+func TestCaptureFailurePreservesProcessCause(t *testing.T) {
+ for _, mode := range []string{"read", "control"} {
+ t.Run(mode, func(t *testing.T) {
+ t.Setenv("PIIK_NATIVEHOST_PIPE_FIXTURE", "backend")
+ t.Setenv("PIIK_CAPTURE_LIVE_FAILURE", mode)
+ t.Setenv("PIIK_CAPTURE_STARTING", string(captureStatePayload(t,
+ `{"state":"starting","codec":"vp8","adapterIndex":0,"adapterName":"Fixture","adapterIdentity":"0:0","encoderName":"VP8","encoderIdentity":"vp8"}`)))
+ t.Setenv("PIIK_CAPTURE_ACTIVE", string(captureStatePayload(t, `{"state":"active","codec":"vp8","width":1280,"height":720,"fps":30}`)))
+ executable, err := os.Executable()
+ if err != nil {
+ t.Fatal(err)
+ }
+ options := nativecapture.VideoOptions{Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"}, Codec: "vp8",
+ Profile: nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"}, OutputGroups: 4}
+ stream, err := nativecapture.StartVideo(t.Context(), executable, options)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer stream.Close()
+ engine, err := mediaedge.NewEngine(mediaedge.EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer engine.Close()
+ preparationFailed := errors.New("replacement could not start")
+ session := &Session{ctx: t.Context(), engine: engine, stream: stream, videoOptions: options,
+ capture: videoCaptureFunc{video: func(context.Context, nativecapture.VideoOptions) (CaptureStream, error) {
+ return nil, preparationFailed
+ }}, edgeCapacity: 4, ready: make(chan error, 1), events: func(context.Context, Event) {}}
+ err = session.runVideo()
+ if session.source != nil {
+ defer session.source.Close()
+ }
+ if err == nil || !strings.Contains(err.Error(), "windows-runtime") || !strings.Contains(err.Error(), "0x887a0005") || (mode == "read" && !errors.Is(err, io.EOF)) {
+ t.Fatalf("live capture lost its read/process failure: %v", err)
+ }
+ if mode == "control" && errors.Is(err, io.EOF) {
+ t.Fatal("control fixture ended through the reader instead of its closed input pipe")
+ }
+ failure := nativecapture.FailureDetails(err)
+ if failure == nil || failure.Stage != "windows-runtime" || failure.HRESULT != "0x887a0005" || failure.ExitCode != 2 {
+ t.Fatalf("failure event lost its typed helper codes: %+v", failure)
+ }
+ var exit *exec.ExitError
+ if !errors.As(err, &exit) || exit.ExitCode() != 2 || strings.Contains(err.Error(), "private") {
+ t.Fatalf("live capture changed or exposed its process failure: %v", err)
+ }
+ })
+ }
+}
+
+func TestDeviceLossRecoversWithRetainedMedia(t *testing.T) {
+ for _, mode := range []string{"read", "control"} {
+ t.Run(mode, func(t *testing.T) {
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ t.Setenv("PIIK_NATIVEHOST_PIPE_FIXTURE", "audio-recovery")
+ failure := filepath.Join(t.TempDir(), "device-failure")
+ t.Setenv("PIIK_VIDEO_DEVICE_FAILURE", failure)
+ if mode == "control" {
+ t.Setenv("PIIK_VIDEO_DEVICE_CONTROL", "1")
+ }
+ executable, err := os.Executable()
+ check(err)
+ ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
+ defer cancel()
+ events := make(chan Event, 64)
+ var active atomic.Uint32
+ video := nativecapture.VideoOptions{Codec: "vp8", Profile: audioRecoveryProfile(),
+ Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Device fixture"}}
+ session, err := Start(ctx, Options{ShareID: "device-recovery", CaptureProcess: executable,
+ Video: video, Profile: QualityProfile{Video: video.Profile, AudioBitrate: 64_000},
+ AudioEnabled: true, EdgeCapacity: 1, BindAddress: "127.0.0.1:0", Events: func(_ context.Context, event Event) {
+ if event.Type == "capture-state" && event.State == "active" {
+ active.Add(1)
+ }
+ select {
+ case events <- event:
+ case <-ctx.Done():
+ }
+ }})
+ check(err)
+ defer session.Close()
+ settings := webrtc.SettingEngine{}
+ settings.SetIncludeLoopbackCandidate(true)
+ settings.SetNetworkTypes([]webrtc.NetworkType{webrtc.NetworkTypeUDP4})
+ receiver, err := webrtc.NewAPI(webrtc.WithSettingEngine(settings)).NewPeerConnection(webrtc.Configuration{})
+ check(err)
+ defer receiver.Close()
+ var audioPackets, videoPackets, videoTracks atomic.Uint32
+ var clockBroken atomic.Bool
+ receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
+ videoTrack := track.Kind() == webrtc.RTPCodecTypeVideo
+ if videoTrack {
+ videoTracks.Add(1)
+ }
+ var previous uint32
+ for {
+ packet, _, err := track.ReadRTP()
+ if err != nil {
+ return
+ }
+ if videoTrack {
+ if previous != 0 && int32(packet.Timestamp-previous) <= 0 {
+ clockBroken.Store(true)
+ }
+ previous = packet.Timestamp
+ videoPackets.Add(1)
+ } else {
+ audioPackets.Add(1)
+ }
+ }
+ })
+ offer, err := session.PrepareLocalEdge("device-edge", "device-offer", false)
+ check(err)
+ check(receiver.SetRemoteDescription(offer))
+ for _, transceiver := range receiver.GetTransceivers() {
+ if transceiver.Kind() != webrtc.RTPCodecTypeAudio {
+ continue
+ }
+ for _, codec := range transceiver.Receiver().GetParameters().Codecs {
+ if codec.MimeType == webrtc.MimeTypeOpus {
+ codec.SDPFmtpLine = "stereo=1;maxaveragebitrate=510000"
+ check(transceiver.SetCodecPreferences([]webrtc.RTPCodecParameters{codec}))
+ break
+ }
+ }
+ }
+ go func() {
+ for {
+ select {
+ case event := <-events:
+ if event.Type == "edge-candidate" && event.Candidate != nil {
+ _ = receiver.AddICECandidate(*event.Candidate)
+ }
+ case <-ctx.Done():
+ return
+ }
+ }
+ }()
+ gathered := webrtc.GatheringCompletePromise(receiver)
+ answer, err := receiver.CreateAnswer(nil)
+ check(err)
+ check(receiver.SetLocalDescription(answer))
+ select {
+ case <-gathered:
+ case <-ctx.Done():
+ t.Fatal("receiver gathering timed out")
+ }
+ check(session.SetAnswer("device-edge", *receiver.LocalDescription(), "device-offer"))
+ wait := func(condition func() bool, label string) {
+ t.Helper()
+ for !condition() {
+ select {
+ case <-ctx.Done():
+ t.Fatal(label)
+ case <-session.Done():
+ t.Fatal("share ended during", label)
+ case <-time.After(10 * time.Millisecond):
+ }
+ }
+ }
+ wait(func() bool { return audioPackets.Load() >= 3 && videoPackets.Load() >= 3 }, "initial media")
+ stream, audioStream := session.currentStream(), session.currentAudioStream()
+ source, audioSource, edge := session.source, session.audioSource, session.edge("device-edge")
+ session.mu.Lock()
+ selectedOptions := session.videoOptions
+ session.mu.Unlock()
+ beforeAudio := audioPackets.Load()
+ check(os.WriteFile(failure, nil, 0600))
+ wait(func() bool { return active.Load() >= 2 }, "replacement installation")
+ afterVideo := videoPackets.Load()
+ wait(func() bool { return audioPackets.Load() >= beforeAudio+5 && videoPackets.Load() >= afterVideo+5 }, "media after device recovery")
+ if session.currentStream() == stream || session.currentAudioStream() != audioStream || session.source != source ||
+ session.audioSource != audioSource || session.edge("device-edge") != edge || videoTracks.Load() != 1 || clockBroken.Load() {
+ t.Fatal("device recovery replaced media owners or reset the receiver clock")
+ }
+ session.mu.Lock()
+ actualOptions := session.videoOptions
+ session.mu.Unlock()
+ if actualOptions != selectedOptions {
+ t.Fatal("recovery changed capture source/profile/backend")
+ }
+ })
+ }
+}
+
+func TestDeviceLossRecoveryYieldsToCaptureOperations(t *testing.T) {
+ for _, mode := range []string{"profile", "stop"} {
+ t.Run(mode, func(t *testing.T) {
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ t.Setenv("PIIK_NATIVEHOST_PIPE_FIXTURE", "audio-recovery")
+ failure := filepath.Join(t.TempDir(), "device-failure")
+ t.Setenv("PIIK_VIDEO_DEVICE_FAILURE", failure)
+ executable, err := os.Executable()
+ check(err)
+ ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+ defer cancel()
+ entered, release, finished := make(chan struct{}), make(chan struct{}), make(chan struct{})
+ unblock := sync.OnceFunc(func() { close(release) })
+ defer unblock()
+ var starts atomic.Uint32
+ capture := videoCaptureFunc{processCapture: processCapture{executable}}
+ capture.video = func(ctx context.Context, options nativecapture.VideoOptions) (CaptureStream, error) {
+ if starts.Add(1) == 2 {
+ close(entered)
+ defer close(finished)
+ select {
+ case <-release:
+ case <-ctx.Done():
+ return nil, ctx.Err()
+ }
+ }
+ return capture.processCapture.Video(ctx, options)
+ }
+ recovering := make(chan struct{}, 1)
+ previousLog := slog.Default()
+ defer slog.SetDefault(previousLog)
+ slog.SetDefault(slog.New(slog.NewTextHandler(io.Discard, &slog.HandlerOptions{Level: slog.LevelDebug,
+ ReplaceAttr: func(_ []string, attr slog.Attr) slog.Attr {
+ if attr.Key == "event" && attr.Value.String() == "capture-device-recovering" {
+ select {
+ case recovering <- struct{}{}:
+ default:
+ }
+ }
+ return attr
+ }})))
+ video := nativecapture.VideoOptions{Codec: "vp8", Profile: audioRecoveryProfile(),
+ Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Operation fixture"}}
+ session, err := Start(ctx, Options{ShareID: "device-operation", Capture: capture,
+ Video: video, Profile: QualityProfile{Video: video.Profile, AudioBitrate: 64_000}, EdgeCapacity: 1, BindAddress: "127.0.0.1:0"})
+ check(err)
+ defer session.Close()
+ completed := make(chan error, 1)
+ next := QualityProfile{Video: video.Profile, AudioBitrate: 64_000}
+ next.Video.Width, next.Video.Height = 640, 360
+ if mode == "profile" {
+ go func() { completed <- session.UpdateProfile(next) }()
+ select {
+ case <-entered:
+ case <-ctx.Done():
+ t.Fatal("profile preparation did not start")
+ }
+ }
+ check(os.WriteFile(failure, nil, 0600))
+ select {
+ case <-recovering:
+ case <-ctx.Done():
+ t.Fatal("reader did not identify device loss")
+ }
+ if mode == "stop" {
+ select {
+ case <-entered:
+ case <-ctx.Done():
+ t.Fatal("recovery preparation did not start")
+ }
+ go func() { completed <- session.Close() }()
+ } else {
+ unblock()
+ }
+ select {
+ case err := <-completed:
+ check(err)
+ case <-ctx.Done():
+ t.Fatal("capture operation deadlocked with its reader")
+ }
+ select {
+ case <-finished:
+ default:
+ t.Fatal("capture operation returned before preparation retired")
+ }
+ if starts.Load() != 2 {
+ t.Fatal("recovery replaced a newer user operation")
+ }
+ if mode == "profile" {
+ if session.profile != next || session.videoOptions.Profile != next.Video {
+ t.Fatal("recovery overwrote the user's profile")
+ }
+ select {
+ case <-session.Done():
+ t.Fatal("profile update ended the share")
+ default:
+ }
+ }
+ })
+ }
+}
+
func TestCaptureFailureBelongsToSelectedStream(t *testing.T) {
for _, replace := range []bool{false, true} {
t.Run(strconv.FormatBool(replace), func(t *testing.T) {
diff --git a/internal/app/remoteinput/diagnostics.go b/internal/app/remoteinput/diagnostics.go
new file mode 100644
index 00000000..372a0c1c
--- /dev/null
+++ b/internal/app/remoteinput/diagnostics.go
@@ -0,0 +1,87 @@
+package remoteinput
+
+// Incoming events and sink Apply calls are separate counts: held-input
+// deduplication can avoid a call, while text and release cleanup can make several.
+// API acceptance is Apply's return value, never proof of OS delivery. Reset
+// calls are not expanded into inferred per-key/button API operations.
+type DiagnosticFamilyCounters struct {
+ Parsed uint64 `json:"parsed"`
+ Authorized uint64 `json:"authorized"`
+ TargetReady uint64 `json:"targetReady"`
+ APIAttempts uint64 `json:"apiAttempts"`
+ APIAccepted uint64 `json:"apiAccepted"`
+ APIRejected uint64 `json:"apiRejected"`
+}
+
+// ReceivedPackets counts current, known-endpoint payloads, not raw channel traffic.
+// Packet type is unknown before decryption and parsing. Those totals include
+// control/gamepad messages; family counters cover keyboard and mouse actions.
+type DiagnosticCounters struct {
+ ReceivedPackets uint64 `json:"receivedPackets"`
+ DecryptedPackets uint64 `json:"decryptedPackets"`
+ ParsedEvents uint64 `json:"parsedEvents"`
+ Mouse DiagnosticFamilyCounters `json:"mouse"`
+ Keyboard DiagnosticFamilyCounters `json:"keyboard"`
+}
+
+type DiagnosticSnapshot struct {
+ Counters DiagnosticCounters `json:"counters"`
+ ActiveGrants int `json:"activeGrants"`
+ SuspendedGrants int `json:"suspendedGrants"`
+ PendingKeyReleases int `json:"pendingKeyReleases"`
+ PendingButtonReleases int `json:"pendingButtonReleases"`
+}
+
+// Diagnostics contains counts only, retained for this Manager's lifetime.
+// It returns a value before the report writer acquires any of its locks.
+func (m *Manager) Diagnostics() DiagnosticSnapshot {
+ if m == nil {
+ return DiagnosticSnapshot{}
+ }
+ m.mu.Lock()
+ defer m.mu.Unlock()
+ result := DiagnosticSnapshot{
+ Counters: m.diagnostics,
+ PendingKeyReleases: len(m.pendingKeys),
+ PendingButtonReleases: len(m.pendingButtons),
+ }
+ for _, c := range m.connections {
+ if c.grant == "" {
+ continue
+ }
+ if c.suspended {
+ result.SuspendedGrants++
+ } else {
+ result.ActiveGrants++
+ }
+ }
+ return result
+}
+
+// Caller holds the Manager lock, as for the input operation itself.
+func (m *Manager) inputFamily(kind string) *DiagnosticFamilyCounters {
+ switch kind {
+ case "move", "relative", "button", "wheel":
+ return &m.diagnostics.Mouse
+ case "key", "text":
+ return &m.diagnostics.Keyboard
+ default:
+ return nil
+ }
+}
+
+func (m *Manager) applyInput(b bounds, event inputEvent) error {
+ family := m.inputFamily(event.Type)
+ if family != nil {
+ family.APIAttempts++
+ }
+ err := m.sink.Apply(b, event)
+ if family != nil {
+ if err == nil {
+ family.APIAccepted++
+ } else {
+ family.APIRejected++
+ }
+ }
+ return err
+}
diff --git a/internal/app/remoteinput/held.go b/internal/app/remoteinput/held.go
index 7bcba947..97dddbad 100644
--- a/internal/app/remoteinput/held.go
+++ b/internal/app/remoteinput/held.go
@@ -47,7 +47,7 @@ func (m *Manager) resetHeld(s *Session) error {
// Failed releases outlive their departing controller. Only releases are retried;
// pointer movement, presses and committed text are never replayed.
func (m *Manager) releaseKey(code string) error {
- err := m.sink.Apply(bounds{}, inputEvent{Type: "key", Code: code})
+ err := m.applyInput(bounds{}, inputEvent{Type: "key", Code: code})
if err != nil {
if m.pendingKeys == nil {
m.pendingKeys = make(map[string]bool)
@@ -60,7 +60,7 @@ func (m *Manager) releaseKey(code string) error {
}
func (m *Manager) releaseButton(button int) error {
- err := m.sink.Apply(bounds{}, inputEvent{Type: "button", Button: button})
+ err := m.applyInput(bounds{}, inputEvent{Type: "button", Button: button})
if err != nil {
if m.pendingButtons == nil {
m.pendingButtons = make(map[int]bool)
@@ -109,7 +109,7 @@ func (m *Manager) applyHeld(c *connection, b bounds, event inputEvent) error {
if event.Down {
// A repeat from this holder is input; another holder shares the press.
if c.keys[event.Code] || !m.keyHeld(event.Code) {
- if err := m.sink.Apply(b, event); err != nil {
+ if err := m.applyInput(b, event); err != nil {
return err
}
}
@@ -132,7 +132,7 @@ func (m *Manager) applyHeld(c *connection, b bounds, event inputEvent) error {
}
if event.Down {
if !m.buttonHeld(event.Button) {
- if err := m.sink.Apply(b, event); err != nil {
+ if err := m.applyInput(b, event); err != nil {
return err
}
}
@@ -147,7 +147,7 @@ func (m *Manager) applyHeld(c *connection, b bounds, event inputEvent) error {
}
}
default:
- return m.sink.Apply(b, event)
+ return m.applyInput(b, event)
}
return nil
}
diff --git a/internal/app/remoteinput/manager.go b/internal/app/remoteinput/manager.go
index 231f8784..0dccfbd6 100644
--- a/internal/app/remoteinput/manager.go
+++ b/internal/app/remoteinput/manager.go
@@ -69,6 +69,7 @@ type Manager struct {
gamepads gamepadBackend
gamepadOpening bool
connections map[endpointKey]*connection
+ diagnostics DiagnosticCounters
sessions map[*Session]struct{}
owner *Session // The one source allowed to inject into the desktop.
closed bool
@@ -320,10 +321,12 @@ func (s *Session) receivePacket(edgeID string, packet inputroute.Packet, current
if current != nil && !current() {
return
}
+ m.diagnostics.ReceivedPackets++
plain, err := c.cipher.open(packet.Data, packet.Epoch)
if err != nil {
return
}
+ m.diagnostics.DecryptedPackets++
if c.edgeID != "" && (c.edgeID != edgeID || c.epoch != packet.Epoch) {
m.revoke(c)
}
@@ -555,6 +558,11 @@ func (m *Manager) receiveLocked(c *connection, plain []byte) {
if decoder.Decode(&event) != nil || decoder.Decode(&struct{}{}) != io.EOF {
return
}
+ m.diagnostics.ParsedEvents++
+ family := m.inputFamily(event.Type)
+ if family != nil {
+ family.Parsed++
+ }
switch event.Type {
case "sync":
m.send(c, m.state(c))
@@ -601,6 +609,9 @@ func (m *Manager) receiveLocked(c *connection, plain []byte) {
if c.grant == "" || event.Grant != c.grant || !c.owner.available() {
return
}
+ if family != nil {
+ family.Authorized++
+ }
if event.Type == "gamepad" || event.Type == "gamepad-reset" || event.Type == "gamepad-stop" {
m.receiveGamepad(c, event)
return
@@ -621,6 +632,9 @@ func (m *Manager) receiveLocked(c *connection, plain []byte) {
if !focused {
return
}
+ if family != nil {
+ family.TargetReady++
+ }
var err error
if event.Type == "text" {
err = applyText(controllerTextSink{m, c}, b, event.Text)
diff --git a/internal/app/remoteinput/manager_test.go b/internal/app/remoteinput/manager_test.go
index ae040386..623c4cee 100644
--- a/internal/app/remoteinput/manager_test.go
+++ b/internal/app/remoteinput/manager_test.go
@@ -1,6 +1,7 @@
package remoteinput
import (
+ "archive/zip"
"bytes"
"context"
"crypto/rand"
@@ -9,6 +10,7 @@ import (
"encoding/hex"
"encoding/json"
"errors"
+ "io"
"log/slog"
"strings"
"sync/atomic"
@@ -22,6 +24,114 @@ import (
"github.com/pion/webrtc/v4"
)
+func TestInputDiagnosticsDistinguishStagesWithoutInputValues(t *testing.T) {
+ sink := &recordedInput{events: make(chan inputEvent, 32), resets: make(chan struct{}, 16), focused: true}
+ m := &Manager{sink: sink, gamepads: &recordedGamepads{}, connections: map[endpointKey]*connection{}}
+ s, peers := coPlaySession(t, m, true)
+ c := peers[0]
+ packet := func(plain []byte) {
+ header := make([]byte, 20)
+ binary.BigEndian.PutUint64(header, c.cipher.received+1)
+ _, _ = rand.Read(header[8:])
+ data := c.cipher.aead.Seal(header, header[8:], plain, inputAAD(0, header[:8], c.epoch))
+ s.receivePacket(c.edgeID, inputroute.Packet{ID: c.id, Epoch: c.epoch, Data: base64.StdEncoding.EncodeToString(data)}, nil)
+ }
+ input := func(event inputEvent) {
+ if event.Grant == "" {
+ event.Grant = c.grant
+ }
+ data, _ := json.Marshal(event)
+ packet(data)
+ }
+ before := m.Diagnostics()
+ s.receivePacket(c.edgeID, inputroute.Packet{ID: c.id, Epoch: c.epoch, Data: "not-base64"}, nil)
+ packet([]byte("not-json"))
+ input(inputEvent{Type: "request", Intent: strings.Repeat("f", 32)})
+ input(inputEvent{Type: "move", Grant: "wrong-grant", X: .37, Y: .29})
+ sink.focused = false
+ input(inputEvent{Type: "move", X: .37, Y: .29})
+ suspended := m.Diagnostics()
+ if suspended.ActiveGrants != 0 || suspended.SuspendedGrants != 1 || len(sink.events) != 0 ||
+ suspended.Counters.Mouse != (DiagnosticFamilyCounters{Parsed: 2, Authorized: 1}) {
+ t.Fatalf("quiet target suspension lost its stage: %+v", suspended)
+ }
+ sink.focused = true
+ input(inputEvent{Type: "move", X: .37, Y: .29})
+ sink.applyErr = errors.New("test sink rejection")
+ input(inputEvent{Type: "button", Down: true})
+ sink.applyErr = nil
+ input(inputEvent{Type: "button", Down: true})
+ input(inputEvent{Type: "button", Down: true}) // Held deduplication is not another API acceptance.
+ input(inputEvent{Type: "key", Code: "KeyZ", Down: true})
+ accepted := m.Diagnostics()
+ if accepted.Counters.ReceivedPackets-before.Counters.ReceivedPackets != 10 ||
+ accepted.Counters.DecryptedPackets-before.Counters.DecryptedPackets != 9 ||
+ accepted.Counters.ParsedEvents-before.Counters.ParsedEvents != 8 ||
+ accepted.Counters.Mouse != (DiagnosticFamilyCounters{Parsed: 6, Authorized: 5, TargetReady: 4, APIAttempts: 3, APIAccepted: 2, APIRejected: 1}) ||
+ accepted.Counters.Keyboard != (DiagnosticFamilyCounters{Parsed: 1, Authorized: 1, TargetReady: 1, APIAttempts: 1, APIAccepted: 1}) ||
+ accepted.ActiveGrants != 1 || accepted.SuspendedGrants != 0 {
+ t.Fatalf("ingress, authorization and sink API results were conflated: %+v", accepted)
+ }
+ sink.applyErr = errors.New("test release rejection")
+ input(inputEvent{Type: "key", Code: "KeyZ"})
+ input(inputEvent{Type: "button"})
+ pending := m.Diagnostics()
+ if pending.PendingKeyReleases != 1 || pending.PendingButtonReleases != 1 {
+ t.Fatalf("release counts missing: %+v", pending)
+ }
+ // Export the actual bounded report context. The snapshot must not retry the
+ // rejected cleanup or expose the held input, credentials, or connection IDs.
+ events, grant := len(sink.events), c.grant
+ recorder, err := diagnostics.Open(t.TempDir(), "client", "test")
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { _ = recorder.Close() })
+ recorder.Context("remoteInput", m.Diagnostics())
+ path, err := recorder.Export()
+ if err != nil {
+ t.Fatal(err)
+ }
+ archive, err := zip.OpenReader(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer archive.Close()
+ var content []byte
+ for _, file := range archive.File {
+ if file.Name == "context.json" {
+ reader, err := file.Open()
+ if err != nil {
+ t.Fatal(err)
+ }
+ content, err = io.ReadAll(reader)
+ reader.Close()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ }
+ var context struct {
+ RemoteInput DiagnosticSnapshot `json:"remoteInput"`
+ }
+ if err := json.Unmarshal(content, &context); err != nil || context.RemoteInput != pending {
+ t.Fatalf("ZIP omitted the current input counts: %s, %v", content, err)
+ }
+ for _, private := range []string{grant, c.secret, c.id, c.peerID, "KeyZ", `"x":`, `"y":`, `"text":`, `"code":`} {
+ if strings.Contains(string(content), private) {
+ t.Fatalf("diagnostic context contains input values or identity: %q", private)
+ }
+ }
+ t.Logf("exported input diagnostic context: %s", content)
+ if m.Diagnostics() != pending || len(sink.events) != events || c.grant != grant {
+ t.Fatal("observing/exporting diagnostics changed input or cleanup state")
+ }
+ sink.applyErr = nil
+ if err := m.retryReleases(); err != nil {
+ t.Fatal(err)
+ }
+}
+
type recordedInput struct {
authorize func() error
events chan inputEvent
diff --git a/internal/media/forwarding/encoded_source.go b/internal/media/forwarding/encoded_source.go
index 6b7c9f6e..12e8f0f3 100644
--- a/internal/media/forwarding/encoded_source.go
+++ b/internal/media/forwarding/encoded_source.go
@@ -89,6 +89,11 @@ func (input *EncodedSource) WriteFrame(layer int, frame encoded.Frame) error {
frame.PTS < input.anchorPTS || frame.PTS > input.inputPTS {
return errors.New("encoded output has no matching source input")
}
+ // Buffers and packetizers survive input replacement, but each new encoder
+ // generation must supply an independently decodable frame for every layer.
+ if input.rtpBytes[layer] == 0 && !frame.Recovery {
+ return nil
+ }
sourceTime := input.anchorTime.Add(frame.PTS - input.anchorPTS)
packets, err := input.packetizers[layer].Packetize(frame.Data, sourceTime.Sub(input.originTime), frame.Duration)
if err != nil {
diff --git a/internal/media/forwarding/encoded_source_test.go b/internal/media/forwarding/encoded_source_test.go
index 017e5f94..7b5d1395 100644
--- a/internal/media/forwarding/encoded_source_test.go
+++ b/internal/media/forwarding/encoded_source_test.go
@@ -43,7 +43,7 @@ func TestCodecBudgetUsesActualSourceFraming(t *testing.T) {
for layer, size := range []int{80, 4000} {
data := make([]byte, size)
copy(data, sfu.VP8KeyFrame8x8)
- if err = input.WriteFrame(layer, encoded.Frame{Data: data, PTS: pts, Duration: period}); err != nil {
+ if err = input.WriteFrame(layer, encoded.Frame{Data: data, PTS: pts, Duration: period, Recovery: true}); err != nil {
t.Fatal(err)
}
}
@@ -86,7 +86,7 @@ func TestEncodedSourceH264MetadataPreservesCorrelation(t *testing.T) {
if err := input.BeginFrame(pts, origin.Add(pts)); err != nil {
t.Fatal(err)
}
- if err := input.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: period}); err != nil {
+ if err := input.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: period, Recovery: index == 0}); err != nil {
t.Fatal(err)
}
}
@@ -130,3 +130,64 @@ func TestEncodedSourceH264MetadataPreservesCorrelation(t *testing.T) {
t.Fatal("SEI was incorrectly treated as packetless metadata")
}
}
+
+func TestEncodedSourceGenerationStartsEachLayerWithRecovery(t *testing.T) {
+ input, err := NewEncodedSource(clockSourceOptions("generation-recovery"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer input.Close()
+ check := func(err error) {
+ t.Helper()
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ delta := append([]byte(nil), sfu.VP8KeyFrame8x8...)
+ delta[0] |= 1
+ const period = 40 * time.Millisecond
+ origin := time.Now()
+ for generation := range 2 {
+ if generation != 0 {
+ input.BeginGeneration()
+ }
+ at := origin.Add(time.Duration(generation) * 5 * time.Second)
+ packets := append([]uint32(nil), input.packets...)
+ reports := []proto.Message{input.GetAllBuffers()[0].GetSenderReportData(), input.GetAllBuffers()[1].GetSenderReportData()}
+ write := func(layer int, pts time.Duration, recovery bool) {
+ t.Helper()
+ data := delta
+ if recovery {
+ data = sfu.VP8KeyFrame8x8
+ }
+ check(input.WriteFrame(layer, encoded.Frame{Data: data, PTS: pts, Duration: period, Recovery: recovery}))
+ }
+ check(input.BeginFrame(0, at))
+ for layer := range 2 {
+ write(layer, 0, false)
+ if input.packets[layer] != packets[layer] || input.rtpBytes[layer] != 0 || input.codecBytes[layer] != 0 ||
+ !proto.Equal(reports[layer], input.GetAllBuffers()[layer].GetSenderReportData()) {
+ t.Fatal("delta crossed a fresh generation or advanced its clock/cost", generation, layer)
+ }
+ }
+ check(input.BeginFrame(period, at.Add(period)))
+ write(0, period, true)
+ write(1, period, false)
+ if input.packets[0] != packets[0]+1 || input.packets[1] != packets[1] {
+ t.Fatal("one layer's recovery admitted another layer's delta")
+ }
+ check(input.BeginFrame(2*period, at.Add(2*period)))
+ write(0, 2*period, false)
+ write(1, 2*period, true)
+ if input.packets[0] != packets[0]+2 || input.packets[1] != packets[1]+1 {
+ t.Fatal("recovered layer did not resume normal frames")
+ }
+ for layer := range 2 {
+ report := input.GetAllBuffers()[layer].GetSenderReportData()
+ captured, ok := input.ReferenceTime(layer, report.RtpTimestamp)
+ if !ok || captured.Sub(at.Add(2*period)).Abs() > time.Microsecond {
+ t.Fatal("recovery changed the source clock")
+ }
+ }
+ }
+}
diff --git a/internal/media/forwarding/publication_recovery_test.go b/internal/media/forwarding/publication_recovery_test.go
index f05821c4..39699557 100644
--- a/internal/media/forwarding/publication_recovery_test.go
+++ b/internal/media/forwarding/publication_recovery_test.go
@@ -193,7 +193,7 @@ func checkPublicationRecovery(t *testing.T, scenario string) {
data := make([]byte, max(len(sfu.VP8KeyFrame8x8)+8, int(bitrate/8/30)))
copy(data, sfu.VP8KeyFrame8x8)
binary.BigEndian.PutUint64(data[len(data)-8:], uint64(time.Now().UnixNano()))
- check(source.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: period}))
+ check(source.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: period, Recovery: true}))
frame++
if now.Sub(lastSample) < time.Second {
continue
diff --git a/internal/media/forwarding/recovery_test.go b/internal/media/forwarding/recovery_test.go
index 6ca55de9..4a65e83d 100644
--- a/internal/media/forwarding/recovery_test.go
+++ b/internal/media/forwarding/recovery_test.go
@@ -268,7 +268,7 @@ func checkTransportRecovery(t *testing.T, scenario string) {
}
data := make([]byte, max(len(sfu.VP8KeyFrame8x8), int(source.CodecBudget(layer, bitrate)/8/30)))
copy(data, sfu.VP8KeyFrame8x8)
- check(source.WriteFrame(layer, encoded.Frame{Data: data, PTS: pts, Duration: period}))
+ check(source.WriteFrame(layer, encoded.Frame{Data: data, PTS: pts, Duration: period, Recovery: true}))
}
frame++
if now.Sub(lastReport) >= time.Second {
diff --git a/internal/media/forwarding/source_clock_test.go b/internal/media/forwarding/source_clock_test.go
index 6de2d4ea..22333630 100644
--- a/internal/media/forwarding/source_clock_test.go
+++ b/internal/media/forwarding/source_clock_test.go
@@ -72,7 +72,7 @@ func TestEncodedSourceClockIgnoresEncoderDelay(t *testing.T) {
}
write := func(layer int, pts time.Duration) {
t.Helper()
- check(input.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 30}))
+ check(input.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 30, Recovery: true}))
}
// The first low output took 500 ms; the high output is current. Both carry
// their correct capture PTS, and later low output catches up completely.
@@ -132,7 +132,7 @@ func TestDerivedSourceSharesRemoteClock(t *testing.T) {
t.Fatal("original reference is absent")
}
check(derived.BeginFrame(pts, captured))
- check(derived.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 30}))
+ check(derived.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 30, Recovery: true}))
high := projectedSourceClock(t, original, 1, &stats[1])
low := projectedSourceClock(t, derived.Source, 0, &stats[0])
if mismatch := low - high; mismatch.Abs() > time.Millisecond {
diff --git a/internal/media/forwarding/source_registration_test.go b/internal/media/forwarding/source_registration_test.go
index 67c35a54..ebaef8ea 100644
--- a/internal/media/forwarding/source_registration_test.go
+++ b/internal/media/forwarding/source_registration_test.go
@@ -52,7 +52,7 @@ func TestSourceRegistrationRetainsCurrentClock(t *testing.T) {
t.Cleanup(input.Close)
check(input.BeginFrame(0, at))
for layer := 0; layer < 2; layer++ {
- check(input.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, Duration: time.Second / 30}))
+ check(input.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, Duration: time.Second / 30, Recovery: true}))
}
return input
}
@@ -104,7 +104,7 @@ func TestSourceRegistrationRetainsCurrentClock(t *testing.T) {
updated <- err
return
}
- updated <- input.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: 33 * time.Millisecond, Duration: time.Second / 30})
+ updated <- input.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: 33 * time.Millisecond, Duration: time.Second / 30, Recovery: true})
}()
wait(held.live)
unblock()
diff --git a/internal/media/forwarding/warm_source_test.go b/internal/media/forwarding/warm_source_test.go
index 9143eb2b..4670d4b1 100644
--- a/internal/media/forwarding/warm_source_test.go
+++ b/internal/media/forwarding/warm_source_test.go
@@ -56,7 +56,7 @@ func TestTransportBootstrapsWarmRawSource(t *testing.T) {
producerError <- err
return
}
- if err := source.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: time.Second / 30}); err != nil {
+ if err := source.WriteFrame(0, encoded.Frame{Data: data, PTS: pts, Duration: time.Second / 30, Recovery: true}); err != nil {
producerError <- err
return
}
diff --git a/mobile/bridge/runtime.go b/mobile/bridge/runtime.go
index 81a3fcfc..3342833f 100644
--- a/mobile/bridge/runtime.go
+++ b/mobile/bridge/runtime.go
@@ -96,6 +96,7 @@ func Start(platform Platform, lanAddress string, codecCapabilities string, dataD
mediaCapabilities := catalog.mediaCapabilities()
mediaCapabilities.Microphone = capabilities.Microphone
mediaCapabilities.SourceCancellation = true
+ mediaCapabilities.CaptureFailureDetails = true
r.control, err = loopback.Start(ctx, loopback.Options{
ActivationURL: activationURL,
MaxControlSessions: 1,
diff --git a/native/capture/windows/main.cpp b/native/capture/windows/main.cpp
index 6e3a8e0c..ecd1357a 100644
--- a/native/capture/windows/main.cpp
+++ b/native/capture/windows/main.cpp
@@ -2280,7 +2280,9 @@ void RunVideoCapture(ProductArguments arguments) {
submit(input);
}
} catch (...) {
+ const HRESULT removed = device.device->GetDeviceRemovedReason();
cleanup();
+ Check(removed, "capture-device-lost");
throw;
}
}
diff --git a/scripts/embedded-sfu-gate.ts b/scripts/embedded-sfu-gate.ts
index b1cb668b..8eff42d2 100644
--- a/scripts/embedded-sfu-gate.ts
+++ b/scripts/embedded-sfu-gate.ts
@@ -162,7 +162,9 @@ async function main(): Promise<void> {
server.stderr.on("data", (data: Buffer) => { serverError = (serverError + data.toString()).slice(-16000); });
await waitForSample((deadline) => fetchJsonBefore<{ status: string }>(`${backend}/healthz`, deadline),
() => true, 15_000);
- vite = await createViteServer({ root: ROOT, configFile: false, appType: "custom", logLevel: "silent",
+ vite = await createViteServer({ root: ROOT, cacheDir: join(BUILD_ROOT, ".vite"),
+ configFile: false, appType: "custom", logLevel: "silent",
+ optimizeDeps: { entries: ["scripts/embedded-sfu-page.ts"] },
server: { host: "127.0.0.1", port: vitePort, strictPort: true, watch: null,
proxy: { "/api": { target: backend }, "/signal": { target: backend, ws: true } } },
plugins: [{ name: "embedded-sfu-gate", configureServer(instance) {
diff --git a/scripts/network-lab/network.ts b/scripts/network-lab/network.ts
index 1d3eaa80..acc02558 100644
--- a/scripts/network-lab/network.ts
+++ b/scripts/network-lab/network.ts
@@ -14,6 +14,15 @@ export function command(program: string, args: string[], input?: string): string
maxBuffer: 2 * 1024 * 1024, stdio: ["pipe", "pipe", "pipe"], input }).trim();
}
+function routeState(ipv6 = false): string {
+ const routes: Record<string, unknown>[] = JSON.parse(command("ip", [
+ ...(ipv6 ? ["-6"] : []), "-j", "route", "show", "table", "all",
+ ]));
+ // Kernel route expiry counts down while the host configuration stays unchanged.
+ for (const route of routes) delete route.expires;
+ return JSON.stringify(routes);
+}
+
// Owns only namespaces and veth pairs created by this run. All routing,
// firewall and sysctl changes happen inside those namespaces.
export class LabNetwork {
@@ -22,8 +31,8 @@ export class LabNetwork {
private readonly pendingLinks = new Set<string>();
private readonly shaped = new Set<string>();
private readonly udpAddresses = new Map<number, string>();
- private readonly originalRoutes = command("ip", ["-j", "route", "show", "table", "all"]);
- private readonly originalRoutes6 = command("ip", ["-6", "-j", "route", "show", "table", "all"]);
+ private readonly originalRoutes = routeState();
+ private readonly originalRoutes6 = routeState(true);
private readonly originalRules = command("nft", ["-s", "list", "ruleset"]);
readonly wan = `${this.prefix}-wan`;
@@ -192,8 +201,8 @@ add rule ip lab_nat postrouting oifname "wan0" snat to ${before}
return {
namespacesRemoved: failures.length === 0 && !remaining.includes(this.prefix),
linksRemoved: !JSON.parse(command("ip", ["-j", "link", "show"])).some((link: { ifname: string }) => this.pendingLinks.has(link.ifname)),
- hostNetworkUnchanged: command("ip", ["-j", "route", "show", "table", "all"]) === this.originalRoutes &&
- command("ip", ["-6", "-j", "route", "show", "table", "all"]) === this.originalRoutes6 &&
+ hostNetworkUnchanged: routeState() === this.originalRoutes &&
+ routeState(true) === this.originalRoutes6 &&
command("nft", ["-s", "list", "ruleset"]) === this.originalRules,
};
}
diff --git a/scripts/package-app-candidate.mjs b/scripts/package-app-candidate.mjs
index 48648599..78a1dffb 100644
--- a/scripts/package-app-candidate.mjs
+++ b/scripts/package-app-candidate.mjs
@@ -4,6 +4,7 @@ import { spawn, spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import {
chmodSync,
+ copyFileSync,
createWriteStream,
existsSync,
mkdirSync,
@@ -19,6 +20,7 @@ import { basename, dirname, isAbsolute, join, relative, resolve } from "node:pat
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import { fileURLToPath } from "node:url";
+import { parseArgs } from "node:util";
import { appPackageTarget, appPackageDirectory, CLOUDFLARED_VERSION } from "./app-package-targets.mjs";
import { createZip, extractZip, tarExecutable } from "./archive-tool.mjs";
@@ -187,10 +189,10 @@ async function verifyLocalPackage(root, target, temporaryRoot) {
if (!healthReady) {
try {
const response = await fetch(healthURL, {
- headers: { Connection: "close" },
signal: AbortSignal.timeout(500),
});
- healthReady = response.ok && (await response.json())?.status === "ok";
+ if (response.ok) healthReady = (await response.json())?.status === "ok";
+ else await response.body?.cancel();
} catch {}
}
if (healthReady && appReady) break;
@@ -201,9 +203,9 @@ async function verifyLocalPackage(root, target, temporaryRoot) {
}
// The Web notices ship inside the binary; the embedded assets serve them.
const notices = await fetch(noticeURL, {
- headers: { Connection: "close" },
signal: AbortSignal.timeout(2_000),
});
+ if (!notices.ok) await notices.body?.cancel();
if (!notices.ok || (await notices.text()).length === 0) {
fail("Packaged App did not serve its Web third-party notices");
}
@@ -222,10 +224,10 @@ async function verifyLocalPackage(root, target, temporaryRoot) {
const closeDeadline = Date.now() + 5_000;
while (Date.now() < closeDeadline) {
try {
- await fetch(healthURL, {
- headers: { Connection: "close" },
+ const response = await fetch(healthURL, {
signal: AbortSignal.timeout(300),
});
+ await response.arrayBuffer();
} catch {
return;
}
@@ -309,16 +311,20 @@ function verifyPlatformAssets(root, target) {
}
}
-if (process.argv.length !== 5) {
+const { values, positionals } = parseArgs({
+ args: process.argv.slice(2), allowPositionals: true,
+ options: { "tunnel-asset": { type: "string" } },
+});
+if (positionals.length !== 3) {
fail(
- "Usage: node scripts/package-app-candidate.mjs <server-release-directory> <target> <new-output-directory>",
+ "Usage: node scripts/package-app-candidate.mjs <server-release-directory> <target> <new-output-directory> [--tunnel-asset <raw-release-asset>]",
);
}
const repositoryRoot = realpathSync(resolve(dirname(fileURLToPath(import.meta.url)), ".."));
-const serverRoot = realpathSync(resolve(process.argv[2]));
-const target = appPackageTarget(process.argv[3]);
-const outputRoot = resolve(process.cwd(), process.argv[4]);
+const serverRoot = realpathSync(resolve(positionals[0]));
+const target = appPackageTarget(positionals[1]);
+const outputRoot = resolve(process.cwd(), positionals[2]);
if (!target) fail("App package target is invalid");
if (process.platform !== target.nodePlatform || process.arch !== target.nodeArch) {
fail(`App candidate ${target.id} requires its native runner`);
@@ -332,10 +338,14 @@ const temporaryRoot = resetBuildWorkspace(repositoryRoot, "app-package", target.
let outputOwned = false;
try {
const tunnelDownload = join(temporaryRoot, target.tunnelAsset);
- await download(
- `https://github.com/cloudflare/cloudflared/releases/download/${CLOUDFLARED_VERSION}/${target.tunnelAsset}`,
- tunnelDownload,
- );
+ if (values["tunnel-asset"] !== undefined) {
+ copyFileSync(realpathSync(resolve(values["tunnel-asset"])), tunnelDownload);
+ } else {
+ await download(
+ `https://github.com/cloudflare/cloudflared/releases/download/${CLOUDFLARED_VERSION}/${target.tunnelAsset}`,
+ tunnelDownload,
+ );
+ }
if (sha256(tunnelDownload) !== target.tunnelSha256) {
fail("Public-link runtime digest does not match the pinned release");
}
diff --git a/scripts/room-voice-gate.ts b/scripts/room-voice-gate.ts
index c369aa46..5d50814b 100644
--- a/scripts/room-voice-gate.ts
+++ b/scripts/room-voice-gate.ts
@@ -188,11 +188,11 @@ async function main(): Promise<void> {
server.stdout.resume();
server.stderr.on("data", (data: Buffer) => { serverLog = (serverLog + data.toString()).slice(-12000); });
await waitForSample(deadline => fetchJsonBefore(`${backend}/healthz`, deadline), () => true, 15_000);
- vite = await createViteServer({ root: ROOT, logLevel: "error", server: { host: "127.0.0.1", port: vitePort, watch: null,
+ vite = await createViteServer({ root: ROOT, cacheDir: join(OUTPUT, ".vite"), logLevel: "error", server: { host: "127.0.0.1", port: vitePort, watch: null,
strictPort: true, proxy: { "/api": { target: backend }, "/healthz": { target: backend }, "/signal": { target: backend, ws: true } } } });
await vite.listen();
if (legacyRoot && legacyPort) {
- legacyVite = await createViteServer({ root: legacyRoot, logLevel: "error", optimizeDeps: { entries: ["index.html"] }, server: { host: "127.0.0.1", port: legacyPort,
+ legacyVite = await createViteServer({ root: legacyRoot, cacheDir: join(OUTPUT, ".vite-legacy"), logLevel: "error", optimizeDeps: { entries: ["index.html"] }, server: { host: "127.0.0.1", port: legacyPort,
watch: null, strictPort: true, proxy: { "/api": { target: backend }, "/healthz": { target: backend }, "/signal": { target: backend, ws: true } } } });
await legacyVite.listen();
}
diff --git a/site/index.html b/site/index.html
index d728f467..4c77daf7 100644
--- a/site/index.html
+++ b/site/index.html
@@ -905,7 +905,7 @@
<p class="download-variant">iPhone / iPad</p>
<a class="lr-btn button download-button" href="https://demo.piik.tv"
><span lang="en">Use online</span><span lang="zh-CN">使用在线版</span><span aria-hidden="true">↗</span></a>
- <p class="download-note"><span lang="en">Watch, chat and talk in your browser.</span><span lang="zh-CN">通过浏览器观看、聊天与语音。</span></p>
+ <p class="download-note"><span lang="en">Use Piik in your browser. No installation needed.</span><span lang="zh-CN">无需安装,在浏览器中直接使用。</span></p>
</article>
</div>
</section>
diff --git a/src/client/components/living/Header.tsx b/src/client/components/living/Header.tsx
index c4d947c0..e94efcb1 100644
--- a/src/client/components/living/Header.tsx
+++ b/src/client/components/living/Header.tsx
@@ -1,5 +1,5 @@
// App header: brand mark, LED connection state, language selection and theme.
-import { useId, useRef, useState, useSyncExternalStore, type ComponentProps } from "react";
+import { useId, useLayoutEffect, useRef, useState, useSyncExternalStore, type ComponentProps } from "react";
import { createPortal } from "react-dom";
import { browserDebugEnabled, debugError, downloadBrowserDebug, withBrowserDebug } from "../../lib/debug";
import { BrandMark } from "./BrandMark";
@@ -85,15 +85,35 @@ const webVersion = import.meta.env.VITE_PIIK_VERSION ?? "development";
const webRelease = /^v\d+\.\d+\.\d+$/.test(webVersion)
? `https://github.com/TNTcraftHIM/Piik/releases/tag/${webVersion}` : null;
-function BrowserHelpMenu() {
+function BrowserHelpContent({ active }: { active: boolean }) {
const { lang, vis, t } = useCopy();
+ const guide = useProjectLink(`/docs/${lang === "zh" ? "zh/" : ""}troubleshooting.html`,
+ `https://github.com/TNTcraftHIM/Piik/blob/main/docs/guide/troubleshooting${lang === "zh" ? ".zh-CN" : ""}.md`, active);
+ return <div className={`lr-help-content${vis ? " is-visual" : ""}`}>
+ <header><strong>Piik Web</strong>
+ {webRelease ? <a href={webRelease} target="_blank" rel="noopener noreferrer"
+ aria-label={`Piik Web ${webVersion}`}>{webVersion}</a> : <span>{webVersion}</span>}
+ </header>
+ <nav aria-label={t("help.title")}>
+ {active && <>
+ <Tooltip kind="hint-help" text={vis ? undefined : t("shareHelp.guide")} place="below" align="end">
+ <a className="lr-btn is-compact is-quiet" href={guide} target="_blank" rel="noopener noreferrer" aria-label={t("shareHelp.guide")}>
+ <Glyph name="book" size={16} />{!vis && <span className="lr-cap">{t("help.troubleshooting")}</span>}
+ </a>
+ </Tooltip>
+ <BrowserDiagnosticsButton size="compact" quiet />
+ </>}
+ </nav>
+ </div>;
+}
+
+function BrowserHelpMenu() {
+ const { vis, t } = useCopy();
const id = useId();
const trigger = useRef<HTMLButtonElement>(null);
const panel = useRef<HTMLDivElement>(null);
const [open, setOpen] = useState(false);
useAnchoredPopover(trigger, panel);
- const guide = useProjectLink(`/docs/${lang === "zh" ? "zh/" : ""}troubleshooting.html`,
- `https://github.com/TNTcraftHIM/Piik/blob/main/docs/guide/troubleshooting${lang === "zh" ? ".zh-CN" : ""}.md`, open);
return <>
<Tooltip kind="hint-help" text={vis ? undefined : t("help.hint")} disabled={open} place="below" align="end">
<button ref={trigger} type="button" className="lr-btn is-compact is-quiet"
@@ -102,29 +122,44 @@ function BrowserHelpMenu() {
</button>
</Tooltip>
{createPortal(<div ref={panel} id={id} popover="auto" role="dialog" aria-label={t("help.title")}
- className={`lr-anchored-card lr-help-menu${vis ? " is-visual" : ""}`} onToggle={event => {
+ className="lr-anchored-card lr-help-menu" onToggle={event => {
if (event.target === event.currentTarget) setOpen(event.newState === "open");
}}>
- <header><strong>Piik Web</strong>
- {webRelease ? <a href={webRelease} target="_blank" rel="noopener noreferrer"
- aria-label={`Piik Web ${webVersion}`}>{webVersion}</a> : <span>{webVersion}</span>}
- </header>
- <nav aria-label={t("help.title")}>
- {open && <>
- <Tooltip kind="hint-help" text={vis ? undefined : t("shareHelp.guide")} place="below" align="end">
- <a className="lr-btn is-compact is-quiet" href={guide} target="_blank" rel="noopener noreferrer" aria-label={t("shareHelp.guide")}>
- <Glyph name="book" size={16} />{!vis && <span className="lr-cap">{t("help.troubleshooting")}</span>}
- </a>
- </Tooltip>
- <BrowserDiagnosticsButton size="compact" quiet />
- </>}
- </nav>
+ <BrowserHelpContent active={open} />
</div>, document.body)}
</>;
}
export function HeaderControls({ diagnosticControl, appControl }: { diagnosticControl?: React.ReactNode; appControl?: React.ReactNode } = {}) {
const { vis, t } = useCopy();
+ const controls = useRef<HTMLDivElement>(null), trigger = useRef<HTMLSpanElement>(null), options = useRef<HTMLDivElement>(null);
+ const focusInline = useRef(false);
+ const id = useId();
+ const [compact, setCompact] = useState(false), [open, setOpen] = useState(false);
+ useAnchoredPopover(trigger, options, { enabled: compact });
+ useLayoutEffect(() => {
+ const header = controls.current?.closest(".lr-top") ?? controls.current;
+ if (!header) return;
+ const resize = () => {
+ const narrow = header.clientWidth <= 600;
+ if (!narrow && options.current?.hasAttribute("popover")) {
+ const focused = header.ownerDocument.activeElement;
+ focusInline.current = options.current.contains(focused) || Boolean(trigger.current?.contains(focused));
+ if (options.current.matches(":popover-open")) options.current.hidePopover();
+ }
+ if (!narrow) options.current?.removeAttribute("style");
+ setCompact(narrow);
+ };
+ const observer = new ResizeObserver(resize);
+ observer.observe(header); resize();
+ return () => observer.disconnect();
+ }, []);
+ useLayoutEffect(() => {
+ if (!compact && focusInline.current) {
+ focusInline.current = false;
+ (options.current?.querySelector<HTMLElement>('[aria-pressed="true"]') ?? options.current?.querySelector<HTMLElement>("button, a"))?.focus();
+ }
+ }, [compact]);
const theme = useSyncExternalStore(subscribeTheme, currentTheme);
const themeTitle = t(theme === "dark" ? "theme.light" : "theme.dark");
const themeButton = (
@@ -147,14 +182,25 @@ export function HeaderControls({ diagnosticControl, appControl }: { diagnosticCo
</button>
);
return (
- <span className="lr-header-controls">
+ <div ref={controls} className={`lr-top-right lr-header-controls${compact ? " is-overflow" : ""}`}>
{appControl}
- <LanguageControl />
- <Tooltip kind={theme === "dark" ? "hint-theme-light" : "hint-theme-dark"} text={vis ? undefined : themeTitle} place="below" align="end">
- {themeButton}
- </Tooltip>
- {diagnosticControl === undefined ? <BrowserHelpMenu /> : diagnosticControl}
- </span>
+ {compact && <span ref={trigger} className="lr-header-more">
+ <Btn icon="sliders" title="header.more" hint="hint-advanced" size="compact" popoverTarget={id} controls={id} expanded={open} />
+ </span>}
+ <div ref={options} id={id} popover={compact ? "auto" : undefined} role={compact ? "dialog" : undefined}
+ aria-label={compact ? t("header.more") : undefined}
+ className={`lr-header-options${compact ? " lr-anchored-card" : ""}`} onToggle={event => {
+ if (event.target === event.currentTarget) setOpen(event.newState === "open");
+ }}>
+ <div className="lr-header-settings">
+ <LanguageControl />
+ <Tooltip kind={theme === "dark" ? "hint-theme-light" : "hint-theme-dark"} text={vis ? undefined : themeTitle} place="below" align="end">
+ {themeButton}
+ </Tooltip>
+ </div>
+ {diagnosticControl === undefined ? compact ? <BrowserHelpContent active={open} /> : <BrowserHelpMenu /> : diagnosticControl}
+ </div>
+ </div>
);
}
@@ -174,20 +220,20 @@ export function AppHeader({
const { t } = useCopy();
return (
<header className="lr-top">
- <span className="lr-header-context">
- {homeHref !== "/client" && <a
- className="lr-brand"
- href={withBrowserDebug(homeHref)}
- aria-label={t("brand.home")}
- aria-current={atHome ? "page" : undefined}
- >
- {!atHome && <Glyph name="chevron" size={14} className="lr-home-arrow" />}
- <BrandMark size={34} motion="once" />
- </a>}
+ {homeHref !== "/client" && <a
+ className="lr-brand"
+ href={withBrowserDebug(homeHref)}
+ aria-label={t("brand.home")}
+ aria-current={atHome ? "page" : undefined}
+ >
+ {!atHome && <Glyph name="chevron" size={14} className="lr-home-arrow" />}
+ <BrandMark size={34} motion="once" />
+ </a>}
+ <div className="lr-top-right">
{led}
- </span>
- <HeaderControls diagnosticControl={diagnosticControl}
- appControl={homeHref !== "/client" ? <AppConnectionButton onConnected={onAppConnected} /> : undefined} />
+ <HeaderControls diagnosticControl={diagnosticControl}
+ appControl={homeHref !== "/client" ? <AppConnectionButton onConnected={onAppConnected} /> : undefined} />
+ </div>
</header>
);
}
diff --git a/src/client/components/living/PlaybackSettings.tsx b/src/client/components/living/PlaybackSettings.tsx
index 9427c5ed..024d3ced 100644
--- a/src/client/components/living/PlaybackSettings.tsx
+++ b/src/client/components/living/PlaybackSettings.tsx
@@ -7,6 +7,7 @@ import { Glyph } from "../../ui/icons";
import { Btn, Cap, Chip, FieldCap, Pill, RowGroup, SwitchItem } from "./primitives";
import { NumberControl } from "./NumberControl";
import { Tooltip } from "./Tooltip";
+import "./playback-controls.css";
export interface PlaybackSettingsProps {
pictureFit: VideoFit;
@@ -56,8 +57,8 @@ export function PlaybackPictureControls({ settings, captions = false }: {
hintVars={{ current: t(`playback.fit.${settings.pictureFit}`), next: t(`playback.fit.${nextFit}`) }}
onClick={() => settings.setPictureFit(nextFit)} />
<Btn icon="rotatePicture" title="playback.rotation" cap={captions ? "playback.rotation" : undefined}
- hint="hint-picture-rotation" hintText="playback.rotationHint" pressed={settings.pictureRotation !== 0}
- onClick={() => settings.setPictureRotation(settings.pictureRotation === 0 ? 90 : 0)} />
+ hint="hint-picture-rotation" hintText="playback.rotationHint"
+ onClick={() => settings.setPictureRotation((settings.pictureRotation + 90) % 360 as VideoRotation)} />
</>;
}
diff --git a/src/client/components/living/playback-controls.css b/src/client/components/living/playback-controls.css
index ea3c632d..cb8c3e92 100644
--- a/src/client/components/living/playback-controls.css
+++ b/src/client/components/living/playback-controls.css
@@ -57,7 +57,7 @@
.lr-playback :is(.lr-remote-controls, .lr-playback-view)::before, .lr-playback-window::before { content: ""; flex: none; width: 1px; height: 20px; margin-right: 4px; background: var(--control-edge); }
.lr-playback-picture-actions { display: inline-flex; align-items: center; gap: 4px; }
.lr-playback-more { display: inline-flex; }
-.lr-playback-more.is-compact-only, .lr-playback-compact-picture { display: none; }
+.lr-playback-more.is-compact-only, .lr-playback .lr-playback-compact-picture { display: none; }
.lr-playback-picture-options { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 8px; }
.lr-playback-picture-options .lr-btn { width: 100%; padding: 0 8px; }
.lr-playback-menu { position: fixed; inset: auto; margin: 0; padding: 8px; max-width: calc(100vw - 16px); overflow: auto; overscroll-behavior: contain; }
diff --git a/src/client/locales/en.ts b/src/client/locales/en.ts
index 80e134b3..d10956af 100644
--- a/src/client/locales/en.ts
+++ b/src/client/locales/en.ts
@@ -136,7 +136,7 @@ export const en: Record<CopyKey, string> = {
"playback.changeFit": "Switch picture fit",
"playback.fitCycle": "Current: {current}. Switch to {next}. Only changes your view.",
"playback.rotation": "Rotate picture",
- "playback.rotationHint": "Rotate the picture locally without changing the shared source.",
+ "playback.rotationHint": "Rotate the picture 90° clockwise on this device.",
"playback.connection": "Connection",
"playback.relayVolunteer": "Help relay to more viewers",
"playback.relayVolunteerHint": "Uses more upload bandwidth for this visit.",
@@ -794,6 +794,7 @@ export const en: Record<CopyKey, string> = {
"mode.language": "Interface language",
"mode.more": "More languages",
+ "header.more": "More options",
"mode.vis": "Visual only",
"theme.dark": "Switch to dark mode",
"theme.light": "Switch to light mode",
diff --git a/src/client/locales/zh.ts b/src/client/locales/zh.ts
index 0210ada6..04c1dd7c 100644
--- a/src/client/locales/zh.ts
+++ b/src/client/locales/zh.ts
@@ -135,8 +135,8 @@ export const zh = {
"playback.windowControls": "播放器窗口",
"playback.changeFit": "切换画面适配",
"playback.fitCycle": "当前:{current}。点击切换为{next},只影响自己的画面。",
- "playback.rotation": "横竖切换",
- "playback.rotationHint": "旋转当前画面的显示方向,不影响分享来源。",
+ "playback.rotation": "旋转画面",
+ "playback.rotationHint": "将本机观看的画面顺时针旋转 90°。",
"playback.connection": "连接",
"playback.relayVolunteer": "帮助更多观众转发",
"playback.relayVolunteerHint": "仅本次观看生效,会使用更多上行带宽。",
@@ -794,6 +794,7 @@ export const zh = {
"mode.language": "界面语言",
"mode.more": "更多语言",
+ "header.more": "更多选项",
"mode.vis": "纯视觉",
"theme.dark": "切换到暗色模式",
"theme.light": "切换到浅色模式",
diff --git a/src/client/media/audio-capture.ts b/src/client/media/audio-capture.ts
index 2be31af3..a6a0c73b 100644
--- a/src/client/media/audio-capture.ts
+++ b/src/client/media/audio-capture.ts
@@ -5,6 +5,7 @@ type DisplayMediaAudioHints = DisplayMediaStreamOptions & {
type BrowserAudioConstraints = MediaTrackConstraints & {
voiceIsolation?: ConstrainBoolean;
+ restrictOwnAudio?: ConstrainBoolean;
};
export function audioInputConstraints(voiceProcessing: boolean): BrowserAudioConstraints {
@@ -23,7 +24,8 @@ export function displayMediaOptions(
// Chromium web display capture otherwise defaults to speech processing.
return {
video,
- audio: audioInputConstraints(false),
+ // Avoid recapturing this page's room voice where the Browser supports it.
+ audio: { ...audioInputConstraints(false), restrictOwnAudio: true } as BrowserAudioConstraints,
systemAudio: "include",
windowAudio: "window",
} as DisplayMediaAudioHints;
diff --git a/src/client/media/browser-encoding-output.ts b/src/client/media/browser-encoding-output.ts
index 40a74908..707bd096 100644
--- a/src/client/media/browser-encoding-output.ts
+++ b/src/client/media/browser-encoding-output.ts
@@ -1,10 +1,11 @@
import { debugError, debugEvent } from "../lib/debug";
+import { supportsFrameOrientationNormalization } from "./sender-video-track";
// One codec-sized macroblock; transport timing is not a quality layer.
export const BROWSER_CARRIER_SIZE = 16;
export function supportsBrowserEncoding(): boolean {
- return typeof RTCRtpSender !== "undefined" && typeof RTCEncodedVideoFrame === "function" &&
+ return supportsFrameOrientationNormalization() && typeof RTCRtpSender !== "undefined" && typeof RTCEncodedVideoFrame === "function" &&
typeof HTMLCanvasElement !== "undefined" && typeof HTMLCanvasElement.prototype.captureStream === "function" &&
typeof (RTCRtpSender.prototype as RTCRtpSender & { createEncodedStreams?: unknown }).createEncodedStreams === "function";
}
diff --git a/src/client/media/browser-encoding-producer.ts b/src/client/media/browser-encoding-producer.ts
index e0ed1418..a7e57c9e 100644
--- a/src/client/media/browser-encoding-producer.ts
+++ b/src/client/media/browser-encoding-producer.ts
@@ -57,7 +57,7 @@ export class BrowserEncodingProducer {
this.budget = initialBudget;
try {
if (this.source.readyState !== "live") throw new Error("Browser encoding source ended");
- const track = this.input = cloneSenderVideoTrack(this.source, () => this.fail());
+ const track = this.input = cloneSenderVideoTrack(this.source, () => this.fail(), { normalizeOrientation: true });
track.enabled = !this.paused;
this.source.addEventListener("ended", this.fail);
track.addEventListener("ended", this.fail);
@@ -122,7 +122,7 @@ export class BrowserEncodingProducer {
await receive.setRemoteDescription({ type: "offer", sdp: normalizeVideoOfferSdp(send.localDescription!.sdp) });
await flushReceive();
this.checkAlive();
- await receive.setLocalDescription(this.seedLocalAnswer(await receive.createAnswer()));
+ await receive.setLocalDescription(this.prepareLocalAnswer(await receive.createAnswer()));
this.checkAlive();
await send.setRemoteDescription(receive.localDescription!);
await flushSend();
@@ -222,13 +222,18 @@ export class BrowserEncodingProducer {
return this.videoSender;
}
- private seedLocalAnswer(answer: RTCSessionDescriptionInit): RTCSessionDescriptionInit {
+ private prepareLocalAnswer(answer: RTCSessionDescriptionInit): RTCSessionDescriptionInit {
const kbps = Math.floor(Math.min(this.budget, this.desiredProfile.maxBitrate) / 1_000);
- if (!Number.isFinite(kbps) || kbps <= 0 || !answer.sdp) return answer;
+ if (!answer.sdp) return answer;
const session = parse(answer.sdp);
const media = session.media.find((section) => section.type === "video" && section.port !== 0);
- const codec = media?.rtp.find((entry) => `video/${entry.codec}`.toLowerCase() === this.codec.mimeType.toLowerCase());
- if (!media || !codec) return answer;
+ if (!media) return answer;
+ // Shared frames already have normalized pixels and also feed peers without
+ // CVO; do not leave orientation in a local-only RTP extension.
+ const cvo = media.ext?.some(extension => extension.uri === "urn:3gpp:video-orientation");
+ if (cvo) media.ext = media.ext?.filter(extension => extension.uri !== "urn:3gpp:video-orientation");
+ const codec = media.rtp.find((entry) => `video/${entry.codec}`.toLowerCase() === this.codec.mimeType.toLowerCase());
+ if (!Number.isFinite(kbps) || kbps <= 0 || !codec) return cvo ? { ...answer, sdp: write(session) } : answer;
const format = media.fmtp.find((entry) => entry.payload === codec.payload);
const parameters = (format?.config ?? "").split(";").map((part) => part.trim())
.filter((part) => part && !/^x-google-start-bitrate\s*=/i.test(part));
diff --git a/src/client/media/sender-video-track.ts b/src/client/media/sender-video-track.ts
index 43dd7af0..1be59c8e 100644
--- a/src/client/media/sender-video-track.ts
+++ b/src/client/media/sender-video-track.ts
@@ -9,16 +9,25 @@ type FrameAPIs = {
};
const inputs = new WeakMap<MediaStreamTrack, { track: MediaStreamTrack; abort: AbortController }>();
+export function supportsFrameOrientationNormalization(): boolean {
+ const api = globalThis as typeof globalThis & FrameAPIs;
+ return typeof api.MediaStreamTrackProcessor === "function" && typeof api.MediaStreamTrackGenerator === "function" &&
+ typeof VideoFrame === "function" && "rotation" in VideoFrame.prototype && "flip" in VideoFrame.prototype &&
+ typeof createImageBitmap === "function";
+}
+
/** The sender owns this track, including its optional display-capture input. */
export function cloneSenderVideoTrack(
source: MediaStreamTrack,
onEnded: (track: MediaStreamTrack) => void,
+ { normalizeOrientation = false }: { normalizeOrientation?: boolean } = {},
): MediaStreamTrack {
+ if (normalizeOrientation && !supportsFrameOrientationNormalization()) throw new Error("Frame orientation normalization unavailable");
const input = source.clone();
input.contentHint = source.contentHint || "motion";
input.enabled = source.enabled;
const { MediaStreamTrackProcessor: Processor, MediaStreamTrackGenerator: Generator } = globalThis as typeof globalThis & FrameAPIs;
- if (!Processor || !Generator || !source.getSettings().displaySurface || input.readyState !== "live") return input;
+ if (!Processor || !Generator || (!normalizeOrientation && !source.getSettings().displaySurface) || input.readyState !== "live") return input;
let output: FrameTrack | undefined;
let readable: ReadableStream<VideoFrame> | undefined;
@@ -27,12 +36,13 @@ export function cloneSenderVideoTrack(
readable = new Processor({ track: input, maxBufferSize: 1 }).readable;
} catch (error) {
output?.stop();
+ if (normalizeOrientation) { input.stop(); throw error; }
debugError("capture", "sender-frame-isolation-unavailable", error);
return input;
}
// Chromium forwards an encoder's soft pixel target as a capture hard limit.
- // Forward frames without transforming them so sender adaptation cannot shrink
- // its own recovery input. No new encoder, frame clock or resolution policy.
+ // Isolate capture from sender adaptation; shared encoders also normalize
+ // orientation before their encoded frames cross CVO capabilities.
const abort = new AbortController();
const track = output;
const frames = readable;
@@ -41,7 +51,24 @@ export function cloneSenderVideoTrack(
track.enabled = source.enabled;
inputs.set(track, { track: input, abort });
void applySenderCaptureConstraints(track, input.getConstraints())
- .then(() => frames.pipeTo(track.writable, { signal: abort.signal }))
+ .then(() => {
+ const normalized = normalizeOrientation ? frames.pipeThrough(new TransformStream<VideoFrame, VideoFrame>({
+ async transform(frame, controller) {
+ const oriented = frame as VideoFrame & { readonly rotation: number; readonly flip: boolean };
+ if (!oriented.rotation && !oriented.flip) { controller.enqueue(frame); return; }
+ let bitmap: ImageBitmap | undefined;
+ try {
+ // VideoFrame display dimensions already include quarter-turn rotation.
+ bitmap = await createImageBitmap(frame, {
+ resizeWidth: frame.displayWidth, resizeHeight: frame.displayHeight,
+ });
+ const next = new VideoFrame(bitmap, { timestamp: frame.timestamp, ...(frame.duration === null ? {} : { duration: frame.duration }) });
+ try { controller.enqueue(next); } catch (error) { next.close(); throw error; }
+ } finally { bitmap?.close(); frame.close(); }
+ },
+ }), { signal: abort.signal }) : frames;
+ return normalized.pipeTo(track.writable, { signal: abort.signal });
+ })
.catch((error) => {
if (!abort.signal.aborted) debugError("capture", "sender-frame-isolation-failed", error);
}).finally(() => {
@@ -63,6 +90,7 @@ export function senderCaptureTrack(track: MediaStreamTrack): MediaStreamTrack {
export function applySenderCaptureConstraints(track: MediaStreamTrack, constraints: MediaTrackConstraints): Promise<void> {
const input = inputs.get(track)?.track;
if (!input) return track.applyConstraints(constraints);
+ if (!input.getSettings().displaySurface) return input.applyConstraints(constraints);
const rate = constraints.frameRate;
// A Processor does not request idle refresh frames as an RTC sender does.
// Preserve the native display-capture refresh with a standard constraint;
diff --git a/src/client/native/client.ts b/src/client/native/client.ts
index 23c79808..b60dc07d 100644
--- a/src/client/native/client.ts
+++ b/src/client/native/client.ts
@@ -355,6 +355,7 @@ export class NativeClient {
...(this.health.nativeMedia.remoteInput ? { remoteInput: true } : {}),
...(this.health.nativeMedia.captureBorderControl ? { showCaptureBorder } : {}),
...(this.health.nativeMedia.sourceCancellation ? { reportSourceCancellation: true } : {}),
+ ...(this.health.nativeMedia.captureFailureDetails ? { reportCaptureFailure: true } : {}),
},
shareStartedResponseSchema,
input.source.kind === "picker" ? null : REQUEST_TIMEOUT_MS,
diff --git a/src/client/native/wire.ts b/src/client/native/wire.ts
index e105e1aa..d9030d66 100644
--- a/src/client/native/wire.ts
+++ b/src/client/native/wire.ts
@@ -53,6 +53,7 @@ export const nativeHealthSchema = nativeDiscoveryIdentitySchema.extend({
microphone: z.boolean().default(false),
shareVolume: z.boolean().optional(),
sourceCancellation: z.boolean().optional(),
+ captureFailureDetails: z.boolean().optional(),
// API availability, not Windows approval or observed border visibility.
captureBorderControl: z.boolean().default(false),
hardwareH264: z.boolean().default(false),
@@ -443,8 +444,13 @@ export const nativeEventSchema = z.discriminatedUnion("type", [
...eventBase,
type: z.literal("share-ended"),
failed: z.boolean(),
+ failure: z.object({
+ stage: z.string().regex(/^[a-z][a-z0-9-]{0,63}$/).optional(),
+ hresult: z.string().regex(/^0x[0-9a-f]{8}$/).optional(),
+ exitCode: z.number().int().min(-0x80000000).max(0xffffffff),
+ }).strict().optional(),
})
- .strict(),
+ .strict().refine(value => value.failed || !value.failure),
]);
export type NativeClientEvent = z.infer<typeof nativeEventSchema>;
diff --git a/src/client/styles.css b/src/client/styles.css
index a0c300dd..769b8927 100644
--- a/src/client/styles.css
+++ b/src/client/styles.css
@@ -91,7 +91,7 @@ ul { margin: 0; padding: 0; list-style: none; }
.lr-top {
display: flex;
- flex-wrap: wrap;
+ flex-wrap: nowrap;
align-items: center;
justify-content: space-between;
gap: 8px 12px;
@@ -115,9 +115,16 @@ ul { margin: 0; padding: 0; list-style: none; }
.lr-brand:active { transform: scale(.92); }
.lr-home-arrow { transform: rotate(90deg); color: var(--muted); }
-.lr-header-context, .lr-header-controls { display: inline-flex; flex: none; gap: 10px; align-items: center; }
-.lr-header-context:empty { display: none; }
-.lr-header-controls { margin-inline-start: auto; }
+.lr-top-right { display: inline-flex; gap: 10px; align-items: center; }
+.lr-top > .lr-top-right { margin-left: auto; min-width: 0; flex: none; justify-content: flex-end; }
+.lr-header-controls { flex: none; }
+.lr-header-options:not([popover]), .lr-header-options:not([popover]) > .lr-header-settings { display: contents; }
+.lr-header-more { display: inline-flex; }
+.lr-header-options:is(:popover-open, .\:popover-open) { display: grid; gap: 16px; }
+.lr-header-settings { display: flex; align-items: center; justify-content: space-between; gap: 12px; }
+.lr-header-options .lr-help-content { padding-top: 14px; border-top: 1px solid var(--control-edge); }
+.lr-top:has(.lr-header-controls.is-overflow) .lr-leds { width: 44px; height: 44px; padding: 0; justify-content: center; }
+.lr-top:has(.lr-header-controls.is-overflow) .lr-leds-label { display: none; }
/* Discrete signaling state, independent of picture quality. */
.lr-leds {
@@ -936,14 +943,14 @@ body.lr-theater-open .lr-top { visibility: hidden; }
.lr-app-connect-help .lr-app-connect-optional { color: var(--muted); font-size: 12px; margin-bottom: 0; }
@media (prefers-reduced-motion: reduce) { .lr-anchored-card:is(:popover-open, .\:popover-open) { animation: none; } }
.lr-help-menu { width: min(260px, calc(100vw - 16px)); }
-.lr-help-menu header { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 12px; margin: 0 2px 14px; }
-.lr-help-menu header strong { font-size: 14px; }
-.lr-help-menu header :is(a, span) { color: var(--muted); font-size: 12px; overflow-wrap: anywhere; font-variant-numeric: tabular-nums; }
-.lr-help-menu header a { text-underline-offset: 3px; }
-.lr-help-menu nav { display: grid; gap: 4px; }
-.lr-help-menu nav .lr-btn { width: 100%; justify-content: flex-start; }
-.lr-help-menu.is-visual nav { display: flex; justify-content: center; gap: 12px; }
-.lr-help-menu.is-visual nav .lr-btn { width: var(--control-compact-size); justify-content: center; }
+.lr-help-content header { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 12px; margin: 0 2px 14px; }
+.lr-help-content header strong { font-size: 14px; }
+.lr-help-content header :is(a, span) { color: var(--muted); font-size: 12px; overflow-wrap: anywhere; font-variant-numeric: tabular-nums; }
+.lr-help-content header a { text-underline-offset: 3px; }
+.lr-help-content nav { display: grid; gap: 4px; }
+.lr-help-content nav .lr-btn { width: 100%; justify-content: flex-start; }
+.lr-help-content.is-visual nav { display: flex; justify-content: center; gap: 12px; }
+.lr-help-content.is-visual nav .lr-btn { width: var(--control-compact-size); justify-content: center; }
.lr-cap { font-size: var(--control-label-size); font-weight: 600; color: inherit; white-space: nowrap; }
.lr-chip {
@@ -1797,10 +1804,14 @@ body.lr-theater-open .lr-top { visibility: hidden; }
.lr-source-option { min-height: 58px; padding: 6px 8px; }
.lr-source-option-preview { height: 90px; }
.lr-source-option-icon { width: 34px; height: 34px; }
- .lr-header-context, .lr-header-controls { gap: 8px; }
+ .lr-top-right { gap: 8px; }
.lr-leds { gap: 4px; padding: 6px; }
.lr-top .lr-btn.is-compact { width: var(--control-compact-size); padding: 0; }
.lr-top .lr-btn.is-compact .lr-cap { display: none; }
+ .lr-top .lr-header-options[popover] .lr-btn.is-compact { width: auto; padding-inline: 12px; }
+ .lr-top .lr-header-options[popover] .lr-cap { display: inline; }
+ .lr-top .lr-header-options[popover] .lr-help-content nav .lr-btn { width: 100%; }
+ .lr-top .lr-header-options[popover] .lr-help-content.is-visual nav .lr-btn { width: var(--control-compact-size); padding: 0; }
}
@media (orientation: landscape) and (max-height: 500px) {
@@ -1947,7 +1958,7 @@ details[open] > summary > .lr-sharing-chevron { transform: rotate(180deg); }
.lr-client-modes { grid-template-columns: 1fr; }
.lr-lcd { gap: 4px; padding-inline: 6px; font-size: 18px; }
.lr-top { column-gap: 4px; padding-inline: 10px; }
- .lr-header-context, .lr-header-controls { gap: 4px; }
+ .lr-top-right { gap: 4px; }
.lr-brand { gap: 0; padding-inline: 0; }
.lr-home-arrow { width: 10px; }
.lr-tiles .lr-tile { padding-inline: 4px; }
diff --git a/tests/app-candidate.test.ts b/tests/app-candidate.test.ts
index 9b7ffefb..f6fc8759 100644
--- a/tests/app-candidate.test.ts
+++ b/tests/app-candidate.test.ts
@@ -47,7 +47,7 @@ vi.mock("../scripts/archive-tool.mjs", () => ({
tarExecutable: () => "fixture-tar",
}));
-it.each(["archive-failure", "existing", "output-created", "dirty"] as const)(
+it.each(["archive-failure", "existing", "output-created", "dirty", "raw-asset", "raw-digest-mismatch"] as const)(
"cleans only this App candidate's incomplete output (%s)", async (scenario) => {
const root = mkdtempSync(join(tmpdir(), "piik-app-candidate-"));
const server = join(root, "server");
@@ -66,17 +66,25 @@ it.each(["archive-failure", "existing", "output-created", "dirty"] as const)(
writeFileSync(join(output, "user.txt"), "keep");
}
process.argv = [process.execPath, "package-app-candidate.mjs", server, "fixture", output];
+ const rawAsset = scenario === "raw-asset" || scenario === "raw-digest-mismatch";
+ if (rawAsset) {
+ const asset = join(root, "raw-release-asset");
+ writeFileSync(asset, scenario === "raw-asset" ? "fixture tunnel" : "wrong bytes");
+ process.argv.push("--tunnel-asset", asset);
+ }
vi.resetModules();
const candidate = import("../scripts/package-app-candidate.mjs");
const expected = { "archive-failure": "archive interrupted", existing: "must not already exist",
- "output-created": "EEXIST", dirty: "clean Git checkout" }[scenario];
+ "output-created": "EEXIST", dirty: "clean Git checkout", "raw-asset": "archive interrupted",
+ "raw-digest-mismatch": "digest does not match the pinned release" }[scenario];
await expect(candidate).rejects.toThrow(expected);
if (scenario === "existing" || scenario === "output-created") {
expect(readFileSync(join(output, "user.txt"), "utf8")).toBe("keep");
} else {
expect(existsSync(output)).toBe(false);
}
- expect(source.assembled).toBe(scenario === "archive-failure" || scenario === "output-created");
+ expect(source.assembled).toBe(scenario === "archive-failure" || scenario === "output-created" || scenario === "raw-asset");
+ if (rawAsset) expect(fetch).not.toHaveBeenCalled();
expect(existsSync(source.workspace)).toBe(false);
} finally {
process.argv = previousArguments;
diff --git a/tests/audio-capture.test.ts b/tests/audio-capture.test.ts
index b6226cdd..d47493c0 100644
--- a/tests/audio-capture.test.ts
+++ b/tests/audio-capture.test.ts
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
-import { displayMediaOptions } from "../src/client/media/audio-capture.ts";
+import { audioInputConstraints, displayMediaOptions } from "../src/client/media/audio-capture.ts";
describe("browser display audio capture", () => {
it("keeps screen audio free of speech processing and prefers stereo", () => {
@@ -13,9 +13,12 @@ describe("browser display audio capture", () => {
autoGainControl: false,
voiceIsolation: false,
channelCount: { ideal: 2 },
+ restrictOwnAudio: true,
},
systemAudio: "include",
windowAudio: "window",
});
+ expect(audioInputConstraints(true)).not.toHaveProperty("restrictOwnAudio");
+ expect(audioInputConstraints(false)).not.toHaveProperty("restrictOwnAudio");
});
});
diff --git a/tests/browser-debug.test.ts b/tests/browser-debug.test.ts
index 3b8e0a1e..25c2352a 100644
--- a/tests/browser-debug.test.ts
+++ b/tests/browser-debug.test.ts
@@ -20,6 +20,23 @@ afterEach(() => {
});
describe("opt-in Browser diagnostics", () => {
+ it("exports validated Native capture failure codes through the existing event recorder", async () => {
+ browser();
+ const debug = await import("../src/client/lib/debug");
+ const { nativeEventSchema } = await import("../src/client/native/wire");
+ const ended = { version: 10, type: "share-ended", shareId: "share_123456", failed: true,
+ failure: { stage: "windows-runtime", hresult: "0x887a0005", exitCode: 2 } };
+ debug.debugEvent("native", "event", nativeEventSchema.parse(ended));
+ const report = JSON.parse(await debug.exportBrowserDebug());
+ expect(report.events).toContainEqual(expect.objectContaining({ scope: "native", event: "event", details: ended }));
+ for (const failure of [{ ...ended.failure, detail: "private" }, { ...ended.failure, stage: "private/path" },
+ { ...ended.failure, hresult: "private" }, { ...ended.failure, exitCode: 2.5 }]) {
+ expect(nativeEventSchema.safeParse({ ...ended, failure }).success).toBe(false);
+ }
+ expect(nativeEventSchema.safeParse({ ...ended, failed: false }).success).toBe(false);
+ expect(nativeEventSchema.safeParse({ ...ended, failed: false, failure: undefined }).success).toBe(true);
+ });
+
it("distinguishes Host input failure from a local stalled queue without exporting input authority or text", async () => {
browser();
const debug = await import("../src/client/lib/debug");
diff --git a/tests/browser-encoding-producer.test.ts b/tests/browser-encoding-producer.test.ts
index 1e43e94b..baf2a0c5 100644
--- a/tests/browser-encoding-producer.test.ts
+++ b/tests/browser-encoding-producer.test.ts
@@ -1,6 +1,12 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { BrowserEncodingProducer } from "../src/client/media/browser-encoding-producer";
import { applyVideoCaptureProfile, configureVideoSender, QUALITY_PROFILES } from "../src/client/media/quality";
+import { cloneSenderVideoTrack } from "../src/client/media/sender-video-track";
+
+vi.mock("../src/client/media/sender-video-track", async (original) => ({
+ ...await original<typeof import("../src/client/media/sender-video-track")>(),
+ cloneSenderVideoTrack: vi.fn((source: MediaStreamTrack) => source.clone()),
+}));
vi.mock("../src/client/media/quality", async (original) => ({
...await original<typeof import("../src/client/media/quality")>(),
@@ -83,6 +89,7 @@ describe("Browser encoding producer startup", () => {
const localAnswer = (codec: string, fmtp = "") => ({ type: "answer" as const, sdp: [
"v=0", "o=- 1 1 IN IP4 127.0.0.1", "s=-", "t=0 0",
"m=video 9 UDP/TLS/RTP/SAVPF 96", "c=IN IP4 0.0.0.0", "a=recvonly",
+ "a=extmap:3 urn:3gpp:video-orientation", "a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid",
`a=rtpmap:96 ${codec}/90000`, ...(fmtp ? [`a=fmtp:96 ${fmtp}`] : []), "",
].join("\r\n") });
@@ -94,8 +101,11 @@ describe("Browser encoding producer startup", () => {
vi.spyOn(Connection.prototype, "createAnswer").mockResolvedValue(localAnswer(name, fmtp));
const { producer } = createProducer(codec);
await producer.start(3_333_999);
+ expect(cloneSenderVideoTrack).toHaveBeenCalledWith(expect.anything(), expect.any(Function), { normalizeOrientation: true });
const [send, receive] = Connection.instances;
const answer = receive.localDescription!;
+ expect(answer.sdp).not.toContain("urn:3gpp:video-orientation");
+ expect(answer.sdp).toContain("a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid");
expect(answer.sdp).toContain("x-google-start-bitrate=3333");
if (fmtp) expect(answer.sdp).toContain(fmtp);
expect(send.remoteDescription).toBe(answer);
@@ -119,7 +129,8 @@ describe("Browser encoding producer startup", () => {
vi.spyOn(Connection.prototype, "createAnswer").mockResolvedValue(answer);
const { producer } = createProducer();
await producer.start(budget);
- expect(Connection.instances[1].localDescription).toBe(answer);
+ expect(Connection.instances[1].localDescription!.sdp).not.toContain("x-google-start-bitrate");
+ expect(Connection.instances[1].localDescription!.sdp).not.toContain("urn:3gpp:video-orientation");
});
it("bounds the start hint by the Host ceiling without duplicating an existing hint", async () => {
diff --git a/tests/native-client.test.ts b/tests/native-client.test.ts
index 214516cb..b7a118ec 100644
--- a/tests/native-client.test.ts
+++ b/tests/native-client.test.ts
@@ -620,12 +620,14 @@ describe("native App private wire", () => {
it.each([undefined, false, true])("gates optional capture commands on advertised support: %s", async (supported) => {
const requests: Record<string, unknown>[] = [];
class CaptureSocket extends EventTarget {
+ static current: CaptureSocket;
static readonly OPEN = 1;
static readonly CLOSING = 2;
readyState = CaptureSocket.OPEN;
protocol = `piik-client-v10.${health.instanceToken}`;
constructor() {
super();
+ CaptureSocket.current = this;
queueMicrotask(() => this.dispatchEvent(new Event("open")));
}
close() { this.readyState = CaptureSocket.CLOSING; }
@@ -647,7 +649,7 @@ describe("native App private wire", () => {
vi.stubGlobal("WebSocket", CaptureSocket);
vi.stubGlobal("window", { setTimeout, clearTimeout, location: new URL("https://piik.example/") });
vi.stubGlobal("fetch", vi.fn(async () => new Response(JSON.stringify({
- ...health, nativeMedia: { ...health.nativeMedia, captureBorderControl: supported, remoteInput: supported, sourceCancellation: supported },
+ ...health, nativeMedia: { ...health.nativeMedia, captureBorderControl: supported, remoteInput: supported, sourceCancellation: supported, captureFailureDetails: supported },
}))));
const client = await NativeClient.connect();
expect(client).not.toBeNull();
@@ -670,6 +672,7 @@ describe("native App private wire", () => {
const extension = supported ? { showCaptureBorder: showCaptureBorder ?? false, reportSourceCancellation: true } : {};
expect(requests.at(-2)).toEqual({
version: NATIVE_CLIENT_PROTOCOL, id: expect.any(String), type: "start-share", ...input, ...extension,
+ ...(supported ? { reportCaptureFailure: true } : {}),
...(supported ? { remoteInput: true } : {}),
});
expect(requests.at(-1)).toEqual({
@@ -677,6 +680,14 @@ describe("native App private wire", () => {
shareId: input.shareId, source: input.source, audio: true, adapterIndex: 0, encoderIndex: 0, ...extension,
});
}
+ const onEvent = vi.fn(), onClose = vi.fn();
+ client!.onEvent(onEvent);
+ client!.onClose(onClose);
+ const ended = { version: NATIVE_CLIENT_PROTOCOL, type: "share-ended", shareId: input.shareId, failed: true,
+ ...(supported ? { failure: { stage: "windows-runtime", hresult: "0x887a0005", exitCode: 2 } } : {}) };
+ CaptureSocket.current.dispatchEvent(new MessageEvent("message", { data: JSON.stringify(ended) }));
+ expect(onEvent).toHaveBeenCalledWith(ended);
+ expect(onClose).not.toHaveBeenCalled();
const unsent = vi.fn();
vi.spyOn(CaptureSocket.prototype, "send").mockImplementationOnce(() => { throw new Error("send failed"); });
await expect(client!.startShare(input, unsent)).rejects.toMatchObject({ reason: "send-failed" });
diff --git a/tests/quality.test.ts b/tests/quality.test.ts
index ef65e1a9..4fd379a1 100644
--- a/tests/quality.test.ts
+++ b/tests/quality.test.ts
@@ -175,6 +175,7 @@ describe("realtime quality controls", () => {
autoGainControl: false,
voiceIsolation: false,
channelCount: { ideal: 2 },
+ restrictOwnAudio: true,
},
systemAudio: "include",
windowAudio: "window",
diff --git a/tests/sender-video-track.test.ts b/tests/sender-video-track.test.ts
index 6c60e7c5..1965b708 100644
--- a/tests/sender-video-track.test.ts
+++ b/tests/sender-video-track.test.ts
@@ -111,6 +111,37 @@ describe("sender display-frame ownership", () => {
expect(p.processor).not.toHaveBeenCalled();
});
+ it("normalizes only oriented shared frames and releases their snapshots", async () => {
+ const p = platform(), source = new Track();source.settings = { width: 640, height: 480 };
+ class Frame {
+ readonly close = vi.fn();
+ get rotation() { return 0; }
+ get flip() { return false; }
+ constructor(readonly bitmap: ImageBitmap, readonly init: VideoFrameInit) {}
+ }
+ const bitmap = { close: vi.fn() } as unknown as ImageBitmap;
+ const snapshot = vi.fn(async () => bitmap);
+ vi.stubGlobal("VideoFrame", Frame);
+ vi.stubGlobal("createImageBitmap", snapshot);
+ const track = cloneSenderVideoTrack(asTrack(source), vi.fn(), { normalizeOrientation: true });owned.push(track);
+ const input = source.clone.mock.results[0]!.value;
+ expect(input.applyConstraints).toHaveBeenCalledWith(source.getConstraints());
+ const plain = { rotation: 0, flip: false, close: vi.fn() } as unknown as VideoFrame;
+ const oriented = { rotation: 90, flip: true, displayWidth: 480, displayHeight: 640, timestamp: 123, duration: 456, close: vi.fn() } as unknown as VideoFrame;
+ p.frames().enqueue(plain);p.frames().enqueue(oriented);
+ await flush();
+ expect(p.write.mock.calls[0]![0]).toBe(plain);
+ expect(snapshot).toHaveBeenCalledExactlyOnceWith(oriented, { resizeWidth: 480, resizeHeight: 640 });
+ expect(p.write.mock.calls[1]![0]).toMatchObject({ bitmap, init: { timestamp: 123, duration: 456 } });
+ expect(plain.close).toHaveBeenCalledOnce();
+ expect(oriented.close).toHaveBeenCalledOnce();
+ expect(bitmap.close).toHaveBeenCalledOnce();
+ expect((p.write.mock.calls[1]![0] as VideoFrame).close).toHaveBeenCalledOnce();
+ stopSenderVideoTrack(track);await flush();
+ expect(p.cancel).toHaveBeenCalledOnce();
+ expect(source.stop).not.toHaveBeenCalled();
+ });
+
it("retires both ends if idle-refresh configuration fails before piping", async () => {
const p = platform(), source = new Track(), input = new Track(), ended = vi.fn();
source.clone.mockReturnValue(input);
diff --git a/vite.config.ts b/vite.config.ts
index dfdb0087..5c127961 100644
--- a/vite.config.ts
+++ b/vite.config.ts
@@ -9,6 +9,8 @@ const server = "http://127.0.0.1:8788";
export default defineConfig({
plugins: [react()],
+ // Local build output includes Browser profiles, not application entries.
+ optimizeDeps: { entries: ["index.html", "site/**/*.html"] },
build: {
outDir: "internal/server/webassets/dist",
emptyOutDir: false,