Skip to content

armgit / Piik

Public Imported from GitHub
0

Free, open-source screen sharing for private live streams with friends. Watch together in a browser or self-host Piik. 免费开源的私密屏幕共享,支持游戏直播、一起看电影与自部署。

fix(native): preserve jittered capture timestamps (#78)

AAA218a8b31d82d
diff --git a/docs/adr/0006-fixed-high-native-sender-canary.md b/docs/adr/0006-fixed-high-native-sender-canary.md
index 58935b16..7d7c51d3 100644
--- a/docs/adr/0006-fixed-high-native-sender-canary.md
+++ b/docs/adr/0006-fixed-high-native-sender-canary.md
@@ -128,6 +128,19 @@ send returned, but the retained fatal marker and missing diagnostics do not prov
where or why the product path stopped. No raw cause may be inferred. There is
still no Viewer, two-viewer, FIFO, or physical or hardware-encoder proof.
+A source follow-up identifies and removes one concrete fatal path without relabeling
+that retained run. WebCodecs output duration is nullable; the local sender uses
+33,333 microseconds when it is absent, while a valid next capture timestamp may
+arrive slightly earlier. The prior RTP timeline rejected that strictly
+increasing pair as `encoded frame timestamps overlap`. The candidate now derives
+RTP cadence from adjacent strictly increasing source timestamps and uses duration
+only to report positive source gaps. Focused tests retain dropped-frame timing,
+non-increasing rejection, and sub-sample progress; the real `/media` handler also
+accepts two frames at 1,000,000 and 1,033,000 microseconds when the first reports
+33,333 microseconds. This proves the source path no longer raises that fatal, not
+that it caused the retained browser failure. No post-fix Chrome smoke ran, so
+ADR-0006 remains product-gate no-go.
+
## Staged Revalidation
Any further run requires separate authorization and must stop at the first
diff --git a/docs/project-memory.md b/docs/project-memory.md
index b22214d3..38682173 100644
--- a/docs/project-memory.md
+++ b/docs/project-memory.md
@@ -23,7 +23,7 @@ Last updated: 2026-08-20
- Use direct host P2P for one or two viewers. Keep ADR-0004 off broadly; use an isolated exact-room candidate until resource, quality, recovery, SFU/UDP, bounded-failure, and browser/mobile gates pass.
- Browser relays resend remote `MediaStreamTrack` values and re-encode at each hop; WebRTC does not guarantee a shared encoder across peer connections, so measure the cost.
- Keep experiments bounded: the standard representation sequence is below; native RTP relay, encoded-object striping, and FEC remain separate.
-- ADR-0006 stays no-go: one evidence-clarification run retained VP8 output and one first-generation binary `send()` return, then no post-send diagnostics and at least one sanitized fatal marker. The media-counter zeros are initial values, not Go reports; no Viewer was created, cleanup passed 5/5, and no product root cause is proven. A separate MF fixture proved RTX H.264 hardware over 360/360 frames, but `42c01f` misses Pion's default fmtp; Viewer interop remains unproven. Native v2 stays memory-only and leaves SQLite/Web unchanged.
+- ADR-0006 stays no-go: the clarification run retained VP8 output and one first-generation binary `send()` return, then no post-send diagnostics and a sanitized fatal marker; media zeros are initial, no Viewer was created, and cleanup passed 5/5. Source now removes one deterministic timestamp-overlap fatal by deriving RTP cadence from increasing capture timestamps rather than nullable WebCodecs duration. Focused/handler Go tests pass, but the retained fatal category and post-fix Chrome/Viewer proof are absent. The MF RTX H.264 fixture passed 360/360, but `42c01f` misses Pion's default fmtp. Native v2 remains memory-only and leaves SQLite/Web unchanged.
- Mobile Web Host is unsupported; feature-detect and fail clearly. Mobile Viewer stays leaf-only. After Windows native, gate Android 14+; iOS waits for stable iOS 27 ScreenCaptureKit.
- ADR-0005 accepts direct/peer UDP, bounded SFU roots, then optional selected-edge TURN. Source has a default-off, undeployed one-shot rebuild after SFU; participant-wide TURN remains rejected and production stays STUN-only.
- Keep the controller exact-room only: room `1` is the STUN/SFU smoke. Preserve sticky progressing P2P, mobile leaves and break-before-make. Recovery spends one attempt per layer (ICE restart, same-parent rebuild, alternate peer, then SFU), never three identical retries; active SFU gets one fresh grant before Peer failback.
diff --git a/docs/research/native-shared-encode-sender.md b/docs/research/native-shared-encode-sender.md
index f116bbdc..2deb3021 100644
--- a/docs/research/native-shared-encode-sender.md
+++ b/docs/research/native-shared-encode-sender.md
@@ -311,6 +311,28 @@ second Viewer, FIFO, or production run followed. The result remains
or why the product path stopped, so no underlying close or raw cause may be
inferred.
+A later static follow-up identified a deterministic source fatal candidate, but
+the retained run did not preserve the fatal category needed to attribute its
+failure to that branch. WebCodecs defines `EncodedVideoChunk.duration` as
+nullable and copies output timestamp and duration from the input `VideoFrame`;
+it does not establish the prior product assumption that the next timestamp must
+be at least the previous timestamp plus duration. The local sender substitutes
+33,333 microseconds when duration is absent, so ordinary capture jitter such as
+1,000,000 then 1,033,000 microseconds previously reached `encoded frame
+timestamps overlap` despite strictly increasing source time.
+
+The source candidate now packetizes the first frame without advancing Pion's
+clock, then advances it by each positive adjacent source-timestamp delta before
+packetizing the next frame. Reported duration remains only for positive
+dropped-source-time diagnostics; equal or decreasing timestamps still fail
+closed. Focused timeline tests cover shorter-than-duration jitter, dropped
+source time, non-increasing input, and a positive delta below one 90 kHz sample.
+A real `/media` handler test carries the two jittered frames through decode and
+fanout with two frames written and positive source RTP. All Native Go packages
+pass. These checks prove that specific fatal path is removed; without a post-fix
+Chrome smoke they do not classify the retained run or prove Viewer delivery.
+The WebCodecs and pinned Pion packetizer sources were rechecked 2026-08-20.
+
The frozen probe continues to bind the first actual local media WebSocket as
gate-local generation 1. A second bridge generation saturates at 2, stops
accumulating the first generation's counters, and fails the Viewer signal,
diff --git a/docs/status.md b/docs/status.md
index 7dd44efe..05b13a64 100644
--- a/docs/status.md
+++ b/docs/status.md
@@ -35,7 +35,7 @@ and is not part of production.
- Host A+B and authenticated P2P Viewer C are sanitized, generation-bound, read-only, and fail closed for stale, ambiguous, or SFU-fed evidence; no raw media metadata is retained.
- SVC is `no-go-web-svc-cross-path-hardware-contract`: no direct/peer selection, cross-PC shared encode, or portable hardware proof; pinned screen share is `L1T3`. No browser run was warranted.
- The `a11a73d` built-in TURN canary is rejected: local allocation passed, but direct Host/Pion Viewer failed before forced relay. Full rollback restored STUN-only client ICE and zero allocations.
-- Native remains no-go: fixed VP8, one encoder and a generation-1 binary send returned; no later diagnostics arrived, a sanitized fatal marker was retained, and the media zeros are initial values rather than Go reports. No Viewer was created, cleanup passed 5/5, and no product root cause is proven.
+- Native remains no-go: the retained VP8 run reached one encoder and generation-1 binary send, then no diagnostics, a fatal marker and initial media zeros; no Viewer. Source removes a deterministic timestamp-overlap fatal and passes timeline/`/media` tests, but the retained category and post-fix Chrome/Viewer proof are absent.
- Access protocol/config/HTTP/storage/SQLite/signaling focused tests pass, including commit-first teardown and v1 rollback.
- Selected-edge typecheck and 131 focused control tests pass; no TURN allocation/media evidence exists.
@@ -61,9 +61,9 @@ The recovery target uses one ICE restart, one same-parent rebuild, one alternate
SFU; it never runs three identical retries or abandons progressing P2P early.
Next gate exactly-two/Dynacast-off BWE on a zero-child SFU leaf, then root
-evacuation. Native stopped after one binary send return: no later diagnostics,
-a retained fatal marker, and uninterpretable media zeros. Classify this boundary
-before any new run.
+evacuation. Native source accepts increasing capture timestamps shorter than
+duration, but has no post-fix browser evidence. The next authorized run must
+prove stage-1 bridge ingress/source RTP before a Viewer.
Audio A/B/C may proceed without displacing P0.
Ordinary Peer ICE stays STUN-only; participant-wide TURN is removed. Selected-edge
is source-complete/default-off/undeployed. Next run one isolated forced-relay
diff --git a/native/sender/internal/app/server_test.go b/native/sender/internal/app/server_test.go
index 6cc8768f..753f9a8e 100644
--- a/native/sender/internal/app/server_test.go
+++ b/native/sender/internal/app/server_test.go
@@ -176,7 +176,7 @@ func TestAcceptedMediaWithoutConfigFailsWithinTheHandshakeDeadline(t *testing.T)
assertAbandonRoom(t, remoteDone)
}
-func TestConfiguredMediaBinaryFrameReachesFanout(t *testing.T) {
+func TestConfiguredMediaFramesWithCaptureJitterReachFanout(t *testing.T) {
application, events, terminal := startObservedTestApplication(t)
connection := openConfiguredMedia(t, application)
@@ -191,11 +191,19 @@ func TestConfiguredMediaBinaryFrameReachesFanout(t *testing.T) {
if err != nil {
t.Fatalf("write encoded frame: %v", err)
}
+ payload[0] = 2
+ binary.BigEndian.PutUint64(payload[1:9], 1_033_000)
+ writeContext, cancelWrite = context.WithTimeout(context.Background(), time.Second)
+ err = connection.Write(writeContext, websocket.MessageBinary, payload)
+ cancelWrite()
+ if err != nil {
+ t.Fatalf("write jittered encoded frame: %v", err)
+ }
diagnostics := awaitRemoteEvent(t, events, "diagnostics", 5*time.Second)
- if diagnostics.Media == nil || diagnostics.Media.FramesWritten == 0 ||
+ if diagnostics.Media == nil || diagnostics.Media.FramesWritten < 2 ||
diagnostics.Media.SourceRTPPacketsWritten == 0 {
- t.Fatalf("binary frame did not reach fanout: %+v", diagnostics.Media)
+ t.Fatalf("binary frames did not reach fanout: %+v", diagnostics.Media)
}
if err = connection.Close(websocket.StatusNormalClosure, "test complete"); err != nil {
t.Fatalf("close configured media: %v", err)
diff --git a/native/sender/internal/media/fanout.go b/native/sender/internal/media/fanout.go
index d331dcd1..d6bb8094 100644
--- a/native/sender/internal/media/fanout.go
+++ b/native/sender/internal/media/fanout.go
@@ -192,32 +192,26 @@ func newFrameTimeline() *frameTimeline {
func (timeline *frameTimeline) Packetize(frame Frame) ([]*rtp.Packet, uint64, error) {
var gapMicros uint64
+ // Source timestamps own RTP cadence. WebCodecs duration is nullable and can
+ // extend past the next live-capture timestamp, so it is diagnostic only.
if timeline.hasPrevious {
if frame.TimestampMicros <= timeline.previousTimestampMicros {
return nil, 0, errors.New("encoded frame timestamp did not increase")
}
- if timeline.previousTimestampMicros > math.MaxUint64-timeline.previousDurationMicros {
- return nil, 0, errors.New("encoded frame timestamp overflowed")
+ deltaMicros := frame.TimestampMicros - timeline.previousTimestampMicros
+ if deltaMicros > timeline.previousDurationMicros {
+ gapMicros = deltaMicros - timeline.previousDurationMicros
}
- expected := timeline.previousTimestampMicros + timeline.previousDurationMicros
- if frame.TimestampMicros < expected {
- return nil, 0, errors.New("encoded frame timestamps overlap")
- }
- gapMicros = frame.TimestampMicros - expected
- gapSamples, err := microsToSamples(gapMicros)
+ deltaSamples, err := microsToSamples(deltaMicros)
if err != nil {
return nil, 0, err
}
- timeline.packetizer.SkipSamples(gapSamples)
- }
- durationSamples, err := microsToSamples(frame.DurationMicros)
- if err != nil {
- return nil, 0, err
- }
- if durationSamples == 0 {
- durationSamples = uint32(videoClockRate / 30)
+ if deltaSamples == 0 {
+ deltaSamples = 1
+ }
+ timeline.packetizer.SkipSamples(deltaSamples)
}
- packets := timeline.packetizer.Packetize(frame.Data, durationSamples)
+ packets := timeline.packetizer.Packetize(frame.Data, 0)
timeline.hasPrevious = true
timeline.previousTimestampMicros = frame.TimestampMicros
timeline.previousDurationMicros = frame.DurationMicros
diff --git a/native/sender/internal/media/fanout_test.go b/native/sender/internal/media/fanout_test.go
index 454d0196..869cc6e2 100644
--- a/native/sender/internal/media/fanout_test.go
+++ b/native/sender/internal/media/fanout_test.go
@@ -38,23 +38,64 @@ func TestFrameTimelinePreservesDroppedSourceTime(t *testing.T) {
}
}
-func TestFrameTimelineRejectsOverlappingSourceTime(t *testing.T) {
+func TestFrameTimelineAcceptsCaptureJitterShorterThanReportedDuration(t *testing.T) {
timeline := newFrameTimeline()
- _, _, err := timeline.Packetize(Frame{
- TimestampMicros: 100,
- DurationMicros: 50,
+ first, _, err := timeline.Packetize(Frame{
+ TimestampMicros: 100_000,
+ DurationMicros: 33_333,
Data: []byte{1},
})
if err != nil {
t.Fatal(err)
}
- _, _, err = timeline.Packetize(Frame{
- TimestampMicros: 149,
- DurationMicros: 50,
+ second, gap, err := timeline.Packetize(Frame{
+ TimestampMicros: 133_000,
+ DurationMicros: 33_333,
Data: []byte{2},
})
- if err == nil {
- t.Fatal("overlapping source timestamps were accepted")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if gap != 0 {
+ t.Fatalf("capture jitter reported a source gap of %d microseconds", gap)
+ }
+ if delta := second[0].Timestamp - first[0].Timestamp; delta != 2_970 {
+ t.Fatalf("RTP timestamp delta = %d, want 2970 samples", delta)
+ }
+}
+
+func TestFrameTimelineRejectsNonIncreasingSourceTime(t *testing.T) {
+ for _, timestamp := range []uint64{100, 99} {
+ timeline := newFrameTimeline()
+ if _, _, err := timeline.Packetize(Frame{
+ TimestampMicros: 100,
+ DurationMicros: 1,
+ Data: []byte{1},
+ }); err != nil {
+ t.Fatal(err)
+ }
+ if _, _, err := timeline.Packetize(Frame{
+ TimestampMicros: timestamp,
+ DurationMicros: 1,
+ Data: []byte{2},
+ }); err == nil {
+ t.Fatalf("non-increasing source timestamp %d was accepted", timestamp)
+ }
+ }
+}
+
+func TestFrameTimelineAdvancesForPositiveSubsampleDelta(t *testing.T) {
+ timeline := newFrameTimeline()
+ first, _, err := timeline.Packetize(Frame{TimestampMicros: 1, DurationMicros: 1, Data: []byte{1}})
+ if err != nil {
+ t.Fatal(err)
+ }
+ second, _, err := timeline.Packetize(Frame{TimestampMicros: 2, DurationMicros: 1, Data: []byte{2}})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if delta := second[0].Timestamp - first[0].Timestamp; delta != 1 {
+ t.Fatalf("RTP timestamp delta = %d, want 1 sample", delta)
}
}