* sfu: filter VP9 and AV1 simulcast temporal layers with the dependency descriptor
Every simulcast codec gets an explicit temporal layer selector. With the
dependency descriptor, VP8, VP9 and AV1 filter temporal layers by frame and
forward the descriptor. Without it, a pass-through selector forwards every
temporal layer and reports the target as reached, so a pending target cannot
block allocation. VP8DD becomes the codec-neutral DDTemporal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: VP8-only padding header in DDTemporal, structure gate on presence
DDTemporal returns the one byte VP8 payload descriptor for padding only for
VP8. The descriptor forwarder takes a structure from any packet that carries
one, as the parser accepts it only on the first packet of a frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers across migration
The forwarder state carries the last dependency descriptor frame number
sent, so a migrated subscriber's first key frame continues the numbers
instead of being dropped as older than its last structure. Bumps protocol
for RTPForwarderState.dependency_descriptor_forwarder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: pass on a pending descriptor seed when migrating again
A forwarder seeded before its codec was known returns the pending seed in
its state, so a second migration keeps the frame numbers continuous.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: forward the dependency descriptor for VP8 simulcast
libwebrtc sends VP8 with a one byte payload descriptor when it sends the
dependency descriptor, so a VP8 subscriber gets neither picture IDs nor the
descriptor and finds frame references by sequence number alone. Under loss
with temporal filtering it stalls until a key frame.
DDForwarder rewrites the publisher's descriptor for each forwarded packet:
frame numbers continue across simulcast layer switches and stream restarts,
the structure goes out with the current layer's key frame, and the active
decode targets follow the forwarded temporal layer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers from going back
Rebase frame numbers only when a new stream would go back past the last
number sent: on a layer switch, a stream restart (also when it starts with a
delta frame) and a new forwarder on the same downtrack, which now carries
over the last number sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: set VP8 descriptor decode targets from the packet's layer
The active decode targets use the temporal layer the packet is forwarded at,
not the selector's next layer on the last packet before a down switch. They
also go out on key frames, combined with the publisher's own mask.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: carry the current active decode targets on every DD packet
The parser puts the stream's current mask on every packet, so a mask change
on a dropped packet still reaches the VP8 descriptor forwarder, which no
longer tracks the publisher's mask itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: take the first DD active decode targets mask at any sequence number
A publisher whose first packet has RTP sequence number 0 never had its key
frame's mask applied, so every packet reported no active decode targets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A filtered packet that follows a sequence number gap now keeps its slot
and gets padding in it. Rolling it back moved the last outgoing number
below the missing packets' slots, so their retransmissions were rejected
and the subscriber stalled on the hole until a key frame.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: parse VP8 and VP9 payload descriptors without an allocation
Replace ExtPacket.Payload with typed VP8 and VP9 pointer fields that point
at backing structs inside the ExtPacket, so parsing does not allocate and a
wrong codec type is a compile error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: copy the VP8 descriptor in the test packet helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The key frame requester re-armed its timer on every loop, so every video down
track woke up every 200 ms to 1 s for its whole life, even when the layer was
locked and there was nothing to do.
Re-arm the timer only while the requested layer is not locked, or while the
lock can change without an event: an acquisition grace is armed, or layers are
selected with the dependency descriptor, which can lose the lock on the packet
path. Post the event on resync, mute, max published layer change and packet
path layer switch, since those can change the lock without an allocation.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On a gap the RTP munger gives the missing sequence numbers outgoing slots.
If a late packet then arrives and is dropped, nothing is ever sent in its
slot, and a subscriber that finds frame references by sequence number
stalls until a key frame. The forwarder now flags such drops, and the
downtrack sends a one byte padding only packet in the slot.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: filter VP8 temporal layers by frame when the dependency descriptor is sent
With the dependency descriptor, libwebrtc sends a one byte VP8 payload
descriptor without a picture ID, so the VP8 munger's picture ID based
temporal drop logic stops filtering after the first gap. A new VP8DD codec
munger decides once per frame from the descriptor's frame number and
temporal ID and applies that decision to every packet of the frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: use the VP8 munger when picture IDs come with the dependency descriptor
VP8DD needs the one byte VP8 payload descriptor that libwebrtc sends with
the dependency descriptor. If a publisher keeps picture IDs next to the
descriptor, they need rewriting, so the forwarder switches to the VP8
munger and keeps switching temporal layers with the descriptor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
libwebrtc strips the VP8 payload descriptor when it sends the dependency
descriptor, so the layer sync bit that #4947 waits for is never set and a
subscriber that lowers its frame rate stays at the base layer until a key
frame. A new temporal layer selector reads only the dependency descriptor:
it switches up at a switch point whose referenced frames were all
forwarded. The forwarder uses it for VP8 when the extension is negotiated
and falls back to the Y-bit selector when the descriptor is not sent.
Fixes#4973.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu/buffer: parse the dependency descriptor without allocating
Every VP9 and AV1 SVC packet allocated six objects in the dependency
descriptor parse: the descriptor, a clone of the frame template with
its three slices, a resolution copy, and a frame entity that took the
address of a parameter.
The pooled ExtDependencyDescriptor now owns the storage for the
descriptor, the template and the resolution, with fixed arrays for the
template's slices. The reader copies the matching template into that
storage.
Delta frame parse: 6 allocs, 176 B, 174 ns -> 0 allocs, 0 B, 122 ns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/dd: never copy a parsed frame into a structure template
A caller can hand Unmarshal a descriptor whose FrameDependencies points
at one of the structure's templates. Copying the packet's custom fields
into it would change the template for every later packet, so the reader
allocates a fresh template in that case.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
A large frame number jump made addEntity walk every skipped frame,
twice. The ring holds only numEntries frames, so older ones get
overwritten anyway. Both loops now start at most numEntries behind
the new frame, like PacketHistory and FrameIntegrityChecker.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu/buffer: stop allocating per ingested packet
Buffer.Write allocated six objects for every RTP packet. They were
the parsed rtp.Packet, a copy of it in patchExtPacket, and the flow
state, whose address two rare log calls took. The rest were an
AbsCaptureTime, present on about one packet per second, and the
extension slice that pion grows with append.
Unmarshal into a pooled scratch packet. Give ExtPacket its own backing
rtp.Packet and an [8]rtp.Extension array, and copy the header into
them. Patch the payload in place. If the abs-capture-time extension is
present, parse it. Log a copy of the flow state.
Per packet, Write plus ReadExtended. Opus: 6 allocs, 448 B -> 0.
VP8: 6 allocs, 416 B -> 1 alloc, 32 B. Opus 240 -> 142 ns, VP8 269 -> 204 ns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: point extension payloads at the bucket copy on read
Queued ExtPackets kept extension payload slices into the write input,
which the transport reuses for the next packet. ReadExtended now parses
the header again from the bucket copy, so extensions and CSRCs point
into RawPacket like the payload does.
The test writes every packet through one input buffer and checks the
audio level of each packet read back.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: test that a dropped padding packet keeps the shifted sequence number on read
The bucket stores the shifted number, so re-parsing the header on read
returns it. This test guards that.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
RedReceiver kept pointers to the last forwarded packets for its
redundant blocks, but their payload is in the read buffer that
forwardRTP reuses for every packet, so the blocks carried the current
packet's bytes. Copy the history packets into slots RedReceiver owns.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix: use the most common recent packet rate in connection quality
The loop that finds the mode packet rate compared a bin's count with the
index of the bin picked so far, so it did not find the most common rate.
Take the bin with the highest count, skipping rates below 10 pps so that
idle, DTX and static content rates do not become the reference. Halve the
counts at each mode update so that the mode follows a lasting change of
rate, and keep the previous mode through a long stretch of low rates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix: start the mode timer from the start time
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix: do not read invalid delta
In TWCC feedback, only the small delta and large delta symbols carry a
receive delta. The send side BWE loop used a delta for every received
symbol, so the reserved symbol read a delta that does not exist. Skip
that symbol and do not read past the last delta.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test: assert loss count and delta use for the reserved symbol
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: report forwarding latency as p90 instead of the mean
The forwarding-latency metric was the mean transit over all forwarded
packets. A mean is dominated by a few slow outliers, so a handful of
packets stalled on the forward path (e.g. goroutine scheduling latency)
inflated the whole node's reported latency even when nearly every packet
was forwarded promptly.
Report p90 instead: p90 rising means roughly a tenth of forwarded packets
are slow, a broad signal of systemic forwarding load rather than a sparse
tail. To read a percentile over the report window, the mergeable
per-interval summary now keeps a small power-of-two-bucket histogram of
transit instead of running moments (sum, sum-of-squares).
Drop the jitter (transit std dev) gauge: nothing consumed it, and any
spread is derivable from the forward-latency histogram. The protobuf
ForwardJitter field is left in place, now unset, to deprecate separately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: clamp forwarding percentile to the observed [min, max]
Bucket interpolation assumes a uniform fill, so a single 20ms packet (or
uniform traffic) could report a p90 above every observed sample. Clamp the
interpolated value to the summary's already-tracked min/max, so a quantile
never falls outside the data. Exact for single samples and repeated
identical latencies.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: fix stale p50 comment in the percentile test
The reported metric is p90; the test comment still said p50 replaced the
mean. Reword it to reflect that a percentile, not the mean, is reported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: quarter-octave forwarding-latency buckets for threshold resolution
Octave buckets are too coarse near the overload thresholds: 300us falls in
[256,512), so a p90 clustered at ~265us and one at ~500us interpolate to the
same value and would trip (or not) identically. Split each octave into four
linear sub-buckets so the two land in different buckets, on the correct side
of the threshold. Min/max clamping alone does not fix this once a node has a
high tail, since its max no longer bounds the interpolation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: interpolate percentiles within each bucket's observed range
Replace the quarter-octave split with plain octave buckets that also carry
the observed [min, max] of their samples, and interpolate a percentile
within that range instead of the bucket's nominal edges. This is exact when
a bucket's samples cluster, so a p90 near an overload threshold that falls
mid-bucket lands on the correct side of it regardless of bucket width -- no
threshold-aware boundaries needed. It subsumes the min/max clamp, since an
estimate can no longer leave the observed samples.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: cover the 500us cluster in the threshold test
Assert milos's full review example exactly: 265us and 500us clusters that
octave-nominal interpolation both read as ~341us now read 265us and 500us.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: make forwardSummary.addSample a pointer receiver
The summary grew from a small moments struct into a per-bucket histogram
(~700 bytes), so the value-receiver addSample copied the whole summary on
every drained sample in the flush loop. Mutate in place instead: ~28ns ->
~2ns per sample in the background fold, no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: eighth-octave buckets for accuracy near thresholds
Split each octave into 8 linear sub-buckets (was plain octaves). With the
per-bucket min/max interpolation this reports a tight p90 cluster exactly
even when it sits mid-octave: 850x265us + 150x410us now reads 410us (above a
400us threshold) instead of 395.5us, and a lognormal p90 lands within ~0.2us
of exact. Per-sample add cost is unchanged (~2ns); cost is ~5KB per summary
and a larger but per-report merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* agent: release JobTerminate handler when job ends before registration
JobRequest registered the job's JobTerminate handler after AssignJob
returned, so an ended UpdateJobStatus or worker disconnect handled in
between deregistered nothing and the handler leaked. AssignJob also sent
the assignment before recording the job as running, so an immediate
ended update was dropped.
Record the job before sending the assignment, and after registering the
handler recheck under h.mu that the job is still running and its worker
still registered, deregistering otherwise.
Fixes#4901
* replace utils.CloneProto with proto.CloneOf
* agent: wait on worker readiness and poll for released handlers in test
* sfu: don't hold bindLock across the blank-frame flush in CloseWithFlush
CloseWithFlush held bindLock across the up-to-flushTimeout (1s) wait for
the closing blank-frame flush. bindLock is a control-plane lock, also
taken by Bind/SetConnected/ReceiverRestart, so every close serialized
those behind its flush wait. This was the single largest accumulator in
mutex contention profiling under track churn, aggregated across all
closing downtracks.
The flush runs in its own goroutine (writeBlankFrameRTP) and is cancelled
via blankFramesGeneration, so the lock guards nothing during the wait;
release it and re-acquire for the unbind/delete teardown.
Guard the control-plane mutators on isClosed so none can act on a
downtrack through the newly opened window. Bind in particular was
reachable: its bindState != unbound check does not catch a closed track,
since close leaves the state unbound.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: drop redundant isClosed guard in Bind
Bind already handles a closed downtrack after codec matching (returns the
codec with no error, preserving pion's non-error contract and never
rebuilding state). The early guard returned errDownTrackClosed, which
handleRemoteAnswerReceived treats as a negotiation failure, so a
close/negotiation race could abort the remote answer. Remove it and the
error type.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* ccutils: use millisecond resolution for probe interval backoff
ProbeSignal grew probeInterval via Seconds(), which truncates the float
to the nearest nanosecond before scaling by time.Second:
time.Duration(probeInterval.Seconds() * BackoffFactor) * time.Second
With BackoffFactor=1.5 and the default BaseInterval of 3 s, the first
congestion signal produced 4 s instead of 4.5 s, and the error compounded
on every subsequent signal (4 s → 6 s → 9 s → 13 s ... instead of
4.5 s → 6.75 s → 10.125 s ...). The interval converges to MaxInterval
later than intended, so the server probes more frequently during sustained
congestion than the configuration requests.
probeDuration in the same function already uses the correct idiom:
time.Duration(float64(probeDuration.Milliseconds()) * Factor) * time.Millisecond
Apply the same pattern to probeInterval so both fields are computed
consistently with millisecond precision.
Found by a defect-hunting pipeline I build and run ([Dev-next-gen](https://github.com/Dev-next-gen)), using Claude Code with Anthropic's Claude Opus 5.
* ccutils: remove probe interval backoff test
Requested by reviewer.
* sfu: add opt-in verbatim abs-capture-time forwarding
Downtrack ACT forwarding gates on an RTCP sender report and
unconditionally rewrites the capture timestamp into SFU clock domain,
which suits cross-stream A/V sync but breaks consumers that need the
original publisher capture time (e.g. end-to-end latency measurement).
Add ForwardAbsCaptureTimeVerbatim (RTC config: forward_abs_capture_time_verbatim),
off by default, which skips both the gate and the rewrite.
* fix: use DisableSenderReportPassThrough setting to also gate abs-capture-time forwarding
* sfu: broadcast RTP to down tracks without a per-packet closure
Every forwarded packet allocated a closure capturing the packet and layer,
and an atomic write counter that escaped with it, even for tracks with a
single subscriber. Add BroadcastRTP, which carries the packet and layer in
the worker state and sums the writes per worker. The serial path no longer
allocates; the parallel path allocates the shared state and the worker
funcval only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: reuse the forwarded packet copy in the RED receivers
Both RED receivers copied the ExtPacket (and the opus one the rtp.Packet)
per forwarded packet, and the copies moved to the heap because the
broadcast takes their address. Keep them on the receiver like the existing
redPayloadBuf: ForwardRTP runs on one goroutine and down tracks do not keep
the packet past WriteRTP.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: skip BroadcastRTP allocation assertion under the race detector
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: group BroadcastRTP and its interfaces after the spreader methods
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
sync.Pool drops a quarter of returned items when built with -race, so the
pooled send path averages about one allocation per packet in CI and the
zero-allocation assertion flips between passing and failing. Keep the
extension checks and skip only the allocation count under race.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
SendPacket allocated four times per down track write: a 3 byte slice for
abs-send-time, a 2 byte slice for transport-cc, and two appends to a nil
Extensions slice because the pooled header was reset to a zero value.
Give pacer.Packet fixed scratch arrays for the two extensions and marshal
into them. The Packet is pooled and owned by one send until SendPacket
returns, so nothing is shared across down tracks. Keep the pooled header's
Extensions capacity across reuse, both when returning it to the pool and
when a down track initializes it.
Adds a test that asserts zero allocations per SendPacket.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
CodecMunger.UpdateAndGet returned a freshly allocated slice for every
forwarded packet on every down track. Return the header in a fixed array
with a length instead and marshal into it with MarshalTo. TranslationParams
carries the array; WriteRTP slices it locally.
UpdateAndGet now allocates zero times per call (was one). Escape analysis
confirms TranslationParams stays on the stack.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
pion no longer starts the repair stream reader when a custom BufferFactory
is set, so the mid/rid/rsid extensions were never observed and simulcast RTX
streams were never paired with their primary streams.
Extract the extensions on the buffer write path instead. Migrated publishers
send no extensions at all, so pair those from SimTracks.
Adds an integration test covering both paths, and moves the vnet setup it
shares with the downtrack test into pkg/testutils/vnettest.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
VP9 SVC sends each spatial layer as its own encoded frame,
so a picture carries one trailer per layer, but only the
top layer's last packet has the RTP marker bit set.
Fix https://github.com/livekit/egress/issues/1347
Flush the downtrack sequencer on stream restart (Resync, ReceiverRestart,
codec change) so NACK retransmissions can't use metadata that no longer
matches the resynced bucket. Add a defensive bounds guard on the RTX and
forward payload slicing.
Cap the PacketHistory and FrameIntegrityChecker catch-up loops to the ring
size so a large sequence/frame-number jump can't drive a big per-packet
iteration count.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the per-NACK-packet goroutine spawn in DownTrack.handleRTCP with a
single long-lived worker that coalesces pending NACKs into one retransmit
pass. Pending sequence numbers are capped so a high NACK arrival rate cannot
grow goroutine count or memory unboundedly.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* log high stream start latency.
There is something wrong in measurement as audio is showing high p99
latency. Must be misattributing samples. So, logging for high latency to
understand this better.
* use correct variable
* time since create
* Fix getRefLayerRTPTimestamp off-by-one that can panic on max layer index.
Reject ref/target layers with >= len(refInfos) so layer==len is an error instead of an out-of-range index.
* Remove historical comment from ref-layer bounds test.
Keep the regression coverage without referencing the old bounds check in source.
* Record subscribe stream start time in prometheus.
Adjust for mutes, i. e. take the last unmute time as the start point and
calculate time till the first byte is sent.
* close the tiny window of race
* Prevent long tail sample when publisher glitches.
Thanks to @milos-lk for this.
Publisher restarting would have reset the layer and would have caused a
sample with very high stream start time. We only need to capture when we
do a dummy start or when the state is seeded to a different node upon
migration.
* reduce a diff
* test
* changed the wrong thing, thank you Devin
* Tests for down stream packet push.
A recent issue (padding bit in RTP header) surfaced a gap which slipped
through due to lack of tests. Changes in pion/rtp were not adopted
properly.
So, adding some tests (thank you Claude for the heavy lifting) to test
the down stream packet path using the whole pion chain.
Split out some interfaces so it is easier to have it all in one place
and create fakes.
Will help adding more tests, for example include the upstream path also
in the integration test. May have to create more interfaces and make
things testable, but this is a start.
* missed file
* rtx specific test
