* sfu: filter VP9 and AV1 simulcast temporal layers with the dependency descriptor
Every simulcast codec gets an explicit temporal layer selector. With the
dependency descriptor, VP8, VP9 and AV1 filter temporal layers by frame and
forward the descriptor. Without it, a pass-through selector forwards every
temporal layer and reports the target as reached, so a pending target cannot
block allocation. VP8DD becomes the codec-neutral DDTemporal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: VP8-only padding header in DDTemporal, structure gate on presence
DDTemporal returns the one byte VP8 payload descriptor for padding only for
VP8. The descriptor forwarder takes a structure from any packet that carries
one, as the parser accepts it only on the first packet of a frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers across migration
The forwarder state carries the last dependency descriptor frame number
sent, so a migrated subscriber's first key frame continues the numbers
instead of being dropped as older than its last structure. Bumps protocol
for RTPForwarderState.dependency_descriptor_forwarder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: pass on a pending descriptor seed when migrating again
A forwarder seeded before its codec was known returns the pending seed in
its state, so a second migration keeps the frame numbers continuous.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: forward the dependency descriptor for VP8 simulcast
libwebrtc sends VP8 with a one byte payload descriptor when it sends the
dependency descriptor, so a VP8 subscriber gets neither picture IDs nor the
descriptor and finds frame references by sequence number alone. Under loss
with temporal filtering it stalls until a key frame.
DDForwarder rewrites the publisher's descriptor for each forwarded packet:
frame numbers continue across simulcast layer switches and stream restarts,
the structure goes out with the current layer's key frame, and the active
decode targets follow the forwarded temporal layer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers from going back
Rebase frame numbers only when a new stream would go back past the last
number sent: on a layer switch, a stream restart (also when it starts with a
delta frame) and a new forwarder on the same downtrack, which now carries
over the last number sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: set VP8 descriptor decode targets from the packet's layer
The active decode targets use the temporal layer the packet is forwarded at,
not the selector's next layer on the last packet before a down switch. They
also go out on key frames, combined with the publisher's own mask.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: carry the current active decode targets on every DD packet
The parser puts the stream's current mask on every packet, so a mask change
on a dropped packet still reaches the VP8 descriptor forwarder, which no
longer tracks the publisher's mask itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: take the first DD active decode targets mask at any sequence number
A publisher whose first packet has RTP sequence number 0 never had its key
frame's mask applied, so every packet reported no active decode targets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: parse VP8 and VP9 payload descriptors without an allocation
Replace ExtPacket.Payload with typed VP8 and VP9 pointer fields that point
at backing structs inside the ExtPacket, so parsing does not allocate and a
wrong codec type is a compile error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: copy the VP8 descriptor in the test packet helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
libwebrtc strips the VP8 payload descriptor when it sends the dependency
descriptor, so the layer sync bit that #4947 waits for is never set and a
subscriber that lowers its frame rate stays at the base layer until a key
frame. A new temporal layer selector reads only the dependency descriptor:
it switches up at a switch point whose referenced frames were all
forwarded. The forwarder uses it for VP8 when the extension is negotiated
and falls back to the Y-bit selector when the descriptor is not sent.
Fixes#4973.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A large frame number jump made addEntity walk every skipped frame,
twice. The ring holds only numEntries frames, so older ones get
overwritten anyway. Both loops now start at most numEntries behind
the new frame, like PacketHistory and FrameIntegrityChecker.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: acquire requested video layer directly at HIGH quality by default
Two changes that together remove the visible low->high quality ramp for a new
subscriber (both publisher-first and subscriber-first join orders):
1. Default a subscriber's initial video quality to HIGH on bind instead of LOW
for adaptive stream, so the subscribed max layer is the top layer. Adaptive
stream clients can still scale down afterwards based on viewport.
2. On initial layer acquisition the forwarder/selector latch directly onto the
allocator's target (the requested top layer) instead of opportunistically
latching onto the first lower key frame that arrives. A short
initial-acquisition grace aims the target at the requested layer; if it does
not show up in time, the target falls back to the highest layer seen so
acquisition never stalls.
Always on - no configuration flag.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: gate start-at-desired-quality behind EnableStartAtDesiredQuality flag
Put the "acquire requested video layer directly at HIGH quality" behavior
behind a per-subscriber EnableStartAtDesiredQuality flag (default off, so
the original low->high ramp-up is restored unless enabled).
Plumbed from config.RTC.EnableStartAtDesiredQuality through ParticipantParams
-> SubscribedTrack/DownTrack -> Forwarder -> simulcast selector, gating all
three behavior changes: the HIGH default on bind, the forwarder's
initial-acquisition grace, and the selector's direct-latch-onto-target.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* remove config.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Some logging changes.
Trying to chase a case of large sequence number gap on subscriber side
where packets are sent after a long time.
* return values instead of logging
* Add support for "abs-capture-time" extension.
Currently, it is just passed through from publisher -> subscriber side.
TODO: Need to store in sequencer and restore for retransmission.
* abs-capture-time in retransmissions
* clean up
* fix test
* more test fixes
* more test fixes
* more test fixes
* log only when size is non-zero
* log on both sides for debugging
* add marshal/unmarshal
* normalize abs capture time to SFU clock
* comment out adding abs-capture-time from registered extensions
* Add debug to understand VP9 freezes.
Have reports of VP9 freezing in some rooms.
Some data indicates that NACKs are received by SFU, but cannot get RTP
packet when that happens. It is possible that the NACKs are all from
dropped packets. Adding some debug to understand drops/NACKs better.
* enable DD debug
* comment out DD debug
* markers
* add back log about diff length mismatch
* add back key frame mismatch logging
* log skipped drops also
* Do server PLI when sync is required.
A few changes
- Run key frame requester goroutine always. Runs every 200 ms which is
not bad.
- Post a key frame request when server knows it needs one, like after an
allocation. This ensures that the initial request is not delayed.
- Periodic check will ensure PLI for cases like all frame chains of a
dependency descriptor being broken.
* simplify
When starting from scratch (like mute -> unmute), it is possible
that the check sync does not detect a broken chain. That results
in PLIs not being sent and the video frozen till a gratuitous key
frame arrives.
Unclear why there are not PLIs from client side. That is something else to
dig into.
SVC has only one stream and when calculating reference time stamp,
irrespective of reference layer, reference time stamp will be the
same as the given time stamp as there is only one stream and no offset.
TODO: Need better all around SVC handling.
* Remove parked layer feature.
Not worth the added complexity.
Several reasons
- Not seeing black frames on pub mute always.
- If they are there, it can consume more than 30kbps if the parked layer
is high res. That is wasted bandwidth downstream when pub is muted.
- On resume, client some time sends PLI and that triggers a key frame
request.
But, leaving the separate `PubMuted` flag in forwarder in case we can
use it for better handling.
* need the request spatial
