* feat: acquire requested video layer directly at HIGH quality by default
Two changes that together remove the visible low->high quality ramp for a new
subscriber (both publisher-first and subscriber-first join orders):
1. Default a subscriber's initial video quality to HIGH on bind instead of LOW
for adaptive stream, so the subscribed max layer is the top layer. Adaptive
stream clients can still scale down afterwards based on viewport.
2. On initial layer acquisition the forwarder/selector latch directly onto the
allocator's target (the requested top layer) instead of opportunistically
latching onto the first lower key frame that arrives. A short
initial-acquisition grace aims the target at the requested layer; if it does
not show up in time, the target falls back to the highest layer seen so
acquisition never stalls.
Always on - no configuration flag.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: gate start-at-desired-quality behind EnableStartAtDesiredQuality flag
Put the "acquire requested video layer directly at HIGH quality" behavior
behind a per-subscriber EnableStartAtDesiredQuality flag (default off, so
the original low->high ramp-up is restored unless enabled).
Plumbed from config.RTC.EnableStartAtDesiredQuality through ParticipantParams
-> SubscribedTrack/DownTrack -> Forwarder -> simulcast selector, gating all
three behavior changes: the HIGH default on bind, the forwarder's
initial-acquisition grace, and the selector's direct-latch-onto-target.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* remove config.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Removing some logs which have not been useful in terms of insights other
than saying that there are a bunch of packets missing. Going to start
looking at gaps in terms of time if the inter-packet gap is too high.
Also, using logging these events as first 20 and then every 200.
* Do not restart receiver on codec change mid-session.
This is not supported and was an erroneous change during the
receiver_base/buffer_base + RTP stream restart consolidation. Also make
the codec munger creation more resilient.
* fix test
* Some logging changes.
Trying to chase a case of large sequence number gap on subscriber side
where packets are sent after a long time.
* return values instead of logging
Packets not being forwarded were getting included in forwarding stats
calculation and skewing the measurement towards a smaller number.
The latency measurement does not include the batch IO of packets on
send. With a 2ms batching, that will add an average latency of 1ms.
* Prevent leakage of previous codec after codec regression.
In the window between forwarder restart and determining codec, the old
codec packet could leak through. Prevent tha by doing the restart and
codec determination atomically on a codec regression.
* tidy
* use locked function
* Use the optimal allocation function for opportunistic allocation.
Allocation functions set the `lastAllocation` state also.
This might have been causing an e2e failure with v1 client on migration.
* annotate args
When SetMaxSpatialLayer() is called with target/current layers in
InvalidLayerSpatial state, opportunistically initialize the target
layer to avoid dropped packets during async stream allocator
initialization.
Guards:
- Only sets target if not congestion-throttled (isDeficientLocked)
- Does not set current layer (deferred to keyframe-based forwarder start)
- Logs at Debug level to avoid log noise
This prevents undefined layer state during manual subscription
with immediate quality upgrades (WithAutoSubscribe(false) +
SetVideoQuality(HIGH)).
It is posible that a subscriber joins when a publisher has reconnected
and has received a flood of retransmitted packets due to NACKing the
gap caused by the publisher reconnecting. Starting on that spurt means
the subscriber gets a burst of unpaced packets that could lead to issues
with calculating render time (especially obvious in cases like egress).
The browser could send rtp packets of svc encoding without
DD extension while the sdp negotiates it, sfu detects extension
in rtp packet for this case.
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"
This reverts commit cdfbb106d1.
* Revert "Keep simulcast information tied to receiver. (#3563)"
This reverts commit ed5e2f16b2.
* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"
This reverts commit ad010cfc43.
* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"
This reverts commit 15f565510c.
* chore(deps): update protocol
* Keep simulcast information tied to receiver.
`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.
Keep track of simulcast flag in receiver.
Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track
* clean up
* clean up
* clean up
* clean up
* test
* Store SdpCid and IsSimulcast in Trackinfo
* clean up
* mock
* fix(video): determine svc/simulcast from SDP for advanced codecs
* fix(explicit-svc): cleanup
* fix(explicit-svc): remove from list on close/remove
* fix(explicit-svc): reorder VLS selection, cleanup
* fix(explicit-svc): todo comments for temporal layer selector
* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
* Normalize mime type and add utilities.
An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.
Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.
* more locations
* test
* Paul feedback
* MimeType type
* more consolidation
* Remove unused
* test
* test
* mime type as int
* use string method
* Pass error details and timeouts. (#3402)
* go mod tidy (#3408)
* Rename CHANGELOG to CHANGELOG.md (#3391)
Enables markdown features in this otherwise already markdown'ish formatted document
* Update config.go to properly process bool env vars (#3382)
Fixes issue https://github.com/livekit/livekit/issues/3381
* fix(deps): update go deps (#3341)
Generated by renovateBot
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Use a Twirp server hook to send API call details to telemetry. (#3401)
* Use a Twirp server hook to send API call details to telemetry.
* mage generate and clean up
* Add project_id
* deps
* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
participant_id, track_id as appropriate
- Store status as int
* deps
* Update pkg/sfu/mime/mimetype.go
* Fix prefer codec test
* handle down track mime changes
---------
Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
There are cases where the probe result has enough headroom to up
allocate all deficient tracks. Mainly happens after a loss scenario
where the estimate is actually still high.
After boosting once, there was a check for the track to hit the desired
layer before boosting again. But, that is not really necessary. Can
boost target and forwarder should resolve to the latest target.
Removing that check in the forwarder.
Also, adding a gratuitous boost check in stream allocator periodic ping
when deficient to look for opportunities to boost.
* file output
* wake under lock
* keep track of RTX bytes separately
* packet group
* Packet group of 50ms
* Minor refactoring
* rate calculator
* send bit rate
* WIP
* comment
* reduce packet infos size
* extended twcc seq num
* fix packet info
* WIP
* queuing delay
* refactor
* config
* callbacks
* fixes
* clean up
* remove debug file, fix rate calculation
* fmt
* fix probes
* format
* notes
* check loss
* tweak detection settings
* 24-bit wrap
* clean up a bit
* limit symbol list to number of packets
* fmt
* clean up
* lost
* fixes
* fmt
* rename
* fixes
* fmt
* use min/max
* hold on early warning of congestion
* make note about need for all optimal allocation on hold release
* estimate trend in congested state
* tweaks
* quantized
* fmt
* TrendDetector generics
* CTR trend
* tweaks
* config
* config
* comments
* clean up
* consistent naming
* pariticpant level setting
* log usage mode
* feedback
* Make lite version of RTPStatsReceiver called RTPStatsReceiverLite.
Refactor around that.
Will probably make some more flavors to have lighter versions still.
* update deps
* use MarshalLogArray
* use util
* Splitting out rtp stats stuff into its own package.
Going to be making some lighter versions of these.
Will be cleaner to have all of these grouped together.
So, as a first step, just making a package for it.
* tests
With protobuf marshalling/unmarshalling, the default struct was getting
used and nil checks for non existence were failing. That means the right
layers were not reported active on migration.
Also, restore the wrapped loggers as some fields needs some conversion
before logging.
Also, log when RTCP sender report for reference layer is received.
Cleaning up a bunch of wrapped logger calls as we already have
logger.Proto which I forgot about while doing those wrappers.
To increase visibility of ICE reconnect, logging reconnected at Infow
level. Otherwise, it is hard to see if an ICE restart finished
successfully.
Also, cleaning up ICEConnectionDetails a bit. Just separate out
read-only fields into its own struct and use it for read-only export.
* Cache RTCP sender report in forwarder state.
To be used in migration.
TODO: need to check more places to operate pure in unix nano rather than
converting.
* match name
CloudWebRTC