* Notes on wht to do
- Should targetLayers be altered while doing opportunistic locking
- Should targetLayers be altered in any other path than stream allocator path?
- Lock to layer as long as it is <= opportunistic layer
- When not congested, opportunistic can be highest
- When congested, opportunistic could be nil or lowest if paused is not allowed
- When muting, can we hold on to current layers (or keep it as previous) and
restore on unmute.
- Store current/target in forwarder state and restore on seeding
- Watch for looking for targetLayers, etc. when looking to insert padding
packets. There may be an assumption about restarting on key frame and hence
okay to insert padding when target layers are invalid. This may not be true
any more when doing opportunistic forwarding.
- Can we distinguish between publisher mute or dynacast (i. e. publisher side
stopping) vs subscriber mute and do something useful? Publisher side mute
could mean continuity in sequence numbers on a restart (might be able to
catch it with opportunistic forwarding). But, there is the challenge of
unmute from publisher via signalling channel vs media. If media is arriving,
should subscribers do opportunistic forwarding before publisher mute state
update happens?
- Maybe introduce a mode where forwarding continues to a frame end (of course
with a time limit just in case the end of frame packet is lost) and then
insert silence/padding packets?
- Ensure that audio blank frame insertion does not suffer from frame boundary
issues.
* pub/sub mute separate + more notes on things to check
* WIP commit, more notes
* WIP commit
* WIP commit
* WIP commit
* WIP commit
* WIP commit
* WIP commit
* clean up
* slightly better comments
* Do not stop on unmute
* do not inject blank frames when pub muted
* do not forward on audio publisher mute
* Add Timer to detect dtls failure quickly
* Fix pc state check in timeout after ice
* More strict conditions to switch candidate type
* log for signal interuppt
* typo
* Ignore inactive media.
Seeing some errors of
"track info not published prior to track".
Happens when trying to configure publisher answer for audio
for DTX and stereo and trying to find a pending track.
There are cases where there are inactive m-lines in SDP
that is hitting this.
* fix sense
When AdaptiveStream is enabled, default the subscriber to LOW quality stream
we would want LOW instead of OFF for a couple of reasons
1. when a subscriber unsubscribes from a track, we would forget their previously defined settings
depending on client implementation, subscription on/off is kept separately from adaptive stream
So when there are no changes to desired resolution, but the user re-subscribes, we may leave stream at OFF
2. when interacting with dynacast *and* adaptive stream. If the publisher was not publishing at the
time of subscription, we might not be able to trigger adaptive stream updates on the client side
(since there aren't any video frames coming through). this will leave the stream "stuck" on off, without
a trigger to re-enable it
Added a new manager to handle all subscription needs. Implemented using reconciler pattern. The goals are:
improve subscription resilience by separating desired state and current state
reduce complexity of synchronous processing
better detect failures with the ability to trigger full reconnect
* Use local time base for NTP in RTCP Sender Report for downtracks.
More details in comments in code.
* Remove debug
* RTCPSenderReportInfo -> RTCPSenderReportDataExt
* Get rid of sender report data pointer checks
* add prometheus stats for rtt/jitter/packet loss
* add track source to metrics
* better packet loss bins
* add track type to metrics
* remove source from AnalyticsStat
* regenerate telemetry service fake
* compute loss from per stream packet count
Added a flag to throttle dynacast messages to the publisher.
Added a helper method to check if a client supports setting `active`
field in RTP sender encoding. This can be used to disable dynacast based
on some other feature flag/config settings.
Related to livekit/protocol#273
This PR adds:
- ParticipantResumed - for when ICE restart or migration had occurred
- TrackPublishRequested - when we initiate a publication
- TrackSubscribeRequested - when we initiate a subscription
- TrackMuted - publisher muted track
- TrackUnmuted - publisher unmuted track
- TrackPublish/TrackSubcribe events will indicate when those actions have been successful, to differentiate.
* Fix handling of non-monotonic timestamps
Timed version is inspired by Hybrid Clock. We used to have a mixed behavior
by using time.Time:
* during local comparisons, it does increment monotonically
* when deserializing remote timestamps, we lose that attribute
So it's possible for two requests to be sent in the same microsecond, and
for the latter one to be dropped.
To fix that behavior, I'm switching it to keeping timestamps to consolidate
that behavior, and accepting multiple updates in the same ms by incrementing ticks.
Also using @paulwe's idea of a version generator.
* initial commit
* add correct label
* clean up
* more cleanup on adding stats
* cleanup
* move things to pub and sub monitors, ensure stats are correctly updated
* fix merge conflict
* Fix panic on MacOS (#1296)
* fixing last feedback
Co-authored-by: Raja Subramanian <raja.gobi@tutanota.com>
* WIP commit
* comment
* clean up
* remove unused stuff
* cleaner comment
* remove unused stuff
* remove unused stuff
* more comments
* TrackSender method to handle RTCP sender report data
* fix test
* push rtcp sender report data to down tracks
* Need payload type for codec id mapping in relay protocol
* rename variable a bit
* Do not get tripped by default values.
The following scenario declared the second message a dupe incorrectly
- UpdateSubscription{subscribe: true}: This message initialized quality
to default which is livekit.VideoQuality_LOW
- UpdateTrackSettings{quality: livekit.VideoQuality_LOW} - this one got
tripped as duplicate because the previous message initialized quality
to LOW.
Fix it by recording whether track settings have been seen.
no auto subscribe + quality setting to LOW test failed before this
change and passes with this change.
* patch all track setting fields
* Add interface and ipfilter to udpmux option
* validate external ip is accessable by client
* add context
* use external ip only for firefox
* fix mapping error
* Update pion/ice and use external ip only for firefox
* Use single external ip for NAT1To1Ips if validate failed
* update pion/ice
* Split stream tracker impl from base
* slight re-arrangement of code
* fps based stream tracker
* MinFPS config
* switch back to packet based tracker
* use video config by default to handle sources without type
GetSelectedICECandidatePair can return nil for the candidate pair if not
available even if the error is not nil. Protect against the nil
de-reference panic.
message.
There are is a sequence where a dupe could be detected due to patching
which could lead to issues.
The sequence is
- UpdataTrackSettings with some values
- UpdateSubscription with Subcribe: false - this will patch from above
track settings
- UpdateSubscription with Subscribe: true - this will continue patching
- UpdateTrackSettings with the same settings as in the first step - this
will be declared a dupe because the track is enabled and the patched
settings will declare no change in settings.
This is okay in the current code as subscription settings are cached at
participant level and applied when somebody re-subscribes. But, that
down stream processing can change any time.
So, when processing `UpdateSubscription` message, just do not patch.
If a later `UpdateTrackSettings` comes along, let it pass even if it
is not changing anything.
* Initial commit of signal deduper.
Idea is protect against signal storm from misbehaving clients.
Design:
- SignalDeduper interface with one method to handle a SignalRequest and
return if dupe or not.
- Signal specific deduper. Could have made a single de-duper which could
handle all signal message types, but making it per type so that the
code is cleaner.
- Some module (like the router) can instantiate whatever signal types
it wants to de-dupe. When a signal message is received, that module
can run the signal message through the list of de-dupers and
potentially drop the message if any of the de-dupers declare that the
message is a dupe. Making it a list makes things a little bit
inefficient, but keeps things cleaner. Hopefully, not many de-dupers
will be needed so that the inefficiency is not pronounced.
* re-arrange comments
* helper function
* add ParticipantClosed
David Zhao