* dynacast: re-send subscribed qualities after every publisher answer
The SFU answers publisher offers without the simulcast pause markers
(SetIgnoreRidPauseForRecv), so applying the answer re-enables in the
browser the layers that dynacast had paused. Dynacast only notifies
changes, so those layers stayed on until the subscribed quality of the
track changed again.
Send the publisher its committed subscribed qualities again 1 s after
each answer, once the client has applied it: livekit-client handles the
next signal message without waiting for the answer to be applied.
Fixes https://github.com/livekit/livekit/issues/4906
* dynacast: resend once per burst of publisher answers, only for paused layers
In SPC, every subscription change of a participant goes through a new publisher offer and answer, so answers come in bursts when a room is busy. Each answer scheduled its own resend of the subscribed qualities, sending the same update several times in a row.
Thist change keeps one pending resend per participant instead. It fires 2 s after the last answer, and never later than 8 s after the first answer of the burst, so a steady stream of answers cannot postpone it forever. It also skips the resend for tracks whose committed qualities are all HIGH: with every layer enabled, an answer has nothing paused to re-enable. We can save those messages.
* Increase receiver loadbalance threshold when batch io enabled
With batch IO the send syscall moves to the BatchConn flush goroutine, so WriteRTP only
translates the packet and enqueues it. Increash the threshold to reduce
* lint
* Reduce locking in media track + telemetry listener on move participant.
Telemetry listener was not getting it from new room on room move.
* update comment
* fix signal bytes reporting
* room aware telemetry listener in media track
* data track telemetry listener
* Fix publish track count on migration in.
https://github.com/livekit/livekit/pull/4707 addressed the case of
publish tracks overcounting due to synthesised track publish on migrate
in. But, it introduced an issue where published tracks count could go
negative because unpublish subtracted the counter irrespective of the
track actually migrated in or not.
Fix it by keeping track of local publish.
Also, the older code was skipping publisher track count increase if the
synthesised publish was handled first. Address it by checking if the
track is actually new (i. e. fresh local publish) when the track was
already created in the migrate in path.
* fix pub time for tracks published after migration
* test
* prevent multiple track egresses
Reverting
- https://github.com/livekit/livekit/pull/4521
- https://github.com/livekit/livekit/pull/4525
There are TWCC feedback packets that are larger than MTU. Seems to
happen under a couple of conditions
1. Bad client data, i. e. severely out-of-order packets, bad sequence
numbers, etc.
2. On an ICE restart - this is rare, but it seemed to be flaky network
with some packets arriving and some not and causing a lot of gaps.
Either case, not much to do. If fargmentation/re-assembly back to
publisher works, the feedback will make it through. If not, feedbacks
will be missed and clients have to work with some missing data which is
not unexpected and the protocol is designed to handle.
However, filed pion/interceptor issue just in case - https://github.com/pion/interceptor/issues/416
* rtc: report participant kind code and details
Plumb ParticipantKind and KindDetails through MediaTrack and
BytesTrackStats so track-level reporting can record the numeric kind
code plus details codes on every participant_session aggregation,
alongside the existing Kind string. Also picks up the new kind fields
on resolved BytesSignalStats participants.
Adds deployment/agentID/version to the agent worker logger.
* Log large packets receive/send.
Seeing cases of servers reporting need for segmentation/re-assembly of
packets. So, logging packet receive/send for RTP/RTCP to check if
anything is seeing more than 1400 byte packets.
* log downtrack RTCP too
* Legacy TrackInfo.Simulcast flag.
When AddTrack did not send SimulcastCodecs, the legacy `Simulcast` flag
was not set. Fix it by setting the flag when a second layer is
published.
* staticcheck
* use the existing PrimaryReceiver function
* Use Muted in TrackInfo to propagated published track muted.
When the track is muted as a receiver is created, the receiver
potentially was not getting the muted property. That would result in
quality scorer expecting packets.
Use TrackInfo consistently for mute and apply the mute on start up of a
receiver.
* update mute of subscriptions
Release to Docker / docker (push) Failing after 3m42s
* Key telemetry stats work using combination of roomID, participantID
With forwarded participant, the same participantID can existing in two
rooms.
NOTE: This does not yet allow a participant session to report its
events/track stats into multiple rooms. That would require regitering
multiple listeners (from rooms a participant is forwarded to).
* missed file
* data channel stats
* PR comments + pass in room name so that telemetry events have proper room name also
In some paths, it is better to preserve pre-recorded time. So, make the
base implementations preserve the RTCP Sender Report receive time.
Also, add a method to enable forwarding packet arrival time. Could be
used across relay.
* Refactor receiver and buffer into Base and higher layer.
To be able to share code/functionality with relay.
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* clean up
* deps
* fix test
* fix test
* Add support for RTP stream restart.
When an unhandled packet is encountered, try a restart sequence.
Restart happens when 5 packets with contiguous sequence numbers and same
or increasing time stamps are received. Note that this does not work for
B-frame type of scenarios, but that is true for receive path handling
even before this. As WebRTC does not use B-frames, it is fine. But,
needs to be looked at again if B-frames are necessary.
It is controlled by a config that is disabled by default.
* clean up
* debug log
Due to SDP ordering, Pion did not provide track ID on a receiver.
Pion needs a=msid line to be before a=ssrc line -> need to check if this
is a spec requirement
Because of the above, it had empty id for the receiver in onTrack.
That matched a published track because we do not duplicate SdpCid in
TrackInfo if the SDP cid matches the signal cid. But, the search checks
both and it matched on empty SDP cid.
Do not accept empty ids in searches to prevent this from happening.
- Move downTrack instantiation to SubscribedTrack as it should own that
DownTrack. Still more to do here as `DownTrack` is fetched from
`SubscribedTrack` in a few places and used. Would like to avoid that,
but doing this initially.
- Use an interface from sfu.Downtrack and replace a bunch of callbacks.
SubscribedTrack is the implementation for DownTrackListener.
* Populate SDP cid in track info when available.
- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
id for AddTrack and actual SDP offer. So, have to look up using both
on server side. To make it easier, store both (only if different) in
TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
around doing simulcast codec for audio so that something like PSTN can
use G.711 without any transcoding. So, just keep it consistent between
audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
pending track or an already published track if the new offer is for a
back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
times.
- Clean up MediaTrack interface a bit and remove unneeded methods.
* WIP
* WIP
* deps
* stream allocator mime aware
* clean up
* populate SDP cid before munging
* interface methods
The browser could send rtp packets of svc encoding without
DD extension while the sdp negotiates it, sfu detects extension
in rtp packet for this case.
* Add simulcast support for WHIP.
- General change to have rids be anything.
- One issue is rid ordering not matching quality ordering, will need
some dynamic layer quality determination for that.
* clean up
* deps
* test
* Make sure moving out track has been unsubscribed
Remove start time checking in subscription manager
as We always use new track ID for republished track at #3020
so there is no race condition now.
Also RemoveSubscriber for moving out tracks for safety,
the subscription manager will handle the removed event but
RemoveSubscriber again will not be bad.
* Clear subscriber node max quality for moving out tracks
* Add Moving participant to another room
it is implemented in cloud only since the destination
room can exist in different node with the source room
* Update pkg/service/errors.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename
* test panic
* fake LocalParticipantHelper
* revert delete line
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"
This reverts commit cdfbb106d1.
* Revert "Keep simulcast information tied to receiver. (#3563)"
This reverts commit ed5e2f16b2.
* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"
This reverts commit ad010cfc43.
* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"
This reverts commit 15f565510c.
* chore(deps): update protocol
This is a bit annoying and painful. Audio does not have the information
in the track info (because it does not use Codecs, it should probably at
some point to make it consistent).
* Keep simulcast information tied to receiver.
`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.
Keep track of simulcast flag in receiver.
Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track
* clean up
* clean up
* clean up
* clean up
* test
* Store SdpCid and IsSimulcast in Trackinfo
* clean up
* mock
* fix(video): determine svc/simulcast from SDP for advanced codecs
* fix(explicit-svc): cleanup
* fix(explicit-svc): remove from list on close/remove
* fix(explicit-svc): reorder VLS selection, cleanup
* fix(explicit-svc): todo comments for temporal layer selector
* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
David Chen