Reverting
- https://github.com/livekit/livekit/pull/4521
- https://github.com/livekit/livekit/pull/4525
There are TWCC feedback packets that are larger than MTU. Seems to
happen under a couple of conditions
1. Bad client data, i. e. severely out-of-order packets, bad sequence
numbers, etc.
2. On an ICE restart - this is rare, but it seemed to be flaky network
with some packets arriving and some not and causing a lot of gaps.
Either case, not much to do. If fargmentation/re-assembly back to
publisher works, the feedback will make it through. If not, feedbacks
will be missed and clients have to work with some missing data which is
not unexpected and the protocol is designed to handle.
However, filed pion/interceptor issue just in case - https://github.com/pion/interceptor/issues/416
* rtc: report participant kind code and details
Plumb ParticipantKind and KindDetails through MediaTrack and
BytesTrackStats so track-level reporting can record the numeric kind
code plus details codes on every participant_session aggregation,
alongside the existing Kind string. Also picks up the new kind fields
on resolved BytesSignalStats participants.
Adds deployment/agentID/version to the agent worker logger.
* Log large packets receive/send.
Seeing cases of servers reporting need for segmentation/re-assembly of
packets. So, logging packet receive/send for RTP/RTCP to check if
anything is seeing more than 1400 byte packets.
* log downtrack RTCP too
* Legacy TrackInfo.Simulcast flag.
When AddTrack did not send SimulcastCodecs, the legacy `Simulcast` flag
was not set. Fix it by setting the flag when a second layer is
published.
* staticcheck
* use the existing PrimaryReceiver function
* Use Muted in TrackInfo to propagated published track muted.
When the track is muted as a receiver is created, the receiver
potentially was not getting the muted property. That would result in
quality scorer expecting packets.
Use TrackInfo consistently for mute and apply the mute on start up of a
receiver.
* update mute of subscriptions
Release to Docker / docker (push) Failing after 3m42s
* Key telemetry stats work using combination of roomID, participantID
With forwarded participant, the same participantID can existing in two
rooms.
NOTE: This does not yet allow a participant session to report its
events/track stats into multiple rooms. That would require regitering
multiple listeners (from rooms a participant is forwarded to).
* missed file
* data channel stats
* PR comments + pass in room name so that telemetry events have proper room name also
In some paths, it is better to preserve pre-recorded time. So, make the
base implementations preserve the RTCP Sender Report receive time.
Also, add a method to enable forwarding packet arrival time. Could be
used across relay.
* Refactor receiver and buffer into Base and higher layer.
To be able to share code/functionality with relay.
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* clean up
* deps
* fix test
* fix test
* Add support for RTP stream restart.
When an unhandled packet is encountered, try a restart sequence.
Restart happens when 5 packets with contiguous sequence numbers and same
or increasing time stamps are received. Note that this does not work for
B-frame type of scenarios, but that is true for receive path handling
even before this. As WebRTC does not use B-frames, it is fine. But,
needs to be looked at again if B-frames are necessary.
It is controlled by a config that is disabled by default.
* clean up
* debug log
Due to SDP ordering, Pion did not provide track ID on a receiver.
Pion needs a=msid line to be before a=ssrc line -> need to check if this
is a spec requirement
Because of the above, it had empty id for the receiver in onTrack.
That matched a published track because we do not duplicate SdpCid in
TrackInfo if the SDP cid matches the signal cid. But, the search checks
both and it matched on empty SDP cid.
Do not accept empty ids in searches to prevent this from happening.
- Move downTrack instantiation to SubscribedTrack as it should own that
DownTrack. Still more to do here as `DownTrack` is fetched from
`SubscribedTrack` in a few places and used. Would like to avoid that,
but doing this initially.
- Use an interface from sfu.Downtrack and replace a bunch of callbacks.
SubscribedTrack is the implementation for DownTrackListener.
* Populate SDP cid in track info when available.
- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
id for AddTrack and actual SDP offer. So, have to look up using both
on server side. To make it easier, store both (only if different) in
TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
around doing simulcast codec for audio so that something like PSTN can
use G.711 without any transcoding. So, just keep it consistent between
audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
pending track or an already published track if the new offer is for a
back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
times.
- Clean up MediaTrack interface a bit and remove unneeded methods.
* WIP
* WIP
* deps
* stream allocator mime aware
* clean up
* populate SDP cid before munging
* interface methods
The browser could send rtp packets of svc encoding without
DD extension while the sdp negotiates it, sfu detects extension
in rtp packet for this case.
* Add simulcast support for WHIP.
- General change to have rids be anything.
- One issue is rid ordering not matching quality ordering, will need
some dynamic layer quality determination for that.
* clean up
* deps
* test
* Make sure moving out track has been unsubscribed
Remove start time checking in subscription manager
as We always use new track ID for republished track at #3020
so there is no race condition now.
Also RemoveSubscriber for moving out tracks for safety,
the subscription manager will handle the removed event but
RemoveSubscriber again will not be bad.
* Clear subscriber node max quality for moving out tracks
* Add Moving participant to another room
it is implemented in cloud only since the destination
room can exist in different node with the source room
* Update pkg/service/errors.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename
* test panic
* fake LocalParticipantHelper
* revert delete line
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"
This reverts commit cdfbb106d1.
* Revert "Keep simulcast information tied to receiver. (#3563)"
This reverts commit ed5e2f16b2.
* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"
This reverts commit ad010cfc43.
* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"
This reverts commit 15f565510c.
* chore(deps): update protocol
This is a bit annoying and painful. Audio does not have the information
in the track info (because it does not use Codecs, it should probably at
some point to make it consistent).
* Keep simulcast information tied to receiver.
`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.
Keep track of simulcast flag in receiver.
Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track
* clean up
* clean up
* clean up
* clean up
* test
* Store SdpCid and IsSimulcast in Trackinfo
* clean up
* mock
* fix(video): determine svc/simulcast from SDP for advanced codecs
* fix(explicit-svc): cleanup
* fix(explicit-svc): remove from list on close/remove
* fix(explicit-svc): reorder VLS selection, cleanup
* fix(explicit-svc): todo comments for temporal layer selector
* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
* Send regressed codec upstream stats to analytics.
There is more work to do for analytics for simulcast codec, i. e. when
both codecs are published. Shorter term, ensure that analytics are sent
for regressed codec if active.
* lock access to regressed codec received
There are two very very edge case scenarios this is triyng to address.
Scenario 1:
-----------
- both pA and pB migrating
- pA migrates first and subscribes to pB via remote track of pB
- while the above subscribe is happening, pB also migrates and
closes the remote track
- by the time the subscribe set up completes, it realises that
the remote track is not open any more and removes itself as
subscriber
- but that removal is using the wrong `isExpectedToResume` as clearing
all receivers has not run yet which is what caches the
`isExpectedToResume`.
- That meant, the down track transceiver is not cached and hence not
re-used when re-subscribing via pB's local track
- Fix it by caching the expected to resume when changing receiver state
to `closing`.
Scenario 2:
-----------
- both pA and pB migrating
- pA migrates first and subscribes to pB via remote track of pB
- while the above subscribe is happening, pB also migrates and
closes the remote track
- pB's local track is published before the remote track can be fully
closed and all the subscribers removed. That local track gets added
to track manager.
- While the remote track is cleaning, subscription manager triggers
again to for pA to subscribe to pB's track. The track manager now
resolves to the local track.
- Local track subscription progresses. As the remote track clean up is
not finished, the transceiver is not cached. So, the local track based
subscription creates a new transceiver and that ends up causing
duplicate tracks in the SDP offer.
- Fix it by creating a FIFO in track manager and only resolve using the
first one. So, in the above case, till the remote track is fully
cleaned up, the track manager will resolve to that. Yes, the
subscriptions itself will fail as the track is not in open state (i. e.
it might be in `closing` state), but that is fine as subscription
manager will eventually resolve to the local track and proper
transceiver re-use can happen.
* Normalize mime type and add utilities.
An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.
Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.
* more locations
* test
* Paul feedback
* MimeType type
* more consolidation
* Remove unused
* test
* test
* mime type as int
* use string method
* Pass error details and timeouts. (#3402)
* go mod tidy (#3408)
* Rename CHANGELOG to CHANGELOG.md (#3391)
Enables markdown features in this otherwise already markdown'ish formatted document
* Update config.go to properly process bool env vars (#3382)
Fixes issue https://github.com/livekit/livekit/issues/3381
* fix(deps): update go deps (#3341)
Generated by renovateBot
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Use a Twirp server hook to send API call details to telemetry. (#3401)
* Use a Twirp server hook to send API call details to telemetry.
* mage generate and clean up
* Add project_id
* deps
* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
participant_id, track_id as appropriate
- Store status as int
* deps
* Update pkg/sfu/mime/mimetype.go
* Fix prefer codec test
* handle down track mime changes
---------
Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
TrackSubscribed is meant to give publishers an indication when the subscriber
is ready to receive its audio. When there are hidden recorders in the room,
we do not want them to trigger this event.
* Don't wait rtp packet to fire track
Create track from sdp instead of first rtp packet,
it is consistent with the browser behavior and
will accelerate the track publication.
* fix test
Raja Subramanian