Tested with eyevinn client.
There are a few issues to figure out still
1. Simulcast - how?
2. For simulcast, how to know width, height so that adaptive stream can
work.
3. The layer added is dummy. It works, but connection quality scoring
would be incorrect (will always say excellent) without bitrate.
Will need some dynamic update of `TrackInfo` based on actual stream for
all of this to fit well into our system, but the simple video support
works for now.
Was setting the state to ACTIVE prematurely to enable the subscription
inter-lock in one shot signalling mode. But, that is incorrectly
changing state.
Use a callback to indicate subscriber ready and let the participant
ACTIVE happen when the connection actually establishes.
Had made the change to align `participant active` to after the ICE
connection is done and that log could list all candidates.
But, with one shot signalling, the state change has to be early to wait
on (auto) subscriptions of track of other participant. So, state has to
be changed early.
* Add support for WHIP ICE Trickle/Restart.
Tested a bit using the WHIP client at https://github.com/Eyevinn/whip,
but needs a lot more testing. ICERestart is not tested yet.
* comment
* clean up
* Prevent migration race.
Comments in code. Briefly, due to race, the remote participant/track
could be closed early leading to missing subscription post-migration.
* fixes
* Unlabeled (pass through) data channels.
Support data channels than can pass through raw data without any LK
protocol marshaling/unmarshaling.
* statischeck
* test
* error -> warn
* reset data message callback
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"
This reverts commit cdfbb106d1.
* Revert "Keep simulcast information tied to receiver. (#3563)"
This reverts commit ed5e2f16b2.
* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"
This reverts commit ad010cfc43.
* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"
This reverts commit 15f565510c.
* chore(deps): update protocol
This is a bit annoying and painful. Audio does not have the information
in the track info (because it does not use Codecs, it should probably at
some point to make it consistent).
* Keep simulcast information tied to receiver.
`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.
Keep track of simulcast flag in receiver.
Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track
* clean up
* clean up
* clean up
* clean up
* test
* Store SdpCid and IsSimulcast in Trackinfo
* clean up
* mock
* Rework node stats a bit.
Related protocol PR - https://github.com/livekit/protocol/pull/1023
- Make a config for node stats measurements. Wanted to put the config in
`routing` package, but a circular dependency forced me to put in
config.go
- Make rate calculations explicit, i. e. requested via config.
Previously, it had some odd checks to decide when to calculate rate
and it would have been calculating over different windows.
- Report signal/data channel bytes every 5 seconds to stats collection
module. Previously, it was doing it every 30 seconds and that meant
some windows could have had a large spike
NOTE: Still need to think about this for load calculations as a large
number of participants leaving could flush in a small window and that
could report a large spike in bytes/packets. Maybe need to ignore
signal bytes for load calculation?
* deps
* use default node stats config if given config is nil
* split out node stats into a struct for re-use
* update config
* fix(video): determine svc/simulcast from SDP for advanced codecs
* fix(explicit-svc): cleanup
* fix(explicit-svc): remove from list on close/remove
* fix(explicit-svc): reorder VLS selection, cleanup
* fix(explicit-svc): todo comments for temporal layer selector
* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
Seeing a lot of queuing delay based back offs. Trying a couple of things
1. Accept a bit more queuing.
2. An option to try a different pacer. Would like to try with pass
through. That will produce some out-of-order packets. Remains to be
seen if it will have a negative impact.
* Send regressed codec upstream stats to analytics.
There is more work to do for analytics for simulcast codec, i. e. when
both codecs are published. Shorter term, ensure that analytics are sent
for regressed codec if active.
* lock access to regressed codec received
This happens due to improper track type.
One possible option is to check the mime type of the track and calculate
the size, but there are other places in the code which work off provided
track type. So, just doing a defensive fix. Have to review code for all
uses of type and how it affects things if client provides incorrect
type.
Clean up the tracks in the synchronous path and remove track from track
manager. This is not strictly required in a single node case. But,
multi-node needs this. So, doing this here for consistency.
There are two very very edge case scenarios this is triyng to address.
Scenario 1:
-----------
- both pA and pB migrating
- pA migrates first and subscribes to pB via remote track of pB
- while the above subscribe is happening, pB also migrates and
closes the remote track
- by the time the subscribe set up completes, it realises that
the remote track is not open any more and removes itself as
subscriber
- but that removal is using the wrong `isExpectedToResume` as clearing
all receivers has not run yet which is what caches the
`isExpectedToResume`.
- That meant, the down track transceiver is not cached and hence not
re-used when re-subscribing via pB's local track
- Fix it by caching the expected to resume when changing receiver state
to `closing`.
Scenario 2:
-----------
- both pA and pB migrating
- pA migrates first and subscribes to pB via remote track of pB
- while the above subscribe is happening, pB also migrates and
closes the remote track
- pB's local track is published before the remote track can be fully
closed and all the subscribers removed. That local track gets added
to track manager.
- While the remote track is cleaning, subscription manager triggers
again to for pA to subscribe to pB's track. The track manager now
resolves to the local track.
- Local track subscription progresses. As the remote track clean up is
not finished, the transceiver is not cached. So, the local track based
subscription creates a new transceiver and that ends up causing
duplicate tracks in the SDP offer.
- Fix it by creating a FIFO in track manager and only resolve using the
first one. So, in the above case, till the remote track is fully
cleaned up, the track manager will resolve to that. Yes, the
subscriptions itself will fail as the track is not in open state (i. e.
it might be in `closing` state), but that is fine as subscription
manager will eventually resolve to the local track and proper
transceiver re-use can happen.
* Use atomic to store codec.
It can change on up stream codec change, but not seeing any racy
behaviour with atomic access.
Reverting the previous change to mute with this change.
* no mime arg
Need to re-visit the bind lock scope and maybe make the codec/mime
atomic and access them without bind lock. But, doing a whack-a-mole a
bit first to move things forward. Will look at making them atomics.
* Dependent participants should not trigger count towards FirstJoinedAt
According to the API, empty timeout should be honored as long as no
independent participant joins the room. If we counted Agents and Egress
as part of FirstJoinedAt, it would have the side effect of using
departureTimeout instead of emptyTimeout for idle calculations.
* use Room logger
