A few more candidates to think about demoting
- Publisher mute changes
- Forwarder -> layer lock/upgrade/downgrade/overshoot adjusting
- StreamAllocator
UpdateSubscription had a shortcoming where when it couldn't find the
participant, it ignored the request.
This PR further removes the reliance of current publisher state from
subscribers.
- SubscribeToTrack only takes in a trackID
- Introduced RoomTrackManager to maintain all published tracks to a room
- Added TrackUnpublished event to clearly indicate when a track has been removed
- SubscribeRequested event no longer include information about the publisher
Added a new manager to handle all subscription needs. Implemented using reconciler pattern. The goals are:
improve subscription resilience by separating desired state and current state
reduce complexity of synchronous processing
better detect failures with the ability to trigger full reconnect
* Fix handling of non-monotonic timestamps
Timed version is inspired by Hybrid Clock. We used to have a mixed behavior
by using time.Time:
* during local comparisons, it does increment monotonically
* when deserializing remote timestamps, we lose that attribute
So it's possible for two requests to be sent in the same microsecond, and
for the latter one to be dropped.
To fix that behavior, I'm switching it to keeping timestamps to consolidate
that behavior, and accepting multiple updates in the same ms by incrementing ticks.
Also using @paulwe's idea of a version generator.
There were some failures with missing media. The only thing I could
see between working and non-working case is when media forwarding
starts. So, delay media forwarding till peer connection is connected.
Also, add a subscribe op only if a subscribe/unsubscribe queuing is
successful. There was a recent change to not queue a subscribe when
the participant is closed/disconnected. This got the subscribe op
counter out of whack.
Also not holding lock while resolving by sid. Not pretty.
But, we should deprecate SID based permissions.
Also, wire changes, possibly due to redis cluster PR?
With migration in, once the local track is published, the
remote track should be closed. Add a flag to `RemovePublishedTrack`
to control the close behaviour. Invoke `Close` if specified.
Without, the remote track is not closed if it is waiting to resolve,
i. e. not yet attached. That remote track is left hanging.
* Prevent track subscriptions/adding receivers after close
With subscribe/unsubscribe queuing, a subscribe may be
attempted after a call to `RemoveAllSubscribers`.
So, renaming `RemoveAllSubscribers` to `InitiateClose`
and maintaining state that track is in the process of closing.
* Mime specific remove
* Remove unused error
* do not add receiver when closing
* Promoting a few logs to Info
Also, adding a couple of more info logs which I will remove later
after some debugging.
* mime type
* Protect pause/max layer
* notify even if not bound
* WIP commit
* Refactor media loss proxy
* Use DynacastQuality and MediaLossProxy from MediaTrack
* fix test
* Remove unused param
* Remove unused interfaces
* Move interface methods to local
* Split out DynacastManager
* have to add codec to dynacast manager
* RUnlock
* fix restart
* Adding API to force quality and also maintain closed state
* Address PR comments
* Do not hold lock while invoking resolver.
Resolver is in room and it will grab its lock.
It is called from partcipant when checking permissions.
Permissions processing uses the participant lock.
So, not a good idea to call a room function with
participant lock held. Avoid that.
Also, use a full lock in the add/remove subscription
path. This is to ensure that permissions path and
subscription path (like subscribing to new participant's
tracks) do not race. As subscriptions are queued now
on the subscriber side, this should be fine.
* Revoke path resolution in ops queue goroutine
* fix test
* Keep track of pending subscriber operations.
This is required to determine if a receiver does not have
any subscription.
* correct spelling of queuing
* lock around hasPermission
* Send permissions update on subscribe.
The permission `allowed` update was happening only when
processing pending subscriptions (which happens only on
subscription permissions update).
It is possible that subscription happens through other
paths (like subscribing new participant to tracks).
In that path, we were checking if the track has permissions
and adding to pending. But, we were not checking if
the track is in pending and if it is in there, removing
in on successful subscription and sending an update.
Fix that.
* log more fields in error
* Use a queue for add/remove subscribe operations.
If subscribe/unsubscribe happens very quickly, the subscription
state gets mixed up as things are keyed off of subscriberID.
Use a queue of subscribe operations and process it serially.
* set up callback for down track added
* move the queue on unexpected type
* move the queue if removeSubscirber does not have a subscribed track
* Fixed unclean DownTrack close when removed before bound.
When a DownTrack is closed before it had a chance to be bound to a
transceiver, we'd skip close and leave it hanging. This is unlikely in
normal operations. However, it can be seen with permissions and
subscription APIs.
* remove remaining peerID references
* Support participant identity in permissions
It is harder for clients to update permissions by SID as remote
reconnecting means a new SID for that participant. Using participant
identity is a better option.
For now, participant SID is also supported. Internally, it will
get mapped to identity. Server code uses identity throughout after
doing any necessary conversion from SID -> Identity.
* Address comments
* Prevent stats update if the deltas are empty
* increase force interval
* static check
* Change max delay to 30 seconds
* Restart the max subscribed quality timer on ICE restart.
Force an update after a restart to let clients apply dynacast settings.
Also, set the max expected layer to HIGH on restart so that stream
tracker starts up fast on a restart.
* fix test
* feat: unpublish tracks after publish permissions are revoked.
Uses protocol 7 to indicate client support, otherwise it attempts to
mute the tracks.
Also sends back permissions objects of all participants, and cleaned up
our handling of various permissions attributes.
* fix static check
