* Add ability to roll back video layer selection.
Not currently useful, but it is possible to do things like not
applying a layer switch if the switch point time stamp is too far back.
Add ability to roll back a layer switch and invoke rollback if
a packet was selected for forwarding, but a subsequent error or decision
to drop the packet can rollback layer switch if that was the switching
packet.
In current code, the paths where a packet can be dropped after selection
does not happen at switch points. So, it was okay to apply the selection
unconditionally. But, adding the call to rollback in the current code
also in all paths where packet is dropped after selection for consistent
code flow.
* separate switch for temporal layer
* Push track quality to poor on a bandwidth constrained pause.
* add tests
* scale distance by divisor
* fix test distance to desired
* wait longer for subscription manager to reconcile
* Prevenet anachronous sample reading.
Not so pretty way of solving this. Please let me know if you have
thoughts.
Passing in time allows testing easier. But, that also leads to
time reversal problems. Example scenario
1. Connection stats worker gets a time and initiates quality
calculation.
2. A layer transition is recorded after that.
3. By the time, scorer is called to calculate score with time from Step
1, there is time reversal and results in anachronous sample.
One option is to use a scorer lock in connection stats module and wrap
all calls to scorer in that lock, but that does not prevent the passed
in time stamps themselves getting out of order. Also, stand alond use
of scorer in some other context will be problematic.
Doing the hybrid thing of taking current time in scorer if passed in
time is zero so that scorer lock domain controls it.
* use zero time everywhere in normal flow
* make APIs with and without time passed in as Paul suggested
* Close subscriptions promptly
Two things:
-----------
1. Because the desired is not changed, the notifiers are not notified
that the subscription is not observing any more. So, that holds
a refernce to the subscription manager.
Address the above by setting `setDesired` to false on all subscriptions
when subscription manager closes. That will remove observer from the
notifiers.
2. When subscription manager is closed, the down track close
is invoked which flows back (with onClose callback of downtrack) to
subscription manager "handleSubscribedTrackClose". That callback
handler sets the subscribed track to nil for that subscription.
A couple of scenarios here
a. Without the above change, desired could have been true and it would
have looked that the track needs to try subscription again because
`needsSubscribe == true` (desired == true && subscribedTrack == nil)
b. Even with the change above, there is a new condition of
`desired == false && subscribedTrack == nil` and there was no handler
for that condition in the reconciler.
Address this by adding a `needsCleanup` function and delete subscription
from the map. Note that the reconciler may not be running to execute
this action as subscription manager would have closed the `closeCh`, but
doing the code in the interest of proper clean up.
* clean up
* Delete down track from receiver in close always.
I think with the parallel close in goroutines, it so happens that
peer connection can get closed first and unbind the track.
The delete down track and RTCP reader close was inside if `bound` block.
So, they were not running leaving a dangling down track in the receiver.
* fix tests
* fix test
* Make congestion controller probe config
* Wait for enough estimate samples
* fixes
* format
* limit number of times a packet is ACKed
* ramp up probe duration
* go format
* correct comment
* restore default
* add float64 type to generated CLI
* Use net.JoinHostPort to build "host:port" strings for `net.Listen`
net.JoinHostPort provides a unified way of building strings of the form
"Host:Port", abstracting the particular syntax requirements of some
methods in the `net` package (namely, that IPv4 addresses can be given
as-is to `net.Listen`, but IPv6 addresses must be given enclosed in
square brackets).
This change makes sense because an address such as `[::1]` is *not* a
valid IPv6 address; the square brackets are just a detail particular to
the Go `net` library. As such, this syntax shouldn't be exposed to the
user, and configuration should just accept valid IPv6 addresses and
convert them as needed for usage within the code.
* Use '--bind' CLI flag to also filter RTC bind address
The local address passed to a command such as
livekit-server --dev --bind 127.0.0.1
was being used as binding address for the TCP WebSocket port, but was
being ignored for RTC connections.
With `--dev`, the conf.RTC.UDPPort config is set to 7882, which enables
"UDP muxing" mechanism. Without interface or address filtering, Pion
would try to bind to port 7882 on *all* interfaces.
This was failing on a system with IPv6 enabled, when trying to bind to
an IPv6 address of the `docker0` interface. It seems to make sense that
the user-passed bind addresses are also honored for the RTC port
bindings.
* Pacer interface to send packets
* notify outside lock
* use select
* use pass through pacer
* add error to OnSent
* Remove log which could get noisy
* Starting TWCC work (#1727)
* add packet time
* WIP commit
* WIP commit
* WIP commit
* minor comments
* Some measurements (#1736)
* WIP commit
* some notes
* WIP commit
* variable name change and do not post to closed channel
* unlock
* clean up
* comment
* Hooking up some more bits for TWCC (#1752)
* wake under lock
* Pacer in down stream path.
Splitting out only the pacer from a feature branch to
introduce the concept of pacer.
Currently, there should be no difference in functionality
as a pass through pacer is used.
Another implementation exists which is just put it in a queue and send
it from one goroutine.
A potential implementation to try would be data paced by bandwidth
estimate. That could include priority queues and such.
But, the main goal here is to introduce notion of pacer in the down
stream path and prepare for more congestion control possibilities down
the line.
* Don't need peak detector
* remove throttling of write IO errors
- Increase max interval between probes to 2 minutes.
- Use a minimum probe rate of 200 kbps. This is to ensure that
the probe rate is decent and can produce a stronger signal.
* Don't update dependency info if unordered packet received
* Trace all active svc chains for downtrack
* Try to keep lower decode target decodable
* remove comments
* Test case
* clean code
* solve comments
When migrating muted track, need to set potential codecs.
For audio, there may not be `simulcast_codecs` in `AddTrack`.
Hence when migrating a muted track, the potential codecs are not set.
That results in no receivers in relay up track (because all this
could happen before the audio track is unmuted).
So, look at MimeType in TrackInfo (this will be set in OnTrack) and
use that as potential codec.
* Do not process events after participant close.
Avoid processing transport events after participant/transport close.
It causes error logs which are not really errors, but distracting noise.
* correct comment
* Full reconnect on publication mismatch on resume.
It is possible that publications mismatch on resume. An example sequence
- Client sends `AddTrack` for `trackA`
- Server never receives it due to signalling connection breakage.
- Client could do a resume (reconnect=1) noticing signalling connection
breakage.
- Client's view thinks that `trackA` is known to server, but server does
not know about it.
- A subsequence offer containing `trackA` triggers `trackInfo not
available before track publish` and the track does not get published.
Detect the case of missing track and issue a full reconnect.
* UpdateSubscriptions from sync state a la cloud
* add missing shouldReconnect
* Close participant on full reconnect.
A full reconnect == irrecoverable error. Participant cannot continue.
So, close the participant when issuing a full reconnect.
That should prevent subscription manager reconcile till the participant
is finally closed down when participant is stale.
* format
A subscription in subscription manager could live till the source
track goes away even though the participant with that subscription
is long gone due to closure on source track removal. Handle it by using
trackID to look up on source track removal.
Also, logging SDPs when a negotiation failure happens to check
if there are any mismatches.
* Simplify sliding window collapse.
Keep the same value collapsing simple.
Add it to sliding window as long as same value is received for longer
than collapse threshold.
But, add a prune with three conditions to process the siliding window
to ensure only valid samples are kept.
* flip the order of validity window and same value pruning
* increase collapse threshold to 0.5 seconds during non-probe
1. Probe end time needs to include the probe cluster running time also.
2. Apply collapse window only within the sliding window. This is to
prevent cases of some old data declaring congestion. For example,
an estimate could have fallen 15 seconds ago and there might have
been a bunch of estimates at that fallen value. And the whole
sliding window could have that value at some point. But, a further
drop may trigger congestion detection. But, that might be acting too
fast, i. e. on one instance of value fall. Change it so that we
detect if there is a fall within the sliding window and apply
collapse based on that.
Till now only video was using simulated publish when migrating on mute.
But, with `pauseUpstream() + replaceTrack(null)`, it is possible that
client does not send any data when muted.
I do not think there is a problem to do this (even when cleint is
actually using mute which sends silence frames).
boks1971