Commit Graph

1555 Commits

Author SHA1 Message Date
Raja Subramanian
9702d3b541 A couple of more opportunities in stream allocator. (#1906)
1. When re-allocating for a track in DEFICIENT state, try to use
   available headroom to accommodate change before trying to steal
   bits from other tracks.
2. If the changing track gives back bits (because of muting or
   moving to a lower layer subscription), use the returned bits
   to try and boost deficient track(s).
2023-07-26 15:35:07 +05:30
Raja Subramanian
0484a68342 Plug a couple of holes in stream transitions. (#1905)
* Plug a couple of holes in stream transitions.

1. Missed negative sign meant stealing bits from other tracks was not
   working.
2. When a track change (mute, unmute, subscription change) cannot be
   allocated, explicitly pause so that stream state update happens.

Refactor stream state update a bit to make it a bit cleaner.

* correct comment
2023-07-26 13:36:58 +05:30
Raja Subramanian
5ae1387c68 Return a copy of down tracks from spreader. (#1902)
As shadow copy can change, do not return as is.
Also use the broacast function to broadcast up track changes to down
tracks.
2023-07-25 19:00:43 +05:30
Raja Subramanian
ffd6dc2210 Packet level ddebug logs. (#1900)
Only for debugging for a bit. Not for deploy.
2023-07-25 13:53:21 +05:30
Raja Subramanian
43fa6f57d1 A very simple leaky bucket pacer. (#1899) 2023-07-23 10:11:35 +05:30
Raja Subramanian
7e6aa00426 Remove unused fields left over from refactor (#1897) 2023-07-21 16:23:00 +05:30
Paul Wells
6c20c7eb15 add test for removing disconnected participants on signal close (#1896)
* add test for removing disconnected participants on signal close

* cleanup
2023-07-20 21:21:40 -07:00
Paul Wells
3980d049c9 close disconnected participants when signal channel fails (#1895)
* close disconnected participants when signal channel fails

* fix typefake

* update reason
2023-07-20 19:23:35 -07:00
Paul Wells
6ad1e1598d move signal server start to server start (#1894)
* move signal server start to server start

* fix test
2023-07-20 19:13:27 -07:00
kannonski
cf4801064d changing key file permissions control (#1893) 2023-07-19 14:23:30 -07:00
Raja Subramanian
dd995899bf Handle extreme case of sender report lagging. (#1892) 2023-07-19 12:50:03 +05:30
Raja Subramanian
cf8cf1a87f Forgot to log important bits :-( (#1891) 2023-07-19 10:22:51 +05:30
Raja Subramanian
66de9ff4a0 Add debug log for RTCP sender report. (#1890)
* Add debug log for RTCP sender report.

Temporary to collect more data. Hitting scenarios under congestion
where the sender report gets off sync. Need some data to pore through
and understand and implement changes.

* Debugw
2023-07-18 23:21:06 +05:30
Raja Subramanian
f41b93657e Log a bit more in sender report warp report. (#1888) 2023-07-18 09:14:41 +05:30
Paul Wells
9f3c975b1c leave signal context open after stream closes (#1887) 2023-07-16 20:24:11 -07:00
David Zhao
5d1d454a98 Fix missed label arg in logger (#1886) 2023-07-16 20:05:41 -07:00
Paul Wells
5535916ff2 prevent signal context from closing before room setup finishes (#1885) 2023-07-16 19:01:53 -07:00
Paul Wells
8784449fc6 manually cancel signal relay context (#1884) 2023-07-16 15:03:47 -07:00
Paul Wells
7dc60bb1bf start reading signal messages before session handler finishes (#1883)
* start reading signal messages before session handler finishes

* fix err scope
2023-07-16 13:40:53 -07:00
Raja Subramanian
11e1eb00fa Attempt to avoid out-of-order max subscribed layer notifications. (#1882)
* Check for request layer lock only in the goroutine

* check before sending PLI

* max layer notifier worker

* test cleanup

* clean up

* do notification in the callback
2023-07-16 23:28:20 +05:30
Raja Subramanian
469f1cd073 Minor changes to publisher bool. (#1880)
* Minor changes to publisher bool.

* address feedback
2023-07-15 12:43:05 +05:30
Raja Subramanian
4c02a6d717 Time stamp adjustments v2 (I think) (#1875)
* WIP commit

* WIP commit

* WIP commit

* Some clean up
- Removed a chatty debug log
- some spelling, punctuation correction in comments
- missed an `Abs` in check, add it.
2023-07-14 11:47:07 +05:30
Benjamin Pracht
68e5fa8e1c Allow listing ingress by id (#1874) 2023-07-14 09:11:55 +08:00
David Zhao
557fe7c9d3 Mark room as dirty after track published changes (#1878)
Ensure that we are recomputing NumPublished when needed
2023-07-13 16:33:04 -07:00
Raja Subramanian
e746fe14e1 Mark active when switching to parked layer. (#1873)
* Mark active when switching to parked layer.

Parked layer lock is not a switch. It is just a restart at the same
layer.

* make explicit bool for switching
2023-07-13 10:42:23 +05:30
Raja Subramanian
8dc2c005c3 Add ability to roll back video layer selection. (#1871)
* Add ability to roll back video layer selection.

Not currently useful, but it is possible to do things like not
applying a layer switch if the switch point time stamp is too far back.

Add ability to roll back a layer switch and invoke rollback if
a packet was selected for forwarding, but a subsequent error or decision
to drop the packet can rollback layer switch if that was the switching
packet.

In current code, the paths where a packet can be dropped after selection
does not happen at switch points. So, it was okay to apply the selection
unconditionally. But, adding the call to rollback in the current code
also in all paths where packet is dropped after selection for consistent
code flow.

* separate switch for temporal layer
2023-07-12 14:12:00 +05:30
Raja Subramanian
ed867fafe5 Log unexpected ICE connection states (#1870) 2023-07-12 10:28:36 +05:30
Raja Subramanian
5459bd2931 Push track quality to poor on a bandwidth constrained pause. (#1867)
* Push track quality to poor on a bandwidth constrained pause.

* add tests

* scale distance by divisor

* fix test distance to desired

* wait longer for subscription manager to reconcile
2023-07-11 15:29:35 +05:30
Raja Subramanian
1cb74b9e1b Check for desired before clean up. (#1865)
Fix a potential race between needsCleanup checking and a re-subscribe
setting desired back to true.
2023-07-10 13:20:57 +05:30
Raja Subramanian
e6f5f2f344 Prevent anachronous sample reading. (#1863)
* Prevenet anachronous sample reading.

Not so pretty way of solving this. Please let me know if you have
thoughts.

Passing in time allows testing easier. But, that also leads to
time reversal problems. Example scenario
1. Connection stats worker gets a time and initiates quality
   calculation.
2. A layer transition is recorded after that.
3. By the time, scorer is called to calculate score with time from Step
   1, there is time reversal and results in anachronous sample.

One option is to use a scorer lock in connection stats module and wrap
all calls to scorer in that lock, but that does not prevent the passed
in time stamps themselves getting out of order. Also, stand alond use
of scorer in some other context will be problematic.

Doing the hybrid thing of taking current time in scorer if passed in
time is zero so that scorer lock domain controls it.

* use zero time everywhere in normal flow

* make APIs with and without time passed in as Paul suggested
2023-07-10 08:39:52 +05:30
David Zhao
42a7d52272 Return 404 with DeleteRoom/RemoveParticipant when deleting non-existent resources (#1860)
Fixes #1587
2023-07-08 22:30:49 -07:00
David Zhao
cbec68ae44 Do not use cancellable context for Redis operations (#1859) 2023-07-08 12:06:31 -07:00
Raja Subramanian
bf3732b898 Remove noisy debug logs. (#1858) 2023-07-08 11:58:56 +05:30
David Zhao
3e71ea3d77 Fixed hidden participant update (#1857) 2023-07-07 13:36:15 -07:00
cnderrauber
873c87f24b Fix nack issue for svc codecs (#1856)
* Fix nack issue for svc codecs

* Fix test
2023-07-07 15:46:18 +08:00
David Zhao
919355c873 Log additional details when updating participant permissions (#1855)
To help track down sporadic updateParticipant failures
2023-07-06 23:38:01 -07:00
Raja Subramanian
6198aac7a8 Make a default config variable (#1848)
* Make a default config variable

* decoder.Decode needs pointer to config
2023-07-03 23:49:03 +05:30
lukasIO
7e96c98dc3 Select highest layer of equal dimensions (#1841)
* Select highest layer of equal dimensions

* clean up test
2023-07-03 17:32:28 +02:00
Raja Subramanian
869f23a054 Close subscriptions promptly (#1845)
* Close subscriptions promptly

Two things:
-----------
1. Because the desired is not changed, the notifiers are not notified
that the subscription is not observing any more. So, that holds
a refernce to the subscription manager.

Address the above by setting `setDesired` to false on all subscriptions
when subscription manager closes. That will remove observer from the
notifiers.

2. When subscription manager is closed, the down track close
is invoked which flows back (with onClose callback of downtrack) to
subscription manager "handleSubscribedTrackClose". That callback
handler sets the subscribed track to nil for that subscription.

A couple of scenarios here
a. Without the above change, desired could have been true and it would
have looked that the track needs to try subscription again because
`needsSubscribe == true` (desired == true && subscribedTrack == nil)

b. Even with the change above, there is a new condition of
`desired == false && subscribedTrack == nil` and there was no handler
for that condition in the reconciler.

Address this by adding a `needsCleanup` function and delete subscription
from the map. Note that the reconciler may not be running to execute
this action as subscription manager would have closed the `closeCh`, but
doing the code in the interest of proper clean up.

* clean up
2023-07-01 12:31:51 +05:30
Raja Subramanian
e3954d1d64 Use timed aggregator. (#1843)
* Use timed aggregator.

For aggregate bitrate and average distance from desired.

Also, clean up debug added to track leak.

* update deps
2023-07-01 10:21:15 +05:30
Raja Subramanian
06f9b574cb Delete down track from receiver in close always. (#1842)
* Delete down track from receiver in close always.

I think with the parallel close in goroutines, it so happens that
peer connection can get closed first and unbind the track.

The delete down track and RTCP reader close was inside if `bound` block.
So, they were not running leaving a dangling down track in the receiver.

* fix tests

* fix test
2023-06-30 20:44:57 +05:30
Raja Subramanian
496656627e Logging more to understand layer transition leak better. (#1840) 2023-06-30 11:59:53 +05:30
Raja Subramanian
69a1e572be Attempt to reduce disruption due to probe. (#1839)
* Make congestion controller probe config

* Wait for enough estimate samples

* fixes

* format

* limit number of times a packet is ACKed

* ramp up probe duration

* go format

* correct comment

* restore default

* add float64 type to generated CLI
2023-06-30 11:09:46 +05:30
David Zhao
7be9e2258d Upgrade to Pion 3.0.11, disable active TCP (#1836) 2023-06-28 16:53:58 -07:00
Juan Navarro
2668073c29 Honor bind address passed as --bind also for RTC ports (#1815)
* Use net.JoinHostPort to build "host:port" strings for `net.Listen`

net.JoinHostPort provides a unified way of building strings of the form
"Host:Port", abstracting the particular syntax requirements of some
methods in the `net` package (namely, that IPv4 addresses can be given
as-is to `net.Listen`, but IPv6 addresses must be given enclosed in
square brackets).

This change makes sense because an address such as `[::1]` is *not* a
valid IPv6 address; the square brackets are just a detail particular to
the Go `net` library. As such, this syntax shouldn't be exposed to the
user, and configuration should just accept valid IPv6 addresses and
convert them as needed for usage within the code.

* Use '--bind' CLI flag to also filter RTC bind address

The local address passed to a command such as

    livekit-server --dev --bind 127.0.0.1

was being used as binding address for the TCP WebSocket port, but was
being ignored for RTC connections.

With `--dev`, the conf.RTC.UDPPort config is set to 7882, which enables
"UDP muxing" mechanism. Without interface or address filtering, Pion
would try to bind to port 7882 on *all* interfaces.

This was failing on a system with IPv6 enabled, when trying to bind to
an IPv6 address of the `docker0` interface. It seems to make sense that
the user-passed bind addresses are also honored for the RTC port
bindings.
2023-06-28 16:52:43 -07:00
Raja Subramanian
eaf70d5549 Pacer in down stream path. (#1835)
* Pacer interface to send packets

* notify outside lock

* use select

* use pass through pacer

* add error to OnSent

* Remove log which could get noisy

* Starting TWCC work (#1727)

* add packet time

* WIP commit

* WIP commit

* WIP commit

* minor comments

* Some measurements (#1736)

* WIP commit

* some notes

* WIP commit

* variable name change and do not post to closed channel

* unlock

* clean up

* comment

* Hooking up some more bits for TWCC (#1752)

* wake under lock

* Pacer in down stream path.

Splitting out only the pacer from a feature branch to
introduce the concept of pacer.

Currently, there should be no difference in functionality
as a pass through pacer is used.

Another implementation exists which is just put it in a queue and send
it from one goroutine.

A potential implementation to try would be data paced by bandwidth
estimate. That could include priority queues and such.

But, the main goal here is to introduce notion of pacer in the down
stream path and prepare for more congestion control possibilities down
the line.

* Don't need peak detector

* remove throttling of write IO errors
2023-06-28 13:22:44 +05:30
Raja Subramanian
2b0a470474 Less flapping in probe. (#1834)
- Increase max interval between probes to 2 minutes.
- Use a minimum probe rate of 200 kbps. This is to ensure that
the probe rate is decent and can produce a stronger signal.
2023-06-28 12:48:38 +05:30
Raja Subramanian
cea41e4189 Discount out-of-order packets in downstream score. (#1831)
* Discount out-of-order packets in downstream score.

More notes inline.

* correct comment

* clean up comment
2023-06-27 17:44:53 +05:30
cnderrauber
5b975af55f Refine dependency descriptor based selection forwarder (#1808)
* Don't update dependency info if unordered packet received

* Trace all active svc chains for downtrack

* Try to keep lower decode target decodable

* remove comments

* Test case

* clean code

* solve comments
2023-06-27 15:11:06 +08:00
Raja Subramanian
2896aeb126 Set potential codecs for tracks without simulcast codecs. (#1828)
When migrating muted track, need to set potential codecs.
For audio, there may not be `simulcast_codecs` in `AddTrack`.
Hence when migrating a muted track, the potential codecs are not set.
That results in no receivers in relay up track (because all this
could happen before the audio track is unmuted).

So, look at MimeType in TrackInfo (this will be set in OnTrack) and
use that as potential codec.
2023-06-27 04:34:41 +05:30