116 Commits

Author SHA1 Message Date
cnderrauber
e1cc9d6b3c Fix log marshal error (#2295) 2023-12-06 00:08:48 +08:00
David Zhao
3fe124c87f Log cleanup pass (#2285)
* Log cleanup pass

Demoted a bunch of logs to DEBUG, consolidated logs.

* use context logger and fix context var usage

* moved common error types, fixed tests
2023-12-02 15:07:31 -08:00
Raja Subramanian
440f00bcac Declare audio inactive if stale. (#2229)
* Declare audio inactive if stale.

Stale samples were used to declare audio active.
Maintain last update time and declare inactive if samples are stale.

* correct comment

* spelling

* check level in test
2023-11-08 11:13:39 +05:30
cnderrauber
f247b68ed6 Make sure dd selector uses correct keyframe to select packets (#2218)
* Make sure dd selector uses correct keyframe to select packets

* Fix test case

* remove unsed field
2023-11-03 17:49:02 +08:00
Raja Subramanian
0bdfdb0c49 Squelching DD reader error. (#2215)
Squelching Structure is nil error as it can happen on packets
received before a key frame is received.
2023-11-02 11:10:28 +05:30
cnderrauber
1f0ba21854 Fix svc: Drop frame is earlier than current keyframe (#2196)
* Fix svc: Drop frame is earlier than current keyframe

* Log detail of dependencydescriptor
2023-10-27 13:57:03 +08:00
Raja Subramanian
3e4cd3a161 Accept more range for first packet time adjustment. (#2150) 2023-10-17 23:52:14 +05:30
Raja Subramanian
e6e3e2a729 sligtly easier readability (#2121) 2023-10-02 22:47:40 +05:30
Pingos
4f9467040e Bind() function fails when mime == "audio/red" (#2104) 2023-09-26 13:36:54 +08:00
Raja Subramanian
a55c50f61d Throttle packet errors/warns a bit. (#2068)
* Throttle packet errors/warns a bit.

In very bad network conditions, it is possible that packets
arrive out-of-order, have choppy behaviour.

Use some counters and temper logs.

* slight change in comment
2023-09-13 15:52:10 +05:30
Raja Subramanian
c09d8d0878 Split RTPStats into receiver and sender. (#2055)
* Split RTPStats into receiver and sender.

For receiver, short types are input and need to calculate extended type.

For sender (subscriber), it can operate only in extended type.
This makes the subscriber side a little simpler and should make it more
efficient as it can do simple comparisons in extended type space.

There was also an issue with subscriber using shorter type and
calculating extended type. When subscriber starts after the publisher
has already rolled over in sequence number OR timestamp, when
subsequent publisher side sender reports are used to adjust subscriber
time stamps, they were out of whack. Using extended type on subscriber
does not face that.

* fix test

* extended types from sequencer

* log
2023-09-11 07:33:39 +05:30
Raja Subramanian
bc5b4d68af Do NACKs and reports always. (#2022)
* Do NACKs and reports always.

With padding packet drops, it is possible that a lot of packets
go by without RTCP RR.

Do NACKs and RTCP RR always.

* remove local variable
2023-08-31 12:41:25 +05:30
Raja Subramanian
790954bbe9 Use RTCP SR to resync. (#2021)
Remove packet debug code that was added temporarily.
2023-08-31 11:45:42 +05:30
Raja Subramanian
6e3a20ebf4 Temporary packet debug (#2018) 2023-08-31 00:33:00 +05:30
Raja Subramanian
da52167cd9 Adjust extended sequence number to account for dropped packets (#2017) 2023-08-30 23:08:53 +05:30
Raja Subramanian
9afb0873ae Do not process packets not processed by RTPStats. (#2015)
Seeing the case of a stream starting with padding packets
on migration. As publisher in that case is always sending packets.
it is possible that the new node gets padding packets at the start.
Processing them in buffer leads to trying to drop that padding packet
and adding an exclusion range. That fails because the extended
sequence number is not available for unprocessed packets.

It is okay to drop them as they will be dropped anyway.
But, they are useful for bandwidth estimation. So, headers are processed
even if the packet is RTPStats unprocessed.
2023-08-30 19:43:24 +05:30
Raja Subramanian
126872047d Handle duplicate padding packet in the up stream. (#2012)
* Handle duplicate padding packet in the up stream.

The following sequence would have produce incorrect results
- Sequence number 39 - regular packet - offset = 0
- Sequence number 40 - padding only - drop - offset = 1
- Sequence number 40 - padding only duplicate - was not dropped (this is
  the bug) - apply offet - sequence number becomes 39 and clashes with
  previous packet
- Sequence number 41 - regular packet - apply offset - goes through as 40.
- Sequence number 40 again - does not get dropped - will pass through as 39.

* fix duplicate dropping

* fix tests

* accept repeat last value as padding injection could cause that

* use exclusion ranges

* more UT and more specific errors
2023-08-30 16:46:39 +05:30
Raja Subramanian
a48660fa77 Make extended sequence number 64-bit. (#2003) 2023-08-27 21:26:31 +05:30
Raja Subramanian
3b30f49ad5 Extended type for RTP timestamp. (#2001) 2023-08-27 17:28:44 +05:30
Raja Subramanian
55d5edcf73 Use range map in RTPMunger. (#2000)
* WIP commit

* Make lastSN 32-bit

* Remove unused TSCycles
2023-08-27 10:49:17 +05:30
Raja Subramanian
36dadbacb2 Drop padding only packets on publisher side. (#1990)
* Drop padding only packet on publisher side.

* add UT

* update deps

* remove debug

* add fast path short cut

* correct comment

* fix test

* fix for Linux
2023-08-24 09:09:42 +05:30
Raja Subramanian
0e9ec9a21e Ignore lagging layer switches. (#1948) 2023-08-09 17:42:33 +05:30
David Zhao
debd75fa15 Integrate logger components (#1933)
* Integrate logger components

Dividing into the following components
* pub - publisher
* pub.sfu
* sub - subscriber
* transport
* transport.pion
* transport.cc
* api
* webhook

* update go modules
2023-08-03 13:31:17 -07:00
cnderrauber
f7a1776f4c Add control of playout delay (#1838)
* Add control of playout delay

Add config to enable playout delay. The delay will be limited by
[min,max] in the config option and calculated by upstream & downstream
RTT.

* check protocol version to enable playout delay

* Move config to room, limit playout-delay update interval, solve comments

* Remove adaptive playout-delay

* Remove unused config
2023-08-02 16:12:23 +08:00
David Zhao
981fb7cac7 Adding license notices (#1913)
* Adding license notices

* remove from config
2023-07-27 16:43:19 -07:00
cnderrauber
5b975af55f Refine dependency descriptor based selection forwarder (#1808)
* Don't update dependency info if unordered packet received

* Trace all active svc chains for downtrack

* Try to keep lower decode target decodable

* remove comments

* Test case

* clean code

* solve comments
2023-06-27 15:11:06 +08:00
Raja Subramanian
9809b8bc3a Use nack queue params. (#1789)
* Use nack queue params.

* fix test
2023-06-12 13:01:02 +05:30
cnderrauber
c91889edfd Add dependency descriptor stream tracker for svc codecs (#1788)
* Add dependency descriptor stream tracker for svc codecs

* Solve comments
2023-06-12 15:07:47 +08:00
Raja Subramanian
f518f5d743 Log head SN when packet cannot be fetched (#1780) 2023-06-09 12:13:06 +05:30
Raja Subramanian
0354626bfc Adjust sender report time stamp for slow publishers. (#1740)
It is possible that publisher paces the media.
So, RTCP sender report from publisher could be ahead of
what is being fowarded by a good amount (have seen up to 2 seconds
ahead). Using the forwarded time stamp for RTCP sender report
in the down stream leads to jumps back and forth in the down track
RTCP sender report.

So, look at the publisher's RTCP sender report to check for it being
ahead and use the publisher rate as a guide.
2023-05-25 21:55:54 +05:30
Raja Subramanian
25d6fd751f Cleaning up smoothed OWD calculation for sender report. (#1684)
* Keep track of expected RTP time stamp and control drift.

- Use monotonic clock in RTCP Sender Report and packet times
- Keep the time stamp close to expected time stamp on layer/SSRC
  switches

* clean up

* fix test compile

* more test compile failures

* anticipatory clean up

* further clean up

* add received sender report logging
2023-05-05 13:14:12 +05:30
Raja Subramanian
15078eb9f4 Keep track of expected RTP time stamp and control drift. (#1681)
* Keep track of expected RTP time stamp and control drift.

- Use monotonic clock in RTCP Sender Report and packet times
- Keep the time stamp close to expected time stamp on layer/SSRC
  switches

* clean up

* fix test compile

* more test compile failures
2023-05-04 13:00:57 +05:30
Raja Subramanian
69fb5e51a2 Fix stutter in forwarding path when using dependency descriptor (#1600)
* Decode chains

* clean up

* clean up

* decode targets only on publisher side

* comment out supported codecs

* fix test compile

* fix another test compile

* Adding TODO notes

* chainID -> chainIdx

* do not need to check for switch up point when using chains, as long as chain integrity is good, can switch

* more comments

* address comments
2023-04-12 17:30:54 +05:30
Raja Subramanian
c56e37f3fe Fix VP9 stutter in non-DD and some other misc changes (#1595)
* WIP commit

* WIP commit

* clean up

* remove todo

* fix test
2023-04-10 12:31:07 +05:30
Raja Subramanian
e32eaa451f Refactor video layer selector (#1588)
* WIP commit

* WIP commit

* fix test

* FPS for VP9

* WIP commit

* test changes

* WIP commit

* h264

* codec munger

* forwarder state

* clean up a bit

* dd interface

* WIP commit

* WIP commit

* WIP commit

* WIP commit

* more TODO notes

* overshoot interface

* clean up

* clean up isTemporalSupported

* wait for key frame to resume

* clean up VP8 payload descriptor stuff

* temporal layer selector

* comment out vp9 and av1

* space

* fix test compile

* append bytes

* fix tests

* fix test
2023-04-08 10:57:57 +05:30
cnderrauber
fb301e6e75 Add vp9 svc support by Dependency Descriptor (#1586)
* Add VP9 SVC support

* Fix preferred fps does not work

* Fix forwarder test
2023-04-06 21:51:33 +08:00
renovate[bot]
2fce780ce8 Update go deps (#1402)
* Update go deps

Generated by renovateBot

* use generics with Deque

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: David Zhao <dz@livekit.io>
2023-03-25 23:17:04 -07:00
David Colburn
191a9e8014 update core to 0.0.5 (#1540)
* update core

* sort imports

* fix typos

* redundant types
2023-03-22 16:53:23 -07:00
David Zhao
5ff72a99b9 Report publish & subscribe RTPStats as Telemetry events (#1506) 2023-03-10 10:28:54 -08:00
cnderrauber
3a4a294a92 Enable upstream nack for opus only audio track (#1343) 2023-01-28 16:02:38 +08:00
Raja Subramanian
c696626fe8 Use local time base for NTP in RTCP Sender Report for downtracks. (#1321)
* Use local time base for NTP in RTCP Sender Report for downtracks.

More details in comments in code.

* Remove debug

* RTCPSenderReportInfo -> RTCPSenderReportDataExt

* Get rid of sender report data pointer checks
2023-01-25 11:00:15 +05:30
Raja Subramanian
2b89c821ab An attempt to use publisher side RTCP sender report while forwarding (#1286)
* WIP commit

* comment

* clean up

* remove unused stuff

* cleaner comment

* remove unused stuff

* remove unused stuff

* more comments

* TrackSender method to handle RTCP sender report data

* fix test

* push rtcp sender report data to down tracks

* Need payload type for codec id mapping in relay protocol

* rename variable a  bit
2023-01-06 14:07:18 +05:30
Raja Subramanian
2b031a5112 Introducing frame based stream tracker. (#1267)
* Split stream tracker impl from base

* slight re-arrangement of code

* fps based stream tracker

* MinFPS config

* switch back to packet based tracker

* use video config by default to handle sources without type
2022-12-28 13:00:21 +05:30
David Zhao
7a1273151f Update to new logging library, using sampling participant logger (#1219) 2022-12-09 00:09:03 -08:00
Raja Subramanian
aba18accd9 Prevent rtx buffer and forwarding path colliding (#1174)
* Prevent rtx buffer and forwarding path colliding

Received packets are put into RTX buffer which is
a circular buffer and the packet (sequence number) is
queued for forwarding. If the RTX buffer fills up
and cycles before forwarding happens, forwarding
would pick the wrong packet (as it is holding a
reference to a byte slice in the RTX buffer) to forward.

Prevent it by moving reading from RTX buffer just
before forwarding. Adds an extra copy from RTX buffer
-> temp buffer for forwarding, but ensures that forwarding
buffer is not used by another go routine.

* Revert some changes from previous commit

Details:
- Do all forward processing as before.
- One difference is not load raw packet into ExtPacket.
- Load raw packet into provided buffer when module that reads
using ReadExtended calls that function. If the packet is
not there in the retransmission buffer, that packet will be
dropped. This is the case we are trying to fix, i. e. the RTX
buffer has cycled before ReadExtended could pull the packet.
This makes a copy into the provided buffer so that the data
does not change underneath.

* Remove debug comment

* Oops missed a function call
2022-11-19 13:19:49 +05:30
Raja Subramanian
379301c0bc A few more log tweaks (#1159) 2022-11-11 13:44:14 +05:30
cnderrauber
4783db34ef Remove DD extension when AV1 not preferred (#1129) 2022-10-27 15:51:50 +08:00
cnderrauber
7a7fc09372 Add fps calculator for VP8 and DependencyDescriptor (#1110)
* Add fps calculator for VP8 and DependencyDescriptor

* clean code

* unit test

* clean code

* solve comment
2022-10-26 09:28:28 +08:00
Raja Subramanian
794c74360b Do not log duplicate packet error. (#1116)
Also, use bucket.Err* as it has been moved to mediatransportutil
2022-10-23 09:39:50 +05:30
Raja Subramanian
792349cc56 Split out mediatransportutil (#1071) 2022-10-06 23:55:59 +05:30