* Loss based congestion signal detector.
It uses the same approach of thresholding + duration to detect
region of operation and further derive early warning/congested states.
A gutter is used for indeterminate region just like the queuing delay
based case.
The two approaches (queuing delay and loss) are treated independently,
i. e. packet groups have to satify the same type of condition (queuing
delay OR loss) to build up congestion.
The aggregate congestion signal is triggered if either one triggers.
Maybe, there is a way to accept hybrid signalling (i. e. each group
satisfying either threhsold adds up to congestion signal detection), but
needs more experimentation. For now, keeping them separate.
* apply max threshold
* clean up
* spelling
* file output
* wake under lock
* keep track of RTX bytes separately
* packet group
* Packet group of 50ms
* Minor refactoring
* rate calculator
* send bit rate
* WIP
* comment
* reduce packet infos size
* extended twcc seq num
* fix packet info
* WIP
* queuing delay
* refactor
* config
* callbacks
* fixes
* clean up
* remove debug file, fix rate calculation
* fmt
* fix probes
* format
* notes
* check loss
* tweak detection settings
* 24-bit wrap
* clean up a bit
* limit symbol list to number of packets
* fmt
* clean up
* lost
* fixes
* fmt
* rename
* fixes
* fmt
* use min/max
* hold on early warning of congestion
* make note about need for all optimal allocation on hold release
* estimate trend in congested state
* tweaks
* quantized
* fmt
* TrendDetector generics
* CTR trend
* tweaks
* config
* config
* comments
* clean up
* consistent naming
* pariticpant level setting
* log usage mode
* feedback
* De-centralize some configs to where they are used.
And make default variables.
Renaming a bit, but these are all internal config and have not been
added to documented config.
* Keep documented config as is.
* test
* typo
No functional effect, but was logging more than expected drift in the
down stream direction. Reason is that when passing through, we could be
using an older report. But, the adjustment was applied to the monotonic
clock and not the RTP timestamp. So, it looked like more time had
elapsed for the same RTP clock elapsed and logging higher than expected
drift. Correcting it so that the log is not misleading/confusing.
Revert part of my previous commit. I vaguely remembered there was a reason for
having code like that, but did not remember the details and ended up
consolidating. The issue is that time needs to be grabbed under lock so
that two events happening close to each other do not get order swapped.
* Set FEC enabled properly in connection stats module.
With RED, the FEC indication is in primary codec.
Also, clean up some bits that were not necessary (TrackInfoAvailable is
not needed)
TODO: There are still a couple of things to figure out
- If codec is RED, Opus is added as second codec synthetically using
https://github.com/livekit/livekit/blob/33098337fc17705bbdb3283c7a7034aa6b2f3745/pkg/rtc/mediaengine.go#L31
which hard codecs FEC enabled. Ideally, we should get the primary
codec parameters from SDP offer.
- The WebRTCReceiver does not have information about primary codec. For
now, just setting FEC to true when RED is enabled. It is okay as it
just affects when we declare quality drops, but ideally the primary
codec should be retrieved from SDP offer.
* clean up and comment
* full prop check
Also, logging a bit more about quality changes to understand why
high(ish) loss does not drop quality. Will remove the loss thresholded
logging after collecting some data.
Transport will send offer immediately if last
negotiation is before debounce interval in #1929,
it will cost two negotiation for a/v tracks if a
pubisher publishes two tracks at same time like
screenshare or enable mic/camera. This change use
a small debounce interval in this case to avoid this issue.
* Make lite version of RTPStatsReceiver called RTPStatsReceiverLite.
Refactor around that.
Will probably make some more flavors to have lighter versions still.
* update deps
* use MarshalLogArray
* use util
* Splitting out rtp stats stuff into its own package.
Going to be making some lighter versions of these.
Will be cleaner to have all of these grouped together.
So, as a first step, just making a package for it.
* tests
* Initial plumbing for metrics.
This implements
- metrics received from participant.
- callback to room.
- room distributes it to all other participants (excluding the sending
participant).
- other participants forward to client.
- counting metrics bytes in data channel stats
TODO:
- recording/processing/batching
- should recording/processing/batching happen on publisher side or
subscriber side?
- should metrics be echoed back to publisher?
- grants to publish/subscribe metrics.
* mage generate
* clear OnMetrics on close
* - CanSubscribeMetrics permission.
- Echo back to sender.
* update deps
* No destination identities for metrics
* WIP
* use normalized timestamp for server injected timestamps
* compile
* debug log metrics batch
* correct comment
* add baseTime to wire
* protocol dep
* Scope metrics forwarding to only participants that a participant is
subscribed to.
Also remove the participant_metrics.go file as it was not doing anything
useful.
* update comment
* utils.ErrorIsOneOf
* couple of more utils.CloneProto
* Log only when not nil.
Default logging confuses debugging as we call using nil as well to make
the call site simpler. And logging a nil makes it look like it is
incorrect seeding. `nil` fields do not seed. So, don't log when `nil`.
* log SDP
With protobuf marshalling/unmarshalling, the default struct was getting
used and nil checks for non existence were failing. That means the right
layers were not reported active on migration.
Also, restore the wrapped loggers as some fields needs some conversion
before logging.
Also, log when RTCP sender report for reference layer is received.
Cleaning up a bunch of wrapped logger calls as we already have
logger.Proto which I forgot about while doing those wrappers.
To increase visibility of ICE reconnect, logging reconnected at Infow
level. Otherwise, it is hard to see if an ICE restart finished
successfully.
Also, cleaning up ICEConnectionDetails a bit. Just separate out
read-only fields into its own struct and use it for read-only export.
* Cache RTCP sender report in forwarder state.
To be used in migration.
TODO: need to check more places to operate pure in unix nano rather than
converting.
* match name
* Record out-of-packet count/rate in prom.
Adding a field to AnalyticsStream to make this easier to report.
Let me know if adding to AnalyticsStream is not ok.
Will set up a protocol PR if it is okay.
* deps
But, do not record first packet time on an out-of-order packet.
It so happens that packets get out-of-order a lot more across relay.
And it turns out with some H.264 stream, the first few packets of a key
frame are very small (may be SPS/PPS, haven't checked), they get
out-of-oder quite a lot, so much so a down track never starts even it
has 20 - 25 key frames have passed through.
* Negotiate downttrack for subscriber before receiver is ready
This change will save 1 round sdp negotiation time for
subscribing to simulcast-codec or remote node track
* solve comment
* Fix simulcast-codec case
Raja Subramanian