Also, logging a bit more about quality changes to understand why
high(ish) loss does not drop quality. Will remove the loss thresholded
logging after collecting some data.
Transport will send offer immediately if last
negotiation is before debounce interval in #1929,
it will cost two negotiation for a/v tracks if a
pubisher publishes two tracks at same time like
screenshare or enable mic/camera. This change use
a small debounce interval in this case to avoid this issue.
* Make lite version of RTPStatsReceiver called RTPStatsReceiverLite.
Refactor around that.
Will probably make some more flavors to have lighter versions still.
* update deps
* use MarshalLogArray
* use util
* Splitting out rtp stats stuff into its own package.
Going to be making some lighter versions of these.
Will be cleaner to have all of these grouped together.
So, as a first step, just making a package for it.
* tests
* Initial plumbing for metrics.
This implements
- metrics received from participant.
- callback to room.
- room distributes it to all other participants (excluding the sending
participant).
- other participants forward to client.
- counting metrics bytes in data channel stats
TODO:
- recording/processing/batching
- should recording/processing/batching happen on publisher side or
subscriber side?
- should metrics be echoed back to publisher?
- grants to publish/subscribe metrics.
* mage generate
* clear OnMetrics on close
* - CanSubscribeMetrics permission.
- Echo back to sender.
* update deps
* No destination identities for metrics
* WIP
* use normalized timestamp for server injected timestamps
* compile
* debug log metrics batch
* correct comment
* add baseTime to wire
* protocol dep
* Scope metrics forwarding to only participants that a participant is
subscribed to.
Also remove the participant_metrics.go file as it was not doing anything
useful.
* update comment
* utils.ErrorIsOneOf
* couple of more utils.CloneProto
* Log only when not nil.
Default logging confuses debugging as we call using nil as well to make
the call site simpler. And logging a nil makes it look like it is
incorrect seeding. `nil` fields do not seed. So, don't log when `nil`.
* log SDP
With protobuf marshalling/unmarshalling, the default struct was getting
used and nil checks for non existence were failing. That means the right
layers were not reported active on migration.
Also, restore the wrapped loggers as some fields needs some conversion
before logging.
Also, log when RTCP sender report for reference layer is received.
Cleaning up a bunch of wrapped logger calls as we already have
logger.Proto which I forgot about while doing those wrappers.
To increase visibility of ICE reconnect, logging reconnected at Infow
level. Otherwise, it is hard to see if an ICE restart finished
successfully.
Also, cleaning up ICEConnectionDetails a bit. Just separate out
read-only fields into its own struct and use it for read-only export.
* Cache RTCP sender report in forwarder state.
To be used in migration.
TODO: need to check more places to operate pure in unix nano rather than
converting.
* match name
* Record out-of-packet count/rate in prom.
Adding a field to AnalyticsStream to make this easier to report.
Let me know if adding to AnalyticsStream is not ok.
Will set up a protocol PR if it is okay.
* deps
But, do not record first packet time on an out-of-order packet.
It so happens that packets get out-of-order a lot more across relay.
And it turns out with some H.264 stream, the first few packets of a key
frame are very small (may be SPS/PPS, haven't checked), they get
out-of-oder quite a lot, so much so a down track never starts even it
has 20 - 25 key frames have passed through.
* Negotiate downttrack for subscriber before receiver is ready
This change will save 1 round sdp negotiation time for
subscribing to simulcast-codec or remote node track
* solve comment
* Fix simulcast-codec case
There are cases where the very first packet on resume is an out-of-order
packet. In that case, the gap in both sequence number and time stamp is
a small(ish) negative number. With a high threshold to declare very old
packet, the condition does not trip and the packet gets through and
treated as a packet that has rolled over.
It should be fine to have smaller threshold (in fact, it is probably
okay to have something a little over 1.0 too) as the expected jump is
calculated based on elapsed time since last packet receive and new
packets should be coming in with a diff close to that. So, a factor of
just over 1.0 to prevent false triggers should be fine. Using 1.5 for
now.
* Handle another old packet condition.
With this detection, the sequence number can be rolled over even when TS
rollover is not possible. For example, a track at 300 pps can rollver
the sequence number space in minutes compared to 13h+ for video time
stamp to roll over.
* fix typo
* Refactor propagation delay estimator.
NOTE: It is not possible to calculate OWD (one-way-delay) in a passive
fashion. So, this should not be used for anything requiring high
precision.
But, mainly factoring it out as a separate object just in case it can be
re-used.
TODO:
- probably has some edge case that is not handled well
- maybe path change detection can be improved
- will write UT later. This is just purely splitting it out from what
was embedded in RTPStatsReceiver.
* fix labels
* precision -> accuracy
* Reset DD tracker layers when muted.
@cnderrauber, I think this is okay to do, but please let me know if
there are gotchas in there.
* copy
* more compact form
It is possible that old packets arrive on receiver. If subscriber starts
on that, the first packet time would be incorrect. Do not start
forwarding on out-of-order packets.
There are cases of small negative sequence number jump and small
positive time stamp jump. Those should not force rollover. Maybe, they
should be dropped, but just logging for now till we learn more.
* Fix forced rollover of RTP time stamp.
Was erroneously forcing a rollover when the timestamp jump actually has
room to accommodate large jumps. For example, before pause ts = 10, then
eight hour pause, restart ts = 10 + (8 * 00 * 60 * 90000) = 2592000010
(at 90000 clock rate for video). In normal processing, it will look like
out-of-order as the difference 2592000000 is more than half the 32-bit
range. But, forcing a roll over is incorrect.
Fix by calculating excess over the full range and then account for wrap
around.
* log potential ts rollover
* clamp at min 0
Raja Subramanian