* Log only when not nil.
Default logging confuses debugging as we call using nil as well to make
the call site simpler. And logging a nil makes it look like it is
incorrect seeding. `nil` fields do not seed. So, don't log when `nil`.
* log SDP
With protobuf marshalling/unmarshalling, the default struct was getting
used and nil checks for non existence were failing. That means the right
layers were not reported active on migration.
Also, restore the wrapped loggers as some fields needs some conversion
before logging.
Also, log when RTCP sender report for reference layer is received.
Cleaning up a bunch of wrapped logger calls as we already have
logger.Proto which I forgot about while doing those wrappers.
* Do not remove from subscription map on unsubscribe.
Notes in line as to why.
* Avoiding the extra check. It is fine to check under lock for clean up.
It is done in other places.
* comments
To increase visibility of ICE reconnect, logging reconnected at Infow
level. Otherwise, it is hard to see if an ICE restart finished
successfully.
Also, cleaning up ICEConnectionDetails a bit. Just separate out
read-only fields into its own struct and use it for read-only export.
* Cache RTCP sender report in forwarder state.
To be used in migration.
TODO: need to check more places to operate pure in unix nano rather than
converting.
* match name
When a user includes a trailing slash in LIVEKIT_URL, it would produce
double slashes in the path, i.e. `https://myhost.livekit.cloud//twirp/RoomService.ListRooms`
Currently the server will send a 302 MOVED response, causing Twirp requests to fail.
We now remove the double slash in front within the middleware.
* Record out-of-packet count/rate in prom.
Adding a field to AnalyticsStream to make this easier to report.
Let me know if adding to AnalyticsStream is not ok.
Will set up a protocol PR if it is okay.
* deps
* Ref count the stats worker.
NOTE: Don't liek this much, but wanted to open this get some 👀 on
this and get feedback.
There are two entities, one for counting signal bytes and another for
media stats. They both send `ParticipantJoined` and `ParticipantLeft`
event.
In the case of a participant resume, as the old web socket
connection is closed, that triggers a signal stats counter close. That
would call `ParticipantLeft` and that would close the stats worker.
The closed stats worker got reaped in `FlushStats` after three minutes.
So, all events after that did not have a worker and hence went
unreported including missing participant_left webhook because it relied
on checking if a participant was ever connected and that needed to check
the worker state.
Using a ref count to keep track of join/leaves. And not close the worker
until ref count goes down to 0.
* create a stats worker on resume
* revert incorrect changes
* transfer connected state
* transfer connected state when creating worker
* resolve participant on a resume
* exponential backoff when calling CreateRoom
* dz+bb review: check ctx cancellation, remove retry max
* bb review: fix loop condition
* raja review: use timer, bring back max tries
But, do not record first packet time on an out-of-order packet.
It so happens that packets get out-of-order a lot more across relay.
And it turns out with some H.264 stream, the first few packets of a key
frame are very small (may be SPS/PPS, haven't checked), they get
out-of-oder quite a lot, so much so a down track never starts even it
has 20 - 25 key frames have passed through.
* Negotiate downttrack for subscriber before receiver is ready
This change will save 1 round sdp negotiation time for
subscribing to simulcast-codec or remote node track
* solve comment
* Fix simulcast-codec case
* Do not ICE restart on an idle/not yet started peer connection.
* Skip ICE restart on unestablished peer connection.
For publish only participants, the subscriber peer connection is not
negotiated. So, ICE restart was hitting an error while trying to restart
the SUBSCRIBER peer connection.
* use ICE gathering state as peer connection state may not have changed if first offer/answer was missed
* speed up track publication
Add metrics for track publication and subscription
Return EnabledCodecs in JoinResponse so client can
choose codec without server side codec fallback
Cache remote webrtc track without AddTrackRequest to
let client send publisher offer before AddTrackRequest response
* go mod
* clean code
There are cases where the very first packet on resume is an out-of-order
packet. In that case, the gap in both sequence number and time stamp is
a small(ish) negative number. With a high threshold to declare very old
packet, the condition does not trip and the packet gets through and
treated as a packet that has rolled over.
It should be fine to have smaller threshold (in fact, it is probably
okay to have something a little over 1.0 too) as the expected jump is
calculated based on elapsed time since last packet receive and new
packets should be coming in with a diff close to that. So, a factor of
just over 1.0 to prevent false triggers should be fine. Using 1.5 for
now.
