- With probing the packet rate can get high suddenly and remote may not
have sent receiver report as it might be sending for the non-spikey
rate. That causes metadata cache overflows. So, give RTX more cahe.
- Don't need a large cache for primary as either reports come in
regularly (or they are missing for a long time and having a biger
cache is not the solution for that, so reduce primary cache size)
- Check for receiver report falling exactly back by (1 << 16). Had done
that change in the inside for loop, but missed the top level check :-(
This is mostly to clean up forwarder state cache for already started
tracks.
A scenario like the following could apply the seed twice and end up with
an incorrect state resulting in a large jump
- Participant A let's say is the one showing the problem
- Participant A migrates first. So, it tries to restore its down track states by querying state from the previous node.
- But, its down tracks start before the response can be received. However, it remains in the cache.
- Participant B migrates from a different node to where Participant A. So, the down track of Participant A gets switched from relay up track publisher -> local up track publisher.
- I am guessing the seeding gets applied twice in this case and the cached value from step 3 above causes the huge jump.
In those cases, the cache needs to be cleaned up.
(NOTE: I think this seeding of down track on migration is not necessary
as the SSRC of down track changes and the remote side seems to be
treating it like a fresh start because of that. But, doing this step
first and will remove the related parts after observing for a bit more)
Also, moving fetching forwarder state to a goroutine as it involves a
network call to the previous node via Director.
Seeing some panic due to sender view being nil. It is possible to have
nil sender view and not-nil receiver view. For analytics, only sender
view is used. Handle nil properly.
Receiver view is used for connection quality.
Sender view is used for analytics. One thing that this introduces is
that sender view uses the packet loss information from receiver view as
true loss is available only in the RTCP Receiver Reports received from
the remote side. So, the time alignment is off, i. e. receiver report
happens periodically and it includes information till the time at which
it was sent from remote side, but sender could have sent more packets
after that time.
The split should ensure that analytics does not rely on remote side
sending proper receiver repoerts albeit at slight misalignment of loss
statistic for remotes that send RTCP RR (which should be majority of the
cases)
Actually, was caused by down track not initialising mime, but it is good
to ignore unknown mime.
Also, added ulpfec sa SDP has that and there was a conversion which was
returning unknown. Also, note that the mime types use audio for RED and
video for flexfec and ulpfec although they are not media type dependent.
Maybe, at point need to introduce `MimeTypeAudioRED` and
`MimeTypeVideoRED`.
* Normalize mime type and add utilities.
An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.
Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.
* more locations
* test
* Paul feedback
* MimeType type
* more consolidation
* Remove unused
* test
* test
* mime type as int
* use string method
* Pass error details and timeouts. (#3402)
* go mod tidy (#3408)
* Rename CHANGELOG to CHANGELOG.md (#3391)
Enables markdown features in this otherwise already markdown'ish formatted document
* Update config.go to properly process bool env vars (#3382)
Fixes issue https://github.com/livekit/livekit/issues/3381
* fix(deps): update go deps (#3341)
Generated by renovateBot
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Use a Twirp server hook to send API call details to telemetry. (#3401)
* Use a Twirp server hook to send API call details to telemetry.
* mage generate and clean up
* Add project_id
* deps
* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
participant_id, track_id as appropriate
- Store status as int
* deps
* Update pkg/sfu/mime/mimetype.go
* Fix prefer codec test
* handle down track mime changes
---------
Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
* Use a Twirp server hook to send API call details to telemetry.
* mage generate and clean up
* Add project_id
* deps
* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
participant_id, track_id as appropriate
- Store status as int
* deps
While working on a different service, noticed delegation of setting up
routes to the service itself. So, making that change and making some
methods internal only as there is no need to export them outside the
service.
* Do not skip due to large RR interval.
With sequence number adjustment, it will report some packets missing
which is fine.
* do not seed if already initialized
No functional change, just logging reason was confusing.
Also, log no packets case. Seeing some instances in staging where there
are periods of no packets received. Trying to understand better.
initially.
Out-of-order reception is detected and dropped before this change. By
moving the adjustment to that point, potentially more reports can be
processed.
TrackSubscribed is meant to give publishers an indication when the subscriber
is ready to receive its audio. When there are hidden recorders in the room,
we do not want them to trigger this event.
My previous PR was trying to fix a premature transition from CONGESTED -> NONE state.
But, that introduced a bug which accelerated the transition from EARLY_WARNING -> CONGESTED state due to holding the `queuingRegion` state variable.
Fix by not holding `queuingRegion` in state and take result of
processing runs.
NOTE: This can still use some work to make the state machine cleaner.
Initially, I was passing around a bunch of variables which was uncouth.
Then moved things to state which made code easier to read and also log
different things, but it causes problems like the ones above. Will take
another look and think more about making it more robust.
