* Connectino quality misc changes
1. Call scorer.Update() with nil stat when no data available so that
scorer can synthesise window with proper window time.
2. Substract out loss in interval to account for packets not sent at
all.
3. Fix `packetsNotFound` variable in `getIntervalStats`. I remember this
working at some point. Not sure if I fat fingered in another PR and
deleted the increment line.
4. Logging a bit more when no packets expected. Those can get noisy
especially when track is muted. But, seeing some unexplained
instances of no packets leading to quality drop. So, temporary logging
to get a bit more information.
* correct spelling
* Limit packet score minimum to 0.0
* Make connection quality not too optimistic.
With score normalization, the quality indicator showed good
under conditions which should have normally showed some badness.
So, a few things in this PR
- Do not normalize scores
- Pick the weakest link as the representative score (moving away from
averaging)
- For down track direction, when reporting delta stats, take the number
of packets sent actually. If there are holes in the feed (upstream
packet loss), down tracks should not be penalised for that loss.
State of things in connection quality feature
- Audio uses rtcscore-go (with a change to accommodate RED codec). This
follows the E-model.
- Camera uses rtcscore-go. No change here. NOTE: THe rtscore here is
purely based on bits per pixel per frame (bpf). This has the following
existing issues (no change, these were already there)
o Does not take packet loss, jitter, rtt into account
o Expected frame rate is not available. So, measured frame rate is
used as expected frame rate also. If expected frame rate were available,
the score could be reduced for lower frame rates.
- Screen share tracks: No change. This uses the very old simple loss
based thresholding for scoring. As the bit rate varies a lot based on
content and rtcscore video algorithm used for camera relies on
bits per pixel per frame, this could produce a very low value
(large width/height encoded in a small number of bits because of static content)
and hence a low score. So, the old loss based thresholding is used.
* clean up
* update rtcscore pointer
* fix tests
* log lines reformat
* WIP commit
* WIP commit
* update mute of receiver
* WIP commit
* WIP commit
* start adding tests
* take min score if quality matches
* start adding bytes based scoring
* clean up
* more clean up
* Use Fuse
* log quality drop
* Periodically report bitrate to down track.
For connection quality based on bitrate for down tracks,
the measured rate should be used. That is to ensure that
down track quality measurement does not get affected by
publisher side changes negatively (or positively).
Report the optimal bit rate to connection quality scorer
every second so that scorer has a continuously updating
picture of the stream and can compare the actual bit rate
against expected optimal bitrate more reliably.
Doing it at time like allocation, the bitrate may not be
accurate (or may not even be available). So, a periodic update
is necessary.
* add transition at allocation times
* clean up debug log
* - Use number of windows for wait to make things simpler
- track no layer expected case
- always update transition
- always call updateScore
* Make connection quality not too optimistic.
With score normalization, the quality indicator showed good
under conditions which should have normally showed some badness.
So, a few things in this PR
- Do not normalize scores
- Pick the weakest link as the representative score (moving away from
averaging)
- For down track direction, when reporting delta stats, take the number
of packets sent actually. If there are holes in the feed (upstream
packet loss), down tracks should not be penalised for that loss.
State of things in connection quality feature
- Audio uses rtcscore-go (with a change to accommodate RED codec). This
follows the E-model.
- Camera uses rtcscore-go. No change here. NOTE: THe rtscore here is
purely based on bits per pixel per frame (bpf). This has the following
existing issues (no change, these were already there)
o Does not take packet loss, jitter, rtt into account
o Expected frame rate is not available. So, measured frame rate is
used as expected frame rate also. If expected frame rate were available,
the score could be reduced for lower frame rates.
- Screen share tracks: No change. This uses the very old simple loss
based thresholding for scoring. As the bit rate varies a lot based on
content and rtcscore video algorithm used for camera relies on
bits per pixel per frame, this could produce a very low value
(large width/height encoded in a small number of bits because of static content)
and hence a low score. So, the old loss based thresholding is used.
* clean up
* update rtcscore pointer
* fix tests
* log lines reformat
* WIP commit
* WIP commit
* update mute of receiver
* WIP commit
* WIP commit
* start adding tests
* take min score if quality matches
* start adding bytes based scoring
* clean up
* more clean up
* Use Fuse
* log quality drop
* clean up debug log
* - Use number of windows for wait to make things simpler
- track no layer expected case
- always update transition
- always call updateScore
In the following order, got the wrong layer
- Max layer is 0, max published is 0, request layer is 0
- Current locks to 0.
- Max changes to 1. Nothing changes as 1 is not published yet.
- Max published changes to 1.
- As curernt layer is valid, available and locked to request layer, it
was kept. But, it should have checked if the request layer changed
and updated accordingly.
In the following scenario, PLI layer lock got stuck at the wrong layer
- target is at 2 to allow overshoot
- current gets to 1, but can't get higher because publisher is not
publishing higher layer
- max layer changed to 0
Because of adjusting for overshoot only when current == target, it never
happened and layer lock PLI kept asking for layer 0. Although, key
frames were received, switch did not happen.
Always check for overshoot adjustment possibility against current layer.
When we unsubscribe from a speaker, SendSpeakerUpdates will drop updates
from that speaker. This has the side effect of dropping the "clearing"
message that we are sending as well.
* Change lock scope of access to RTCP sender report data.
Forwarder calls back to get time stamp offset.
Holding buffer lock is a much bigger scoped lock.
Reduce lock scope and cache latest sender report under its own lock.
And use that cache when calculating time stamp offset.
* move sr cache to stream tracker manager for re-use in relay
* cache before spread
When the publisher stops publishing, the individual receivers would close
attached DownTracks first before notifying MediaTrackReceiver callbacks.
This means #1454 does not fix the issue entirely since there is still a window
when we can subscribe to a closing track.
A couple of other bits
1. Use request layer for sending PLI on bind and connected.
2. When adjusting for overshoot, do not adjust target unless current is
at max. If not, it could get stuck in a lower layer in the following
scenario
a. Overshoot to layer 2
b. Max layer is 1, start sending PLI
c. Get key frame for layer 0, adjust for overshoot as we have
something at a layer lower than max.
d. Adjust for overshoot.
e. Setting target to max means that current and target are equal
and no further adjustment happens.
Due to the order of events in MediaTrackReceiver and friends, SubscribedTrack
will be closed before the track is removed from RoomTrackManager.
Because of this, when a track is unpublished, it's possible to be subscribed
to the track as it's closing.
By introducing a closing state, we'd prevent accidental subscription to
closing tracks.
* Do not overshoot when layer is locked.
One more challenging case. When current layer is already locked,
should not set up for overshoot.
* set target to current
Missed this in the last commit. Sorry.
The case of
- locking to a layer
- that layer stops
- re-allocation
This should trigger a key frame request to the max available layer.
So, have to set the request layer to max available.
Addressing edge case where a layer stopped before bitrate could be
measured. Purely bit rate based change deduction missed this as
the before and after did not have bit rates.
Use available layers to look for changes, especially currently
forwarding layer going away.
Also, simplifying bits. Only in the optimal allocation path,
these things are required. When congested, bitrate is always needed.
So, for optimal path, just look at available layer changes and adjust.
Don't need to look for bitrate based layer changes. Clean up that code.
The following sequence caused early migration complete declaration
1. Audio track received
2. Audio track published callback in progress
3. Video track received, this clears the pending track
4. Audio track published callback finishes. This checks for pending
tracks. As nothing is pending migration complete declared.
5. Due to the above, the remote video track is closed as not resuming.
That causes an unsubscription.
Fix
- Wait till publish callback to finish to remove a track from pending
fully.
- Introducing a new map as pending tracks is used in OnClose too. So,
did not want to delay removing from it as a close could happen while
publish callback is happening.
Also, moving the publish callback to a go routine (just like the recent
change for running those in a go routine for migrated muted tracks)
