* Use Muted in TrackInfo to propagated published track muted.
When the track is muted as a receiver is created, the receiver
potentially was not getting the muted property. That would result in
quality scorer expecting packets.
Use TrackInfo consistently for mute and apply the mute on start up of a
receiver.
* update mute of subscriptions
* Do not increase max expected layer on track info update.
When max expected layer increases, the corresponding trackers are reset
so that first packets from those layers can trigger a layer detected
change enabling quick detection of layer start.
A track info update changing max to what is in track info could set the
max expected to be higher without resetting the tracker. And that would
cause dynacast induced max layer change to miss tracker reset too.
Sequence
- dynacast sets max expected to 0
- track info update sets it to 2
- dynacast sets it to 1 --> this should have reset tracker on layer 1,
but because it is less than current max (2), it is skipped.
* thank you CodeRabbit
* force update on start
Need to re-visit the bind lock scope and maybe make the codec/mime
atomic and access them without bind lock. But, doing a whack-a-mole a
bit first to move things forward. Will look at making them atomics.
* De-centralize some configs to where they are used.
And make default variables.
Renaming a bit, but these are all internal config and have not been
added to documented config.
* Keep documented config as is.
* test
* typo
* Validate RTP packets.
Check version, payload type (if available) and SSRC (if available)
and drop bad packets. And let repair mechanisms take effect for those
packets.
* address data race reported by test
* fix an unlock and test packets
Previously, the bit rate interval config was checked first. That would
have returned `!ok` for invalid layers. A recent change to prevent
duplicate tracker addition re-arranged the code and the tracker array
was accessed out-of-bounds.
Unclear why an invalid layer is passed in. Need to investigate that.
Was hitting the edge case mentioned in the (now deleted in this PR)
comments. It is fine to reset and let it declare available again.
Available layer handler will ignore repeats.
When relaying buffers are stopped and restarted. On a restart,
the buffer adds a tracker. But, the tracker is not destroyed till the
end. So, the old tracker and new tracker for the same layer stomp on
each other and declare layer unavailable (the old tracker is not getting
any packets).
Fix by not creating a new tracker if one exists already.
* Move caching of publisher sender report to subscriber side.
Please see inline for descriptive comments on why. Basically,
pause/unpause using replaceTrack(null)/replaceTrack(actualTrack) can
cause time stamp in sender report sent to subscribers jump ahead.
This prevents that.
With the caching on subscriber side, cleaning up the caching on
publisher side.
* fix compile, test still failing, need to debug
* skip reference TS for testing
* Forward publisher sender report.
Publisher side RTCP sernfer report is rebased to SFU time base
and used to send sender rerport to subscriber.
Will wait to merge till previous versions are out as this will require a
bunch of testing.
* - Add rebased report drift
- update protocol dep
- fix path change check, it has to check against delta of propagation
delay and not propagation delay as the two side clocks could be way
off.
* Consolidate TrackInfo.
TrackInfo was spread across a bit. Consolidating it.
* TODO comments
* test
* update TrackInfo on SSRC change
* further consolidation
* log mimes only
* update receivers on SSRC set
* clone proto on return
* feedback: break loop on mime match
* prevent data race
* Prevent out-of-bounds access.
Don't know which codec causes a spatial layer three access.
Returning nil and also logging so that we know the trackID of offending
track.
* spelling
SVC has only one stream and when calculating reference time stamp,
irrespective of reference layer, reference time stamp will be the
same as the given time stamp as there is only one stream and no offset.
TODO: Need better all around SVC handling.
* Use 32-bit time stamp to get reference time stamp on a switch.
With relay and dyncast and migration, it is possible that different
layers of a simulcast get out of sync in terms of extended type,
i. e. layer 0 could keep running and its timestamp could have
wrapped around and bumped the extended timestamp. But, another layer
could start and stop.
One possible solution is sending the extended timestamp across relay.
But, that breaks down during migration if publisher has started afresh.
Subscriber could still be using extended range.
So, use 32-bit timestamp to infer reference timestamp and patch it with
expected extended time stamp to derive the extended reference.
* use calculated value
* make it test friendly
Seeing a good chunk of logs using default offset.
And it is concentrated heavily on few tracks.
Logging more to understand this better before
potentially demoting this log.
* WIP commit
* WIP commit
* WIP commit
* Some clean up
- Removed a chatty debug log
- some spelling, punctuation correction in comments
- missed an `Abs` in check, add it.
It is possible that publisher paces the media.
So, RTCP sender report from publisher could be ahead of
what is being fowarded by a good amount (have seen up to 2 seconds
ahead). Using the forwarded time stamp for RTCP sender report
in the down stream leads to jumps back and forth in the down track
RTCP sender report.
So, look at the publisher's RTCP sender report to check for it being
ahead and use the publisher rate as a guide.
* Experimental flag to try time stamp adjustment to control drift.
There is a config to enable this.
Using a PID controller to try and keep the sample rate at expected
value. Need to be seen if this works well. Adjustment are limited
to 25 ms max at a time to ensure there are no large jumps.
And it is applied when doing RTCP sender report which happens
once in 5 seconds currently for both audio and video tracks.
A nice introduction to PID controllers - https://alphaville.github.io/qub/pid-101/#/
Implementation borrowed from - https://github.com/pms67/PID
A few things TODO
1. PID controller tuning is a process. Have picked values from test from
that implementation above. May not be the best. Need to try.
2. Can potentially run this more often. Rather than running it only when
running RTCP sender report (which is once in 5 seconds now), can
potentially run it every second and limit the amount of change to
something like 10 ms max.
* remove unused variable
* debug log a bit more
* Keep track of expected RTP time stamp and control drift.
- Use monotonic clock in RTCP Sender Report and packet times
- Keep the time stamp close to expected time stamp on layer/SSRC
switches
* clean up
* fix test compile
* more test compile failures
* anticipatory clean up
* further clean up
* add received sender report logging
* Keep track of expected RTP time stamp and control drift.
- Use monotonic clock in RTCP Sender Report and packet times
- Keep the time stamp close to expected time stamp on layer/SSRC
switches
* clean up
* fix test compile
* more test compile failures
A few more candidates to think about demoting
- Publisher mute changes
- Forwarder -> layer lock/upgrade/downgrade/overshoot adjusting
- StreamAllocator
A few things
1. Have to use expected layer in upstream distance to desired. Using
min(published, expected) means if expected is higher than published, it was not caught as a missed layer.
2. Forgot to remove layer transition update in one place. It was still constrained to screen share.
This caused quality to not pick up after constraint is released.
3. Switching to max layer cannot be marked on max published. Same as point #1 above. Otherwise,
dynacast would kick in and turn off highest layer.
Raja Subramanian