Found another reason for potential stalling on layer switch up.
When switching up, it is possible that the key frame of the packet
at higher layer arrives very close to a packet in the currently
forwarded layer.
An inversion happens when higher layer packet arrives earlier
(as timestamped at receicver), but gets scheduled for forwarding
later (as seen by forwarder).
Because of the order of processing goroutines,
it is possible to have negative diff when trying to adjust
RTP timestamps for layer switch. A negative diff results in
large jump in RTP timestamp. Client stalls, sends PLI three seconds
later (3 seconds is a Chrome thing, not sure about others), waits
for another key frame and starts again. In the mean time, the
video is frozen.
* Handle an edge in layer lock.
A very edge case
- Available layer: [0, 1, 2], but bitrate is not yet available.
We set it to layer 2 awaiting measurement.
- Measurement for layers 0 and 1 come through.
- Still no key frame for layer 2.
- Finalize layers runs and sees that bitrate is available for 0 and 1.
It finalizes layer 1.
- Layer 1 key frame comes (because we asked key frame of layer 2,
publisher sends key frame for all layers). Locks to layer 1.
- No more events happen to switch to layer 2.
Changes
-------
- Move bit rate measurement to StreamTrackerManager. Allows re-use
in relay.
- Make bit rate availability (from zero -> non-zero) an event
and let it flow through the stream allocation flow so that we
always have an event when bit rate measurement becomes available.
This gets rid of finalization which I was unhappy with it anyway as
it was polling every second.
- Removing REMB stuff from buffer. We do not use it.
It is incorrect anyway. REMB should be ay peer connection level.
* Fix test
* fix test
* Simplify allocate
* Simplify/clean up
* Key frames
- Keep track of key frame stats
- Split out PLI from down track used for purpose of layer locking.
This will give us a good picture of down stream issues forcing a PLI.
- Use key frame requester whenever there is a layer lock required.
Not just the first key frame. With the synchronous thing, the counter
was just ridiculously high like 150 or something because of all
the initial padding packets. Also, use RTT in key frame requester.
* send first PLI before waiting
* Turn off key frame requester when disabled
* simplify
* Random clean up
* Split out StreamTrackerManager for re-use
* Reset the correct tracker
* use generation counter to exit coroutine
* start only for video and when enabled
* Add RemoveAllTrackers method
* Add a resync API to sfu.DownTrack
Also passing in logger with context into sfu package. More to do here
with proper logging context in all modules, but this is a start
* Remove debug code
* fix tests
* Prevent multiple resume notifications on track
When returning previous allocation as is, reset change to none.
Ideally, would like to have change as a separate return and not in last
allocation. But, also prefer to have last allocation state. For now,
resetting change.
* log bandwidth estimate only on decrease
* Fix forwarding status deduction
- When muted OR when there are no available layers, declare optimal
- When target layer is the maximum it can achieve taking available
layers into account even if they are not the maximum subscribed layer,
it is still optimal as there is nothing better available.
* Fix and add more tests for forwarding status
* WIP commit
* deficient handling
* Add missing ProvisionalAllocatePrepare
* adjust state on track removal
* Increase test timeout
* - Add comments about cooperative routines
- Take down transition if available in cooperative scheme
- Use layer comparison when taking down transition. Because of when the
bitrate is measured, it is not always guaranteed bandwidthDelta is -ve
when moving down.
- Do not add track to stream allocator till bind.
* make comment better
* a bit more clear comments
* Use OnBind on subscribed track
* Stream Allocator Try 3
Making an intermediate PR to do
- Special treatment for screen share tracks
- When allocating all tracks,
o try to stream all tracks by starting with the lowest layer
o multi-pass across tracks to get a more even distribution
Not yet done:
-------------
In deficient state,
o Allocate a specific track on a change
o Steal from other tracks
* Correct sense of managed track
* have to range to copy
* generate
* fix VideoLayers compare
* Use t.simulcasted
* WIP commit
* test padding only with a gap scenario
* Debug
* Fix video corruption, need buffer to include payload and translated header size
* Revert incorrect change
* Fix VP8 translation to return buffer with proper length
* Restore 7-bit mode
* WIP commit
* Clean up
* More clean up and tests compiling again
* Fix tests
* Make VP8 packet translation thread-safe.
Was using one packet from pool for all VP8 translation which was not thread safe.
Grab packets from the pool when needed for VP8 translation and return to pool after done.
Do not grab packet from pool if the header size between incoming and translated matches.
That also saves copying the packet payload.
* Keep Get/Put in the same function.
Raja Subramanian