* Adjust stream allocator ping interval based on state.
In steady state, does a 15 second ping.
While deficient, to be able to react to probes faster, it pings at 100ms
interval.
* clean up
* log ops queue not able to wake up
* Tweaks tresholds for logging high forwarding latency/jitter.
Previous attempt showed skewed jitter (i. e. more than 10x latency),
But, no large latency.
So, reducing the latency treshold to declare high latency.
And also keeping track of lowest/highest per reporting window and
logging those along with short term and long term measurements.
NOTE: previously short term and long term were separate calls with locks
acquired. Now, it is all in one lock. So, it does increase the lock
duration a bit, but hopefully not by too much as the welford merge for
short term would go over 20 samples (at 50 ms sampling interval and 1 s
reporting window).
* revert skew factor
* Log some information around high forwarding latency.
Latency is not 0 after switching to microseconds resolution.
But, still seeing high jitter. Logging a bit more to understand under
what conditions it happens.
More notes inline.
* compact
Latency is always 0, but jitter is high.
Not sure how that happens as latency is the welford mean and jitter is
welford standard deviation. Feels like some mis-labeling.
Anyhow, switching to microseconds units to get better resolution.
* Add debugging from DD frame number wrap around.
On a DD parser restart, the extended highest sequence number oes not
seem to be updated. Adding some debug to understand it better.
* more logs
* log incoming sequence number and frame number
* Set publisher codec preferences after setting remote description
Munging SDP prior to setting remote description was becoming problematic
in single peer connection mode. In that mode, it is possible that a
subscribe track m-section is added which sets the fmtp of H.265 to a
value that is different from when that client publishes. That gets
locked in as negotiated codecs when pion processes remote description.
Later when the client publishes H.265, the H.265 does only partial
match. So, if we munge offer and send it to SetRemoteDescription, the
H.265 does only a partial match due to different fmtp line and that gets
put at the end of the list. So, the answer does not enforce the
preferred codec. Changing pion to put partial match up front is more
risky given other projects. So, switch codec preferences to after remote
description is set and directly operate on transceiver which is a better
place to make these changes without munging SDP.
This fixes the case of
- firefox joins first
- Chrome preferring H.265 joining next. This causes a subscribe track
m-section (for firefox's tracks) to be created first. So, the
preferred codec munging was not working. Works after this change.
* clean up
* mage generate
* test
* clean up
- Move downTrack instantiation to SubscribedTrack as it should own that
DownTrack. Still more to do here as `DownTrack` is fetched from
`SubscribedTrack` in a few places and used. Would like to avoid that,
but doing this initially.
- Use an interface from sfu.Downtrack and replace a bunch of callbacks.
SubscribedTrack is the implementation for DownTrackListener.
* Do not send both asb-send-time and twcc.
In single peer connection mode, both extensions are set on the media
engine and both would be negotiated. Unfortunately, pion/webrtc does
not yet support RTPSender.SetParameters() which would allow setting
specific header extensions for the sender. So, check for TWCC enabled
and use it. If not, do abs-send-time if that is enabled.
* check BWE type
* comment
* Populate SDP cid in track info when available.
- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
id for AddTrack and actual SDP offer. So, have to look up using both
on server side. To make it easier, store both (only if different) in
TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
around doing simulcast codec for audio so that something like PSTN can
use G.711 without any transcoding. So, just keep it consistent between
audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
pending track or an already published track if the new offer is for a
back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
times.
- Clean up MediaTrack interface a bit and remove unneeded methods.
* WIP
* WIP
* deps
* stream allocator mime aware
* clean up
* populate SDP cid before munging
* interface methods
Normalize the rids in SDP to known patterns.
Currently,
- LK protocol uses q;h;f
- Sean's OBS WHIP uses 0;1;2
As the ordering in SDP could be different, normalize to known order.
For RIDs not in the known set, just use it as is.
* SVC with RID -> spatial layer mapping
There are cases where an SVC track comes in with a RID.
As there is no RID announced in SDP, it maps to invalid layer.
Seems to happen with older browsers.
* test
The browser could send rtp packets of svc encoding without
DD extension while the sdp negotiates it, sfu detects extension
in rtp packet for this case.
* Add simulcast support for WHIP.
- General change to have rids be anything.
- One issue is rid ordering not matching quality ordering, will need
some dynamic layer quality determination for that.
* clean up
* deps
* test
* Add a trend check before declaring joint queuing region.
Seeing cases where the propagated queuing delay drops from one group to
next. Both groups are above threhold. It also recovers majority of the
time. So, introducing a trend check before declaring that queuing delay
is in joint queuing region. It is set 0.8 by default which means the
queueing delay should be trending up strongly before being declared
joint queuing region.
* deps
* Send initial participant update only after a participant becomes active.
There are cases where apps send data to remote participant as soon as
client emits `ParticipantConnected`. But, that time point would not have
a fully established client (i. e. the media connection + data channel
establishment is still in progress).
This PR changes the initial participant update to be sent from server
side only when a participant becomes `ACTIVE`, i.e fully connected
(media channel established and data channels open).
It is supported for clients using protocol version > 15.
@cnderrauber bumping up the protocol version in this PR. Move support is
also conditioned on protocol version > 15, but that PR did not ump
protocol version. Please let me know if there are issues bumping
protocol version.
* check for joining states in broadcast
* have to check on other participant
* test
* make helper for sending participant updates
* test
* make utility of pushAndDeque
* test
* consolidate getting other participants
* remove extra cast
* debug
* debug
* typo
* stop transceiver that is not bound
* logs
* log
* check for ever bound
* clean up
* clean up
* Call Broadcast in lock scope.
Seems like there is a possible window where things can hang forever
if a goroutine enters the Wait) after the lock is released but before
Broadcast gets called, it will never see that broadcast and will hang forever.
* RLock
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"
This reverts commit cdfbb106d1.
* Revert "Keep simulcast information tied to receiver. (#3563)"
This reverts commit ed5e2f16b2.
* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"
This reverts commit ad010cfc43.
* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"
This reverts commit 15f565510c.
* chore(deps): update protocol
* Keep simulcast information tied to receiver.
`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.
Keep track of simulcast flag in receiver.
Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track
* clean up
* clean up
* clean up
* clean up
* test
* Store SdpCid and IsSimulcast in Trackinfo
* clean up
* mock
* fix(video): determine svc/simulcast from SDP for advanced codecs
* fix(explicit-svc): cleanup
* fix(explicit-svc): remove from list on close/remove
* fix(explicit-svc): reorder VLS selection, cleanup
* fix(explicit-svc): todo comments for temporal layer selector
* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
Seeing a lot of queuing delay based back offs. Trying a couple of things
1. Accept a bit more queuing.
2. An option to try a different pacer. Would like to try with pass
through. That will produce some out-of-order packets. Remains to be
seen if it will have a negative impact.
