mirror of
https://github.com/livekit/livekit.git
synced 2026-09-17 03:44:48 +00:00
Raja Subramanian
7b92530461
A bunch of edges to note here RED packet does not have sequence number for redundant blocks. It only has timestamp offset compared to the primary payload. The receivers are supposed to use just timestamp to sequence the payload and decode. But, when converting from RED -> Opus, the packets extracted from RED packet should be assigned a sequence number before they can be forwarded. The simple rule is, if packet N contains X redundant payloads, they are assigned sequence number of N - X to N - 1. However there are cases like the following sequence (with 1 packet redundancy) - Seq num 10, timestamp 2000, forwarded - Seq num 11 is lost - Seq num 12 has a redundant payload. Seq num 12 has timestamp of 4000. Ideally would expect the redundant payload to have a timestamp offset of 1000, so the redundant payload can be mapped to sequence number 11 and timestamp 3000 (4000 - 1000). But, in the problematic case, it has an offset of 3000 resulting in sequence number 11 and timestamp of 1000 causing an inversion with packet at sequence number 10. Unclear if this a publisher issue, i. e. packing RED wrong or if this is some expected behaviour with DTX. i. e. the DTX packets are not included in redundant payload. For example, the sequence - Seq num 10 -> DTX - Seq num 11 -> DTX -> lost - Seq num 12 -> Regular packet and include sequence num 9 as that is the last regular packet. Anyhow, detect this condition and drop the time inverted packet. Note however this handles only inversion against the highest sent packet sequence number and timestamp. So, some old packet inverted with some other old packet getting forwarded will get through. That has been the case always though and detecting that would be expensive and complicated. At least for egress, will also look at adding a check for inversion so that it can catch it before sending it down the gstreamer pipeline. As the egress uses a jitter buffer with ordered sequence number emits, it will be simpler to detect timestamp going back when sequence number is moving forward (of course the mute/dtx challenege is there).