* Use Muted in TrackInfo to propagated published track muted.
When the track is muted as a receiver is created, the receiver
potentially was not getting the muted property. That would result in
quality scorer expecting packets.
Use TrackInfo consistently for mute and apply the mute on start up of a
receiver.
* update mute of subscriptions
* fix: ensure num_participants is accurate in webhook events (#4265)
Three fixes for stale/incorrect num_participants in webhook payloads:
1. Move participant map insertion before MarkDirty in join path so
updateProto() counts the new participant.
2. Use fresh room.ToProto() for participant_joined webhook instead of
a stale snapshot captured at session start.
3. Remove direct NumParticipants-- in leave path (inconsistent with
updateProto's IsDependent check), force immediate proto update,
and wait for completion before triggering onClose callbacks.
* fix: use ToProtoConsistent for webhook events instead of forcing immediate updates
* Update go deps
Generated by renovateBot
* update api usage
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: David Zhao <dz@livekit.io>
* Log join duration.
Also revert the "unresolved" init. Defeated the purpose of log resolver
as it was resolving with those values even if not forced. Instead set it
to "unresolved" if not set when forced.
Join duration is not reset if resolver is reset as that happens on
moving a participant and there is no new join duration in that case.
* explode
Some e2e is failing due to subscriptions happening late and the expected
order of m-lines is different. Not a hard failure, but logging more to
make seeing this easie.
For a participant migrating out, the track could be resumed on a
different node, but ending on the migrating out node. So, `flush` should
be used to indicate if track is going to be resumed.
Subscription can switch between remote track and local track or
vice-versa. When that happens, closing the subscribed track of one or
the other asynchronously means the re-subscribe could race with
subscribed track closing.
Keeping the case of `isExpectedToResume` sync to prevent the race.
Would be good to support multiple subscribed tracks per subscription.
So, when subscribed track closes, subscription manager can check and
close the correct subscribed track. But, it gets complex to clearly
determine if a subccription is pending or not and other events. So,
keeping it sync.
Removing some logs which have not been useful in terms of insights other
than saying that there are a bunch of packets missing. Going to start
looking at gaps in terms of time if the inter-packet gap is too high.
Also, using logging these events as first 20 and then every 200.
A bunch of edges to note here
RED packet does not have sequence number for redundant blocks. It only
has timestamp offset compared to the primary payload. The receivers are
supposed to use just timestamp to sequence the payload and decode.
But, when converting from RED -> Opus, the packets extracted from RED
packet should be assigned a sequence number before they can be
forwarded. The simple rule is, if packet N contains X redundant
payloads, they are assigned sequence number of N - X to N - 1.
However there are cases like the following sequence (with 1 packet
redundancy)
- Seq num 10, timestamp 2000, forwarded
- Seq num 11 is lost
- Seq num 12 has a redundant payload. Seq num 12 has timestamp of 4000.
Ideally would expect the redundant payload to have a timestamp offset
of 1000, so the redundant payload can be mapped to sequence number 11
and timestamp 3000 (4000 - 1000). But, in the problematic case, it has
an offset of 3000 resulting in sequence number 11 and timestamp of
1000 causing an inversion with packet at sequence number 10.
Unclear if this a publisher issue, i. e. packing RED wrong or if this is
some expected behaviour with DTX. i. e. the DTX packets are not included
in redundant payload. For example, the sequence
- Seq num 10 -> DTX
- Seq num 11 -> DTX -> lost
- Seq num 12 -> Regular packet and include sequence num 9 as that is the
last regular packet.
Anyhow, detect this condition and drop the time inverted packet.
Note however this handles only inversion against the highest sent packet
sequence number and timestamp. So, some old packet inverted with some
other old packet getting forwarded will get through. That has been the
case always though and detecting that would be expensive and
complicated.
At least for egress, will also look at adding a check for inversion so
that it can catch it before sending it down the gstreamer pipeline. As
the egress uses a jitter buffer with ordered sequence number emits, it
will be simpler to detect timestamp going back when sequence number is
moving forward (of course the mute/dtx challenege is there).
* Log time inversion between incoming packets
Log of timestamp inversion within a red packet did not show anything.
Log across packets. Not dropping till there is more evidence of the
cause.
* save
* comment
* Guard against timestamp inversion in RED -> Opus conversion.
Seeing timestamp inversion (sequence number is +1, but timestamp is
-960, i.e. 20ms) in the RED -> Opus conversion path. Not able to spot
any bugs in code. So, logging details upon detection and also dropping
the packet. If not dropped, downstream components like Egress treat it
as a big timestamp jump (because sequence number is moving forward) and
try to adjust pts which ends up causing drops.
* do not log time reversal at the start
* typo
When a subscriber disconnects, observer closures registered on the
publisher's TrackChangedNotifier and TrackRemovedNotifier were never
removed. These closures capture the SubscriptionManager, which holds
the ParticipantImpl, preventing the entire participant object graph
(PCTransport, SDPs, RTP stats, DownTracks) from being garbage collected.
In rooms with many participants that disconnect and reconnect frequently,
this causes unbounded memory growth proportional to the number of
disconnect events. The leaked memory is not recoverable while the room
remains open.
Clear notifiers in both handleSubscribedTrackClose (individual
subscription teardown) and SubscriptionManager.Close (full participant
teardown), matching the existing cleanup in handleSourceTrackRemoved.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Close both peer connections to aid migration.
In single peer connection case, that would close publisher peer
connection.
@cnderrauber I don't remember why we only closed subscriber peer
connection. I am thinking it is okay to close both (or the publisher
peer connection in single peer connection mode). Please let me know if I
am missing something.
* log change only
And match design to RTP header extension, i. e. the padding for
extensions is not at per extension level (which was the case before),
but has been changed to padding the aggregate of all extensions in this
PR.
