* sfu: restart the dependency descriptor parser of a publisher stream after a gap
libwebrtc numbers dependency descriptor frames with one counter across all
simulcast layers. A layer that dynacast paused for a long time resumes with a
frame number more than half the 16 bit range away from its last one. The
publisher buffer's parser never restarted, so it unwrapped the resumed frames
against the stale number and later lost a full wrap. Every packet of the layer
then fell below the structure thresholds and was dropped until the numbers
caught up, about 16 minutes for three 22 fps layers.
Enable the parser restart after a 30 s gap that relayed tracks already use, and
start the key frame seeder on a restart, as the restart drops the structure the
following packets need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: request a key frame only when a restarted stream has no structure
The parser now reports the restart on the packet that caused it, so the
buffer does not read the parser's state. A stream that resumes with a key
frame carries its structure and needs no key frame request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: ignore a nil publisher sender report in the forwarder
A stream restart replaces the receiver stats between the sender report
callback and its read, so the report can be nil. SetRefSenderReport now
keeps the previous report instead of reading the nil one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* dynacast: skip a codec regression when the target codec has no quality
HandleCodecRegression looked up the target codec after it released the
lock. If the manager closed in between, or the target mime type was
unknown, the lookup returned nil and RegressTo called Restart or Replace on
it. The target is now looked up with the source, and a missing one stops
the regression.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* service: create the internal record for a stored room that has none
RedisStore.LoadRoom returns a nil RoomInternal without an error when the
room exists but its internal field does not, for example after a delete
races a store. CreateRoom then wrote egress, agent, playout delay or sync
settings into the nil record. It now starts from an empty record, which
the store call writes back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: drop a dependency descriptor whose chains do not match the structure
The parser takes a new structure from any packet, but DDForwarder keeps
the structure of the last packet it forwarded. When the publisher attached
a structure with fewer chains to a packet this downtrack dropped, the next
descriptor had fewer chain diffs than the forwarder's structure, and the
writer indexed past them. The forwarder now drops such a packet, and the
writer returns an error instead of indexing past the chain diffs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* deps: bump mediatransportutil for the empty VP9 payload fix
A VP9 packet without a dependency descriptor and with a 1-byte payload such
as 0x08 parses to an empty VP9 payload, and IsVP9KeyFrame read its first
byte. mediatransportutil#106 returns false for an empty payload. This pins
the transport-v4 commit that carries it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep the template test comment with its test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* dynacast: create the target quality only for a valid regression
A target quality starts a timer that notifies the publisher. Look it up
only after the source codec is found, so a rejected regression creates
nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* service: restore the internal defaults for a room that lost them
A stored room without an internal record now gets the configured playout
delay and sync streams defaults. Its own settings, such as the empty
timeout and codecs, stay as stored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Throttle PLIs with an atomic before taking the rtpstats lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reset the PLI throttle when the stream restarts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: replace forward scheduling delay with processing per subscriber
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: describe processing per subscriber as per write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* telemetry: name forward processing metrics per write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin protocol with NodeStats forward_processing_per_write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: report forward scheduling delay next to forwarding latency
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep full forwarding transit when packing scheduling delay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: saturate reported forwarding p90s instead of wrapping
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu/buffer: stop allocating per ingested packet
Buffer.Write allocated six objects for every RTP packet. They were
the parsed rtp.Packet, a copy of it in patchExtPacket, and the flow
state, whose address two rare log calls took. The rest were an
AbsCaptureTime, present on about one packet per second, and the
extension slice that pion grows with append.
Unmarshal into a pooled scratch packet. Give ExtPacket its own backing
rtp.Packet and an [8]rtp.Extension array, and copy the header into
them. Patch the payload in place. If the abs-capture-time extension is
present, parse it. Log a copy of the flow state.
Per packet, Write plus ReadExtended. Opus: 6 allocs, 448 B -> 0.
VP8: 6 allocs, 416 B -> 1 alloc, 32 B. Opus 240 -> 142 ns, VP8 269 -> 204 ns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: point extension payloads at the bucket copy on read
Queued ExtPackets kept extension payload slices into the write input,
which the transport reuses for the next packet. ReadExtended now parses
the header again from the bucket copy, so extensions and CSRCs point
into RawPacket like the payload does.
The test writes every packet through one input buffer and checks the
audio level of each packet read back.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: test that a dropped padding packet keeps the shifted sequence number on read
The bucket stores the shifted number, so re-parsing the header on read
returns it. This test guards that.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Do not restart receiver on codec change mid-session.
This is not supported and was an erroneous change during the
receiver_base/buffer_base + RTP stream restart consolidation. Also make
the codec munger creation more resilient.
* fix test
* Rework receiver restart.
- Protect against concurrent restarts
- Clean up and consolidate code around restarts
- Use `RestartStream` of buffer rather than creating new buffers.
* fix test
This is not all of it as it is not possible (or at least I do not know
of a way) to get all suggestions for a repo/project. Did this via loop
searching mainly and taking the modernize suggestions.
* Use sync.Pool for objects in packet path.
Seeing cases of forwarding latency spikes that aling with GC.
This might be a bit overkill, but using sync.Pool for small +
short-lived objects in packet path.
Before this, all these were increasing in alloc_space heap profile
samples over time. With these, there is no increase (actually the lines
corresponding to geting from pool does not even show up in heap
accounting when doing `list` in `pprof`)
* merge
* Paul feedback
* Remove alloc in packet forwarding path.
Unlikely logger creation was doing allocs. Replace it with a function
like in rtpstats_receiver.go so that allocations do not happen
unnecessarily.
* variable rename
* one more place
* Splitting out rtp stats stuff into its own package.
Going to be making some lighter versions of these.
Will be cleaner to have all of these grouped together.
So, as a first step, just making a package for it.
* tests
* Validate RTP packets.
Check version, payload type (if available) and SSRC (if available)
and drop bad packets. And let repair mechanisms take effect for those
packets.
* address data race reported by test
* fix an unlock and test packets
* Disable audio loss proxying.
Added a config which is off by default.
With audio NACKs, that is the preferred repair mechanism.
With RED, repair is built in via packet redundancy to recover from
isolated losses.
So, proxying is not required. But, leaving it in there with a config
that is disabled by default.
* fix test
* Prevent rtx buffer and forwarding path colliding
Received packets are put into RTX buffer which is
a circular buffer and the packet (sequence number) is
queued for forwarding. If the RTX buffer fills up
and cycles before forwarding happens, forwarding
would pick the wrong packet (as it is holding a
reference to a byte slice in the RTX buffer) to forward.
Prevent it by moving reading from RTX buffer just
before forwarding. Adds an extra copy from RTX buffer
-> temp buffer for forwarding, but ensures that forwarding
buffer is not used by another go routine.
* Revert some changes from previous commit
Details:
- Do all forward processing as before.
- One difference is not load raw packet into ExtPacket.
- Load raw packet into provided buffer when module that reads
using ReadExtended calls that function. If the packet is
not there in the retransmission buffer, that packet will be
dropped. This is the case we are trying to fix, i. e. the RTX
buffer has cycled before ReadExtended could pull the packet.
This makes a copy into the provided buffer so that the data
does not change underneath.
* Remove debug comment
* Oops missed a function call
* WIP commit
* fix test
* More clean up
* cast to right size
* use local variable
* set a default RTT
* Do not log RTX, although need to figure out source of RTX
* fix test
* Cleaning/simplifying some buffer bits
1. NACKs are always inserted in order. So, get rid of
bunch of out-of-order handling in there and simplify.
2. For now, removing triggering a key frame from NACKs.
Let subs drive it.
3. Move to 16-bit sequence numbers except for receiver
report handling. Simplify bits about unwrapping sequence
number on all packets.
4. Remove unused code.
* remove unused field
* Handle multiple codecs in renegotiation
update pion to v3.1.9 for answer same order of codec as publisher.
register enable codecs in subscriber peerconnectin created.
add codec parameter to buffer.bind
buffer should use the codec of TrackRemote as it's codec mime.
sent h264blankframe when DownTrack closing
* Separate from ion-sfu
changes:
1. extract pkg/buffer, twcc, sfu, relay, stats, logger
2. to solve cycle import, move ion-sfu/pkg/logger to pkg/sfu/logger
3. replace pion/ion-sfu => ./
reason: will change import pion/ion-sfu/pkg/* to livekit-server/pkg/*
after this pr merged. Just not change any code in this pr, because it
will confused with the separate code from ion-sfu in review.
* Move code from ion-sfu to pkg/sfu
* fix build error for resovle conflict
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
