RED depacketizing now lives in mediatransportutil so the SFU and
replay export share one implementation. Drop the local copies; the
encoder round-trip tests now exercise the shared package.
Callers passed mono.Now() as a time.Time, which made it easy to pass
time.Now() by mistake and mix clocks. Take mono nanoseconds instead.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
LastSenderReportTime() returned time.Unix(0, srNewest.At). But At is a
mono timestamp, so the result is not a real wall time. Connection quality
compares it to time.Now() to decide if RTCP is stale. If the wall clock
drifts from mono, for example after an NTP step, streams can look stale
when they are not, or the reverse.
- Convert At to wall time: now minus the mono time since the report.
- Return zero time when At is 0, so the scorer skips the stale check
(before, it got the 1970 epoch and saw the stream as stale).
Follow-up to #5005, which fixed the same mix-up in DLSR.
Tests: rtpstats and connectionquality pass. No new test, because a
test process cannot make the mono and wall clocks drift.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: apply simulcast rids from SDP for WHIP (one-shot) publishers
In one-shot signalling mode HandleRemoteDescription set the remote offer
directly and never called OnSetRemoteDescriptionOffer, so
updateRidsFromSDP did not run for WHIP participants. Every simulcast layer
then mapped to spatial layer 0 and all but the first were rejected with
'duplicate layer'.
* rtc: test that a one-shot simulcast offer applies rids to the pending track
* rtc: use h/q rids in one-shot simulcast test, cover single-layer offer
Name the simulcast test layers medium/h and low/q per convention, and add
TestOneShotSingleLayerOfferUnchanged to check a one-shot offer without rids
still yields a single layer with rid "" at spatial layer 0.
* agent: fix flaky TestAgent job dispatch
Retry JobRequest until the worker subscribes the job topic, because
registration is async and the topic is subscribed after the register response.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* agent: report last JobRequest error on dispatch timeout
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: reclaim abandoned pending tracks
A pending track is freed only when its first media arrives or the
participant leaves. A client that calls AddTrack and then gives up
before any media flows leaves the slot held for the whole session.
After 20 such slots, every later AddTrack is rejected.
Reclaim these slots when the cap is hit, if no media arrived and the
slot is older than the publish timeout. Keep the supervisor monitor in
step for queued tracks that share a track id. Add a publish_timeout
config value, default 30s.
Fixes: #4970
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* rtc: fix publish_timeout comment to not name an internal constant
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* rtc: release supervisor-timed-out pending tracks at the limit
Replace the createdAt sweep with the supervisor's publish timeout. The
supervisor counts time only while the publisher is connected and the
track is unmuted, and never reports a live track, so a successor queued
behind a live track is not released.
Restart the publish timer when a queued publish reaches the front of
the queue, so it does not inherit the clock of the track ahead of it.
Write the publication error under the write lock, since it is now read
from another goroutine.
Drop the publish_timeout config value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: skip stale supervisor timeouts and keep live monitors
TimedOutPublications re-checks the monitor, because the cached error is
cleared only on the next tick. A track that published since is not
reported, so its queued successor is not released.
RemovePublication keeps a monitor that still holds a published track.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: drop late media for released pending tracks
Media or an offer with no matching pending track falls back to the first
pending track of the same type. After a release, late media for the
released track could take another pending track's identity.
Record the released track's signal and SDP client ids, the 32 most
recent, and drop media or offers for them. An AddTrack that reuses the
client id clears it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: clear all ids of a released track when it is re-added
A re-add cleared only the signal client id, so a Firefox republish that
keeps the old SDP id had its offer and media dropped. Keep the ids of
each released track together, and clear the whole track when a new
AddTrack reuses its signal or simulcast codec client id. The limit now
counts released tracks, and a track ages out with all of its ids.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
An unnamed deployment was addressed as "default" and that spelling leaked
into scope keys, so embedders keyed endpoints differently from job dispatch.
Map "_" back to "" at the URL boundary, as the agent name already does,
and drop NormalizeDeployment.
LoadIngress reads the ingress info and its state from two separate Redis
keys non-atomically, while DeleteIngress removes both keys in a single
transaction. If the delete lands between the two reads, LoadIngress
returns an IngressInfo with a nil State and no error, and
UpdateIngressState panics dereferencing info.State.Status — taking down
the whole server with every room it hosts.
The race fires in practice when an ingress session is being torn down by
DeleteIngress and the ingress service reports the session state change
concurrently: with short-lived per-room WHIP ingresses this crashed
production servers within minutes (observed on v1.13.5 and v1.13.6,
SIGSEGV at ioservice_ingress.go:82, addr=0x8).
Guard the nil State by treating it as an empty state, matching how the
store's UpdateIngressState already tolerates a missing state key.
* turn: refuse TCP relays and limit relays to this host to media ports
Embedded TURN let any participant relay to arbitrary peers (#4924):
- RFC 6062 TCP relays, accepted since pion/turn v5, let a client open TCP
connections to any permitted address, e.g. SSH on the node itself.
WebRTC clients never request them, so they are now refused for both
embedded and standalone TURN.
- Relays to the host's own addresses are delivered locally and bypass any
firewall in front of the host. Embedded TURN now only relays to this
host's RTC media UDP ports.
- Carrier-grade NAT, NAT64 and a few other special-purpose ranges are
added to the restricted peer ranges.
The issue also suggested allowing relays only to the media node's IPs.
That is not done here because, with the changes above, the remaining
relays do not form a security concern: they are UDP only, to public
addresses outside the restricted ranges, and reach a peer exactly as the
client could from its own address (no amplification, no spoofed source).
This is standard TURN behavior. Limiting destinations to media nodes would
also require TURN server to know media node all endpoints, for little benefit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* solve comment
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: filter VP9 and AV1 simulcast temporal layers with the dependency descriptor
Every simulcast codec gets an explicit temporal layer selector. With the
dependency descriptor, VP8, VP9 and AV1 filter temporal layers by frame and
forward the descriptor. Without it, a pass-through selector forwards every
temporal layer and reports the target as reached, so a pending target cannot
block allocation. VP8DD becomes the codec-neutral DDTemporal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: VP8-only padding header in DDTemporal, structure gate on presence
DDTemporal returns the one byte VP8 payload descriptor for padding only for
VP8. The descriptor forwarder takes a structure from any packet that carries
one, as the parser accepts it only on the first packet of a frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers across migration
The forwarder state carries the last dependency descriptor frame number
sent, so a migrated subscriber's first key frame continues the numbers
instead of being dropped as older than its last structure. Bumps protocol
for RTPForwarderState.dependency_descriptor_forwarder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: pass on a pending descriptor seed when migrating again
A forwarder seeded before its codec was known returns the pending seed in
its state, so a second migration keeps the frame numbers continuous.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: forward the dependency descriptor for VP8 simulcast
libwebrtc sends VP8 with a one byte payload descriptor when it sends the
dependency descriptor, so a VP8 subscriber gets neither picture IDs nor the
descriptor and finds frame references by sequence number alone. Under loss
with temporal filtering it stalls until a key frame.
DDForwarder rewrites the publisher's descriptor for each forwarded packet:
frame numbers continue across simulcast layer switches and stream restarts,
the structure goes out with the current layer's key frame, and the active
decode targets follow the forwarded temporal layer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep VP8 descriptor frame numbers from going back
Rebase frame numbers only when a new stream would go back past the last
number sent: on a layer switch, a stream restart (also when it starts with a
delta frame) and a new forwarder on the same downtrack, which now carries
over the last number sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: set VP8 descriptor decode targets from the packet's layer
The active decode targets use the temporal layer the packet is forwarded at,
not the selector's next layer on the last packet before a down switch. They
also go out on key frames, combined with the publisher's own mask.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: carry the current active decode targets on every DD packet
The parser puts the stream's current mask on every packet, so a mask change
on a dropped packet still reaches the VP8 descriptor forwarder, which no
longer tracks the publisher's mask itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: take the first DD active decode targets mask at any sequence number
A publisher whose first packet has RTP sequence number 0 never had its key
frame's mask applied, so every packet reported no active decode targets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A filtered packet that follows a sequence number gap now keeps its slot
and gets padding in it. Rolling it back moved the last outgoing number
below the missing packets' slots, so their retransmissions were rejected
and the subscriber stalled on the hole until a key frame.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: parse VP8 and VP9 payload descriptors without an allocation
Replace ExtPacket.Payload with typed VP8 and VP9 pointer fields that point
at backing structs inside the ExtPacket, so parsing does not allocate and a
wrong codec type is a compile error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: copy the VP8 descriptor in the test packet helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The key frame requester re-armed its timer on every loop, so every video down
track woke up every 200 ms to 1 s for its whole life, even when the layer was
locked and there was nothing to do.
Re-arm the timer only while the requested layer is not locked, or while the
lock can change without an event: an acquisition grace is armed, or layers are
selected with the dependency descriptor, which can lose the lock on the packet
path. Post the event on resync, mute, max published layer change and packet
path layer switch, since those can change the lock without an allocation.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On a gap the RTP munger gives the missing sequence numbers outgoing slots.
If a late packet then arrives and is dropped, nothing is ever sent in its
slot, and a subscriber that finds frame references by sequence number
stalls until a key frame. The forwarder now flags such drops, and the
downtrack sends a one byte padding only packet in the slot.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: filter VP8 temporal layers by frame when the dependency descriptor is sent
With the dependency descriptor, libwebrtc sends a one byte VP8 payload
descriptor without a picture ID, so the VP8 munger's picture ID based
temporal drop logic stops filtering after the first gap. A new VP8DD codec
munger decides once per frame from the descriptor's frame number and
temporal ID and applies that decision to every packet of the frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: use the VP8 munger when picture IDs come with the dependency descriptor
VP8DD needs the one byte VP8 payload descriptor that libwebrtc sends with
the dependency descriptor. If a publisher keeps picture IDs next to the
descriptor, they need rewriting, so the forwarder switches to the VP8
munger and keeps switching temporal layers with the descriptor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
GetRecentlyDisconnectedParticipants walked the participant update cache
with Get, which moves each entry to the front of the LRU. Walking from
newest to oldest left the cache in reverse order, so after a reconnect
the next updates evicted the most recently updated participants first.
In a room where the subscriber has tracked 128 or more participants, a
participant that had just disconnected was then dropped from the cache
and a later reconnect did not report it as disconnected, and the
out-of-order version check was lost for recently updated participants.
Use Peek, which reads an entry without changing its recency.
libwebrtc strips the VP8 payload descriptor when it sends the dependency
descriptor, so the layer sync bit that #4947 waits for is never set and a
subscriber that lowers its frame rate stays at the base layer until a key
frame. A new temporal layer selector reads only the dependency descriptor:
it switches up at a switch point whose referenced frames were all
forwarded. The forwarder uses it for VP8 when the extension is negotiated
and falls back to the Y-bit selector when the descriptor is not sent.
Fixes#4973.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Firefox 155 enabled AV1 for WebRTC by default, but Firefox 155/156
decode only the base spatial layer of an AV1 SVC stream. SVC is not
visible in SDP, so such a subscriber that negotiates AV1 is forwarded
the publisher's AV1 SVC stream instead of its backup codec, rendering
black whenever a higher layer is selected. Firefox 157 decodes the
higher layers correctly (client-sdk-js#2116), so it keeps AV1.
Removing AV1 from the subscriber's enabled codecs keeps it out of the
SFU's offers and answers, so the down track binds to the backup codec.
Firefox never publishes AV1 either, so it is disabled in both
directions, the same as Safari.
The Firefox for Android H.264 rule is switched to merge so it no longer
drops the AV1 rule for that browser.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* sfu/buffer: parse the dependency descriptor without allocating
Every VP9 and AV1 SVC packet allocated six objects in the dependency
descriptor parse: the descriptor, a clone of the frame template with
its three slices, a resolution copy, and a frame entity that took the
address of a parameter.
The pooled ExtDependencyDescriptor now owns the storage for the
descriptor, the template and the resolution, with fixed arrays for the
template's slices. The reader copies the matching template into that
storage.
Delta frame parse: 6 allocs, 176 B, 174 ns -> 0 allocs, 0 B, 122 ns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/dd: never copy a parsed frame into a structure template
A caller can hand Unmarshal a descriptor whose FrameDependencies points
at one of the structure's templates. Copying the packet's custom fields
into it would change the template for every later packet, so the reader
allocates a fresh template in that case.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
A large frame number jump made addEntity walk every skipped frame,
twice. The ring holds only numEntries frames, so older ones get
overwritten anyway. Both loops now start at most numEntries behind
the new frame, like PacketHistory and FrameIntegrityChecker.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* sfu/buffer: stop allocating per ingested packet
Buffer.Write allocated six objects for every RTP packet. They were
the parsed rtp.Packet, a copy of it in patchExtPacket, and the flow
state, whose address two rare log calls took. The rest were an
AbsCaptureTime, present on about one packet per second, and the
extension slice that pion grows with append.
Unmarshal into a pooled scratch packet. Give ExtPacket its own backing
rtp.Packet and an [8]rtp.Extension array, and copy the header into
them. Patch the payload in place. If the abs-capture-time extension is
present, parse it. Log a copy of the flow state.
Per packet, Write plus ReadExtended. Opus: 6 allocs, 448 B -> 0.
VP8: 6 allocs, 416 B -> 1 alloc, 32 B. Opus 240 -> 142 ns, VP8 269 -> 204 ns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: point extension payloads at the bucket copy on read
Queued ExtPackets kept extension payload slices into the write input,
which the transport reuses for the next packet. ReadExtended now parses
the header again from the bucket copy, so extensions and CSRCs point
into RawPacket like the payload does.
The test writes every packet through one input buffer and checks the
audio level of each packet read back.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu/buffer: test that a dropped padding packet keeps the shifted sequence number on read
The bucket stores the shifted number, so re-parsing the header on read
returns it. This test guards that.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The static client config disabled H.264 publish for Firefox on Linux or
Android. The Android workaround came from #2183, but the linux clause
only ever hit desktop Linux Firefox, which publishes H.264 fine through
OpenH264. Narrow the rule to Android only.
Fixes#4929
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* bump protocol to get unlock on no key fix
* update warp pointer
* update protocol and psrpc deps
* update protocol dep
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
A subscribe that found the reconcile queue full was dropped and waited
for the 3 s ticker. Flag the overflow, and have the worker reconcile
all subscriptions after its next queued item.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
