A client sets SimulcastCodec.VideoLayerMode on AddTrack. Any value other
than MODE_UNUSED is used as-is, so a forged out-of-range value flows into
the forwarder. Reject it in ValidVideoLayers, alongside the quality check.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Pins the transport-v4 branch of mediatransportutil at 03518fb, which
adds mediatransportutil#103 on top of #101.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* service: send queued webhooks before shutdown completes
Start returned, and the process exited, while the room_finished webhooks
of the rooms closed by the shutdown were still queued. Stop the telemetry
service at the end of Start so it flushes its queue and drains the webhook
notifier, waiting up to 5s.
* Bump protocol to pick up the ResourceURLNotifier drain fix
ResourceURLNotifier.Stop(false) now waits for the queued events to be sent (livekit/protocol#1901), which the shutdown change relies on.
* telemetry: keep the analytics of webhooks sent during stop
The notifier's processed hook queues a WEBHOOK analytics event for each webhook it sends, but Stop stopped the jobs queue before draining the notifier, so those events were dropped. Run the queued events with a barrier, drain the notifier, then stop the queue.
* Pass CPU stats options through prometheus.Init
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add node_stats.host_cpu config to measure whole-host CPU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Bump protocol to ignore the CPU request with host CPU stats
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On mute the forwarder resyncs sequence numbers, timestamps and VP8 picture
IDs, but DDForwarder kept its frame number offset. libwebrtc counts DD
frames across all simulcast layers, so after a long mute the frame number
sent on unmute jumped by more than half the 16 bit range. The subscriber
took every frame as older than the last decoded one and dropped it, a
black tile until the numbers wrapped past it again.
Resync DDForwarder with the forwarder, and move the numbers to just after
the last one sent when they jump too far ahead.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Throttle PLIs with an atomic before taking the rtpstats lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reset the PLI throttle when the stream restarts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
GetClientIP returned the whole X-Forwarded-For header, so a request that
passed through more than one proxy set ClientInfo.Address to a list like
"198.51.100.2, 10.0.0.5" instead of an IP. Each proxy appends to the
header, so the client is the first entry.
Refresh tokens were signed with the first configured key pair, which is
an arbitrary map entry when several keys are configured. Keep the join
token's API key on the participant and sign with it when its secret is
still configured, falling back to the first key pair otherwise.
The server sets the spatial layer, rid, ssrc and repair ssrc of each
video layer from SDP and RTP. Clear these fields on AddTrack request
layers, including each simulcast codec's layers.
Clamp the layer in ReceiverBase.SetMaxExpectedSpatialLayer to
[InvalidLayerSpatial, DefaultMaxLayerSpatial].
Keep a client set video layer mode, but warn when it differs from the
server determined mode.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* config: reject out-of-range values in generated numeric flags
Generated flags for uint8, uint16, uint32, int32 and float32 config
fields were parsed at a wider type and then converted with
reflect.Value.Convert, which wraps silently. For example
LIVEKIT_TURN_RELAY_RANGE_START=70000 became 4464, and
LIVEKIT_LIMIT_MAX_METADATA_SIZE=4294967296 became 0, which turns the
limit off. The YAML config rejects the same values.
Use the cli flag type that matches each field's width so the parse
fails with the env var or flag name in the error.
* config: cover uint, int8 and int16 in generated numeric flags
Move the numeric cases into numericCLIFlag and add Int8Flag, Int16Flag and UintFlag, so every numeric kind gets a flag of its own width even though no config field uses these three yet. TestNumericCLIFlagWidths checks each kind directly with an in-range value and one that does not fit.
* config: return an error from numericCLIFlag for unsupported kinds
numericCLIFlag returned nil when no case matched, and GenerateCLIFlags appended it unchecked. It now returns (cli.Flag, error) and the caller returns the error like the default case does.
Pins the transport-v4 branch of mediatransportutil (dd8e31a plus
mediatransportutil#101), because master moved to pion/transport/v5,
which the warp forks of ice, webrtc and dtls do not support yet.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The test mutates info2 (room "another-egress-test") to COMPLETE with an
EndedAt 24h in the past, then persists `info` instead of `info2`. So info2
stays STARTING forever, CleanEndedEgress never removes it, and the record
leaks into whatever Redis the suite ran against.
The leak is invisible because the final assertion only lists the primary
room. It surfaces on the NEXT run: utils_test.go:49 prefers a reachable
localhost:6379 over an ephemeral dockertest container, so against a
persistent Redis the list-all assertion grows by one record per run and
TestEgressStore starts failing with "should have 2 item(s), but has 3".
With info2 persisted, CleanEndedEgress removes both records and the test
cleans up after itself. Verified: 3 consecutive runs green, zero keys left.
With auto_create on, CreateDispatch creates the room through router.CreateRoom without checking limits.max_room_name_length, so it could create a room that CreateRoom and the join paths reject.
* sfu: replace forward scheduling delay with processing per subscriber
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: describe processing per subscriber as per write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* telemetry: name forward processing metrics per write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin protocol with NodeStats forward_processing_per_write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- pkg/service/redisstore.go: after the retry loop, return an error wrapping redis.TxFailedErr instead of nil. CreateIngress now fails instead of returning an ingress that was never written, StoreIngress stops before creating the orphan state key, and a dropped UpdateIngressState reaches the caller's existing error log.
- pkg/service/ingress.go: the Fix nil pointer dereference in IOInfoService.UpdateIngressState on concurrent DeleteIngress #4806 guard applied to the two Twirp handlers, treating a nil State as empty. UpdateIngress then applies and stores the request; DeleteIngress deletes without calling the ingress service.
- pkg/service/redisstore.go: UpdateIngress now fails with ErrIngressNotFound when the ingress no longer exists, checked inside the WATCH on the ingress hash so a later delete fails the EXEC. Without it an update racing a DeleteIngress recreated the deleted ingress and its stream key; this was possible before this PR whenever the delete landed after LoadIngress, and the guard in 2 widened the window to the gap between its two reads.
The guard stays at the call sites to match #4806 and keep API responses unchanged.
* chore: add AttachJob interface on Worker
allows for a worker to take ownership of an existing job
* ensure we can handle stateless jobs
* set initial state in AssignJob
Servers that embed pkg/service and run their own join paths need the
same join limit check. Rename checkConnectLimits to CheckConnectLimits.
No behavior change.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* service: enforce participant name length on join
StartSession checked the token's metadata and attributes against the configured limits but not its name, so a participant could join with a name longer than limits.max_participant_name_length, which UpdateParticipant and the signal UpdateMetadata path reject.
* service: consolidate join grant-limit checks on the validate path
Extract checkGrantLimits and run it from ValidateConnectRequest and
WHIPService.validateCreate so the /rtc/validate endpoint and the WHIP
ingest path reject over-limit name, metadata and attributes the same way
a real join does. StartSession keeps its own checks as the backstop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* service: run the full join limit check in StartSession
StartSession now calls checkConnectLimits, so the RTC node checks
identity, room name, participant name, metadata and attributes itself.
ValidateConnectRequest runs the check after the publish suffix, so it
measures the same identity StartSession sees. checkConnectLimits returns
a plain error, and the HTTP callers map it to 400. The name-only
StartSession test is dropped for TestConnectLimits, which gains a
publish-suffix case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* service: test the StartSession limit check directly
TestStartSessionConnectLimits runs the same over-limit cases as
TestConnectLimits through StartSession on the RTC node. The cases move
into an exported ConnectLimitCases table so both test packages share it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: ANDROIDHASSAN <84630851+ANDROIDHASSAN@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* sfu: report forward scheduling delay next to forwarding latency
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: keep full forwarding transit when packing scheduling delay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: saturate reported forwarding p90s instead of wrapping
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The WHIP create handler never called ValidateCreateRoom, so with room.auto_create: false a publisher token without roomCreate could create any room over WHIP, while the same join over /rtc gets 404.
* sfu: cap retransmissions per down track (rtc.rtx config, off by default)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: rtx budget review fixes (sequencer undo, sent bytes, newest first, rtx_budget)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: rtx budget second review (order unchanged when off, keep NACK window, metric in down track)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* sfu: rtx budget sorts NACKs newest first by extended sequence number
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: wait for the recorded data track subscription before revoking in test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: drop a subscription whose canSubscribe was revoked while it was in flight
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Thread the issuing API key from the /rtc join path to the media node via
StartSession, and include it in the "starting RTC session" log and the
twirp API-request log, so participant activity can be correlated with the
key. Logs the key id only, never the secret.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* rtc: measure subscribe pacing delay in shadow
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* prometheus: raise the subscribe pace shadow top bucket to 120 s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: pace video subscribes per node in shadow, record the wait only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: subscribe pacer doc wording, make the wiring test independent of timing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: sample shadow pacing at arrival, skip one-shot, keep sub-millisecond waits
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: take the shadow pacing token before setup, in arrival order
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* rtc: build shadow pacing buckets in one pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add upstream FlexFEC recovery support
* Guard short TWCC extensions in FEC packets
* Release completed FlexFEC states
* Validate FlexFEC payload type range
* Invoke FEC callbacks after unlocking
* Reduce FlexFEC protection state allocations
* Avoid allocations for complete FlexFEC packets
* Optimize ordered FlexFEC packet insertion
* Reuse RTP packet storage for FlexFEC recovery
* Limit pending FlexFEC repair packets
* Lower FlexFEC repair window to two seconds
* Harden upstream FlexFEC handling
* Test FlexFEC with E2EE media
* Fix FlexFEC negotiation with libwebrtc publishers
* Enhance payload length validation in decoder
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* remove the ability to configure payload type for FlexFEC
* moved some helpers into protocol
* Handle big gaps in media sequence numbers
Reset buffers when a big gap in media sequence numbers is detected.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* bump protocol & psrpc deps
* Handle FEC packet recovery failure
Increment FEC packets discarded counter and remove packet.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* do not NACK on fec recovered packet (#4896)
* do not NACK on fec recovered packet
* fix test race in CI
* Fix FEC write return after merging RTX feedback changes
* PR fixes for FlexFEC implementation (#4898)
* do not NACK on fec recovered packet
* refactor common code into unlockAndInvokeFECRecovery
* add FEC stats to RTPStats
* add log for when fecdecoder unavailable after creation
* rename markAsFECRepair to setFECRepairBufferLimit for clarity
* refactor packet generation for tests into util
* fix test race in CI
* move the unlocks back inline for readability
* update RTPstats
* Fix FEC write return after merging RTX feedback changes
* Use shared RTP packet helper in merged FEC NACK test
* Remove redundant return from FEC write handler
* packet post process (#5020)
* packet post process
* remove redundant writeRTX
* send out RTCP (NACK and RR) from packetPostProcess only
* handle FEC decoder on receiver restart (#5032)
* handle FEC decoder on receiver restart
* move onStreamRestartLocked into BufferBaseParams
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Raja Subramanian <raja.gobi@tutanota.com>
Pick up protocol#1879 (round-trip 7s, default response 15s) so a
server-initiated PerformRpc tolerates the same ack and response windows
as the client SDK caller.
Also log an RPC response the v1 caller cannot decode (compressed_payload
or data-stream) instead of dropping it, which otherwise surfaces only as
a response timeout.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
TestDownTrackResumesAfterBandwidthPause called logger.InitFromConfig
(SetLogger) mid-run while the hwstats CPU monitor goroutine, started by
prometheus.Init in forwardstats_test, read the same non-atomic logger
package globals. go test -race flagged the write-vs-read and failed CI.
Set the global logger once in TestMain, before any test starts that
goroutine, and drop the per-test InitFromConfig.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
