* test: stop registering the global DefaultConfig with the logger
InitLoggerFromConfig hands the pointer to zaputil.ComponentLeveler as its
level resolver, and protocol a879e94 gave logger.Config a
ResolveComponentLevel that takes c.lock. The two test packages passed
&config.DefaultConfig.Logging, so every first-time component level
resolution locked a mutex inside the global DefaultConfig while
config.NewConfig marshalled that same global from another goroutine:
Read at 0x39baa08 yaml.Marshal -> pkg/config/config.go:633 (NewConfig)
Write at 0x39baa08 ComponentLeveler.resolve -> zaputil/leveler.go:117
That tripped the race detector in pkg/rtc TestPreferMediaCodecForPublisher
and broke CI on master.
Register a test-local LoggingConfig instead, so the logger never touches
the global that NewConfig reads. DefaultConfig.Logging only ever sets
PionLevel, so behavior is unchanged. A plain struct copy is not an option
here: logger.Config holds a mutex and vet's copylocks would reject it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: make RTCClient callbacks safe to set while the client runs
The scenario tests assign c2.OnDataReceived after waitUntilConnected, so
the assignment raced the data channel goroutine already reading the field
in handleDataMessage:
Read test/client.(*RTCClient).handleDataMessage client.go:1146
Write test.scenarioDataPublish.func1 scenarios.go:160
This is what failed TestMultinodeDataPublishing on master.
Replace the three exported callback fields with atomic pointers behind
SetOnConnected/SetOnDataReceived/SetOnDataUnlabeledReceived. OnConnected
and OnDataUnlabeledReceived have no writers today, but they are read from
the same background goroutines and would race the moment one appeared, so
all three move together rather than leaving a split API on one struct.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* sfu: broadcast RTP to down tracks without a per-packet closure
Every forwarded packet allocated a closure capturing the packet and layer,
and an atomic write counter that escaped with it, even for tracks with a
single subscriber. Add BroadcastRTP, which carries the packet and layer in
the worker state and sums the writes per worker. The serial path no longer
allocates; the parallel path allocates the shared state and the worker
funcval only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: reuse the forwarded packet copy in the RED receivers
Both RED receivers copied the ExtPacket (and the opus one the rtp.Packet)
per forwarded packet, and the copies moved to the heap because the
broadcast takes their address. Keep them on the receiver like the existing
redPayloadBuf: ForwardRTP runs on one goroutine and down tracks do not keep
the packet past WriteRTP.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: skip BroadcastRTP allocation assertion under the race detector
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* sfu: group BroadcastRTP and its interfaces after the spreader methods
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
sync.Pool drops a quarter of returned items when built with -race, so the
pooled send path averages about one allocation per packet in CI and the
zero-allocation assertion flips between passing and failing. Keep the
extension checks and skip only the allocation count under race.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
SendPacket allocated four times per down track write: a 3 byte slice for
abs-send-time, a 2 byte slice for transport-cc, and two appends to a nil
Extensions slice because the pooled header was reset to a zero value.
Give pacer.Packet fixed scratch arrays for the two extensions and marshal
into them. The Packet is pooled and owned by one send until SendPacket
returns, so nothing is shared across down tracks. Keep the pooled header's
Extensions capacity across reuse, both when returning it to the pool and
when a down track initializes it.
Adds a test that asserts zero allocations per SendPacket.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
CodecMunger.UpdateAndGet returned a freshly allocated slice for every
forwarded packet on every down track. Return the header in a fixed array
with a length instead and marshal into it with MarshalTo. TranslationParams
carries the array; WriteRTP slices it locally.
UpdateAndGet now allocates zero times per call (was one). Escape analysis
confirms TranslationParams stays on the stack.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
SubscriptionManager tracks data-track subscriptions but exposed no accessor
for the active ones, so callers could see subscribed and published media tracks
and published data tracks, but not subscribed data tracks. Add
GetSubscribedDataTracks(), returning the bound data down-tracks, mirroring
GetSubscribedTracks() for media, so a participant's full track set can be
accounted for. Regenerated the LocalParticipant fake.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* telemetry: do not recreate a stats worker for a released guard
A ParticipantActive overtaken by the participant's close arrives with a
guard ParticipantLeft already released and replaced the closed worker
with one nothing could release.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* telemetry: handle a released guard independently of map presence
A released guard reaching getOrCreateWorker after the closed worker was
reaped still created a zero-reference worker. Return nil as found
instead, and make SetConnected nil-safe for ParticipantActive.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* telemetry: add visibility into stats worker reference underflow
Log the paths that can leave a stats worker with no references, so the
`-1` never-closed cases seen in production can be traced to their origin.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* telemetry: guard StatsWorker.MarshalLogObject against a nil receiver
The new worker-created log passes the existing worker, which is a typed
nil when there is none. Mirror ReferenceGuard.MarshalLogObject.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* turn: accept PROXY protocol on the TCP listener
Behind a TLS-terminating or reverse proxy that dials from its own address,
the embedded TURN server reports the proxy's address to the client as
XOR-MAPPED-ADDRESS. Firefox rejects a loopback or wildcard mapped address
and abandons the allocation, so relay-only clients never get a relay
candidate (#4851).
Add turn.proxy_protocol. When set, the TCP listener requires a PROXY
protocol v1/v2 header on every connection and uses the client address it
carries; connections without the header are rejected. The header is read
before TLS, so it works with both the built-in TLS listener and
external_tls.
* turn: only trust PROXY headers from configured proxies
A PROXY header from any peer that can reach the port would let a direct
client claim an arbitrary source address. Add
turn.proxy_protocol_trusted_cidrs, defaulting to loopback, and close
connections from any other address before reading the header.
getCPUStats has no callers: node CPU load comes from hwstats.CPUStats in
GetNodeStats, and nothing in the tree reads getCPUStats. Only getLoadAvg
is still used.
Remove the function from both the windows and non-windows variants along
with the state it kept. getLoadAvg is untouched, and go-osstat stays a
direct dependency through it.
Co-authored-by: XiaoShao <26596822+xiaoshao9704@users.noreply.github.com>
getMessageBus built the bus from the redis client alone, so there was no
way to reach the gzip settings psrpc v0.7.6 added at the bus boundary.
Take rpc.PSRPCConfig, which the wire graph already provides, and pass its
bus options to both the redis and the local bus.
Compression is off by default. A peer on an older psrpc cannot decode a
compressed payload and drops it silently, so egress, ingress, SIP and
agent workers all have to be upgraded before quality is raised.
* rtc: send connection quality to participants that subscribe after their first update
Added tracking for participants' connection quality updates to ensure all subscribed participants receive necessary updates, even if they were added after the last quality announcement
* rtc: record sent connection quality only after a successful send
A failed SendConnectionQualityUpdate must not mark the participant as informed, otherwise the update is never retried while qualities stay stable. Also simplify the untold-subscription check.
* ingress: add opt-in support for udp:// URL pull ingress
URL pull ingress previously accepted only http, https and srt source
URLs. Add udp:// as an accepted scheme, gated behind a new
`ingress.enable_udp_url_pull` config option that defaults to false, so
unauthenticated UDP sources are not reachable unless the operator opts
in.
Also add a counterfeiter fake for IngressLauncher and a test covering
the scheme validation matrix.
The ingress handler binds a local socket on the caller supplied address
and port instead of connecting out like the http and srt sources do.
Spell out what that means for operators in both the config field doc
comment and config-sample.yaml: caller controlled local port binding,
unauthenticated and spoofable input, and multicast relaying of traffic
on the handler's local network.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: hold signal messages until the ReconnectResponse goes out
Clients take the ReconnectResponse as the first message on a resumed or
migrated in signal connection, anything ahead of it is dropped. Hold
messages back until it has been written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: keep queued participant updates on a path that always flushes
Queue only while the participant is not ready, that queue is always
drained by the join or reconnect response. Log a dropped SDP, it leaves
the negotiation waiting until the state machine recovers it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: flush queued updates whenever the connection opens
Queue participant updates while the handshake is pending again, and give
the signaller a hook that fires when the connection opens, on an explicit
open and on the handshake window expiring, so the queue always drains.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: drive the handshake window off a timer and read the gate atomically
The window ran only when something asked whether the handshake was
pending, so a connection with nothing else to send held its queue.
Reading the gate under the lock the flush takes closes the race where an
update queued just after a flush.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: ignore a handshake timeout from a wait that has ended
Stop cannot cancel a timeout that is already running, so tag each wait
and let a timeout act only on its own. Count opens atomically in the
test, the timer fires on its own goroutine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The link target was passed through url.PathEscape, which turns the '?' of
turn:host:3478?transport=udp into %3F. The target of a Link header field
is a URI reference and "?transport=" is part of TURN URI syntax
, not a query string, so a publisher that percent-decodes the
target ends up with an unparsable host.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* telemetry: support roomID change for a participant
A room can get a new id while participants are connected. Key stats
workers as map[roomID]map[participantID] so moving a room is a single
map splice, and add reKeyRoom/RoomIDChanged to do the move.
Stats collected before the change are sealed off with the room they
were collected in so they stay attributed to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* telemetry: close superseded worker on re-key collision
Only one worker can be keyed at (room, participant). If a re-key lands
on a room that already has a worker for the same participant, keep the
one already filed there and close the superseded one so it drains and
is reaped instead of lingering in the flush list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* telemetry: hand references to the successor on force close
A ReferenceGuard records that it activated some worker, not which one,
so a superseded worker cannot just drop its references - the survivor
would be left with references it never sees released and would never
close. Hand them over instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* report a reason on room-ended telemetry
Room.Close already took a ParticipantCloseReason, but OnClose was func()
with no arguments, so the reason was structurally dropped before it could
reach RoomEnded.
Add types.RoomCloseReason and carry it through Close -> OnClose -> RoomEnded,
where it lands on both the analytics event and the room_finished webhook.
Room.Close now takes the room reason and derives the participant reason from
it, so the two can never disagree. ToParticipantCloseReason maps each reason
to exactly what its call site passed before, so no participant-facing
behaviour changes; a table test pins that mapping.
* fix(test): update webhook test for RoomCloseReason
test/ was outside the packages checked before pushing, so this Close call
site was missed. Also assert the reason reaches the room_finished webhook,
which the unit test cannot cover since it runs with a nil notifier.
* Carry worker kind details into agent job tokens.
WorkerRegistration gains a server-controlled KindDetails field that is
passed to BuildAgentToken when assigning jobs, so kind details flow into
the participant join token for the whole session.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: shishir gowda <shishir@livekit.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
* utils: make Median generic, overflow-safe, and add comprehensive
* tests: fix staticcheck unused variable warning in changenotifier_test.go
* tests: switch from assert to require for consistency with existing tests
* trigger ci rerun
In migration cases, dummy receiver trackInfo is used by relay tracks to
set up the receivers and those need the proper track info.
Also check for proper receiver when adding a migrated track.
TestUnsubscribe checked that the changed-notifier observer was gone as
soon as the unsubscribe had settled, but setDesired leaves the
RemoveObserver call to a goroutine of its own and nothing the test waits
on orders against it, so CI caught the assertion running first.
TestSubscribe has the same defect on the unsubscribed callback, which
unmarkSubscribedTo delivers with a bare go while the subscribed one is
called inline.
Wait for both, each with a message of its own, so that a leak that is
real still says which of them broke.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Let the docker-backed service tests be skipped with a flag.
TestMain called log.Fatalf when it could not reach a docker daemon, so
the whole package refused to run without one, including every test in it
that needs no container at all.
Record why docker is unavailable instead, and gate the tests that want a
container on it. A run asks to go without them with -docker=false;
otherwise a missing daemon still fails the package, so an unreachable
daemon stays a broken build rather than a run that quietly covers less
than the last one did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pion no longer starts the repair stream reader when a custom BufferFactory
is set, so the mid/rid/rsid extensions were never observed and simulcast RTX
streams were never paired with their primary streams.
Extract the extensions on the buffer write path instead. Migrated publishers
send no extensions at all, so pair those from SimTracks.
Adds an integration test covering both paths, and moves the vnet setup it
shares with the downtrack test into pkg/testutils/vnettest.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
VP9 SVC sends each spatial layer as its own encoded frame,
so a picture carries one trailer per layer, but only the
top layer's last packet has the RTP marker bit set.
Fix https://github.com/livekit/egress/issues/1347
* Reduce locking in media track + telemetry listener on move participant.
Telemetry listener was not getting it from new room on room move.
* update comment
* fix signal bytes reporting
* room aware telemetry listener in media track
* data track telemetry listener
SendData API messages do not have sender ID or sequence number. So, they
were not cached and hence excluded from the reliable caching feature
which is meant to provide reliability of data channel messages between
the time other participants see participant as ACTIVE (which happens on
ICE connected) and data channel being open (DTLS done + data channels
opened).
Add a small cache for that and flush those messages on data channel
establishment.
* Check for ICE connection before closing participant on signal close.
Only close the participant if ICE has not connected by the time signal
source is closed. If ICE had connected, candidates have been exchanged
and link was established. So, it should be resumable.
Waiting for DTLS closed the participant sometimes in the windowa after
ICE connection, but before DTLS finishes and that unnecessarily closed
the participant forcing a full reconnect.
* variable name
* Flush pending signal responses before closing the web socket.
When the request direction of a signalling connection goes away, the
web socket was closed right away. Responses that the participant had
already sent were dropped.
This loses the leave request on migration. The media node writes
leave(RESUME) and closes the signalling connection just after. The
close won the race, so the client saw a plain web socket close with
code 1000 and never got the leave. It then did a full reconnect
instead of a resume.
Now the response pump is signalled first and drains what is pending,
then the web socket is closed. The producer closes the response
source after its last write, so draining till the source is closed is
a complete flush. A deadline bounds the case where the source stays
open.
The web socket is still closed on all paths, so the ping worker does
not leak.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Give the response pump a margin over the drain deadline.
Both waits used the same timeout and started at about the same time.
So when the drain ran to its deadline, the outer wait could give up at
the same moment and close the web socket while the pump was still
writing. That write failed and the message was lost. It also logged a
timeout even though nothing was stuck.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
- Renovate
config:recommended (config:base is deprecated) and matchPackageNames globs instead of the deprecated matchPackagePrefixes.
Vulnerability alerts get a fast path: 2-day quarantine, no concurrency/hourly/schedule limits.
Go modules are no longer grouped into one "go deps" PR — each gets its own, so a bad bump can be reverted alone. The pion modules stay grouped as a documented exception: they're co-released and interdependent, so individual PRs wouldn't build.
First-party github.com/livekit/** skips the 2-week quarantine.
go.mod's go directive is no longer an update target — the build toolchain is pinned in the Dockerfile instead.
Dockerfile deps get pinDigests; the golang image is ungrouped with separateMinorPatch so a patch and a minor bump are each separately approvable.
Custom manager to bump the builder image's -alpineA.B suffix together with its digest, which the stock docker manager holds fixed.
- Pinning
Both Dockerfiles pin golang and alpine by digest alongside the readable tag.
GOTOOLCHAIN=local so a go.mod bump fails loudly instead of silently downloading a different toolchain.
apk upgrade in the runtime stage — a digest pin plus the 2-week quarantine would otherwise ship base-package CVEs Alpine has already fixed. This relies on a cold layer cache, which holds today because the release workflow configures no buildx cache; there's a comment saying so.
Workflows resolve the Go version from the Dockerfile via .github/scripts/go-version.sh, so tests, releases and images share one toolchain.
- Tools
All four code generators now come from the module graph, and tools/tools.go (the pre-Go-1.24 blank-import idiom) is replaced by go.mod tool directives:
tool how why
goimports go tool lives in x/tools — its own module is the one being selected
gotestfmt go tool zero dependencies, nothing to skew
wire go run pins x/tools v0.24.1; building it in our graph changes its output
counterfeiter go run unchanged, matches its //go:generate directives
The wire distinction is load-bearing. Building wire inside our module raises it from the x/tools v0.24.1 it pins to our v0.48.0, and that module version difference changes what it generates: it falls back to v/v2/v3 instead of deriving real identifiers from the type. wire_gen.go is regenerated here to match the in-module build — a cosmetic rename of 9 lines, with no other change to the generated code.
golangci-lint deliberately keeps its action rather than becoming a tool: it pins its own x/tools (v0.44.0 vs our v0.48.0) for the analyzers it bundles, adding it to go.mod would double our go.mod/go.sum (158→338 / 441→889 lines), and the action supplies caching, only-new-issues and PR annotations that invoking a binary can't. Its version stays manual by request.
* Redact stream keys in UpdateStream API log fields
UpdateStream add/remove urls were logged raw into the API request log,
including rtmp stream keys and mux/twitch shorthand keys. Redact them
with utils.RedactStreamKey before appending log fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Bump protocol for query-value redaction fallback
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Flush the downtrack sequencer on stream restart (Resync, ReceiverRestart,
codec change) so NACK retransmissions can't use metadata that no longer
matches the resynced bucket. Add a defensive bounds guard on the RTX and
forward payload slicing.
Cap the PacketHistory and FrameIntegrityChecker catch-up loops to the ring
size so a large sequence/frame-number jump can't drive a big per-packet
iteration count.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
shak31599