Commit Graph

332 Commits

Author SHA1 Message Date
cnderrauber
b61799ecd8 Ignore parse addr error when add remote candidate (#4264) 2026-01-26 14:37:02 +08:00
Anunay Maheshwari
0c33b8c671 chore: move codecs/mime stuff to protocol (#4255) 2026-01-20 20:54:32 +05:30
Raja Subramanian
335f4c33fb Swap result sink atomically rather than closing and setting. (#4216)
To prevent WriteMessage after Close potentially.
2026-01-03 03:17:58 +05:30
Raja Subramanian
08793bea89 Use active at time to check for track not bound timeout. (#4206) 2025-12-29 20:32:08 +05:30
Raja Subramanian
1df1316b85 Move OnDataTrackMessage to participant listener to support replay. (#4178) 2025-12-19 13:08:44 +05:30
Raja Subramanian
e7601251bc Make data message naming a bit more consistent. (#4177)
* Make data message naming a bit more consistent.

OnDataPacket and OnDataMessage half-way in the chain made it confusing
(for me at least). Use same name throughout.
API still uses SendDataPacket, but that is not harder to read.

* test
2025-12-19 12:09:16 +05:30
Raja Subramanian
a04e566dbf Use published track for model access in data down track. (#4176)
* Use published track for model access in data down track.

No need to pass in the model.

* implementation type assertion

* remove redundant log key

* one more type

* Close interface method

* clean up redundant log key

* remove unused method

* some interface changes to support data tracks in replay

* correct mock signature
2025-12-19 11:00:18 +05:30
Raja Subramanian
5964efbba5 Ensure subscribe data track handles are unique (#4162) 2025-12-16 13:52:05 +05:30
Raja Subramanian
64f3d1e972 switch participant callbacks to room to listener interface (#4136)
* switch participant callbacks to room to listener interface

* mage generate

* clean up

* clear listener

* clean up

* use interface in up data track manager

* tweaks

* Paul feedback - should reduce the diff as this keeps the room handlers as is except making methods for a couple of anonymous handlers

* clean up
2025-12-08 15:59:45 +05:30
Raja Subramanian
7954748d7a Data tracks (#4089)
* WIP

* WIP

* Starting to add some signalling integration testing.

* Working tests.

* fix tests

* Forward data packets (#4096)

* WIP commit

* WIP

* WIP

* fix forwarding

* address PR comments

* move some methods from LocalParticipant to Participant interface

* handle subscription update

* add extensions and tests

* more packet tests

* add test for replace extension and fix a bug

* update protocol and add config
2025-12-04 10:44:34 +05:30
Paul Wells
3d73703152 add idempotent reference count to telemetry stats worker (#3964)
* add idempotent reference guard to telemetry stats worker

* tidy

* sync

* tidy
2025-09-29 02:35:16 -07:00
Anunay Maheshwari
990c5fafbb feat: server rpc apis (#3904)
* feat: server rpc apis

* fix: cleanup

* fix: move rpc impl to participant

* cleanup

* cleanup, psrpc errors

* remove TODO comment

* update protocol, handle participant disconnect case

* add ephemeral participant identity to rpc data packet

* fix: panic

* chore(deps): bump proto
2025-09-24 13:16:27 +05:30
Raja Subramanian
df6c26dbf6 Subscrbed audio codecs - update from remote nodes. (#3921) 2025-09-12 13:01:00 +05:30
cnderrauber
76645fad5e Rpcs for ingress proxy WHIP (#3911)
See https://github.com/livekit/protocol/pull/1194
2025-09-09 22:49:42 +08:00
Raja Subramanian
07c40cf349 Use RequestResponse to report protocol handling errors (#3895)
* Send errors as necessary

* WIP

* WIP

* WIP

* WIP

* deps

* static check

* fix test

* test

* test

* logs
2025-09-04 11:27:07 +05:30
Raja Subramanian
890fd94249 Single peer connection mode (#3873)
* WIP

* check using protocol version

* revert

* clean up

* sdp cid argument

* WIP

* WIP

* test

* clean up

* clean up

* fixes

* clean up

* clean up

* clean up

* conditional checks

* tests for both dual and single peer connection

* test

* test

* test

* type check

* test

* todo

* munges

* combined config

* populate mid

* limit to receive only

* clean up

* clean up

* clean up

* older test

* clean up

* alternative audio codec

* dtx

* don't need to copy

* Anunay feedback

* use the available peer connection

* publisher check

* WIP

* WIP

* WIP

* no mid

* media sections requirement

* mage generate

* WIP

* WIP

* set data channel receive size for test

* handle early media better

* WIP

* do not do ICERestart if no subscriber

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* start up subscriber RTCP worker

* WIP

* WIP

* clean up

* clean up

* flag to indicate use of single peer connection

* remove unused interface method

* clean up

* clean up

* Jie feedback #1

* deps

* do not access subscriber in one shot mode

* more places for one shot mode

* more one shot fixes

* deps

* deps

* test
2025-08-28 12:16:18 +05:30
Raja Subramanian
fa5f4ef33c Populate SDP cid in track info when available. (#3845)
* Populate SDP cid in track info when available.

- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
  id for AddTrack and actual SDP offer. So, have to look up using both
  on server side. To make it easier, store both (only if different) in
  TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
  around doing simulcast codec for audio so that something like PSTN can
  use G.711 without any transcoding. So, just keep it consistent between
  audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
  pending track or an already published track if the new offer is for a
  back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
  times.
- Clean up MediaTrack interface a bit and remove unneeded methods.

* WIP

* WIP

* deps

* stream allocator mime aware

* clean up

* populate SDP cid before munging

* interface methods
2025-08-13 10:53:16 +05:30
Raja Subramanian
1b2289137d Support video layer mode from client and make most of the code mime aware (#3843) 2025-08-09 21:26:11 +05:30
Raja Subramanian
34a491309f Delete v2 signalling (#3835)
* Delete v2 signalling

* deps

* mage generate

* deelte unused file
2025-08-06 09:08:09 +05:30
Raja Subramanian
db4bc127e8 Get to the point of connecting publisher PC and using it for async signalling (#3822)
* starting signalling DC work

* WIP

* plumbing data channel

* add datachannel message sink file

* mage generate

* clean up
2025-08-01 18:50:28 +05:30
Raja Subramanian
10103449c5 Add country label to edge prom stats. (#3816)
* Add country label to edge prom stats.

* data channel country stats

* test

* pub/sub time country
2025-07-24 13:23:05 +05:30
Raja Subramanian
a75295fcd5 More v2 signalling changes (#3814)
* More v2 signalling changes

- Use wire message in relay signal
- Generate offerId for SDP for sync mode also
- more checks for SDP offer/answer state to see if there are cases of
  missed/duplicate messages

* clean up
2025-07-24 10:14:01 +05:30
Raja Subramanian
0a1bfd3092 Signal handling interfaces and participant specific HTTP PATCH. (#3804)
* HTTP PATCH on participant sid

* rename

* signal handler

* new files

* clean up

* clean up reassembly

* handle messages on server side

* deps

* staticcheck

* clean up
2025-07-20 22:47:36 +05:30
Raja Subramanian
7837c8e595 starting signaller interface (#3802)
* starting signaller interface

* WIP

* WIP

* WIP

* typo

* connect response check

* WIP

* clean up

* move signal handling to participant fully

* service methods

* type assertions for interfaces
2025-07-20 13:48:40 +05:30
Raja Subramanian
2a6a9b8a4a Grouping all signal messages into participant_signal. (#3801)
Currently, it is a bit of a mish-mash
- some compose the message fully and just call send()
- some give parameters and the message is composed in
  participant_signal.go

Was thinking about making an interface for signalling and have v1/v2
impls, but did not want to repeat composing messages if there are common
messages. And some of those function reach into `ParicipantImpl` object
and use information (simple example of p.IsReady()) which would become
more elaborate if the signaller is split out into its own struct.

Maybe, just need to make an interface for the sink and send to the
correct sink based on v1 /v2 signal transport.

But, for now, just grouping all signal messaages in one file
so that it is easier to manage later.
2025-07-18 15:24:52 +05:30
Raja Subramanian
b9a44c3fbf Signalling V2 protocol implementation start (#3794)
* WIP

* name

* refactor validate

* WIP

* WIP

* signal cache initial impl

* HandleConnect in room manager

* generate subscriber offer

* handle ConnectRequest as stand alone

* segmentation, reassembly

* clean up

* rearrange

* lock scope

* support metadata in connect request

* prom

* add SifTrailer to ConnectResponse

* prom for get offer error counter

* RtcInit counter

* Jie feedback

* signal client

* consolidate v1 and v2 into SignalClient

* clean up

* comment

* deps

* mage generate

* fix tests

* pass around roomName and participantIdentity

* mage generate
2025-07-18 00:01:21 +05:30
Raja Subramanian
9fc4ddbe47 ClearAllReceivers interface is used to pause relay tracks. (#3761)
Restore interface.
2025-06-24 16:27:14 -07:00
Raja Subramanian
1216113b35 Do not need to just clean up receivers. Remove that interface. (#3760)
The tracks are always closed.
2025-06-24 12:54:34 -07:00
cnderrauber
e754a86068 return error when moving egree/agent participant (#3741) 2025-06-18 15:10:08 +08:00
Raja Subramanian
a9e2911645 Add Id to SDP signalling messages. (#3722)
* Add Id to SDP signalling messages.

Allows matching up offer/answer.
For now, the subscriber answer just logs if there is a mismatch.

* correct variable name

* test

* fixing rid

* revert

* clean up
2025-06-12 14:39:25 -07:00
cnderrauber
fdf9b85290 e2e reliability for data channel (#3716)
* e2e reliability for data channel

* solve comments
2025-06-11 11:14:30 +08:00
Paul Wells
630aa7d970 implement observability for room metrics (#3712)
* implement observability for room metrics

* deps

* test

* test

* Raja feedback

* cleanup
2025-06-09 09:32:58 -07:00
cnderrauber
dbb70e0f06 Fix dynacast quality for moving out tracks (#3664)
* Make sure moving out track has been unsubscribed

Remove start time checking in subscription manager
as We always use new track ID for republished track at #3020
so there is no race condition now.

Also RemoveSubscriber for moving out tracks for safety,
the subscription manager will handle the removed event but
RemoveSubscriber again will not be bad.

* Clear subscriber node max quality for moving out tracks
2025-05-14 12:54:33 +08:00
Raja Subramanian
5172af1526 ~Send initial participant update only after a participant becomes active.~ - General clean up (#3655)
* Send initial participant update only after a participant becomes active.

There are cases where apps send data to remote participant as soon as
client emits `ParticipantConnected`. But, that time point would not have
a fully established client (i. e. the media connection + data channel
establishment is still in progress).

This PR changes the initial participant update to be sent from server
side only when a participant becomes `ACTIVE`, i.e fully connected
(media channel established and data channels open).

It is supported for clients using protocol version > 15.

@cnderrauber bumping up the protocol version in this PR. Move support is
also conditioned on protocol version > 15, but that PR did not ump
protocol version. Please let me know if there are issues bumping
protocol version.

* check for joining states in broadcast

* have to check on other participant

* test

* make helper for sending participant updates

* test

* make utility of pushAndDeque

* test

* consolidate getting other participants

* remove extra cast

* debug

* debug

* typo

* stop transceiver that is not bound

* logs

* log

* check for ever bound

* clean up

* clean up
2025-05-13 14:46:35 +05:30
cnderrauber
793b383a52 Add Moving participant to another room (#3648)
* Add Moving participant to another room

it is implemented in cloud only since the destination
room can exist in different node with the source room

* Update pkg/service/errors.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* rename

* test panic

* fake LocalParticipantHelper

* revert delete line

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-08 12:58:24 +08:00
Raja Subramanian
f69ab68045 Populate the sender identity when translating to user packet. (#3628) 2025-04-25 23:44:43 +05:30
Raja Subramanian
e14905581c Forward data between WHIP client and non-WHIP client (#3627)
* Forward data between WHIP client and non-WHIP client

* fix test
2025-04-25 23:20:08 +05:30
Raja Subramanian
d8cf54399f Determine TURN connection type and no fallback for TURN/TLS. (#3612) 2025-04-22 11:42:05 +05:30
Raja Subramanian
7e16106a0e Add OnSubscirberReady callback on LocalParticipant. (#3600)
Was setting the state to ACTIVE prematurely to enable the subscription
inter-lock in one shot signalling mode. But, that is incorrectly
changing state.

Use a callback to indicate subscriber ready and let the participant
ACTIVE happen when the connection actually establishes.
2025-04-09 15:02:27 +05:30
Raja Subramanian
35ac5f561a Add support for WHIP ICE Trickle/Restart. (#3596)
* Add support for WHIP ICE Trickle/Restart.

Tested a bit using the WHIP client at https://github.com/Eyevinn/whip,
but needs a lot more testing. ICERestart is not tested yet.

* comment

* clean up
2025-04-09 00:20:58 +05:30
Raja Subramanian
e24fe77bc7 map PEER_CONNECTION_DISCONNECTED -> CONNECTION_TIMEOUT (#3591) 2025-04-08 00:00:10 +05:30
Paul Wells
ee08aede5b skip out of order participant state updates (#3583)
* skip out of order participant state updates

* avoid nil last active pointer

* cleanup

* protect state change callbacks

* gen

* test
2025-04-06 06:44:50 -07:00
Raja Subramanian
d08487bf83 Unlabeled (pass through) data channels. (#3567)
* Unlabeled (pass through) data channels.

Support data channels than can pass through raw data without any LK
protocol marshaling/unmarshaling.

* statischeck

* test

* error -> warn

* reset data message callback
2025-04-01 21:59:31 +05:30
Anunay Maheshwari
52ce18d56e fix: revert recent changes to determine simulcast from sdp (#3565)
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"

This reverts commit cdfbb106d1.

* Revert "Keep simulcast information tied to receiver. (#3563)"

This reverts commit ed5e2f16b2.

* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"

This reverts commit ad010cfc43.

* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"

This reverts commit 15f565510c.

* chore(deps): update protocol
2025-03-31 23:15:44 +05:30
Raja Subramanian
ed5e2f16b2 Keep simulcast information tied to receiver. (#3563)
* Keep simulcast information tied to receiver.

`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.

Keep track of simulcast flag in receiver.

Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track

* clean up

* clean up

* clean up

* clean up

* test

* Store SdpCid and IsSimulcast in Trackinfo

* clean up

* mock
2025-03-31 19:25:57 +05:30
Anunay Maheshwari
15f565510c fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)
* fix(video): determine svc/simulcast from SDP for advanced codecs

* fix(explicit-svc): cleanup

* fix(explicit-svc): remove from list on close/remove

* fix(explicit-svc): reorder VLS selection, cleanup

* fix(explicit-svc): todo comments for temporal layer selector

* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
2025-03-26 16:22:32 +05:30
Raja Subramanian
9551c52c85 Try 2 to consolidate mime type (#3407)
* Normalize mime type and add utilities.

An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.

Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.

* more locations

* test

* Paul feedback

* MimeType type

* more consolidation

* Remove unused

* test

* test

* mime type as int

* use string method

* Pass error details and timeouts. (#3402)

* go mod tidy (#3408)

* Rename CHANGELOG to CHANGELOG.md (#3391)

Enables markdown features in this otherwise already markdown'ish formatted document

* Update config.go to properly process bool env vars (#3382)

Fixes issue https://github.com/livekit/livekit/issues/3381

* fix(deps): update go deps (#3341)

Generated by renovateBot

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Use a Twirp server hook to send API call details to telemetry. (#3401)

* Use a Twirp server hook to send API call details to telemetry.

* mage generate and clean up

* Add project_id

* deps

* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
  participant_id, track_id as appropriate
- Store status as int

* deps

* Update pkg/sfu/mime/mimetype.go

* Fix prefer codec test

* handle down track mime changes

---------

Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
2025-02-10 10:44:15 +05:30
cnderrauber
aeec75edeb H265 supoort and codec regression (#3358)
* H265 supoort and codec regression

Support H265 codec.
Add optional codec regression for subscribers don't
support advanced codecs like H265, AV1, VP9.

* restart forwarder on upstream codec change

* tests

* Reneogitate new codec if client doesn't support change

* Add option to disable codec regression

---------

Co-authored-by: boks1971 <raja.gobi@tutanota.com>
2025-02-06 11:56:49 +08:00
Paul Wells
3970e965f6 move ConnectedAt to Participant interface (#3383) 2025-01-30 05:45:47 -08:00
Raja Subramanian
db75092aa6 Use nonce in data messages to de-dupe SendData API. (#3366)
* Use nonce in data messages to de-dupe SendData API.

With API retries, it is possible that some data messages are sent
multiple times. Check the nonce (if present) to de-dupe data messages
before forwarding it to the clients.

* add nonce to UserPacket

* defer unlock

* 128-bit UUID as nonce

* - UUID nonce
- make a UserPacketDeduper module for easier re-use.

* deps

* feedback
2025-01-27 15:35:37 +05:30