Commit Graph

324 Commits

Author SHA1 Message Date
Raja Subramanian 64f3d1e972 switch participant callbacks to room to listener interface (#4136)
* switch participant callbacks to room to listener interface

* mage generate

* clean up

* clear listener

* clean up

* use interface in up data track manager

* tweaks

* Paul feedback - should reduce the diff as this keeps the room handlers as is except making methods for a couple of anonymous handlers

* clean up
2025-12-08 15:59:45 +05:30
Raja Subramanian 7954748d7a Data tracks (#4089)
* WIP

* WIP

* Starting to add some signalling integration testing.

* Working tests.

* fix tests

* Forward data packets (#4096)

* WIP commit

* WIP

* WIP

* fix forwarding

* address PR comments

* move some methods from LocalParticipant to Participant interface

* handle subscription update

* add extensions and tests

* more packet tests

* add test for replace extension and fix a bug

* update protocol and add config
2025-12-04 10:44:34 +05:30
Paul Wells 3d73703152 add idempotent reference count to telemetry stats worker (#3964)
* add idempotent reference guard to telemetry stats worker

* tidy

* sync

* tidy
2025-09-29 02:35:16 -07:00
Anunay Maheshwari 990c5fafbb feat: server rpc apis (#3904)
* feat: server rpc apis

* fix: cleanup

* fix: move rpc impl to participant

* cleanup

* cleanup, psrpc errors

* remove TODO comment

* update protocol, handle participant disconnect case

* add ephemeral participant identity to rpc data packet

* fix: panic

* chore(deps): bump proto
2025-09-24 13:16:27 +05:30
Raja Subramanian df6c26dbf6 Subscrbed audio codecs - update from remote nodes. (#3921) 2025-09-12 13:01:00 +05:30
cnderrauber 76645fad5e Rpcs for ingress proxy WHIP (#3911)
See https://github.com/livekit/protocol/pull/1194
2025-09-09 22:49:42 +08:00
Raja Subramanian 07c40cf349 Use RequestResponse to report protocol handling errors (#3895)
* Send errors as necessary

* WIP

* WIP

* WIP

* WIP

* deps

* static check

* fix test

* test

* test

* logs
2025-09-04 11:27:07 +05:30
Raja Subramanian 890fd94249 Single peer connection mode (#3873)
* WIP

* check using protocol version

* revert

* clean up

* sdp cid argument

* WIP

* WIP

* test

* clean up

* clean up

* fixes

* clean up

* clean up

* clean up

* conditional checks

* tests for both dual and single peer connection

* test

* test

* test

* type check

* test

* todo

* munges

* combined config

* populate mid

* limit to receive only

* clean up

* clean up

* clean up

* older test

* clean up

* alternative audio codec

* dtx

* don't need to copy

* Anunay feedback

* use the available peer connection

* publisher check

* WIP

* WIP

* WIP

* no mid

* media sections requirement

* mage generate

* WIP

* WIP

* set data channel receive size for test

* handle early media better

* WIP

* do not do ICERestart if no subscriber

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* start up subscriber RTCP worker

* WIP

* WIP

* clean up

* clean up

* flag to indicate use of single peer connection

* remove unused interface method

* clean up

* clean up

* Jie feedback #1

* deps

* do not access subscriber in one shot mode

* more places for one shot mode

* more one shot fixes

* deps

* deps

* test
2025-08-28 12:16:18 +05:30
Raja Subramanian fa5f4ef33c Populate SDP cid in track info when available. (#3845)
* Populate SDP cid in track info when available.

- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
  id for AddTrack and actual SDP offer. So, have to look up using both
  on server side. To make it easier, store both (only if different) in
  TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
  around doing simulcast codec for audio so that something like PSTN can
  use G.711 without any transcoding. So, just keep it consistent between
  audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
  pending track or an already published track if the new offer is for a
  back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
  times.
- Clean up MediaTrack interface a bit and remove unneeded methods.

* WIP

* WIP

* deps

* stream allocator mime aware

* clean up

* populate SDP cid before munging

* interface methods
2025-08-13 10:53:16 +05:30
Raja Subramanian 1b2289137d Support video layer mode from client and make most of the code mime aware (#3843) 2025-08-09 21:26:11 +05:30
Raja Subramanian 34a491309f Delete v2 signalling (#3835)
* Delete v2 signalling

* deps

* mage generate

* deelte unused file
2025-08-06 09:08:09 +05:30
Raja Subramanian db4bc127e8 Get to the point of connecting publisher PC and using it for async signalling (#3822)
* starting signalling DC work

* WIP

* plumbing data channel

* add datachannel message sink file

* mage generate

* clean up
2025-08-01 18:50:28 +05:30
Raja Subramanian 10103449c5 Add country label to edge prom stats. (#3816)
* Add country label to edge prom stats.

* data channel country stats

* test

* pub/sub time country
2025-07-24 13:23:05 +05:30
Raja Subramanian a75295fcd5 More v2 signalling changes (#3814)
* More v2 signalling changes

- Use wire message in relay signal
- Generate offerId for SDP for sync mode also
- more checks for SDP offer/answer state to see if there are cases of
  missed/duplicate messages

* clean up
2025-07-24 10:14:01 +05:30
Raja Subramanian 0a1bfd3092 Signal handling interfaces and participant specific HTTP PATCH. (#3804)
* HTTP PATCH on participant sid

* rename

* signal handler

* new files

* clean up

* clean up reassembly

* handle messages on server side

* deps

* staticcheck

* clean up
2025-07-20 22:47:36 +05:30
Raja Subramanian 7837c8e595 starting signaller interface (#3802)
* starting signaller interface

* WIP

* WIP

* WIP

* typo

* connect response check

* WIP

* clean up

* move signal handling to participant fully

* service methods

* type assertions for interfaces
2025-07-20 13:48:40 +05:30
Raja Subramanian 2a6a9b8a4a Grouping all signal messages into participant_signal. (#3801)
Currently, it is a bit of a mish-mash
- some compose the message fully and just call send()
- some give parameters and the message is composed in
  participant_signal.go

Was thinking about making an interface for signalling and have v1/v2
impls, but did not want to repeat composing messages if there are common
messages. And some of those function reach into `ParicipantImpl` object
and use information (simple example of p.IsReady()) which would become
more elaborate if the signaller is split out into its own struct.

Maybe, just need to make an interface for the sink and send to the
correct sink based on v1 /v2 signal transport.

But, for now, just grouping all signal messaages in one file
so that it is easier to manage later.
2025-07-18 15:24:52 +05:30
Raja Subramanian b9a44c3fbf Signalling V2 protocol implementation start (#3794)
* WIP

* name

* refactor validate

* WIP

* WIP

* signal cache initial impl

* HandleConnect in room manager

* generate subscriber offer

* handle ConnectRequest as stand alone

* segmentation, reassembly

* clean up

* rearrange

* lock scope

* support metadata in connect request

* prom

* add SifTrailer to ConnectResponse

* prom for get offer error counter

* RtcInit counter

* Jie feedback

* signal client

* consolidate v1 and v2 into SignalClient

* clean up

* comment

* deps

* mage generate

* fix tests

* pass around roomName and participantIdentity

* mage generate
2025-07-18 00:01:21 +05:30
Raja Subramanian 9fc4ddbe47 ClearAllReceivers interface is used to pause relay tracks. (#3761)
Restore interface.
2025-06-24 16:27:14 -07:00
Raja Subramanian 1216113b35 Do not need to just clean up receivers. Remove that interface. (#3760)
The tracks are always closed.
2025-06-24 12:54:34 -07:00
cnderrauber e754a86068 return error when moving egree/agent participant (#3741) 2025-06-18 15:10:08 +08:00
Raja Subramanian a9e2911645 Add Id to SDP signalling messages. (#3722)
* Add Id to SDP signalling messages.

Allows matching up offer/answer.
For now, the subscriber answer just logs if there is a mismatch.

* correct variable name

* test

* fixing rid

* revert

* clean up
2025-06-12 14:39:25 -07:00
cnderrauber fdf9b85290 e2e reliability for data channel (#3716)
* e2e reliability for data channel

* solve comments
2025-06-11 11:14:30 +08:00
Paul Wells 630aa7d970 implement observability for room metrics (#3712)
* implement observability for room metrics

* deps

* test

* test

* Raja feedback

* cleanup
2025-06-09 09:32:58 -07:00
cnderrauber dbb70e0f06 Fix dynacast quality for moving out tracks (#3664)
* Make sure moving out track has been unsubscribed

Remove start time checking in subscription manager
as We always use new track ID for republished track at #3020
so there is no race condition now.

Also RemoveSubscriber for moving out tracks for safety,
the subscription manager will handle the removed event but
RemoveSubscriber again will not be bad.

* Clear subscriber node max quality for moving out tracks
2025-05-14 12:54:33 +08:00
Raja Subramanian 5172af1526 ~Send initial participant update only after a participant becomes active.~ - General clean up (#3655)
* Send initial participant update only after a participant becomes active.

There are cases where apps send data to remote participant as soon as
client emits `ParticipantConnected`. But, that time point would not have
a fully established client (i. e. the media connection + data channel
establishment is still in progress).

This PR changes the initial participant update to be sent from server
side only when a participant becomes `ACTIVE`, i.e fully connected
(media channel established and data channels open).

It is supported for clients using protocol version > 15.

@cnderrauber bumping up the protocol version in this PR. Move support is
also conditioned on protocol version > 15, but that PR did not ump
protocol version. Please let me know if there are issues bumping
protocol version.

* check for joining states in broadcast

* have to check on other participant

* test

* make helper for sending participant updates

* test

* make utility of pushAndDeque

* test

* consolidate getting other participants

* remove extra cast

* debug

* debug

* typo

* stop transceiver that is not bound

* logs

* log

* check for ever bound

* clean up

* clean up
2025-05-13 14:46:35 +05:30
cnderrauber 793b383a52 Add Moving participant to another room (#3648)
* Add Moving participant to another room

it is implemented in cloud only since the destination
room can exist in different node with the source room

* Update pkg/service/errors.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* rename

* test panic

* fake LocalParticipantHelper

* revert delete line

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-08 12:58:24 +08:00
Raja Subramanian f69ab68045 Populate the sender identity when translating to user packet. (#3628) 2025-04-25 23:44:43 +05:30
Raja Subramanian e14905581c Forward data between WHIP client and non-WHIP client (#3627)
* Forward data between WHIP client and non-WHIP client

* fix test
2025-04-25 23:20:08 +05:30
Raja Subramanian d8cf54399f Determine TURN connection type and no fallback for TURN/TLS. (#3612) 2025-04-22 11:42:05 +05:30
Raja Subramanian 7e16106a0e Add OnSubscirberReady callback on LocalParticipant. (#3600)
Was setting the state to ACTIVE prematurely to enable the subscription
inter-lock in one shot signalling mode. But, that is incorrectly
changing state.

Use a callback to indicate subscriber ready and let the participant
ACTIVE happen when the connection actually establishes.
2025-04-09 15:02:27 +05:30
Raja Subramanian 35ac5f561a Add support for WHIP ICE Trickle/Restart. (#3596)
* Add support for WHIP ICE Trickle/Restart.

Tested a bit using the WHIP client at https://github.com/Eyevinn/whip,
but needs a lot more testing. ICERestart is not tested yet.

* comment

* clean up
2025-04-09 00:20:58 +05:30
Raja Subramanian e24fe77bc7 map PEER_CONNECTION_DISCONNECTED -> CONNECTION_TIMEOUT (#3591) 2025-04-08 00:00:10 +05:30
Paul Wells ee08aede5b skip out of order participant state updates (#3583)
* skip out of order participant state updates

* avoid nil last active pointer

* cleanup

* protect state change callbacks

* gen

* test
2025-04-06 06:44:50 -07:00
Raja Subramanian d08487bf83 Unlabeled (pass through) data channels. (#3567)
* Unlabeled (pass through) data channels.

Support data channels than can pass through raw data without any LK
protocol marshaling/unmarshaling.

* statischeck

* test

* error -> warn

* reset data message callback
2025-04-01 21:59:31 +05:30
Anunay Maheshwari 52ce18d56e fix: revert recent changes to determine simulcast from sdp (#3565)
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"

This reverts commit cdfbb106d1.

* Revert "Keep simulcast information tied to receiver. (#3563)"

This reverts commit ed5e2f16b2.

* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"

This reverts commit ad010cfc43.

* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"

This reverts commit 15f565510c.

* chore(deps): update protocol
2025-03-31 23:15:44 +05:30
Raja Subramanian ed5e2f16b2 Keep simulcast information tied to receiver. (#3563)
* Keep simulcast information tied to receiver.

`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.

Keep track of simulcast flag in receiver.

Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track

* clean up

* clean up

* clean up

* clean up

* test

* Store SdpCid and IsSimulcast in Trackinfo

* clean up

* mock
2025-03-31 19:25:57 +05:30
Anunay Maheshwari 15f565510c fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)
* fix(video): determine svc/simulcast from SDP for advanced codecs

* fix(explicit-svc): cleanup

* fix(explicit-svc): remove from list on close/remove

* fix(explicit-svc): reorder VLS selection, cleanup

* fix(explicit-svc): todo comments for temporal layer selector

* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
2025-03-26 16:22:32 +05:30
Raja Subramanian 9551c52c85 Try 2 to consolidate mime type (#3407)
* Normalize mime type and add utilities.

An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.

Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.

* more locations

* test

* Paul feedback

* MimeType type

* more consolidation

* Remove unused

* test

* test

* mime type as int

* use string method

* Pass error details and timeouts. (#3402)

* go mod tidy (#3408)

* Rename CHANGELOG to CHANGELOG.md (#3391)

Enables markdown features in this otherwise already markdown'ish formatted document

* Update config.go to properly process bool env vars (#3382)

Fixes issue https://github.com/livekit/livekit/issues/3381

* fix(deps): update go deps (#3341)

Generated by renovateBot

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Use a Twirp server hook to send API call details to telemetry. (#3401)

* Use a Twirp server hook to send API call details to telemetry.

* mage generate and clean up

* Add project_id

* deps

* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
  participant_id, track_id as appropriate
- Store status as int

* deps

* Update pkg/sfu/mime/mimetype.go

* Fix prefer codec test

* handle down track mime changes

---------

Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
2025-02-10 10:44:15 +05:30
cnderrauber aeec75edeb H265 supoort and codec regression (#3358)
* H265 supoort and codec regression

Support H265 codec.
Add optional codec regression for subscribers don't
support advanced codecs like H265, AV1, VP9.

* restart forwarder on upstream codec change

* tests

* Reneogitate new codec if client doesn't support change

* Add option to disable codec regression

---------

Co-authored-by: boks1971 <raja.gobi@tutanota.com>
2025-02-06 11:56:49 +08:00
Paul Wells 3970e965f6 move ConnectedAt to Participant interface (#3383) 2025-01-30 05:45:47 -08:00
Raja Subramanian db75092aa6 Use nonce in data messages to de-dupe SendData API. (#3366)
* Use nonce in data messages to de-dupe SendData API.

With API retries, it is possible that some data messages are sent
multiple times. Check the nonce (if present) to de-dupe data messages
before forwarding it to the clients.

* add nonce to UserPacket

* defer unlock

* 128-bit UUID as nonce

* - UUID nonce
- make a UserPacketDeduper module for easier re-use.

* deps

* feedback
2025-01-27 15:35:37 +05:30
Raja Subramanian 28c39efa06 Exempt egress participant from track permissions. (#3322)
* Exempt egress participant from track permissions.

* test
2025-01-11 12:50:34 +05:30
cnderrauber 54f9f7de51 upgrade to pion/webrtc v4 (#3213) 2024-11-28 16:05:38 +08:00
Raja Subramanian 23285744ba Server side metrics (#3198)
* mbb WIP

* deps

* WIP

* WIP

* remove unused file

* Switch to enabled

* misc

* deps

* mediatransportutil update

* Typo

* Set ParticipantIdentity in metrics data packets

* use uint32 as JSON decoder does not unmarshal time.Duration
2024-11-25 13:10:48 +05:30
Raja Subramanian 3498e53650 Participant method to check a track by name is subscribed. (#3192)
* Set down track connected flag in one-shot-signalling mode.

Also, added maintaing ICE candidates for info purposes.
And doing analytics events (have to maintain the subscription inside
subscriptionmanager to get list of subscribed tracks, so added enough
bits from the async path into sync path to get the analytics bits also)

* comment typo

* method to check if a track name is subscribed
2024-11-22 07:43:38 +05:30
Raja Subramanian 9f25603213 One shot signalling mode (#3188)
* WIP

* comment

* Verify method on LocalParticipant

* cleanup

* clean up

* pass in one-shot-mode to StartSession

* null message source and sink

* feedback and also remove check in ParticipantImpl for one-shot-mode-filtering as a null sink can be used for that
2024-11-21 09:33:28 +05:30
Raja Subramanian 487a3fc3fb ICE candidate marking (#3128)
- Update filtered if dropping a pending candidate later.
- Ordinal for selected pair so it is easy to see which got selected
  later.
2024-10-22 20:23:55 +05:30
Denys Smirnov af15f211d0 Forward new disconnect reasons for SIP. (#3056) 2024-10-03 16:36:47 +03:00
Paul Wells d26ade0076 add helpers for partiicpant reconnect and version (#3034) 2024-09-22 23:37:35 -07:00