Commit Graph

1382 Commits

Author SHA1 Message Date
Anunay Maheshwari
990c5fafbb feat: server rpc apis (#3904)
* feat: server rpc apis

* fix: cleanup

* fix: move rpc impl to participant

* cleanup

* cleanup, psrpc errors

* remove TODO comment

* update protocol, handle participant disconnect case

* add ephemeral participant identity to rpc data packet

* fix: panic

* chore(deps): bump proto
2025-09-24 13:16:27 +05:30
Raja Subramanian
d7f928783f Avoid matching on empty track id. (#3937)
Due to SDP ordering, Pion did not provide track ID on a receiver.

Pion needs a=msid line to be before a=ssrc line -> need to check if this
is a spec requirement

Because of the above, it had empty id for the receiver in onTrack.
That matched a published track because we do not duplicate SdpCid in
TrackInfo if the SDP cid matches the signal cid. But, the search checks
both and it matched on empty SDP cid.

Do not accept empty ids in searches to prevent this from happening.
2025-09-19 15:14:17 +05:30
Raja Subramanian
2c30a0640a Fix dynacast subscriber node clearing on move participant. (#3926) 2025-09-14 15:20:20 +05:30
Raja Subramanian
6489237e33 Simulcast audio fixes (#3925)
* Simulcast audio fixes

* clean up
2025-09-14 09:41:40 +05:30
Raja Subramanian
9f0ab8706f Wait for SetRemoteDescription before configuring senders. (#3924)
* Wait for `SetRemoteDescription` before configuring senders.

Details inline in code.

* comment
2025-09-12 23:24:56 +05:30
Raja Subramanian
df6c26dbf6 Subscrbed audio codecs - update from remote nodes. (#3921) 2025-09-12 13:01:00 +05:30
Raja Subramanian
798fa76110 Support simulcasting of audio (#3920)
* WIP

* WIP

* new files

* clean up

* test

* Deps

* clean up

* clean up

* goimports latest
2025-09-12 10:20:04 +05:30
Raja Subramanian
eee2001a31 Set publisher codec preferences after setting remote description (#3913)
* Set publisher codec preferences after setting remote description

Munging SDP prior to setting remote description was becoming problematic
in single peer connection mode. In that mode, it is possible that a
subscribe track m-section is added which sets the fmtp of H.265 to a
value that is different from when that client publishes. That gets
locked in as negotiated codecs when pion processes remote description.
Later when the client publishes H.265, the H.265 does only partial
match. So, if we munge offer and send it to SetRemoteDescription, the
H.265 does only a partial match due to different fmtp line and that gets
put at the end of the list. So, the answer does not enforce the
preferred codec. Changing pion to put partial match up front is more
risky given other projects. So, switch codec preferences to after remote
description is set and directly operate on transceiver which is a better
place to make these changes without munging SDP.

This fixes the case of
- firefox joins first
- Chrome preferring H.265 joining next. This causes a subscribe track
  m-section (for firefox's tracks) to be created first. So, the
  preferred codec munging was not working. Works after this change.

* clean up

* mage generate

* test

* clean up
2025-09-10 18:28:36 +05:30
Hamdan
fc995533e1 add incoming request id to request response message (#3912) 2025-09-10 08:58:12 +05:30
cnderrauber
76645fad5e Rpcs for ingress proxy WHIP (#3911)
See https://github.com/livekit/protocol/pull/1194
2025-09-09 22:49:42 +08:00
Raja Subramanian
991a4a4f53 Refactor subscribedTrack + mediaTrackSubscriptions. (#3908)
- Move downTrack instantiation to SubscribedTrack as it should own that
  DownTrack. Still more to do here as `DownTrack` is fetched from
  `SubscribedTrack` in a few places and used. Would like to avoid that,
  but doing this initially.
- Use an interface from sfu.Downtrack and replace a bunch of callbacks.
  SubscribedTrack is the implementation for DownTrackListener.
2025-09-08 18:20:19 +05:30
Raja Subramanian
07c40cf349 Use RequestResponse to report protocol handling errors (#3895)
* Send errors as necessary

* WIP

* WIP

* WIP

* WIP

* deps

* static check

* fix test

* test

* test

* logs
2025-09-04 11:27:07 +05:30
Raja Subramanian
f7291fdaa8 Do not send both asb-send-time and twcc. (#3890)
* Do not send both asb-send-time and twcc.

In single peer connection mode, both extensions are set on the media
engine and both would be negotiated. Unfortunately, pion/webrtc does
not yet support RTPSender.SetParameters() which would allow setting
specific header extensions for the sender. So, check for TWCC enabled
and use it. If not, do abs-send-time if that is enabled.

* check BWE type

* comment
2025-08-30 19:22:14 +05:30
Raja Subramanian
21b42fa6d9 Do not advertise NACK for RED. (#3889)
Set the transceiver codecs first in single peer connection mode and
then do audio specific config so that NACK is removed for RED.
2025-08-30 17:34:44 +05:30
Raja Subramanian
38f7906ecb Handle migration better in single peer connection case. (#3886)
* Handle migration better in single peer connection case.

Have to avoid creating a sender for publish tracks.
Otherwise, the sender messes up the answer SDP because
it forced to inactive and that gets used by type match when setting
remote description offer.

Create sender only for m-lines which was used to send.

* remove unnecessary RemoveTrack
2025-08-29 18:18:32 +05:30
Raja Subramanian
091e3c13a1 Revert to using answer for migration case. (#3884)
Was breaking migration in cases where there was inactive transceivers
because of direction check.

This change will break single peer connection case, but can look at that
one later.
2025-08-29 12:58:42 +05:30
Paul Wells
2aeadf14dc init ua parser once (#3883) 2025-08-28 05:20:25 -07:00
Raja Subramanian
890fd94249 Single peer connection mode (#3873)
* WIP

* check using protocol version

* revert

* clean up

* sdp cid argument

* WIP

* WIP

* test

* clean up

* clean up

* fixes

* clean up

* clean up

* clean up

* conditional checks

* tests for both dual and single peer connection

* test

* test

* test

* type check

* test

* todo

* munges

* combined config

* populate mid

* limit to receive only

* clean up

* clean up

* clean up

* older test

* clean up

* alternative audio codec

* dtx

* don't need to copy

* Anunay feedback

* use the available peer connection

* publisher check

* WIP

* WIP

* WIP

* no mid

* media sections requirement

* mage generate

* WIP

* WIP

* set data channel receive size for test

* handle early media better

* WIP

* do not do ICERestart if no subscriber

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* start up subscriber RTCP worker

* WIP

* WIP

* clean up

* clean up

* flag to indicate use of single peer connection

* remove unused interface method

* clean up

* clean up

* Jie feedback #1

* deps

* do not access subscriber in one shot mode

* more places for one shot mode

* more one shot fixes

* deps

* deps

* test
2025-08-28 12:16:18 +05:30
Paul Wells
8d270e2a0f chunk room updates (#3880)
* chunk room updates

* move to config

* typo

* default
2025-08-26 09:16:00 -07:00
Raja Subramanian
d62336e1a6 Remove unnecessary check (#3870) 2025-08-20 09:24:13 +05:30
cnderrauber
b660c3b582 Extract video size from media stream (#3856)
* Extract video size from media stream

* fix test
2025-08-18 09:06:57 +08:00
Raja Subramanian
456b870936 Fix missed unlock (#3861)
* Fix missed unlock

* select alternative codec if no codec specified
2025-08-17 13:08:45 +05:30
Raja Subramanian
d500806eed Handle no codecs in track info. (#3859)
* Handle no codecs in track info.

Not sure how it happens, but seeing a panic caused by no codecs in track
info. Avoiding that and logging information when it happens.

* log request
2025-08-16 00:18:38 +05:30
Raja Subramanian
11b240d67d Log track settings more. (#3853) 2025-08-14 20:26:15 +05:30
Raja Subramanian
1aa0f96374 Log signal messages on media node. (#3852) 2025-08-14 18:12:17 +05:30
Raja Subramanian
a370bb2054 Support G.711 A-law and U-law (#3849)
* More codecs

* clean up

* clean up

* add to unprocessed for nil mime type

* enhance tests to check for audio codec preferences also
2025-08-13 14:49:07 +05:30
Raja Subramanian
fa5f4ef33c Populate SDP cid in track info when available. (#3845)
* Populate SDP cid in track info when available.

- Adding SDP cid to TrackInfo. Browsers like FF uses a different stream
  id for AddTrack and actual SDP offer. So, have to look up using both
  on server side. To make it easier, store both (only if different) in
  TrackInfo.
- Use a codec in TrackInfo for audio also. There is some discussion
  around doing simulcast codec for audio so that something like PSTN can
  use G.711 without any transcoding. So, just keep it consistent between
  audio and video.
- Populate SDP cid when SDP offer is received. It could populate a
  pending track or an already published track if the new offer is for a
  back up codec where the primary codec is already published.
- Passed around parsed offer to more places to avoid parsing multiple
  times.
- Clean up MediaTrack interface a bit and remove unneeded methods.

* WIP

* WIP

* deps

* stream allocator mime aware

* clean up

* populate SDP cid before munging

* interface methods
2025-08-13 10:53:16 +05:30
Raja Subramanian
1b2289137d Support video layer mode from client and make most of the code mime aware (#3843) 2025-08-09 21:26:11 +05:30
Raja Subramanian
f2da4444b4 Support per simulcast codec layers. (#3840)
* different layers

* deps
2025-08-08 19:53:38 +05:30
Raja Subramanian
f275f592ed handle SyncState in join request (#3839)
* handle SyncState in join request

* check for nil in function
2025-08-08 10:14:02 +05:30
Raja Subramanian
5ca1626439 Support join request as proto + base64 encoded query param (#3836)
* Support join request as proto + base64 encoded query param

* joinPublish

* staticcheck

* deps

* tests

* gzip

* test

* deps

* clean up
2025-08-07 11:13:27 +05:30
Raja Subramanian
7dea101286 Clean up missed v2 pieces (#3837)
* Clean up missed v2 pieces

* missed stuff
2025-08-06 22:30:50 +05:30
Raja Subramanian
34a491309f Delete v2 signalling (#3835)
* Delete v2 signalling

* deps

* mage generate

* deelte unused file
2025-08-06 09:08:09 +05:30
Raja Subramanian
db4bc127e8 Get to the point of connecting publisher PC and using it for async signalling (#3822)
* starting signalling DC work

* WIP

* plumbing data channel

* add datachannel message sink file

* mage generate

* clean up
2025-08-01 18:50:28 +05:30
Raja Subramanian
e3155b146b Get to the point of establishing subscriber peer connection. (#3821)
Proper message id checks.

Some refactoring for common code in rtcv2service handlers.

Instantiate signalling based on synchronous local candidates flag, but
needs better defined code.
2025-07-25 14:49:04 +05:30
Raja Subramanian
a7ce138239 HTTP DELETE of participant session (#3819) 2025-07-24 23:02:45 +05:30
Raja Subramanian
01de0e365f Do not send leave if nil (to older clients) (#3817) 2025-07-24 17:24:35 +05:30
Raja Subramanian
10103449c5 Add country label to edge prom stats. (#3816)
* Add country label to edge prom stats.

* data channel country stats

* test

* pub/sub time country
2025-07-24 13:23:05 +05:30
Raja Subramanian
a75295fcd5 More v2 signalling changes (#3814)
* More v2 signalling changes

- Use wire message in relay signal
- Generate offerId for SDP for sync mode also
- more checks for SDP offer/answer state to see if there are cases of
  missed/duplicate messages

* clean up
2025-07-24 10:14:01 +05:30
Raja Subramanian
b20db94dc9 Validation end point for v2 signalling. (#3811)
* WIP

* stricter check

* WIP

* WIP

* clean up
2025-07-23 11:59:30 +05:30
Raja Subramanian
fffc2ac090 Use signalling utils from protocol (#3807)
* Use signalling utils from protocol

* staticcheck
2025-07-21 18:15:06 +05:30
Raja Subramanian
f5fc82d344 Filling out messages unlikely to change in v2. (#3806)
* Filling out messages unlikely to change in v2.

* deps

* remove defensive nil checks
2025-07-21 12:33:52 +05:30
Raja Subramanian
1c99b9ad5f Split signal segmenter and reassembler. (#3805)
* Split signal segmenter and reassembler.

As reassembler may need to run a goroutine for pruning, splitting it up
so that we do not have unnecessary goroutines. Keeping the UT unified
though.

Also, for now, trying to see if `Prune` can be called from external
place to avoid creating a goroutine and have extra state/functions to
close the reassembler and clean up goroutine etc. May still have to do
goroutine, but have not created one for now.

* fix: do not increase packetId for each fragment of a packet
2025-07-21 11:10:20 +05:30
Raja Subramanian
0a1bfd3092 Signal handling interfaces and participant specific HTTP PATCH. (#3804)
* HTTP PATCH on participant sid

* rename

* signal handler

* new files

* clean up

* clean up reassembly

* handle messages on server side

* deps

* staticcheck

* clean up
2025-07-20 22:47:36 +05:30
Raja Subramanian
7837c8e595 starting signaller interface (#3802)
* starting signaller interface

* WIP

* WIP

* WIP

* typo

* connect response check

* WIP

* clean up

* move signal handling to participant fully

* service methods

* type assertions for interfaces
2025-07-20 13:48:40 +05:30
Raja Subramanian
2a6a9b8a4a Grouping all signal messages into participant_signal. (#3801)
Currently, it is a bit of a mish-mash
- some compose the message fully and just call send()
- some give parameters and the message is composed in
  participant_signal.go

Was thinking about making an interface for signalling and have v1/v2
impls, but did not want to repeat composing messages if there are common
messages. And some of those function reach into `ParicipantImpl` object
and use information (simple example of p.IsReady()) which would become
more elaborate if the signaller is split out into its own struct.

Maybe, just need to make an interface for the sink and send to the
correct sink based on v1 /v2 signal transport.

But, for now, just grouping all signal messaages in one file
so that it is easier to manage later.
2025-07-18 15:24:52 +05:30
Raja Subramanian
078c01fad3 Signal v2: envelope and fragments as wire message format. (#3800)
* Signal v2: envelope and fragments as wire message format.

Please see details in https://github.com/livekit/protocol/pull/1133

* log connect response
2025-07-18 15:24:32 +05:30
Raja Subramanian
b9a44c3fbf Signalling V2 protocol implementation start (#3794)
* WIP

* name

* refactor validate

* WIP

* WIP

* signal cache initial impl

* HandleConnect in room manager

* generate subscriber offer

* handle ConnectRequest as stand alone

* segmentation, reassembly

* clean up

* rearrange

* lock scope

* support metadata in connect request

* prom

* add SifTrailer to ConnectResponse

* prom for get offer error counter

* RtcInit counter

* Jie feedback

* signal client

* consolidate v1 and v2 into SignalClient

* clean up

* comment

* deps

* mage generate

* fix tests

* pass around roomName and participantIdentity

* mage generate
2025-07-18 00:01:21 +05:30
cnderrauber
51bbe8c52b Set participant active when peerconnection connected (#3790)
Don't wait datachannel ready (sctp handshake)
2025-07-14 10:59:59 +08:00
Raja Subramanian
40028dc33c Normalize known rids. (#3779)
Normalize the rids in SDP to known patterns.
Currently,
- LK protocol uses q;h;f
- Sean's OBS WHIP uses 0;1;2

As the ordering in SDP could be different, normalize to known order.

For RIDs not in the known set, just use it as is.
2025-07-06 13:32:21 +05:30