Commit Graph

297 Commits

Author SHA1 Message Date
Raja Subramanian
f69ab68045 Populate the sender identity when translating to user packet. (#3628) 2025-04-25 23:44:43 +05:30
Raja Subramanian
e14905581c Forward data between WHIP client and non-WHIP client (#3627)
* Forward data between WHIP client and non-WHIP client

* fix test
2025-04-25 23:20:08 +05:30
Raja Subramanian
d8cf54399f Determine TURN connection type and no fallback for TURN/TLS. (#3612) 2025-04-22 11:42:05 +05:30
Raja Subramanian
7e16106a0e Add OnSubscirberReady callback on LocalParticipant. (#3600)
Was setting the state to ACTIVE prematurely to enable the subscription
inter-lock in one shot signalling mode. But, that is incorrectly
changing state.

Use a callback to indicate subscriber ready and let the participant
ACTIVE happen when the connection actually establishes.
2025-04-09 15:02:27 +05:30
Raja Subramanian
35ac5f561a Add support for WHIP ICE Trickle/Restart. (#3596)
* Add support for WHIP ICE Trickle/Restart.

Tested a bit using the WHIP client at https://github.com/Eyevinn/whip,
but needs a lot more testing. ICERestart is not tested yet.

* comment

* clean up
2025-04-09 00:20:58 +05:30
Raja Subramanian
e24fe77bc7 map PEER_CONNECTION_DISCONNECTED -> CONNECTION_TIMEOUT (#3591) 2025-04-08 00:00:10 +05:30
Paul Wells
ee08aede5b skip out of order participant state updates (#3583)
* skip out of order participant state updates

* avoid nil last active pointer

* cleanup

* protect state change callbacks

* gen

* test
2025-04-06 06:44:50 -07:00
Raja Subramanian
d08487bf83 Unlabeled (pass through) data channels. (#3567)
* Unlabeled (pass through) data channels.

Support data channels than can pass through raw data without any LK
protocol marshaling/unmarshaling.

* statischeck

* test

* error -> warn

* reset data message callback
2025-04-01 21:59:31 +05:30
Anunay Maheshwari
52ce18d56e fix: revert recent changes to determine simulcast from sdp (#3565)
* Revert "Audio uses signal SignalCid and SdpCid. (#3564)"

This reverts commit cdfbb106d1.

* Revert "Keep simulcast information tied to receiver. (#3563)"

This reverts commit ed5e2f16b2.

* Revert "chore(logs): log VLS type for VP9/AV1 (#3561)"

This reverts commit ad010cfc43.

* Revert "fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)"

This reverts commit 15f565510c.

* chore(deps): update protocol
2025-03-31 23:15:44 +05:30
Raja Subramanian
ed5e2f16b2 Keep simulcast information tied to receiver. (#3563)
* Keep simulcast information tied to receiver.

`simulcast` flag in `TrackInfo` is at track lavel. With codec simulcast,
the primary codec (in most cases) is SVC and the backup codec is
simulcast. Back up codec publish changing the track info setting to true
meant that the primary receiver was treated as simulcast if a subscriber
for primary codec joined after the backup codec was published.

Keep track of simulcast flag in receiver.

Also, TrackInfo Cids are from signal. So, keep track of SDP cids
separately. The `simulcastTrackIds` map uses SDP cid. Clean up by all
the SDP cids of a track

* clean up

* clean up

* clean up

* clean up

* test

* Store SdpCid and IsSimulcast in Trackinfo

* clean up

* mock
2025-03-31 19:25:57 +05:30
Anunay Maheshwari
15f565510c fix(video): determine svc/simulcast from SDP for advanced codecs (#3549)
* fix(video): determine svc/simulcast from SDP for advanced codecs

* fix(explicit-svc): cleanup

* fix(explicit-svc): remove from list on close/remove

* fix(explicit-svc): reorder VLS selection, cleanup

* fix(explicit-svc): todo comments for temporal layer selector

* fix(explicit-svc): remove from simulcastTrackIds even if client does not support unpublish
2025-03-26 16:22:32 +05:30
Raja Subramanian
9551c52c85 Try 2 to consolidate mime type (#3407)
* Normalize mime type and add utilities.

An attempt to normalize mime type and avoid string compares remembering
to do case insensitive search.

Not the best solution. Open to ideas. But, define our own mime types
(just in case Pion changes things and Pion also does not have red mime
type defined which should be easy to add though) and tried to use it everywhere.
But, as we get a bunch of callbacks and info from Pion, needed conversion in
more places than I anticipated. And also makes it necessary to carry
that cognitive load of what comes from Pion and needing to process it
properly.

* more locations

* test

* Paul feedback

* MimeType type

* more consolidation

* Remove unused

* test

* test

* mime type as int

* use string method

* Pass error details and timeouts. (#3402)

* go mod tidy (#3408)

* Rename CHANGELOG to CHANGELOG.md (#3391)

Enables markdown features in this otherwise already markdown'ish formatted document

* Update config.go to properly process bool env vars (#3382)

Fixes issue https://github.com/livekit/livekit/issues/3381

* fix(deps): update go deps (#3341)

Generated by renovateBot

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Use a Twirp server hook to send API call details to telemetry. (#3401)

* Use a Twirp server hook to send API call details to telemetry.

* mage generate and clean up

* Add project_id

* deps

* - Redact requests
- Do not store responses
- Extract top level fields room_name, room_id, participant_identity,
  participant_id, track_id as appropriate
- Store status as int

* deps

* Update pkg/sfu/mime/mimetype.go

* Fix prefer codec test

* handle down track mime changes

---------

Co-authored-by: Denys Smirnov <dennwc@pm.me>
Co-authored-by: Philzen <Philzen@users.noreply.github.com>
Co-authored-by: Pablo Fuente Pérez <pablofuenteperez@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Paul Wells <paulwe@gmail.com>
Co-authored-by: cnderrauber <zengjie9004@gmail.com>
2025-02-10 10:44:15 +05:30
cnderrauber
aeec75edeb H265 supoort and codec regression (#3358)
* H265 supoort and codec regression

Support H265 codec.
Add optional codec regression for subscribers don't
support advanced codecs like H265, AV1, VP9.

* restart forwarder on upstream codec change

* tests

* Reneogitate new codec if client doesn't support change

* Add option to disable codec regression

---------

Co-authored-by: boks1971 <raja.gobi@tutanota.com>
2025-02-06 11:56:49 +08:00
Paul Wells
3970e965f6 move ConnectedAt to Participant interface (#3383) 2025-01-30 05:45:47 -08:00
Raja Subramanian
db75092aa6 Use nonce in data messages to de-dupe SendData API. (#3366)
* Use nonce in data messages to de-dupe SendData API.

With API retries, it is possible that some data messages are sent
multiple times. Check the nonce (if present) to de-dupe data messages
before forwarding it to the clients.

* add nonce to UserPacket

* defer unlock

* 128-bit UUID as nonce

* - UUID nonce
- make a UserPacketDeduper module for easier re-use.

* deps

* feedback
2025-01-27 15:35:37 +05:30
Raja Subramanian
28c39efa06 Exempt egress participant from track permissions. (#3322)
* Exempt egress participant from track permissions.

* test
2025-01-11 12:50:34 +05:30
cnderrauber
54f9f7de51 upgrade to pion/webrtc v4 (#3213) 2024-11-28 16:05:38 +08:00
Raja Subramanian
23285744ba Server side metrics (#3198)
* mbb WIP

* deps

* WIP

* WIP

* remove unused file

* Switch to enabled

* misc

* deps

* mediatransportutil update

* Typo

* Set ParticipantIdentity in metrics data packets

* use uint32 as JSON decoder does not unmarshal time.Duration
2024-11-25 13:10:48 +05:30
Raja Subramanian
3498e53650 Participant method to check a track by name is subscribed. (#3192)
* Set down track connected flag in one-shot-signalling mode.

Also, added maintaing ICE candidates for info purposes.
And doing analytics events (have to maintain the subscription inside
subscriptionmanager to get list of subscribed tracks, so added enough
bits from the async path into sync path to get the analytics bits also)

* comment typo

* method to check if a track name is subscribed
2024-11-22 07:43:38 +05:30
Raja Subramanian
9f25603213 One shot signalling mode (#3188)
* WIP

* comment

* Verify method on LocalParticipant

* cleanup

* clean up

* pass in one-shot-mode to StartSession

* null message source and sink

* feedback and also remove check in ParticipantImpl for one-shot-mode-filtering as a null sink can be used for that
2024-11-21 09:33:28 +05:30
Raja Subramanian
487a3fc3fb ICE candidate marking (#3128)
- Update filtered if dropping a pending candidate later.
- Ordinal for selected pair so it is easy to see which got selected
  later.
2024-10-22 20:23:55 +05:30
Denys Smirnov
af15f211d0 Forward new disconnect reasons for SIP. (#3056) 2024-10-03 16:36:47 +03:00
Paul Wells
d26ade0076 add helpers for partiicpant reconnect and version (#3034) 2024-09-22 23:37:35 -07:00
Paul Wells
61179d5624 simplify agent registration (#3018)
* simplify agent registration

* testutils

* deps

* fix

* gen

* cleanup

* lower job load
2024-09-19 05:58:30 -07:00
Paul Wells
370f386dd0 add convenience methods to participant (#3016)
* add convenience methods to participant

* tidy
2024-09-19 00:20:22 -07:00
Raja Subramanian
7df6f86693 Initial plumbing for metrics. (#2950)
* Initial plumbing for metrics.

This implements
- metrics received from participant.
- callback to room.
- room distributes it to all other participants (excluding the sending
  participant).
- other participants forward to client.
- counting metrics bytes in data channel stats

TODO:
  - recording/processing/batching
  - should recording/processing/batching happen on publisher side or
    subscriber side?
  - should metrics be echoed back to publisher?
  - grants to publish/subscribe metrics.

* mage generate

* clear OnMetrics on close

* - CanSubscribeMetrics permission.
- Echo back to sender.

* update deps

* No destination identities for metrics

* WIP

* use normalized timestamp for server injected timestamps

* compile

* debug log metrics batch

* correct comment

* add baseTime to wire

* protocol dep

* Scope metrics forwarding to only participants that a participant is
subscribed to.

Also remove the participant_metrics.go file as it was not doing anything
useful.

* update comment

* utils.ErrorIsOneOf

* couple of more utils.CloneProto
2024-09-19 11:42:31 +05:30
Raja Subramanian
1b5bb4dddc Log ICE reconnected. (#2999)
To increase visibility of ICE reconnect, logging reconnected at Infow
level. Otherwise, it is hard to see if an ICE restart finished
successfully.

Also, cleaning up ICEConnectionDetails a bit. Just separate out
read-only fields into its own struct and use it for read-only export.
2024-09-12 13:16:41 +05:30
cnderrauber
16896e4d66 Add FastPublish in JoinResponse (#2964)
Set FastPublish in JoinResponse for participant can publish
and no ice fallback setting.
2024-08-28 12:00:28 +08:00
Raja Subramanian
e9f26c21e7 Panic fix for nil candidate check. (#2957) 2024-08-25 10:12:32 +05:30
Raja Subramanian
56e67c9d25 Properly exclude mDNS when not trickling also. (#2956) 2024-08-24 23:23:23 +05:30
cnderrauber
947e8f5909 Speed up track publication (#2952)
* speed up track publication

Add metrics for track publication and subscription

Return EnabledCodecs in JoinResponse so client can
choose codec without server side codec fallback

Cache remote webrtc track without AddTrackRequest to
let client send publisher offer before AddTrackRequest response

* go mod

* clean code
2024-08-23 18:38:32 +08:00
Raja Subramanian
9a6b4c6956 Send signal response for non-error cases also. (#2926)
* Send signal response for non-error cases also.

Gated by protocol version 15 or higher for sending OK response.

* explicit reason init
2024-08-12 22:25:34 +05:30
cnderrauber
a8730b04b8 move TrackSubscribed trigger to MediaSubscription (#2916) 2024-08-07 22:30:52 +08:00
Raja Subramanian
13ee1aca28 Delay getting forwarder state till migration is complete. (#2909) 2024-08-06 12:45:46 +05:30
Raja Subramanian
8c323330b6 Store subscriber forwarder state (#2907)
* Forwarder state for migrating participant.

* clean up

* update protocol deps

* cleanup debug
2024-08-05 21:13:07 +05:30
David Zhao
b1fbca066f Send the correct room closed reason to clients (#2901)
* Send the correct room closed reason to clients

* update go mod
2024-08-01 19:40:06 -07:00
Raja Subramanian
f7695cf9f4 Move track update methods to LocalParticipant. (#2868) 2024-07-16 08:09:46 +05:30
Raja Subramanian
faa66d1138 Make sender report pass through an option. (#2861)
Enabled by default.

Also, tweak the long term propagation delay a bit. The first propagation
delay itself was too high and the long term initialized with a high
value. Prevent that and also ensure large negtaives do not have an
effect by using a lower bound of 0. Lower bound of 0 is okay as the main
purpose is to track sustained high positive values.
2024-07-15 11:27:31 +05:30
Raja Subramanian
09e3aef859 Check size limits on metadata and name set from client. (#2850)
* Send error response when update metadata fails.

Keeping it simple for the first implementation.
- Send error response only if request_id != 0
- Two kinds of errors notified
  o does not have permissions - NOT_ALLOWED
  o attributes exceeds size limits -  INVALID_ARGUMENT

* Check size limits on metadata and name set from client.

Added a name length limit also.

* check name length in service update participant path also

* limit check in limit config

* update protocol

* longer keys
2024-07-12 09:57:17 +05:30
Raja Subramanian
b8847cd8e9 Send error response when update metadata fails. (#2849)
Keeping it simple for the first implementation.
- Send error response only if request_id != 0
- Two kinds of errors notified
  o does not have permissions - NOT_ALLOWED
  o attributes exceeds size limits -  INVALID_ARGUMENT
2024-07-10 23:26:44 +05:30
cnderrauber
7a5c90dc01 log non-trickle candidate in details (#2832)
* log non-trickle candidate in details

* fix test
2024-07-03 14:16:58 +08:00
Paul Wells
e511464d3d add handler interface to receive agent worker updates (#2830)
* add handler interface to receive agent worker updates

* cleanup
2024-07-02 13:11:08 -07:00
David Zhao
7a774cc82a Support for participant attributes (#2806)
* Support for participant attributes

* move metadata setters to LocalParticipant

* address feedback

* forward error

* update go mod

* update attributes first
2024-06-19 23:14:19 -07:00
Raja Subramanian
ef838e4fa2 Indicate if track is expectd to be resumed in onClose callback. (#2800)
That is the main change. Changed variable name to `isExpectedToResume`
everywhere to be consistent.

Planning to use the callback value in relays to determine if the down
track should be closed or switched to a different up track.
2024-06-17 23:51:00 +05:30
Raja Subramanian
d9910ead61 Add back TrafficStats used by cloud. (#2736) 2024-05-27 13:43:20 +05:30
Raja Subramanian
991ad5a0ea Remove unused traffic load (#2734) 2024-05-27 10:53:23 +05:30
Raja Subramanian
80a4d021b9 Add simulate leave request close reason (#2713) 2024-05-10 20:02:25 +05:30
cnderrauber
cfa82c6ad4 increase protocol version to support optional publisher datachannel (#2693) 2024-04-28 15:09:24 +08:00
Raja Subramanian
79f6506553 Clean up UpdateVideoLayers (#2685) 2024-04-26 12:21:40 +05:30
Raja Subramanian
90b47424b5 Handle UpdateLocalAudioTrack and UpdateLocalVideoTrack. (#2684)
* Handle UpdateLocalAudioTrack and UpdateLocalVideoTrack.

- Update the TrackInfo
- NOTE: populating Stereo and DisableDtx fields although there are
  features now.
- The audio features in UpdateLocalAudioTrack is applied as is,
  i. e. the update has the latest set of features.
- Emits a track update which will broadcast a participant update.

TODO:
-----
- Telemetry event with track update?

* update deps
2024-04-26 12:06:35 +05:30