Commit Graph

152 Commits

Author SHA1 Message Date
Raja Subramanian
14321f21bf Make OpsQueueParams to make it easier to understand args. (#2578) 2024-03-14 10:27:24 +05:30
Paul Wells
ad341d41f5 start telemetry participant worker to collect signal stats (#2538)
* start telemetry participant worker to collect signal stats

* format

* resolve room

* tidy
2024-03-03 02:47:51 -08:00
Denys Smirnov
f5eb6c8a95 Update usage of core.Fuse. (#2519) 2024-02-28 03:48:58 +02:00
Raja Subramanian
7649e4ffab Post data and signal stats once in 5 minutes (#2518) 2024-02-27 15:45:32 +05:30
Paul Wells
e5b8e25064 use shared psrpc utils (#2506)
* use shared psrpc utils

* fix

* deps
2024-02-24 00:38:49 -08:00
Raja Subramanian
5ac5bd236a Let track events go through after participant close. (#2487)
* Let track events go through after participant close.

Also, reducing lock scope in telemetry service.

* use shadow
2024-02-17 13:40:07 +05:30
Mathew Kamkar
7508560fde larger buckets for jitter prometheus histogram (#2468) 2024-02-09 12:09:51 -08:00
Raja Subramanian
b71d373f4a Use Deque in ops queue. (#2418)
* Use Seque in ops queue.

Standardizing some uses
- Change OpsQueue to use Deque so that it can grow/shrink as necessary and
  need not worry about channel getting full and dropping events.
- Change StreamAllocator and TelemetryService to use OpsQueue so that
  they also need not worry about channel size and overflows.

* Address feedback

* delete obvious comment

* clean up
2024-01-28 13:48:30 +05:30
Paul Wells
c726cbf2ba increase max session start time bin size (#2380) 2024-01-12 03:49:23 -08:00
Paul Wells
2fe2a9c9f2 add session start time metric (#2377) 2024-01-11 23:23:51 -08:00
shishirng
3770fbce64 Analytics: send local node room state/info (#2335)
* Analytics: send local node room state/info

Signed-off-by: shishir gowda <shishir@livekit.io>
2023-12-22 18:59:04 -05:00
Raja Subramanian
dcff75a516 Record number of data messages in prometheus. (#2282) 2023-12-01 16:10:57 +05:30
Raja Subramanian
53542b09a0 Participant traffic load. (#2262)
* Participant traffic load.

Capturing information about participant traffic
- Upstream/Downstream
- Audio/Video/Data
- Packets/Bytes

This captures a notion of how much traffic load a participant is
generating.

Can be used to make allocation decisions.

* Clean up

* SIP patches

* reporter goroutine

* unlock

* move traffic stats from protocol

* check type
2023-11-26 23:05:00 +05:30
Raja Subramanian
56dd399684 Use a worker to report signal/data stats. (#2260)
* Use a worker to report signal/data stats.

Was checking if reporting is needed on every update.
The check is wasted work if volume of signal/data messages is high
as reporting happens only once in 10 seconds.

Changing to a worker based on a timer. And also aligning with
telemetry reporting interval which defaults to 30 seconds.

* Remove unused constant
2023-11-22 11:47:15 +05:30
Paul Wells
f4a984d446 preallocate prometheus packet counters (#1942) 2023-08-08 01:06:14 -07:00
David Zhao
981fb7cac7 Adding license notices (#1913)
* Adding license notices

* remove from config
2023-07-27 16:43:19 -07:00
Benjamin Pracht
552e3758d5 Add IngressUpdated event (#1775) 2023-06-16 10:58:49 -07:00
David Zhao
f71544e27a Do not send ParticipantJoined webhook if connection was resumed (#1795)
* Do not send ParticipantJoined webhook if connection was resumed

* isResume -> isMigration
2023-06-15 15:39:04 -07:00
shishirng
2dd4e1365b Send EgressUpdated event (#1792)
Signed-off-by: shishir gowda <shishir@livekit.io>
2023-06-14 18:56:07 -04:00
David Zhao
7e5a7ae79f Fixed windows build (#1768) 2023-06-04 00:17:25 -07:00
Benjamin Pracht
e7879a46fc Add ingress telemetry support (#1763) 2023-06-02 17:38:19 -07:00
David Zhao
956735ae05 Fix node stats updates on Windows (#1748)
Because we aren't able to get CPU count/load info on Windows, they are
stubbed out to return placeholders. This restores compatibility to run
on Windows.
2023-05-29 10:53:08 -07:00
shishirng
3de51181ec Fix setting minscore - initialized to 0 (#1725)
Signed-off-by: shishir gowda <shishir@livekit.io>
2023-05-19 11:00:32 -04:00
shishirng
2e93d386fe send min/median connection score along with avg (#1720)
* send min/median connection score along with avg
* guard against divide by zero for avg score calculation
* update median calculation

Signed-off-by: shishir gowda <shishir@livekit.io>
2023-05-18 13:50:54 -04:00
Raja Subramanian
a085afc6ee Send quality stats to prometheus. (#1708) 2023-05-12 09:44:03 +05:30
David Colburn
2ccee369a6 update notifier (#1702) 2023-05-09 20:52:22 -07:00
David Colburn
ab6c994db4 update protocol/psrpc (#1643)
* update protocol/psrpc

* metadata references
2023-04-21 12:43:20 -07:00
David Zhao
40ceddd18b Integrate QueuedNotifier, fixes out-of-order delivery (#1615) 2023-04-15 01:20:23 -07:00
Paul Wells
6636e37664 add prometheus psrpc metrics observer (#1571)
* add prometheus psrpc metrics observer

* record rpc error counts

* update psrpc

* update protocol
2023-04-05 03:50:43 -07:00
David Colburn
108b251045 egress updated webhook (#1555) 2023-03-27 16:34:44 -07:00
David Colburn
191a9e8014 update core to 0.0.5 (#1540)
* update core

* sort imports

* fix typos

* redundant types
2023-03-22 16:53:23 -07:00
David Zhao
5ff72a99b9 Report publish & subscribe RTPStats as Telemetry events (#1506) 2023-03-10 10:28:54 -08:00
shishirng
8856ce6422 Bump up interval for sending telemetry stats to 30 seconds (#1430)
Signed-off-by: shishir gowda <shishir@livekit.io>
2023-02-16 15:53:58 -05:00
Dan McFaul
1848a21eda add configurable environment value (#1421)
* add configurable prometheus env label

* Update pkg/config/config.go

Co-authored-by: Mathew Kamkar <578302+matkam@users.noreply.github.com>

* Update cmd/server/main.go

Co-authored-by: Mathew Kamkar <578302+matkam@users.noreply.github.com>

* Update config-sample.yaml

Co-authored-by: Mathew Kamkar <578302+matkam@users.noreply.github.com>

* set config.Environment value to dev when in dev mode

* be more precise for config-sample

---------

Co-authored-by: Mathew Kamkar <578302+matkam@users.noreply.github.com>
2023-02-15 14:41:44 -07:00
Mathew Kamkar
937256d89e don't error when get tc stats fails (#1386) 2023-02-10 10:05:45 -08:00
David Zhao
2851a8ac98 Improved robustness of subscription stack (#1382)
UpdateSubscription had a shortcoming where when it couldn't find the
participant, it ignored the request.

This PR further removes the reliance of current publisher state from
subscribers.
- SubscribeToTrack only takes in a trackID
- Introduced RoomTrackManager to maintain all published tracks to a room
- Added TrackUnpublished event to clearly indicate when a track has been removed
- SubscribeRequested event no longer include information about the publisher
2023-02-06 18:08:26 -08:00
cnderrauber
8b6dab780c Add reconnect reason and signal rtt calculation (#1381)
* Add connect reason and signal rtt calculate

* Update protocol

* solve comment
2023-02-06 11:12:25 +08:00
David Zhao
b023c531c2 Fix incorrect unsubscribed track telemetry (#1350)
- only log unsubscribe on close if Track was actually bound
- update subscribed counters even if a failure had been logged before
2023-01-30 10:16:21 -08:00
Paul Wells
52fd0a641b adjust jitter histogram buckets (#1347)
* adjust jitter histogram buckets

* typo
2023-01-29 22:45:32 -08:00
David Zhao
2fa46e2df4 Retry initial connection attempt should it fail (#1335)
Sometimes the initial selected node could fail. In that case, we'll give it a few more attempts to locate a media node for the session instead of failing it after the first try.
2023-01-25 22:59:57 -08:00
David Zhao
cd6b8b80b9 feat: SubscriptionManager to consolidate subscription handling (#1317)
Added a new manager to handle all subscription needs. Implemented using reconciler pattern. The goals are:

improve subscription resilience by separating desired state and current state
reduce complexity of synchronous processing
better detect failures with the ability to trigger full reconnect
2023-01-24 23:06:16 -08:00
Dan McFaul
9e3ca1e989 adding rtc_init stat (#1316)
* adding rtc_initiated stat

* clean up signal and rtc init/connected

* update naming and break out stats update funcs

* update protocol dependency
2023-01-23 12:49:15 -07:00
Paul Wells
1ef7c46fd7 publish stream stats to prometheus (#1313)
* add prometheus stats for rtt/jitter/packet loss

* add track source to metrics

* better packet loss bins

* add track type to metrics

* remove source from AnalyticsStat

* regenerate telemetry service fake

* compute loss from per stream packet count
2023-01-19 19:37:15 -08:00
Benjamin Pracht
edc39da0b1 Add TwirpRequestStatusReporter twirp server hook to count requests (#1309) 2023-01-18 11:53:20 -08:00
David Zhao
732309a8c1 Added track success & muted events (#1308)
Related to livekit/protocol#273

This PR adds:
- ParticipantResumed - for when ICE restart or migration had occurred
- TrackPublishRequested - when we initiate a publication
- TrackSubscribeRequested - when we initiate a subscription
- TrackMuted - publisher muted track
- TrackUnmuted - publisher unmuted track
- TrackPublish/TrackSubcribe events will indicate when those actions have been successful, to differentiate.
2023-01-15 15:40:20 -08:00
Dan McFaul
4d6f0cd0f7 Stats collect v2 (#1291)
* initial commit

* add correct label

* clean up

* more cleanup on adding stats

* cleanup

* move things to pub and sub monitors, ensure stats are correctly updated

* fix merge conflict

* Fix panic on MacOS (#1296)

* fixing last feedback

Co-authored-by: Raja Subramanian <raja.gobi@tutanota.com>
2023-01-11 14:49:50 -07:00
Raja Subramanian
1db218a5b1 Fix panic on MacOS (#1296) 2023-01-11 10:08:56 +05:30
Mathew Kamkar
7c970da974 add memory used and total to node stats (#1293)
* add memory used and total to node stats

* raja review: consistency

* update protocol
2023-01-10 12:32:04 -08:00
shishirng
dceb624b83 Send participant object in telemetry on Active event (#1282)
Signed-off-by: shishir gowda <shishir@livekit.io>
2023-01-04 08:55:49 -05:00
David Zhao
c1d7dbd4fc Tweaks to prometheus participant counter (#1240)
* Tweaks to prometheus participant counter

Ensure that we don't miss adding a count in migration scenarios

* avoid nil ICEConfig
2022-12-19 14:30:14 -08:00