Commit Graph

66 Commits

Author SHA1 Message Date
Raja Subramanian
a19ca69f5f Prevent stats update if the deltas are empty (#619)
* Prevent stats update if the deltas are empty

* increase force interval

* static check

* Change max delay to 30 seconds
2022-04-18 22:51:34 +05:30
Raja Subramanian
a98d955284 Delta stats throughout (#615)
* Use delta stats throughout and avoid calculating deltas in telemetry

* Fix a few things after testing

* Remove debug

* Fix tests

* delete instead of setting to nil

* Point to the latest protocol
2022-04-16 21:11:32 +05:30
Raja Subramanian
92009b6428 Consistently stop tickers (#593) 2022-04-05 20:42:06 +05:30
David Colburn
0b8a180554 Code inspection (#581)
* Code inspection

* fix [4]int64 conversiong
2022-03-30 13:49:53 -07:00
shishirng
a6bb59b159 handle deltas being null leading to crash (#567)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-03-25 19:18:32 -04:00
shishirng
579d3d1a19 Check if current stats < prev and guard against underflow (#563)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-03-23 15:16:59 -04:00
Raja Subramanian
076eb1c8ae Dampen stream allocator (#551)
* WIP commit

* WIP commit

* WIP commit

* format

* NACK window

* Remove layer when it is expected to stop

* Remove debug
2022-03-22 22:23:22 +05:30
Mathew Kamkar
cf63da2e64 prometheus livekit_room_total node_id label 2022-03-21 16:43:01 -07:00
David Zhao
f14c452f8c Telemetry and webhook improvements. (#535)
* Telemetry and webhook improvements.

* avoid blocking on telemetry channel - increase channel size and drop when full
* send ParticipantJoined webhook when fully joined (i.e. on ParticipantActive)
* send TrackPublished & TrackUnpublished webhooks
* increase number of parallel webhook workers to 50

* update protocol
2022-03-18 23:20:33 -07:00
Mathew Kamkar
cac6d22a72 store cpu load in node stats (#524)
* store cpu load in node stats

* num cpus uint32

* cpu load selector test

* dep update
2022-03-16 14:51:22 -07:00
shishirng
cd2a7c2447 Telemetry: send video layers in TrackPublishedUpdate event (#500)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-03-10 14:49:01 -05:00
shishirng
c34b907d58 Add checks to prevent bytes/packet counts from going -ve (#499)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-03-09 16:51:23 -05:00
shishirng
57ecec73d7 Send participantInfo on participant left event to store identity (#498)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-03-09 14:35:01 -05:00
shishirng
c3a3fb569d add track publisher info in track subscribed event (#473)
* add track publisher info in track subscribed event

Signed-off-by: shishir gowda <shishir@livekit.io>

* update protocol ver

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-28 13:48:02 -05:00
Raja Subramanian
2706dc130f Replace sync/atomic usage with uber/atomic (#471) 2022-02-28 09:57:17 +05:30
Raja Subramanian
0170cc1cb6 Staticcheck (#464)
Using `go get -u honnef.co/go/tools/cmd/staticcheck`
Uneaarthed a couple of real bugs
2022-02-25 12:04:08 +05:30
David Colburn
20f21cce2b Egress (#455)
* egress updates

* pass egressInfo to delete

* update typefakes

* export StartEgress

* update protocol

* new rpc, rename stores

* add json tag

* update tests

* update protocol
2022-02-24 14:57:14 -08:00
shishirng
3e7fae96ea Add telemetry method to capture max video_quality (#457)
* Add telemetry method to capture max video_quality

Signed-off-by: shishir gowda <shishir@livekit.io>

* Telemetry fakes

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update go mod dep

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-22 19:08:49 -05:00
shishirng
7fcb887eb8 use delta bytes in window to identify max layer (#442)
total_bytes is aggregate, when we switch from higher layer to lower
layer, it takes time for lower layers total_bytes to catch up to
stopped higher layers

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-17 15:15:10 -05:00
shishirng
c534099e3a fix connection_scores not being sent to telemetry during delta calc (#439)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-16 19:31:59 -05:00
shishirng
8680f6fd23 Send trackInfo object in TRACK_SUBSCRIBED event (#431)
Need track details in subscribed events

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-10 16:48:16 -05:00
shishirng
e96e8e7f97 Clean up closed tracks stats and handle -ve packet_lost (#430)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-10 11:21:17 -05:00
shishirng
6f7e6c4556 Compute delta stats to send downstream (#426)
* Compute delta stats to send downstream

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update tests: total_packets should be diff between 2 packets

First packet was 1, second was 4. diff should be 3

Signed-off-by: shishir gowda <shishir@livekit.io>

* If there are no videoLayers, do not sent in Stats

For audio and Downstream tracks, we do not  get layers

Signed-off-by: shishir gowda <shishir@livekit.io>

* Use prev Max layer for current delta and update layer info for next round
2022-02-09 20:45:53 -05:00
Raja Subramanian
222b02aa73 RTT (#420)
* Consolidating PLI throttle

Use the throttler in `sfu.WebRTCReceiver`.

Does change shape of config object.

* Move PLIThrottleConfig to sfu.WebRTCReceiver

* fix test compile

* Cleaning up unused stuff

* improve readability

* RTT

- Calculate down track RTT using RTCP Receiver report
- Surface it back to the participant
- Participant updates all its published trackes
  (throttled to limit update to once in 5 seconds)
- That propagates to all the upstream sfu.Buffer and the nacker.
  So, we will have RTT throttled NACKs.

* rtt callback
2022-02-09 09:34:40 +05:30
Raja Subramanian
36289bbca7 FPS (#410)
* WIP commit

* WIP commit

* WIP commit

* WIP commit

* WIP commit

* WIP commit

* Clean up

* Clean up

* Store RTT in stats

* spelling mistake

* Make tests compile

* Fix test compilation error

* fix tests

* clone

* latest protocol
2022-02-08 12:53:14 +05:30
David Zhao
a6eb4290d3 Generate telemetry stubs (#412) 2022-02-07 23:15:24 -08:00
shishirng
32b56e0fd6 Add ParticipantActive telemetry method (#411)
* Add ParticipantActive telemetry method

Signed-off-by: shishir gowda <shishir@livekit.io>

* fix test

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update go mod

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-02-07 17:23:39 -05:00
David Colburn
7bbd238188 clean up logs and imports (#400) 2022-02-03 14:20:19 -07:00
David Colburn
3d132730f9 replace entire nodeStats object (#393) 2022-01-31 17:09:36 -07:00
shishirng
1e156025b4 Store client meta on participant join (#380)
* Store client meta on participant join

capture region, time_to_connect, ip, node

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update proto dep

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-01-27 15:44:03 -05:00
shishirng
26eea78b54 Telemetry connection scores (#377)
* octets - total bytes needs to be uint64

uint32 wraps at 4GB

Signed-off-by: shishir gowda <shishir@livekit.io>

* Cleanup stats handler to use connectionQuality stats

remove per packet rtcp handlers, buffer stats

* cleanup connection stats

* Update mediatrack to store rtcp stats in connection stats

* Update downstream handling of connection stats and telemetry

* Update telemetry tests

Signed-off-by: shishir gowda <shishir@livekit.io>

* Misc fixes

Signed-off-by: shishir gowda <shishir@livekit.io>

* Minor fix to avoid accessing buffer before its allocated

Signed-off-by: shishir gowda <shishir@livekit.io>

* start updateStats worker in AddReciever()

Signed-off-by: shishir gowda <shishir@livekit.io>

* Use previous score to calculate avg scores

* Restructure connectionStats

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-01-27 11:24:54 -05:00
shishirng
56ebd521f9 Telemetry capture published track updates (#367)
* Telemetry capture published track updates

Signed-off-by: shishir gowda <shishir@livekit.io>

* Updated OnVideoLayerUpdate to take slice of layers

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update proto dep

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-01-24 14:38:04 -05:00
shishirng
829e183863 pass client info object in participant joined request (#358)
* pass client info object in participant joined request

Signed-off-by: shishir gowda <shishir@livekit.io>

* Update tests

Signed-off-by: shishir gowda <shishir@livekit.io>
2022-01-20 15:29:34 -05:00
David Colburn
5bea9debb7 Code cleanup (#353) 2022-01-19 02:13:06 -08:00
shishirng
cf830191e3 Add client info to participant_joined event (#337)
Signed-off-by: shishir gowda <shishir@livekit.io>
2022-01-13 17:45:21 -05:00
Raja Subramanian
995c1e8676 type aliases -> type definitions (#312)
* type alias -> type definition

* participantSid -> participantID in a comment

* Update protocol
2022-01-03 12:43:58 +05:30
Raja Subramanian
3a9009ae12 type definition of room name (#311)
* WIP commit

* update protocol

* Fixing a test and catching one place where casting was missed

* Fix one more spot which need conversion from livekit.RoomName -> string

* do not covert list
2022-01-02 16:49:16 +05:30
shishirng
007177d095 room_sid to room_id update in analytics proto (#306)
Signed-off-by: shishir gowda <shishir@livekit.io>
2021-12-30 15:26:14 -05:00
Artur Shellunts
6f84b36ccf Use RTT field in analytics stats (#304)
* Update protocol to v0.11.7

* Use RTT field in analytics stats

Instead of Delay
2021-12-30 13:13:40 +01:00
Raja Subramanian
07db1ba726 Some more files with types (#302) 2021-12-30 16:43:20 +05:30
Artur Shellunts
1b66fe1e23 Fix handling of PacketLost (#296)
They come accumulated already.
2021-12-30 11:06:00 +01:00
Artur Shellunts
1fa194297f Implement calculating of RTT for down tracks (#300)
* Implement calculating of RTT for down tracks
* Fix code style issues
2021-12-29 19:58:41 +01:00
Artur Shellunts
b744a9c2ba Implement event loop for telemetry service (#297)
It allows all actions/events to run in the same go routine.
Therefore no synchronization primitives are needed inside
telemetry service implementation.
2021-12-29 19:51:12 +01:00
Artur Shellunts
828d490755 Imlement analytics per track statistics (#281)
New tests for telemetry + implementation of per track statistics
2021-12-29 19:29:24 +01:00
Artur Shellunts
2209edce20 Make TelemetryService testable (#276)
* Make TelemetryService testable

Timer is extracted for better testability of telemetryservice.

Divided TelemetryService to internal part that:
- contains business logic
- does not contain timer
- and therefore testable

and external part that:
- does not contain business logic
- contains timer to send analytics every 10 seconds for all participants.
- does not need tests

* Add Test_AnalyticsSentWhenParticipantLeaves

* Fix test
2021-12-28 12:54:56 +01:00
David Zhao
87a799bae2 Include unique id and timestamp with webhook events (#291)
Resolves #230
2021-12-27 23:33:06 -08:00
shishirng
2e4ccd2577 Send client sdk type when participant joins in telemetry (#275)
Signed-off-by: shishir gowda <shishir@livekit.io>
2021-12-21 15:35:27 -05:00
David Colburn
0b71a46239 missed some typos 2021-12-17 13:25:41 -08:00
David Colburn
faa870de3d Move callbacks out of messageRouter (#269)
* move callbacks out of messageRouter

* OCD

* more OCD

* fix forwarder test

* even more OCD

* maximum OCD

* package name collision, copy lock by value
2021-12-17 13:19:23 -08:00
Artur Shellunts
0d229388ec Telemetry test (#258)
* Add first test for telemetry

- First test for telemetry package that tests totalBytes field in downstream stats.
- Add fake_analytics_service
2021-12-14 14:13:21 +01:00