# LiveKit RTC Protocol

LiveKit uses a WebSocket connection for signaling. Each client should start the RTC session by connecting to `/rtc`. Once connected, LiveKit uses protocol buffers to exchange messages between the server and client. The messages are detailed in [rtc.proto](../proto/rtc.proto). Client always sends `SignalRequest` and will receive `SignalResponse` from the server.

Like with other requests to LiveKit, the WebSocket connection needs to be [authenticated](authentication.md).

## Joining a room

When the WebSocket connection is established, server will first provide information about the joined room, and waits for client to perform the handshake. Flow:

1. server sends a `JoinResponse`, which includes room information, the current participant's data, and information about other participants in the room.
2. client sends `offer`, and sets its local description.
3. server accepts connection and sends an `answer`.
4. ICE connectivity is established
5. server notifies other participants of the new participant
6. server subscribes new client to existing tracks in the room

## Handling negotiations

Negotiation is use to describe the offer/answer process in WebRTC. After the initial offer/answer process to establish the connection, negotiations are needed whenever track changes are made to the current session. Because WebRTC is a peer to peer protocol, negotiations can be initiated by either party as needed.

### Server initiated negotiations

LiveKit server will need to negotiate the connection with each participant when the participant subscribes to new tracks.

1. server sends an `offer` to client
2. client calls `setRemoteDescription`, and create an `answer` for the server
3. server receives answer and concludes the negotiation cycle

### Client initiated negotiations

Because each of the peers could initiate a negotiation at any point, this creates a synchronization headache. Depending on timing, there could be a race condition in which the peers would receive unexpected responses from the other side. This is called [glare](https://tools.ietf.org/agenda/82/slides/rtcweb-10.pdf).

In LiveKit, we've added a layer of synchronization so that negotiations are more deterministic. The server is the authority determining who should be issuing offers in a negotiation cycle. When the client wants to issue an offer to the server, it needs to follow this flow.

1. client's onnegotiationneeded callback is triggered
2. client sends server a `negotiate` (`NegotiationRequest`)
3. server determines when it's ready for a new offer, and notifies client with `NegotiationResponse`
4. when client receives `negotiate` (`NegotiationResponse`), giving it permission to send an offer. Generates a new offer and sends it.
5. server accepts the offer and sends an `answer`
6. client calls `setRemoteDescription` with that session description.

now the negotiation is complete and track changes have been applied.

## Receiving subscribed tracks

When the client is subscribed to other participants tracks, it needs to coordinate the metadata (sent in `ParticipantInfo.tracks`) with the WebRTC `ontrack` callback. For a audio or video track, `TrackInfo.sid` will match `MediaStreamTrack.id`. For a data track, `TrackInfo.sid` will match `RTCDataChannel.id.toString()`.

## Publishing tracks

To publish a track, client needs to first notify server with track metadata before actually adding the track.

1. client sends `add_track` with the `cid` field set to the client generated id for the track. (`MediaStreamTrack.id` or name of data track)
2. server adds it to a list of pending tracks and sends back `track_published`
3. client receives track_published, then:
   a. for a audio/video track, client adds the `MediaStreamTrack` to its `PeerConnection`
   b .for a data track, create the channel with `PeerConnection.createDataChannel`

The above sequence will trigger `onnegotiationneeded`

## Muting/Unmuting tracks

Muting the track requires setting `MediaStreamTrack.enabled` to false, and then notifying the server with a `mute` message.

Disabling the media track simply will not remove it from the stream, but to [send packets with 0 values periodically](https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/enabled). Because this will be a waste of bandwidth, LiveKit has optimizations to skip packet forwarding to other participants when the track is muted.

Follow similar steps for unmuting