Raja SubramanianandGitHub
fdde44d926
There are cases where the RTP time stamp does not increment acros mute/unmute. Seems to happen fairly consistently with React Native clients. Something like the following happens - Track is progressing - Mute at `t = x`, assume RTP time stamp at the point is `y` and RTP clock rate is `z`. - Through mute, more RTCP sender reports come in from publisher and the RTP time stamp in those reports are progressing at expected rate of `z` RTP clock ticks per second. - Forwarding path uses those sender reports from publisher to build the sender report for subscribers. - Unmute happens at `t = x + a` seconds. - Ideally packets coming in after that, should have a time stamp of `y + (a * z)`, but they tend to have something a little bit more than `y`. - RTCP sender reports also have a time stamp that goes back. SFU ignores these. - Mean while the forwarding path has adjusted to the new RTP time stamp base and it has calculated a TS offset (from publisher -> subscriber). Effectively, that offset comes out close to `(a * z)`, i. e. jump corresponding to the mute interval. - When it is time to send a RTCP sender report to subscriber, the old sender report from publisher is used (as intervening ones from publisher were rejected because time stamp is moving backwards). The problem is that the old report is used with new offset. So, it looks like time stamp jumped ahead by `a` seconds. Address it by storing time stamp offset at the time of receiving the publisher side sender report. And use that while sending subscriber side sender report. There are very edge cases where this can get mismatched, but should be rare. Hopefully, this should prevent unnecessary jumps in time stamp in RTCP sender report to subscribers.
LiveKit: Real-time video, audio and data for developers
LiveKit is an open source project that provides scalable, multi-user conferencing based on WebRTC. It's designed to provide everything you need to build real-time video audio data capabilities in your applications.
LiveKit's server is written in Go, using the awesome Pion WebRTC implementation.
Features
- Scalable, distributed WebRTC SFU (Selective Forwarding Unit)
- Modern, full-featured client SDKs
- Built for production, supports JWT authentication
- Robust networking and connectivity, UDP/TCP/TURN
- Easy to deploy: single binary, Docker or Kubernetes
- Advanced features including:
- speaker detection
- simulcast
- end-to-end optimizations
- selective subscription
- moderation APIs
- end-to-end encryption
- SVC codecs (VP9, AV1)
- webhooks
- distributed and multi-region
Documentation & Guides
Live Demos
- LiveKit Meet (source)
- Spatial Audio (source)
- Livestreaming from OBS Studio (source)
- AI voice assistant using ChatGPT (source)
Ecosystem
- Agents: build real-time multimodal AI applications with programmable backend participants
- Egress: record or multi-stream rooms and export individual tracks
- Ingress: ingest streams from external sources like RTMP, WHIP, HLS, or OBS Studio
SDKs & Tools
Client SDKs
Client SDKs enable your frontend to include interactive, multi-user experiences.
| Language | Repo | Declarative UI | Links |
|---|---|---|---|
| JavaScript (TypeScript) | client-sdk-js | React | docs | JS example | React example |
| Swift (iOS / MacOS) | client-sdk-swift | Swift UI | docs | example |
| Kotlin (Android) | client-sdk-android | Compose | docs | example | Compose example |
| Flutter (all platforms) | client-sdk-flutter | native | docs | example |
| Unity WebGL | client-sdk-unity-web | docs | |
| React Native (beta) | client-sdk-react-native | native | |
| Rust | client-sdk-rust |
Server SDKs
Server SDKs enable your backend to generate access tokens, call server APIs, and receive webhooks. In addition, the Go SDK includes client capabilities, enabling you to build automations that behave like end-users.
| Language | Repo | Docs |
|---|---|---|
| Go | server-sdk-go | docs |
| JavaScript (TypeScript) | server-sdk-js | docs |
| Ruby | server-sdk-ruby | |
| Java (Kotlin) | server-sdk-kotlin | |
| Python (community) | python-sdks | |
| PHP (community) | agence104/livekit-server-sdk-php |
Tools
- CLI - command line interface & load tester
- Docker image
- Helm charts
Install
Tip
We recommend installing LiveKit CLI along with the server. It lets you access server APIs, create tokens, and generate test traffic.
The following will install LiveKit's media server:
MacOS
brew install livekit
Linux
curl -sSL https://get.livekit.io | bash
Windows
Download the latest release here
Getting Started
Starting LiveKit
Start LiveKit in development mode by running livekit-server --dev. It'll use a placeholder API key/secret pair.
API Key: devkey
API Secret: secret
To customize your setup for production, refer to our deployment docs
Creating access token
A user connecting to a LiveKit room requires an access token. Access tokens (JWT) encode the user's identity and the room permissions they've been granted. You can generate a token with our CLI:
livekit-cli create-token \
--api-key devkey --api-secret secret \
--join --room my-first-room --identity user1 \
--valid-for 24h
Test with example app
Head over to our example app and enter a generated token to connect to your LiveKit server. This app is built with our React SDK.
Once connected, your video and audio are now being published to your new LiveKit instance!
Simulating a test publisher
livekit-cli join-room \
--url ws://localhost:7880 \
--api-key devkey --api-secret secret \
--room my-first-room --identity bot-user1 \
--publish-demo
This command publishes a looped demo video to a room. Due to how the video clip was encoded (keyframes every 3s), there's a slight delay before the browser has sufficient data to begin rendering frames. This is an artifact of the simulation.
Deployment
Use LiveKit Cloud
LiveKit Cloud is the fastest and most reliable way to run LiveKit. Every project gets free monthly bandwidth and transcoding credits.
Sign up for LiveKit Cloud.
Self-host
Read our deployment docs for more information.
Building from source
Pre-requisites:
- Go 1.20+ is installed
- GOPATH/bin is in your PATH
Then run
git clone https://github.com/livekit/livekit
cd livekit
./bootstrap.sh
mage
Contributing
We welcome your contributions toward improving LiveKit! Please join us on Slack to discuss your ideas and/or PRs.
License
LiveKit server is licensed under Apache License v2.0.
| LiveKit Ecosystem | |
|---|---|
| Real-time SDKs | React Components · JavaScript · iOS/macOS · Android · Flutter · React Native · Rust · Python · Unity (web) · Unity (beta) |
| Server APIs | Node.js · Golang · Ruby · Java/Kotlin · Python · Rust · PHP (community) |
| Agents Frameworks | Python · Playground |
| Services | Livekit server · Egress · Ingress · SIP |
| Resources | Docs · Example apps · Cloud · Self-hosting · CLI |