ParseManifest builds a router.Router rather than a slice of compiled templates,
Manifest aliases it, and Route keeps the template and the public flag the front
reads. Declared methods become a mask at build time, so a method no route could
declare masks to 0 and matches nothing, as an undeclared one already did.
Three behavior changes ride along.
Templates anchor as `\n?$`. A manifest declaring private /files/{p:path} before
public /files/{p} routed GET /files/x%0A to the public route here and to the
private one on the worker, because the path convertor refuses a newline where
str accepts it. The access gate reads the public flag of the route the front
picked, so it opened on a route starlette would not have selected.
slashAlternatePaths derives the escaped form from the transform it applied
rather than from the result's suffix. A request path of // trimmed to /, the
caller saw a trailing slash on that result, took the append branch, and handed
the worker a request line of /// against a decoded path of /.
The decoded path is capped at MaxPathLength, answering 414 above it. Matching
runs before anything sizes the request head, so net/http's own limit was the
only bound on what the matcher was handed.
A table too ambiguous to decide now dispatches rather than 404s: the worker's
own router still knows, and the preamble carries no route for it to be told
about. Nothing here knows whether that route is public, so the request needs a
grant. Dispatch keys on the match set, since a candidate can now join it
without resolving a route.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Manifest matching runs a compiled regex per route over a linear scan, once per
candidate registration, on the serving path. At the 256-route cap that is up to
256 RE2 executions per replica per request.
The router compiles a manifest into a compressed trie instead. Starlette
selects the first route declared that matches, so the trie cannot resolve
static before wildcard the way net/http's does: every node carries the lowest
route index in its subtree, the walk tracks the lowest full match found and
prunes any subtree that cannot improve on it, and inserting in declaration
order leaves edges and leaves sorted by that index with no sort pass.
Params are not segment-aligned - starlette allows /f/{name}.{ext}, a {p:path}
anywhere, and a float whose fraction backtracks - so the walk must search. An
edge is single when nothing below it can start with a byte its own convertor
could have consumed, which is a property of the target node and so is settled
at build time; a single edge takes its greedy run and descends once. Templates
whose params are segment-aligned are entirely single and never search. What
remains is bounded by a step budget, and exhausting it returns ResultOverBudget
rather than a route the cut-short search cannot vouch for.
Templates parse without regexp. Scanning the grammar by hand keeps a brace that
opens nothing well-formed as an ordinary literal, which is what starlette's
finditer does. Anchoring is `\n?$` rather than `$`: python's '$' matches before
one trailing newline where Go's does not, and [^/] accepts a newline where .
refuses one, so the two convertors diverge in opposite directions on a decoded
%0A.
The regex implementation moves into the tests as an oracle, carrying its own
parser so it references the hand-rolled scanner as well as the trie.
FuzzMatchAgainstOracle generates a table and a request and asserts the two
agree on both the winning template and the result. Matching allocates nothing,
asserted by AllocsPerRun on hit, 405, miss and on the searching path.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
StartWebTransport built the node's only HTTP/3 listener from inside AgentService:
it minted the TLS config, bound the UDP socket, assembled an h3 mux and returned
a stop closure LivekitServer held in agentWTStop. Every other listener is
constructed in NewLivekitServer and delegates to a service, and agents are the
first consumer of HTTP/3 rather than the last.
webtransport.go now carries the transport with no agent references:
WebTransportTLS, NewWebTransportServer, NewWebTransportHandler,
ListenWebTransport, and WithWebTransportServer/GetWebTransportServer, which put
the serving server in the request context so a route that upgrades is a
conventional http.Handler instead of a closure over the server. The listener is
built in NewLivekitServer beside the prometheus and debug servers, bound in Start
with the other listeners, and stopped once doneChan unblocks, so no stop closure
is held on LivekitServer. /agent is a route on its mux, as it is on the API mux.
UpgradeWebTransport unwraps the ResponseWriter before upgrading. webtransport-go
type-asserts it to http3.Settingser and http3.HTTPStreamer without checking and
negroni wraps the writer on every chain, so the h3 listener could not carry a
middleware chain at all. It now runs the same recovery, api-key auth and path
normalization as the TCP chain.
AgentService splits into AgentWSService and AgentWTService over the shared
AgentHandler, and the endpoint front becomes AgentEndpointService. wire builds
all four and threads one endpoint.Registry into the handler and the front.
NewAgentHandler takes the config and builds its own ServerInfo, so endpointsConfig
and singleAPIKey are set at construction and the psrpc server is registered
against a fully formed handler; it previously bound one whose embedded
*AgentHandler was still nil.
The listener's port and certificate move to a top-level webtransport config
block. The listener no longer reads agents.endpoints.disabled, which continues to
refuse endpoint registrations. ListenWebTransport binds one socket per bind
address, where the h3 listener ignored bind_addresses that every TCP listener
honoured.
NewWorkerRegisterer takes a variadic WorkerRegisterHandler, each handler reading
the request and filling its part of the response and the registration, replacing
the single EndpointSettingsFunc threaded through HandleRegister.
EndpointRegisterHandler is the endpoint declaration's turn, and
endpoint.NegotiateSettings holds the protocol negotiation the OSS and cloud
handshakes each carried a copy of.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Front is an http.Handler, so fallback, identity and singleKeyFallback were read
on every request with no synchronization while WithFallback, WithIdentity and
WithSingleKeyFallback wrote them on the live object. Each returned the same
pointer, so the chain read as though it built a value. Every call site happens
to run before the handler is mounted, so nothing races today, but nothing in
the type prevents it, and WithSingleKeyFallback took no argument and could only
be turned on. NewFront now takes FrontParams and configuration is fixed at
construction.
NewWorkerRegisterer takes its EndpointSettingsFunc directly, and
HandshakeAgentWorker takes one in place of a variadic of raw closures that
existed only to reach the setter it replaced.
Registration gains NewRegistration and RegistrationParams, moving the Draining
callback off an exported mutable field and folding SetSession into
construction. IsDraining absorbs the nil check at both call sites.
Access carried a three-state ladder as two bools, with "granted implies
credentialed" documented but unenforced. It is now an ordered AccessLevel, so
callers compare a rank rather than combining flags and the invariant holds by
construction.
Registry.Register returned an error that was always nil, with a dead branch at
each call site. Both registry maps and the per-registration session are
read-heavy, so they take RWMutex.
CopyBody returned two errors to separate a source failure from a destination
one. It returns one, wrapping a source failure in *SourceError, which is what
the caller discriminates on.
NewWebTransportServer took a callback to break the handler/server init cycle;
the caller assigns wt.H3.Handler after construction instead. StartWebTransport
reads Development off the service rather than taking a bool, and returns a nil
stop function where it starts no listener.
Smaller: slices.Sort for sort.Strings, for range for an unused counter, a nil
slice for regs[:0:0], p2c generic over its slice so the call site passes a
method expression rather than allocating a closure per request, and streamCode
deduplicated into wire.StreamCode so both peers map reset codes in one place.
Comments on the touched declarations drop remote behavior, migration narration
and contrastive framing, keeping the constraints and invariants.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A committed response that cannot be completed is torn down with
panic(http.ErrAbortHandler): the head is on the wire, so no status can report
the failure, and this is the only teardown net/http offers. HTTP/1.1 closes
without the final chunk and HTTP/2 sends RST_STREAM, so a short body cannot
read as whole.
bridge raised it from inside the response copy loop, through a helper whose
doc had to note that it does not return. It now reports a bridgeOutcome and
ServeHTTP performs the teardown at the handler boundary, where the constraint
that nothing may recover the sentinel is visible to a reader. The outcome
replaces the (done, retryable) pair, whose four combinations only ever carried
three states.
No behavior change.
The name is percent-encoded into its URL path segment, so it needs no charset.
Only "_", "." and ".." are refused, and only for workers declaring endpoints:
all three are unreserved, so no encoded form addresses them distinctly. "_"
now addresses a worker registered with an empty agent name, which the
implicit-dispatch default leaves unset.
Two escaping fixes the escaped-path split depends on:
- the trailing-slash alternate treated a decoded slash as literal, so
/known%2F normalized onto route /known and the worker was handed a path it
does not serve.
- RemoveDoubleSlashes trimmed URL.Path and left RawPath, which makes
EscapedPath() re-encode from Path and lose every escape in the request.
Request logs now carry the encoded path, since a decoded %0A forges a log line.
The front gated non-public routes on a bool that was true for any valid token
in the project, so a roomJoin token minted for an end user reached every
non-public endpoint the project's agents expose.
Replace the resolver's (apiKey, authenticated) pair with an Access value and
give it the agent and deployment from the URL, so the scope rules stay with
the grant in protocol/auth and this package consumes only the verdict.
Credentialed now separates a caller with no credential from one whose
credential lacks the grant: the first is challenged with 401, the second gets
403 and no WWW-Authenticate, since a retry cannot succeed.
A denied request still reaches the fallback. Another node's worker may declare
the same path public, and that node is the authority.
Replace the protobuf frame layer on exchange streams with a length-prefixed
StreamPreamble followed by opaque HTTP/1.1 bytes. QUIC already provides the
multiplexing and per-stream flow control, so the frame layer only re-encoded a
message both peers can already parse, and every SDK had to re-materialize HTTP
from it.
The front serializes a canonical request head from its own parsed
*http.Request and never forwards the client's bytes, which is what keeps
request smuggling out of the worker. Responses are read with
http.ReadResponse, so informational heads need no httptrace hook and the
conformance worker loses its HTTP reconstruction entirely.
Completion splits by outcome: a body ends by its own framing, a failure after
bytes have flowed travels in x-lk-completion / x-lk-error trailers, and a
failure before any byte resets the stream with an HttpStreamResetCode. REFUSED
stays distinct as the retry-safety signal, and the retry/idempotence rules are
unchanged.
The x-lk- prefix is reserved for this signalling: it is stripped from
client-supplied request headers so a caller cannot forge an outcome, and from
responses so it never reaches the end client. Exchange-stream targets are
split from the escaped path so a percent-encoded '?' or '/' cannot change
which resource the worker routes to.
Per-request state moves to attempt.go so the Front and attempt lifecycles stop
interleaving; the remaining files are grouped type-first with free helpers
last. Those moves are ordering only.
Remove the write-only Registration.Endpoints field (matching goes through
Manifest; the multi-node layer no longer replicates the raw route set). Add
Front-level fallback tests (fires with the resolved identity, declined -> 404
with a local worker or 503 without), and fix two stale comments.
Route presence like a reverse proxy keyed on (project, agent_name, deployment):
any of a deployment's workers is a candidate and the worker's own router returns
the real status (a 404 for a path it doesn't serve during a rolling deploy is
returned, not re-relayed). This removes the whole per-node route-advertisement
path: the cuckoo route filter and matcher (routematch.go), the miss-tagging /
retry-past-miss machinery, and the route-depth cap that only existed to bound
the filter. Text routing is now the same shape as voice job dispatch.
A slash mismatch (e.g. /hook/ for a route registered as /hook) is now rewritten
to the registered form and served directly, rather than 307-redirected. Webhook
clients often don't follow redirects, and a redirect from the final routing hop
resends the body and pays the whole routing path twice. Exact form is matched
first, so a route registered with a trailing slash is served as-is.
Use endpoint.CurrentProtocol and a named SessionCloseOK instead of bare
literals, factor the repeated 503 and header-copy blocks into helpers, and drop
a few restating test comments and a redundant assertion. No behavior change.
Replace the capsule mux / credit / attach-pool machinery with one WebTransport
(QUIC) session per worker: a control stream carries registration and status,
and each HTTP exchange rides its own QUIC stream (native mux, per-stream flow
control, half-close, and reset). Encode the HTTP method into the cross-node
route filter so candidate selection is method-aware. Keep the legacy WebSocket
control transport for backward compatibility.
* Carry worker kind details into agent job tokens.
WorkerRegistration gains a server-controlled KindDetails field that is
passed to BuildAgentToken when assigning jobs, so kind details flow into
the participant join token for the whole session.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: shishir gowda <shishir@livekit.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Stream now implements io.ReaderFrom, so http.Request.Write's bufio.Writer
delegates the body to it and a large upload flows in MaxFrameSize frames instead
of ~4KiB ones - 16x fewer frames, marshals, scheduler cycles and websocket
messages. Large frame buffers (>= 16KiB) are drawn from a per-conn sync.Pool and
recycled by the scheduler after each write, so a big upload stops allocating a
buffer per frame; small/webhook bodies stay below the threshold and use a
right-sized make, so the average request is unaffected.
Measured on a 1MiB upload: -82% allocations, -66% latency, -17% bytes; a small
GET/POST is unchanged.
- ReadFrame reads each frame into a reused per-conn buffer via NextReader
instead of letting ReadMessage allocate one per frame; ~41% fewer bytes on a
1MiB download. Safe: proto.Unmarshal copies the payload out and readLoop is
the sole reader.
- the data wire's websocket read is bounded by its own max message size (a full
frame + framing overhead), independent of the signalling message limit, so a
peer cannot OOM the server with an oversized frame even when signalling limits
are disabled. Drops the old coupling that clamped MaxFrameSize under the
signalling limit.
- reuse the 32KiB response-copy buffer and the response-head reader from
sync.Pools; ~27% fewer bytes allocated per small request.
- reuse a per-conn marshal buffer in WriteFrame instead of allocating one per
frame; ~19% fewer bytes on a 1MiB upload. Safe because stream.Write copies
the payload and gorilla WriteMessage does not retain it.
- remove cmd/agent-endpoint-client: the Python SDK has landed, so the manual
sidecar for non-SDK workers is no longer needed.
- move pkg/agent/endpoint/client to pkg/agent/endpoint/conformance (package
conformance) so it reads as the test harness it is, not a product client
SDK; only the acceptance suite drives it.
- worker selection uses power-of-two-choices over live in-flight streams
(InflightStreams / SpareStreams) instead of weighted-random over a
reported load; the conn pick stays least-loaded. In-flight is observed
locally so it is correct for adopted wires, and the reported Load hook is
dropped from the endpoint path.
- the front tags its own routing misses (X-Livekit-Endpoint-Miss) so a relay
caller can distinguish them from a worker app's own 404/405 and retry past
them across the remaining candidates.
- a data-wire attach is bound to the connecting grant's api key, so a leaked
attach token cannot bind a conn to another project's worker.
- remove the dead scope-hook machinery (it only fed the removed bus resolve)
and rename the request identity from "scope" to apiKey.
- a path convertor anywhere in a template (non-terminal, or mixed with literal
text in a segment) now canonicalizes to a terminal glob that spans slashes,
instead of being dropped or narrowed to a single-segment str - the old
behavior silently failed to match such routes (a false negative -> misroute).
- enforce MaxRouteDepth at ParseManifest (wires up RouteDepth), bounding the
replicated filter's prefix count and the matcher's walk.
Shape-matches a request path against a set of route templates via canonical
typed-wildcard tokens and a left-to-right pruned walk: linear in path depth
(no 2^segments), type-aware (int/float/uuid/str/path), glob catch-alls, and a
PrefixIndex interface so the same walk runs over an exact map (local) or a
compact replicated backing (multi-node). Worker stays authoritative for the
exact convertor/method match.
A multi-node layer can now replicate a node's route set and match request
paths against it locally, instead of a manifest digest. Drops the digest
helper added for that (superseded).
Order-independent fnv64a of the route set so a multi-node layer can tell
whether the nodes serving a deployment agree on the manifest (safe to route
without inspecting the path) or are mid rolling-deploy (fall back to a
path-aware resolve).
A wire whose worker is registered but under a different epoch now returns
ErrWrongEpoch (not ErrAttachRejected); pool-full/closed stay ErrAttachRejected.
The attach adopter runs on unknown-worker OR wrong-epoch, so a satellite left
behind by a reconnected worker is refreshed by re-validating against the
registration holder instead of shadowing the new epoch until it is swept.
Serve worker-declared FastAPI routes at /agents/{deployment}/{path} without
any worker-side listener: workers dial a fixed pool of wires speaking
AgentHttp.Frame, the server opens multiplexed streams carrying one opaque
HTTP/1.1 exchange each (two-level credit flow control, prioritized write
scheduler, attach epoch fencing). The front does starlette-exact manifest
matching with per-endpoint public access, typed 401/404/405, a retry table,
and SSE/WebSocket passthrough; a pluggable fallback hook lets multi-node
deployments resolve misses elsewhere. Includes a conformance client, a
manual sidecar, and the acceptance suite.
The auth middleware now reads access_token from the query string only:
FormValue consumed the bodies of proxied url-encoded POSTs.
* agent: thread simulation flag from dispatch to job
Reads simulation from AgentDispatch / RoomAgentDispatch and copies it
onto Job in agent.LaunchJob and the inline room-agent path so workers
see the flag.
Stacked on top of livekit/protocol#1629.
* agent: replace simulation bool with attributes map
Threads the renamed attributes map (was bool simulation) from dispatch
to job and bumps the protocol pseudo-version.
* deps
* rtc: report participant kind code and details
Plumb ParticipantKind and KindDetails through MediaTrack and
BytesTrackStats so track-level reporting can record the numeric kind
code plus details codes on every participant_session aggregation,
alongside the existing Kind string. Also picks up the new kind fields
on resolved BytesSignalStats participants.
Adds deployment/agentID/version to the agent worker logger.
* add AssignmentHook to AssignJob; propagate websocket write errors
- Replace the `url *string` parameter on `Worker.AssignJob` with a
middleware-style `AssignmentHook` so callers can intercept the
`JobAssignment` send (e.g. to set Url, or to gate hedged attempts so
only one assignment is written).
- Remove the `sendRequest` helper. Inline `WriteServerMessage` and
propagate the error: `AssignJob` returns immediately on a failed
availability or assignment write, leaving the job out of
`runningJobs`; `TerminateJob` still updates local bookkeeping when
the wire write fails but surfaces the write error to the caller.
* tidy
This is not all of it as it is not possible (or at least I do not know
of a way) to get all suggestions for a repo/project. Did this via loop
searching mainly and taking the modernize suggestions.
