`NativeHooks::sticky_backlog` loads a backlogged batch, cleaning up rooms
whose sticky events expired, reads the events through the shared event
cache and filters them for the destination. Its completion records the
batch natively. Where the filter would wait for full state, the batch is
left to Python, as before.
`FederationStore` also gains `get_catch_up_outstanding_destinations` for
phase 6. Each query has a differential test against Python's.
The native queue tests now run with MSC4354 on, so the loops look at the
sticky event backlog, and pin that it takes no GIL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`NativeHooks::catch_up` runs `_catch_up_transmission_loop` natively. It
reads the catch-up events and the room's forward extremities, loads
events through the shared event cache, filters the extremities with the
Rust `filter_events_for_server`, sends each room's PDUs through the
queue's sink, records the sticky event gap and moves the catch-up
position on. A destination catching up no longer takes the GIL, unless a
module wants to hear about delivered events.
`FederationStore` gains the catch-up queries, with differential tests
against Python's. The `catch_up` hook is given its queue, and a PDU can
now hold an event loaded in Rust.
Where Python would wait for a room's full state to filter, Rust sends
the original PDU, as it does for a room with partial state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Keep an unchanged device stream's position. The tokens are read after
the change check, so moving to them could skip a message that arrived
in between.
- Decode cross-signing keys as leniently as Python does, including
URL-safe base64.
- Mark the transaction open before awaiting BEGIN, so a dropped task
still rolls back.
- Tell modules about delivered events before the database writes, as
Python does.
- Update the stream change cache before its listeners, and log a
listener that fails.
- Check the metrics through a registry scrape, with a fresh event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
On the native Postgres driver, the Rust queues' `NativeHooks` read the
device EDUs and record finished transactions through `FederationStore`,
on tokio. A transaction then takes no trip to the reactor, unless it
finishes a sticky event batch or a module wants to hear about delivered
events. Catch-up and the sticky event backlog stay Python. SQLite and
the Python Postgres driver keep the Python hooks.
Python skips the device queries for destinations with nothing new by
asking two stream change caches. Rust keeps a mirror of each, per
destination, fed by a new `StreamChangeCache.add_listener`, so it hears
of every change the caches do. The current stream positions live in
Python's ID generators, and are read under the GIL (without a reactor
trip) only when a destination has changed.
The loops now count their transactions, EDUs and requests in Rust.
`synapse.metrics.pre_collect_hooks` adds the counts to the Python
metrics when they are collected. This removes a GIL crossing on every
failed send, and from `_rust_complete`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`FederationStore` ports the store methods the federation transmission
loop calls around each transaction to the native Postgres driver:
`get_new_device_msgs_for_remote`, `delete_device_msgs_for_remote`,
`get_device_updates_by_remote` (with the cross-signing key updates and
cross-signatures), `mark_as_sent_devices_by_remote` and
`set_destination_last_successful_stream_ordering`. The stream change
cache checks those start with are left to the caller.
None of the writes has a Python cache to invalidate. The cross-signing
reads skip Python's caches.
`tests/storage/test_federation_store.py` runs each against the Python
method on one database, through a `FederationStoreForTest` pyclass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Stop catch-up before the next room once the loop is shut down. The Rust
loop cannot cancel the Python catch-up coroutine, and after a shutdown
the transaction manager returns without sending, so the room would be
recorded as sent.
- Record the request metrics when the Python fallback for reading a
transaction response fails.
- Give each transmission loop its own LoopActivity, so a loop that ends
cannot clear the parked flag of the next.
- Count the time spent reading a hook's result as GIL time of its site.
- Correct the AwaitableObserver::finished doc.
- Let test_send_receipts_with_backoff_sender accept either order of its
first two sends, which run at once with the queues in Rust.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`DestinationQueue` holds what waits to go to one destination: PDUs,
EDUs, keyed EDUs, presence and receipts, with Python's merge rules and
limits. The first thing queued starts its transmission loop as a tokio
task, which builds and sends transactions until there is nothing left,
as `_transaction_transmission_loop` does.
Catch-up, the sticky event backlog and the loop's database work stay
Python for now, behind `QueueHooks`. `RustDestinationQueue` is the
pyclass the Python facade wraps, and calls those hooks on it.
`run_python_awaitable_observed` converts a Python result on the reactor
and tells an observer when the awaitable starts and finishes. The loop
uses it so tests can tell a loop waiting on the reactor from one with
work to do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Only build the Rust sender from a real `RustFederationClient`. A test
that mocks the federation client no longer fails with a `TypeError`.
- Log the `TX` and response lines under Python's logger names, so log
configs that name those loggers still apply.
- Move the PDUs and EDUs into the request body, so a transaction holds
one copy of them while it is retried, besides the canonical JSON.
- Drop the unused `time_now` from `Event::to_pdu_value`.
- Set the compare mode per `TransactionManager` in the tests that use
it, not on the class.
- Test that the tracing headers reach the request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`TransactionSender` builds `/send` transactions and sends them through
the Rust federation client, with the options `send_transaction` uses.
It reads the response and returns the ids of the PDUs the remote
accepted. `Event::to_pdu_value` builds the PDU JSON from the Rust event
without the GIL.
`RustTransactionManager` is the Python facade. It takes the events and
EDU dicts, and returns a Deferred. A response body that serde_json
refuses goes to Python's decoder, as `JsonParser` would.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Create the sleeper's `Notified` under the map lock, so a wake that lands
just before the sleep starts is not lost. Count sleeps per destination
and forget the destination in a drop guard, which also runs on cancel.
- Map an SRV `.` target to `ConnectError`, as Python's resolver does.
- Call a custom parser's `finish` when its last write fails, and when the
request is cancelled mid-read.
- Check the path is ASCII after the backoff check, as Python does.
- `put_json` takes `rewrite_age_ts`, which `send_transaction` sets. Any
other `json_data_callback` sends through the Python client.
- Count attempts in the no-retry test, and test the request counters,
the proxy-worker fallback, shutdown and the server-up wake.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Port `MatrixFederationHttpClient._send_request` and `_handle_response`
to Rust, over the phase 1 agent: destination validation and the
whitelist, the retry limiter, `X-Matrix` signing per attempt, the long
and short retry schedules with jitter, `wake_destination`, timeouts on
the virtual clock, the trailing slash retry, and the mapping of each
failure to the exception Python raises.
The JSON parsers run on tokio without the GIL. Python's decoder is the
fallback for the JSON serde_json refuses, such as lone surrogates. A
custom `ByteParser` is fed under the GIL in 64 KiB chunks. The
transaction manager's `json_data_callback` becomes a `rewrite_age_ts`
option, so a retry needs no call into Python.
The retry timings are read through a Rust mirror of
`get_destination_retry_timings`. Writes go to a Python callable, which
invalidates the cache across workers and reports a server coming back.
Every GIL acquisition in the federation code goes through `with_gil`,
which counts calls and hold time per call site.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
A shared .well-known lookup only ran while a caller awaited it. If every
caller gave up, the next request resumed it with its fetch timeout long
expired, and cached a failure for an hour. Run it as its own task.
The TLS handshake and a proxy's CONNECT answer had no timeout. hyper
finishes some connections in a background task that no request timeout
reaches, so a peer that stalled after accepting leaked the task and its
socket. Give each 30 seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The Rust replacement for `MatrixFederationAgent` and its connection pool,
as a library: nothing in Synapse sends through it yet. The request layer
(signing, retries, backoff) comes next and will be its first caller.
`Agent::request` takes a `matrix-federation://` request, applies
`.well-known` delegation to the URI and the `Host` header, and sends it
through hyper-util's pooled client (5 idle connections per host, 120s
idle timeout, HTTP/1.1, no automatic retries), as Python's pool does.
The pool calls our connector, derived from matrix-hyper-federation-
client's `MatrixConnector` (Apache-2.0). It resolves the server name
(SRV), tries each server in order and returns the first error if all
fail. For each server it resolves addresses through the IP range policy,
connects TCP with Twisted's staggered attempts and 30s timeout, or
tunnels through `https_proxy` with `CONNECT` unless `no_proxy` exempts
the host, then does TLS verified against the delegated server name,
never the SRV target. The `.well-known` fetch shares the pool and, as in
Python, blocks IP literals by the blocklist alone.
Failures are a typed `TransportError` (DNS, connect, TLS, and so on), so
the request layer can tell a DNS failure from the rest. The agent
implements `FederationTransport`.
`RustFederationAgentForTest` exposes a blocking `request` to Python. The
trial tests drive it against a real TLS server on a thread, signed by
the test CA, covering the delegation, SRV, SNI, IP literal, blocklist,
certificate verification and connection reuse cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The parts of the outbound federation agent that decide where to connect
and what to trust, each a port of the Python it replaces, with Rust unit
tests. Nothing uses them yet.
- `resolve`: `_matrix-fed._tcp`, then `_matrix._tcp`, then port 8448, as
`MatrixHostnameEndpoint._resolve_server`. SRV records are ordered as
`_sort_server_list` (RFC 2782), cached while every record is live, and
the stale answer is used when a refetch fails. A lone `.` target means
the service is unavailable. SRV lookups go through hickory, and A/AAAA
through the system resolver, as Twisted uses getaddrinfo. Parts derive
from matrix-hyper-federation-client (Apache-2.0).
- `well_known`: `WellKnownResolver`, with Python's cache periods, jitter,
grace period, had-valid memory, size cap and retries, behind a
`WellKnownFetcher` trait so the state machine is tested without HTTP.
Two additions: each fetch attempt has a timeout (`client_timeout`), and
concurrent lookups for one server name share a fetch. An `m.server`
that is not a valid URI authority now counts as an invalid response.
- `policy`: the IP range lists, refusing a name if any of its addresses
is blocked (the DNS rebinding defence), and the `https_proxy`
settings with Python's `no_proxy` matching.
- `tls`: `FederationPolicyForHTTPS` on rustls: platform roots or the
custom CA list, verification off (still sending SNI), the per-host
verification whitelist. rustls has no TLS 1.0 or 1.1, so a minimum
below 1.2 means 1.2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
A dropped Clock::sleep left its waker in VIRTUAL_SLEEPERS until the
virtual time passed its deadline, which a later test case's reactor may
never do. Prune closed wakers when sleeping and when the time moves.
Nothing constructs IpBlocked: a blocked address is a failed DNS lookup,
as in Python.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Add `rust/src/federation`, starting with the `X-Matrix` request
signature and the transport seam the request layer will send through.
The signing code comes from matrix-hyper-federation-client (Apache-2.0),
changed to use Synapse's canonical JSON serializer, Python's header
layout and quoting, and `destination_is` for identity servers. It keeps
that crate's signature test vectors. The body is serialized to canonical
JSON once, then both signed and sent in that form.
The homeserver's signing key crosses to Rust as its algorithm, version
and seed, taken from `hs.signing_key`. Rust never reads the key file.
`FederationTransport` is what the request layer will send through.
`PyTransport` implements it with a Python callable, which is how Rust
level tests will inject responses in place of `FakeTransport`.
A trial test checks the Rust header is byte-identical to the one
`MatrixFederationHttpClient.build_auth_headers` builds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
