Port `_calculate_new_extremities`, `_calculate_new_state_dag_extremities`
and `_prune_extremities` from the persist events controller to
`rust/src/persist/extremities.rs`. They read through an `ExtremityReads`
trait, which the `PersistReader` implements and the unit tests implement
over an in-memory DAG. Metrics go to a `PersistCounts` the caller owns,
so shadow mode can drop them.
Missing prev state events fail with `MissingPrevStateEvents`, which
`persist_error_to_py` turns into the same 500 `SynapseError` as Python.
`PersistReaderForTest` gains the three calculations, and
`test_persist_reader.py` compares each with the Python method on the
same database and the same new events.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
`PersistReader` wraps a `SyncReader` and adds the reads the persist
controller makes before its write transaction: forward and state DAG
extremities, the prev event walks of `_get_events_which_are_prevs` and
`_get_prevs_before_rejected`, `have_seen_events`, memberships by event ID,
`is_local_host_in_room_ignoring_users`, the room version, the sliding sync
joined room check, `get_last_event_pos_in_room` and the soft failed sticky
events. None of them is cached. Each runs once per chunk.
The chain cover reads are free functions over a `Querier`, not reader
methods. The chain cover step reads, allocates chain IDs and writes in one
transaction, so they must run on that transaction's connection. To allow
that, `WriteTxn` now implements `Querier`. A `WriteError` that passes
through `anyhow` on the way out comes back as itself, so a conflict is still
retried.
The persister builds its own `SyncStore` over the shared caches.
`tests/storage/test_persist_reader.py` runs each read against the Python
method or statement it ports, on the native driver.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
`run_write_interaction` is the Rust `runInteraction` for writes: a
`REPEATABLE READ` transaction, committed at the end, and run again from the
start on a serialization failure, a deadlock, a cancelled statement or a
lost connection, up to five attempts as in Python. Constraint violations come
back as their own error, so an expected conflict can be told apart.
Once a statement fails, Postgres has aborted the transaction, and a COMMIT
would roll back without an error. So if the closure catches a statement
error and returns success, the commit is refused with an error.
`WriteTxn::insert_many` and `upsert_many` write a batch in one statement
whatever its size, passing each column as an array to `UNNEST`.
`execute_batch` pipelines a per-row statement for the inserts that need a
sub-select per row.
The tests run against a real server when `SYNAPSE_TEST_POSTGRES_DSN` is set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
Phase 0 of the event persistence plan, apart from the write transaction
helper:
- `experimental_features.rust_event_persistence`, one ordered level
(off, shadow, prepare, transaction, full). Nothing uses the levels yet
beyond building the persister.
- `RustHandlers.persister`: built on the events writer when the level is not
off and the main database uses the native driver. It holds the pool, the
shared caches and the events, backfill and cache stream trackers.
- `PersistEffects`, the Python cache updates a Rust transaction will ask for
in place of `txn.call_after`, applied in one call by
`DataStore._apply_rust_persist_effects`. A test persists batches through
Python and checks that applying the matching effects updates the same
caches. The chunk's stream IDs come back unfinished and are finished last,
as Python's `get_next_mult` does after the transaction's callbacks, so
the current token never passes an event the stream change caches don't
know about.
- `PersistContext`, read from an `EventContext` once per event.
- `Event::to_persistence_value` and `EventInternalMetadata::to_persistence_value`:
the `event_json` columns built without the GIL.
- Rust-side persistence counters, added to the Python metrics by a pre-collect
hook. A homeserver can now register more than one such hook. The persisted
position is reported only when it moved, so it can't pull back the gauge
the Python path sets.
- `SYNAPSE_TEST_RUST_PERSIST` for trial and `RUST_PERSIST` for Complement.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
The event persister will hold the Python `RustConnectionPool` the same way.
The GIL site it reads the pool under is now a parameter, so each user is
counted under its own site.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
`MultiWriterIdGenerator` tracks in memory which IDs this instance has
allocated and not finished, and each writer's position, and derives the
current token from them. That state now lives in a Rust `StreamIdTracker`,
so that Rust code can allocate and finish stream IDs without the GIL. The
generator keeps what touches the database or the reactor: loading the start
positions, the sequence, the `stream_positions` table and notifying
replication.
The logic is a line-by-line port. Two changes in form: the in-flight fetches
are a counted map (they were a sorted multiset), and the position gauge reads
the tracker when scraped instead of being set on each change. The issue 19795
debug line is logged after the lock is released, since logging can take the
GIL.
One behaviour change, a fix: on restart, the loader sorted the rows past
`stream_positions` by their stored ID before negating them. On the backfill
stream that visited our own rows furthest first, so our position ended at
the nearest of them. The rows are now negated first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
Event persistence will take the GIL from tokio too, and should be counted the
same way. Its sites are named with a `persist_` prefix and reported by
`persist_gil_stats`; `federation_gil_stats` reports the rest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UHKYPSc37kWqGACn9aKxKL
With the main database on the native Postgres driver, every event load
goes through the Rust `EventLoader`. `EventsWorkerStore` sends the events
its own lookups miss to `RustStore.load_events`, and `_get_events_from_db`
to `reload_events`. Loads from Python, sync and state resolution then
coalesce in the shared event cache, and an invalidation during a load
stops it caching a stale entry. Python's fetch threads and in-flight map
stay only for other databases.
- `EventLoader::load` goes through `Cache::get_or_load_many`. The extra
redaction events it reads are returned but no longer cached.
- `PoolQuerier` checks its connection out before claiming any loads, so
readers that hold connections while they wait cannot starve a load.
- An unusable event raises `InvalidEventError` or
`DatabaseCorruptionError`, as from Python, which some callers catch.
- Cancelling a Python caller leaves the load running for the others, and
`evt_db_fetch_count` counts only what a caller read itself.
- `RustStore` holds the `DatabasePool` weakly and reads its pool per call.
Tests: trial makes the loads block, as it does Python's transactions
(`_BlockingRustCalls`, renamed from `_BlockingRustSyncHandler`). The
cancellation and outage tests get Rust variants. Two tests stop depending
on how far requests advance the clock or on which `DatabasePool` object
a worker store keeps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
`Cache::get_or_load_many` looks up several keys at once and loads the
misses with one call. Each key coalesces with loads already in flight,
single or batched, and a key invalidated mid-load is answered but not
cached. Waiters now get `None` for a key a batch load did not find, and
a single load waiting on such a key loads it itself.
`Cache::abandon_loads` stops every in-flight load from caching, for an
invalidation that cannot name the keys it affects. A room purge uses it
on the event cache, as the room of an event being loaded isn't known.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
- The server filter read each restricted event's state group before it
checked for partial state, and through a cache that a prefill never
reached. After an event got its full state, a retry could use the old
partial group. It now checks first, then reads those groups from the
database. A prefill of a Python cache now invalidates its Rust mirrors.
- Catch-up ended in two steps, so a PDU that send_pdu skipped in between
was lost until the next catch-up. One lock now covers both.
- The sticky backlog query reads a fresh snapshot per statement, so
events could expire between its cleanup and its retry. It now cleans
up again, a few times, before giving up.
- Tests for each, plus catch-up's batch limit, a failed catch-up send,
module notification, a filtered-out sticky batch, the hand-off to
Python, the backlog cleanup boundary and paging outstanding
destinations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`NativeHooks::sticky_backlog` loads a backlogged batch, cleaning up rooms
whose sticky events expired, reads the events through the shared event
cache and filters them for the destination. Its completion records the
batch natively. Where the filter would wait for full state, the batch is
left to Python, as before.
`FederationStore` also gains `get_catch_up_outstanding_destinations` for
phase 6. Each query has a differential test against Python's.
The native queue tests now run with MSC4354 on, so the loops look at the
sticky event backlog, and pin that it takes no GIL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`NativeHooks::catch_up` runs `_catch_up_transmission_loop` natively. It
reads the catch-up events and the room's forward extremities, loads
events through the shared event cache, filters the extremities with the
Rust `filter_events_for_server`, sends each room's PDUs through the
queue's sink, records the sticky event gap and moves the catch-up
position on. A destination catching up no longer takes the GIL, unless a
module wants to hear about delivered events.
`FederationStore` gains the catch-up queries, with differential tests
against Python's. The `catch_up` hook is given its queue, and a PDU can
now hold an event loaded in Rust.
Where Python would wait for a room's full state to filter, Rust sends
the original PDU, as it does for a room with partial state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`visibility::filter_events_for_server` reads the erased senders, the
partial state rooms, the history visibility and the target server's
memberships at each event through a `SyncReader`, and leaves the
per-event decision to `event_visible_to_server`. That now borrows its
arguments.
Python waits for the full state of an event whose memberships it needs.
Rust cannot, and returns `NeedsFullState` for the caller to handle.
The differential tests compare it with Python's on random rooms, an
outlier and a room with partial state, for every combination of flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Keep an unchanged device stream's position. The tokens are read after
the change check, so moving to them could skip a message that arrived
in between.
- Decode cross-signing keys as leniently as Python does, including
URL-safe base64.
- Mark the transaction open before awaiting BEGIN, so a dropped task
still rolls back.
- Tell modules about delivered events before the database writes, as
Python does.
- Update the stream change cache before its listeners, and log a
listener that fails.
- Check the metrics through a registry scrape, with a fresh event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
On the native Postgres driver, the Rust queues' `NativeHooks` read the
device EDUs and record finished transactions through `FederationStore`,
on tokio. A transaction then takes no trip to the reactor, unless it
finishes a sticky event batch or a module wants to hear about delivered
events. Catch-up and the sticky event backlog stay Python. SQLite and
the Python Postgres driver keep the Python hooks.
Python skips the device queries for destinations with nothing new by
asking two stream change caches. Rust keeps a mirror of each, per
destination, fed by a new `StreamChangeCache.add_listener`, so it hears
of every change the caches do. The current stream positions live in
Python's ID generators, and are read under the GIL (without a reactor
trip) only when a destination has changed.
The loops now count their transactions, EDUs and requests in Rust.
`synapse.metrics.pre_collect_hooks` adds the counts to the Python
metrics when they are collected. This removes a GIL crossing on every
failed send, and from `_rust_complete`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`FederationStore` ports the store methods the federation transmission
loop calls around each transaction to the native Postgres driver:
`get_new_device_msgs_for_remote`, `delete_device_msgs_for_remote`,
`get_device_updates_by_remote` (with the cross-signing key updates and
cross-signatures), `mark_as_sent_devices_by_remote` and
`set_destination_last_successful_stream_ordering`. The stream change
cache checks those start with are left to the caller.
None of the writes has a Python cache to invalidate. The cross-signing
reads skip Python's caches.
`tests/storage/test_federation_store.py` runs each against the Python
method on one database, through a `FederationStoreForTest` pyclass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Stop catch-up before the next room once the loop is shut down. The Rust
loop cannot cancel the Python catch-up coroutine, and after a shutdown
the transaction manager returns without sending, so the room would be
recorded as sent.
- Record the request metrics when the Python fallback for reading a
transaction response fails.
- Give each transmission loop its own LoopActivity, so a loop that ends
cannot clear the parked flag of the next.
- Count the time spent reading a hook's result as GIL time of its site.
- Correct the AwaitableObserver::finished doc.
- Let test_send_receipts_with_backoff_sender accept either order of its
first two sends, which run at once with the queues in Rust.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`DestinationQueue` holds what waits to go to one destination: PDUs,
EDUs, keyed EDUs, presence and receipts, with Python's merge rules and
limits. The first thing queued starts its transmission loop as a tokio
task, which builds and sends transactions until there is nothing left,
as `_transaction_transmission_loop` does.
Catch-up, the sticky event backlog and the loop's database work stay
Python for now, behind `QueueHooks`. `RustDestinationQueue` is the
pyclass the Python facade wraps, and calls those hooks on it.
`run_python_awaitable_observed` converts a Python result on the reactor
and tells an observer when the awaitable starts and finishes. The loop
uses it so tests can tell a loop waiting on the reactor from one with
work to do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Only build the Rust sender from a real `RustFederationClient`. A test
that mocks the federation client no longer fails with a `TypeError`.
- Log the `TX` and response lines under Python's logger names, so log
configs that name those loggers still apply.
- Move the PDUs and EDUs into the request body, so a transaction holds
one copy of them while it is retried, besides the canonical JSON.
- Drop the unused `time_now` from `Event::to_pdu_value`.
- Set the compare mode per `TransactionManager` in the tests that use
it, not on the class.
- Test that the tracing headers reach the request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`TransactionSender` builds `/send` transactions and sends them through
the Rust federation client, with the options `send_transaction` uses.
It reads the response and returns the ids of the PDUs the remote
accepted. `Event::to_pdu_value` builds the PDU JSON from the Rust event
without the GIL.
`RustTransactionManager` is the Python facade. It takes the events and
EDU dicts, and returns a Deferred. A response body that serde_json
refuses goes to Python's decoder, as `JsonParser` would.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
- Create the sleeper's `Notified` under the map lock, so a wake that lands
just before the sleep starts is not lost. Count sleeps per destination
and forget the destination in a drop guard, which also runs on cancel.
- Map an SRV `.` target to `ConnectError`, as Python's resolver does.
- Call a custom parser's `finish` when its last write fails, and when the
request is cancelled mid-read.
- Check the path is ASCII after the backoff check, as Python does.
- `put_json` takes `rewrite_age_ts`, which `send_transaction` sets. Any
other `json_data_callback` sends through the Python client.
- Count attempts in the no-retry test, and test the request counters,
the proxy-worker fallback, shutdown and the server-up wake.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Port `MatrixFederationHttpClient._send_request` and `_handle_response`
to Rust, over the phase 1 agent: destination validation and the
whitelist, the retry limiter, `X-Matrix` signing per attempt, the long
and short retry schedules with jitter, `wake_destination`, timeouts on
the virtual clock, the trailing slash retry, and the mapping of each
failure to the exception Python raises.
The JSON parsers run on tokio without the GIL. Python's decoder is the
fallback for the JSON serde_json refuses, such as lone surrogates. A
custom `ByteParser` is fed under the GIL in 64 KiB chunks. The
transaction manager's `json_data_callback` becomes a `rewrite_age_ts`
option, so a retry needs no call into Python.
The retry timings are read through a Rust mirror of
`get_destination_retry_timings`. Writes go to a Python callable, which
invalidates the cache across workers and reports a server coming back.
Every GIL acquisition in the federation code goes through `with_gil`,
which counts calls and hold time per call site.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
A shared .well-known lookup only ran while a caller awaited it. If every
caller gave up, the next request resumed it with its fetch timeout long
expired, and cached a failure for an hour. Run it as its own task.
The TLS handshake and a proxy's CONNECT answer had no timeout. hyper
finishes some connections in a background task that no request timeout
reaches, so a peer that stalled after accepting leaked the task and its
socket. Give each 30 seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The Rust replacement for `MatrixFederationAgent` and its connection pool,
as a library: nothing in Synapse sends through it yet. The request layer
(signing, retries, backoff) comes next and will be its first caller.
`Agent::request` takes a `matrix-federation://` request, applies
`.well-known` delegation to the URI and the `Host` header, and sends it
through hyper-util's pooled client (5 idle connections per host, 120s
idle timeout, HTTP/1.1, no automatic retries), as Python's pool does.
The pool calls our connector, derived from matrix-hyper-federation-
client's `MatrixConnector` (Apache-2.0). It resolves the server name
(SRV), tries each server in order and returns the first error if all
fail. For each server it resolves addresses through the IP range policy,
connects TCP with Twisted's staggered attempts and 30s timeout, or
tunnels through `https_proxy` with `CONNECT` unless `no_proxy` exempts
the host, then does TLS verified against the delegated server name,
never the SRV target. The `.well-known` fetch shares the pool and, as in
Python, blocks IP literals by the blocklist alone.
Failures are a typed `TransportError` (DNS, connect, TLS, and so on), so
the request layer can tell a DNS failure from the rest. The agent
implements `FederationTransport`.
`RustFederationAgentForTest` exposes a blocking `request` to Python. The
trial tests drive it against a real TLS server on a thread, signed by
the test CA, covering the delegation, SRV, SNI, IP literal, blocklist,
certificate verification and connection reuse cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The parts of the outbound federation agent that decide where to connect
and what to trust, each a port of the Python it replaces, with Rust unit
tests. Nothing uses them yet.
- `resolve`: `_matrix-fed._tcp`, then `_matrix._tcp`, then port 8448, as
`MatrixHostnameEndpoint._resolve_server`. SRV records are ordered as
`_sort_server_list` (RFC 2782), cached while every record is live, and
the stale answer is used when a refetch fails. A lone `.` target means
the service is unavailable. SRV lookups go through hickory, and A/AAAA
through the system resolver, as Twisted uses getaddrinfo. Parts derive
from matrix-hyper-federation-client (Apache-2.0).
- `well_known`: `WellKnownResolver`, with Python's cache periods, jitter,
grace period, had-valid memory, size cap and retries, behind a
`WellKnownFetcher` trait so the state machine is tested without HTTP.
Two additions: each fetch attempt has a timeout (`client_timeout`), and
concurrent lookups for one server name share a fetch. An `m.server`
that is not a valid URI authority now counts as an invalid response.
- `policy`: the IP range lists, refusing a name if any of its addresses
is blocked (the DNS rebinding defence), and the `https_proxy`
settings with Python's `no_proxy` matching.
- `tls`: `FederationPolicyForHTTPS` on rustls: platform roots or the
custom CA list, verification off (still sending SNI), the per-host
verification whitelist. rustls has no TLS 1.0 or 1.1, so a minimum
below 1.2 means 1.2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
A dropped Clock::sleep left its waker in VIRTUAL_SLEEPERS until the
virtual time passed its deadline, which a later test case's reactor may
never do. Prune closed wakers when sleeping and when the time moves.
Nothing constructs IpBlocked: a blocked address is a failed DNS lookup,
as in Python.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Extend the Rust view of the homeserver config with what the federation
request layer and agent need:
- `config.federation`: timeouts, retry counts and delays, destination
backoff, the domain whitelist and the metrics domains.
- `config.tls`: certificate verification, the per-host verification
whitelist, the minimum TLS version, and the custom CA list. Python now
keeps the PEM text of that list too, so Rust never reads the files.
- `config.server`: the federation IP range block and allow lists (taken
from `netaddr.IPSet`s), the user agent suffix and the proxy config.
The verification whitelist holds Python regexes made by
`glob_to_regex`. They translate to Rust's syntax with two changes: `\Z`
becomes `\z` and `re.IGNORECASE` becomes `(?i)`.
The changelog entry needs renaming to the PR number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Add `rust/src/federation`, starting with the `X-Matrix` request
signature and the transport seam the request layer will send through.
The signing code comes from matrix-hyper-federation-client (Apache-2.0),
changed to use Synapse's canonical JSON serializer, Python's header
layout and quoting, and `destination_is` for identity servers. It keeps
that crate's signature test vectors. The body is serialized to canonical
JSON once, then both signed and sent in that form.
The homeserver's signing key crosses to Rust as its algorithm, version
and seed, taken from `hs.signing_key`. Rust never reads the key file.
`FederationTransport` is what the request layer will send through.
`PyTransport` implements it with a Python callable, which is how Rust
level tests will inject responses in place of `FakeTransport`.
A trial test checks the Rust header is byte-identical to the one
`MatrixFederationHttpClient.build_auth_headers` builds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Tokio's timers run on real time, but trial drives a virtual reactor clock
that only moves when a test advances it. A Rust task that sleeps with
`tokio::time::sleep` under trial waits out real time while the test
believes none has passed.
`Clock::sleep` sleeps on the virtual clock while a test has pinned it:
`set_virtual_time_msec` wakes the sleepers that are due. Otherwise it is
`tokio::time::sleep`. Code that sleeps takes a `Sleeper`, so that its
Rust unit tests can use tokio's pausable clock instead, since the
virtual time is process-global and an earlier test may have pinned it.
The federation sender port needs this for its backoff and well-known
retry sleeps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The new `sync/relations.rs` ports the relations handler's
`get_bundled_aggregations`: thread summaries with participation and
ignored users' replies subtracted, references, and the latest edit. The
Rust sync handler bundles them for limited and newly joined timelines,
loads the redactions of the bundled events, and the encoder serializes
them. Python no longer bundles the rooms Rust computes, and the room
entries hand Python the bundles only when something reads the timeline.
A thread whose latest reply is from an ignored user falls back to
Python for that room: finding the latest reply that is not takes the
relations pagination, which is not ported.
The relations queries read the database each time. Python's relations
caches are not mirrored.
Shadow mode now compares the Rust bundles with Python's, in the encoded
rooms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The rooms the Rust sync handler computes now reach the servlet as
`RustJoinedSyncResult` and `RustArchivedSyncResult`. They subclass the
Python entries, and their fields become Python objects only when read.
The servlet encodes them in one call to the new Rust `RoomsEncoder`,
which ports `encode_room` over the existing serializer. If that call
fails, Python encodes them.
The Rust handler loads each room's redactions as it computes the room.
The serializer reads redactions, bundles and module additions through a
`SerializeLookups` trait, so that it takes Rust events as well as Python
maps. Bundled aggregations are still computed in Python.
Shadow mode now also compares the encoded JSON of each room.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`EventsWorkerStore._get_event_cache` is now the Rust `EventCache` that the
sync handler and state resolution read, so each event is held once.
The cache keeps Rust `CachedEvent` values. Python gets them through
`RustEventCache`, a `RustCache` subclass with the `AsyncLruCache` names the
rest of Synapse uses, and each hit wraps the cached event in a new `Event`
object. The wrapper shares the cached event's internal metadata, unsigned
data and signatures, so a change Python makes, such as adding
`prev_content` or flipping `outlier`, lands in the one copy the Rust
readers see. Python's `_event_ref` weak map is checked before the cache,
so while anything holds an event, a lookup returns that same object, as
before. A hit there marks the cache entry as read, and puts the event back
if it was evicted.
- The cache is sized from `caches.event_cache_size` and named
`*getEvent*`, so its metrics and `per_cache_factors` keep their names.
The Rust loader now counts its misses too.
- Invalidating an event drops its chain position as well, and a room
purge drops the room's events through the room index, plus the chain
cover and state group caches, as `RustStore.invalidate_event` and
`clear_event_cache` did. Those two calls are gone.
- `EventsWorkerStore` and `RustWorkerStore` run their `__init__` bodies
in a different order in `DataStore` and `GenericWorkerStore`, as they
sit differently in each MRO. So whichever runs first builds the
`RustStore`. A datastore without `RustWorkerStore`, such as the port
script's, gets a standalone `RustEventCache`.
- The external cache stubs (`get_external`, `set_external`,
`_invalidate_async_get_event_cache` and its room variant) are dropped.
They were documented no-ops.
- The sync shadow diff still ignores `prev_content` on state events.
Python keeps the objects it holds after the cache evicts them, so a Rust
reader can still load a second copy without it. Trial's cache of one
event makes that routine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
With `cache_autotuning` configured, `_expire_old_entries` walks Python's
global cache list from the coldest entry and evicts while jemalloc reports
too much memory allocated. Rust caches were not on that list, so they were
never evicted for memory, which matters once the event cache lives in Rust.
Each Rust cache's LRU order is already a sorted run, so the registry
merges them without a global list: `CacheRegistry::evict_coldest` keeps
evicting from the cache whose coldest entry is coldest, up to a bound and
a count. The Python walk calls it before evicting each Python entry, with
that entry's last access as the bound, so both sides go coldest first
together. It reports the coldest Rust entry left, so Python only calls
again once its walk passes that time. Memory is re-read after each batch
of 100 Rust evictions, as after 100 Python ones, and `min_cache_ttl` holds
for Rust entries too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
Preparation for making the Rust event cache the one `EventsWorkerStore`
reads through.
- `Cache::new_indexed` keeps the keys grouped by a value-derived index key,
maintained on every insert and removal path, and `invalidate_index` drops
one group. A room purge invalidates the event cache by room, and
retention runs that for every room with a policy, so it must not walk
the whole cache.
- `Cache::touch` marks an entry as read without cloning it,
`get_or_count_miss` and `get_uncounted` cover the metrics cases of
Python's `update_metrics`, and `count_miss` records a miss found
elsewhere.
- `CachedEvent` works out its auth event IDs on first use. An event filled
from Python is usually never read by state resolution, so it should not
pay to parse them. `auth_event_ids` is fallible now, and its callers in
state resolution propagate the error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
Phase 4 of the sync-in-Rust plan. `SyncHandler.room_changes` ports
`_get_room_changes_for_incremental_sync` and
`_get_room_changes_for_initial_sync`: which rooms the response reports, and
which the user joined or left since the last sync. It fetches the timelines
of the joined rooms itself, as `get_room_events_stream_for_rooms` does.
`SyncHandler.generate_room_entries` then computes every joined and archived
room's entry, ten at a time. A room it cannot compute, or fails on, comes
back in the result for Python to compute. A panic in a room is that room's
error, so it cannot fail the rooms that have recorded their lazy-loaded
members.
The `state_after` rollback now reads the state deltas with no upper bound,
after the current state, so that it covers every change the current state
may hold.
These replace the per-room `generate_room_entry` and `RoomSyncInput`. The
store gains `event_position`, for the leave events of archived rooms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
Phase 3 of the sync-in-Rust plan: `SyncHandler.generate_room_entry` ports
`_generate_room_entry` for a joined or archived room, reading through the
sync store. It covers the timeline (gap check, filtering, visibility and the
backward pagination loop), the state delta on the full, incremental and
`state_after` paths with the lazy-loaded members cache, the room summary,
sticky events and unread counts. Partial state rooms raise
`RustSyncFallback`. Bundled aggregations stay with Python.
A room records the lazy-loaded members it sends only once its entry is
computed. A room that fails part way records none, so Python can compute it
from the same cache. Python reads and writes that cache through
`ll_member_was_sent` and `ll_member_mark_sent`.
The visibility checks get a batch loader over the store, and the store gains
`events_have_relations` for relation filters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The queries of the per-room sync loop, on the native Postgres driver, each
a port of the store method it is named after. A `SyncReader` is made per
request and borrows a pooled connection for one group of statements at a
time, so that a sync worker with hundreds of requests in flight does not
pin a connection per room; a group that needs one snapshot runs in a
`REPEATABLE READ READ ONLY` transaction. State groups are loaded whole,
with the recursive CTE sent as one simple query per batch, and cached as
full maps.
The caches that copy a `@cached` function (rooms for user, room summary,
partial state rooms, erased users, retention policies, event to state
group, unread counts) live on the Rust store and are attached to the
Python cache each names by the new `RustWorkerStore`, which also takes
over building the Rust store. Unread counts are invalidated by room as
well as by room and user, as the Python tree cache is.
`tests/storage/test_sync_store.py` runs every query and the Python
method against the same database, with multi-writer tokens, redactions
of every kind, and the summary rotation of the unread counts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The sync store's caches are copies of `@cached` functions of the
datastore, and those are invalidated by several paths: a write on the
process, the caches stream, and the current state batch. Rather than tap
each path, a `DeferredCache` now takes mirrors, invalidated with its own
keys whenever it is, and `mirror_rust_cache` attaches a Rust cache as one,
passing the key as the tuple of arguments.
A `tree=True` cache is invalidated by a prefix of its key as well, so the
Rust cache gains `invalidate_matching`, which drops every entry whose key
satisfies a test, and a key type can decode a prefix into one. A key of
two strings, such as room and user, does.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The state res store built events from rows itself, unredacted. Sync sends
events to clients, so it needs the whole of `_get_events_from_db`: the
`redactions` and `room_ban_redactions` tables, the recheck of redactions
whose authority was not verified at persist time, and `get_events_as_list`
on top, with its redact behaviours and the previous content of state
events.
`storage/events.rs` holds that port, over a `Querier` so it runs inside
the state res transaction as well as on a borrowed sync connection. The
event cache entry becomes a `CachedEvent`: the event as persisted and its
redacted form, as Python's `EventCacheEntry`. `StateResEvent` is an alias
of it, and the state res store loads through the shared code.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`LazyLoadedMembersCache` as it is on develop after #20289: an LRU of the
members sent to a device, each stamped with the room position of the
response that carried it, and `start_request` forgetting the entries
recorded after the request's `since`. The per-device caches expire
after thirty minutes without use, as the Python `ExpiringCache` does.
The Rust sync handler holds them, not the store: they are per device and
no data change invalidates them. `SyncHandler.start_request` is its
first entry point.
`tests/sync/test_rust_ll_members.py` mirrors the Python cache's tests
and runs random operation sequences, with multi-writer positions,
through both.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The per-event decision of `filter_and_transform_events_for_client`:
`_check_client_allowed_to_see_event` and the membership, history
visibility, send-to-client and effective visibility checks under it, the
call invite rule, and the membership annotation for MSC4115. The inputs
are the event, the user's membership and the history visibility at it as
events, and flags; loading those is the caller's job, so the module can
serve the sync handler and later ports alike. The erased-sender case
prunes through the existing Rust redaction.
The `allowed` closure in the Python is now a module-level function,
`_check_event_for_client`, with the same body, so that
`tests/sync/test_rust_visibility.py` can compare the two exactly on
random events, state and flags, sticky and outlier events included.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`_calculate_state`, the `use_state_after` roll-back over state deltas of
the full-state path, and the delta loop of the incremental path. The
database reads that produce their inputs stay with the store.
The two Python loops were inline in the handler; they are now functions,
`_roll_back_state_over_deltas` and `_apply_current_state_deltas`, so that
`tests/sync/test_rust_state_delta.py` can compare them with the Rust on
random state maps and deltas. Behaviour is unchanged.
One deliberate difference: when the start and end of the timeline
disagree on a key that is in neither the timeline nor the previous state,
`_calculate_state` sends whichever event its set iteration visits last.
The Rust sends the one from the end of the timeline, the state the client
should end up with. The test allows the Python either.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`Filter` and `FilterCollection` from `synapse.api.filtering`, built from
the filter JSON the Python one was built from, once per request. The
checks are pure functions of the filter and the event, over events,
presence updates and the JSON of ephemeral events and account data. The
one part that needs the database, the `related_by_*` relation checks, is
a trait the store will supply; `needs_relation_check` says when.
The config flags the Python reads (MSC3773, MSC3874, profile updates in
sync) are passed in, so the port has no reach into the config.
`tests/sync/test_rust_filter.py` compares the two on random filters and
random events, with and without the MSCs enabled.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The first of the pure parts of `/sync` that the Rust sync handler needs:
`RoomStreamToken` with its three string forms and its vector clock
arithmetic (`is_before_or_eq`, `copy_and_advance`, `bound_stream_token`),
`persisted_after`, and the pagination helpers of `stream.py` that turn
tokens into bounds and filter the rows that come back by writer. The SQL
is built in the Postgres row-comparison form only, as the handler reads
through the native Postgres driver.
Tokens cross the boundary as the Python class, converted both ways.
The multi-writer filtering is where a bug would be silent, so
`tests/sync/test_rust_stream_token.py` checks every operation against the
Python on random tokens with instance maps. `tests/sync/fixtures.py`
holds what the differential tests that follow share.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`rust_sync` takes `off`, `shadow` or `on`, as `rust_state_res` does, and
`RustHandlers.sync` is built when the main database uses the native Rust
driver. The Python `SyncHandler` reads the option and takes the Rust
handler, warning and keeping the Python path when the driver is not the
Rust one. Nothing is served by Rust yet: the per-room entry points come
with the ports that follow.
Trial gets the same plumbing the state resolution has: `SYNAPSE_TEST_
RUST_SYNC` sets the option on every test homeserver, and the Rust handler
is wrapped so that each entry point's `_blocking` twin answers under the
fake reactor.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
Cancelling a deferred from `create_deferred` failed it at once and left
the tokio task running to completion, its result thrown away. That is
the right behaviour for a cache load, which other callers may be
waiting on, but wrong for work only the cancelled caller wanted: a
`/sync` request's per-room loop, say, which clients abandon constantly
and which holds a pooled connection while it runs.
`create_deferred_with` takes an `OnCancel`. With `Abort`, the deferred's
canceller aborts the task, so its future is dropped where it stands and
whatever it holds is released by `Drop`. The default stays as it was.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The event, chain position, chain link and state map caches lived on the
Rust `StateResolutionHandler`, each a `SizedLru` of its own. The Rust
sync handler will read the event and state group caches too, so they
move to the Rust `Store`, which the main datastore builds and every Rust
handler borrows. They are now `Cache`s in the runtime's registry, so
age-based expiry, cache factors and metrics come with that rather than
being re-implemented, and `SizedLru` goes.
`EventsWorkerStore` now empties them through the store (`RustStore.
invalidate_event` and `clear_event_cache`) rather than through the state
res handler, so they are kept in step with the Python event cache on
every database, and the conflict cache is the one cache the handler
still owns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The Rust `Cache` bounded itself by entry count, through `lru`'s capacity.
The state resolution caches need a bound on the sum of what their values
weigh: a state map counts for its entries, a chain's links for each link,
as Python's `size_callback` caches do. So the bound is now a weight, with
a `Weigher` chosen at construction and the default weighing every value
as one, and the cache enforces it itself over an unbounded `lru`.
Those caches are also filled by readers that load a batch of keys in one
statement rather than through `get_or_load`, so `Cache::insert` is now
public, with `prefill` semantics: it leaves any in-flight load for the
key to answer its waiters and cache what it found.
A cache key that is a single string or integer now decodes from Python,
as the keys of the caches moving here are event IDs and group IDs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The first Rust database function to get a cache, and the first Python
`@cached` function to be replaced by one. The Rust `/versions` handler
calls `Store::is_feature_enabled_for_user` on every authenticated request,
and until now each call ran a query. The Python `is_feature_enabled` had a
cache of the same rows under another name, which every writer would have
had to invalidate alongside. Now there is one cache, keyed on
`(user_id, feature)`, and both sides read through it.
The main datastore builds the Rust `Store` in its `__init__`, over its own
`db_pool`, and `hs.get_rust_store()` hands it to the Rust handlers: the
datastore owns the store, the handlers borrow it. The store's
`is_feature_enabled_for_user` reaches Python as a callable cache object,
bound on the datastore under that name. Calling it answers a hit with an
already-fired Deferred, without leaving the reactor thread, and only a
miss goes through tokio. Being a `RustCache`, it is also what
`_invalidate_cache_and_stream`, replication and `register_cache` take, so
`set_features_for_user` invalidates it with the one call it already made,
and the name is spelled once in Rust, where the cache is built.
`is_feature_enabled` keeps its global-config short cut in Python and
delegates the per-user lookup.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
