The new `sync/relations.rs` ports the relations handler's
`get_bundled_aggregations`: thread summaries with participation and
ignored users' replies subtracted, references, and the latest edit. The
Rust sync handler bundles them for limited and newly joined timelines,
loads the redactions of the bundled events, and the encoder serializes
them. Python no longer bundles the rooms Rust computes, and the room
entries hand Python the bundles only when something reads the timeline.
A thread whose latest reply is from an ignored user falls back to
Python for that room: finding the latest reply that is not takes the
relations pagination, which is not ported.
The relations queries read the database each time. Python's relations
caches are not mirrored.
Shadow mode now compares the Rust bundles with Python's, in the encoded
rooms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The rooms the Rust sync handler computes now reach the servlet as
`RustJoinedSyncResult` and `RustArchivedSyncResult`. They subclass the
Python entries, and their fields become Python objects only when read.
The servlet encodes them in one call to the new Rust `RoomsEncoder`,
which ports `encode_room` over the existing serializer. If that call
fails, Python encodes them.
The Rust handler loads each room's redactions as it computes the room.
The serializer reads redactions, bundles and module additions through a
`SerializeLookups` trait, so that it takes Rust events as well as Python
maps. Bundled aggregations are still computed in Python.
Shadow mode now also compares the encoded JSON of each room.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`rust_sync: on` was refused unless `rust_sync_allow_unshared_event_cache`
was set, because the Rust sync handler held a second copy of every event
it served. Python and Rust now share one event cache, so the gate and its
option go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
`EventsWorkerStore._get_event_cache` is now the Rust `EventCache` that the
sync handler and state resolution read, so each event is held once.
The cache keeps Rust `CachedEvent` values. Python gets them through
`RustEventCache`, a `RustCache` subclass with the `AsyncLruCache` names the
rest of Synapse uses, and each hit wraps the cached event in a new `Event`
object. The wrapper shares the cached event's internal metadata, unsigned
data and signatures, so a change Python makes, such as adding
`prev_content` or flipping `outlier`, lands in the one copy the Rust
readers see. Python's `_event_ref` weak map is checked before the cache,
so while anything holds an event, a lookup returns that same object, as
before. A hit there marks the cache entry as read, and puts the event back
if it was evicted.
- The cache is sized from `caches.event_cache_size` and named
`*getEvent*`, so its metrics and `per_cache_factors` keep their names.
The Rust loader now counts its misses too.
- Invalidating an event drops its chain position as well, and a room
purge drops the room's events through the room index, plus the chain
cover and state group caches, as `RustStore.invalidate_event` and
`clear_event_cache` did. Those two calls are gone.
- `EventsWorkerStore` and `RustWorkerStore` run their `__init__` bodies
in a different order in `DataStore` and `GenericWorkerStore`, as they
sit differently in each MRO. So whichever runs first builds the
`RustStore`. A datastore without `RustWorkerStore`, such as the port
script's, gets a standalone `RustEventCache`.
- The external cache stubs (`get_external`, `set_external`,
`_invalidate_async_get_event_cache` and its room variant) are dropped.
They were documented no-ops.
- The sync shadow diff still ignores `prev_content` on state events.
Python keeps the objects it holds after the cache evicts them, so a Rust
reader can still load a second copy without it. Trial's cache of one
event makes that routine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
With `cache_autotuning` configured, `_expire_old_entries` walks Python's
global cache list from the coldest entry and evicts while jemalloc reports
too much memory allocated. Rust caches were not on that list, so they were
never evicted for memory, which matters once the event cache lives in Rust.
Each Rust cache's LRU order is already a sorted run, so the registry
merges them without a global list: `CacheRegistry::evict_coldest` keeps
evicting from the cache whose coldest entry is coldest, up to a bound and
a count. The Python walk calls it before evicting each Python entry, with
that entry's last access as the bound, so both sides go coldest first
together. It reports the coldest Rust entry left, so Python only calls
again once its walk passes that time. Memory is re-read after each batch
of 100 Rust evictions, as after 100 Python ones, and `min_cache_ttl` holds
for Rust entries too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpBQxvqZTUvHCMm3u6CR7J
With `rust_sync: on`, Python hands the rooms section to the Rust handler in
two calls, fetching the sticky events of newly joined rooms between them.
Python computes the rooms the Rust handler leaves to it. If the Rust
handler fails to work out the room changes, Python computes the whole
section.
Shadow mode runs the Python and Rust sections at once. It compares the room
changes (newly joined and left rooms, invites and knocks) as well as each
room's entry.
Python still computes what feeds the steps before the rooms: the joined
rooms of `get_sync_result_builder`, ephemeral events, room account data,
tags and sticky events. It also drops the invites of ignored users and those
the invite rules ignore.
The unknown room version test also clears the Rust event cache, as the
restart it stands for would.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
`on` serves each joined or archived room from the Rust handler. A room the
Rust handler cannot compute, or fails on, is computed by Python. Python's
fallback rooms share the Rust handler's lazy-loaded members caches, so a
room that moves between the two does not resend members.
`shadow` serves Python's entry and compares the Rust one with it, counting
the outcome in synapse_sync_rust_shadow_total. The two run at once, so the
response waits for the slower. Cancellation stops both.
Config parsing refuses `on` while the Rust and Python event caches are
separate, unless `rust_sync_allow_unshared_event_cache` is set. Under trial,
a Rust error or a shadow mismatch fails the test.
The call invite test now builds its events for the room's own version. An
event of another version is stored, but the Rust event loader cannot read
it back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The queries of the per-room sync loop, on the native Postgres driver, each
a port of the store method it is named after. A `SyncReader` is made per
request and borrows a pooled connection for one group of statements at a
time, so that a sync worker with hundreds of requests in flight does not
pin a connection per room; a group that needs one snapshot runs in a
`REPEATABLE READ READ ONLY` transaction. State groups are loaded whole,
with the recursive CTE sent as one simple query per batch, and cached as
full maps.
The caches that copy a `@cached` function (rooms for user, room summary,
partial state rooms, erased users, retention policies, event to state
group, unread counts) live on the Rust store and are attached to the
Python cache each names by the new `RustWorkerStore`, which also takes
over building the Rust store. Unread counts are invalidated by room as
well as by room and user, as the Python tree cache is.
`tests/storage/test_sync_store.py` runs every query and the Python
method against the same database, with multi-writer tokens, redactions
of every kind, and the summary rotation of the unread counts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The sync store's caches are copies of `@cached` functions of the
datastore, and those are invalidated by several paths: a write on the
process, the caches stream, and the current state batch. Rather than tap
each path, a `DeferredCache` now takes mirrors, invalidated with its own
keys whenever it is, and `mirror_rust_cache` attaches a Rust cache as one,
passing the key as the tuple of arguments.
A `tree=True` cache is invalidated by a prefix of its key as well, so the
Rust cache gains `invalidate_matching`, which drops every entry whose key
satisfies a test, and a key type can decode a prefix into one. A key of
two strings, such as room and user, does.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`LazyLoadedMembersCache` as it is on develop after #20289: an LRU of the
members sent to a device, each stamped with the room position of the
response that carried it, and `start_request` forgetting the entries
recorded after the request's `since`. The per-device caches expire
after thirty minutes without use, as the Python `ExpiringCache` does.
The Rust sync handler holds them, not the store: they are per device and
no data change invalidates them. `SyncHandler.start_request` is its
first entry point.
`tests/sync/test_rust_ll_members.py` mirrors the Python cache's tests
and runs random operation sequences, with multi-writer positions,
through both.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The per-event decision of `filter_and_transform_events_for_client`:
`_check_client_allowed_to_see_event` and the membership, history
visibility, send-to-client and effective visibility checks under it, the
call invite rule, and the membership annotation for MSC4115. The inputs
are the event, the user's membership and the history visibility at it as
events, and flags; loading those is the caller's job, so the module can
serve the sync handler and later ports alike. The erased-sender case
prunes through the existing Rust redaction.
The `allowed` closure in the Python is now a module-level function,
`_check_event_for_client`, with the same body, so that
`tests/sync/test_rust_visibility.py` can compare the two exactly on
random events, state and flags, sticky and outlier events included.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`_calculate_state`, the `use_state_after` roll-back over state deltas of
the full-state path, and the delta loop of the incremental path. The
database reads that produce their inputs stay with the store.
The two Python loops were inline in the handler; they are now functions,
`_roll_back_state_over_deltas` and `_apply_current_state_deltas`, so that
`tests/sync/test_rust_state_delta.py` can compare them with the Rust on
random state maps and deltas. Behaviour is unchanged.
One deliberate difference: when the start and end of the timeline
disagree on a key that is in neither the timeline nor the previous state,
`_calculate_state` sends whichever event its set iteration visits last.
The Rust sends the one from the end of the timeline, the state the client
should end up with. The test allows the Python either.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`Filter` and `FilterCollection` from `synapse.api.filtering`, built from
the filter JSON the Python one was built from, once per request. The
checks are pure functions of the filter and the event, over events,
presence updates and the JSON of ephemeral events and account data. The
one part that needs the database, the `related_by_*` relation checks, is
a trait the store will supply; `needs_relation_check` says when.
The config flags the Python reads (MSC3773, MSC3874, profile updates in
sync) are passed in, so the port has no reach into the config.
`tests/sync/test_rust_filter.py` compares the two on random filters and
random events, with and without the MSCs enabled.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The first of the pure parts of `/sync` that the Rust sync handler needs:
`RoomStreamToken` with its three string forms and its vector clock
arithmetic (`is_before_or_eq`, `copy_and_advance`, `bound_stream_token`),
`persisted_after`, and the pagination helpers of `stream.py` that turn
tokens into bounds and filter the rows that come back by writer. The SQL
is built in the Postgres row-comparison form only, as the handler reads
through the native Postgres driver.
Tokens cross the boundary as the Python class, converted both ways.
The multi-writer filtering is where a bug would be silent, so
`tests/sync/test_rust_stream_token.py` checks every operation against the
Python on random tokens with instance maps. `tests/sync/fixtures.py`
holds what the differential tests that follow share.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
`rust_sync` takes `off`, `shadow` or `on`, as `rust_state_res` does, and
`RustHandlers.sync` is built when the main database uses the native Rust
driver. The Python `SyncHandler` reads the option and takes the Rust
handler, warning and keeping the Python path when the driver is not the
Rust one. Nothing is served by Rust yet: the per-room entry points come
with the ports that follow.
Trial gets the same plumbing the state resolution has: `SYNAPSE_TEST_
RUST_SYNC` sets the option on every test homeserver, and the Rust handler
is wrapped so that each entry point's `_blocking` twin answers under the
fake reactor.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
Cancelling a deferred from `create_deferred` failed it at once and left
the tokio task running to completion, its result thrown away. That is
the right behaviour for a cache load, which other callers may be
waiting on, but wrong for work only the cancelled caller wanted: a
`/sync` request's per-room loop, say, which clients abandon constantly
and which holds a pooled connection while it runs.
`create_deferred_with` takes an `OnCancel`. With `Abort`, the deferred's
canceller aborts the task, so its future is dropped where it stands and
whatever it holds is released by `Drop`. The default stays as it was.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194eNQP5weBtVMg8NKtc6VU
The first Rust database function to get a cache, and the first Python
`@cached` function to be replaced by one. The Rust `/versions` handler
calls `Store::is_feature_enabled_for_user` on every authenticated request,
and until now each call ran a query. The Python `is_feature_enabled` had a
cache of the same rows under another name, which every writer would have
had to invalidate alongside. Now there is one cache, keyed on
`(user_id, feature)`, and both sides read through it.
The main datastore builds the Rust `Store` in its `__init__`, over its own
`db_pool`, and `hs.get_rust_store()` hands it to the Rust handlers: the
datastore owns the store, the handlers borrow it. The store's
`is_feature_enabled_for_user` reaches Python as a callable cache object,
bound on the datastore under that name. Calling it answers a hit with an
already-fired Deferred, without leaving the reactor thread, and only a
miss goes through tokio. Being a `RustCache`, it is also what
`_invalidate_cache_and_stream`, replication and `register_cache` take, so
`set_features_for_user` invalidates it with the one call it already made,
and the name is spelled once in Rust, where the cache is built.
`is_feature_enabled` keeps its global-config short cut in Python and
delegates the per-user lookup.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
With the database work cached, converting the state sets from Python
dicts was the largest piece of a warm resolution: 5,010 entries pulled
out of tuples and interned, under the GIL, before the future starts,
and 2,505 entries built into a dict on the way back. Every group's
state was converted again on every resolution it took part in.
The handler gains resolve_state_groups, which takes the state groups
by ID with their state, as StateResolutionHandler.resolve_state_groups
receives them, and keeps each group's converted StateMap in a cache
keyed on the group ID. State groups are immutable and their IDs are
never reused, so an entry can only go stale by its group being
deleted, which makes it dead weight rather than wrong. The cache is
cleared on a room purge with the others. A group given with fewer
entries than its cached state is a filtered one: it is converted for
that call and does not replace the full map.
The result is returned in the shape of a _StateCacheEntry, chosen as
_make_state_cache_entry does: the input group the resolved state
equals, else the input group it differs least from and the difference,
else the state itself. So the common cases cross the boundary as a
group ID or a small delta rather than the whole state.
StateResolutionHandler.resolve_state_groups uses it in "on" mode, with
the usual fallback to Python for rooms without a chain cover index.
Shadow mode keeps comparing through resolve_events_with_store. The
state sets are now shared maps inside the Rust, so a cached map and a
converted one are handled alike.
On the benchmark room (2,000 members, 1,000 conflicted events) a warm
resolution through the state groups takes 9 to 11 ms end to end once
the groups' state is cached, from 15 to 19 through the flat entry
point; Python warm is 37 to 44.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
With the event cache warm, the chain cover queries were what was left of
the database work in a resolution: the position of every event in the
state sets, and the links out of every chain they are on. Both can be
cached.
A chain position never changes once assigned, so it is cached before
the transaction commits, and dropped with the event in case the event is
purged and received again. The links out of a chain only grow, at the
end: an event appended at sequence number n adds links out of (chain, n)
and changes nothing below. So links read while the chain was known to
reach n are complete for every walk that starts at or below n, and the
cache keeps that bound with them. A walk that needs a chain beyond its
bound refetches it. Both caches go on a room purge, which deletes the
room's chains and links.
The chain cover walk now gathers links first (from the cache, following
the links a backwards walk would follow so reached chains are gathered
too, and from the database for the rest) and then walks them once. The
Python walks each batch of its recursive query as it arrives, but a batch
is closed under reachability, so walking the union comes to the same
thing. Fetching only what a sequence-aware walk reaches, rather than the
full reachability closure, leaves the auth difference and the v2.1
conflicted subgraph unchanged: any forwards path that ends inside a
backwards-reachable range runs through chains the backwards walks
reached, below their bounds. The trial comparison against the Python
runs every case through a handler's caches too.
The three store-side caches are bundled as StateResCaches, so the store
and the invalidation hooks treat them as one. The integer-keyed maps in
the chain cover code and the interner use FxHash, and the reverse
position map is one flat map, which together are worth 2 ms on the warm
path of the benchmark. The stats gained the cache hit counts and
timings for the auth chain difference, event loading and the
connection.
On the benchmark room (2,000 members, 1,000 conflicted events) a warm
resolution runs no chain queries and takes 14 to 20 ms end to end, from
27 to 32 with the event cache alone; Python warm is 38 to 41.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Loading and parsing events was most of a Rust resolution's time: against
a warm Python event cache, Rust was level with Python. The handler now
keeps the events it loads, keyed by event ID, so a resolution in a room
it has seen reads them from memory.
The conflict cache's LRU becomes a generic SizedLru shared by both
caches. The Postgres store takes the event cache, reads it before the
database and inserts what it loads. The stats report event cache hits
and the time spent converting state to and from Python.
Invalidation follows the Python event cache rather than having its own
sources: EventsWorkerStore._invalidate_local_get_event_cache and the
room-wide variant drop the same events from the Rust cache. Those two
methods are where every event cache invalidation ends up, including a
partial-state event's rejection status changing, which is the one
change state resolution can observe.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
YAML reads an unquoted on or off as a boolean, so rust_state_res: on
failed config validation. Map true and false to "on" and "off", and
quote the value the Complement start script writes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
The Rust resolution fires its deferred from the tokio threads, and the
fake reactor only waits for those inside a request. Background work that
resolves state, such as processing a PDU after /send has returned, had
not finished when tests checked its effects, so tests failed with
SYNAPSE_TEST_RUST_STATE_RES set.
The handler gains resolve_blocking, and test homeservers use it when
SYNAPSE_TEST_RUST_STATE_RES is set, as make_fake_db_pool already does
for database transactions. tests/state/test_rust_handler.py still covers
the deferred.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
For v2 and v2.1 rooms on the native driver:
- "on" resolves in Rust, falling back to Python for rooms without a
chain cover index. The per-room CPU and database metrics come from
the Rust stats, as the log context does not see that work.
- "shadow" resolves in Python, then in Rust with a copy of the event
map, logs any difference and counts the outcome in
synapse_state_res_rust_shadow_total.
- "off" (the default) is unchanged.
SYNAPSE_TEST_RUST_STATE_RES sets the mode for every test homeserver.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
At every fork in the test graphs, and in the conflict cache tests, the
Rust resolution over an in-memory store must agree with the Python one.
A new test does the same over 40 random graphs of power level,
membership, join rule and topic events, many of which fail auth.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
When the tests run on the native driver, the auth difference tests also
check that the Rust chain cover port gives the same auth difference and
conflicted subgraph as _get_auth_chain_difference_using_cover_index_txn,
and a new test compares events loaded by Rust with those loaded by
Python.
SYNAPSE_TEST_RUST_DRIVER=1 POSTGRES=1 run-trial.sh tests.storage.test_event_federation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
check_state_dependent_auth_rules and auth_types_for_event now call the
Rust implementation, so every caller (federation, sticky events, state
res v1 and v2) uses it. The Python helpers that only they used are gone.
The ed25519 check for third-party invites stays in Python, as a helper
the Rust rule calls.
auth_types_for_event returns its pairs in the order the Python added
them. Event creation lists the auth events in the set's iteration
order, so a Rust HashSet's random order changed event IDs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
The Rust pool set REPEATABLE READ as the session's default transaction
isolation, so autocommit statements ran at REPEATABLE READ too. psycopg2
applies its isolation level only to the transactions it opens, and
autocommit statements keep the server's default of READ COMMITTED.
Synapse's worker read/write lock release runs in autocommit mode and
relies on that: Complement's TestMSC3757OwnedState failed on the Rust
driver with a foreign-key violation while releasing a lock, and passed
on psycopg2.
Mirror psycopg2: the connection opens its implicit transactions with
`BEGIN ISOLATION LEVEL ...` (REPEATABLE READ unless the engine's
`set_isolation_level` says otherwise), and the pool no longer touches the
session default.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Five call sites repeated the same choice: `execute_values` with a
`VALUES ?` statement on psycopg2, or `execute_batch` with a row of
placeholders elsewhere. `execute_batch_values` makes that choice once,
given the statement and the row width, so the store code no longer
needs to know which driver has psycopg2's extras.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
`make_pool` returned the Rust pool under a `type: ignore` because it is
not an `adbapi.ConnectionPool`. Name the slice of the adbapi interface
`DatabasePool` relies on as a Protocol, which both pools satisfy, and
type `make_pool` and the test harness fakes with it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
The pool tests were written with `inlineCallbacks`; trial runs `async`
test methods directly, so use `await`. Adds a test that `txn_limit`
replaces the pooled connection after the configured number of checkouts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Pool exhaustion was invisible until checkouts timed out. Expose
deadpool's occupancy from the Rust pool and publish it as gauges per
pool (max connections, open connections, idle connections, waiting
checkouts), plus a histogram of the time a checkout waits for a
connection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
`database_engine` was the DBAPI module's name, which for the Rust backend
is its Rust module rather than anything recognisable. Report the
configured engine name there, as before for sqlite3 and psycopg2, and
add `database_driver` so deployments on the native driver can be told
apart.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
The Rust backend prepares and caches statements per connection. DDL that
changes a cached statement's result type, such as a schema delta applied
during a rolling restart, makes Postgres reject the stale plan with
SQLSTATE 0A000. Inside a transaction that error used to propagate as a
plain DatabaseError, which `new_transaction` does not retry, so other
workers saw transient failures until their caches turned over.
The cursor already drops the cached entry before the error propagates,
so treating 0A000 like a deadlock means the retried transaction prepares
a fresh plan.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
The Rust shim has no `psycopg2.extras`, so the bulk write helpers that Synapse
runs via `execute_values` need a shim-compatible path. Add a
`uses_psycopg2_extras` engine capability flag (True for psycopg2, False for the
Rust and SQLite engines) and branch the write paths on it:
- The bulk INSERT/DELETE helpers (`simple_insert_many_txn`,
`simple_upsert_many_txn`, `simple_delete_many_batch_txn`, and the
sliding-sync / relations / state-deletion inserts) use psycopg2's single
multi-row `execute_values` on psycopg2, and `execute_batch` (the shim's
pipelined `executemany`) on the Rust and SQLite backends.
- `execute_values` itself is now psycopg2-only (it asserts
`uses_psycopg2_extras`); no query path splices row values into SQL as
literals.
- The four VALUES-join / RETURNING queries that `execute_values(fetch=True)`
served (in `event_federation` and `end_to_end_keys`) can't be expressed by
`executemany`, so they use a parameterised `unnest(...)` form that works on
all three backends.
Also cast the sequence name in `nextval(?::text)`: the Rust driver binds typed
parameters and can't produce a `regclass` from a name, whereas a text argument
coerces (this is the only parameterised sequence call, and Postgres-only SQL, so
psycopg2 and sqlite are unaffected).
psycopg2 behaviour and performance are unchanged; the Rust and SQLite bulk paths
now match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
A checkout from the Rust connection pool (`ConnectionPool.connect`) previously
blocked indefinitely while waiting for a free slot or establishing a new
connection, so an unreachable or overloaded server could hang the worker
threads handling requests forever.
Bound the whole checkout with a configurable timeout: deadpool's `wait` and
`create` timeouts (enabled via its `rt_tokio_1` runtime feature, driven on our
own tokio runtime). Hitting it raises an OperationalError, which Synapse
already treats as a retryable operational error.
Configurable via the `pool_checkout_timeout` database option (milliseconds;
default 30s, `0` disables), read by the Rust engine and plumbed to both the
connection pool and the bootstrap (pool-of-one) connection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
The Rust backend connected with `NoTls`, so it couldn't talk to a Postgres that
requires (or, as libpq defaults, prefers) TLS — a blocker for most managed /
networked databases. Honour the same libpq keys psycopg2 already forwards, so an
existing `database.args` keeps working: `sslmode`, `sslrootcert`, `sslcert`,
`sslkey`, `sslpassword`.
Implemented on rustls (already in the tree via reqwest; pinned to the `ring`
provider since `aws-lc-rs` needs a C toolchain we don't have). `tokio-postgres`
only knows `sslmode` disable/prefer/require and does no verification itself, so a
new `postgres/tls.rs` drives both the handshake mode and the verifier:
disable -> no TLS
allow/prefer -> encrypt opportunistically, no verification (accept-any)
require -> encrypt, no verification (libpq's `require` does NOT verify)
verify-ca -> verify the chain, not the hostname (WebPKI, name bypassed)
verify-full -> verify chain + hostname (WebPKI)
`sslrootcert` (or the system trust store) supplies the roots; `sslcert`/`sslkey`
add a client certificate for mutual TLS. The cert-path keys and the `verify-*`
modes are stripped from the DSN and passed to the pool as explicit params
(`rust_dbapi.split_ssl_params`), since `tokio-postgres` can't parse them.
Behaviour change: the default (no `sslmode`) is now libpq's `prefer` — attempt
TLS, fall back to plaintext — where it was previously no TLS. This matches
psycopg2; plaintext servers still connect via the fallback.
Caveats vs libpq/openssl: rustls is stricter (requires SANs, rejects a
non-CA self-signed root), and an encrypted client key (`sslpassword`) isn't
supported and is rejected with a clear error.
Validated end-to-end against an SSL Postgres: require / verify-ca / verify-full
connect and use TLS, verify-full with only the system roots correctly rejects an
unknown CA, and hostname is checked (verify-full) / ignored (verify-ca). Unit
tests cover the sslmode->verifier mapping; the Rust-backend storage suites still
pass with the new default (prefer falls back against the plaintext test server).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
`new_transaction` sets a transaction's isolation level (for the receipts,
push-actions and purge paths, which ask for a specific level) by calling
`engine.attempt_to_set_isolation_level` before the transaction and resetting it
after. The Rust engine raised `NotImplementedError` there, so those transactions
errored out on the Rust backend.
The shim has no psycopg2-style `conn.set_isolation_level`, so implement it in
SQL: run `SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL <level>` on
the connection. That sets the session's default for *subsequent* transactions,
so it's committed before the caller's transaction begins (SET is transactional);
`new_transaction` resets it to the default (REPEATABLE READ, which the pool
already applies at connection setup) afterwards, scoping the override to the one
transaction — matching psycopg2's behaviour. The level name comes from a fixed
`IsolationLevel` map, not caller input.
Fixes the isolation-level failures in tests.storage.test_event_push_actions,
tests.storage.test_purge and tests.storage.test_receipts on the Rust backend
(20/20 pass); psycopg2 and sqlite are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Add SYNAPSE_TEST_RUST_DRIVER to run the Postgres test suite against the native
Rust driver (sets use_rust_driver on the test database config, off by default).
Teach the harness's make_fake_db_pool to build a RustConnectionPool for the Rust
engine, with its worker thread pool swapped for the threadless test pool so
queries run deterministically on the reactor during `pump` (the shim's block_on
waits there for the tokio workers). Test-database create/drop is admin work, so
it now goes directly over psycopg2 (always available in tests) rather than the
engine module's `connect`.
Running the suite this way surfaces a few real psycopg2-compatibility gaps:
- `fixup_config_defaults` (was `fixup_default_host`) now also fills libpq's
PGUSER / PGPASSWORD environment defaults, not just the host — tokio-postgres
reads none of them from the environment, but Synapse's test setup relies on
them (as psycopg2 does).
- `build_dsn` skips `None`-valued args, matching psycopg2's handling of unset
kwargs.
With these, a homeserver boots on the Rust backend under the test harness and
starts running real transactions and background updates. The remaining psycopg2
type-coercion / behaviour differences are addressed in the commits that follow;
psycopg2 and sqlite runs are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
`DatabasePool`'s `inner_func` calls `conn.reconnect()` when a connection is
found closed, or to recycle one that has exceeded the per-connection
transaction limit (`txn_limit`). The Rust DBAPI2 adapter had no `reconnect`, so
those paths (off by default, but real) would have raised.
Add `reconnect` to the adapter: it returns the current connection to the pool
(or discards it if unusable) and checks out a fresh one. The adapter now holds
the pool it was checked out of, with an `owns_pool` flag distinguishing a shared
pool (RustConnectionPool — reconnect from it, don't close it) from a bootstrap
pool-of-one (`rust_dbapi.connect` — closed together with the connection).
Tested: reconnect swaps in a working connection; runWithConnection and the
bootstrap/adapter paths are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
A running homeserver exercises three shim capabilities beyond what the unit
tests reached; add them so it boots on the Rust backend end to end:
- `prepare_database` reads `db_conn.autocommit`; add a readable `autocommit`
getter to the shim `Connection` (and the adapter), mirroring psycopg2's
property.
- schema preparation runs multi-statement scripts via
`engine.execute_script_file` -> `cursor.executescript`; route
`LoggingTransaction.executescript` to the shim's multi-statement primitive
for the Rust backend (as it already does for sqlite), and expose
`executescript` on the DBAPI2 adapter cursor.
- store loading binds Python lists as Postgres arrays for `column = ANY($1)` /
`!= ALL($1)` (Synapse's `make_in_list_sql_clause`); add a `PgValue::Array`
variant that classifies a `list` and encodes it as an array of the element
column type, and accept array column types in `ToSql`.
With these, a homeserver boots on the Rust backend: it prepares the full schema
(176 tables), listens, `/health` returns OK, and DB-backed client endpoints
respond correctly.
Tested: array `to_sql` / classification and array-column `accepts` (Rust unit
tests); and, against a live Postgres, `= ANY($1)` list binding, adapter
`executescript`, and the `autocommit` getter. sqlite and psycopg2 homeserver
boots remain green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
The Rust `ConnectionPool` takes the homeserver's `RustRuntime`, whose
tokio runtime drives the pool. `make_pool`, `rust_dbapi.connect` and
`make_conn` now take it too, and `DatabasePool` and `Databases` pass
`hs.get_rust_runtime()`. `make_conn` falls back to a standalone runtime
on the global reactor for the scripts and tests that have no homeserver
to take one from.
The adapter and engine tests build their pools through the shared
`_make_pool` helper for the same reason.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Turn the Rust backend on with `database.use_rust_driver: true` (alongside
`name: psycopg2`). Keeping `name: psycopg2` means the engine is still selected as
Postgres and all the `isinstance(engine, PostgresEngine)` dialect checks hold;
the flag only swaps the driver implementation.
- `create_engine` returns `RustPostgresEngine` instead of `Psycopg2Engine` when
the flag is set.
- `make_pool` gains a `clock` argument and, for the Rust engine, builds a
`RustConnectionPool` (via `_make_rust_pool`) instead of an adbapi pool:
it derives the libpq DSN from the config `args`, sizes threads/connections
to `cp_max`, passes the engine's synchronous_commit / statement_timeout,
starts the pool, and registers a shutdown hook via the clock. Its declared
return type stays `adbapi.ConnectionPool` — `RustConnectionPool` provides the
`_db_pool` subset `DatabasePool` uses (runWithConnection / threadID /
running / threadpool) — so callers that lean on adbapi-specific methods keep
type-checking.
- `RustConnectionPool` takes synchronous_commit / statement_timeout_ms and
threads them into its pooled connections' session setup.
- `DatabasePool` passes its clock to `make_pool`; the test harness's
`make_fake_db_pool` accepts the new argument.
Tested: create_engine honours the flag (and the Rust engine is still a
PostgresEngine, not a Psycopg2Engine); make_pool builds a started
RustConnectionPool that serves a query as `_db_pool`. psycopg2 and sqlite
homeserver boots remain green. A full homeserver boot on the Rust backend isn't
covered here because the test harness patches `make_pool` with a synchronous
adbapi fake.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Startup opens a bootstrap connection via `make_conn` (outside the runtime pool)
and validates it with `engine.check_database` before `prepare_database` runs.
Neither worked for the Rust backend: the Rust module has no `module.connect`,
and check_database/server_version were NotImplementedError stubs. Fill them in:
- `rust_dbapi.build_dsn` turns the libpq-style `args` into a DSN string, and
`rust_dbapi.connect` opens a standalone connection (a pool of one, kept
alive by the returned Connection) for one-off bootstrap use.
- `make_conn` routes the Rust engine through `rust_dbapi.connect` (with the
engine's synchronous_commit / statement_timeout) instead of
`engine.module.connect`; psycopg2 and sqlite are unchanged.
- `RustPostgresEngine.check_database` reads the server version over a cursor
(`SHOW server_version_num`) rather than psycopg2's `conn.server_version`,
and applies the same version / encoding / collation / ctype checks;
`server_version` is derived from the cached value.
Tested: build_dsn quoting; and, against a live Postgres, make_conn +
check_database + server_version + a query. psycopg2 and sqlite homeserver boots
(make_conn → check_database) remain green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Make RustConnectionPool a drop-in for DatabasePool._db_pool so real
runInteraction traffic can flow through it (once make_pool is switched):
- it now hands each function a DBAPI2-adapter connection (rust_dbapi.Connection)
wrapping the pooled shim, so LoggingDatabaseConnection.cursor() yields a
working LoggingTransaction and the engine's in_transaction / is_closed /
set_autocommit operate on it;
- the entry point is named `runWithConnection` (matching
twisted.enterprise.adbapi.ConnectionPool, which database.py calls by that
name), alongside `threadID` for the transaction-limit path. With
`threadpool` and `running` already present, the pool covers the slice of the
adbapi interface DatabasePool uses.
A new test drives a full transaction through the pool the way
DatabasePool.runWithConnection's inner_func does — engine.in_transaction check,
LoggingDatabaseConnection + LoggingTransaction, `?`→`$n` conversion, commit —
off the reactor thread, and gets the result back via the Deferred.
Still outstanding before make_pool can return it: `reconnect` on the connection
(transaction-limit / closed-connection paths) and the startup path
(make_conn / check_database).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Previously RustPostgresEngine subclassed the psycopg2 PostgresEngine, which
dragged psycopg2 into the Rust path and conflated "is Postgres" with "is
psycopg2". Split the hierarchy so both drivers are siblings:
- `PostgresEngine` (new `postgres_base` module) is now the driver-agnostic
base holding the shared SQL-dialect and config logic (single_threaded,
supports_using_any_list, row_id_name, get_db_locale, check_new_database,
lock_table, synchronous_commit / statement_timeout). It has no driver
dependency, so it always imports. Everything touching a live connection,
the DBAPI2 exception module, or the placeholder style is left abstract.
- `Psycopg2Engine(PostgresEngine)` holds the psycopg2 specifics (register_type
/ register_adapter, isolation-level map, conn.status/closed/server_version,
`%s` placeholders, psycopg2 execute path, `uses_psycopg2_extras = True`).
- `RustPostgresEngine(PostgresEngine)` is re-parented onto the base (no longer
inherits psycopg2). It passes the Rust DBAPI2 module to the base and gets
NotImplementedError stubs for the still-psycopg2-shaped check_database /
server_version (part of the deferred startup wiring).
The base keeps the name `PostgresEngine`, so all ~91
`isinstance(engine, PostgresEngine)` checks across the storage layer (which mean
"emit Postgres SQL") hold for both drivers unchanged. `create_engine` now
returns `Psycopg2Engine` for `name == "psycopg2"`.
Verified: full lint clean; Rust engine/adapter tests pass; a psycopg2 homeserver
boots and runs (test_room_search under Postgres) and the sqlite path is
unaffected (test_room_search / test_event_federation under sqlite).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Synapse's LoggingTransaction drives a cursor through the DBAPI2 spelling
(fetchone/fetchmany/fetchall, iteration, rowcount/description as properties),
but the Rust shim cursor exposes fetch_one/fetch_all/fetch_next_batch and
rowcount()/description() as methods. Rather than reshape the Rust API, add thin
Python `Connection`/`Cursor` wrappers that present the DBAPI2 shape and delegate
to the shim:
- `Cursor` maps fetchone/fetchmany/fetchall/__iter__ and rowcount/description
onto the shim, and tracks exhaustion so fetching past the end keeps
returning "no more rows" (the shim raises instead);
- `Connection.cursor()` returns the adapter cursor, and the transaction-control
and engine-facing methods (commit/rollback/close, set_autocommit, is_closed,
in_transaction) delegate straight through.
An end-to-end test drives a real LoggingTransaction backed by the adapter and
RustPostgresEngine: `?` placeholders are converted to `$n`, a query runs, and
rows come back via fetchone/iteration/description.
Not handled yet: execute_batch / execute_values (psycopg2 extras that
LoggingTransaction calls directly for PostgresEngine) still need a routing
change to reach a shim-backed implementation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
