The rooms row for a new room is inserted by the persist transaction,
Rust or Python, when the create event's context is flagged, instead of
by a store_room transaction of its own beforehand. The insert also
claims the room ID: a row already there fails the batch with
RoomIdCollisionError, and create_room runs the whole creation again
under a fresh ID, perturbing the create event's timestamp for v12+
rooms as the old loop did. The room version is prefilled on the upgrade
path too, as the row is no longer there to read.
The chain cover check skips its rooms read when the create event is
among the events, since the row may not be stored yet, and the
createRoom bench advances the fake clock per room so that rooms do not
collide.
Creating a room read the database from Python for things the request
already knows, each a round trip of its own:
- The creator's and each invitee's profile, for the membership events.
`get_profileinfo` is now cached by user ID, invalidated where the
profile is written: setting the displayname or avatar, registering
with a displayname, and deleting the profile.
- Whether a joining user is already joined, in
`persist_and_notify_client_events`, and whether a membership event's
target is joined, in the push rule evaluator. A membership event that
replaces no membership in the room state, which `replaces_state` in its
unsigned data records, is of a user who is not joined, so there is
nothing to read.
- The new room's version, when the invite is built in Python. The room
handler prefills `get_room_version_id` once the room is stored.
On the native driver with the Rust persister the bench's Python
interactions per room fall from 13 to 11 for a plain room and from 19 to
14 for a room with an invite, and the Postgres round trips the proxy
counts from 72 to 70 and from 86 to 81. `SYNAPSE_BENCH_TRACE` prints a
stack for each interaction with a given description.
A join without profile data read the user's display name and then their
avatar URL, two transactions. For a local user with neither in the
content, `get_profileinfo` now reads both at once. A remote user, or
content that has one of the two, takes the same path as before.
If the user has no profile row, the first read used to fail with a 404,
logged, and leave both out. Now both are left out without the log line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
Behind `experimental_features.rust_room_creation: on`, the room handler
collects the events that follow the create event (the creator's join,
power levels, join rules, history visibility, the initial state, name,
topic) as `(type, state_key, content)`, and `LocalEventSigner` builds,
hashes and signs them all in one call. Each event's prev event, depth
and auth events follow from the one before, as `create_event` gives
them for a batch.
Python keeps every check: the requester checks ran when the create
event was built, and each built event still goes through
`validate_builder`, `validate_new` and the relation check before
joining the batch, which persists through `handle_new_client_event` as
before. Invites are still built by `create_event`.
Rooms the Rust builder does not cover are built in Python and counted
as fallbacks: v1 and v2 rooms, MSC4242 state DAG rooms, appservice
senders, modules with `check_event_allowed`, initial `m.room.aliases`
events, and `m.call.invite` or `m.room.create` in the initial state. If
Rust fails, the events are built in Python. If Python rejects them too,
the request was bad and gets Python's error. Otherwise it counts as a
Rust error. In tests,
`SYNAPSE_TEST_RUST_ROOM_CREATION=on` turns the builder on and makes a
failure fail the test.
Python orders auth events by iterating a set, so the order changes with
the process's hash seed. Rust keeps the order of the auth types. A test
creates rooms of several configurations and room versions in Python,
rebuilds their events in Rust from the same inputs, and compares them
with the auth events taken as a set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
Rust could read state groups but not write them. A new
`store_state_group_chain` on the `RustStore` writes the chain
`store_state_deltas_for_batched` stores: an optional full group for the
state before the first event, then one delta group per state event. It
runs in one native transaction, allocating from `state_group_id_seq` as
Python does. A `prev_group` is locked against deletion and its pending
deletion pushed back, and a missing one fails with Python's message.
When the state groups share the main database and that uses the native
driver, `store_state_deltas_for_batched` calls it instead of running a
Python transaction. The context bookkeeping and the cache prefill are
shared by both paths. Room creation, and anything else sending a batch,
saves one Python transaction.
A twin test stores the same batches through both writers and compares
the rows and the groups the contexts get.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
After `batch_persist_unpersisted_contexts`, an event's `EventContext` only
had state group IDs, so every read of the state before the event went to
the state store. For a new room that was eight state-group reads per
`/createRoom`: six from `deduplicate_state_event`, one from the
create-event check in `persist_and_notify_client_events`, and one from the
persister working out the new current state.
Keep the state map the batch was built from on the `EventContext`, as a
private attribute that `get_prev_state_ids` prefers and that is not
serialized, so a context sent to another worker reads the state group as
before. A context that arrives with a state group of its own, which the
batch then replaces, drops its map. `store_state_deltas_for_batched` also prefills the state group
caches with the state after the last event of the batch, through the
helper `store_state_group` now shares.
On the trial harness (SQLite) a plain room drops from 35 to 27 database
interactions, and `/createRoom` from 17 to 10 database transactions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
`/createRoom` persisted the create event on its own, then the creator's
join through `update_membership`, then the rest of the initial state as a
batch. Each round paid for the persist queue, the purge lock, forward
extremities, chain cover, a replication notify and a wake-up of the stats
and presence pollers.
Build the create event and the join against the in-memory state map like
the other initial events, and persist all of them in one batch. The join's
content comes from `RoomMemberHandler.build_creator_join_content`, which
shares the suspension and profile handling with `update_membership_locked`
through the new `_prepare_membership_content`. The MSC4291 create event,
built before the room ID is known, now also leaves its context unpersisted
for the batch.
`store_state_deltas_for_batched` accepts a batch that starts the room: it
stores the empty state before the create event as a full state group, then
the delta chain. If the first context already has a group for the state
before it (a module rewrote the event), the chain builds on that instead.
`calculate_context_info` and `UnpersistedEventContext` treat an empty state
map as known empty state. Before, an empty map tripped an assertion. Room
creation never passed one, but federation could (`compute_event_context`
with an empty remote state), and such an event is now stored with an empty
state group and left to the auth checks.
Two cases keep persisting the create event and the join on their own:
v1 and v2 rooms, whose builder reads prev and auth event hashes from the
database, and servers with a `check_event_allowed` module callback, which
is given the room state from the database and may have the event rebuilt.
On the trial harness (SQLite) a plain room drops from 68 to 35 ms and 26
to 17 database transactions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
`jsonschema.validate` checks the schema against the metaschema on every
call. For power levels that cost more than validating the event: about
9 ms per `/createRoom`. Check the power levels and mentions schemas once
at import and validate with bound validator instances, reporting the same
best-match error.
`create_event` also ran `validate_new` a second time on the event that
`create_new_client_event` had just validated. Drop the repeat.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
`tests/rest/client/bench_createroom.py` times room creation per room in
the trial harness, with optional cProfile dumps and per-room counts of
database interactions and SQL statements. It skips unless
`SYNAPSE_BENCH_CREATEROOM` is set.
`scripts-dev/createroom_bench.py` times it against a real server, reusing
`persist_bench.Server`. That server now lifts the room-creation rate limit
and can run on psycopg2 instead of the native driver.
The test template database's maintenance connection now always uses
psycopg2, as `setup_test_homeserver` does. The Rust driver's `postgres`
module has no `connect`, so it failed under `SYNAPSE_TEST_RUST_DRIVER=1`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVLsNh6akt4ifEizY4D1ta
The phase 6 bench found Rust making 2 to 6 times Python's queries: 8 per
room computed, and 4 on an incremental request where nothing changed.
- Bundled aggregations read the thread summary, edits and references
through Rust caches that mirror Python's `get_thread_summary`,
`get_applicable_edit` and `get_references_for_event`, so Python's
invalidations reach them. The thread summary takes one statement with a
window count, where it took a transaction of up to two. v2 sync in Rust
shares these reads.
- `ignored_users` reads through a mirror of Python's cache.
- The `instance_map` table is cached. It only gains rows, so a stored
token naming a writer the cache lacks drops the cache and reads again.
- The self-leave query is skipped when the membership stream cache says
the user's membership has not changed since the `to_token`, the
shortcut Python's query carries as a TODO.
- On a room's first send, the name and the required state come from one
state read. Rooms with heroes keep a second read for their memberships.
On the bench, initial sync goes from 173 queries to 49, the range
expansion from 662 to 178 (Python makes 335), and a no-op incremental
from 4 to 0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KETBASkkVomGNnbut9NWQq
Rust's `encode_rooms` returned a dict per room, which Python then
JSON-encoded again. On the bench's range expansion those two steps were
17 of Rust's 31 ms under the GIL.
`encode_rooms` now returns the rooms as one `bytes` value of JSON object
members and encodes them with the GIL released. The servlet encodes the
rest of the response on a thread, as `respond_with_json` did, splices
Rust's rooms into the rooms object next to any Python fallback rooms, and
sends the body with `respond_with_json_bytes`.
The bench counts the encoder's time as off the GIL. Rust's reactor time
on the range expansion goes from 0.13 of Python's to 0.06.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
- Count Rust's calls back into Python (the events stream cache) as reactor
time: they hold the GIL in production too.
- Profile only the timed request, not the setup request or the message
sent before it.
- Measure what the plan meant by "500 rooms sent": a [0,19] request on a
connection that sent 500 rooms before. The [0,499] list stays as its own
scenario.
- Say that trial runs Python's database functions on the reactor thread,
so reactor CPU is the GIL time of both sides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
An opt-in trial module that times each request on a Python and a Rust
connection of one user, in turn, for the scenarios of phase 6 of the
sliding sync plan: initial sync, incremental with nothing or one event
changed, a range expansion, a 2000-room account and a connection with 500
rooms sent, each with and without the account data and receipts
extensions. It reports wall time, reactor-thread CPU less the time inside
Rust calls, process CPU, and database time, queries and transactions, and
can dump a cProfile per scenario.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
- Re-raise SlidingSyncUnknownPosition from finish_request. A racing
request on the same connection can clear the position before the
persist, and Python would fail the same way, so a Python retry only
repeats the work.
- Treat an empty events stream cache as a miss, as Python's
get_entities_changed does, so Rust asks the database.
- Copy the room statuses out of the connection state before building
Python objects, so the lock is never held across a Python call.
- Trace the rooms Python computes for Rust, as the Python path does.
- Test the per-room fallback: Python computes a room Rust leaves to it,
and its changes reach the Rust state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
In `rust_sliding_sync: on` mode, `current_sync_for_user` is now four Rust
calls around the Python extensions: load the connection state, compute
the interested rooms (the port of `_compute_interested_rooms_new_tables`),
compute the rooms, and finish the request (unsent and sent rooms, then
the persist).
The connection state is a Rust pyclass. The extensions read and update it
through views with the surface of Python's `PerConnectionState` and
`MutablePerConnectionState`. Receipts tokens stay in their database form
in Rust and become `MultiWriterStreamToken`s only when Python asks.
The whole request falls back to Python when the sliding sync tables are
not filled, for a filter Python does not implement either, and on any
Rust error. Rooms Rust leaves to Python run `get_room_sync_data` over a
Python copy of their slice of the state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
The differential test now also checks that the Python and Rust sides
make the same changes to their connection's state: the room configs and
the lazy-loaded members. It also covers an avatar change, a ban, a
rejected remote invite, a state reset out of a room, and a prev_batch
token that names another writer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L51DoxB8VnH8UjXJFDgpNG
Python validates the body with pydantic as before. Rust parses the same
bytes with serde into `SlidingSyncBody`, which follows the pydantic model
field for field, including where it tells an explicit `null` from a missing
field. Lists and room subscriptions keep document order, and a repeated key
keeps its first position and last value, as the dict `json.loads` builds.
`SlidingSyncRequest` is the body plus what Python adds: the user and device,
the room key and connection position of `pos`, the room key and string suffix
of the token the response runs up to, the admin's soft-failed policy, whether
the background updates have finished, the excluded rooms and the extension
flags. `SlidingSyncHandler._build_rust_request` builds it.
The servlet keeps the body's bytes in `SlidingSyncConfig.body_bytes` when the
Rust handler is in use. A config built without a request is re-encoded.
`tests/rest/client/sliding_sync/test_rust_request.py` parses 400 generated
bodies (unknown fields, nulls, repeated keys, escapes) both ways and
compares, and checks the re-encoding parses the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The new `sync/relations.rs` ports the relations handler's
`get_bundled_aggregations`: thread summaries with participation and
ignored users' replies subtracted, references, and the latest edit. The
Rust sync handler bundles them for limited and newly joined timelines,
loads the redactions of the bundled events, and the encoder serializes
them. Python no longer bundles the rooms Rust computes, and the room
entries hand Python the bundles only when something reads the timeline.
A thread whose latest reply is from an ignored user falls back to
Python for that room: finding the latest reply that is not takes the
relations pagination, which is not ported.
The relations queries read the database each time. Python's relations
caches are not mirrored.
Shadow mode now compares the Rust bundles with Python's, in the encoded
rooms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The rooms the Rust sync handler computes now reach the servlet as
`RustJoinedSyncResult` and `RustArchivedSyncResult`. They subclass the
Python entries, and their fields become Python objects only when read.
The servlet encodes them in one call to the new Rust `RoomsEncoder`,
which ports `encode_room` over the existing serializer. If that call
fails, Python encodes them.
The Rust handler loads each room's redactions as it computes the room.
The serializer reads redactions, bundles and module additions through a
`SerializeLookups` trait, so that it takes Rust events as well as Python
maps. Bundled aggregations are still computed in Python.
Shadow mode now also compares the encoded JSON of each room.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5b7Ruy4cgt27Mjq6Ptxbb
The first Rust database function to get a cache, and the first Python
`@cached` function to be replaced by one. The Rust `/versions` handler
calls `Store::is_feature_enabled_for_user` on every authenticated request,
and until now each call ran a query. The Python `is_feature_enabled` had a
cache of the same rows under another name, which every writer would have
had to invalidate alongside. Now there is one cache, keyed on
`(user_id, feature)`, and both sides read through it.
The main datastore builds the Rust `Store` in its `__init__`, over its own
`db_pool`, and `hs.get_rust_store()` hands it to the Rust handlers: the
datastore owns the store, the handlers borrow it. The store's
`is_feature_enabled_for_user` reaches Python as a callable cache object,
bound on the datastore under that name. Calling it answers a hit with an
already-fired Deferred, without leaving the reactor thread, and only a
miss goes through tokio. Being a `RustCache`, it is also what
`_invalidate_cache_and_stream`, replication and `register_cache` take, so
`set_features_for_user` invalidates it with the one call it already made,
and the name is spelled once in Rust, where the cache is built.
`is_feature_enabled` keeps its global-config short cut in Python and
delegates the per-user lookup.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
These things are not needed now that MSC2965 has stabilised. In any
case, they
should never have been exposed without a config flag in the first place.
This speeds up the /createRoom calls a tiny bit by not checking the
senders push rules.
This helps speed up a lot of test runs in Element Web which
runs a lot of tests in one-person rooms (and creates a heck of a lot of rooms)
## Summary
- default `request_token_inhibit_3pid_errors` to `true` to prevent
account enumeration
- document the new default and the explicit opt-out
- add an upgrade note and regression coverage for the default
Fixes#18415.
Part of #19414 (Support Matrix 1.16).
⚠️ Before merging, wait for the following PRs to be merged first:
- https://github.com/element-hq/synapse/pull/20081
- https://github.com/element-hq/synapse/pull/20169
- https://github.com/element-hq/synapse/pull/20171
---
Since Matrix 1.16, the
[spec](https://spec.matrix.org/v1.19/client-server-api/#get_matrixclientv3sync)
defines the stable identifiers for
[MSC4222](https://github.com/matrix-org/matrix-spec-proposals/pull/4222):
the `use_state_after` query parameter on `GET /_matrix/client/v3/sync`,
and the `state_after` room response field:
> **`use_state_after`** — `boolean` — Controls whether to receive state
changes between the previous sync and the start of the timeline, or
between the previous sync and the end of the timeline. If this is set to
`true`, servers MUST respond with the state between the previous sync
and the end of the timeline in `state_after` and MUST omit `state`. […]
>
> **`state_after`** — […] Required if `use_state_after` was set to true
in the request, even if it is empty. Added in `v1.16`
>
> — [Matrix
Spec](https://spec.matrix.org/v1.19/client-server-api/#get_matrixclientv3sync)
Synapse so far only implemented the unstable identifiers
(`org.matrix.msc4222.use_state_after` /
`org.matrix.msc4222.state_after`), gated behind the `msc4222_enabled`
experimental flag and the per-user experimental features admin API.
This PR:
- accepts the stable `use_state_after` query parameter, unconditionally,
and responds with the stable `state_after` field. The unstable query
parameter is kept for a transition period; clients opting in with it get
the unstable response field name back, so unstable-era clients see an
unchanged wire format. If both parameters are set, the stable one wins.
- removes the `msc4222_enabled` experimental flag and the `msc4222`
per-user feature from the experimental features admin API, since the
behaviour is no longer gated.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
TLDR: `PUT` on a server-default push rule now applies the same existence
check that `GET` already does. A rule that is not served cannot be
modified.
Synapse ships some server-default push rules behind experimental feature
flags, so the set of server-default rules a user has depends on the
server's configuration. Most flags add rules that only exist while the
flag is on.
[MSC4210](https://github.com/matrix-org/matrix-spec-proposals/pull/4210)
works the other way: it removes the three legacy mention rules while its
flag is on. The flags that change the set are:
- `msc1767_enabled`:
[MSC1767](https://github.com/matrix-org/matrix-spec-proposals/pull/1767)
extensible events rules
- `msc3381_polls_enabled`:
[MSC3381](https://github.com/matrix-org/matrix-spec-proposals/pull/3381)
poll rules
- `msc3664_enabled`:
[MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664)
reply rule
- `msc4028_push_encrypted_events`:
[MSC4028](https://github.com/matrix-org/matrix-spec-proposals/pull/4028)
encrypted event rule
- `msc4306_enabled`:
[MSC4306](https://github.com/matrix-org/matrix-spec-proposals/pull/4306)
thread subscription rules
- `msc4210_enabled`: removes `.m.rule.contains_display_name`,
`.m.rule.contains_user_name` and `.m.rule.roomnotif`
`GET /pushrules/` only lists the rules that exist for that
configuration. `GET` on a single rule that is not in the set returns
`404 M_NOT_FOUND`. The same set is used to evaluate notifications, and
to build the `m.push_rules` account data in `/sync`.
`PUT .../enabled` and `PUT .../actions` did not use that set. They
accepted any rule ID that Synapse knows about, whether or not the server
currently serves it. Clients could modify a rule the server had just
said did not exist. The request returned `200`, the change was stored,
and it was then ignored. Both endpoints now return `404 M_NOT_FOUND` for
such a rule, as [the spec
requires](https://spec.matrix.org/v1.19/client-server-api/#put_matrixclientv3pushrulesglobalkindruleidenabled)
for a rule that does not exist.
Before (`msc4210_enabled: true`):
```
GET /_matrix/client/v3/pushrules/global/override/.m.rule.roomnotif
404 {"errcode": "M_NOT_FOUND", "error": "..."}
PUT /_matrix/client/v3/pushrules/global/override/.m.rule.roomnotif/enabled
{"enabled": false}
200 {}
```
After:
```
GET /_matrix/client/v3/pushrules/global/override/.m.rule.roomnotif
404 {"errcode": "M_NOT_FOUND", "error": "..."}
PUT /_matrix/client/v3/pushrules/global/override/.m.rule.roomnotif/enabled
{"enabled": false}
404 {"errcode": "M_NOT_FOUND", "error": "Unknown rule"}
```
This is a no-op for default configurations. All of these flags are off
by default, and rules that are served are still modifiable as before.
Use a new, dedicated endpoint for scheduling delayed events. This should
deprecate the `delay` query parameter on the `/send` and `/state`
endpoints.
Co-authored-by: Paul Chobert <paul@chobert.fr>
Co-authored-by: Eric Eastwood <erice@element.io>
In a similar vein to #20315, this attempts to speed up Synapse when
delayed events aren't in use. Here we skip doing any work if the room
event stream has not advanced.
Fixes: #19661
Part of: MSC4354
First introduced some helpers for querying a sharded stream, which were moved to
https://github.com/element-hq/synapse/pull/20108.
- SQL builder for 'give me rows between two sharded tokens'
- a function to return the 'new' from_token after a limited read
Then make the sticky events portions of both oldschool and sliding sync use
sharded tokens.
This prevents a 'torn sync' problem where, because the events and sticky
events streams are both sharded with the same writers:
1. persisting a sticky event leads to an event row and a `sticky_events`
row being created for event `e` (on writer `W`).
2. Suppose another writer is still persisting, so the position of the
stream for writer W advances, but the global
['linear'](https://element-hq.github.io/synapse/latest/development/synapse_architecture/streams.html?highlight=%E2%80%9Clinear%E2%80%9D#current-stream-id)
(I have been calling this 'baseline') position does not advance yet.
3. sync is sharding-aware for the regular events stream, so `e` appears
as a real-time timeline event, thus coming down sync
4. since sticky events were not sharding-aware, `e` is not detected as a
sticky event in this sync response. The sticky event position doesn't
advance in the sync token, either.
5. (assume the position advances because other writers have finished
writing and so the 'linear' position advances for the sticky events
stream)
6. the client then syncs again, using the new sync token. This time, `e`
appears in the sticky event section.
If we make sticky events sharding-aware in sync, then `e` would appear
for consideration in both the timeline section **and** the sticky event
section in the first sync.
It then gets deduplicated out of the sticky section since it is
appearing in the timeline section in the same sync.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
In `/sync` there is a mode called lazy loaded (LL) members. In this mode
`/sync` does not return all memberships in the room, instead it only
returns a) membership changes, and b) the memberships of senders of
events in the timeline (if not previously sent).
Synapse implements a lazy-loaded member cache to avoid repeatedly
sending down the same memberships when a single user sends multiple
events. It does this by storing for each user/device the memberships it
has previously returned in an LRU cache. Note that it is fine for the
server to return the same membership, clients handle that fine, but it
is a waste of bandwidth.
The current lazy-loaded membership cache has three intertwined flaws:
1. It does not handle retries or clients re-using old tokens.
2. It incorrectly applies the membership cache to membership changes,
not just memberships added by lazy-loading.
3. It does not correctly handle multiple "sync streams" from the same
device.
In this PR we explicitly fix the first two issues in two commits, and
the third issue is mitigated:
1. Alongside each membership in the LL cache we store the `next_batch`
position of the response that carried it. A client that processed a
response syncs from its `next_batch` next. When a request arrives with a
`since` token, we delete every membership recorded after that token, as
they were sent in responses the client never processed. Memberships
recorded at or before `since` are kept.
2. We always return memberships that have changed, even if they appear
in the LL member cache.
These changes also half mitigate the issue of running multiple `/sync`
loops per-device (though we do not see this in the wild), as we'd at
least always return the changed membership states.
Fixes#19978 and #20278, supersedes #20279
Fixes#15871
### What was broken
Sending `{"auth": null}` to any endpoint that requires user-interactive
auth (for example `POST /keys/device_signing/upload`, `DELETE
/devices/{deviceId}` or `POST /register`) returned a 500 instead of the
usual 401 with the available flows. A non-object `auth` on endpoints
that do not validate the body with a model (such as `/register`) could
also 500, e.g. `{"auth": ["session"]}`.
### Root cause
`AuthHandler.check_ui_auth` did `clientdict.pop("auth", {})`, so the
`{}` default only applies when the key is missing. An explicit `null`
came through as `None` and the following `"session" in authdict` raised
`TypeError`. `get_session_id` had the same assumption that `auth` is a
dict.
### Pull Request Checklist
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Signed-off-by: Ankit Jha <ankit.jha@tradomate.one>
This pull request fixes a few issues with the profile update stream,
when a user leaves a room. This is basically just mimicking what we
already had for when someone leaves a room, and they no longer share any
rooms, but in reverse - ie when we leave a room, and no longer share
rooms with some users. This was missed in the implementation when adding
the profile update stream for legacy and sliding sync.
[Fix missing profile update stream rows on user leaving
room](https://github.com/element-hq/synapse/commit/e953322226731a99718ebe13139b9c5f05ae4b62)
When a user leaves a room, profile update stream rows are generated with
the `LEFT_ROOM` action for each user in the room that no longer shares a
room with the user who left the room.
This also needs to happen in reverse. The user who left the room needs
to have a profile update stream row for each user they no longer share a
room with.
If we don't do this, clients may keep stale data around even after they
don't share a room with a user, which may mean they don't know when to
refetch profiles after re-joining a room with the stale profile data
user.
[Clear out old profile update stream rows when leaving a
room](https://github.com/element-hq/synapse/commit/e7642bf141b098c1eefaf74655e5e7e18415500b)
When we leave a room, ensure profile update stream rows are cleared out
for every user we no longer share a room with. This is the same as what
happens when someone else leaves a room, but in reverse. The only
remaining profile update stream row should be the `LEFT_ROOM` action.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
This is to prevent the case where URL previews return a MXC that
immediately 404s because the media in question has been quarantined.
This is mostly to help implementations which currently show an ugly
empty preview due to the MXC being sent down, despite being invalid.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [ ] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
[MSC4140](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/4140-delayed-events-futures.md),
"Account deactivation":
> when an account is deactivated, the homeserver MUST cancel that
account's delayed events which have not yet been added to a room's event
DAG. These cancelled records MAY be removed immediately, including their
stored event content, as an exception to the usual finalised-record
retention policy.
Deactivation currently leaves the user's scheduled delayed events,
including their content, in place, and they are still attempted at their
scheduled time:
| Delayed event scheduled by the user | What happens after deactivation
today |
| --- | --- |
| Message or state event, fires after deactivation has made the user
leave the room | The send fails with a 403 (user not in room) and the
record is dropped |
| Message or state event, fires before deactivation has made the user
leave the room (leaving happens room by room in the background) | The
event is sent |
| `m.room.member` join for themselves in a public room | The deactivated
user re-joins the room |
### What changes
- Deactivating an account (client or admin API) removes the user's
unsent delayed events as its first step, and re-arms the send timer for
whatever is scheduled next.
- Deactivation can run on a worker, while the send timer lives on the
main process, so the cancellation goes through a new replication
request.
- A delayed event whose send has already started is left alone, as with
a normal cancel: it is already on its way into the DAG, and the send
path removes its record itself.
Suspension and locking are unchanged.
### After #19038
Today a cancelled delayed event is simply deleted, so this PR deletes
the user's records too. #19038 changes cancellation to keep the record
and mark it as cancelled ("finalised"), so that clients can look up what
happened to a delayed event. Once it has landed, deactivation should
probably finalise the user's records as cancelled the same way, rather
than delete them. The MSC allows either: it permits removing the records
immediately, content included, as an exception to the usual retention of
finalised records, which is worth doing at least when the user asks to
be erased.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Andrew Ferrazzutti <af_0_af@hotmail.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
Previously the tokio runtime was stashed in a hidden attribute on the
reactor object, installed lazily by whichever Rust code first needed it,
and started via `callWhenRunning`.
Instead, we create a `RustRuntime` (accessible via
`HomeServer.get_rust_runtime()`) that holds any per-reactor Rust state,
such as the tokio runtime. It is constructed lazily on use. Rust
consumers (`HttpClient`, `VersionsHandler`, the Python DB pool wrapper)
now receive the runtime or reactor handle explicitly, and the
`reactor.run()` / manual-startup workarounds in tests are no longer
needed.
We also add helper wrappers in Rust for `Reactor` and `HomeServer` that
exposes the needed functionality.
The aim is to allow us to have a Rust-side clock (mainly to get the
current time), that respects the unit test per-reactor time management.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR fixes the issue described as comment here:
https://github.com/element-hq/synapse/issues/18793#issuecomment-3502202379
In Element Call, this shows up as ghost participants: someone who left
the call keeps being displayed until a later state change refreshes the
room.
The bug is not specific to Element Call: any state event can be
affected, RTC membership just changes often enough to make it visible.
## What happens
Alice has a client syncing against a homeserver where events are
persisted by one worker (the event persister) and `/sync` is served by
another (the sync worker). Her client is parked in a long-poll: `GET
/sync?since=s99&timeout=30000`.
Bob joins a call at the same moment Carol sends a message. Carol's
message reaches the persister first; Bob's `m.call.member` arrives while
that write is still in flight, so the per-room persist queue groups them
into one transaction:
```
events (each gets its own stream ordering):
stream_ordering 100: m.room.message Carol
stream_ordering 101: m.call.member Bob (state)
current_state_delta_stream (how state_after finds state changes):
stream_id 100 ────► (m.call.member, @bob) -> $bob_join_call
▲
└─ stamped with the batch MINIMUM (100), not the event's own 101
(see `_update_current_state_txn`)
```
The transaction commits: both events and the delta row are now in the
database, atomically.
The persister then announces the new events over replication, one RDATA
token per stream ordering — rows are only merged into one token when
they share a position, and 100 and 101 don't. So the sync worker's
events-stream position steps 99 → 100 → 101, and on reaching 100 it
pokes the notifier.
Alice's long-poll wakes at exactly that moment. Her response is built at
the worker's *current* position — `end = 100` — with RDATA 101 still in
the queue:
```
Sync A (since=99, end=100):
timeline: events 99 < ordering ≤ 100 → [Carol's message]
state_after: deltas 99 < stream_id ≤ 100 → [$bob_join_call] ← delivered EARLY
next_batch: s100 ← mid-batch token
```
No race on the client's side is needed: the server *hands out* the
mid-batch token as `next_batch`. Alice's client re-polls with it, as
every sync client does. The worker has meanwhile processed RDATA 101:
```
Sync B (since=100, end=101):
timeline: events 100 < ordering ≤ 101 → [Bob's m.call.member @101] ✓
state_after: deltas 100 < stream_id ≤ 101 → [] row is stamped 100 ✗
```
A state event in the timeline with an empty `state_after`. An MSC4222
client trusts `state_after` over timeline state events, so Alice's copy
of Bob's call membership never updates from this response.
On a single process this cannot happen: the batch's stream IDs are
released as a whole, so the position visible to `/sync` jumps 99 → 101
and `s100` is never handed out. Only a process that learns its position
from replication — any sync worker — ticks through the middle of a
batch.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
Instead, treat them as absent fields as they feel like they should be.
The database implementation detail that these fields have a dedicated
column with `NULL`
when unset is kept to the storage layer.
The goal here is to reduce the amount of special casing needed for these
two original profile fields and treat them a little bit more like
regular profile fields.
Follows: #20003
Follows: #20147 (needed as a bugfix to continue sending them down
oldschool sync when they get deleted. Without #20147, this PR would
break that — which matches how custom profile fields were broken too.)
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Part of: #19414
When
[MSC4133](https://github.com/matrix-org/matrix-spec-proposals/pull/4133)
(custom profile fields) was implemented, the returned value for an unset
display name changed from `200 {}` to `200 { displayname: null }`. This
happened first on the unstable `uk.tcpip.msc4133` path in #17488
(1.123.0), then on the stable path when #18635 (1.135.0) unified the
`displayname`, `avatar_url` and custom field servlets. Neither PR
discussed the change in review, so it looks like an unintended side
effect of the refactor rather than a deliberate decision.
The v1.16 spec mandated to change from returning `200 {}` to `404` but
change was not identified as breaking and was eventually not implemented
in other clients and server. This PR has a sister MSC that proposes to
return to the pre-1.16 error codes:
[MSC4537](https://github.com/matrix-org/matrix-spec-proposals/pull/4537).
Before:
```
GET /_matrix/client/v3/profile/@alice:test/displayname
200 {"displayname": null}
```
After:
```
GET /_matrix/client/v3/profile/@alice:test/displayname
200 {}
```
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
Matthew Hodgson