These existed only to force the tokio thread pool to start before
reactor.run() under the old callWhenRunning-based startup. Now that the
pool starts lazily on first use, the constructions (and the
make_homeserver overrides that existed solely to host them) are dead
weight. The tearDown hooks that fire the reactor shutdown triggers stay:
they are still what stops the tokio pool for tests that exercise the
Rust /versions handler.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyuiV3m44ZbK9o24EnMMES
Rename the HomeServer wrapper accessors to mirror the Python methods they
call (get_rust_runtime, get_clock), so the wrapper reads 1:1 against
synapse/server.py.
Make RustRuntime itself a cheaply-clonable handle (Clone + Deref to
RustRuntimeInner) rather than exposing Arc<RustRuntimeInner> in
signatures. With pyo3's opt-in from_py_object extraction, the same class
serves as both the Python-facing object and the Rust-side handle:
consumers now declare `runtime: RustRuntime` in their constructors and
pyo3 clones it out (an Arc refcount bump), so no separate Py-wrapper
type is needed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyuiV3m44ZbK9o24EnMMES
Same pattern as the Reactor wrapper: a newtype around the Python object
that is the single place naming the HomeServer API surface Rust relies on
(config extraction, get_rust_runtime, get_clock, the main database pool).
RustHandlers and both rendezvous handlers now take it instead of a bare
Py<PyAny>, and public_baseurl moves into the extracted ServerConfig view
rather than a getattr chain.
Also drop the runtime duck-type validation from Reactor: hasattr only
proves a name exists, not that the signature matches, and the Python side
of these boundaries is already type-checked by mypy via the .pyi stubs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyuiV3m44ZbK9o24EnMMES
Previously the tokio runtime was stashed in a hidden attribute on the
reactor object, installed lazily by whichever Rust code first needed it,
and started via callWhenRunning (which never fires under trial's
MemoryReactorClock, hence reactor.run() workarounds in several tests).
Instead, introduce:
* `Reactor`, a typed Rust wrapper around the Twisted reactor. The duck
type is validated once at the FFI boundary, and the wrapper is the
single place naming the Twisted API surface Rust relies on.
* `RustRuntime`, a frozen pyclass wrapping `Arc<RustRuntimeInner>`,
constructed once per homeserver via `hs.get_rust_runtime()`. The tokio
runtime starts lazily on first use; shutdown is driven by a reactor
shutdown trigger holding only a `Weak` reference (so there is no
uncollectable reference cycle through the Rust struct), with a `Drop`
backstop for reactors whose triggers never fire.
Rust consumers (`HttpClient`, `VersionsHandler`, the Python DB pool
wrapper) now receive the runtime or reactor handle explicitly instead of
fishing state out of reactor attributes, and the reactor.run() /
manual-startup workarounds in tests are no longer needed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyuiV3m44ZbK9o24EnMMES
The cache key of `_get_server_keys_json` is a single argument which is
itself a `(server_name, key_id)` tuple, and `store_server_keys_response`
passed that nested tuple straight into the cache invalidation stream.
psycopg2 quietly serialises the inner tuple as a Postgres *record*, so
the `keys` column of `cache_invalidation_stream_by_instance` ended up
holding the record literal as a single string (e.g.
`{"(srv,ed25519:abc)"}`) — which never matches the real cache key on the
receiving side, i.e. the
invalidation has always been a silent no-op on workers. The native Rust
backend's stricter parameter conversion turns the same nested tuple into
a loud `TypeError: unsupported parameter type for postgres: tuple`.
Fix it the same way https://github.com/element-hq/synapse/pull/18899 did
for `_get_e2e_cross_signing_signatures_for_device`, which has the same
nested-tuple key shape: invalidate the local cache directly, JSON-encode
the key for the replication row, and decode it again in
`process_replication_rows`.
Found as part of the the effort to port the database pool to Rust.
This PR uses the new `M_USER_LIMIT_EXCEEDED` common error code instead
of the malformed `M_RESOURCE_LIMIT_EXCEEDED` error code (as reported by
#18749) (spec: MSC4335).
The error code is also changed from `400` to `403` as this matches what
is in the spec for the endpoints:
https://spec.matrix.org/v1.18/client-server-api/#post_matrixmediav3upload
and
https://spec.matrix.org/v1.18/client-server-api/#put_matrixmediav3uploadservernamemediaid
(albeit the latter says that `M_FORBIDDEN` would be returned)
By default a new built-in `media_upload_limit_exceeded.html` template
will be served for the `info_uri`. Administrators can specify an
external URI in config instead.
Compatibility is retained for any existing modules making use the
`MediaUploadLimit` (e.g. via the `get_media_upload_limits_for_user`
callback).
Introduced in: #19473
Noticed in:
https://github.com/element-hq/synapse/pull/19556#discussion_r3505783541
I have not experienced the bug in the real world, it's just something I
noticed by reading.
--
Fix bug in `_prune_device_lists_changes_in_room` when transaction is
retried
The `nonlocal` variable is a footgun as it increments the counter even
though the transaction did not commit yet and may still be retried.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
If presence is disabled after having been enabled, the presence states
in the database (and hence on clients) are frozen at whatever they were
when presence was last enabled: nothing writes to the presence stream
any more and /sync omits the presence section entirely, so clients show
the old presence states forever.
Fix this in two parts:
1. At startup, if presence is disabled but the database still contains
non-offline presence states, the presence writer sends out one final
round of updates marking those users as offline.
2. /sync no longer unconditionally omits presence when presence is
disabled: incremental syncs whose since token is behind the presence
stream still get the straggling updates. As the stream doesn't advance
while presence is disabled, clients catch up once and the check then
short-circuits to a token comparison.
Remote servers already handle this themselves by timing out our users
([`FEDERATION_TIMEOUT`](https://github.com/element-hq/synapse/blob/4d8905a15a417ed0054ec2533d243932d890bbbd/synapse/handlers/presence.py#L194-L198)),
so no federation changes are needed.
Note that this only fixes the issue if presence is fully disabled. If
set to `untracked` we still have the same issue, however since modules
would still write to presence we can't just clobber everything like we
do in this patch.
This does two things, first it adds a config flag to ignore rooms for
the purposes of presence routing.
Secondly, it changes the caching behaviour to try and improve the cache
hit ratio. Previously, the size of the `do_users_share_a_room` cache
(which stores pairs of users) needs to `O(n²)` for the number of online
users, which is infeasible for large servers.
Instead, we call `get_users_in_room` for both the syncing and updated
users. This sounds more expensive, but a) we will already have cached
the syncing user's rooms, and b) we will only calculate the updated
user's rooms once (rather than once per syncing user).
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
- Impose limit of scheduled delayed events
- Update error codes to match latest draft of MSC4140
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Sync workers proxied a full `ReplicationPresenceSetState` call to the
presence writer on every sync request with `affect_presence=True` (via
`user_syncing`), and a `ReplicationBumpPresenceActiveTime` call on every
user action, even though the writer's presence timers only need feeding
every `SYNC_ONLINE_TIMEOUT` / `LAST_ACTIVE_GRANULARITY`. On busy clients
this amounts to tens of no-op replication calls per user per minute, and
the resulting per-update work is the dominant CPU cost on saturated
presence writers.
Track the last relayed `(state, timestamp)` per `(user, device)` on the
worker and suppress unchanged sync-driven repeats within a 25s relay
interval - deliberately below `SYNC_ONLINE_TIMEOUT` (30s) so the
writer's device `last_sync_ts`/`last_active_ts` timers stay fed and
users neither flap offline nor bounce `currently_active`. Genuine state
changes are relayed immediately, explicit (non-sync) set_state calls
always go through and reset the throttle, bumps that might un-idle a
device bypass it, and entries are evicted when a `USER_SYNC` stop is
sent so reconnecting devices are relayed afresh.
This gives the writer-CPU benefit that deployments currently obtain by
tightening `rc_presence` to ~1/29s, without dropping real presence
transitions.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Quentin Gliech <quenting@element.io>
A handful of places in the storage layer bound a value whose Python type
didn't match the declared column type — an `int` into a `TEXT` column,
or a `str` into a `BIGINT` column — and relied on psycopg2's loose
coercion to paper over the mismatch. These are latent correctness bugs:
they only work because the driver silently converts, and a stricter
driver that binds typed parameters rejects them outright.
Found during a Rust port of the database pool where the driver does not
coerce automatically.
Each fix binds the value with the type the column actually declares,
rather than depending on driver-specific coercion. All changes are
behaviour-preserving on psycopg2 and sqlite.
There is also a fix to the multi-writer id-gen tests where we forgot to
commit. This is tangential, but was found during the same effort.
### Changes
- **`device_lists_remote_extremeties.stream_id (TEXT)`** —
`_update_remote_device_list_cache_txn` (typed `stream_id: int`) bound an
int; store it as a string, matching the column and the sibling
`_update_remote_device_list_cache_entry_txn` (typed `stream_id: str`).
The old mismatch, when rejected, was swallowed inside the device-list
resync and hung `query_devices`.
- **`user_filters.filter_id (BIGINT)`** — `get_user_filter` bound the
raw `int | str` (a string, from sync requests). It already validates via
`int(filter_id)`; bind that int so it matches the column.
- **`rejections.last_check (TEXT)`** — both writers stored
`clock.time_msec()` (an int); store the timestamp as a string.
- **user-directory temp position** — the `populate_user_directory`
background update's staging column
`_temp_populate_user_directory_position.position` was `TEXT` but held an
int (read back into a `BIGINT` column). Declare it `BIGINT`. The temp
table is created and dropped within the background update, so there's no
migration.
- **`test_batched_state_group_storing`** — selected from
`state_group_edges` with a stringified `state_group`; bind the int
directly (the column is an integer).
- **Multi-writer id-generator tests** — constructing a
`MultiWriterIdGenerator` prunes stale `stream_positions` rows, but the
harness never committed, so the cleanup only survived because adbapi
keeps one connection per thread with its transaction open. Commit after
construction so it persists regardless of pool semantics (the delete is
legitimate work that should be committed anyway).
This speeds up the cascading delete from
`sliding_sync_connection_positions`, which without an index on
`connection_position` requires a sequential scan of the whole table for
each deleted position.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
