The cached fast path serialised its per-room lookups
(is_partial_state_room, then the hierarchy state cache fill, then room
version and stats), which undid the traversal-level prefetch parallelism
on a cold cache under latency. Gather them concurrently instead, warming
the room-stats and room-version caches that the rest of the path reads.
Since the partial-state check now races the cache fill, drop the cached
hierarchy state again when the room turns out to be partial-stated, and
register get_room_hierarchy_state in _invalidate_state_caches(_all) so
state resets and un-partial-stating invalidate it too.
Two new caches serve the room hierarchy (space summary) endpoints without
touching the database once warm:
* get_room_hierarchy_state (state store): the room's authoritative join
rule and history visibility, the join-rules state id (for the
restricted-allow computation) and the stripped m.space.child events.
Invalidated whenever a state event is persisted in the room (and on
room purge), including over replication.
* get_room_with_stats (room store) is now @cached, invalidated where the
room stats rows are written (update_room_state and the room stats
delta), when the room's directory visibility changes, and on purge.
The handler serves local room entries from these caches; accessibility
checks use the cached authoritative join rule / history visibility plus
the existing per-user cached membership lookups, so a warm request for a
public hierarchy does no DB work. Partial-state and unknown rooms fall
back to the uncached path.
Join rule and history visibility for access decisions are read from
current state, not the stats copies, so access control never rides a
stale asynchronously-updated stats row. Remote rooms were already
TTL-cached by the federation client (5 minutes); change-based
invalidation is not possible across federation.
Ties in child ordering beyond origin_server_ts now break on the child
room id (per MSC2946) rather than the previously arbitrary input order.
Three changes to synapse/handlers/room_summary.py:
* Prefetch the summaries of the next PREFETCH_SUMMARIES (10) rooms in
traversal order concurrently, instead of processing rooms strictly one
at a time. For hierarchies containing remote subspaces this overlaps
the per-subspace federation /hierarchy requests (previously fully
sequential, one round trip per subspace); for local rooms it overlaps
their DB reads. Results are still emitted strictly in traversal order.
* Fetch each room's current state once per summary with a filter
covering the accessibility check, the restricted-join-rules lookup and
the m.space.child edges, instead of three separate current-state
fetches; the child-events fetch previously requested the room's full
unfiltered current state, which is very expensive for large rooms.
* Summarise the children of a space concurrently (rather than
sequentially) when responding to a federation /hierarchy request.
Broken out of #20027 because I'd like to have it land first.
> It seems a lot of time in our trial tests goes towards setting up the
database. (The same is probably true of Complement too)
>
> We haven't done a full schema for about 20 schema versions, so no
surprise!
>
> As a result, I want to produce a full schema soon.
When running `make_full_schema.sh`, these drop statements now cause the
error:
```
Parse error near line 2: table event_search_content may not be dropped
Parse error near line 3: table event_search_segments may not be dropped
Parse error near line 4: table event_search_segdir may not be dropped
Parse error near line 5: table event_search_docsize may not be dropped
Parse error near line 6: table event_search_stat may not be dropped
Parse error near line 7: table user_directory_search_content may not be dropped
Parse error near line 8: table user_directory_search_segments may not be dropped
Parse error near line 9: table user_directory_search_segdir may not be dropped
Parse error near line 10: table user_directory_search_docsize may not be dropped
Parse error near line 11: table user_directory_search_stat may not be dropped
```
It seems SQLite has cracked down on code that edits the internal tables.
Because SQLite dumps the schema with `CREATE TABLE IF NOT EXISTS` for
these virtual tables, it's harmless to leave them in the schema dump.
---------
Signed-off-by: Olivier 'reivilibre' <oliverw@matrix.org>
To make it easy to share to people and aid in debugging when they run into
performance issues.
Unfortunately, the Grafana UI doesn't make the export or import steps
easy so it involves some manual Grafana API calls.
As first explored in
https://github.com/element-hq/synapse-rust-apps/pull/397
Without this change, it was linking to `develop` as we were on the
`develop` branch because of the previous `merge-back` step at this point
in the release process.
Follow-up to https://github.com/element-hq/synapse/pull/19984 as this
was an oversight and I assumed we would still be on the `release-v1.158`
branch by that point.
Fix `RemoteJoinHelper` signing events with mismatched room version
compared to the `room_version` arg. The default room version on
`develop` is `11` but the `RemoteJoinHelper` `room_version` arg defaults
to `10` (room version mismatch). This mismatch wasn't present where this
fix was developed
(https://github.com/element-hq/synapse-private/pull/136) as the default
room version was only recently bumped to `11` via
https://github.com/element-hq/synapse/pull/18680 (not even in a release
yet).
Fixes the CI being broken on `develop` :x::
```
[ERROR]
Traceback (most recent call last):
File "/home/runner/work/synapse/synapse/tests/federation/test_federation_join_upgraded_room.py", line 298, in test_no_transfer_when_tombstone_does_not_match
join_helper.join(local_user_id, local_user_tok)
File "/home/runner/work/synapse/synapse/tests/federation/_remote_join.py", line 350, in join
self._test_case.helper.join(remote_room_id, local_user_id, tok=local_user_tok)
File "/home/runner/work/synapse/synapse/tests/rest/client/utils.py", line 195, in join
return self.change_membership(
File "/home/runner/work/synapse/synapse/tests/rest/client/utils.py", line 333, in change_membership
assert channel.code == expect_code, (
builtins.AssertionError: Expected: 200, got: 400, PUT /_matrix/client/r0/rooms/!remote-room:other.example.com/state/m.room.member/@user1:test?access_token=syt_dXNlcjE_JSEarRndiAGqtydnrdLn_33qlLD -> resp: b'{"errcode":"M_UNKNOWN","error":"No create event in state"}'
tests.federation.test_federation_join_upgraded_room.FederationJoinUpgradedRoomTestCase.test_no_transfer_when_tombstone_does_not_match
```
These tests were originally introduced
https://github.com/element-hq/synapse-private/pull/136 (developed
private as this was part of the Synapse security release) and introduced
into the public codebase via
https://github.com/element-hq/synapse/commit/cbc6934821aab314506c5957223c60c74eeda091
This means we can point to a single source of truth instead of
duplicating the content to the tag and GitHub release. Less to manage
and worry about when you make some updates to the changelog (maintenance
burden). We also get to avoid the content sitting in the vendor lock-in
GitHub releases.
We point to
`https://github.com/element-hq/synapse/blob/{repo.active_branch.name}/CHANGES.md`
as it will have the relevant changelog entry at the top and won't change
as we archive releases on `develop`. Even for RC releases after the main
release goes out, the entry will still be towards the top. We could try
to get the heading anchor for the specific section but I thought that it
wasn't necessary (nice but more complex).
Fixes: https://github.com/element-hq/synapse/security/advisories/GHSA-vh4c-pqh4-w3wq
Fixes: https://github.com/matrix-org/internal-config/issues/1703
The key thing to understand is that in `synapse/util/httpresourcetree.py`,
we create `UnrecognizedRequestResource` and then dangle children (with real resources) off them.
Since `UnrecognizedRequestResource` returns itself as a catch-all 'dynamic child',
this means any `UnrecognizedRequestResource`s with real children can have unlimited path components inserted between it and its child.
So `/_matrix/INSERTED/static/client/login/style.css` or `/_matrix/INSERTED/AS/MANY/AS/I/WANT/static/client/login/style.css` would unexpectedly resolve to the resource.
Client, Federation and Admin APIs wouldn't have been affected because you wouldn't get through the regex routing that they use.
-----
Reviewed-on: https://github.com/element-hq/synapse-private/pull/143
The cache key of `_get_server_keys_json` is a single argument which is
itself a `(server_name, key_id)` tuple, and `store_server_keys_response`
passed that nested tuple straight into the cache invalidation stream.
psycopg2 quietly serialises the inner tuple as a Postgres *record*, so
the `keys` column of `cache_invalidation_stream_by_instance` ended up
holding the record literal as a single string (e.g.
`{"(srv,ed25519:abc)"}`) — which never matches the real cache key on the
receiving side, i.e. the
invalidation has always been a silent no-op on workers. The native Rust
backend's stricter parameter conversion turns the same nested tuple into
a loud `TypeError: unsupported parameter type for postgres: tuple`.
Fix it the same way https://github.com/element-hq/synapse/pull/18899 did
for `_get_e2e_cross_signing_signatures_for_device`, which has the same
nested-tuple key shape: invalidate the local cache directly, JSON-encode
the key for the replication row, and decode it again in
`process_replication_rows`.
Found as part of the the effort to port the database pool to Rust.
This PR uses the new `M_USER_LIMIT_EXCEEDED` common error code instead
of the malformed `M_RESOURCE_LIMIT_EXCEEDED` error code (as reported by
#18749) (spec: MSC4335).
The error code is also changed from `400` to `403` as this matches what
is in the spec for the endpoints:
https://spec.matrix.org/v1.18/client-server-api/#post_matrixmediav3upload
and
https://spec.matrix.org/v1.18/client-server-api/#put_matrixmediav3uploadservernamemediaid
(albeit the latter says that `M_FORBIDDEN` would be returned)
By default a new built-in `media_upload_limit_exceeded.html` template
will be served for the `info_uri`. Administrators can specify an
external URI in config instead.
Compatibility is retained for any existing modules making use the
`MediaUploadLimit` (e.g. via the `get_media_upload_limits_for_user`
callback).
Matthew Hodgson