Broken out of #20027 because I'd like to have it land first.
> It seems a lot of time in our trial tests goes towards setting up the
database. (The same is probably true of Complement too)
>
> We haven't done a full schema for about 20 schema versions, so no
surprise!
>
> As a result, I want to produce a full schema soon.
When running `make_full_schema.sh`, these drop statements now cause the
error:
```
Parse error near line 2: table event_search_content may not be dropped
Parse error near line 3: table event_search_segments may not be dropped
Parse error near line 4: table event_search_segdir may not be dropped
Parse error near line 5: table event_search_docsize may not be dropped
Parse error near line 6: table event_search_stat may not be dropped
Parse error near line 7: table user_directory_search_content may not be dropped
Parse error near line 8: table user_directory_search_segments may not be dropped
Parse error near line 9: table user_directory_search_segdir may not be dropped
Parse error near line 10: table user_directory_search_docsize may not be dropped
Parse error near line 11: table user_directory_search_stat may not be dropped
```
It seems SQLite has cracked down on code that edits the internal tables.
Because SQLite dumps the schema with `CREATE TABLE IF NOT EXISTS` for
these virtual tables, it's harmless to leave them in the schema dump.
---------
Signed-off-by: Olivier 'reivilibre' <oliverw@matrix.org>
To make it easy to share to people and aid in debugging when they run into
performance issues.
Unfortunately, the Grafana UI doesn't make the export or import steps
easy so it involves some manual Grafana API calls.
As first explored in
https://github.com/element-hq/synapse-rust-apps/pull/397
Without this change, it was linking to `develop` as we were on the
`develop` branch because of the previous `merge-back` step at this point
in the release process.
Follow-up to https://github.com/element-hq/synapse/pull/19984 as this
was an oversight and I assumed we would still be on the `release-v1.158`
branch by that point.
Fix `RemoteJoinHelper` signing events with mismatched room version
compared to the `room_version` arg. The default room version on
`develop` is `11` but the `RemoteJoinHelper` `room_version` arg defaults
to `10` (room version mismatch). This mismatch wasn't present where this
fix was developed
(https://github.com/element-hq/synapse-private/pull/136) as the default
room version was only recently bumped to `11` via
https://github.com/element-hq/synapse/pull/18680 (not even in a release
yet).
Fixes the CI being broken on `develop` :x::
```
[ERROR]
Traceback (most recent call last):
File "/home/runner/work/synapse/synapse/tests/federation/test_federation_join_upgraded_room.py", line 298, in test_no_transfer_when_tombstone_does_not_match
join_helper.join(local_user_id, local_user_tok)
File "/home/runner/work/synapse/synapse/tests/federation/_remote_join.py", line 350, in join
self._test_case.helper.join(remote_room_id, local_user_id, tok=local_user_tok)
File "/home/runner/work/synapse/synapse/tests/rest/client/utils.py", line 195, in join
return self.change_membership(
File "/home/runner/work/synapse/synapse/tests/rest/client/utils.py", line 333, in change_membership
assert channel.code == expect_code, (
builtins.AssertionError: Expected: 200, got: 400, PUT /_matrix/client/r0/rooms/!remote-room:other.example.com/state/m.room.member/@user1:test?access_token=syt_dXNlcjE_JSEarRndiAGqtydnrdLn_33qlLD -> resp: b'{"errcode":"M_UNKNOWN","error":"No create event in state"}'
tests.federation.test_federation_join_upgraded_room.FederationJoinUpgradedRoomTestCase.test_no_transfer_when_tombstone_does_not_match
```
These tests were originally introduced
https://github.com/element-hq/synapse-private/pull/136 (developed
private as this was part of the Synapse security release) and introduced
into the public codebase via
https://github.com/element-hq/synapse/commit/cbc6934821aab314506c5957223c60c74eeda091
This means we can point to a single source of truth instead of
duplicating the content to the tag and GitHub release. Less to manage
and worry about when you make some updates to the changelog (maintenance
burden). We also get to avoid the content sitting in the vendor lock-in
GitHub releases.
We point to
`https://github.com/element-hq/synapse/blob/{repo.active_branch.name}/CHANGES.md`
as it will have the relevant changelog entry at the top and won't change
as we archive releases on `develop`. Even for RC releases after the main
release goes out, the entry will still be towards the top. We could try
to get the heading anchor for the specific section but I thought that it
wasn't necessary (nice but more complex).
Fixes: https://github.com/element-hq/synapse/security/advisories/GHSA-vh4c-pqh4-w3wq
Fixes: https://github.com/matrix-org/internal-config/issues/1703
The key thing to understand is that in `synapse/util/httpresourcetree.py`,
we create `UnrecognizedRequestResource` and then dangle children (with real resources) off them.
Since `UnrecognizedRequestResource` returns itself as a catch-all 'dynamic child',
this means any `UnrecognizedRequestResource`s with real children can have unlimited path components inserted between it and its child.
So `/_matrix/INSERTED/static/client/login/style.css` or `/_matrix/INSERTED/AS/MANY/AS/I/WANT/static/client/login/style.css` would unexpectedly resolve to the resource.
Client, Federation and Admin APIs wouldn't have been affected because you wouldn't get through the regex routing that they use.
-----
Reviewed-on: https://github.com/element-hq/synapse-private/pull/143
The cache key of `_get_server_keys_json` is a single argument which is
itself a `(server_name, key_id)` tuple, and `store_server_keys_response`
passed that nested tuple straight into the cache invalidation stream.
psycopg2 quietly serialises the inner tuple as a Postgres *record*, so
the `keys` column of `cache_invalidation_stream_by_instance` ended up
holding the record literal as a single string (e.g.
`{"(srv,ed25519:abc)"}`) — which never matches the real cache key on the
receiving side, i.e. the
invalidation has always been a silent no-op on workers. The native Rust
backend's stricter parameter conversion turns the same nested tuple into
a loud `TypeError: unsupported parameter type for postgres: tuple`.
Fix it the same way https://github.com/element-hq/synapse/pull/18899 did
for `_get_e2e_cross_signing_signatures_for_device`, which has the same
nested-tuple key shape: invalidate the local cache directly, JSON-encode
the key for the replication row, and decode it again in
`process_replication_rows`.
Found as part of the the effort to port the database pool to Rust.
This PR uses the new `M_USER_LIMIT_EXCEEDED` common error code instead
of the malformed `M_RESOURCE_LIMIT_EXCEEDED` error code (as reported by
#18749) (spec: MSC4335).
The error code is also changed from `400` to `403` as this matches what
is in the spec for the endpoints:
https://spec.matrix.org/v1.18/client-server-api/#post_matrixmediav3upload
and
https://spec.matrix.org/v1.18/client-server-api/#put_matrixmediav3uploadservernamemediaid
(albeit the latter says that `M_FORBIDDEN` would be returned)
By default a new built-in `media_upload_limit_exceeded.html` template
will be served for the `info_uri`. Administrators can specify an
external URI in config instead.
Compatibility is retained for any existing modules making use the
`MediaUploadLimit` (e.g. via the `get_media_upload_limits_for_user`
callback).
