Kegan Dougal
abf658c712
WIP MSC4354; stub out soft-failure code for now
2025-09-18 14:33:11 +01:00
reivilibre and GitHub
6fe8137a4a
Configure Synapse to run MSC4306: Thread Subscriptions Complement tests. ( #18819 )
...
Pairs with: https://github.com/matrix-org/complement/pull/795
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org >
2025-09-09 11:40:10 +01:00
reivilibre and GitHub
8344c944b1
Add configurable rate limiting for the creation of rooms. ( #18514 )
...
Default values will be 1 room per minute, with a burst count of 10.
It's hard to imagine most users will be affected by this default rate,
but it's intentionally non-invasive in case of bots or other users that
need to create rooms at a large rate.
Server admins might want to down-tune this on their deployments.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org >
2025-07-24 14:08:02 +00:00
5ea2cf2484
Move device changes off the main process ( #18581 )
...
The main goal of this PR is to handle device list changes onto multiple
writers, off the main process, so that we can have logins happening
whilst Synapse is rolling-restarting.
This is quite an intrusive change, so I would advise to review this
commit by commit; I tried to keep the history as clean as possible.
There are a few things to consider:
- the `device_list_key` in stream tokens becomes a
`MultiWriterStreamToken`, which has a few implications in sync and on
the storage layer
- we had a split between `DeviceHandler` and `DeviceWorkerHandler` for
master vs. worker process. I've kept this split, but making it rather
writer vs. non-writer worker, using method overrides for doing
replication calls when needed
- there are a few operations that need to happen on a single worker at a
time. Instead of using cross-worker locks, for now I made them run on
the first writer on the list
---------
Co-authored-by: Eric Eastwood <erice@element.io >
2025-07-18 09:06:14 +02:00
8010377a88
Add support for MSC4155 Invite filtering ( #18288 )
...
This implements
https://github.com/matrix-org/matrix-spec-proposals/pull/4155 , which
adds support for a new account data type that blocks an invite based on
some conditions in the event contents.
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com >
2025-06-05 11:49:09 +01:00
7be6c711d4
start_for_complement.sh: use more shell builtins ( #18293 )
...
Avoid calling external tools when shell builtins suffice.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Quentin Gliech <quenting@element.io >
2025-04-30 15:53:15 +00:00
5ab05e7b95
docker: use shebangs to invoke generated scripts ( #18295 )
...
When generating scripts from templates, don't add a leading newline so
that their shebangs may be handled correctly.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Quentin Gliech <quenting@element.io >
2025-04-30 14:26:08 +00:00
Andrew Ferrazzutti and GitHub
5c84f25809
complement-synapse: COPY existing dir from base ( #18294 )
...
The base postgres image already has the /var/run/postgresql directory,
and COPY can set file ownership with chown=, so COPY it instead of
making it from scratch & manually setting its ownership.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
2025-04-01 15:51:00 +00:00
Andrew Ferrazzutti and GitHub
a227d20c25
Pass args to start_for_complement.sh ( #18273 )
2025-03-25 14:09:38 +00:00
Andrew Ferrazzutti and GitHub
92a29dcffc
Docker: Use an ARG for debian version more often ( #18272 )
2025-03-25 13:57:55 +00:00
fbb21b29bb
Define delayed event ratelimit category ( #18019 )
...
Apply ratelimiting on delayed event management separately from messages.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [ ] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com >
2025-02-25 17:22:01 +00:00
Erik Johnston and GitHub
b2a187f49b
Disable room list publication by default ( #18175 )
...
This is in line with our general policy of ensuring that the default
config is reasonably locked down.
SyTest PR to fix tests: https://github.com/matrix-org/sytest/pull/1396
2025-02-24 12:06:16 +00:00
Sven Mäder and GitHub
9c5d08fff8
Ratelimit presence updates ( #18000 )
2025-01-24 19:58:01 +00:00
Eric Eastwood and GitHub
e2d757f62d
Increase rc_invites.per_issuer for Complement ( #18072 )
...
It's possible to run into `SynapseError: 429 - Too Many Requests (rc_invites.per_issuer)`
`rc_invites.per_issuer` originally introduced in
https://github.com/matrix-org/synapse/pull/13125
2025-01-13 15:01:00 -06:00
Eric Eastwood and GitHub
b257c7ab19
Be able to test /login/sso/redirect in Complement ( #17986 )
...
Be able to test `/login/sso/redirect` in Complement
Spawning from
https://github.com/element-hq/sbg/pull/421#discussion_r1854926218 where
we have a proxy that intercepts responses to
`/_matrix/client/v3/login/sso/redirect(/{idpId})` in order to upgrade
them to use OAuth 2.0 Pushed Authorization Requests (PAR). We have some
Complement tests in that codebase that go over this flow and these
changes are required [in order for the URL's to line
up](https://github.com/element-hq/synapse/blob/d648c8ce3f4cbf61191b9f5302e405f7b0288677/synapse/rest/client/login.py#L652-L673 ).
2024-12-03 12:54:25 +00:00
Will Hunt and GitHub
f73edbe4d2
Add encrypted appservice extensions to Complement test image. ( #17945 )
2024-11-20 16:35:43 +00:00
Andrew Ferrazzutti and GitHub
5173741c71
Support MSC4140: Delayed events (Futures) ( #17326 )
2024-09-23 13:33:48 +01:00
Richard van der Hoff and GitHub
2c36a679ae
Include user membership on events ( #17282 )
...
MSC4115 has now completed FCP, so we can enable it by default and switch
to the stable identifier.
2024-06-13 21:45:54 +00:00
Benjamin Bouvier and GitHub
c0ea2bf800
synapse complement image: hardcode enabling msc3266 ( #17105 )
...
This is an alternative to
https://github.com/matrix-org/matrix-rust-sdk/issues/3340 where we don't
need to change our CI setup too much in the Rust SDK repository, and
still can test MSC3266.
2024-05-02 11:48:27 +01:00
b548f7803a
Add support for MSC4115 ( #17104 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com >
2024-04-29 15:22:13 +01:00
259442fa4c
bugfix: make msc3967 idempotent ( #16943 )
...
MSC3967 was updated recently to make it more robust to network failures:
> there is an existing cross-signing master key and it exactly matches
the cross-signing master key provided in the request body. If there are
any additional keys provided in the request (self signing key, user
signing key) they MUST also match the existing keys stored on the
server. In other words, the request contains no new keys. If there are
new keys, UIA MUST be performed.
https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/device-signing-upload-uia/proposals/3967-device-signing-upload-uia.md#proposal
This covers the case where the 200 OK is lost in transit so the client
retries the upload, only to then get UIA'd.
Complement tests: https://github.com/matrix-org/complement/pull/713 -
passing example
https://github.com/element-hq/synapse/actions/runs/7976948122/job/21778795094?pr=16943#step:7:8820
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: reivilibre <oliverw@matrix.org >
2024-04-15 10:57:56 +00:00
Andrew Morgan and GitHub
21daa56ee1
Prevent start_for_complement.sh from setting START_POSTGRES to false when it's already set ( #16985 )
...
I have a use case where I'd like the Synapse image to start up a
postgres instance that I can use, but don't want to force Synapse to use
postgres as well.
This commit prevents postgres from being started when it has already
been explicitly enabled elsewhere.
2024-03-21 13:50:51 +00:00
kegsay and GitHub
0c55c76da8
Better complement docs ( #16946 )
2024-02-20 17:14:50 +00:00
Erik Johnston
930dc9e2d3
Fix typo
2023-12-13 16:37:10 +00:00
Erik Johnston
8613f7693e
More renaming
2023-12-13 15:41:11 +00:00
kegsay and GitHub
11a8ae0632
complement: enable dirty runs ( #16520 )
...
* complement: enable dirty runs
* Add changelog
* Set a low connpool limit when running in Complement
Dirty runs can cause many containers to be running concurrently,
which seems to easily exhaust resources on the host. The increased
speedup from dirty runs also seems to use more db connections on
workers, which are misconfigured currently to have
`SUM(workers * cp_max) > max_connections`, causing
```
FATAL: sorry, too many clients already
```
which results in tests failing.
* Try p=2 concurrency to restrict slowness of servers which causes partial state join tests to flake
* Debug logging
* Only run flakey tests
* Only adjust connection pool limits in worker mode
* Move cp vars to somewhere where they get executed in CI
* Move cp values back to where they actually work
* Debug logging
* Try p=1 to see if this makes worker mode happier
* Remove debug logging
2023-10-27 18:29:20 +01:00
Erik Johnston and GitHub
3cf1a3aa17
Use bookwork as docker base image ( #16324 )
2023-09-15 13:14:10 +01:00
Eric Eastwood and GitHub
0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms ( #15748 )
...
Context for why we're removing the implementation:
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734
Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737
Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
874378c052
Docker fully qualified image names ( #15689 )
...
* Fully qualified docker image names for the main Dockerfile and Complement related.
* Fully qualified docker image names for Dockerfiles associated with building Debian release artifacts.
This one is harder and is separate from the other commit in case it wasn't correct or was unwanted. I decided to
do the expansion on the docker images in the Dockerfile itself, instead of the various source places that build
which distribution that is selected, as it would have been more invasive with the scripts breaking up the string
for tagging and such. This one is untested.
* Changelog
* Update docker/Dockerfile-workers
* Update docker/complement/Dockerfile
---------
Co-authored-by: reivilibre <olivier@librepush.net >
2023-05-31 15:13:31 +00:00
003a25ae5c
Additional functionality for declaring worker types in Complement ( #14921 )
...
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org >
2023-03-14 16:29:33 +00:00
realtyem and GitHub
58214dbb9b
Allow enabling the asyncio reactor in complement ( #14858 )
...
Signed-off-by: Jason Little realtyem@gmail.com
2023-02-01 23:42:45 +00:00
22cc93afe3
Enable Faster Remote Room Joins against worker-mode Synapse. ( #14752 )
...
* Enable Complement tests for Faster Remote Room Joins on worker-mode
* (dangerous) Add an override to allow Complement to use FRRJ under workers
* Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org >
* Fix race where we didn't send out replication notification
* MORE HACKS
* Fix get_un_partial_stated_rooms_token to take instance_name
* Fix bad merge
* Remove warning
* Correctly advance un_partial_stated_room_stream
* Fix merge
* Add another notify_replication
* Fixups
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Fix presence test
* Newsfile
* Apply suggestions from code review
* Update changelog.d/14752.misc
Co-authored-by: Erik Johnston <erik@matrix.org >
* lint
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org >
Co-authored-by: Erik Johnston <erik@matrix.org >
2023-01-22 21:10:11 +00:00
Andrew Morgan and GitHub
a7b54ca8d8
Implement MSC3930: polls push rules ( #14787 )
2023-01-19 12:47:10 +00:00
David Robertson and GitHub
5b3af1c7d0
Stabilise serving partial join responses ( #14839 )
...
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.
Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
2023-01-17 12:44:15 +00:00
Andrew Morgan and GitHub
54cd90ea60
Implement MSC3890: Remotely silence local notifications ( #14775 )
2023-01-13 19:32:10 +00:00
Andrew Morgan and GitHub
c4456114e1
Add experimental support for MSC3391: deleting account data ( #14714 )
2023-01-01 03:40:46 +00:00
be3a8a85e3
Add --editable flag to complement.sh which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. ( #14548 )
...
Co-authored-by: Mathieu Velten <mathieuv@matrix.org >
2022-12-07 15:45:31 +00:00
Eric Eastwood and GitHub
8f10c8b054
Move MSC3030 /timestamp_to_event endpoint to stable v1 location ( #14471 )
...
Fix https://github.com/matrix-org/synapse/issues/14390
- Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
- Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`
Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28 15:54:18 -06:00
69814eb282
Allow override for requesting specific worker types for Complement on command line. ( #14324 )
...
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.
* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.
* Update docs to reflect this new ability.
* Changelog
* Don't rely on soft wrapping to format long strings
Good idea dklimpel. Thanks for catching that.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com >
* Small nits just noticed in docs.
* Fixup new line in docs.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com >
2022-11-08 12:34:09 +00:00
Patrick Cloke and GitHub
59ca73006c
Enable testing MSC3874 in complement. ( #14339 )
2022-11-01 13:26:28 -04:00
Jonathan de Jong and GitHub
e440f9674a
Enable URL previews in complement homeserver config. ( #14198 )
2022-10-18 09:52:23 -04:00
c604d2c218
Mark /relations endpoint as usable on workers. ( #14028 )
...
Co-authored-by: Eric Eastwood <erice@element.io >
2022-10-12 06:46:13 -04:00
42dd992bb7
Dockerfile for tests: align comments with current behavior ( #13867 )
...
Signed-off-by: Mathieu Velten <mathieuv@matrix.org >
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com >
2022-09-27 15:55:43 +02:00
Mathieu Velten and GitHub
384dca53d6
complement: init postgres DB directly inside the target image ( #13819 )
...
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume
Signed-off-by: Mathieu Velten <mathieuv@matrix.org >
2022-09-16 17:12:45 +02:00
10e4093839
Call out buildkit is required when building test docker images ( #13338 )
...
Co-authored-by: David Robertson <davidr@element.io >
2022-07-21 14:29:58 +02:00
David Robertson and GitHub
b977867358
Rate limit joins per-room ( #13276 )
2022-07-19 11:45:17 +00:00
Richard van der Hoff and GitHub
512486bbeb
Docker: copy postgres from base image ( #13279 )
...
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
Till and GitHub
fa10468eb4
[Complement] Allow device_name lookup over federation ( #13167 )
2022-07-04 12:34:50 +00:00
9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. ( #13127 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com >
2022-06-30 11:58:12 +00:00
reivilibre and GitHub
90cadcd403
Add a Subject Alternative Name to the certificate generated for Complement tests. ( #13071 )
2022-06-16 12:43:21 +01:00
Kegan Dougal