### Summary
Resolves#18308.
Makes the `TaskScheduler`'s maximum concurrent running tasks
configurable via `task_scheduler.max_concurrent_tasks` in the homeserver
configuration and defaults it to `2` (reduced from the previous
hardcoded limit of `5`).
### Motivation
Twisted's `adbapi.ConnectionPool` defaults to 5 database connections
(`cp_max=5`). When the `TaskScheduler` previously executed up to 5 tasks
concurrently, it could exhaust the entire database connection pool,
starving regular API and synchronization requests on smaller homeserver
instances. Lowering the default to 2 preserves at least 3 connections
for foreground requests while allowing concurrent tasks to progress, and
gives administrators the ability to tune the limit according to their
host and connection pool sizing.
### Changes
- Added `TaskSchedulerConfig` (`task_scheduler.max_concurrent_tasks`)
with validation (must be a positive integer) and registered it in
`HomeServerConfig`.
- Updated `TaskScheduler` to enforce the configured concurrency limit
and updated class default to 2.
- Updated config documentation and JSON schema.
- Added config tests in `tests/config/test_task_scheduler_config.py` and
updated unit tests in `tests/util/test_task_scheduler.py`.
- Added newsfragment in `changelog.d/18308.feature`.
---------
Signed-off-by: nevil06 <nevilansondsouza@gmail.com>
Co-authored-by: Devon Hudson <devonhudson@librem.one>
Ports the logcontext classes to Rust, and gives tokio tasks a captured
logcontext so that work running in (or spawned from) Rust is attributed
to the request that caused it.
1. **Add characterization tests for logcontext error messages and the
filter** — pins the exact `logcontext_error` message shapes, the
abuse-detection code paths and `LoggingContextFilter`'s observable
behaviour, *against the existing Python implementation* (this commit is
green on its own). These are the behavioural contract the port has to
satisfy.
2. **Port `ContextResourceUsage` to a Rust pyclass** — self-contained:
the new `synapse_rust.logcontext` module, the class, its stub and the
re-export.
3. **Move the logcontext storage and `LoggingContext` to Rust** — the
core change; the commit message carries detailed design notes.
Highlights:
- The slot is typed `Option<Py<LoggingContext>>`, with `None`
representing the sentinel. `_Sentinel`/`SENTINEL_CONTEXT` stay pure
Python (unchanged); thin wrappers on
`current_context`/`set_current_context` convert at the boundary, and
pyo3's extraction enforces the type (`TypeError` otherwise).
- The accounting is native: one `getrusage(RUSAGE_THREAD)` read per
switch via libc, inline `stop`/`start` bookkeeping for base
`LoggingContext`s, Python dispatch only for subclasses
(`BackgroundProcessLoggingContext`) so their overrides run. The thread
id comes from `PyThread_get_thread_ident` (the exact
`threading.get_ident()`
value) without calling into Python.
- The hot paths avoid per-operation allocation: names are `Py<PyString>`
(the per-log-record `str(context)`/`server_name` reads are INCREF-only),
error branches materialise strings only when hit.
4. **Attribute Rust-spawned work to the caller's logcontext** —
`create_deferred` captures the caller's context and scopes it onto the
spawned task via a tokio task-local (`LogContextHandle`);
`current_context()` gives the task-local read precedence, so
`LoggingContextFilter`/`pyo3-log` resolve the right context on worker
threads with no per-record stamping. `run_python_awaitable` restores the
captured context (via a `with_logcontext` helper, the Rust
`PreserveLoggingContext`) around Python called back from Rust, so e.g.
`runInteraction` from the Rust `/versions` handler accounts its DB usage
against the right request. Integration tests exercise both guarantees
through real production code paths.
Follow-up work on top of this (separate PR): porting
`BackgroundProcessLoggingContext` natively and removing further `Py<_>`
indirections. The fact that `BackgroundProcessLoggingContext` is a
subclass is what forces some of the warts in this PR: e.g. having to use
`Py<LoggingContext>` everywhere, etc.
We don't try (yet) to make this pure Rust, instead we see this as simply
maintaining the Python logcontext machinery when crossing, rather than
trying to make a Rust equivalent that can be used by pure Rust
dependencies. We probably do want to do that in future, as well as wire
up e.g. CPU recording on Rust side, but that is unnecessary for now.
Adds a `redis.username` config option.
Details:
A `username` without a `password` (or `password_path`) is refused at
startup. Redis has no wire form for a username without a password, and
txredisapi only sends `AUTH` when a password is set, so the username
would otherwise be silently ignored. An explicitly empty password is
accepted, since that is how a `nopass` ACL user is configured.
This relies on txredisapi 1.4.12, the first release to accept a
`username` kwarg. That upstream support was contributed by @karolyi
specifically to unblock this.
Fixes#19238.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Aligns the config schema of `federation_domain_whitelist` more closely
with its semantics and its actual implementation in Synapse. This is
done by making it nullable.
- all behavior of the option is now achievable by setting the config
option to an actual value
- aligned behavior of the property between unset and null
- provide a definitive default value
This also fixes the formatting problem with separating the paragraphs in
the documentation.
This PR implements support for profile updates over Sliding Sync:
https://github.com/matrix-org/matrix-spec-proposals/pull/4262. This pr
may be easier to review as a whole than commit by commit.
This builds on the legacy sync profile updates feature
https://github.com/element-hq/synapse/pull/19556, specifically the
profile updates stream it added.
Submitting for early review to get consensus on implementation. There
are some things we would like to add still, from spec, mainly:
* > Homeservers should only consider a profile field update "accepted"
by a client
> once the client returns with a new /sync request with the next /sync
token,
> NOT just after sending down the profile update. The client may never
receive
> response due to network conditions, or a bug in the client
implementation.
* > When a room enters this subset in this connection for the first
time, all requested
> fields from profiles of users in that room MAY be sent down. This
gives the client
> a base set of information for which future field updates can be
applied on top of.
> The homeserver MAY omit some fields and profiles if it believes that
the client has
> already received them, likewise repeat profiles MAY be sent down based
on homeserver
> implementation.
* > Finally, if the list of fields expands to cover a new field ID,
those fields should
> be sent down for all users that are within the current room subset.
Future incremental
> updates will then include changes to this field.
* Additionally, we would need to implement a lazy loading cache similar
to the legacy sync. (not part of MSC as such)
Depending on review these could either be added to this pr, or to keep
this pr from not growing too much, be added in a follow-up pr, as they
are more enhancement to this base sliding sync profile updates
functionality than a part of the core functionality.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Olivier 'reivilibre' <olivier@librepush.net>
Co-authored-by: Olivier 'reivilibre <oliverw@element.io>
We have an internal usage of `/scheduled_tasks` that would like to fetch
multiple actions at once (janitor).
We also make it so that invalid `status` values now return a 400 rather
than a 500.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
It seems a lot of time in our trial tests goes towards setting up the
database. (The same is probably true of Complement too)
We haven't done a full schema for about 20 schema versions, so no
surprise!
As a result, I want to produce a full schema soon.
In this PR I dust off `make_full_schema.sh` (which seems to have broken
after some SQLite changes)
and add a CI workflow that runs it (producing a diff) when someone
changes the schema.
The CI workflow also adds a sticky comment showing the diff on the
schema, so you can better appreciate the final effect of a change.
---
**Dead changes:**
I wanted to make it possible to generate a versioned full schema without
the manual work,
but you can't run the background updates without essentially starting up
a homeserver,
at which point it might fail because you haven't run all the deltas yet.
There's no actual good way to do this, short of deleting the latest
deltas (+ tweaking code to not crash without them)
or rolling back in the git history.
Backed out those changes, but they're preserved on the PR if interesting.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
The registration of `QuarantinedMediaStream` in
`ReplicationCommandHandler._streams_to_replicate` was missed when the
stream was added, so an instance configured as the
quarantined_media_changes stream writer never sent RDATA/POSITION for it
unless it was the main process.
Also add the stream to the `instance_map` config validation.
Stream was introduced in
https://github.com/element-hq/synapse/pull/19558
Fixes https://github.com/element-hq/synapse/issues/20080
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This PR uses the new `M_USER_LIMIT_EXCEEDED` common error code instead
of the malformed `M_RESOURCE_LIMIT_EXCEEDED` error code (as reported by
#18749) (spec: MSC4335).
The error code is also changed from `400` to `403` as this matches what
is in the spec for the endpoints:
https://spec.matrix.org/v1.18/client-server-api/#post_matrixmediav3upload
and
https://spec.matrix.org/v1.18/client-server-api/#put_matrixmediav3uploadservernamemediaid
(albeit the latter says that `M_FORBIDDEN` would be returned)
By default a new built-in `media_upload_limit_exceeded.html` template
will be served for the `info_uri`. Administrators can specify an
external URI in config instead.
Compatibility is retained for any existing modules making use the
`MediaUploadLimit` (e.g. via the `get_media_upload_limits_for_user`
callback).
This does two things, first it adds a config flag to ignore rooms for
the purposes of presence routing.
Secondly, it changes the caching behaviour to try and improve the cache
hit ratio. Previously, the size of the `do_users_share_a_room` cache
(which stores pairs of users) needs to `O(n²)` for the number of online
users, which is infeasible for large servers.
Instead, we call `get_users_in_room` for both the syncing and updated
users. This sounds more expensive, but a) we will already have cached
the syncing user's rooms, and b) we will only calculate the updated
user's rooms once (rather than once per syncing user).
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds `last_active_granularity`, `sync_online_timeout` and `idle_timeout`
options to the `presence` config section, controlling the previously
hard-coded `LAST_ACTIVE_GRANULARITY`, `SYNC_ONLINE_TIMEOUT` and
`IDLE_TIMER` constants (which remain as the defaults).
This is mainly useful on deployments that ratelimit how often syncs can
affect presence (`rc_presence`): the sync timeout must exceed the
ratelimit interval or users flap offline between syncs, so tuning down
presence traffic currently requires squeezing under the fixed 30s limit.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
After looking into it, just a couple of things to pick a bone at in the
old wording,
which I thought could be clarified for when I next come to look at this
again.
- the claim that there's a fundamental difference; I'd argue there isn't
really, it's just by convention
on some mainstream distros. So I have changed this to 'typically'
- statements that some distros fetch dependencies at build time
(probably does happen, but
traditional distros make a point of not doing this for the reasons you'd
expect).
- This was probably meant to be talking about Debian, but my observation
based on sample size of 3 is that some crates are packaged natively,
others are vendored in the respective application's source package (like
they do for us) and sometime they patch the bounds a bit
There could probably be room to talk about how distros vendoring
packages is a maintenance burden on them,
but I guess it's a bit moot as we would struggle to conform to wide
enough bounds to make everyone
happy (and anyway; I expect the distros that vendor packages have the
tooling to make this easy to
update and we do keep on top of security updates and release
frequently...)
---
Spawning from discussion in
[`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$VttYPPUevn2S_W_rrzg2ZOXWI6aKebk2ganTgrLEWUc?via=jki.re&via=element.io&via=matrix.org)
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
The reason for querying this support was wanting support for SQLite's JSON operators, which are currently not present in the SQLite version found in Ubuntu's oldest supported LTS.
The JSON operators were used in some of the sticky events work (related: #19452).
Our ruling was that we should support Ubuntu oldest LTS equally to Debian oldstable, so support the oldest of the two versions from those.
That makes some kind of sense as it would be difficult to do otherwise without dropping support for that version of Ubuntu altogether, given if we kept publishing packages intended for use with Postgres, there's a risk that an innocent sysadmin would update their SQLite deployment without realising that it is no longer supported.
This was [discussed months ago
(private)](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.fcdvoc88dy5s)
and at [private](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.u48ivjge4qpt).
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Knowledge from @reivilibre in
[`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$2WUfCdA02wZw-6-jhw3QbLQ44BmKJrqLuZ6wjz2r7hk?via=jki.re&via=element.io&via=matrix.org)
on 2025-12-15.
Spawning from me making Rust changes but nothing useful was printed
until I added `-v`,
```shell
$ poetry install --extras all
Installing dependencies from lock file
Package operations: 0 installs, 1 update, 0 removals
- Updating pyjwt (2.11.0 -> 2.12.0)
Installing the current project: matrix-synapse (1.154.0rc1)
Failed to install /home/eric/Documents/github/element/synapse
```
I also see `poetry run maturin develop` suggested but I'd prefer not to
need to install `maturin` as yet another system tool to manage myself.
Added details how synapse syncs the picture claim when
update_profile_information setting is true. Addresses #17836
---------
Co-authored-by: Michael Hlas <3398654+mhlas7@users.noreply.github.com>
