Eric Eastwood
c5e5abb822
Use weakref to Python DatabasePool to allow homeserver to cleanly shutdown ( #20009 )
...
Use weakref to Python `DatabasePool` to allow homeserver to cleanly
shutdown
Spawning from
[upgrading](https://github.com/element-hq/synapse-small-hosts/pull/420#discussion_r3661700252 )
the Synapse version in Synapse Pro for small hosts and seeing our tenant
deprovision tests failing (end-to-end Complement tests). Specifically,
it was failing with a test in
[`TestLogging`](https://github.com/element-hq/synapse-small-hosts/blob/f24ee731ff419890680d1d80ef6eafe44f87f2c2/complement/tests/multi_synapse/logging_test.go#L328-L385 )
in CI but was equally reproducible with any test where we deprovision
(`deployment.StopServer(...)`) like
[`TestProvisionHomeserverTenant/deprovision_homeserver_tenant`](https://github.com/element-hq/synapse-small-hosts/blob/f24ee731ff419890680d1d80ef6eafe44f87f2c2/complement/tests/multi_synapse/provision_test.go#L48-L58 ).
Clean homeserver shutdown specifically regressed with the changes from
https://github.com/element-hq/synapse/pull/19878 . The
`PythonDatabasePoolWrapper` in Rust holds onto a reference to the Python
`DatabasePool` which references the Python `HomeServer` and keeps the
`HomeServer` from being garbage collected on the Python side.
This PR updates `PythonDatabasePoolWrapper` to use a weak reference.
### Dev notes
Previous PR where we had to deal with Rust <-> Python reference cycles,
https://github.com/element-hq/synapse/pull/19837#discussion_r3425358096
PyO3 garbage collector docs (`__traverse__`, `__clear__`):
https://pyo3.rs/v0.28.3/class/protocols#garbage-collector-integration
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
2026-07-28 10:04:48 +01:00
Eric Eastwood
7da21a715d
Update HomeserverTestCase.get_success(...) and friends to drive async Rust (Tokio runtime/thread pool) ( #19871 )
...
This means you can use `get_success(...)` anywhere regardless
of what kind of work needs to be done.
Spawning from adding some more async Rust things in
https://github.com/element-hq/synapse/pull/19846 and wanting something
more standard instead of the custom `till_deferred_has_result(...)` that
has crept in to a few files.
Alternative to https://github.com/element-hq/synapse/pull/19867 spurred
on by [this
comment](https://github.com/element-hq/synapse/pull/19867#discussion_r3441774685 )
from @erikjohnston
### How does this work?
Previously, `get_success(...)` just ran in a hot-loop advancing the
Twisted reactor clock which didn't give any time for other threads to do
some work or acquire the GIL if necessary (whenever there is a hand-off
from Rust to Python, we need the GIL).
Now, `get_success(...)` loops until we see a result (until we hit the
~0.1s real-time timeout). In the loop, we call
[`time.sleep(0)`](https://docs.python.org/3/library/time.html#time.sleep )
which will "Suspend execution of the calling thread [...]" (CPU and GIL)
to allow other threads to do some work. Then like before, we advance the
Twisted reactor clock to run any scheduled callbacks which includes
anything the other threads may have scheduled.
### Does this slow down the entire test suite?
Seems just as fast as before. There is minutes variance in what we had
before and after but both are within the same range of each other.
(see PR for actual before/after timings)
2026-07-02 15:20:38 -05:00
Eric Eastwood
39316672da
Be able to shutdown homeserver that hasn't setup ( #19187 )
...
For example, a homeserver can fail to `setup` if it fails to connect to
the database.
Fix https://github.com/element-hq/synapse/issues/19188
Follow-up to https://github.com/element-hq/synapse/pull/18828
### Background
As part of Element's plan to support a light form of vhosting (virtual
host) (multiple instances of Synapse in the same Python process) (c.f
Synapse Pro for small hosts), we're currently diving into the details
and implications of running multiple instances of Synapse in the same
Python process.
"Clean tenant deprovisioning" tracked internally by
https://github.com/element-hq/synapse-small-hosts/issues/50
2025-12-02 10:58:06 -06:00
Eric Eastwood
e02a6f5e5d
Fix lost logcontext on HomeServer.shutdown() ( #19108 )
...
Same fix as https://github.com/element-hq/synapse/pull/19090
Spawning from working on clean tenant deprovisioning in the Synapse Pro
for small hosts project
(https://github.com/element-hq/synapse-small-hosts/pull/204 ).
2025-11-03 14:07:10 -06:00
Devon Hudson and Eric Eastwood
396de6544a
Cleanly shutdown SynapseHomeServer object ( #18828 )
...
This PR aims to allow for a clean shutdown of the `SynapseHomeServer`
object so that it can be fully deleted and cleaned up by garbage
collection without shutting down the entire python process.
Fix https://github.com/element-hq/synapse-small-hosts/issues/50
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Eric Eastwood <erice@element.io >
2025-10-01 02:42:09 +00:00
Eric Eastwood