Commit Graph
44 Commits
Author SHA1 Message Date
Eric EastwoodandGitHub 7da21a715d Update HomeserverTestCase.get_success(...) and friends to drive async Rust (Tokio runtime/thread pool) (#19871)
This means you can use `get_success(...)` anywhere regardless
of what kind of work needs to be done.

Spawning from adding some more async Rust things in
https://github.com/element-hq/synapse/pull/19846 and wanting something
more standard instead of the custom `till_deferred_has_result(...)` that
has crept in to a few files.

Alternative to https://github.com/element-hq/synapse/pull/19867 spurred
on by [this
comment](https://github.com/element-hq/synapse/pull/19867#discussion_r3441774685)
from @erikjohnston


### How does this work?

Previously, `get_success(...)` just ran in a hot-loop advancing the
Twisted reactor clock which didn't give any time for other threads to do
some work or acquire the GIL if necessary (whenever there is a hand-off
from Rust to Python, we need the GIL).

Now, `get_success(...)` loops until we see a result (until we hit the
~0.1s real-time timeout). In the loop, we call
[`time.sleep(0)`](https://docs.python.org/3/library/time.html#time.sleep)
which will "Suspend execution of the calling thread [...]" (CPU and GIL)
to allow other threads to do some work. Then like before, we advance the
Twisted reactor clock to run any scheduled callbacks which includes
anything the other threads may have scheduled.


### Does this slow down the entire test suite?

Seems just as fast as before. There is minutes variance in what we had
before and after but both are within the same range of each other.

(see PR for actual before/after timings)
2026-07-02 15:20:38 -05:00
Erik JohnstonandGitHub 1bddd25a85 Port Clock functions to use Duration class (#19229)
This changes the arguments in clock functions to be `Duration` and
converts call sites and constants into `Duration`. There are still some
more functions around that should be converted (e.g.
`timeout_deferred`), but we leave that to another PR.

We also changes `.as_secs()` to return a float, as the rounding broke
things subtly. The only reason to keep it (its the same as
`timedelta.total_seconds()`) is for symmetry with `as_millis()`.

Follows on from https://github.com/element-hq/synapse/pull/19223
2025-12-01 13:55:06 +00:00
Andrew FerrazzuttiandGitHub fc244bb592 Use type hinting generics in standard collections (#19046)
aka PEP 585, added in Python 3.9

 - https://peps.python.org/pep-0585/
 - https://docs.astral.sh/ruff/rules/non-pep585-annotation/
2025-10-22 16:48:19 -05:00
Eric EastwoodandGitHub 5a9ca1e3d9 Introduce Clock.call_when_running(...) to include logcontext by default (#18944)
Introduce `Clock.call_when_running(...)` to wrap startup code in a
logcontext, ensuring we can identify which server generated the logs.

Background:

>  Ideally, nothing from the Synapse homeserver would be logged against the `sentinel` 
>  logcontext as we want to know which server the logs came from. In practice, this is not 
>  always the case yet especially outside of request handling. 
>   
>  Global things outside of Synapse (e.g. Twisted reactor code) should run in the 
>  `sentinel` logcontext. It's only when it calls into application code that a logcontext 
>  gets activated. This means the reactor should be started in the `sentinel` logcontext, 
>  and any time an awaitable yields control back to the reactor, it should reset the 
>  logcontext to be the `sentinel` logcontext. This is important to avoid leaking the 
>  current logcontext to the reactor (which would then get picked up and associated with 
>  the next thing the reactor does). 
>
> *-- `docs/log_contexts.md`

Also adds a lint to prefer `Clock.call_when_running(...)` over
`reactor.callWhenRunning(...)`

Part of https://github.com/element-hq/synapse/issues/18905
2025-09-22 10:27:59 -05:00
reivilibreandGitHub a31d53b28f Use twisted.internet.testing module in tests instead of deprecated twisted.test.proto_helpers. (#18728)
Follows: #18727

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2025-07-30 12:32:10 +01:00
Erik JohnstonandGitHub 23740eaa3d Correctly mention previous copyright (#16820)
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
2024-01-23 11:26:48 +00:00
Patrick Cloke 8e1e62c9e0 Update license headers 2023-11-21 15:29:58 -05:00
Patrick ClokeandGitHub 679c691f6f Remove more usages of cursor_to_dict. (#16551)
Mostly to improve type safety.
2023-10-26 15:12:28 -04:00
Patrick ClokeandGitHub 9407d5ba78 Convert simple_select_list and simple_select_list_txn to return lists of tuples (#16505)
This should use fewer allocations and improves type hints.
2023-10-26 13:01:36 -04:00
David RobertsonandGitHub 6525fd65ee Log the details of background update failures (#16212) 2023-09-01 12:41:56 +01:00
Patrick ClokeandGitHub a8a46b1336 Replace simple_async_mock with AsyncMock (#16180)
Python 3.8 has a native AsyncMock, use it instead of a custom
implementation.
2023-08-25 09:27:21 -04:00
Patrick ClokeandGitHub daf11e26ef Replace make_awaitable with AsyncMock (#16179)
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
2023-08-24 19:38:46 -04:00
Erik JohnstonandGitHub e55a9b3e41 Fix downgrading to previous version of Synapse (#15907)
We do this by marking the constraint as deferrable.
2023-07-10 16:24:42 +01:00
Erik JohnstonandGitHub 95a96b21eb Add foreign key constraint to event_forward_extremities. (#15751) 2023-07-05 09:43:19 +00:00
Dirk KlimpelandGitHub 9d21ecf7ce Add type hints to tests files. (#12256) 2022-03-21 09:43:16 -04:00
ShayandGitHub ef3619e61d Add config settings for background update parameters (#11980) 2022-03-11 10:46:45 -08:00
ShayandGitHub 26211fec24 Fix a bug in background updates wherein background updates are never run using the default batch size (#12157) 2022-03-07 09:44:33 -08:00
Richard van der HoffandGitHub e24ff8ebe3 Remove HomeServer.get_datastore() (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
V02460andGitHub 7a7ca8f226 Use mock from standard library (#11588)
Instead of the backported version.
2021-12-20 10:34:46 -05:00
Patrick ClokeandGitHub ed635d3285 Add a license header and comment. (#11479) 2021-12-01 12:51:14 -05:00
d08ef6f155 Make background updates controllable via a plugin (#11306)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-11-29 17:57:06 +01:00
7f9841bdec Lower minumum batch size to 1 for background updates (#11422)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-24 19:21:44 +00:00
Jonathan de JongandGitHub 93729719b8 Use inline type hints in tests/ (#10350)
This PR is tantamount to running:

    python3.8 -m com2ann -v 6 tests/

(com2ann requires python 3.8 to run)
2021-07-13 11:52:58 +01:00
Patrick ClokeandGitHub 0b3112123d Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Eric EastwoodandGitHub 0a00b7ff14 Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick ClokeandGitHub e00816ad98 Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Patrick ClokeandGitHub 30426c7063 Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168) 2020-08-27 07:41:01 -04:00
Erik JohnstonandGitHub a7bdf98d01 Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Richard van der Hoff b4c2234232 Make do_next_background_update return a bool
returning a None or an int that we don't use is confusing.
2020-03-31 17:43:58 +01:00
Richard van der Hoff 26d17b9bdc Make has_completed_background_updates async
(Almost) everywhere that uses it is happy with an awaitable.
2020-03-31 17:43:58 +01:00
Richard van der Hoff cfe8c8ab8e Remove unused start_background_update
This was only used in a unit test, so let's just inline it in the test.
2020-03-31 17:24:06 +01:00
Richard van der HoffandGitHub d20c346544 port BackgroundUpdateTestCase to HomeserverTestCase (#6653) 2020-01-07 14:09:07 +00:00
Erik Johnston 4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Erik Johnston 756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Amber BrownandGitHub 4806651744 Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber BrownandGitHub b36c82576e Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Amber BrownandGitHub 3f189c902e Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Amber BrownandGitHub 99dd975dae Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
blackandAmber Brown 8b3d9b6b19 Run black. 2018-08-10 23:54:09 +10:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Richard van der Hoff 42f4feb2b7 PEP8 2016-07-25 12:25:06 +01:00
Richard van der Hoff 465117d7ca Fix background_update tests
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
Mark Haines 700487a7c7 Fix flake8 warnings for tests 2016-02-19 15:34:38 +00:00
Mark Haines 36c58b18a3 Test for background updates 2015-11-10 15:51:40 +00:00