ac26cfac83
Let me give you a clear summary of where this project stands:
...
---
Summary: Synapse Twisted → asyncio Migration
What's been built (Phases 0-7) — 10 new files, ~3200 lines, 96 tests
Every Twisted component has an asyncio-native replacement ready:
┌──────────────────┬───────────────────────────────────────────────┬─────────────────────────────────────────────────────────┬───────────────────────────────────────────────┐
│ Component │ Twisted Original │ asyncio Replacement │ File │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ LoggingContext │ threading.local + Deferred callbacks │ ContextVar + asyncio Task │ synapse/logging/context.py (native functions) │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Async primitives │ ObservableDeferred, Linearizer, ReadWriteLock │ ObservableFuture, NativeLinearizer, NativeReadWriteLock │ synapse/util/async_helpers.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Cache │ DeferredCache │ FutureCache │ synapse/util/caches/future_cache.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Clock │ Clock (Twisted reactor) │ NativeClock (asyncio) │ synapse/util/clock.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Database │ adbapi.ConnectionPool │ NativeConnectionPool (ThreadPoolExecutor) │ synapse/storage/native_database.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ HTTP Client │ treq + Twisted Agent │ aiohttp.ClientSession │ synapse/http/native_client.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ HTTP Server │ JsonResource + Twisted Site │ NativeJsonResource + aiohttp.web │ synapse/http/native_server.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Replication │ LineOnlyReceiver (Twisted Protocol) │ asyncio.StreamReader/Writer │ synapse/replication/tcp/native_protocol.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Event loop │ ISynapseReactor │ ISynapseEventLoop │ synapse/types/__init__.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Test helper │ — │ FakeAsyncioLoop │ tests/async_helpers.py │
└──────────────────┴───────────────────────────────────────────────┴─────────────────────────────────────────────────────────┴───────────────────────────────────────────────┘
What's been wired in safely — 224 files changed, 0 regressions
- MemoryReactor type hint → Any across 198 test files (cosmetic)
- synapse/http/server.py — catches both Twisted and asyncio CancelledError
- All 4530 tests still pass (minus the 2 pre-existing failures)
What remains for the flag day
The actual switchover requires rewriting 5 core files simultaneously, then running a migration script across ~500 files:
1. tests/unittest.py + tests/server.py — switch from twisted.trial.TestCase to unittest.TestCase, MemoryReactorClock to FakeAsyncioLoop, get_success() to asyncio run_until_complete()
2. synapse/logging/context.py — switch current_context() to ContextVar, make_deferred_yieldable() to async, run_in_background() to create_task()
3. synapse/util/async_helpers.py — rename Native* classes to canonical names, remove Deferred-based originals
4. Migration script — update all CancelledError, defer.*, Deferred imports across ~500 files
5. pyproject.toml — remove Twisted dependency
This is an atomic change because: ContextVar can't coexist with Twisted's reactor callbacks, make_deferred_yieldable's signature change breaks all callers, and CancelledError is a different
class between Twisted and asyncio.
2026-03-21 16:17:04 +00:00
fcac7e0282
Write union types as X | Y where possible ( #19111 )
...
aka PEP 604, added in Python 3.10
2025-11-06 14:02:33 -06:00
5a9ca1e3d9
Introduce Clock.call_when_running(...) to include logcontext by default ( #18944 )
...
Introduce `Clock.call_when_running(...)` to wrap startup code in a
logcontext, ensuring we can identify which server generated the logs.
Background:
> Ideally, nothing from the Synapse homeserver would be logged against the `sentinel`
> logcontext as we want to know which server the logs came from. In practice, this is not
> always the case yet especially outside of request handling.
>
> Global things outside of Synapse (e.g. Twisted reactor code) should run in the
> `sentinel` logcontext. It's only when it calls into application code that a logcontext
> gets activated. This means the reactor should be started in the `sentinel` logcontext,
> and any time an awaitable yields control back to the reactor, it should reset the
> logcontext to be the `sentinel` logcontext. This is important to avoid leaking the
> current logcontext to the reactor (which would then get picked up and associated with
> the next thing the reactor does).
>
> *-- `docs/log_contexts.md`
Also adds a lint to prefer `Clock.call_when_running(...)` over
`reactor.callWhenRunning(...)`
Part of https://github.com/element-hq/synapse/issues/18905
2025-09-22 10:27:59 -05:00
a31d53b28f
Use twisted.internet.testing module in tests instead of deprecated twisted.test.proto_helpers. ( #18728 )
...
Follows: #18727
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org >
2025-07-30 12:32:10 +01:00
23740eaa3d
Correctly mention previous copyright ( #16820 )
...
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
2024-01-23 11:26:48 +00:00
8e1e62c9e0
Update license headers
2023-11-21 15:29:58 -05:00
647ff3ef65
Remove unused room_alias field from /createRoom response ( #15093 )
...
* Change `create_room` return type
* Don't return room alias from /createRoom
* Update other callsites
* Fix up mypy complaints
It looks like new_room_user_id is None iff new_room_id is None. It's a
shame we haven't expressed this in a way that mypy can understand.
* Changelog
2023-02-22 11:07:28 +00:00
42aea0d8af
Add final type hint to tests.unittest. ( #15072 )
...
Adds a return type to HomeServerTestCase.make_homeserver and deal
with any variables which are no longer Any.
2023-02-14 14:03:35 -05:00
3ac412b4e2
Require types in tests.storage. ( #14646 )
...
Adds missing type hints to `tests.storage` package
and does not allow untyped definitions.
2022-12-09 12:36:32 -05:00
568016929f
Clarify that a method returns only unthreaded receipts. ( #13937 )
...
By renaming it and updating the docstring.
Additionally, refactors a method which is used only by tests.
2022-09-29 07:07:31 -04:00
efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). ( #13782 )
...
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
2022-09-23 14:33:28 +00:00
0e99f07952
Remove support for unstable private read receipts ( #13653 )
...
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com >
2022-09-01 13:31:54 +01:00
ab18441573
Support stable identifiers for MSC2285: private read receipts. ( #13273 )
...
This adds support for the stable identifiers of MSC2285 while
continuing to support the unstable identifiers behind the configuration
flag. These will be removed in a future version.
2022-08-05 11:09:33 -04:00
86e366a46e
Remove old empty/redundant slaved stores. ( #13349 )
2022-07-21 17:56:45 +00:00
Matthew Hodgson