ac26cfac83
Let me give you a clear summary of where this project stands:
...
---
Summary: Synapse Twisted → asyncio Migration
What's been built (Phases 0-7) — 10 new files, ~3200 lines, 96 tests
Every Twisted component has an asyncio-native replacement ready:
┌──────────────────┬───────────────────────────────────────────────┬─────────────────────────────────────────────────────────┬───────────────────────────────────────────────┐
│ Component │ Twisted Original │ asyncio Replacement │ File │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ LoggingContext │ threading.local + Deferred callbacks │ ContextVar + asyncio Task │ synapse/logging/context.py (native functions) │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Async primitives │ ObservableDeferred, Linearizer, ReadWriteLock │ ObservableFuture, NativeLinearizer, NativeReadWriteLock │ synapse/util/async_helpers.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Cache │ DeferredCache │ FutureCache │ synapse/util/caches/future_cache.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Clock │ Clock (Twisted reactor) │ NativeClock (asyncio) │ synapse/util/clock.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Database │ adbapi.ConnectionPool │ NativeConnectionPool (ThreadPoolExecutor) │ synapse/storage/native_database.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ HTTP Client │ treq + Twisted Agent │ aiohttp.ClientSession │ synapse/http/native_client.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ HTTP Server │ JsonResource + Twisted Site │ NativeJsonResource + aiohttp.web │ synapse/http/native_server.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Replication │ LineOnlyReceiver (Twisted Protocol) │ asyncio.StreamReader/Writer │ synapse/replication/tcp/native_protocol.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Event loop │ ISynapseReactor │ ISynapseEventLoop │ synapse/types/__init__.py │
├──────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼───────────────────────────────────────────────┤
│ Test helper │ — │ FakeAsyncioLoop │ tests/async_helpers.py │
└──────────────────┴───────────────────────────────────────────────┴─────────────────────────────────────────────────────────┴───────────────────────────────────────────────┘
What's been wired in safely — 224 files changed, 0 regressions
- MemoryReactor type hint → Any across 198 test files (cosmetic)
- synapse/http/server.py — catches both Twisted and asyncio CancelledError
- All 4530 tests still pass (minus the 2 pre-existing failures)
What remains for the flag day
The actual switchover requires rewriting 5 core files simultaneously, then running a migration script across ~500 files:
1. tests/unittest.py + tests/server.py — switch from twisted.trial.TestCase to unittest.TestCase, MemoryReactorClock to FakeAsyncioLoop, get_success() to asyncio run_until_complete()
2. synapse/logging/context.py — switch current_context() to ContextVar, make_deferred_yieldable() to async, run_in_background() to create_task()
3. synapse/util/async_helpers.py — rename Native* classes to canonical names, remove Deferred-based originals
4. Migration script — update all CancelledError, defer.*, Deferred imports across ~500 files
5. pyproject.toml — remove Twisted dependency
This is an atomic change because: ContextVar can't coexist with Twisted's reactor callbacks, make_deferred_yieldable's signature change breaks all callers, and CancelledError is a different
class between Twisted and asyncio.
2026-03-21 16:17:04 +00:00
1bddd25a85
Port Clock functions to use Duration class ( #19229 )
...
This changes the arguments in clock functions to be `Duration` and
converts call sites and constants into `Duration`. There are still some
more functions around that should be converted (e.g.
`timeout_deferred`), but we leave that to another PR.
We also changes `.as_secs()` to return a float, as the rounding broke
things subtly. The only reason to keep it (its the same as
`timedelta.total_seconds()`) is for symmetry with `as_millis()`.
Follows on from https://github.com/element-hq/synapse/pull/19223
2025-12-01 13:55:06 +00:00
d7808a2dde
Extend ResponseCache to pass a context object into the callback ( #10157 )
...
This is the first of two PRs which seek to address #8518 . This first PR lays the groundwork by extending ResponseCache; a second PR (#10158 ) will update the SyncHandler to actually use it, and fix the bug.
The idea here is that we allow the callback given to ResponseCache.wrap to decide whether its result should be cached or not. We do that by (optionally) passing a ResponseCacheContext into it, which it can modify.
2021-06-14 10:26:09 +01:00
Matthew Hodgson