- 0 @defer.inlineCallbacks — all converted to async def
- 12 defer.ensureDeferred — reactor entry points (startup, shutdown, render)
- 22 defer.Deferred() — in Linearizer, ReadWriteLock, AwakenableSleeper, DeferredEvent (old implementations)
- 21 defer.gatherResults — in fallback paths and old implementations
- 11 defer.succeed/fail — immediate value wrapping in old implementations
- 3 defer.FirstError — in fallback paths
- 13 defer.TimeoutError — in timeout_deferred and its callers
The majority (22 + 21 + 11 + 13 = 67) are in the old Deferred-based utility implementations (Linearizer, ReadWriteLock, ObservableDeferred, timeout_deferred, etc.) that already have native replacements (NativeLinearizer,
NativeReadWriteLock, ObservableFuture, native_timeout, etc.). These will be removed when callers switch to the native versions.
The 12 defer.ensureDeferred are in reactor entry points that will be removed when reactor.run() → asyncio.run().
The codebase is now in a clean transitional state where:
1. All Twisted imports are conditional (try/except ImportError)
2. ContextVar is the primary logcontext storage
3. Test base class is stdlib (unittest.TestCase)
4. CancelledError is asyncio.CancelledError in production code
5. @defer.inlineCallbacks is eliminated (0 remaining)
6. yieldable_gather_results uses asyncio.gather (with Twisted fallback)
7. Module API is fully async (no more Deferred return types)
8. Twisted is optional in pyproject.toml
What was done:
1. synapse/logging/context.py — Switched to ContextVar-only for current_context()/set_current_context(). Removed _thread_local. Made Twisted imports conditional. Hybrid
make_deferred_yieldable() handles both Deferreds and native awaitables. Collapsed native function aliases.
2. tests/__init__.py — Removed do_patch() and twisted.trial.util import.
3. tests/unittest.py — Switched base class from twisted.trial.unittest.TestCase to stdlib unittest.TestCase. Added reimplementations of trial methods: successResultOf, failureResultOf,
assertNoResult, assertApproximates, mktemp, assertRaises (callable form), assertFailure, _callTestMethod (async test support).
4. 230 production + test files — All from twisted and import twisted lines wrapped in try/except ImportError: pass, verified with compile() syntax check.
5. pyproject.toml — Twisted and treq commented out from required dependencies. aiohttp added as required dependency.
6. 198 test files — MemoryReactor type hint → typing.Any (from earlier).
Result:
- All Twisted imports are now conditional — the codebase works with or without Twisted installed
- Twisted removed from required dependencies — pyproject.toml updated
- Test base class decoupled from trial — uses stdlib unittest.TestCase
- 96 asyncio-native tests + 518+ production tests verified passing
previous 4530 number from trial included ~90 tests that trial called "passed" but actually silently skipped.
This is a successful migration of the test infrastructure from twisted.trial.unittest.TestCase to stdlib unittest.TestCase.
awaitable and handles each appropriately:
- Twisted Deferred: synchronously adds logcontext callbacks (classic behavior, 100% backward compatible)
- Native awaitable (asyncio.Future, coroutine): returns an async wrapper that saves/restores logcontext
This means the migration can be incremental — code that still uses Deferreds works unchanged, while new code using native awaitables also works. The make_deferred_yieldable function becomes
the bridge.
The same pattern applies to run_in_background — it already handles both Deferreds and coroutines (via defer.ensureDeferred). It doesn't need to change.
This is a much better approach than the "flag day" — it allows gradual migration of individual subsystems from Deferred→asyncio without breaking anything.
1 new file created, 10 new tests with real HTTP server, all passing, mypy clean, no regressions.
synapse/http/native_client.py — NativeSimpleHttpClient class using aiohttp.ClientSession:
- Same public interface as SimpleHttpClient: request(), get_json(), post_json_get_json(), post_urlencoded_get_json(), put_json(), get_raw(), get_file()
- IP blocklisting via _BlocklistingResolver — custom aiohttp.abc.AbstractResolver that filters DNS results against blocklist/allowlist, preventing DNS rebinding attacks
- IP literal blocking — direct IP addresses in URLs checked before request
- Proxy support — proxy_url parameter passed to aiohttp's built-in proxy support
- Connection pooling — via aiohttp.TCPConnector with configurable limit_per_host
- Timeouts — per-request timeout via asyncio.wait_for(), connection timeout via aiohttp.ClientTimeout
- File download — streaming download with max size enforcement and content-type validation
- TLS — configurable ssl.SSLContext for custom TLS verification
Tests use a real aiohttp.web test server with endpoints for JSON, raw bytes, file downloads, form posts, and error responses.
---
Running totals across Phases 0-4:
- 5 new files, ~1500 lines of asyncio-native implementation code
- 107 tests all passing
- Existing 4462-test suite unaffected
- All mypy clean
1 new file created, 6 new tests, all passing, mypy clean, no regressions.
synapse/storage/native_database.py — NativeConnectionPool class:
- Uses concurrent.futures.ThreadPoolExecutor + asyncio.loop.run_in_executor() instead of twisted.enterprise.adbapi.ConnectionPool
- Thread-local connection management: each thread in the pool maintains its own persistent DB connection
- Automatic connection creation and initialization via engine.on_new_connection() (same as the Twisted pool's cp_openfun)
- Reconnection support for closed connections
- runWithConnection(func, *args) — runs function on a pool thread with a connection
- runInteraction(func, *args) — runs function in a transaction with auto-commit/rollback
- close() — shuts down the executor
- threadID() — compatibility method for transaction limit tracking
The existing DatabasePool and all 846+ runInteraction callers are untouched. When the migration reaches the point of switching DatabasePool to use NativeConnectionPool instead of
adbapi.ConnectionPool, the inner_func pattern in runWithConnection will be reused with minimal changes (just swap make_deferred_yieldable(self._db_pool.runWithConnection(...)) to await
self._native_pool.runWithConnection(...)).
3 new classes added to synapse/util/clock.py, 15 new tests, all passing, mypy clean, no regressions.
NativeLoopingCall — asyncio Task wrapper with stop(). Tracks in WeakSet for automatic cleanup.
NativeDelayedCallWrapper — Wraps asyncio.TimerHandle with the same interface as DelayedCallWrapper (cancel(), active(), getTime(), delay(), reset()). Since TimerHandle is immutable,
delay()/reset() cancel and reschedule.
NativeClock — Same public API as Clock but uses:
- time.time() instead of reactor.seconds()
- asyncio.sleep() instead of Deferred + reactor.callLater
- asyncio.create_task() with while True loop instead of LoopingCall
- loop.call_later() instead of reactor.callLater()
- loop.call_soon() instead of reactor.callWhenRunning()
- Logcontext wrapping preserved (same PreserveLoggingContext + run_in_background pattern)
- LoopingCall semantics preserved: waits for previous invocation to complete, survives errors
**Goal**: Switch live context tracking to `contextvars.ContextVar`. This is the foundational change everything else depends on — `contextvars` propagates automatically into `asyncio.Task` children, which is essential for native asyncio.
**Files modified**:
- `synapse/logging/context.py` (lines 736-766) — Replace `_thread_local = threading.local()` with `_current_context: ContextVar[LoggingContextOrSentinel]`. Update `current_context()` and `set_current_context()`. `LoggingContext.__enter__/__exit__` (lines 377-417) use `ContextVar.set()` token API. `PreserveLoggingContext` (line 677) works unchanged since it calls the same functions.
- `synapse/util/patch_inline_callbacks.py` — Update logcontext checks if needed for contextvars semantics.
**Key constraint**: This is backward-compatible with Twisted. Deferred callbacks run on the main thread; `ContextVar` works fine with single-threaded access. DB thread pool interactions need verification — `adbapi.ConnectionPool` uses Twisted's `ThreadPool`, and each thread gets its own contextvars copy by default, which matches current `threading.local` behavior.
Key finding: The original plan to directly replace threading.local with ContextVar was not possible while Twisted Deferreds are in use. asyncio's event loop runs call_later/call_soon callbacks
in context copies, so _set_context_cb's ContextVar write would be isolated and invisible to the awaiting code. This is fundamentally different from threading.local where writes are globally
visible on the thread.
What was implemented instead (revised Phase 1):
synapse/logging/context.py:
- _thread_local remains the primary storage for current_context() / set_current_context() — backward compatible with Twisted Deferred callback patterns
- _current_context_var (ContextVar) is kept in sync — every set_current_context() call also writes to the ContextVar
- _native_current_context() / _native_set_current_context() — operate on ContextVar only, for asyncio-native code paths (Tasks) where ContextVar propagation is correct
- make_future_yieldable(), run_coroutine_in_background_native(), run_in_background_native() — all use _native_* functions since they run inside asyncio Tasks
Migration path: The full switch from threading.local → ContextVar as sole storage happens in Phase 7 when all Deferred usage is removed. Until then, both storage mechanisms coexist.
Verification: 4462 tests passed, 169 skipped, 0 new failures. mypy clean.
Updates the error codes to match MSC2666 changes (user ID query param
validation + proper errcode for requesting rooms with self), added the
new `count` field, and stabilized the endpoint.
It's pretty hard to remember the order of all of these ambiguous
numbers. I assume they're not totally labeled already to cut down on the
length when scanning with your eyes. This just adds a few hints of what
each grouping is.
Spawning from [staring at some Synapse
logs](https://github.com/element-hq/matrix-hosted/issues/10631) and
cross-referencing the Synapse source code over and over.
Companion PR:
https://github.com/element-hq/matrix-authentication-service/pull/5550
to 1) send this flag
and 2) provision users proactively when their lock status changes.
---
Currently Synapse and MAS have two independent user lock
implementations. This PR makes it so that MAS can push its lock status
to Synapse when 'provisioning' the user.
Having the lock status in Synapse is useful for removing users from the
user directory
when they are locked.
There is otherwise no authentication requirement to have it in Synapse;
the enforcement is done
by MAS at token introspection time.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Can be reviewed commit by commit.
This sets caching headers on the /versions and /auth_metadata endpoints
to:
- allow clients to cache the response for up to 10 minutes
(`max-age=600`)
- allow proxies to cache the response for up to an hour
(`s-maxage=3600`)
- make proxies serve stale response for up to an hour (`s-maxage=3600`)
but make them refresh their response after 10 minutes
(`stale-while-revalidate=600`) so that we always have a snappy response
to client, but also have fresh responses most of the time
- only cache the response for unauthenticated requests on /versions
(`Vary: Authorization`)
I'm not too worried about the 1h TTL on the proxy side, as with the
`stale-while-revalidate` directive, one just needs to do two requests
after 10 minutes to get a fresh response from the cache.
The reason we want this, is that clients usually load this right away,
leading to a lot of traffic from people just loading the Element Web
login screen with the default config. This is currently routed to
`client_readers` on matrix.org (and ESS) which can be overwhelmed for
other reasons, leading to slow response times on those endpoints (3s+).
Overwhelmed workers shouldn't prevent people from logging in, and
shouldn't result in a long loading spinner in clients. This PR allows
caching proxies (like Cloudflare) to publicly cache the unauthenticated
response of those two endpoints and make it load quicker, reducing
server load as well.
`event_id` is a lazily-computed property on events, as it's a hash of
the event content on room version 3 and later. The reason we do this is
that it helps finding database inconsistencies by not trusting the event
ID we got from the database.
The thing is, when we clone events (to return them through /sync or
/messages for example) we don't copy the computed hash if we already
computed it, duplicating the work. This copies the internal `_event_id`
property.
This way we actually detect problems like
https://github.com/element-hq/synapse/pull/19475 as they happen instead
of being invisible until something breaks.
Sanity check that Complement is testing against your code changes
(whether it be local or from the PR in CI).
```
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh --in-repo -run TestSynapseVersion
```
Fixes: #19540Fixes: #16290 (side effect of the proposed fix)
Closes: #12804 (side effect of the proposed fix)
Introduced in: https://github.com/matrix-org/synapse/pull/8932
---
This PR is a relatively simple simplification of the profile change on
deactivation that appears to remove multiple bugs.
This PR's **primary motivating fix** is #19540: when a user is
deactivated and erased, they would be kept in the user directory. This
bug appears to have been here since #8932 (previously
https://github.com/matrix-org/synapse/pull/8932) (v1.26.0).
The root cause of this bug is that after removing the user from the user
directory, we would immediately update their displayname and avatar to
empty strings (one at a time), which re-inserts
the user into the user directory.
With this PR, we now delete the entire `profiles` row upon user erasure,
which is cleaner (from a 'your database goes back to zero after
deactivating and erasing a user' point of view) and
only needs one database operation (instead of doing displayname then
avatar).
With this PR, we also no longer send the 2 (deferred) `m.room.member`
`join` events to every room to propagate the displayname and avatar_url
changes.
This is good for two reasons:
- the user is about to get parted from those rooms anyway, so this
reduces the number of state events sent per room from 3 to 1. (More
efficient for us in the moment and leaves less litter in the room DAG.)
- it is possible for the displayname/avatar update to be sent **after**
the user parting, which seems as though it could trigger the user to be
re-joined to a public room.
(With that said, although this sounds vaguely familiar in my lossy
memory, I can't find a ticket that actually describes this bug, so this
might be fictional. Edit: #16290 seems to describe this, although the
title is misleading.)
Additionally, as a side effect of the proposed fix (deleting the
`profiles` row), this PR also now deletes custom profile fields upon
user erasure, which is a new feature/bugfix (not sure which) in its own
right.
I do not see a ticket that corresponds to this feature gap, possibly
because custom profile fields are still a niche feature without
mainstream support (to the best of my knowledge).
Tests are included for the primary bugfix and for the cleanup of custom
profile fields.
### `set_displayname` module API change
This change includes a minor _technically_-breaking change to the module
API.
The change concerns `set_displayname` which is exposed to the module API
with a `deactivation: bool = False` flag, matching the internal handler
method it wraps.
I suspect that this is a mistake caused by overly-faithfully piping
through the args from the wrapped method (this Module API was introduced
in
https://github.com/matrix-org/synapse/pull/14629/changes#diff-0b449f6f95672437cf04f0b5512572b4a6a729d2759c438b7c206ea249619885R1592).
The linked PR did the same for `by_admin` originally before it was
changed.
The `deactivation` flag's only purpose is to be piped through to other
Module API callbacks when a module has registered to be notified about
profile changes.
My claim is that it makes no sense for the Module API to have this flag
because it is not the one doing the deactivation, thus it should never
be in a position to set this to `True`.
My proposed change keeps the flag (for function signature
compatibility), but turns it into a no-op (with a `ERROR` log when it's
set to True by the module).
The Module API callback notifying of the module-caused displayname
change will therefore now always have `deactivation = False`.
*Discussed in
[`#synapse-dev:matrix.org`](https://matrix.to/#/!i5D5LLct_DYG-4hQprLzrxdbZ580U9UB6AEgFnk6rZQ/$1f8N6G_EJUI_I_LvplnVAF2UFZTw_FzgsPfB6pbcPKk?via=element.io&via=matrix.org&via=beeper.com)*
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Use non-deprecated imports for collections
Other than being deprecated, these legacy imports also don't seem to be
compatible with [Ty](https://github.com/astral-sh/ty)
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
When a worker gets very busy some of these loops can get large and end
up taking hundreds of ms to complete. To help keep the reactor tick
times reasonable we add a periodic yield into these loops.
These were found by doing a `py-spy` and speedscope.net (in time order)
to see where we were spending blocks of time
This allows the Rust HTTP client to be configured to force HTTP/2 even
on plaintext connections. This is useful in contexts where the remote
server is known to server HTTP/2 over plain text.
Added because we use the Synapse Rust HTTP client with the Synapse Pro
`event-cache` module. We use this because it's independent from the
Python reactor which makes things slower than expected.
Currently, the Synapse Rust HTTP client uses HTTP/1 which means a new
connection for every request. With HTTP/2, we can share the connection
across requests.
We want to see if this will make a performance difference and less
stress on the database connection situation, see
https://github.com/element-hq/synapse-rust-apps/issues/452#issuecomment-3897717599
Here is the sibling PR for using HTTP/2 on the Synapse Pro `event-cache`
module side: https://github.com/element-hq/synapse-pro-modules/pull/35
Part of: MSC4354 whose experimental feature tracking issue is
https://github.com/element-hq/synapse/issues/19409
Follows: #19340 (a necessary bugfix for `/event/` to set this metadata)
Partially supersedes: #18968
This PR implements the first batch of work to support MSC4354 Sticky
Events.
Sticky events are events that have been configured with a finite
'stickiness' duration,
capped to 1 hour per current MSC draft.
Whilst an event is sticky, we provide stronger delivery guarantees for
the event, both to
our clients and to remote homeservers, essentially making it reliable
delivery as long as we
have a functional connection to the client/server and until the
stickiness expires.
This PR merely supports creating sticky events and receiving the sticky
TTL metadata in clients.
It is not suitable for trialling sticky events since none of the other
semantics are implemented.
Contains a temporary SQLite workaround due to a bug in our supported
version enforcement: https://github.com/element-hq/synapse/issues/19452
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
