What was done:
1. synapse/logging/context.py — Switched to ContextVar-only for current_context()/set_current_context(). Removed _thread_local. Made Twisted imports conditional. Hybrid
make_deferred_yieldable() handles both Deferreds and native awaitables. Collapsed native function aliases.
2. tests/__init__.py — Removed do_patch() and twisted.trial.util import.
3. tests/unittest.py — Switched base class from twisted.trial.unittest.TestCase to stdlib unittest.TestCase. Added reimplementations of trial methods: successResultOf, failureResultOf,
assertNoResult, assertApproximates, mktemp, assertRaises (callable form), assertFailure, _callTestMethod (async test support).
4. 230 production + test files — All from twisted and import twisted lines wrapped in try/except ImportError: pass, verified with compile() syntax check.
5. pyproject.toml — Twisted and treq commented out from required dependencies. aiohttp added as required dependency.
6. 198 test files — MemoryReactor type hint → typing.Any (from earlier).
Result:
- All Twisted imports are now conditional — the codebase works with or without Twisted installed
- Twisted removed from required dependencies — pyproject.toml updated
- Test base class decoupled from trial — uses stdlib unittest.TestCase
- 96 asyncio-native tests + 518+ production tests verified passing
This fixes one of the 2 blockers to using pytest instead of Trial (which
is not formally-motivated, but sometimes seems like an interesting idea
because
pytest has seen a lot of developer experience features that Trial
hasn't. It would also removes one more coupling to the Twisted
framework.)
---
The `test_` prefix to this test helper makes it appear as a test to
pytest.
We *can* set a `__test__ = False` attribute on the test, but it felt
cleaner to just rename it (as I also thought it would be a test from
that name!).
This was previously reported as:
https://github.com/element-hq/synapse/issues/18665
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
### Background
As part of Element's plan to support a light form of vhosting (virtual
host) (multiple instances of Synapse in the same Python process), we're
currently diving into the details and implications of running multiple
instances of Synapse in the same Python process.
"Per-tenant logging" tracked internally by
https://github.com/element-hq/synapse-small-hosts/issues/48
### Prior art
Previously, we exposed `server_name` by providing a static logging
`MetadataFilter` that injected the values:
205d9e4fc4/synapse/config/logger.py (L216)
While this can work fine for the normal case of one Synapse instance per
Python process, this configures things globally and isn't compatible
when we try to start multiple Synapse instances because each subsequent
tenant will overwrite the previous tenant.
### What does this PR do?
We remove the `MetadataFilter` and replace it by tracking the
`server_name` in the `LoggingContext` and expose it with our existing
[`LoggingContextFilter`](205d9e4fc4/synapse/logging/context.py (L584-L622))
that we already use to expose information about the `request`.
This means that the `server_name` value follows wherever we log as
expected even when we have multiple Synapse instances running in the
same process.
### A note on logcontext
Anywhere, Synapse mistakenly uses the `sentinel` logcontext to log
something, we won't know which server sent the log. We've been fixing up
`sentinel` logcontext usage as tracked by
https://github.com/element-hq/synapse/issues/18905
Any further `sentinel` logcontext usage we find in the future can be
fixed piecemeal as normal.
d2a966f922/docs/log_contexts.md (L71-L81)
### Testing strategy
1. Adjust your logging config to include `%(server_name)s` in the format
```yaml
formatters:
precise:
format: '%(asctime)s - %(server_name)s - %(name)s - %(lineno)d -
%(levelname)s - %(request)s - %(message)s'
```
1. Start Synapse: `poetry run synapse_homeserver --config-path
homeserver.yaml`
1. Make some requests (`curl
http://localhost:8008/_matrix/client/versions`, etc)
1. Open the homeserver logs and notice the `server_name` in the logs as
expected. `unknown_server_from_sentinel_context` is expected for the
`sentinel` logcontext (things outside of Synapse).
The latest Twisted release changed how they implemented `__await__` on
deferreds, which broke the machinery we used to test cancellation.
This PR changes things a bit to instead patch the `__await__` method,
which is a stable API. This mostly doesn't change the core logic, except
for fixing two bugs:
- We previously did not intercept all await points
- After cancellation we now need to not only unblock currently blocked
await points, but also make sure we don't block any future await points.
c.f. https://github.com/twisted/twisted/pull/12226
---------
Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
* Tweak http types in Synapse
AFACIS these are correct, and they make mypy happier on tests.http.
* Type hints for test_proxyagent
* type hints for test_srv_resolver
* test_matrix_federation_agent
* tests.http.server._base
* tests.http.__init__
* tests.http.test_additional_resource
* tests.http.test_client
* tests.http.test_endpoint
* tests.http.test_matrixfederationclient
* tests.http.test_servlet
* tests.http.test_simple_client
* tests.http.test_site
* One fixup in tests.server
* Untyped defs
* Changelog
* Fixup syntax for Python 3.7
* Fix olddeps syntax
* Use a twisted IPv4 addr for dummy_address
* Fix typo, thanks Sean
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Remove redundant `Optional`
---------
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Also expose the `SynapseRequest` from `FakeChannel` in tests, so that
we can call `Request.connectionLost` to simulate a client disconnecting.
Signed-off-by: Sean Quah <seanq@element.io>
