Files
synapse/tests/rest/client/test_matrixrtc.py
T
Eric EastwoodandGitHub c63d77a79d Rust database access via Python database connection pool v2 (#19878)
This is a stepping stone before we can go full Rust everywhere. We're
providing a generic interface as we want database access to work in
Synapse and `synapse-rust-apps`. In `synapse-rust-apps`, we will use a
`tokio-postgres` based database connection pool so it's full Rust.

We want to avoid the situation where we have two database connection
pools (one for Python, one for Rust) as we've run into connection
exhaustion problems on Matrix.org before.

As an example of using it and sanity check for all this work (including
tests), I've also ported over the `/versions` handler to the Rust side
with database access. The `/versions` endpoint is the simplest endpoint
I could find that still had some database access. Hopefully the refactor
on `/versions` isn't that controversial as it's not really the point of
this PR. We can always remove it from this PR but it's just here as a
sanity check that all of this works.


### Why `runInteraction(...)`?

Using the same `runInteraction` pattern that we already have in Synapse
means we can port over existing Synapse code/endpoints without much
thought. But this pattern also makes sense because we want[^1]
transactions to have repeatable-read isolation (easy to think about,
less foot-guns). Having everything thappen in a function callback means
we can do retries for serialization/deadlock errors.

[^1]: To note: Ideally, we'd want the least isolation possible but the
problem is that there is no tooling to yell at you when your
queries/logic is wrong so repeatable-read isolation is a great balance.


> When an application receives this error message, it should abort the
current transaction and retry the whole transaction from the beginning.
The second time through, the transaction will see the
previously-committed change as part of its initial view of the database,
so there is no logical conflict in using the new version of the row as
the starting point for the new transaction's update.
> 
> Note that only updating transactions might need to be retried;
read-only transactions will never have serialization conflicts.
>
> *--
https://www.postgresql.org/docs/current/transaction-iso.html#XACT-REPEATABLE-READ*


As a note, this strategy is less of an impedance mismatch (aligns more
closely) with Synapse so the glue code for the `python_db_pool` should
also be simpler.


### How does this interact with logcontext (`LoggingContext`)?

See [docs on log
contexts](https://github.com/element-hq/synapse/blob/4e9f7757f17ba81b8747b7f8f9646d17df145aa3/docs/log_contexts.md)
for more background.

We already support normal logging from Rust -> Python with `pyo3-log`
and `log` but as soon as we pass a thread boundary, everything is logged
against the `sentinel` log context. Normally, we want logs and CPU/DB
usage correlated with the request that spawned the work.

You can see how I took a stab at fixing this in
https://github.com/element-hq/synapse/pull/19846 by capturing the
logcontext in a Tokio task local and re-activating as necessary. For
example, in that PR, I reactivated the logcontext in
`run_python_awaitable(...)` which we use to call `runInteraction(...)`
from the Rust side which means all of the database usage is correlated
with the request as expected. It also means any `log:info!(...)` done in
`run_interaction(...)` is correlated correctly. But there needs to be a
better story for when you want to log everywhere else.

I haven't explored tracking CPU usage on the Rust side.

I've left all of this out of this PR as I think it will be better to
tackle this as a dedicated follow-up. For example, I'm thinking about
instead creating a new `LoggingContext` with the `parent_context` set to
the calling context and try to avoid needing to call
`set_current_context(...)` on the Python side where possible (like
tracking CPU).



### Testing strategy

Added some tests that exercise some `async` Rust handlers for the
`/versions` endpoint:
```
SYNAPSE_TEST_LOG_LEVEL=INFO poetry run trial tests.rest.client.test_versions.VersionsTestCase
```

Real-world:

 1. `poetry run synapse_homeserver --config-path homeserver.yaml`
 1. `GET http://localhost:8008/_matrix/client/versions`
2026-07-07 13:07:35 -05:00

153 lines
5.9 KiB
Python

#
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
#
# [This file includes modifications made by New Vector Limited]
#
#
"""Tests REST events for /rtc/endpoints path."""
from twisted.internet.testing import MemoryReactor
from synapse.rest import admin
from synapse.rest.client import login, matrixrtc, register, room, versions
from synapse.server import HomeServer
from synapse.synapse_rust.http_client import HttpClient
from synapse.util.clock import Clock
from tests import unittest
from tests.unittest import HomeserverTestCase, override_config
PATH_PREFIX = "/_matrix/client/unstable/org.matrix.msc4143"
RTC_ENDPOINT = {"type": "focusA", "required_field": "theField"}
LIVEKIT_ENDPOINT = {
"type": "livekit",
"livekit_service_url": "https://livekit.example.com",
}
class MatrixRtcTestCase(HomeserverTestCase):
"""Tests /rtc/transports Client-Server REST API."""
servlets = [
admin.register_servlets,
room.register_servlets,
login.register_servlets,
register.register_servlets,
matrixrtc.register_servlets,
]
def prepare(
self, reactor: MemoryReactor, clock: Clock, homeserver: HomeServer
) -> None:
self.register_user("alice", "password")
self._alice_tok = self.login("alice", "password")
def test_matrixrtc_endpoint_not_enabled(self) -> None:
channel = self.make_request(
"GET", f"{PATH_PREFIX}/rtc/transports", access_token=self._alice_tok
)
self.assertEqual(404, channel.code, channel.json_body)
self.assertEqual(
"M_UNRECOGNIZED", channel.json_body["errcode"], channel.json_body
)
@override_config({"experimental_features": {"msc4143_enabled": True}})
def test_matrixrtc_endpoint_requires_authentication(self) -> None:
channel = self.make_request("GET", f"{PATH_PREFIX}/rtc/transports")
self.assertEqual(401, channel.code, channel.json_body)
@override_config(
{
"experimental_features": {"msc4143_enabled": True},
"matrix_rtc": {"transports": [RTC_ENDPOINT]},
}
)
def test_matrixrtc_endpoint_contains_expected_transport(self) -> None:
channel = self.make_request(
"GET", f"{PATH_PREFIX}/rtc/transports", access_token=self._alice_tok
)
self.assertEqual(200, channel.code, channel.json_body)
self.assert_dict({"rtc_transports": [RTC_ENDPOINT]}, channel.json_body)
@override_config(
{
"experimental_features": {"msc4143_enabled": True},
"matrix_rtc": {"transports": []},
}
)
def test_matrixrtc_endpoint_no_transports_configured(self) -> None:
channel = self.make_request(
"GET", f"{PATH_PREFIX}/rtc/transports", access_token=self._alice_tok
)
self.assertEqual(200, channel.code, channel.json_body)
self.assert_dict({}, channel.json_body)
@override_config(
{
"experimental_features": {"msc4143_enabled": True},
"matrix_rtc": {"transports": [LIVEKIT_ENDPOINT]},
}
)
def test_matrixrtc_endpoint_livekit_transport(self) -> None:
channel = self.make_request(
"GET", f"{PATH_PREFIX}/rtc/transports", access_token=self._alice_tok
)
self.assertEqual(200, channel.code, channel.json_body)
self.assert_dict({"rtc_transports": [LIVEKIT_ENDPOINT]}, channel.json_body)
class MatrixRtcVersionsTestCase(HomeserverTestCase):
"""Tests that org.matrix.msc4143 is correctly advertised in /versions."""
servlets = [versions.register_servlets]
def make_homeserver(self, reactor: MemoryReactor, clock: Clock) -> HomeServer:
hs = self.setup_test_homeserver()
# XXX: We must create the Rust HTTP client before we call `reactor.run()` below.
# Twisted's `MemoryReactor` doesn't invoke `callWhenRunning` callbacks if it's
# already running and we rely on that to start the Tokio thread pool in Rust. In
# the future, this may not matter, see https://github.com/twisted/twisted/pull/12514
self._http_client = hs.get_proxied_http_client()
_ = HttpClient(
reactor=hs.get_reactor(),
user_agent=self._http_client.user_agent.decode("utf8"),
)
# This triggers the server startup hooks, which starts the Tokio thread pool
reactor.run()
return hs
def tearDown(self) -> None:
# MemoryReactor doesn't trigger the shutdown phases, and we want the
# Tokio thread pool to be stopped
# XXX: This logic should probably get moved somewhere else
shutdown_triggers = self.reactor.triggers.get("shutdown", {})
for phase in ["before", "during", "after"]:
triggers = shutdown_triggers.get(phase, [])
for callbable, args, kwargs in triggers:
callbable(*args, **kwargs)
def test_msc4143_false_by_default(self) -> None:
channel = self.make_request("GET", "/_matrix/client/versions")
self.assertEqual(channel.code, 200, channel.result)
self.assertFalse(channel.json_body["unstable_features"]["org.matrix.msc4143"])
@unittest.override_config({"experimental_features": {"msc4143_enabled": True}})
def test_msc4143_true_if_enabled(self) -> None:
channel = self.make_request("GET", "/_matrix/client/versions")
self.assertEqual(channel.code, 200, channel.result)
self.assertTrue(channel.json_body["unstable_features"]["org.matrix.msc4143"])