mirror of
https://github.com/element-hq/synapse.git
synced 2026-11-11 14:27:48 +00:00
Erik Johnston
26e46786b3
Ports the logcontext classes to Rust, and gives tokio tasks a captured
logcontext so that work running in (or spawned from) Rust is attributed
to the request that caused it.
1. **Add characterization tests for logcontext error messages and the
filter** — pins the exact `logcontext_error` message shapes, the
abuse-detection code paths and `LoggingContextFilter`'s observable
behaviour, *against the existing Python implementation* (this commit is
green on its own). These are the behavioural contract the port has to
satisfy.
2. **Port `ContextResourceUsage` to a Rust pyclass** — self-contained:
the new `synapse_rust.logcontext` module, the class, its stub and the
re-export.
3. **Move the logcontext storage and `LoggingContext` to Rust** — the
core change; the commit message carries detailed design notes.
Highlights:
- The slot is typed `Option<Py<LoggingContext>>`, with `None`
representing the sentinel. `_Sentinel`/`SENTINEL_CONTEXT` stay pure
Python (unchanged); thin wrappers on
`current_context`/`set_current_context` convert at the boundary, and
pyo3's extraction enforces the type (`TypeError` otherwise).
- The accounting is native: one `getrusage(RUSAGE_THREAD)` read per
switch via libc, inline `stop`/`start` bookkeeping for base
`LoggingContext`s, Python dispatch only for subclasses
(`BackgroundProcessLoggingContext`) so their overrides run. The thread
id comes from `PyThread_get_thread_ident` (the exact
`threading.get_ident()`
value) without calling into Python.
- The hot paths avoid per-operation allocation: names are `Py<PyString>`
(the per-log-record `str(context)`/`server_name` reads are INCREF-only),
error branches materialise strings only when hit.
4. **Attribute Rust-spawned work to the caller's logcontext** —
`create_deferred` captures the caller's context and scopes it onto the
spawned task via a tokio task-local (`LogContextHandle`);
`current_context()` gives the task-local read precedence, so
`LoggingContextFilter`/`pyo3-log` resolve the right context on worker
threads with no per-record stamping. `run_python_awaitable` restores the
captured context (via a `with_logcontext` helper, the Rust
`PreserveLoggingContext`) around Python called back from Rust, so e.g.
`runInteraction` from the Rust `/versions` handler accounts its DB usage
against the right request. Integration tests exercise both guarantees
through real production code paths.
Follow-up work on top of this (separate PR): porting
`BackgroundProcessLoggingContext` natively and removing further `Py<_>`
indirections. The fact that `BackgroundProcessLoggingContext` is a
subclass is what forces some of the warts in this PR: e.g. having to use
`Py<LoggingContext>` everywhere, etc.
We don't try (yet) to make this pure Rust, instead we see this as simply
maintaining the Python logcontext machinery when crossing, rather than
trying to make a Rust equivalent that can be used by pure Rust
dependencies. We probably do want to do that in future, as well as wire
up e.g. CPU recording on Rust side, but that is unnecessary for now.
101 lines
2.6 KiB
Rust
101 lines
2.6 KiB
Rust
use std::convert::Infallible;
|
|
|
|
use lazy_static::lazy_static;
|
|
use pyo3::prelude::*;
|
|
use pyo3_log::ResetHandle;
|
|
|
|
pub mod acl;
|
|
pub mod canonical_json;
|
|
pub mod config;
|
|
pub mod deferred;
|
|
pub mod duration;
|
|
pub mod errors;
|
|
pub mod events;
|
|
pub mod handlers;
|
|
pub mod http;
|
|
pub mod http_client;
|
|
pub mod identifier;
|
|
pub mod json;
|
|
pub mod logging;
|
|
pub mod matrix_const;
|
|
pub mod msc4388_rendezvous;
|
|
pub mod push;
|
|
pub mod rendezvous;
|
|
pub mod room_versions;
|
|
pub mod segmenter;
|
|
pub mod storage;
|
|
pub mod tokio_runtime;
|
|
pub mod types;
|
|
|
|
lazy_static! {
|
|
static ref LOGGING_HANDLE: ResetHandle = pyo3_log::init();
|
|
}
|
|
|
|
/// Returns the hash of all the rust source files at the time it was compiled.
|
|
///
|
|
/// Used by python to detect if the rust library is outdated.
|
|
#[pyfunction]
|
|
fn get_rust_file_digest() -> &'static str {
|
|
env!("SYNAPSE_RUST_DIGEST")
|
|
}
|
|
|
|
/// Returns the `rustc` version used when this native module was built.
|
|
///
|
|
/// This value is embedded at build time, so it can be exported as a prometheus metrics.
|
|
#[pyfunction]
|
|
pub fn get_rustc_version() -> &'static str {
|
|
env!("SYNAPSE_RUSTC_VERSION")
|
|
}
|
|
|
|
/// Formats the sum of two numbers as string.
|
|
#[pyfunction]
|
|
#[pyo3(text_signature = "(a, b, /)")]
|
|
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
|
|
Ok((a + b).to_string())
|
|
}
|
|
|
|
/// Reset the cached logging configuration of pyo3-log to pick up any changes
|
|
/// in the Python logging configuration.
|
|
///
|
|
#[pyfunction]
|
|
fn reset_logging_config() {
|
|
LOGGING_HANDLE.reset();
|
|
}
|
|
|
|
/// The entry point for defining the Python module.
|
|
#[pymodule]
|
|
fn synapse_rust(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
|
|
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
|
|
m.add_function(wrap_pyfunction!(get_rust_file_digest, m)?)?;
|
|
m.add_function(wrap_pyfunction!(get_rustc_version, m)?)?;
|
|
m.add_function(wrap_pyfunction!(reset_logging_config, m)?)?;
|
|
|
|
acl::register_module(py, m)?;
|
|
logging::context::register_module(py, m)?;
|
|
deferred::register_module(py, m)?;
|
|
push::register_module(py, m)?;
|
|
events::register_module(py, m)?;
|
|
handlers::register_module(py, m)?;
|
|
http_client::register_module(py, m)?;
|
|
rendezvous::register_module(py, m)?;
|
|
msc4388_rendezvous::register_module(py, m)?;
|
|
segmenter::register_module(py, m)?;
|
|
room_versions::register_module(py, m)?;
|
|
types::register_module(py, m)?;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
pub trait UnwrapInfallible<T> {
|
|
fn unwrap_infallible(self) -> T;
|
|
}
|
|
|
|
impl<T> UnwrapInfallible<T> for Result<T, Infallible> {
|
|
fn unwrap_infallible(self) -> T {
|
|
match self {
|
|
Ok(val) => val,
|
|
Err(never) => match never {},
|
|
}
|
|
}
|
|
}
|