mirror of
https://github.com/element-hq/synapse.git
synced 2026-08-15 00:30:06 +00:00
Erik JohnstonandClaude Opus 4.8
7e8da7619d
The Postgres driver is async but the Python-facing API will be synchronous, so we need a way to drive tokio futures to completion from a sync, GIL-holding method. Add a process-global, lazily-initialised multi-thread tokio runtime (`database::runtime`) whose worker threads only drive the connection tasks: the actual blocking wait happens on the calling Python thread, so a small fixed pool can't starve itself. Add the `block_on` / `block_on_result` / `block_on_next` extension traits (`database::postgres::helpers`) that block on that runtime while releasing the GIL (`py.detach`), so other Python threads keep running while we wait. `pg_err_to_py` maps a driver error into a Python `RuntimeError`. `BlockingPostgresStream` is implemented generically over `Pin<&mut Fuse<S>>` rather than just `RowStream`. This is deliberate: it lets the cursor state machine added in the next change be unit-tested against an in-memory fake stream, with no live database. The module doc spells out why polling a `RowStream` off the runtime (the non-blocking fast path) is sound, and why the `Fuse` bound is required rather than merely assumed. These modules are `pub` for now so the not-yet-consumed items don't trip clippy's `dead_code` lint; later changes tighten that. Unit tests cover the ready / pending / exhausted stream paths (using an in-memory stream and a `yield_now`-based pending future) and the runtime's shared-instance behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
50 lines
1.9 KiB
Rust
50 lines
1.9 KiB
Rust
//! Shared tokio runtime used by the async-backed database drivers (currently
|
|
//! just the Postgres backend). The runtime is created lazily on first use and
|
|
//! lives for the lifetime of the process.
|
|
|
|
use once_cell::sync::Lazy;
|
|
use tokio::runtime::Runtime;
|
|
|
|
static RUNTIME: Lazy<Runtime> = Lazy::new(|| {
|
|
tokio::runtime::Builder::new_multi_thread()
|
|
// A small fixed pool is enough: these threads only drive the I/O-bound
|
|
// connection tasks and the futures blocked on them (the actual blocking
|
|
// wait happens on the calling Python thread, see `helpers::block_on`),
|
|
// so we don't need one worker per connection.
|
|
//
|
|
// This is also why a fixed pool of two can't deadlock: every blocking
|
|
// wait (`Runtime::block_on`) runs on the *calling* Python thread, never
|
|
// on a worker, so the workers are always free to drive the connection
|
|
// tasks that those waits are waiting on. Moving a blocking wait onto a
|
|
// worker (e.g. via `Handle::block_on` from inside the runtime) would
|
|
// break that invariant.
|
|
.worker_threads(2)
|
|
.enable_all()
|
|
.thread_name("synapse-db")
|
|
.build()
|
|
.expect("failed to build tokio runtime for synapse database backend")
|
|
});
|
|
|
|
pub fn runtime() -> &'static Runtime {
|
|
&RUNTIME
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn runtime_runs_futures_and_is_shared() {
|
|
// The runtime drives a future to completion...
|
|
assert_eq!(runtime().block_on(async { 1 + 1 }), 2);
|
|
|
|
// ...and is the same instance on every call (it's a process-global
|
|
// `Lazy`), so a second `block_on` reuses it rather than spinning up a
|
|
// fresh runtime.
|
|
let first = runtime() as *const Runtime;
|
|
let second = runtime() as *const Runtime;
|
|
assert_eq!(first, second);
|
|
assert_eq!(runtime().block_on(async { "ok" }), "ok");
|
|
}
|
|
}
|