Add a threadpool bridge for running DB functions against pooled connections

`RustConnectionPool` runs synchronous Synapse transaction functions off the
reactor thread against the native Rust connection pool. Each
`run_with_connection(func, *args)` call runs, on a dedicated Twisted thread
pool, `func(conn, *args)` with a `Connection` checked out of the Rust pool, and
returns a Deferred that fires on the reactor with the result (or an errback if
it raised). It builds on `defer_to_threadpool`, so log contexts are preserved
across the hop, and returns the connection to the pool afterwards (clean →
reused, mid-transaction/broken → discarded, per the shim's disposal rules).

The Rust pool sizes 1:1 with the thread count, and `PyConnectionPool.close()`
(new) closes all idle connections so the owner can drop the server connections
deterministically. The bridge is lifecycle-owner-managed (start/close) and
takes no clock, keeping it trivially testable; slotting it into
`DatabasePool.make_pool` — which also needs engine-level support for the shim
connection (in_transaction, is_connection_closed, autocommit/isolation,
reconnect) — is left to a follow-up.

Tested against a live Postgres over the real reactor (Synapse's in-memory test
reactor mocks the DB thread pool out, so a plain Twisted trial TestCase is used):
result/return-value, arg forwarding, exception-to-errback, connection reuse
across calls, and concurrent checkouts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
This commit is contained in:
Erik Johnston
2026-07-06 09:46:05 +00:00
co-authored by Claude Opus 4.8
parent 28e9f95279
commit 9f7024a78a
3 changed files with 321 additions and 0 deletions
+10
View File
@@ -123,6 +123,16 @@ impl PyConnectionPool {
let conn = self.pool.get().block_on(py).map_err(pool_err_to_py)?;
Ok(Connection::new(conn))
}
/// Close the pool, closing every idle connection.
///
/// After this, [`connect`](Self::connect) fails; a connection still checked
/// out is closed when it is returned. Idempotent. This lets the owning
/// Python code drop the pool's server connections deterministically rather
/// than waiting for garbage collection.
fn close(&self) {
self.pool.close();
}
}
/// Map a `deadpool` checkout failure onto the DBAPI2 exception hierarchy.