The pool tests were written with `inlineCallbacks`; trial runs `async`
test methods directly, so use `await`. Adds a test that `txn_limit`
replaces the pooled connection after the configured number of checkouts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Pool exhaustion was invisible until checkouts timed out. Expose
deadpool's occupancy from the Rust pool and publish it as gauges per
pool (max connections, open connections, idle connections, waiting
checkouts), plus a histogram of the time a checkout waits for a
connection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
A checkout from the Rust connection pool (`ConnectionPool.connect`) previously
blocked indefinitely while waiting for a free slot or establishing a new
connection, so an unreachable or overloaded server could hang the worker
threads handling requests forever.
Bound the whole checkout with a configurable timeout: deadpool's `wait` and
`create` timeouts (enabled via its `rt_tokio_1` runtime feature, driven on our
own tokio runtime). Hitting it raises an OperationalError, which Synapse
already treats as a retryable operational error.
Configurable via the `pool_checkout_timeout` database option (milliseconds;
default 30s, `0` disables), read by the Rust engine and plumbed to both the
connection pool and the bootstrap (pool-of-one) connection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
The Rust `ConnectionPool` takes the homeserver's `RustRuntime`, whose
tokio runtime drives the pool. `make_pool`, `rust_dbapi.connect` and
`make_conn` now take it too, and `DatabasePool` and `Databases` pass
`hs.get_rust_runtime()`. `make_conn` falls back to a standalone runtime
on the global reactor for the scripts and tests that have no homeserver
to take one from.
The adapter and engine tests build their pools through the shared
`_make_pool` helper for the same reason.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULdieJyQoHNzeHq42H2M1p
Turn the Rust backend on with `database.use_rust_driver: true` (alongside
`name: psycopg2`). Keeping `name: psycopg2` means the engine is still selected as
Postgres and all the `isinstance(engine, PostgresEngine)` dialect checks hold;
the flag only swaps the driver implementation.
- `create_engine` returns `RustPostgresEngine` instead of `Psycopg2Engine` when
the flag is set.
- `make_pool` gains a `clock` argument and, for the Rust engine, builds a
`RustConnectionPool` (via `_make_rust_pool`) instead of an adbapi pool:
it derives the libpq DSN from the config `args`, sizes threads/connections
to `cp_max`, passes the engine's synchronous_commit / statement_timeout,
starts the pool, and registers a shutdown hook via the clock. Its declared
return type stays `adbapi.ConnectionPool` — `RustConnectionPool` provides the
`_db_pool` subset `DatabasePool` uses (runWithConnection / threadID /
running / threadpool) — so callers that lean on adbapi-specific methods keep
type-checking.
- `RustConnectionPool` takes synchronous_commit / statement_timeout_ms and
threads them into its pooled connections' session setup.
- `DatabasePool` passes its clock to `make_pool`; the test harness's
`make_fake_db_pool` accepts the new argument.
Tested: create_engine honours the flag (and the Rust engine is still a
PostgresEngine, not a Psycopg2Engine); make_pool builds a started
RustConnectionPool that serves a query as `_db_pool`. psycopg2 and sqlite
homeserver boots remain green. A full homeserver boot on the Rust backend isn't
covered here because the test harness patches `make_pool` with a synchronous
adbapi fake.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Make RustConnectionPool a drop-in for DatabasePool._db_pool so real
runInteraction traffic can flow through it (once make_pool is switched):
- it now hands each function a DBAPI2-adapter connection (rust_dbapi.Connection)
wrapping the pooled shim, so LoggingDatabaseConnection.cursor() yields a
working LoggingTransaction and the engine's in_transaction / is_closed /
set_autocommit operate on it;
- the entry point is named `runWithConnection` (matching
twisted.enterprise.adbapi.ConnectionPool, which database.py calls by that
name), alongside `threadID` for the transaction-limit path. With
`threadpool` and `running` already present, the pool covers the slice of the
adbapi interface DatabasePool uses.
A new test drives a full transaction through the pool the way
DatabasePool.runWithConnection's inner_func does — engine.in_transaction check,
LoggingDatabaseConnection + LoggingTransaction, `?`→`$n` conversion, commit —
off the reactor thread, and gets the result back via the Deferred.
Still outstanding before make_pool can return it: `reconnect` on the connection
(transaction-limit / closed-connection paths) and the startup path
(make_conn / check_database).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
Match adbapi's `_runWithConnection` contract in `RustConnectionPool._run`:
a transaction the function leaves open is committed when it returns and
rolled back when it raises, before the connection goes back to the pool.
The commit makes callers that rely on the pool's implicit commit durable,
and is a no-op after functions that commit themselves. The rollback
matters more: without it every raising transaction function hands a
mid-transaction connection back to the shim, which discards it (tearing
down the TCP session) rather than returning it to the pool for reuse.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ti5yoaVfjHYZ3LmzXFxSwq
`RustConnectionPool` runs synchronous Synapse transaction functions off the
reactor thread against the native Rust connection pool. Each
`run_with_connection(func, *args)` call runs, on a dedicated Twisted thread
pool, `func(conn, *args)` with a `Connection` checked out of the Rust pool, and
returns a Deferred that fires on the reactor with the result (or an errback if
it raised). It builds on `defer_to_threadpool`, so log contexts are preserved
across the hop, and returns the connection to the pool afterwards (clean →
reused, mid-transaction/broken → discarded, per the shim's disposal rules).
The Rust pool sizes 1:1 with the thread count, and `PyConnectionPool.close()`
(new) closes all idle connections so the owner can drop the server connections
deterministically. The bridge is lifecycle-owner-managed (start/close) and
takes no clock, keeping it trivially testable; slotting it into
`DatabasePool.make_pool` — which also needs engine-level support for the shim
connection (in_transaction, is_connection_closed, autocommit/isolation,
reconnect) — is left to a follow-up.
Tested against a live Postgres over the real reactor (Synapse's in-memory test
reactor mocks the DB thread pool out, so a plain Twisted trial TestCase is used):
result/return-value, arg forwarding, exception-to-errback, connection reuse
across calls, and concurrent checkouts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W3G4M92AmwSSZCbmtMJU3d
