Previously every `tokio_postgres` error became a bare `RuntimeError`. Synapse's
transaction driver, though, branches on the *type* of a database error and on
its `pgcode`: `new_transaction` retries `OperationalError`, retries deadlocks it
recognises via `is_deadlock` (which reads `pgcode`) on a `DatabaseError`, and
`simple_upsert` retries `IntegrityError`. With everything collapsed to
`RuntimeError` none of that fired.
Add just the distinctions Synapse acts on, rather than psycopg2's full PEP-249
hierarchy: `Error` -> `DatabaseError` -> {`OperationalError`, `IntegrityError`},
exposed on the `postgres` submodule, each instance tagged with `pgcode` (the
SQLSTATE string, or `None`). A small classifier maps the SQLSTATE class:
constraint violations (`23`) to `IntegrityError`, connection/resource classes
(`08`/`53`/`57`/`58`) to `OperationalError`, everything else (incl. `40*`
deadlocks, which retry via `pgcode`) to `DatabaseError`. Codeless errors are
split with `is_closed()`: a lost connection is operational, any other (a bad
parameter, a failed connect) is a plain `DatabaseError` so it isn't retried.
Errors surfacing while a result stream is drained (the usual case for an
`INSERT` constraint violation) now route through the same mapping, so they carry
the right class and `pgcode` too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`tokio_postgres` and libpq disagree on the default host when a DSN gives
no `host=`: tokio-postgres falls back to localhost, whereas libpq (which
`psql` and the rest of Synapse use) applies its configurable compiled-in
default — typically the Unix socket directory — and honours `PGHOST`.
To keep Synapse's existing connection behaviour, `connect()` now runs the
DSN through `fixup_default_host`: if the parsed config has neither a host
nor a hostaddr, it asks libpq what its default would be (via
`PQconnectStart` on an empty conninfo, which applies libpq's defaults/env
without opening a socket) and sets that on the tokio-postgres config.
The libpq call lives in a small hand-written safe wrapper
(`database::postgres::libpq`) over the `pq-sys` crate. pq-sys ships
pre-generated bindings and links the system libpq itself, so — unlike a
bindgen-based binding crate — this needs no libclang to build. The
behaviour is exercised by the Python integration tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the Python-facing `Connection` / `Cursor` pair and the `connect`
factory, implementing enough of the PEP-249 (DBAPI2) shape for Synapse's
needs.
A `Connection` owns a single `tokio_postgres::Client`. The client *moves*
between the connection and an in-flight cursor rather than being shared:
it lives in the connection between interactions, is taken out for the
duration of a cursor, and is handed back when the cursor finishes. That
single-owner baton (an `Option<Client>` slot on each side) makes it
structurally impossible to use the connection mid-transaction or to run
two overlapping transactions on one socket — both become a clean "already
closed" error.
The transaction lifecycle (`cursor` opens with `BEGIN`; `finish`
COMMIT/ROLLBACKs and hands the client back; `CursorGuard` rolls back an
abandoned transaction on drop) is included here. On any
transaction-control error the client is dropped rather than returned,
closing the socket — safer than handing a possibly-broken connection back
to what will become a connection pool.
`connect()` parses a libpq-style DSN, blocks until connected, and spawns
the long-lived connection task onto the shared runtime (the libpq
default-host fixup is a follow-up). The cursor query methods
(`execute` / `fetch_one` / `fetch_all` / `fetch_next_batch` / `rowcount`)
delegate to the `CursorQueryState` machine.
`run_interaction` — the high-level glue that opens a cursor, runs the
callback, and commits/rolls back — follows in the next change, so
`cursor`/`finish`/`CursorGuard` carry a transitional `#[allow(dead_code)]`
until then. Now that the value/helpers/cursor_state modules are consumed
internally, their visibility is tightened from `pub` back to private.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Model the lifecycle of a cursor's most recent query as an explicit state
machine (Idle / Active / Closed) so that illegal field combinations are
unrepresentable and fetching past the end of a result set is a clean,
specific error rather than the spurious "connection closed" you get from
re-polling a finished stream. The row stream is fused so an
exhausted-but-not-yet-reported stream can sit safely in `Active`.
This change adds `fetch_one`, `fetch_all` and `rowcount` (the batched
`fetch_next_batch` follows separately). On a stream error a cursor resets
to `Idle`; on normal exhaustion it moves to `Closed`, retaining the
PEP-249 rowcount from the command tag.
The state machine is generic over the stream type, defaulting to
`RowStream`, via a small `CursorRowStream` trait that abstracts the three
things the logic needs (the affected-row count, row->PyTuple conversion,
and error rendering). This is what lets the state transitions, exhaustion
handling and error recovery be unit-tested against an in-memory fake
stream, with no live Postgres server.
`cursor_state` is `pub` for now so its not-yet-consumed items don't trip
clippy's dead_code lint; the connection code wires it up and tightens that
later.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the `block_on`/`block_on_result`/`block_on_next` helpers the Postgres
backend uses to drive its async `tokio-postgres` futures to completion from
sync, GIL-holding Python methods, releasing the GIL for the wait. They take
a `tokio::runtime::Handle` and block on it from the calling (Python) thread.
Rather than give the DB backend a runtime of its own, they use the
extension's existing shared runtime (`tokio_runtime::PyTokioRuntime`, stored
on the reactor). `start` is made idempotent and a `runtime_handle` accessor
starts it on demand, so a caller that needs a connection before the reactor
is running still gets a handle; once the reactor runs, its
`callWhenRunning(start)` hook is a no-op.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RPEeXx2fAG67o6u4CnmC8W
Introduce the Python<->Postgres value-mapping layer that the forthcoming
Rust DBAPI2 backend will build on, plus the module scaffolding to hang it
off (`synapse_rust.database.postgres`).
`PgValue` is an owned representation of a bound parameter that implements
`tokio_postgres::types::ToSql`, encoding into Postgres' binary wire format
according to the column type from the prepared statement (so a single
Python `int` becomes INT2/INT4/INT8 as appropriate, with range checks).
`PythonPgFromSql` is the decode counterpart, turning a column's wire bytes
back into the natural Python object (and SQL NULL into `None`). The two
`accepts` lists are kept in sync via tests.
The `value` submodule is exposed as `pub` for now so its
not-yet-consumed public items don't trip clippy's `dead_code` lint; later
changes that wire it into the cursor/connection code tighten that back up.
Covers int / float / bool / str / bytes / None; lists and richer types
(json, decimal, timestamps) are left to a follow-up.
The unit tests exercise the whole mapping without a live Postgres server,
including the float4 lossy narrowing, integer width boundaries, the
WrongType and out-of-range paths, and the unsupported/non-UTF-8 decode
errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
