The problem is that while `pillow@12.3.0` supports Python >= 3.10,
`pillow@13.0.0` only supports Python > 3.11, and we haven't dropped
support for Python 3.10 yet (tracked by
https://github.com/element-hq/synapse/issues/20041).
`pillow@13.0.0` includes some important security fixes:
https://pillow.readthedocs.io/en/stable/releasenotes/13.0.0.html
### How was this change made?
1. `poetry update pillow`
1. Manually change the `"pillow"` `markers` in `poetry.lock` to split at
Python 3.11 instead of 3.14
Please note this is not a durable change. `poetry` will try to revert
this any time we try to make a change (requires diligence to make sure
we keep changing it back).
This maintenance burden will disappear once we drop Python 3.10 support
(tracked by https://github.com/element-hq/synapse/issues/20041).
### Alternatives
If we were up for updating the minimum version of `pillow` in
`pyproject.toml`, we could do something like the following which would
resolve in `poetry.lock` durably how we want,
`pyproject.toml`
```
# Pillow 13 dropped Python 3.10 but has security fixes. Split so 3.11+ can pick up
# the security fixes.
#
# FIXME: Remove the split once we drop Python 3.10 support.
"Pillow>=10.0.1,<13; python_version < '3.11'",
"Pillow>=13.0.0; python_version >= '3.11'",
```
It looks like we [previously
bumped](https://github.com/element-hq/synapse/blob/4853cd4238e100183fd7b0852ddf3e8eef894658/pyproject.toml#L54-L56)
the minimum version of `Pillow` for another CVE
(https://github.com/matrix-org/synapse/pull/16347)
Although, I'm guessing we'd rather leave the minimum version alone and
only update `poetry.lock`.
I think the problem with updating the minimum is that the packagers for
Linux distros will have to deal with not having the latest `pillow`
version available and may patch `pillow` a different way.
The problem is that while `pillow@12.3.0` supports Python >= 3.10,
`pillow@13.0.0` only supports Python > 3.11, and we haven't dropped
support for Python 3.10 yet (tracked by
https://github.com/element-hq/synapse/issues/20041).
`pillow@13.0.0` includes some important security fixes:
https://pillow.readthedocs.io/en/stable/releasenotes/13.0.0.html
### How was this change made?
1. `poetry update pillow`
1. Manually change the `"pillow"` `markers` in `poetry.lock` to split at
Python 3.11 instead of 3.14
Please note this is not a durable change. `poetry` will try to revert
this any time we try to make a change (requires diligence to make sure
we keep changing it back).
This maintenance burden will disappear once we drop Python 3.10 support
(tracked by https://github.com/element-hq/synapse/issues/20041).
### Alternatives
If we were up for updating the minimum version of `pillow` in
`pyproject.toml`, we could do something like the following which would
resolve in `poetry.lock` durably how we want,
`pyproject.toml`
```
# Pillow 13 dropped Python 3.10 but has security fixes. Split so 3.11+ can pick up
# the security fixes.
#
# FIXME: Remove the split once we drop Python 3.10 support.
"Pillow>=10.0.1,<13; python_version < '3.11'",
"Pillow>=13.0.0; python_version >= '3.11'",
```
It looks like we [previously
bumped](https://github.com/element-hq/synapse/blob/4853cd4238e100183fd7b0852ddf3e8eef894658/pyproject.toml#L54-L56)
the minimum version of `Pillow` for another CVE
(https://github.com/matrix-org/synapse/pull/16347)
Although, I'm guessing we'd rather leave the minimum version alone and
only update `poetry.lock`.
I think the problem with updating the minimum is that the packagers for
Linux distros will have to deal with not having the latest `pillow`
version available and may patch `pillow` a different way.
Fixes#19583
The nginx example in `docs/reverse_proxy.md` sets `proxy_set_header Host
$host:$server_port;`. The server block listens on 443, so nginx sends
`Host: matrix.example.com:443`. With `public_baseurl:
https://matrix.example.com/`, the request URI never starts with
`public_baseurl`, and `SsoRedirectServlet` keeps redirecting to itself.
That breaks SSO even when the internal and external ports are the same.
The line came in with #17994.
This switches the example to `proxy_set_header Host $host;`, as
@anoadragon453 recommended in #18359. I added a short comment for setups
where clients use a non-default port: use `$http_host` and put that port
in `public_baseurl`. The setup in #18359 (external port 50080) needs
that variant.
I left `docker/conf-workers/nginx.conf.j2` alone. Its listener port
matches `public_baseurl`, so `$host:$server_port` is right there.
This is a docs-only change, so there are no tests.
Thanks to @shibotto for reporting this and for pointing at the fix.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
It looks like our CI is sometimes being run with Python 3.15, where
`rpds-py` doesn't have wheels for our current version.
Unsure why it's intermittent.
Run:
https://github.com/element-hq/synapse/actions/runs/39464528041/job/118772376619
```
| error: failed to run custom build command for `pyo3-ffi v0.27.2`
|
| Caused by:
| process didn't exit successfully: `/tmp/tmp72_cy3_y/rpds_py-0.30.0/target/release/build/pyo3-ffi-ae84d9db989de8cd/build-script-build` (exit status: 1)
| --- stdout
| cargo:rustc-check-cfg=cfg(Py_LIMITED_API)
| cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)
| cargo:rustc-check-cfg=cfg(PyPy)
| cargo:rustc-check-cfg=cfg(GraalPy)
| cargo:rustc-check-cfg=cfg(py_sys_config, values("Py_DEBUG", "Py_REF_DEBUG", "Py_TRACE_REFS", "COUNT_ALLOCS"))
| cargo:rustc-check-cfg=cfg(pyo3_disable_reference_pool)
| cargo:rustc-check-cfg=cfg(pyo3_leak_on_drop_without_reference_pool)
| cargo:rustc-check-cfg=cfg(Py_3_7)
| cargo:rustc-check-cfg=cfg(Py_3_8)
| cargo:rustc-check-cfg=cfg(Py_3_9)
| cargo:rustc-check-cfg=cfg(Py_3_10)
| cargo:rustc-check-cfg=cfg(Py_3_11)
| cargo:rustc-check-cfg=cfg(Py_3_12)
| cargo:rustc-check-cfg=cfg(Py_3_13)
| cargo:rustc-check-cfg=cfg(Py_3_14)
| cargo:rustc-check-cfg=cfg(Py_3_15)
| cargo:rerun-if-env-changed=PYO3_CROSS
| cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
| cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
| cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY
|
| --- stderr
| error: the configured Python version (3.15) is newer than PyO3's maximum supported version (3.14)
| = help: this package is being built with PyO3 version 0.27.2
| = help: check https://crates.io/crates/pyo3 for the latest PyO3 version available
| = help: updating this package to the latest version of PyO3 may provide compatibility with this Python version
| = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
| warning: build failed, waiting for other jobs to finish...
| 💥 maturin failed
| Caused by: Failed to build a native library through cargo
| Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.15-64bit" PYO3_PYTHON="/tmp/tmpy90bal2g/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmpy90bal2g/.venv/bin/python" "cargo" "rustc" "--profile" "release" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmp72_cy3_y/rpds_py-0.30.0/Cargo.toml" "--lib"`
| Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmpy90bal2g/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with rpds-py (0.30.0)
- not supporting PEP 517 builds
- not specifying PEP 517 build requirements correctly
- the build requirements are incompatible with your operating system or Python version
- the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "rpds-py (==0.30.0)".
PEP517 build of a dependency failed
```
```
| error: failed to run custom build command for `pyo3-ffi v0.26.0`
|
| Caused by:
| process didn't exit successfully: `/tmp/tmpy_lu888j/pydantic_core-2.41.5/target/release/build/pyo3-ffi-83252c0f9cb4430a/build-script-build` (exit status: 1)
| --- stdout
| cargo:rustc-check-cfg=cfg(Py_LIMITED_API)
| cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)
| cargo:rustc-check-cfg=cfg(PyPy)
| cargo:rustc-check-cfg=cfg(GraalPy)
| cargo:rustc-check-cfg=cfg(py_sys_config, values("Py_DEBUG", "Py_REF_DEBUG", "Py_TRACE_REFS", "COUNT_ALLOCS"))
| cargo:rustc-check-cfg=cfg(pyo3_disable_reference_pool)
| cargo:rustc-check-cfg=cfg(pyo3_leak_on_drop_without_reference_pool)
| cargo:rustc-check-cfg=cfg(Py_3_7)
| cargo:rustc-check-cfg=cfg(Py_3_8)
| cargo:rustc-check-cfg=cfg(Py_3_9)
| cargo:rustc-check-cfg=cfg(Py_3_10)
| cargo:rustc-check-cfg=cfg(Py_3_11)
| cargo:rustc-check-cfg=cfg(Py_3_12)
| cargo:rustc-check-cfg=cfg(Py_3_13)
| cargo:rustc-check-cfg=cfg(Py_3_14)
| cargo:rustc-check-cfg=cfg(Py_3_15)
| cargo:rerun-if-env-changed=PYO3_CROSS
| cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
| cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
| cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY
|
| --- stderr
| error: the configured Python interpreter version (3.15) is newer than PyO3's maximum supported version (3.14)
| = help: please check if an updated version of PyO3 is available. Current version: 0.26.0
| = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
| warning: build failed, waiting for other jobs to finish...
| 💥 maturin failed
| Caused by: Failed to build a native library through cargo
| Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.15-64bit" PYO3_PYTHON="/tmp/tmprci5om5_/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmprci5om5_/.venv/bin/python" "cargo" "rustc" "--profile" "release" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmpy_lu888j/pydantic_core-2.41.5/Cargo.toml" "--lib" "--crate-type" "cdylib"`
| Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmprci5om5_/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with pydantic-core (2.41.5)
- not supporting PEP 517 builds
- not specifying PEP 517 build requirements correctly
- the build requirements are incompatible with your operating system or Python version
- the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "pydantic-core (==2.41.5)".
```
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Lays the two foundations the Rust DBAPI2 backend needs before any
connection code: a Python↔Postgres value-mapping layer, and the helpers
that drive async work from sync Python.
Value mapping:
- `PgValue` — enum that implements `tokio_postgres::types::ToSql`,
encoding into Postgres' binary wire format per the prepared-statement
column type (so a single Python `int` becomes INT2/INT4/INT8 with range
checks).
- `PythonPgFromSql` — the decode counterpart, turning wire bytes back
into Python objects (SQL NULL → `None`).
- Covers int / float / bool / str / bytes / None; lists etc are left to
a follow-up.
Blocking helpers:
- `block_on` / `block_on_pg_result` / `block_on_next` drive
`tokio-postgres` futures to completion from sync, GIL-holding Python
methods, releasing the GIL for the wait.
Both modules are `pub` for now so not-yet-consumed items don't trip
clippy's `dead_code` lint; later PRs in the stack tighten visibility.
Everything here is unit-tested with no live Postgres server.
## What the MSC says
From MSC4140's [Auth rule
evaluation](https://github.com/matrix-org/matrix-spec-proposals/blob/b7cb09664fed684388ba656459d08cfaf786c1d1/proposals/4140-delayed-events-futures.md#auth-rule-evaluation):
> The server SHOULD also check the event could be sent when it is
scheduled and reject the request if not. In particular, servers SHOULD
check the sender is in the room (or otherwise authorized to send the
event) to avoid storing events for rooms the user is not in.
## This PR
- Checks the event against the room's auth rules as if it were sent now,
and returns the same 403 `M_FORBIDDEN` that `/send` and `/state` would.
- Changes to the sender's own membership are exempt: a join or knock can
be sent from outside the room, which is how this PR reads the MSC's "or
otherwise authorized". They are checked by the auth rules when they are
sent.
Here are a few examples:
| Delayed event | Before | After |
| ----------------------------------------------------------------- |
--------------------------------------- |
---------------------------------------- |
| `m.room.message` from outside the room | 200 `delay_id`, fails at send
time | 403 `M_FORBIDDEN` |
| Kick of another member from outside the room | 200 `delay_id`, fails
at send time | 403 `M_FORBIDDEN` |
| `m.room.name` from a member whose power level is too low | 200
`delay_id`, fails at send time | 403 `M_FORBIDDEN` |
| **Exception**: join of a public room from outside it | 200 `delay_id`,
joins at send time | unchanged |
| Join from outside the room, then a message due after it | both
accepted, message sent once joined | join accepted, message 403
`M_FORBIDDEN` |
Part of: MSC4354 (Sticky Events, Experimental Feature tracking issue
#19409)
When a user is the sender of a sticky event that comes down sync, that
user is now in scope for having their lazy-loaded membership sent down
in sync.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Hugh Nimmo-Smith