A user who left a configured server announcement room, such as a sender
removed from its config, is now treated like a user who has never been
in it. Since the room is world-readable, they read its current history
and state instead of the room as it was when they left.
Add the server announcements store to workers, which now look rooms up
in it when checking room access.
Run the history tests for a former announcement sender who was removed
from the room before the announcement was sent. /messages and
/rooms/{roomId}/initialSync fail for them: their leave membership limits
what they can read to the room as it was when they left.
A local user who has never been in an announcement room can read it
through /messages, /context, /event, /relations, /threads and
/rooms/{roomId}/initialSync, because the room is world-readable.
This includes the following:
- Implements ['Implicit Forward
Pagination'](https://github.com/matrix-org/matrix-spec/issues/2378).
Remove the extension's `since` and `next_batch` fields. The position is
now tracked via the global `pos`, with per-room connection state
recording how far each room has been sent so that `limit` paginates
without holding `pos` back and lost responses are re-delivered.
- Discussed in
https://github.com/matrix-org/matrix-spec-proposals/pull/4480#discussion_r4165887827
- Honour the `lists` and `rooms` fields.
- Send all unexpired sticky events when a room enters scope for the
first time or is newly joined, and catch up on events sent while a room
was out of scope when it re-enters.
- Enabling the extension partway through a connection returns all
unexpired sticky events for in-scope rooms, and re-enabling it after
disabling it returns the unexpired sticky events sent in the meantime.
I tried to split this up into standalone commits though the bulk of the
change is in the third commit.
---------
Co-authored-by: Olivier 'reivilibre <git.oo@emunest.net>
The problem is that while `pillow@12.3.0` supports Python >= 3.10,
`pillow@13.0.0` only supports Python > 3.11, and we haven't dropped
support for Python 3.10 yet (tracked by
https://github.com/element-hq/synapse/issues/20041).
`pillow@13.0.0` includes some important security fixes:
https://pillow.readthedocs.io/en/stable/releasenotes/13.0.0.html
### How was this change made?
1. `poetry update pillow`
1. Manually change the `"pillow"` `markers` in `poetry.lock` to split at
Python 3.11 instead of 3.14
Please note this is not a durable change. `poetry` will try to revert
this any time we try to make a change (requires diligence to make sure
we keep changing it back).
This maintenance burden will disappear once we drop Python 3.10 support
(tracked by https://github.com/element-hq/synapse/issues/20041).
### Alternatives
If we were up for updating the minimum version of `pillow` in
`pyproject.toml`, we could do something like the following which would
resolve in `poetry.lock` durably how we want,
`pyproject.toml`
```
# Pillow 13 dropped Python 3.10 but has security fixes. Split so 3.11+ can pick up
# the security fixes.
#
# FIXME: Remove the split once we drop Python 3.10 support.
"Pillow>=10.0.1,<13; python_version < '3.11'",
"Pillow>=13.0.0; python_version >= '3.11'",
```
It looks like we [previously
bumped](https://github.com/element-hq/synapse/blob/4853cd4238e100183fd7b0852ddf3e8eef894658/pyproject.toml#L54-L56)
the minimum version of `Pillow` for another CVE
(https://github.com/matrix-org/synapse/pull/16347)
Although, I'm guessing we'd rather leave the minimum version alone and
only update `poetry.lock`.
I think the problem with updating the minimum is that the packagers for
Linux distros will have to deal with not having the latest `pillow`
version available and may patch `pillow` a different way.
The problem is that while `pillow@12.3.0` supports Python >= 3.10,
`pillow@13.0.0` only supports Python > 3.11, and we haven't dropped
support for Python 3.10 yet (tracked by
https://github.com/element-hq/synapse/issues/20041).
`pillow@13.0.0` includes some important security fixes:
https://pillow.readthedocs.io/en/stable/releasenotes/13.0.0.html
### How was this change made?
1. `poetry update pillow`
1. Manually change the `"pillow"` `markers` in `poetry.lock` to split at
Python 3.11 instead of 3.14
Please note this is not a durable change. `poetry` will try to revert
this any time we try to make a change (requires diligence to make sure
we keep changing it back).
This maintenance burden will disappear once we drop Python 3.10 support
(tracked by https://github.com/element-hq/synapse/issues/20041).
### Alternatives
If we were up for updating the minimum version of `pillow` in
`pyproject.toml`, we could do something like the following which would
resolve in `poetry.lock` durably how we want,
`pyproject.toml`
```
# Pillow 13 dropped Python 3.10 but has security fixes. Split so 3.11+ can pick up
# the security fixes.
#
# FIXME: Remove the split once we drop Python 3.10 support.
"Pillow>=10.0.1,<13; python_version < '3.11'",
"Pillow>=13.0.0; python_version >= '3.11'",
```
It looks like we [previously
bumped](https://github.com/element-hq/synapse/blob/4853cd4238e100183fd7b0852ddf3e8eef894658/pyproject.toml#L54-L56)
the minimum version of `Pillow` for another CVE
(https://github.com/matrix-org/synapse/pull/16347)
Although, I'm guessing we'd rather leave the minimum version alone and
only update `poetry.lock`.
I think the problem with updating the minimum is that the packagers for
Linux distros will have to deal with not having the latest `pillow`
version available and may patch `pillow` a different way.
Fixes#19583
The nginx example in `docs/reverse_proxy.md` sets `proxy_set_header Host
$host:$server_port;`. The server block listens on 443, so nginx sends
`Host: matrix.example.com:443`. With `public_baseurl:
https://matrix.example.com/`, the request URI never starts with
`public_baseurl`, and `SsoRedirectServlet` keeps redirecting to itself.
That breaks SSO even when the internal and external ports are the same.
The line came in with #17994.
This switches the example to `proxy_set_header Host $host;`, as
@anoadragon453 recommended in #18359. I added a short comment for setups
where clients use a non-default port: use `$http_host` and put that port
in `public_baseurl`. The setup in #18359 (external port 50080) needs
that variant.
I left `docker/conf-workers/nginx.conf.j2` alone. Its listener port
matches `public_baseurl`, so `$host:$server_port` is right there.
This is a docs-only change, so there are no tests.
Thanks to @shibotto for reporting this and for pointing at the fix.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
It looks like our CI is sometimes being run with Python 3.15, where
`rpds-py` doesn't have wheels for our current version.
Unsure why it's intermittent.
Run:
https://github.com/element-hq/synapse/actions/runs/39464528041/job/118772376619
```
| error: failed to run custom build command for `pyo3-ffi v0.27.2`
|
| Caused by:
| process didn't exit successfully: `/tmp/tmp72_cy3_y/rpds_py-0.30.0/target/release/build/pyo3-ffi-ae84d9db989de8cd/build-script-build` (exit status: 1)
| --- stdout
| cargo:rustc-check-cfg=cfg(Py_LIMITED_API)
| cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)
| cargo:rustc-check-cfg=cfg(PyPy)
| cargo:rustc-check-cfg=cfg(GraalPy)
| cargo:rustc-check-cfg=cfg(py_sys_config, values("Py_DEBUG", "Py_REF_DEBUG", "Py_TRACE_REFS", "COUNT_ALLOCS"))
| cargo:rustc-check-cfg=cfg(pyo3_disable_reference_pool)
| cargo:rustc-check-cfg=cfg(pyo3_leak_on_drop_without_reference_pool)
| cargo:rustc-check-cfg=cfg(Py_3_7)
| cargo:rustc-check-cfg=cfg(Py_3_8)
| cargo:rustc-check-cfg=cfg(Py_3_9)
| cargo:rustc-check-cfg=cfg(Py_3_10)
| cargo:rustc-check-cfg=cfg(Py_3_11)
| cargo:rustc-check-cfg=cfg(Py_3_12)
| cargo:rustc-check-cfg=cfg(Py_3_13)
| cargo:rustc-check-cfg=cfg(Py_3_14)
| cargo:rustc-check-cfg=cfg(Py_3_15)
| cargo:rerun-if-env-changed=PYO3_CROSS
| cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
| cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
| cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY
|
| --- stderr
| error: the configured Python version (3.15) is newer than PyO3's maximum supported version (3.14)
| = help: this package is being built with PyO3 version 0.27.2
| = help: check https://crates.io/crates/pyo3 for the latest PyO3 version available
| = help: updating this package to the latest version of PyO3 may provide compatibility with this Python version
| = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
| warning: build failed, waiting for other jobs to finish...
| 💥 maturin failed
| Caused by: Failed to build a native library through cargo
| Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.15-64bit" PYO3_PYTHON="/tmp/tmpy90bal2g/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmpy90bal2g/.venv/bin/python" "cargo" "rustc" "--profile" "release" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmp72_cy3_y/rpds_py-0.30.0/Cargo.toml" "--lib"`
| Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmpy90bal2g/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with rpds-py (0.30.0)
- not supporting PEP 517 builds
- not specifying PEP 517 build requirements correctly
- the build requirements are incompatible with your operating system or Python version
- the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "rpds-py (==0.30.0)".
PEP517 build of a dependency failed
```
```
| error: failed to run custom build command for `pyo3-ffi v0.26.0`
|
| Caused by:
| process didn't exit successfully: `/tmp/tmpy_lu888j/pydantic_core-2.41.5/target/release/build/pyo3-ffi-83252c0f9cb4430a/build-script-build` (exit status: 1)
| --- stdout
| cargo:rustc-check-cfg=cfg(Py_LIMITED_API)
| cargo:rustc-check-cfg=cfg(Py_GIL_DISABLED)
| cargo:rustc-check-cfg=cfg(PyPy)
| cargo:rustc-check-cfg=cfg(GraalPy)
| cargo:rustc-check-cfg=cfg(py_sys_config, values("Py_DEBUG", "Py_REF_DEBUG", "Py_TRACE_REFS", "COUNT_ALLOCS"))
| cargo:rustc-check-cfg=cfg(pyo3_disable_reference_pool)
| cargo:rustc-check-cfg=cfg(pyo3_leak_on_drop_without_reference_pool)
| cargo:rustc-check-cfg=cfg(Py_3_7)
| cargo:rustc-check-cfg=cfg(Py_3_8)
| cargo:rustc-check-cfg=cfg(Py_3_9)
| cargo:rustc-check-cfg=cfg(Py_3_10)
| cargo:rustc-check-cfg=cfg(Py_3_11)
| cargo:rustc-check-cfg=cfg(Py_3_12)
| cargo:rustc-check-cfg=cfg(Py_3_13)
| cargo:rustc-check-cfg=cfg(Py_3_14)
| cargo:rustc-check-cfg=cfg(Py_3_15)
| cargo:rerun-if-env-changed=PYO3_CROSS
| cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
| cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
| cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
| cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY
|
| --- stderr
| error: the configured Python interpreter version (3.15) is newer than PyO3's maximum supported version (3.14)
| = help: please check if an updated version of PyO3 is available. Current version: 0.26.0
| = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
| warning: build failed, waiting for other jobs to finish...
| 💥 maturin failed
| Caused by: Failed to build a native library through cargo
| Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_BUILD_EXTENSION_MODULE="1" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.15-64bit" PYO3_PYTHON="/tmp/tmprci5om5_/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmprci5om5_/.venv/bin/python" "cargo" "rustc" "--profile" "release" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmpy_lu888j/pydantic_core-2.41.5/Cargo.toml" "--lib" "--crate-type" "cdylib"`
| Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmprci5om5_/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with pydantic-core (2.41.5)
- not supporting PEP 517 builds
- not specifying PEP 517 build requirements correctly
- the build requirements are incompatible with your operating system or Python version
- the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "pydantic-core (==2.41.5)".
```
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Lays the two foundations the Rust DBAPI2 backend needs before any
connection code: a Python↔Postgres value-mapping layer, and the helpers
that drive async work from sync Python.
Value mapping:
- `PgValue` — enum that implements `tokio_postgres::types::ToSql`,
encoding into Postgres' binary wire format per the prepared-statement
column type (so a single Python `int` becomes INT2/INT4/INT8 with range
checks).
- `PythonPgFromSql` — the decode counterpart, turning wire bytes back
into Python objects (SQL NULL → `None`).
- Covers int / float / bool / str / bytes / None; lists etc are left to
a follow-up.
Blocking helpers:
- `block_on` / `block_on_pg_result` / `block_on_next` drive
`tokio-postgres` futures to completion from sync, GIL-holding Python
methods, releasing the GIL for the wait.
Both modules are `pub` for now so not-yet-consumed items don't trip
clippy's `dead_code` lint; later PRs in the stack tighten visibility.
Everything here is unit-tested with no live Postgres server.
Paul Chobert