This moves the reference counting from PyO3 into standard Rust types,
allowing the class to be used natively from Rust without needing a
Python runtime.
Follows: #19365
Part of: MSC4354 Sticky Events (experimental feature #19409)
This PR introduces a `spam_checker_spammy` flag, analogous to
`policy_server_spammy`, as an explicit flag
that an event was decided to be spammy by a spam-checker module.
The original Sticky Events PR (#18968) just reused
`policy_server_spammy`, but it didn't sit right with me
because we (at least appear to be experimenting with features that)
allow users to opt-in to seeing
`policy_server_spammy` events (presumably for moderation purposes).
Keeping these flags separate felt best, therefore.
As for why we need this flag: soon soft-failed status won't be
permanent, at least for sticky events.
The spam checker modules currently work by making events soft-failed.
We want to prevent spammy events from getting
reconsidered/un-soft-failed, so it seems like we need
a flag to track spam-checker spamminess *separately* from soft-failed.
Should be commit-by-commit friendly, but is also small.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
The Rust port of `KNOWN_ROOM_VERSIONS` (#19589) made `__contains__`
strict about key types, raising `TypeError` when called with `None`
instead of returning `False` like a Python dict would.
This broke `/sync` for rooms with a NULL `room_version` in the database.
```
File "/home/synapse/src/synapse/handlers/sync.py", line 2628, in _get_room_changes_for_initial_sync
if event.room_version_id not in KNOWN_ROOM_VERSIONS:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument 'key': 'NoneType' object cannot be cast as 'str'
```
# What is done?
- resolves#19403
- Adds build-time Rust compiler detection and captures the rustc
--version value during the build.
- Exposes the captured compiler version from the Rust extension via a
new Python-callable function.
- Exports a new Prometheus metric for rustc version.
# How to test?
- compile `poetry install`
- add `enable_metrics: true` and
```yaml
resources:
- compress: false
names:
- client
- federation
- metrics
```
to homeserver.yaml
- start synapse
- find the rustc version at `http://localhost:8008/_synapse/metrics`
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Quentin Gliech <quenting@element.io>
This is a simplification so that `unsigned` only includes "simple"
values, to make it easier to port to Rust.
Reviewable commit-by-commit
Summary:
1. **Add `recheck` column to `redactions` table**
A new boolean `recheck` column (default true) is added to the
`redactions` table. This captures whether a redaction needs its sender
domain checked at read time — required for room v3+ where redactions are
accepted speculatively and later validated. When persisting a new
redaction, `recheck` is set directly from
`event.internal_metadata.need_to_check_redaction()`.
It's fine if initially we recheck all redactions, as it only results in
a little more CPU overhead (as we always pull out the redaction event
regardless).
2. **Backfill `recheck` via background update**
A background update (`redactions_recheck`) backfills the new column for
existing rows by reading `recheck_redaction` from each event's
`internal_metadata` JSON. This avoids loading full event objects by
reading `event_json` directly via a SQL JOIN.
3. **Don't fetch confirmed redaction events from the DB**
Previously, when loading events, Synapse recursively fetched all
redaction events regardless of whether they needed domain rechecking.
Now `_fetch_event_rows` reads the `recheck` column and splits redactions
into two lists:
- `unconfirmed_redactions` — need fetching and domain validation
- `confirmed_redactions` — already validated, applied directly without
fetching the event
This avoids unnecessary DB reads for the common case of
already-confirmed redactions.
4. **Move `redacted_because` population to `EventClientSerializer`**
Previously, `redacted_because` (the full redaction event object) was
stored in `event.unsigned` at DB fetch time, coupling storage-layer code
to client serialization concerns. This is removed from
`_maybe_redact_event_row` and moved into
`EventClientSerializer.serialize_event`, which fetches the redaction
event on demand. The storage layer now only sets
`unsigned["redacted_by"]` (the redaction event ID).
5. **Always use `EventClientSerializer`**
The standalone `serialize_event` function was made private
(`_serialize_event`). All external callers — `rest/client/room.py`,
`rest/admin/events.py, appservice/api.py`, and `tests` — were updated to
use `EventClientSerializer.serialize_event` / `serialize_events`,
ensuring
`redacted_because` is always populated correctly via the serializer.
6. **Batch-fetch redaction events in `serialize_events`**
`serialize_events` now collects all `redacted_by` IDs from the event
batch upfront and fetches them in a single `get_events` call, passing
the result as a `redaction_map` to each `serialize_event` call. This
reduces N individual DB round-trips to one when serializing a batch of
events that includes redacted events.
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Principally so that we can share the same room version configuration
between Python and Rust.
For the most part, this is a direct port. Some special handling has had
to go into `KNOWN_ROOM_VERSIONS` so that it can be sensibly shared
between Python and Rust, since we do update it during config parsing.
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This allows the Rust HTTP client to be configured to force HTTP/2 even
on plaintext connections. This is useful in contexts where the remote
server is known to server HTTP/2 over plain text.
Added because we use the Synapse Rust HTTP client with the Synapse Pro
`event-cache` module. We use this because it's independent from the
Python reactor which makes things slower than expected.
Currently, the Synapse Rust HTTP client uses HTTP/1 which means a new
connection for every request. With HTTP/2, we can share the connection
across requests.
We want to see if this will make a performance difference and less
stress on the database connection situation, see
https://github.com/element-hq/synapse-rust-apps/issues/452#issuecomment-3897717599
Here is the sibling PR for using HTTP/2 on the Synapse Pro `event-cache`
module side: https://github.com/element-hq/synapse-pro-modules/pull/35
Hello,
I'm writing on behalf of the Citadel product developed by ERCOM.
This PR bumps `pyo3` from 0.26.0 to 0.27.2 and `pythonize` from 0.26.0
to 0.27.0.
For the code migration I followed the guide found here:
[link](https://pyo3.rs/v0.27.0/migration.html).
This changes the arguments in clock functions to be `Duration` and
converts call sites and constants into `Duration`. There are still some
more functions around that should be converted (e.g.
`timeout_deferred`), but we leave that to another PR.
We also changes `.as_secs()` to return a float, as the rounding broke
things subtly. The only reason to keep it (its the same as
`timedelta.total_seconds()`) is for symmetry with `as_millis()`.
Follows on from https://github.com/element-hq/synapse/pull/19223
Erik Johnston