Bringing it up to parity with the other 404 StoreErrors naming the table
name already.
More helpful response when debugging issues with incomplete relations.
Concretly: a user that got partly removed and then reinstated, missing
an entry in 'profiles' table.
This should also contribute to a better understanding of #2807 and #2173
Not aware of an open ticket for this.
I came across it when I accidentally broke the feature even more (as
part of another piece of work),
then discovered there weren't tests for this.
So this is overall a low-priority drive-by fix.
Requires a fix to SyTest https://github.com/matrix-org/sytest/pull/1426
(as it depended on the bug).
<ol>
<li>
Add a test for purging rooms with `delete_local_events=False` \
Parameterised by room version, this test currently succeeds
on v2 but fails on v12.
This is because the condition checking for local events relies
on the old event ID format, which has not been used since v2.
</li>
<li>
Fix delete_local_events=False for room versions above v2 \
The event ID format changing means that we have to rely on `sender`
to know the origin of an event
</li>
</ol>
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
When I run `ruff` locally, either `ruff` or one of the linting scripts
adds `.ruff_cache` to the `.gitignore` file. This PR adds that line so
that running linters doesn't result in a dirty git working tree (and to
ignore ruff's cache, of course).
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
Fixes: #19844
Concretely, this changes `ResponseCache` to unset cache entries once
they resolve to a `Failure`.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
Running `scripts-dev/lint.sh` locally revealed a failure, so here are the fixes that the linter
applied + a manual change to address an issue that couldn't be
auto-fixed.
The Complement integration-test job prints the full raw `go test -json`
stream straight to the GitHub Actions build log. That's an enormous,
unreadable wall of JSON, and the GitHub web UI renders very large logs
poorly — making the run hard to view and slow to load.
Instead, let's store the raw JSON and upload it as an artefact. We still
render failing test output as before.
We will still stream when tests finish (i.e. PASS / FAIL) so that people
can see that things are actually running. However, all other output
(such as logs) are hidden.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
When building the v1.155.0rc1, the release notification said the
workflow was successful
but it had actually failed.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
The sytest `After /purge_history users still get pushed for new
messages` is flaky. The flakiness exposes a real bug rather than a
test-timing issue.
Notification counts are stored in two places: `event_push_actions` (one
row per unread event) and `event_push_summary` (aggregate counts
populated periodically by `_rotate_notifs`, which runs on a 30-second
timer). `_purge_history_txn` deletes the purged events' rows from
`event_push_actions` but never adjusts `event_push_summary` (only the
full-room `purge_room` drops that table).
So the result depends on a race: if rotation hasn't fired, counts come
live from `event_push_actions`, the purge removes the right rows, and
the count is correct. If rotation fires before the purge — more likely
under the slower
multi-postgres/workers/asyncio CI config — the events get folded into
`event_push_summary`, the purge then deletes the underlying
`event_push_actions` rows but leaves the summary untouched, and the
count comes out inflated.
### Fix
Before deleting the rotated rows from `event_push_actions`, decrement
`event_push_summary` by the amount attributable to the events being
deleted. The decrement mirrors the counting logic in
`_rotate_notifs_before_txn`: only rows that were already rotated
(`stream_ordering <= event_push_summary_stream_ordering`) and that fall
after the summary's receipt are subtracted, so it stays correct in the
presence of read receipts and unread/highlight rows. The SQL avoids
`UPDATE ... FROM` and CTEs so it works on both SQLite and Postgres.
End-of-purge cache invalidation already covers
`get_unread_event_push_actions_by_room_for_user`.
### Tests
Adds `test_count_aggregation_after_purge`, which forces a rotation
before purging and asserts the aggregate count reflects only the
surviving events, covering read receipts and a subsequent re-rotation.
It fails (`3 != 1`) without the fix.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
I.e. `default_config("test")` and `default_config("test", False)` are so
opaque and hard to connect the dots with.
Spawning from trying to figure out what the `server_name` is set as for
our `HomeserverTestCase` in order to reference it in
https://github.com/element-hq/synapse/pull/19848#discussion_r3397455309
This was all done through claude.
## Fix flaky `test_edu_large_messages_not_splitting_one_user` (`TooLong`
under `trial -jN`)
### Problem
The "Build .deb packages" CI step intermittently failed with:
```
twisted.protocols.amp.TooLong
...in
tests.rest.client.test_sendtodevice.SendToDeviceTestCase.test_edu_large_messages_not_splitting_one_user
```
The deb build runs the suite with `twisted.trial -j2`. In that mode,
worker log events are shipped to the manager process over Twisted's AMP
protocol, which encodes each value with a 2-byte length prefix — so any
single log line of **64 KiB or more** raises `TooLong`.
### Root cause
Not a bug in the to-device/EDU logic, and **not** debug logging enabled
by the build (it runs at the default `ERROR` level). It's a **log-level
leak between tests sharing a `-j2` worker**:
- `tests/logging/test_loggers.py::ExplicitlyConfiguredLoggerTestCase`
calls `root_logger.setLevel(logging.DEBUG)` directly and never restores
it (no `setUp`/`tearDown`/`addCleanup`).
- When that test runs before
`test_edu_large_messages_not_splitting_one_user` **in the same worker
process**, the root logger is left at `DEBUG`.
- That test deliberately builds an EDU of exactly `SOFT_MAX_EDU_SIZE -
1` (65 535) bytes. Storing it triggers `synapse/storage/database.py`'s
`[SQL values]` DEBUG log, which dumps the full query params — producing
a **65 708-byte**
line that overflows AMP's cap.
It looks "flaky" purely because of `-j2` scheduling: whether the two
tests land on the same worker, and in what order.
### Fix
Three commits:
1. **Restore the root logger level in
`ExplicitlyConfiguredLoggerTestCase`** — `addCleanup` to put the level
back. Fixes the root cause.
2. **Truncate oversized log lines in the test log handler**
(`ToTwistedHandler.emit`) — caps lines at 1000 chars so no debug line
can break `trial -jN`, regardless of which query is logged (defense in
depth).
3. **Truncate values in `[SQL values]` debug logging** — caps the logged
param repr at 1000 chars (guarded by `isEnabledFor(DEBUG)` to keep the
hot path lazy). Keeps production debug logs sane too.
### Testing
- The reproduction (`trial -j2 tests.logging.test_loggers <the EDU
test>`) went from **~3/8 failing** to **10/10 passing**.
- Confirmed the root level is restored after the logging tests, and that
the `[SQL values]` line is now capped (~50 KB with a `[truncated]`
marker, was 65 708).
- `ruff` + `mypy` clean; `tests.logging.test_loggers` and
`tests.rest.client.test_sendtodevice` pass (14/14).
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Eric Eastwood <erice@element.io>
When building v1.155.0rc1, flaky deb builds combined with fail-fast
behaviour
meant that I had to press the retry button 5 times to get a full set.
Without fail-fast, I suspect 1 or 2 retries would have done the job.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
This is in prep for converting the event serialization to Rust.
This is a fairly mechanical port, except that we store the appservice ID
rather than the appservice object. This avoids us having to store a
`Py<..>` (or port the appservice object over).
The reason for querying this support was wanting support for SQLite's JSON operators, which are currently not present in the SQLite version found in Ubuntu's oldest supported LTS.
The JSON operators were used in some of the sticky events work (related: #19452).
Our ruling was that we should support Ubuntu oldest LTS equally to Debian oldstable, so support the oldest of the two versions from those.
That makes some kind of sense as it would be difficult to do otherwise without dropping support for that version of Ubuntu altogether, given if we kept publishing packages intended for use with Postgres, there's a risk that an innocent sysadmin would update their SQLite deployment without realising that it is no longer supported.
This was [discussed months ago
(private)](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.fcdvoc88dy5s)
and at [private](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.u48ivjge4qpt).
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
Fix#2860
Also cleans up comments around plans to define `ALLOWED_SCHEMES` as we can rely on Bleach's `ALLOWED_PROTOCOLS` defaults (`http`, `https` and `mailto`).
Knowledge from @reivilibre in
[`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$2WUfCdA02wZw-6-jhw3QbLQ44BmKJrqLuZ6wjz2r7hk?via=jki.re&via=element.io&via=matrix.org)
on 2025-12-15.
Spawning from me making Rust changes but nothing useful was printed
until I added `-v`,
```shell
$ poetry install --extras all
Installing dependencies from lock file
Package operations: 0 installs, 1 update, 0 removals
- Updating pyjwt (2.11.0 -> 2.12.0)
Installing the current project: matrix-synapse (1.154.0rc1)
Failed to install /home/eric/Documents/github/element/synapse
```
I also see `poetry run maturin develop` suggested but I'd prefer not to
need to install `maturin` as yet another system tool to manage myself.
Follow on from #19701
This (more or less) matches what we had before. Otherwise we just get a
default `<builtins.Event at 0x...>`
---------
Co-authored-by: Eric Eastwood <erice@element.io>
Follow on from https://github.com/element-hq/synapse/pull/19701.
Unfortunately serde has a bug when using `#[serde(flatten)]` with
`arbitrary-precision` feature when handling integers that fit in a i128
when doing `serde_json::from_value`. See
https://github.com/serde-rs/serde/issues/2230.
The `depythonize` hits the same issue. To fix this we make it so we only
parse events from strings and not values.
This is based on https://github.com/element-hq/synapse/pull/18416, which
got reverted (#19614) due to it incorrectly rejecting to-device messages
to users with many devices (and thus breaking message sending).
Fix https://github.com/element-hq/synapse/issues/17035
A to-device message content looks like:
```jsonc
{
"@user:domain": {"device1": {...}, "device2": {...}},
...
}
```
The previous PR would split up into multiple EDUs, each with a subset of
the users. However, if one user's entry was too large it would not
further split it up and then error out.
The main change in this PR is to allow splitting up a single user into
multiple EDUs.
Other changes:
1. Rename to `SOFT_MAX_EDU_SIZE` to indicate that we sometimes send EDUs
with larger size than that, and its more a target than a hard limit.
2. Check early if any to-device message (to a specific device) is too
large to send, even if we're not going to send it over federation. This
ensures that we catch issues where clients try to send too large
to-device.
This still means that if a client send a large individual to-device
message it will fail, but I don't believe we ever send such large
to-device messages (normally they're in the range of a few KB).
---
I ended up changing the implementation a bunch to make it easy to reuse
the code to split up dictionaries. Instead of repeatedly splitting up
the EDU until each bit fits into the size, we instead record the size of
each entry in the dict and instead split up based on cumulative size.
This means we call `encode_canonical_json` on each entry rather than
once on the entire struct, but its not significantly slower to do so.
--
cc @MatMaul @MadLittleMods
---------
Co-authored-by: Mathieu Velten <matmaul@gmail.com>
Co-authored-by: mcalinghee <mcalinghee.dev@gmail.com>
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Follow on from #19701.
Some Synapse servers may have events in their database that don't pass
the canonical JSON checks. This is bad, but we still want to be able to
load them nonetheless.
Ports the event class to Rust.
The main difference here are:
1. There is now a single event class
2. We now validate a lot more at event construction time than we
previously did (we basically checked nothing before). This required some
changes to the tests, including
https://github.com/matrix-org/sytest/pull/1423
Reviewable commit-by-commit.
### Overview of Event Rust structure
The format of the event struct in Rust is quite different than that in
Python.
The top-level looks like:
```rust
pub struct Event {
/// The parsed event JSON.
fields: FormattedEvent,
/// The event ID. For format v1 this is read directly from the JSON;
/// for v2+ it is computed from the canonical-JSON hash at
/// construction time and cached here.
event_id: Arc<str>,
/// Synapse-internal per-event state that lives outside the federated
/// JSON (e.g. outlier flag, soft-failure, stream positions).
#[pyo3(get)]
internal_metadata: EventInternalMetadata,
/// The room version this event was parsed for.
#[pyo3(get)]
room_version: &'static RoomVersion,
/// `None` for accepted events; otherwise a short reason set by auth
/// when the event was rejected.
rejected_reason: Option<Box<str>>,
}
```
which includes the actual parsed event in `FormattedEvent`, plus the
rest of the event metadata.
```rust
pub struct FormattedEvent<E = Arc<EventFormatEnum>> {
#[serde(default)]
pub signatures: Signatures,
#[serde(default)]
pub unsigned: Unsigned,
#[serde(flatten)]
pub specific_fields: E,
#[serde(flatten)]
pub common_fields: Arc<EventCommonFields>,
}
```
The struct is further split into the common fields, format specific
fields, plus the signatures and unsigned. We split out the signature and
unsigned fields as they are mutable, so when we clone the event we can
still share the common and specific fields and only copy signature and
unsigned.
The `specific_fields` are the fields that depend on the format version.
They can either be a specific format (e.g. `E = EventFormatV1`) or a
type-erased enum `EventFormatEnum` that is across all room versions:
```rust
pub enum EventFormatEnum {
V1(EventFormatV1),
V2V3(EventFormatV2V3),
V4(EventFormatV4),
VMSC4242(EventFormatVMSC4242),
}
```
For example:
```rust
/// Shared flat-list encoding of `auth_events` and `prev_events`, reused
/// by every format from v2/v3 onwards.
#[derive(Serialize, Deserialize)]
pub struct SimpleAuthPrevEvents {
pub auth_events: Vec<String>,
pub prev_events: Vec<String>,
}
/// Version-specific fields for room versions 3-10.
#[derive(Serialize, Deserialize)]
pub struct EventFormatV2V3 {
pub room_id: Box<str>,
#[serde(flatten)]
pub auth_prev_events: SimpleAuthPrevEvents,
}
```
### Dev notes
As discussed in
[`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$3gTjDO440GbAz57cXcCawwiyFLiD0crrarvS1uhzKOY?via=jki.re&via=element.io&via=matrix.org)
---------
Co-authored-by: Eric Eastwood <erice@element.io>
Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
<h2>3.12 (2026-04-21)</h2>
<ul>
<li>Update to Unicode 17.0.0.</li>
<li>Issue a deprecation warning for the transitional argument.</li>
<li>Added lazy-loading to provide some performance improvements.</li>
<li>Removed vestiges of code related to Python 2 support, including
segmentation of data structures specific to Jython.</li>
</ul>
<p>Thanks to Rodrigo Nogueira for contributions to this release.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.11...v3.15">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/element-hq/synapse/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Eric Eastwood