Follows: #19468
The main change is from this comment
https://github.com/element-hq/synapse/pull/19468#discussion_r2810364196
I am pretty sure it's safe and was tempted to add it to that PR, but for
easier bisection and reversion in case it goes wrong, thought a separate
commit would be the best.
The other drive-by change is a boolean logic simplification
Simplify condition (boolean equivalence)
Don't fetch name state from `meta_room_state` since it's no longer used
there
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
As per the spec, a room with m.room.name value that is absent, null or
empty should be treated as if there is no m.room.name event at all:
https://spec.matrix.org/v1.17/client-server-api/#mroomname
This fetches the full m.room.name event and checks the content.name
instead of only checking the existence of the m.room.name event. This
results in correctly sending heroes for those rooms.
Fixes: https://github.com/element-hq/synapse/issues/19447
Signed-off-by: Joe Groocock <me@frebib.net>
when an access token had a refresh token associated to it in the
database, deleting this refresh token (for example when deleting the
device using it) would cascade delete the access token, which wouldn't
be returned by the sql query that was supposed to delete it on its own,
and an empty array was passed to the cache invalidation function.
Fixes: #19689
# What
This PR fixes a bug I found when I run synapse (from dockerhub) and
register a `check_event_allowed` callback and my client makes use of the
mentions field in messages (`cinny:latest`). The bug doesn't appear when
the `check_event_allowed` callback is not loaded.
After some digging I noticed that the current validation of the mentions
doesn't work when an event has been frozen with `event.freeze()`. For
the messages this seems to happen when a the `check_event_allowed` is
registered (but not otherwise), see [where the event is frozen for
check_event_allowed
callback](https://github.com/element-hq/synapse/blob/b0fc0b7a612a42e6f15b87dee2a1db4c383645fb/synapse/module_api/callbacks/third_party_event_rules_callbacks.py#L289)
and [where the validation function is
called](https://github.com/element-hq/synapse/blob/b0fc0b7a612a42e6f15b87dee2a1db4c383645fb/synapse/handlers/message.py#L1404).
To have a minimal reproduction example, the following scripts fails on
`develop` but succeeds in this branch:
``` python
from synapse.api.room_versions import RoomVersions
from synapse.events import EventBase, make_event_from_dict
from synapse.events.validator import EventValidator
from tests.utils import default_config
def make_message_event(content: dict) -> EventBase:
return make_event_from_dict(
{
"room_id": "!room:test",
"type": "m.room.message",
"sender": "@alice:test",
"content": content,
"auth_events": [],
"prev_events": [],
"hashes": {"sha256": "aGVsbG8="},
"signatures": {},
"depth": 1,
"origin_server_ts": 1000,
},
room_version=RoomVersions.V9,
)
event = make_message_event(
{
"msgtype": "m.text",
"body": "@moderator:example.com hello",
"m.mentions": {"user_ids": ["@moderator:jailbreak-challenge.aqtiveguard.com"]},
}
)
EventValidator().validate_new(event, default_config) # Ok
event.freeze()
EventValidator().validate_new(event, default_config) # throws
# pydantic_core._pydantic_core.ValidationError: 1 validation error for Mentions
# Input should be a valid dictionary or instance of Mentions [type=model_type, input_value=immutabledict({'user_ids'...nge.aqtiveguard.com',)}), input_type=immutabledict]
# For further information visit https://errors.pydantic.dev/2.12/v/model_type
```
# How
I made the validation logic also validate the transformation performed
by the freezing process, namely:
- `immutabledict` validates as `dict`. (was already implemented for
POWER_LEVELS)
- `tuple` validates as array (added this to the validator in this PR).
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier 'reivilibre <oliverw@matrix.org>
This isn't fixing any particular issue. It's just a follow-up I thought
about after merging https://github.com/element-hq/synapse/pull/19611
since we're now also dealing with backfill points in the nearby range
ahead of the `current_depth`. And it's possible that the previous sort
could bias to all nearby backfill points ahead of the `current_depth`
that don't extend into the visible window of events we're paginating
through.
Added details how synapse syncs the picture claim when
update_profile_information setting is true. Addresses #17836
---------
Co-authored-by: Michael Hlas <3398654+mhlas7@users.noreply.github.com>
When we port the `Event` class to Rust, the constructor will check for
the existence of required fields. To support that, we tidy up the test
code where we construct fake events to add all the required fields.
There should be no behavioural changes.
Review commit-by-commit.
The reason for the change is to make it easier to support these checks
when porting event class to Rust.
Previously, code that needed to access `prev_state_events` had to
combine a `room_version.msc4242_state_dags` boolean check with an
`isinstance(event, FrozenEventVMSC4242)` cast (or `cast()`) for the type
checker. Introduce `supports_msc4242_state_dag()` in a new
`synapse/events/py_protocol.py` which does both in one step via
`TypeIs[MSC4242Event]`, removing the need to import the concrete
`FrozenEventVMSC4242` class at every call site.
`MSC4242Event` is an `EventBase` subclass used purely for type narrowing
— it's marked with a metaclass that rejects `isinstance()` to make
accidental runtime use loud.
No behavioural change: callers continue to gate on the same room version
flag and access the same `prev_state_events` attribute.
This is in prep for using the room versions more from Rust.
Main changes:
- Change it so each room version is defined as a delta to the last one.
This is a cosmetic change that makes it easier to ensure the room
version definitions are correct (as they're defined as deltas from
previous versions).
- Move constants to `RoomVersion` constants, like `RoomVersion::V1`, for
convenience.
- Change visibility of various attributes.
Convert `prev_state_events` to use `StrCollection` rather than requiring
it to be a mutable list. None of the usages require it to be a proper
list, and besides, events are immutable and therefore so should
`event.prev_state_events`.
Based on #19708.
This is on the path to porting the entire event class to Rust, as
`event.content` will then return the new Rust class `JsonObject`.
This PR adds a pure Rust `JsonObject` class that is a `Mapping`
representing a json-style object. It uses `serde_json::Value` as its
in-memory representation and `pythonize` for conversion when a field is
looked up on the object.
I'm not thrilled with the name, but couldn't think of a better one.
This also adds `JsonObject` handling to the JSON serialisation functions
we use, as well as to the `freeze(..)` function.
Reviewable commit-by-commit.
Better to retry more quickly than have workers wait around. 5 seconds is
still a reasonable gap in time to not overwhelm anything.
This matters most in cross-worker scenarios. When locks are on the same
worker, when the lock holder releases, we signal to other locks (with
the same name/key) that they should try reacquiring the lock
immediately. But locks on other workers only re-check based on their
retry `_timeout_interval`.
Updating to 5 seconds to match the previous intentions based on the
[flawed
code](https://github.com/element-hq/synapse/blob/6100f6e4f7fb0c72f1ae2802683ebc811c0e3a77/synapse/handlers/worker_lock.py#L278).
We can assume they were trying to have 5 seconds as the max value to
retry.
Spawning from
https://github.com/element-hq/synapse/pull/19394#discussion_r3168458070
Similar to #19706, let's port the `unsigned` field into a Rust class.
This does change things a bit in that we now define exactly what
unsigned fields that are allowed to be added to an event, and what
actually gets persisted. This should be a noop though, as we carefully
filter out what unsigned fields we allow in from federation, for example
As a side effect of this cleanup, I think this fixes handling
`unsigned.age` on events received over federation.
This is another stepping stone in porting the event class fully to Rust.
The new `Signatures` class is relatively simple, as we actually don't
interact with it that much in the code. It does *not* implement
`Mapping` or `MutableMapping` as that takes quite a lot of effort that
we don't need, even though it would be more ergonomic.
Fixes the symptoms of https://github.com/element-hq/synapse/issues/19315
/ https://github.com/element-hq/synapse/issues/19588 but not the
underlying reason causing the number to grow so large in the first
place.
```
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
```
Copied from the original pull request on [Famedly's Synapse
repo](https://github.com/famedly/synapse/pull/221) (with some edits):
Basing the time interval around a 5 seconds leaves a big window of
waiting especially as this window is doubled each retry, when another
worker could be making progress but can not.
Right now, the retry interval in seconds looks like `[0.2, 5, 10, 20,
40, 80, 160, 320, (continues to double)]` after which logging should
start about excessive times and (relatively quickly) end up with an
extremely large retry interval with an unrealistic expectation past the
heat death of the universe. 1 year in seconds = 31,536,000.
With this change, retry intervals in seconds should look more like:
```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
60, < never goes higher than this
]
```
Logging about excessive wait times will start at 10 minutes.
<details>
<summary>Previous breakdown when we were using 15 minutes</summary>
```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
102.4, # 1.7 minutes
204.8, # 3.41 minutes
409.6, # 6.83 minutes
819.2, # 13.65 minutes < logging about excessive times will start here, 13th iteration
900, # 15 minutes < never goes higher than this
]
```
</details>
Further suggested work in this area could be to define the cap, the
retry interval starting point and the multiplier depending on how
frequently this lock should be checked. See data below for reasons why.
Increasing the jitter range may also be a good idea
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
This comes from
https://github.com/erikjohnston/rust-signed-json/blob/main/src/json.rs.
We need to be able to serialise canonical JSON in Rust to be able to
calculate event IDs once we port the event class to Rust.
We could instead make the above a properly published crate, but feels
easier to pull it into Synapse utils.
When upgrading a room to v12, we accidentally ended up mutating the
content of the old power level. Since we cache events, this meant any
future usage of the old power level event would see the wrong content
(until it dropped from the cache).
This meant that the creator of the new room would not be able to perform
admin actions on the old room. Any federation requests for the event
would fail the hash checks, since the content had been changed.
All in all, quite a nasty bug.
MSC3266 is merged in v1.15, let's stabilize it as part of #18731
1. Add support for the stable `/_matrix/client/v1/room_summary/`
endpoint, keeping both unstable endpoints for compat
2. Remove the experimental `msc3266_enabled` flag
This fixes the bug described in #19713 (and double-checked against the
SDK integration test, which now passes with this change). A sync
response must be returned immediately if a room subscription
configuration change caused a new non-empty response (checked with `if
response` in the code) to be produced.
Fixes#19713.
Fixes#18844.
---------
Co-authored-by: Erik Johnston <erik@matrix.org>
Currently synapse returns `M_FORBIDDEN` when trying to use the account
deactivation API, if the server admin disabled displayname changes. This
is undesirable, since it prevents GDPR erasure without admin
interaction. The admin API seems to work fine though. This also only
seems to affect the deactivate API, when the erase flag is true.
Relevant endpoint:
https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3accountdeactivate
This change only removes the checked for condition that the displayname
and profile avatar are allowed to be changed per the configuration
setting. If a user is deleting themselves, why is that denied?
There did not seem to be a basic test for this endpoint that checks the
`erase` usage, so that was added as well as checking the above mentioned
behavior.
Both `__getitem__` and `.user_id` were removed in #19680 to simplify the
event class. However, `EventBase` is exposed to modules who might also
make use of those methods, so let's reinstate them (but otherwise not
reinstate the usage of them in the code).
Follows on from #19473.
We should be recording where we have deleted up to in the same
transaction as we perform the delete, rather than at the end. This code
only starts deleting rows after a month (and the original PR isn't in a
release yet), so no server should have run into this problem yet.
Also let's log more regularly, as the initial set of deletions will
likely take a long time.
Fixes#13043
The usages of the table mostly already correctly handled if we don't
have old entries, as that was needed when we first added the table.
I arbitrarily set the prune time to 30 days. The only use for old
entries is for sync streams that haven't synced since then, and we
should very rarely see sync streams that haven't been used in 30 days.
Reviewable commit-by-commit.
---------
Co-authored-by: Olivier 'reivilibre' <oliverw@element.io>
Co-authored-by: Olivier 'reivilibre' <olivier@librepush.net>
shcherbak