Follows: #19468
The main change is from this comment
https://github.com/element-hq/synapse/pull/19468#discussion_r2810364196
I am pretty sure it's safe and was tempted to add it to that PR, but for
easier bisection and reversion in case it goes wrong, thought a separate
commit would be the best.
The other drive-by change is a boolean logic simplification
Simplify condition (boolean equivalence)
Don't fetch name state from `meta_room_state` since it's no longer used
there
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
As per the spec, a room with m.room.name value that is absent, null or
empty should be treated as if there is no m.room.name event at all:
https://spec.matrix.org/v1.17/client-server-api/#mroomname
This fetches the full m.room.name event and checks the content.name
instead of only checking the existence of the m.room.name event. This
results in correctly sending heroes for those rooms.
Fixes: https://github.com/element-hq/synapse/issues/19447
Signed-off-by: Joe Groocock <me@frebib.net>
when an access token had a refresh token associated to it in the
database, deleting this refresh token (for example when deleting the
device using it) would cascade delete the access token, which wouldn't
be returned by the sql query that was supposed to delete it on its own,
and an empty array was passed to the cache invalidation function.
Fixes: #19689
# What
This PR fixes a bug I found when I run synapse (from dockerhub) and
register a `check_event_allowed` callback and my client makes use of the
mentions field in messages (`cinny:latest`). The bug doesn't appear when
the `check_event_allowed` callback is not loaded.
After some digging I noticed that the current validation of the mentions
doesn't work when an event has been frozen with `event.freeze()`. For
the messages this seems to happen when a the `check_event_allowed` is
registered (but not otherwise), see [where the event is frozen for
check_event_allowed
callback](https://github.com/element-hq/synapse/blob/b0fc0b7a612a42e6f15b87dee2a1db4c383645fb/synapse/module_api/callbacks/third_party_event_rules_callbacks.py#L289)
and [where the validation function is
called](https://github.com/element-hq/synapse/blob/b0fc0b7a612a42e6f15b87dee2a1db4c383645fb/synapse/handlers/message.py#L1404).
To have a minimal reproduction example, the following scripts fails on
`develop` but succeeds in this branch:
``` python
from synapse.api.room_versions import RoomVersions
from synapse.events import EventBase, make_event_from_dict
from synapse.events.validator import EventValidator
from tests.utils import default_config
def make_message_event(content: dict) -> EventBase:
return make_event_from_dict(
{
"room_id": "!room:test",
"type": "m.room.message",
"sender": "@alice:test",
"content": content,
"auth_events": [],
"prev_events": [],
"hashes": {"sha256": "aGVsbG8="},
"signatures": {},
"depth": 1,
"origin_server_ts": 1000,
},
room_version=RoomVersions.V9,
)
event = make_message_event(
{
"msgtype": "m.text",
"body": "@moderator:example.com hello",
"m.mentions": {"user_ids": ["@moderator:jailbreak-challenge.aqtiveguard.com"]},
}
)
EventValidator().validate_new(event, default_config) # Ok
event.freeze()
EventValidator().validate_new(event, default_config) # throws
# pydantic_core._pydantic_core.ValidationError: 1 validation error for Mentions
# Input should be a valid dictionary or instance of Mentions [type=model_type, input_value=immutabledict({'user_ids'...nge.aqtiveguard.com',)}), input_type=immutabledict]
# For further information visit https://errors.pydantic.dev/2.12/v/model_type
```
# How
I made the validation logic also validate the transformation performed
by the freezing process, namely:
- `immutabledict` validates as `dict`. (was already implemented for
POWER_LEVELS)
- `tuple` validates as array (added this to the validator in this PR).
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier 'reivilibre <oliverw@matrix.org>
This isn't fixing any particular issue. It's just a follow-up I thought
about after merging https://github.com/element-hq/synapse/pull/19611
since we're now also dealing with backfill points in the nearby range
ahead of the `current_depth`. And it's possible that the previous sort
could bias to all nearby backfill points ahead of the `current_depth`
that don't extend into the visible window of events we're paginating
through.
Added details how synapse syncs the picture claim when
update_profile_information setting is true. Addresses #17836
---------
Co-authored-by: Michael Hlas <3398654+mhlas7@users.noreply.github.com>
When we port the `Event` class to Rust, the constructor will check for
the existence of required fields. To support that, we tidy up the test
code where we construct fake events to add all the required fields.
There should be no behavioural changes.
Review commit-by-commit.
The reason for the change is to make it easier to support these checks
when porting event class to Rust.
Previously, code that needed to access `prev_state_events` had to
combine a `room_version.msc4242_state_dags` boolean check with an
`isinstance(event, FrozenEventVMSC4242)` cast (or `cast()`) for the type
checker. Introduce `supports_msc4242_state_dag()` in a new
`synapse/events/py_protocol.py` which does both in one step via
`TypeIs[MSC4242Event]`, removing the need to import the concrete
`FrozenEventVMSC4242` class at every call site.
`MSC4242Event` is an `EventBase` subclass used purely for type narrowing
— it's marked with a metaclass that rejects `isinstance()` to make
accidental runtime use loud.
No behavioural change: callers continue to gate on the same room version
flag and access the same `prev_state_events` attribute.
This is in prep for using the room versions more from Rust.
Main changes:
- Change it so each room version is defined as a delta to the last one.
This is a cosmetic change that makes it easier to ensure the room
version definitions are correct (as they're defined as deltas from
previous versions).
- Move constants to `RoomVersion` constants, like `RoomVersion::V1`, for
convenience.
- Change visibility of various attributes.
Convert `prev_state_events` to use `StrCollection` rather than requiring
it to be a mutable list. None of the usages require it to be a proper
list, and besides, events are immutable and therefore so should
`event.prev_state_events`.
Based on #19708.
This is on the path to porting the entire event class to Rust, as
`event.content` will then return the new Rust class `JsonObject`.
This PR adds a pure Rust `JsonObject` class that is a `Mapping`
representing a json-style object. It uses `serde_json::Value` as its
in-memory representation and `pythonize` for conversion when a field is
looked up on the object.
I'm not thrilled with the name, but couldn't think of a better one.
This also adds `JsonObject` handling to the JSON serialisation functions
we use, as well as to the `freeze(..)` function.
Reviewable commit-by-commit.
Better to retry more quickly than have workers wait around. 5 seconds is
still a reasonable gap in time to not overwhelm anything.
This matters most in cross-worker scenarios. When locks are on the same
worker, when the lock holder releases, we signal to other locks (with
the same name/key) that they should try reacquiring the lock
immediately. But locks on other workers only re-check based on their
retry `_timeout_interval`.
Updating to 5 seconds to match the previous intentions based on the
[flawed
code](https://github.com/element-hq/synapse/blob/6100f6e4f7fb0c72f1ae2802683ebc811c0e3a77/synapse/handlers/worker_lock.py#L278).
We can assume they were trying to have 5 seconds as the max value to
retry.
Spawning from
https://github.com/element-hq/synapse/pull/19394#discussion_r3168458070
Fixes the symptoms of https://github.com/element-hq/synapse/issues/19315
/ https://github.com/element-hq/synapse/issues/19588 but not the
underlying reason causing the number to grow so large in the first
place.
```
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
```
Copied from the original pull request on [Famedly's Synapse
repo](https://github.com/famedly/synapse/pull/221) (with some edits):
Basing the time interval around a 5 seconds leaves a big window of
waiting especially as this window is doubled each retry, when another
worker could be making progress but can not.
Right now, the retry interval in seconds looks like `[0.2, 5, 10, 20,
40, 80, 160, 320, (continues to double)]` after which logging should
start about excessive times and (relatively quickly) end up with an
extremely large retry interval with an unrealistic expectation past the
heat death of the universe. 1 year in seconds = 31,536,000.
With this change, retry intervals in seconds should look more like:
```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
60, < never goes higher than this
]
```
Logging about excessive wait times will start at 10 minutes.
<details>
<summary>Previous breakdown when we were using 15 minutes</summary>
```
[
0.2,
0.4,
0.8,
1.6,
3.2,
6.4,
12.8,
25.6,
51.2,
102.4, # 1.7 minutes
204.8, # 3.41 minutes
409.6, # 6.83 minutes
819.2, # 13.65 minutes < logging about excessive times will start here, 13th iteration
900, # 15 minutes < never goes higher than this
]
```
</details>
Further suggested work in this area could be to define the cap, the
retry interval starting point and the multiplier depending on how
frequently this lock should be checked. See data below for reasons why.
Increasing the jitter range may also be a good idea
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
(cherry picked from commit 3f58bc50df)
Similar to #19706, let's port the `unsigned` field into a Rust class.
This does change things a bit in that we now define exactly what
unsigned fields that are allowed to be added to an event, and what
actually gets persisted. This should be a noop though, as we carefully
filter out what unsigned fields we allow in from federation, for example
As a side effect of this cleanup, I think this fixes handling
`unsigned.age` on events received over federation.
Richard van der Hoff