Spawning from
https://github.com/element-hq/synapse/pull/20143#discussion_r4080708454
`RestServlet.register` read `PATTERNS` via `getattr`, so it was typed as
`Any` and mypy never checked subclasses' patterns against what
`HttpServer.register_paths` expects.
This declares the attribute so that classes extending `RestServlet`
provide the right type for the `PATTERNS` attribute.
Add a metric for the number of times we see a stall due to being given a
future token.
This would have caught
https://github.com/element-hq/synapse/issues/20080, where lots of sync
streams got stuck due to a stream not getting replicated correctly.
## Summary
Clarify the `--exists-ok` command-line help text for
`register_new_matrix_user`.
The previous help text said:
> Do not fail if user already exists.
This could be interpreted as the existing user's account being updated.
The new wording explicitly states that the existing user account will
not be updated.
This also aligns the CLI help text with the existing Debian man page
documentation.
## Issue
Fixes#20112
## Testing
* `git diff --check`
* Reviewed the staged diff to confirm only the intended help-text change
was included.
Tokio tasks currently complete by taking the GIL and calling
`reactor.callFromThread`. Taking the GIL may block for a period of time,
and we do not want that to happen on the tokio reactor threads (as that
can block other work from happening).
To avoid this, we instead add a work queue that we can add to from Rust
without taking the GIL, which is drained by the reactor. We signal to
the Twisted reactor that it should wake up by using a unix socket pair,
which is exactly how `reactor.callFromThread` works.
---
The self-pipe trick is basically where you create a pair of unix sockets
connected to each other. One end is added to the reactor so the reactor
is woken up when there are bytes to read, and when another thread needs
to wake up the reactor it just needs to write a byte into the other unix
socket.
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
This pull request fixes a few issues with the profile update stream,
when a user leaves a room. This is basically just mimicking what we
already had for when someone leaves a room, and they no longer share any
rooms, but in reverse - ie when we leave a room, and no longer share
rooms with some users. This was missed in the implementation when adding
the profile update stream for legacy and sliding sync.
[Fix missing profile update stream rows on user leaving
room](https://github.com/element-hq/synapse/commit/e953322226731a99718ebe13139b9c5f05ae4b62)
When a user leaves a room, profile update stream rows are generated with
the `LEFT_ROOM` action for each user in the room that no longer shares a
room with the user who left the room.
This also needs to happen in reverse. The user who left the room needs
to have a profile update stream row for each user they no longer share a
room with.
If we don't do this, clients may keep stale data around even after they
don't share a room with a user, which may mean they don't know when to
refetch profiles after re-joining a room with the stale profile data
user.
[Clear out old profile update stream rows when leaving a
room](https://github.com/element-hq/synapse/commit/e7642bf141b098c1eefaf74655e5e7e18415500b)
When we leave a room, ensure profile update stream rows are cleared out
for every user we no longer share a room with. This is the same as what
happens when someone else leaves a room, but in reverse. The only
remaining profile update stream row should be the `LEFT_ROOM` action.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
This is to prevent the case where URL previews return a MXC that
immediately 404s because the media in question has been quarantined.
This is mostly to help implementations which currently show an ugly
empty preview due to the MXC being sent down, despite being invalid.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [ ] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
Previously, the worker endpoint pattern included only the multi-item
lookup and `/restart`. Adjust the pattern so that it includes the
per-`delay_id` lookup endpoint as well.
Also adjust the worker documentation to use this same endpoint pattern.
Follow-up of #20210
[MSC4140](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/4140-delayed-events-futures.md),
"Account deactivation":
> when an account is deactivated, the homeserver MUST cancel that
account's delayed events which have not yet been added to a room's event
DAG. These cancelled records MAY be removed immediately, including their
stored event content, as an exception to the usual finalised-record
retention policy.
Deactivation currently leaves the user's scheduled delayed events,
including their content, in place, and they are still attempted at their
scheduled time:
| Delayed event scheduled by the user | What happens after deactivation
today |
| --- | --- |
| Message or state event, fires after deactivation has made the user
leave the room | The send fails with a 403 (user not in room) and the
record is dropped |
| Message or state event, fires before deactivation has made the user
leave the room (leaving happens room by room in the background) | The
event is sent |
| `m.room.member` join for themselves in a public room | The deactivated
user re-joins the room |
### What changes
- Deactivating an account (client or admin API) removes the user's
unsent delayed events as its first step, and re-arms the send timer for
whatever is scheduled next.
- Deactivation can run on a worker, while the send timer lives on the
main process, so the cancellation goes through a new replication
request.
- A delayed event whose send has already started is left alone, as with
a normal cancel: it is already on its way into the DAG, and the send
path removes its record itself.
Suspension and locking are unchanged.
### After #19038
Today a cancelled delayed event is simply deleted, so this PR deletes
the user's records too. #19038 changes cancellation to keep the record
and mark it as cancelled ("finalised"), so that clients can look up what
happened to a delayed event. Once it has landed, deactivation should
probably finalise the user's records as cancelled the same way, rather
than delete them. The MSC allows either: it permits removing the records
immediately, content included, as an exception to the usual retention of
finalised records, which is worth doing at least when the user asks to
be erased.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Andrew Ferrazzutti <af_0_af@hotmail.com>
Co-authored-by: Andrew Ferrazzutti <andrewf@element.io>
As I was working on https://github.com/element-hq/synapse/pull/20218, I
noticed what seemed to be an illegal configuration of synapse.
- `require_auth_for_profile_requests`: blocks profile requests unless
authenticated
- `limit_profile_requests_to_users_who_share_rooms`: blocks profile
requests unless authenticated user share a room with requested user
This, I think, should be an illegal config:
```
require_auth_for_profile_requests = false
limit_profile_requests_to_users_who_share_rooms = true
```
As of now, with such a config the shared-room check is never applied: a
profile can be requested anonymously, and also by an authenticated user
who doesn't share a room.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Olivier 'reivilibre' <oliverw@element.io>
The count of unread notifications is computed in two phases:
1. From the summaries (`event_push_summary`) when those are up to date
with the user's last read receipt
2. Otherwise from counting the push actions (`event_push_actions`) after
that receipt
The problem in this process is that the threads whose summary is up to
date are identified with only their `thread_id`. But all main timelines
share the same `"main"` thread_id, so as soon as one room has an
up-to-date summary, phase 2 skips the main timeline of every room.
The fix identifies the up-to-date summaries with their `(room_id,
thread_id)` pair.
Authored by @sandhose. Found while investigating the
`TestThreadedReceipts` Complement flake (#15517, #18537), but it is a
bug on its own.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Quentin Gliech <quentingliech@gmail.com>
Co-authored-by: Devon Hudson <devonhudson@librem.one>
We sometimes see a lot of `ERROR` logs like the following:
```
Closing scope Scope<... master.write_bytes_to_request> which is not the currently-active one None
```
This error is generated by `opentracing.Scope` checking that it is the
"active" one. Synapse tracks "active" spans via logcontexts, so this
indirectly asserts that the scope is closed in the context it was opened
in. However, the producer methods are often called from the reactor and
therefore withing the sentinel logcontext, which produces the error
above.
This specifically happens when the producer tries to write large
responses but gets paused, and then later resumes.
The fix is to simply use `Span` directly, rather than scopes. `Span`
does not perform the checks.
I noticed this when deploying #19979, though it is unrelated.
### Summary
Resolves#18308.
Makes the `TaskScheduler`'s maximum concurrent running tasks
configurable via `task_scheduler.max_concurrent_tasks` in the homeserver
configuration and defaults it to `2` (reduced from the previous
hardcoded limit of `5`).
### Motivation
Twisted's `adbapi.ConnectionPool` defaults to 5 database connections
(`cp_max=5`). When the `TaskScheduler` previously executed up to 5 tasks
concurrently, it could exhaust the entire database connection pool,
starving regular API and synchronization requests on smaller homeserver
instances. Lowering the default to 2 preserves at least 3 connections
for foreground requests while allowing concurrent tasks to progress, and
gives administrators the ability to tune the limit according to their
host and connection pool sizing.
### Changes
- Added `TaskSchedulerConfig` (`task_scheduler.max_concurrent_tasks`)
with validation (must be a positive integer) and registered it in
`HomeServerConfig`.
- Updated `TaskScheduler` to enforce the configured concurrency limit
and updated class default to 2.
- Updated config documentation and JSON schema.
- Added config tests in `tests/config/test_task_scheduler_config.py` and
updated unit tests in `tests/util/test_task_scheduler.py`.
- Added newsfragment in `changelog.d/18308.feature`.
---------
Signed-off-by: nevil06 <nevilansondsouza@gmail.com>
Co-authored-by: Devon Hudson <devonhudson@librem.one>
When a stream advances in the database but stops being replicated to a
process, that process's view of the stream freezes. Requests that wait
for it to catch up to a token issued by another worker then time out and
return empty responses indefinitely (see #20080), and nothing exported
said so.
Report `get_current_token` from every ID generator, on every process.
The value is comparable between processes, so a stream that has stopped
reaching one of them shows up as divergence with no client traffic
needed. It is also the position that `wait_for_stream_token` waits on,
so its divergence is the failure itself rather than a proxy for it.
Being a watermark over gapless runs of persisted IDs, it also catches a
single writer of a sharded stream going quiet, which a maximum across
writers would hide behind the writers still being replicated.
Previously the tokio runtime was stashed in a hidden attribute on the
reactor object, installed lazily by whichever Rust code first needed it,
and started via `callWhenRunning`.
Instead, we create a `RustRuntime` (accessible via
`HomeServer.get_rust_runtime()`) that holds any per-reactor Rust state,
such as the tokio runtime. It is constructed lazily on use. Rust
consumers (`HttpClient`, `VersionsHandler`, the Python DB pool wrapper)
now receive the runtime or reactor handle explicitly, and the
`reactor.run()` / manual-startup workarounds in tests are no longer
needed.
We also add helper wrappers in Rust for `Reactor` and `HomeServer` that
exposes the needed functionality.
The aim is to allow us to have a Rust-side clock (mainly to get the
current time), that respects the unit test per-reactor time management.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This is an attempt to better cache the cases where there are a large
number of extremities to resolve over, which keep slightly changing.
This spawns from seeing issues on matrix.org.
We already have a cache over the exact state groups being resolved.
However, we can do better by caching the inputs into state res (i.e. the
conflicted sets), which are more likely to be constant across repeated
state res in a room. We key this cache based on a sha256 hash, on the
assumption that this will never conflict.
Also includes a commit that removes needless copying of the state.
Ports the logcontext classes to Rust, and gives tokio tasks a captured
logcontext so that work running in (or spawned from) Rust is attributed
to the request that caused it.
1. **Add characterization tests for logcontext error messages and the
filter** — pins the exact `logcontext_error` message shapes, the
abuse-detection code paths and `LoggingContextFilter`'s observable
behaviour, *against the existing Python implementation* (this commit is
green on its own). These are the behavioural contract the port has to
satisfy.
2. **Port `ContextResourceUsage` to a Rust pyclass** — self-contained:
the new `synapse_rust.logcontext` module, the class, its stub and the
re-export.
3. **Move the logcontext storage and `LoggingContext` to Rust** — the
core change; the commit message carries detailed design notes.
Highlights:
- The slot is typed `Option<Py<LoggingContext>>`, with `None`
representing the sentinel. `_Sentinel`/`SENTINEL_CONTEXT` stay pure
Python (unchanged); thin wrappers on
`current_context`/`set_current_context` convert at the boundary, and
pyo3's extraction enforces the type (`TypeError` otherwise).
- The accounting is native: one `getrusage(RUSAGE_THREAD)` read per
switch via libc, inline `stop`/`start` bookkeeping for base
`LoggingContext`s, Python dispatch only for subclasses
(`BackgroundProcessLoggingContext`) so their overrides run. The thread
id comes from `PyThread_get_thread_ident` (the exact
`threading.get_ident()`
value) without calling into Python.
- The hot paths avoid per-operation allocation: names are `Py<PyString>`
(the per-log-record `str(context)`/`server_name` reads are INCREF-only),
error branches materialise strings only when hit.
4. **Attribute Rust-spawned work to the caller's logcontext** —
`create_deferred` captures the caller's context and scopes it onto the
spawned task via a tokio task-local (`LogContextHandle`);
`current_context()` gives the task-local read precedence, so
`LoggingContextFilter`/`pyo3-log` resolve the right context on worker
threads with no per-record stamping. `run_python_awaitable` restores the
captured context (via a `with_logcontext` helper, the Rust
`PreserveLoggingContext`) around Python called back from Rust, so e.g.
`runInteraction` from the Rust `/versions` handler accounts its DB usage
against the right request. Integration tests exercise both guarantees
through real production code paths.
Follow-up work on top of this (separate PR): porting
`BackgroundProcessLoggingContext` natively and removing further `Py<_>`
indirections. The fact that `BackgroundProcessLoggingContext` is a
subclass is what forces some of the warts in this PR: e.g. having to use
`Py<LoggingContext>` everywhere, etc.
We don't try (yet) to make this pure Rust, instead we see this as simply
maintaining the Python logcontext machinery when crossing, rather than
trying to make a Rust equivalent that can be used by pure Rust
dependencies. We probably do want to do that in future, as well as wire
up e.g. CPU recording on Rust side, but that is unnecessary for now.
For state filters that ask for concrete types. This allows us to cache
the common case of asking for a specific type/state key.
I noticed a bunch of queries in the jaeger traces that could be cached.
Part of #18731 (Support Matrix 1.15).
Since Matrix 1.15, the
[spec](https://spec.matrix.org/v1.19/client-server-api/#get_matrixclientv1roomsroomidhierarchy)
defines `allowed_room_ids` on the room summaries returned by `GET
/_matrix/client/v1/rooms/{roomId}/hierarchy`, so that clients can tell
whether a restricted room can be joined or only knocked at:
> **`allowed_room_ids`** — `[Room ID]` — If the room is a [restricted
room](https://spec.matrix.org/v1.19/client-server-api/#restricted-rooms),
these are the room IDs which are specified by the join rules. Empty or
omitted otherwise. Added in `v1.15`
>
> — [Matrix
Spec](https://spec.matrix.org/v1.19/client-server-api/#get_matrixclientv1roomsroomidhierarchy)
Synapse currently strips the field from the client `/hierarchy` response
before returning it — a guard added in matrix-org/synapse#12175 (2022),
correct at the time, when the spec defined the field for federation only
and it was leaking into client responses. The other two surfaces that
define the field (`/room_summary` and the federation `/hierarchy`)
already return it.
This PR removes the strip in `_RoomEntry.as_json` (and the now-unused
`for_client` parameter), so client hierarchy entries include
`allowed_room_ids` for restricted rooms, both local and received over
federation. The first commit adds the failing test coverage, the second
removes the strip.
---
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
This partially fixes the bug
https://github.com/element-hq/synapse/issues/20116
Device list update EDUs from non-compliant (grandfathered historical)
user IDs are currently accepted over federation, stored, and surfaced to
clients in `/sync`'s `device_lists.changed` array.
> For current room versions, servers must still accept events using such
user IDs over federation; however they SHOULD NOT forward such user IDs
to clients when referenced outside the context of an event. For example,
device list updates from non-compliant user IDs would be dropped by the
receiving server.
>
> -- [Matrix
spec](https://spec.matrix.org/v1.14/appendices/#historical-user-ids),
clarified in Matrix v1.14 by
[matrix-spec#1506](https://github.com/matrix-org/matrix-spec/issues/1506)
### Problem Example
A remote server sends an `m.device_list_update` EDU for
`@héllo:remote.example` (non-ASCII localpart, outside the compliant
U+0021–U+007E range). Synapse:
- accepts and processes the update (resyncing the user's device list if
needed)
- stores it in the remote device list cache
- forwards `@héllo:remote.example` to local clients via
`device_lists.changed` in `/sync` (**the leak** — a non-compliant user
ID referenced outside event context)
---
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
This PR fixes the issue described as comment here:
https://github.com/element-hq/synapse/issues/18793#issuecomment-3502202379
In Element Call, this shows up as ghost participants: someone who left
the call keeps being displayed until a later state change refreshes the
room.
The bug is not specific to Element Call: any state event can be
affected, RTC membership just changes often enough to make it visible.
## What happens
Alice has a client syncing against a homeserver where events are
persisted by one worker (the event persister) and `/sync` is served by
another (the sync worker). Her client is parked in a long-poll: `GET
/sync?since=s99&timeout=30000`.
Bob joins a call at the same moment Carol sends a message. Carol's
message reaches the persister first; Bob's `m.call.member` arrives while
that write is still in flight, so the per-room persist queue groups them
into one transaction:
```
events (each gets its own stream ordering):
stream_ordering 100: m.room.message Carol
stream_ordering 101: m.call.member Bob (state)
current_state_delta_stream (how state_after finds state changes):
stream_id 100 ────► (m.call.member, @bob) -> $bob_join_call
▲
└─ stamped with the batch MINIMUM (100), not the event's own 101
(see `_update_current_state_txn`)
```
The transaction commits: both events and the delta row are now in the
database, atomically.
The persister then announces the new events over replication, one RDATA
token per stream ordering — rows are only merged into one token when
they share a position, and 100 and 101 don't. So the sync worker's
events-stream position steps 99 → 100 → 101, and on reaching 100 it
pokes the notifier.
Alice's long-poll wakes at exactly that moment. Her response is built at
the worker's *current* position — `end = 100` — with RDATA 101 still in
the queue:
```
Sync A (since=99, end=100):
timeline: events 99 < ordering ≤ 100 → [Carol's message]
state_after: deltas 99 < stream_id ≤ 100 → [$bob_join_call] ← delivered EARLY
next_batch: s100 ← mid-batch token
```
No race on the client's side is needed: the server *hands out* the
mid-batch token as `next_batch`. Alice's client re-polls with it, as
every sync client does. The worker has meanwhile processed RDATA 101:
```
Sync B (since=100, end=101):
timeline: events 100 < ordering ≤ 101 → [Bob's m.call.member @101] ✓
state_after: deltas 100 < stream_id ≤ 101 → [] row is stamped 100 ✗
```
A state event in the timeline with an empty `state_after`. An MSC4222
client trusts `state_after` over timeline state events, so Alice's copy
of Bob's call membership never updates from this response.
On a single process this cannot happen: the batch's stream IDs are
released as a whole, so the position visible to `/sync` jumps 99 → 101
and `s100` is never handed out. Only a process that learns its position
from replication — any sync worker — ticks through the middle of a
batch.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
Fix#19905
Directly retrieve the `m.room.join_rules` event so the room summary data
is correct and matches `allowed_room_ids`. Otherwise there could be a
mis-match delay or the `join_rule` could be missing altogether.
Adds a `redis.username` config option.
Details:
A `username` without a `password` (or `password_path`) is refused at
startup. Redis has no wire form for a username without a password, and
txredisapi only sends `AUTH` when a password is set, so the username
would otherwise be silently ignored. An explicitly empty password is
accepted, since that is how a `nopass` ACL user is configured.
This relies on txredisapi 1.4.12, the first release to accept a
`username` kwarg. That upstream support was contributed by @karolyi
specifically to unblock this.
Fixes#19238.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Instead, treat them as absent fields as they feel like they should be.
The database implementation detail that these fields have a dedicated
column with `NULL`
when unset is kept to the storage layer.
The goal here is to reduce the amount of special casing needed for these
two original profile fields and treat them a little bit more like
regular profile fields.
Follows: #20003
Follows: #20147 (needed as a bugfix to continue sending them down
oldschool sync when they get deleted. Without #20147, this PR would
break that — which matches how custom profile fields were broken too.)
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
A key refactoring for, and split out of,
https://github.com/element-hq/synapse/pull/20165
Would be easier to land first to isolate the diff.
Should be a standalone change with no behavioural change.
Motivation is that #20165 will round-robin between 'main queue'
transactions and 'sticky event' transactions.
To keep the data flow clear, I wanted to insert a typed struct (well,
`attrs` dataclass) as an interface between the 'preparation' of a
transaction and its 'completion'.
Doing this whilst keeping the asynchronous context manager style did not
lead to a readable result in my opinion.
(I would also say the async context manager is a touch 'magic' /
obscures control flow, but I suspect this is largely down to opinion.)
Replace _TransactionQueueManager with prepare/complete transaction
methods
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
dependabot[bot]