A membership event carries the claim about who its member key belongs to, and
a join that makes no claim leaves its sender attributable to nobody: not
routable, not shown to clients, soft failed by every server that receives
anything further from them. The joins this server sent to member key rooms
made no claim.
The gap was in where the linkage information is filled in. For events a client
asks us to create, that happens while the event dict is being translated into
member keys, which is also where a client's own attempt to assert linkage
information is thrown away. A join sent over federation never goes through
that path: the template comes from the resident server, we fill in the
client's content and sign it, and nothing along the way says who we are.
So fill it in there too, once the room version coming back from `/make_join`
shows the room wants it: our domain, and the user's localpart unless they are
erased, in which case their name is not published again. Whatever the client
asked for is dropped first, as it is everywhere else — a client may not say
who a member key belongs to, only the server holding it may. The event is then
signed by the member key and co-signed by us, which is what makes the claim
worth anything to the server receiving it.
Also stop recording a link against a member key this server holds. Links
describe other servers' keys, and for our own we already know the answer, so
the row would be at best redundant. It becomes actively wrong once a member
key is moved between servers: the new server's claim about it is still not
ours to write down against our own copy.
The tests drive the outbound half with a stubbed resident server, so what we
put on the wire is checked rather than assumed. The tests for the inbound half
build their events by hand and so never exercised this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A member key names no server, so a room full of them is a room whose members
cannot be routed to, served to, or shown to anyone. What a key stands for is
the linkage information on its `m.room.member` event, and this teaches the
server to read it: another server's member can now join one of our member key
rooms over federation, and we learn who they are from the event that put them
there.
A member key's link is the linkage information on its *current*
`m.room.member` event, and nothing else. A member who has left has no link,
because the event that removed them was somebody else's and asserts nothing
about them. Making the mapping a function of the room's current state means
every server that has the state agrees on it, no history is needed to compute
it, and an event arriving late or by backfill cannot change it.
Two signatures decide what a claim is worth. The member key's own makes the
claim theirs, and without it the claim is ignored entirely: on an event the
member sent, the `sender` signature is already that; on an event somebody
else sent about them, it is the signature under `state_key`. The current
server signing key of the claimed domain makes the claim *verified*, and that
is the half that needs the network, so it is the half that can fail for
reasons that are nobody's fault: a domain that does not answer leaves the link
unverified rather than refused, and unverified links are re-checked on a small
bounded loop so that a member whose server was briefly unreachable does not
stay invisible until they next change their membership.
What a link is used for depends on which half holds. The claimed domain is
where the room's traffic goes, verified or not: an unverified claim is
somewhere to send to, and grants nothing. Only a verified link makes a server
a member server — able to be told the room's history, to be served its state,
to be treated as present in it — because anybody can claim anybody's domain.
Events from a key with no verified link are soft failed, so their state
changes still count (membership and power levels are decided by keys, not by
who holds them) but their messages are not shown to clients.
Joining a room of this kind works from either end. A joining server does not
know the room's version until `/make_join` answers, so it cannot know whether
to name its user by ID or by key: it sends the user ID in the path and the key
in `org.matrix.msc4430.member_key`, which a server that has never heard of
MSC4430 ignores. The key is held in memory until the room turns out to want
it, so joining a room without member keys leaves nothing behind, and it is
stored before the join is sent, so a join that succeeds is never one we cannot
sign for afterwards. A resident server accepts a key in either place and
refuses a bare user ID for such a room. Because the sender of a membership
event is a key and not a user ID, the check that the sender belongs to the
requesting server no longer applies: the origin is simply who relayed it, and
what the member claims about their own server is in the event, signed by them.
The end-to-end tests build the remote half by hand, so that a server can be
made to vouch for its member, decline to, vouch for a different one, or lie
about somebody else's. They cover a verified join, an unverified one, a
re-check that succeeds on a later attempt, a member key that moves to another
server and takes its member with it, and a moderator who bans a member while
naming a domain they control and changes nothing about who that member is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An `m.room.member` event's `content.localpart` and `content.domain` claim who
the member key in `state_key` belongs to, and the server named by
`content.domain` signs the event to vouch for the claim. That signature is
not enough on its own: a moderator kicking or banning a member can name a
domain they control, vouch for the claim with that domain's signing key, and
produce a verifiable but false statement about somebody else — defacing an
innocent member with an abusive localpart, or pointing them at a server that
then receives their traffic.
So require the claim to come from the member too. The member key named by
`state_key` signs the event under the entity `state_key`, with the member
keys' constant key ID, and linkage information on an event the member did not
send is ignored unless it is there. A moderator cannot produce that signature
without the member's key, and the absence of one costs them nothing: the kick
or the ban still takes effect, since this is not a validity rule. It only
decides whether anyone believes what the event says about who was kicked.
The events this server builds follow suit. A membership event sent by one of
our users about another — an invite — now carries the target's own signature
alongside ours, which is what makes it worth anything to the servers that
receive it. A kick, a ban or an unban carries no linkage information at all:
we would be asserting something we are not entitled to assert, and the
target's localpart has no business being published on somebody else's event.
Members' own events are unchanged; their `sender` signature is already their
assertion, and a second copy of it would say nothing more.
This replaces the previous arrangement, where the server's signature vouching
for linkage information was made under the constant `state_key` rather than
under the server's own name. That distinguished a claim about the target from
one about the sender, but it did not establish who was entitled to make it.
The entity name goes back to the server's name, as everywhere else in Matrix,
and `state_key` now names what it says: the member key in the event's state
key, signing for itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An `m.room.member` event in a member key room carries two signatures: the
member key named by `sender` signs the event, and the server named by
`content.domain` co-signs the claim that the key in `state_key` belongs to
`content.localpart` there. That co-signature was made under the signing
server's own name, which leaves a reader unable to tell what is being
asserted: an invite is sent by one member key and co-signed by the
*invitee's* server, so a signature by `example.com` on it could plausibly
stand behind either the sender or the target.
Name the entity for the member the claim is about instead. The constant
`state_key` joins the existing constant `sender`, and a signature under it
is, by construction, an assertion about the member named by the event's
`state_key` whoever sent the event. The key ID stays the signing server's
real key ID, since that is what names the published key a verifier must
check it against; only member keys use the constant `ed25519:1`.
A signature under a server's name is now refused outright rather than
treated as linkage information, so the only entities an event in such a
room may carry are `sender`, `state_key` and the authorising member key of
a restricted join.
Nothing yet verifies a co-signature, so this changes only the events this
server produces, and only in member key rooms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Kicks, bans and unbans, power levels and room upgrades in a member key room,
through the client API, on one homeserver. A moderator can now act on any
member, including one whose key this server holds nothing for (as federation
will bring them), and a room can be upgraded into a member key version and
from one member key version to another. Rooms without member keys are
untouched.
Naming a member. Clients name a membership target on `/invite`, `/kick`,
`/ban`, `/unban` and `PUT /state/m.room.member/{target}` by user ID, as
ever, or in a member key room by key.
`MemberKeyHandler.parse_membership_target` turns what the client sent into a
`RoomMemberIdentity` (a `UserID` or a `MemberKey`): a user ID is a user; a
key we hold is the user behind it, so that everything the membership code
does for a user of ours (suspension, invite rules, profiles, notifications,
the user-keyed membership tables) applies, and the user ID becomes the key
again when the event is built; any other key, one nobody vouches for or one
with a verified link to a member of another server, is a `MemberKey`, which
`update_membership` accepts for a kick, a ban or an unban and refuses for
anything else with 400 (there is nobody to invite, join or knock as), and
which the event then names as it is. `update_membership_locked` and
`_local_membership_update` take the `MemberKey` through the leave and ban
paths and skip what only a user has: an account to be suspended, a profile,
an invite to look up, a device to wake, a "left room" to record. A
`third_party_signed` block (a third-party invite to exchange on a join) is
refused with 400 in these rooms whatever the target: exchanging it builds a
membership event from the user IDs the client sent. A `MemberKey` is never
ours (`hs.is_mine`), so a kick of an invited key goes the way a kick of a
remote invitee does. `kick_guest_users`, which found our guests by the
domain of their state keys and so skipped every member of such a room, finds
them through the keys we hold. The event the kick or ban produces is sent by
the moderator's key and signed by it alone: `translate_event_dict` lets a
membership of `leave` or `ban` whose sender is not its target name any key,
and attaches no linkage information for a key we do not hold (we vouch for
nothing about it), while a target we do hold gets ours and our co-signature
as before. A member may still name their own key on the state API, which is
handled as themselves.
Power levels. The `users` of an `m.room.power_levels` event are translated
the same way as membership targets: a user ID of ours becomes the user's key
in the room (a user with no key yet is given one, so that a power level can
be granted ahead of a join, as in any room), a key is kept as it is (so any
member can be named by key), a user ID of another server is refused with
400, naming the same member twice (by user ID and by key) is refused with
400 rather than letting one entry win, and anything else is left for the
auth rules to reject. This covers the power levels a room is created with
(`power_level_content_override`) as well as later events.
Third-party invites bind a user ID to a membership, which means nothing for
keys: `do_3pid_invite` refuses them in these rooms with 400, before any
identity server is contacted, and so does an `m.room.third_party_invite`
state event.
Upgrades. `/upgrade` into a member key version works, from a room with
member keys or without one. The members a clone carries over (the power
levels' `users`, the bans, and with `auto_member` the joins and the pending
invites) are named in the old room as it knows them, which in a member key
room are keys that mean nothing in another room; `clone_existing_room` and
`_update_upgraded_room_pls` carry them by the users behind them
(`_users_behind_room_identities`, `_power_levels_by_user`), so that building
the new room's events mints new keys for them, and the creator removal that
MSC4289 versions need is applied to the user-keyed map. Before anything is
created, and once the requester is known to be allowed to send the tombstone
(the refusal names members of the room), `_check_upgrade_member_identities`
refuses an upgrade of a member key room to a version without member keys
(400, `M_UNSUPPORTED_ROOM_VERSION`), and an upgrade into a member key
version whose old room's power levels or bans name a member who is not one
of our users (400, naming them), since the new room could not hold them; the
old room is left as it was, with no tombstone. (The check and the clone read
the room's state separately; a power level or ban naming such a member that
lands between the two is refused when the clone builds the event, after the
new room's create event exists, as any upgrade that fails midway.) A joined
or invited member that maps to no user is left behind with a log line rather
than refusing the upgrade: there is nobody to join or invite.
`additional_creators` on `/upgrade` into these versions are validated as
user IDs (they become keys in the create event). Two things the clone
needed: the signing helper and `translate_event_dict` accept the frozen
content of a copied event (a `Mapping`, not a `dict`; translation copies it
before editing), and `create_and_send_new_client_events` chains
`prev_state_events` through a batch in an MSC4242 state DAG room, each state
event taking the previous batched state event as its parent, so that a join
sent in the same batch as its invite is authorised against it (the auth
check already looked for parents within the batch; the builder did not
produce them). And the rejections `auto_member` sends into the old room for
the invites it re-sends are, in a state DAG room, sent after the tombstone
rather than during the clone: the tombstone is built before the clone and
sent after it, so whatever the clone sends into the old room is the
tombstone's sibling, and in a state DAG room the merge of a rejection with
the tombstone's branch, where the invite still stands, is left to state
resolution with nothing to order the two by (the rejected invite came back
as an invite on some runs). `clone_existing_room` returns those events for
`_upgrade_room` to send once the tombstone is in. Rooms without a state DAG
take the previous path in both places.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
What a client sees of a member key room: every event names a member key as
its sender and, through MSC4428, as its stable ID; `member_info` resolves
the keys to users; the requester is recognised by their key wherever the
server decides what they may see or which events are theirs; and `/members`
and `/joined_members` are keyed by key. One homeserver only, but the code
answers for members of other servers too (a verified link names the user it
claims, anything else is nobody).
`member_info`. MSC4428's map, which only named erased users (the stable ID
of an ordinary room is the user ID and the entry is omitted), now gives
every member key an entry: the user behind a key we hold, the user named by
a verified link for another server's key, and `null` for an erased user or a
key nobody vouches for (an unverified claim is anyone's to make). The
resolution is one batched store lookup (`get_user_ids_for_identities`,
exposed by the member key handler as `user_ids_for_identities`), so the map
costs rooms without member keys nothing. The map already covers `/sync`
(full and incremental), sliding sync, `/members` and `/joined_members`,
which are keyed by state key and so by key; nothing there changes. An
erasure is recorded on the change stream under the stable ID that changed,
which in these rooms is the member's key, so the entry that flips to `null`
is the one clients hold.
Visibility. `filter_and_transform_events_for_client` decided what the
requester may see from their membership under their user ID, which the state
of a member key room does not hold: a joined user saw nothing. It now looks
the requester up under their key in each member key room the batch touches,
for both the allow check and the membership the event is annotated with, and
maps senders back to users, per room (a key is only a user in the room it
was looked up in), for the two per-user checks: erased senders, whose events
are pruned for users who joined later, so a key with no attributable user is
not erased; and ignored senders, where a member may be ignored as the user
behind their key or as the key itself, since a key with no attributable user
is the only handle a client has on it.
`filter_event_for_clients_with_state`, which push evaluation uses to decide
who may see an event, does the same per local user, for the ignore lists as
for the memberships, including for the out-of-band membership events it
returns to their target. The push evaluator itself resolves the sender (so a
user is not notified of their own events), the invitee and the inviter (so
an invite notifies its target, and the invitee's invite rules see the
inviter as a user, where one is attributable).
The requester's own events. The serializer echoes `transaction_id` and
`delay_id` only to the sender's own session, by comparing the sender with
the requester's user ID, which in a member key room never matches.
`SerializeEventConfig.requester_member_keys` (a new keyword-only argument,
`None` by default, so existing constructions are unchanged) carries the
requester's keys and the Rust serializer accepts a sender among them;
`create_config` fills it from the cached per-user keys when the feature is
on. Keys are unique across rooms, so a key of the requester's can only ever
be their own sender.
Sync. Where the handlers compared the requester's user ID with a sender or
a membership state key, they now use the identity the room knows them by
(`room_identity_for_user`: their key where they have one, their user ID
otherwise, or the cached map of their keys per room): the incremental sync
reads the requester's previous membership under their key, so a change to
their own membership (a display name) is an ordinary timeline event rather
than the room sent again as newly joined, and a leave is newly left; the
heroes of a nameless room no longer include the requester; the lazy-loading
full sync fetches the requester's own membership under their key; a leave
the user sent themselves is told from a kick by their keys per room, in
sync and in the sliding sync room lists; the join-race guard asks the
user-keyed membership table; and sliding sync's `$ME` is the requester's
key, including in the computation of which required state has changed
between requests. An invite's sender is resolved to a user (where
attributable) for the ignore list and the invite rules, in sync and in
both sliding sync room lists: a key on the ignore list hides the invite
too, and an unattributable key no longer reaches the invite rules, which
parse user IDs and would have failed the whole sync. The members handed to
presence and to device lists, by sync, by the device handler for newly
joined and left rooms, and by the room initial sync, are mapped to users
first, so clients are told about users' devices and presence, not keys'.
The three storage queries that find a user's membership changes in
`current_state_delta_stream` and `room_memberships` share one clause that,
with the feature on, also matches the keys we hold for the user, each
scoped to its room (keys are public strings anyone can name as a state key
elsewhere) and spelled out from the cached per-user keys so that every arm
of the clause can use the tables' indexes; a user with no keys, and the
feature being off, get the legacy clause byte for byte. Sliding sync's
"newly joined" and "newly left" detection, and the device handler's
shared-room tracking, run on those queries.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With `msc4430_enabled` on, the member key room version is registered and a
user can create such a room, join it, send events, change their own
membership, invite another user of this server and leave, all by user ID as
today, while every event in the room names member keys and is signed by
them. One homeserver only; members of other servers, moderation (kick, ban,
power levels), restricted joins and what clients are served back (`/sync`,
`/event`, `/members`) are outside this change.
Where a user ID becomes a key. Clients speak in user IDs, so the translation
happens once, in `EventCreationHandler.create_event`, before the event is
built: `MemberKeyHandler.translate_event_dict` replaces the `sender`, the
`state_key` of an `m.room.member` event and the `additional_creators` of a
create event with the users' member keys in the room, creating a key for a
user who has none (a key never changes, so a user who leaves and re-joins
keeps theirs). A membership event about one of our users gets our linkage
information (`content.localpart` and `content.domain`); whatever the client
put there is dropped first, since linkage information is a server's
assertion, and an erased user gets no `localpart`, so their name is not
published again. A user ID of another server, as sender, target or
additional creator, is refused (403 or 400) rather than quietly sent as a
user ID the room cannot hold, and so is a member key the client chose itself
as a target or additional creator: a key nobody vouches for would be a seat
in the room for whoever holds its private half (the membership servlets
still parse user IDs, so a key as a target is refused there too). A room
with a remote invitee is refused before anything is created, not by the
invite after the room exists, and a room cannot be upgraded into this
version (the clone names user IDs in power levels, bans and invites). The
handler is also where the rest of the server asks what an identity means:
`local_user_id_for_identity` (ours, by user ID or by a key we hold in that
room), `user_id_for_identity` (also the verified link of another server's
key), `domain_for_identity` (routing only: our name for our keys, the
claimed domain of a linked key) and `room_identity_for_user` (how to look a
user of ours up in a room's state: their key where they have one, their user
ID otherwise, which the state then does not contain, as it should not).
Three other places compared the sender with a user ID and now resolve it: a
suspended user redacting their own event, the admin-redaction rate limit,
and room participation.
The create event. In these room versions the room ID is derived from the
create event, and the create event names and is signed by the creator's key,
so the key has to exist before the room ID does. `translate_event_dict`
generates such keys as "pending" (an expiring in-memory cache), they sign the
create event from there, and once the event is built `create_event` stores
them under the new room ID; if building fails they are dropped at once. A
pending key is ours only for the create event itself (no room yet): in any
existing room it is nobody's, so a key minted for one room cannot act in
another while it waits. A user named twice by one create event (as creator
and additional creator) gets one key; a create event sent into an existing
room is refused before any key is made; and the room ID collision retry
starts each attempt from the content the client sent, not the translated
one. Room creation then looks the creator's membership up by their key in
the state map it batches the rest of the initial state against, and refuses
the version when the flag is off even though the registry (process wide) may
know it, which is also why the tests can run flag-off cases.
`default_room_version` cannot name this version: the server config validates
it before the experimental config registers it, so rooms are created with an
explicit `room_version`.
Signatures. `EventBuilder.build` asks `MemberKeyHandler.get_signers_for_event_dict`
who signs and `create_local_event_from_event_dict` signs with exactly those:
the sender's member key under the entity `sender`, if we hold it; our server
signing key on an `m.room.member` event whose target is a key we hold and
whose linkage information is exactly what we vouch for about that user (our
domain, their localpart unless they are erased, in which case none), which
is what lets other servers verify the member's link; and the authorising
member's key on a restricted join that names one of ours. An event whose
sender is not a key we hold gets no signature at all (it is only hashed), so
the server never signs as a key it does not own; the co-signature is decided
from the store, never from the dict, so no caller can have us vouch for a
name that is not the user's; and the signing helper refuses any signer the
event may not carry, so a wrong choice here could not produce one either. A
third-party rules module that replaces an event is as untrusted as a client:
its dict is translated again, and it may not change who authorised a join.
The builder's validation accepts canonical member keys as sender and
membership target in these rooms, and nothing else.
Membership handling. `update_membership_locked` looks the target and the
requester up in the room's state under their room identity (their key once
they have one), so re-joins, duplicate events and the ban checks see the
right membership; the previous membership of a new event is read under the
event's own state key; whether this server is in the room is answered from
the members that map to our users rather than by domain; a pending invite is
read by user ID (that table is keyed by user) and the inviter's server is
found through `domain_for_identity`, so a join via a remote inviter would
still route; the join notification maps the joining key to its user; and an
invite in such a room is refused with 400 unless the invitee is a key we
hold.
Hosts in a room. Registering the version makes the "which servers are in
this room" paths reachable for the first time, and they are what the
federation sender consults for every event. They took the domain of every
member; a member key has none, and on Postgres the SQL that extracts domains
from state keys returned a `None` host for such a room.
`get_hosts_for_identities` on the member keys store now gives the server
behind each identity (a user ID's domain, our own name for keys we hold, the
claimed domain of a key we have a link for, nothing for the rest), and
`_get_joined_hosts`, `get_current_hosts_in_room` and
`get_current_hosts_in_room_ordered` use it for member key rooms (the ordered
form is unordered there: nothing in the state keys to sort by). The sender
itself decided whether an event was ours by the sender's domain, which for a
key said "not ours" and dropped every event of such a room before reaching
any of this; it now asks the identity map, and finds the server of a
rescinded invite's target the same way rather than by parsing the key, which
would have raised outside the guard that protects the sending loop. A member
key room holds only our own members, so its hosts are this server and
nothing is sent.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
In an MSC4430 member key room the membership state keys and senders are
per-room Ed25519 keys, not user IDs. The room's own tables keep those keys
(`current_state_events`, `room_memberships`, the state and auth chains), but
several tables and caches are keyed by *user* and are what everything
downstream of persistence consults to answer "which rooms is this user in":
`local_current_membership`, the sliding sync membership snapshots,
`event_txn_id_device_id` (a foreign key to `devices`), `get_rooms_for_user`
and its relatives, and the membership stream cache that gates incremental
sync. This change fills those with the user behind each key we hold, on the
main process and on workers, and leaves rooms without member keys on exactly
the path they took before. The member key room version is not registered, so
none of it is reachable.
The local identity map. Storage cannot import the handler that owns identity
resolution for the rest of the server, so the storage half lives on the member
keys store: `MemberKeysWorkerStore.get_local_identity_map_txn(txn, room_id,
identities)` and its async counterpart `get_local_identity_map`. Both turn
room member identities into local user IDs: our users' IDs map to themselves,
member keys we hold *in that room* map to their user, and everything else
(other servers' users, other members' keys, and our own keys for other rooms)
is dropped. Member keys are only looked up when `msc4430_enabled` is on; off,
a member-key-shaped state key is as malformed as it always was and is dropped
as `is_mine_id` dropped it. The lookup is one batched query, and none at all
when no identity is a member key, so rooms without member keys cost nothing.
The map is built once per persisted batch at the top of `_persist_events_txn`
from every sender, every membership state key and every membership change in
the state delta, and threaded to `_persist_transaction_ids_txn`,
`_store_room_members_txn` and `_update_current_state_txn`; the only identities
not known until inside the transaction are the room's members when we are
leaving it, which `_update_current_state_txn` adds with a second lookup. The
partial-state resync path (`update_current_state`) builds its own map for its
delta. The async form serves `_calculate_sliding_sync_table_changes` and the
persistence controller, from the cached per-key lookups; keys never change
once created, so a cached answer can only be missing, never wrong.
What is keyed by user now. `_update_current_state_txn` writes
`local_current_membership` rows for the mapped users; the sliding sync
snapshot rows carry the mapped user (and the member key as `sender`, which is
what the room says); `_store_room_members_txn` does the same for out-of-band
memberships (remote invites and their rejections); `_persist_transaction_ids_txn`
stores the mapped sender, and `get_already_persisted_events` (the
persistence-time deduplication of a transaction ID) looks it up the same way,
so two racing sends of one transaction ID in a member key room still
collapse to one event; a sender the map does not know is stored as before,
so the foreign key makes a key that fails to map fail loudly rather than
silently drop the transaction ID; the persisted-events metric counts events
from our keys as local.
Caches and workers. Membership cache invalidations are keyed by the event's
state key, in `_invalidate_caches_for_event` (every event, and the events
stream for workers) and in `_invalidate_state_caches` (current state changes,
and the caches stream). In a member key room that key is the member key, so
the user-keyed caches `_invalidate_caches_for_event` touches
(`get_rooms_for_user`, `_get_rooms_for_local_user_where_membership_is_inner`,
`get_sliding_sync_rooms_for_user_from_membership_snapshots`,
`get_invited_rooms_for_local_user`, `did_forget`,
`get_forgotten_rooms_for_user` and the membership stream cache) are now also
invalidated for the user behind the key: once per membership event at the
event's own position, next to the per-event invalidation, and once from
`_update_current_state_txn` at the batch's position for members with no
event in the batch (a state reset, or everyone in the room when we leave it).
Workers get these as a row of their own on the caches stream
(`MEMBER_KEY_USERS_CACHE_NAME`, keys `[room_id, stream_ordering, *user_ids]`,
handled by `_invalidate_user_caches_for_member_keys`): the events stream rows
name the key and a worker cannot map it, and the current-state rows would
record the change at the caches stream's own position, which the membership
stream cache ignores when it is below the worker's starting event position
and misplaces when it is above. Only users whose state key is a key get these
rows; a user whose state key is their user ID has already had exactly these
invalidations, so rooms without member keys see no extra rows.
`_invalidate_state_caches` itself no longer parses member keys as user IDs: a
key names no host, so a membership change under one drops the
`is_host_joined` / `is_host_invited` entries for the whole room, which needs
those two caches to be tree caches keyed by room first. That is the one
change not gated on member keys: the caches change shape for every room (an
entry can be dropped by its room prefix), the alternative being to flush them
for every room on every change in a member key room. The replication client's
notifier wake-up on workers parsed the membership state key as a user ID,
which for a key raised and aborted the whole batch of replication rows
(nothing in that batch, in any room, woke a sync on that worker); it now
wakes the user behind one of our keys, and nobody for anyone else's.
Reads. `get_rooms_for_user` and `get_rooms_for_users` union
`local_current_membership` for local users when the flag is on; it is keyed
by user ID in every room and agrees with `current_state_events` in rooms
without member keys, so the union adds only the member key rooms. With the
flag off the legacy query runs byte for byte.
`get_membership_changes_for_user` also matches the user's member keys
(`room_memberships.user_id` is the state key) when the flag is on, scoped to
each event's room: keys are public strings, so a membership event in room B
naming the user's room A key is not a membership change of theirs. In the
persistence controller, extremity pruning groups senders by origin to drop
an old extremity sooner when the same origin sends again; in a member key
room a key is its own origin (the stricter reading), and the "never drop our
own events" walk uses the map rather than `is_mine_id`.
`_is_server_still_joined` maps the delta's member keys to users and passes
user IDs to `is_local_host_in_room_ignoring_users`, which queries
`local_current_membership`; before this the server would not have noticed
its last member leaving a member key room.
Remote member keys. Those reach `handle_potentially_left_users_txn` as
opaque identities (nothing in the map), where "do we still share a room with
them" is answered from `current_state_events` by state key and the
unsubscribe is a no-op delete plus one cache invalidation.
Run against the unchanged code, the tests show how an unadapted site fails:
thirteen of the fourteen persisting tests end in `MemberKeyHasNoDomainError`
(500) from `_invalidate_state_caches`, the first site on every path to parse
a member key as a user ID, which hides the ones behind it
(`_update_current_state_txn`, `_calculate_sliding_sync_table_changes`,
`_persist_transaction_ids_txn`, `_prune_extremities`,
`_is_server_still_joined`, the replication client, all adapted here). The one
silent failure was the out-of-band path in `_store_room_members_txn`:
`is_mine_id` answered False for the invitee's key and the invite was simply
never recorded for the user.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
In a room whose members are MSC4430 member keys, the `sender` of every event
and the `state_key` of every membership event is a per-room Ed25519 key, and
the event is signed by that key rather than by the sender's server. This
change teaches the two places every received event must pass, signature
verification and the auth rules, to work on keys. The member key room
version is not registered, so none of it is reachable; it is the acceptance
boundary every server has to agree on, so it depends on nothing but the
event.
Signatures (`_check_sigs_on_pdu`): in a member key room the event must carry a
valid signature under the entity `sender` by the key decoded from its `sender`,
with key ID `ed25519:1`, over the redacted event as for server signatures; a
restricted join must additionally carry a valid signature by the member key
named in `join_authorised_via_users_server`, under that member key string. No
server signature is required and no key is fetched, so every server reaches
the same verdict. The sender and the authoriser are parsed strictly: a
non-canonical base64 encoding of a key is not that key (otherwise one public
key would have two identities), and a user ID is rejected. A server
co-signature on a membership event's linkage information is not checked here,
on purpose: the MSC makes it decide whether the member's link is verified, not
whether the event is accepted, and some membership events legitimately carry
no linkage information at all. Hash checking, redaction on a hash mismatch,
the policy server and the spam checker are unchanged.
Auth rules (`event_auth`): `validate_event_for_room_version` checks that the
sender is a canonical member key and has signed the event, that a membership
event's `state_key` is a canonical member key, and that a restricted join
names a canonical member key as authoriser and carries its signature. The
`m.federate` domain comparison is skipped in member key rooms, in both the
general and the membership rule: keys have no domain, so it cannot be
evaluated, and in such rooms `m.federate: false` means no federation at all.
Power level `users` and `additional_creators` must be member keys;
`check_valid_additional_creators` therefore takes the room version, and the
room upgrade servlet passes the new one. Third-party invites need no rule
change: the rule compares `signed.mxid` with the `state_key` and verifies
`signed` with the key published in the inviter's `m.room.third_party_invite`
event, so an identity server that signs the member key produces an
acceptable invite; how it learns the key is unspecified. Everything else in
the rules already works on opaque identities: membership lookups, power
levels, creator power and the restricted-join authoriser are all compared as
strings. A membership event with no `membership` in its content is handled
without raising on both new paths (the legacy paths have the same latent
`KeyError`).
Redactions: in room versions since v3 a redaction by a sender without redact
power is applied only if both senders are from the same server, a check
deferred until the redacted event is known and made in three places (the event
cache, the read path, and persistence of an existing redaction). Those three
now share `redaction_sender_matches`, which keeps the server comparison for
every other room version and, in member key rooms where senders have no
server, applies only when the two senders are the very same key. That is
stricter than before (a server's other users can no longer redact a member's
events without power), and the only rule that needs no network.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MSC4430 identifies room members by a per-user, per-room Ed25519 "member key"
rather than a user ID, and lets a member's `m.room.member` event carry the user
ID it claims to be (`content.localpart`, `content.domain`), co-signed by the
claimed domain's server signing key. This adds the two tables and the store
behind that, with no callers: nothing in the server creates keys or learns links
yet, and the member key room version cannot be used.
`msc4430_member_keys` holds the keys we hold for our own users, one per (local
user, room), with the private seed. `MemberKeysWorkerStore.create_local_member_key`
is insert-if-absent and returns whichever key is stored, so a user's key in a
room can never change once it exists, however many concurrent callers try (the
earlier monolithic attempt raised `IntegrityError` here). The store derives the
public form from the signing key it is given, so the two columns cannot
disagree, and rejects any key ID other than `ed25519:1`. Every lookup by key
takes the room the key was seen in: a key of ours from room A appearing in room
B's events does not resolve there (the earlier attempt mapped keys without a
room and gave users phantom memberships). Lookups are cached per (room, user),
per (room, key), per user and per room, and `create_local_member_key`
invalidates all four on the cache stream, so workers see new keys.
`msc4430_member_links` holds what we have learned about keys we do not hold:
the claimed domain (routing information, kept whether verified or not), the
claimed localpart (NULL once the event that carried it is redacted, after
which the key maps to no user), and whether the link is verified. A verified
link always names the `m.room.member` event and the server signing key ID that
proved it; the `MemberLink` type and a CHECK constraint both enforce this, so
"verified" means exactly one thing. Our own keys get no link row: a link is a
claim someone else made, and the keys we hold are already known to be ours.
`upsert_member_link` is a plain replace that returns the previously stored link,
so a caller can see what changed (the mapped user, the domain); which events
may update a link is the caller's policy, not the store's, and
`get_member_link_txn` plus `upsert_member_link_txn` let it read, decide and
write in one transaction. `mark_links_unverified(domain, key_id)` downgrades
every link a rotated or withdrawn server key vouched for, in one bulk cache
invalidation, keeping the event and key ID so the links can be found again
(`get_links_for_signing_key`) if the events are re-signed; a key ID alone does
not identify a key, so re-checking a link means re-checking the co-signature
on its event against the domain's current key material, which the schema
says. `get_member_keys_for_domain_in_room(verified_only=...)` separates routing
(claimed) from trust (verified), and answers for our own server name from the
keys we hold, never from links: a link claiming our domain for a key we do not
hold is somebody else's claim. `get_member_links_for_claimed_user` is the
reverse mapping; it returns every claimant (verified first) rather than picking
one, because an unverified claim is anyone's to make and that lookup is how a
user ID in an invite, kick or power level change becomes a key.
`clear_link_localpart_for_redacted_event_txn` is the erasure hook (it clears
every link learned from the redacted event). Purging a room clears its links;
the keys we hold deliberately survive a purge, since a user's key in a room
must not change if the room is re-joined. The stored seed cannot be revoked
(MSC4430 has no member key rotation; only rotating the server key helps), and
the schema says so.
The store is mixed into `RoomMemberWorkerStore` (so it is present on workers)
rather than registered on `DataStore`, because the membership queries are the
code that will need it. `LocalMemberKey` and `MemberLink` are frozen attrs
types in `synapse/storage/member_keys.py`, next to the other storage return
types; `SigningKey` is added to the mypy plugin's immutable types so a
`LocalMemberKey` can be a cached value. `synapse_port_db` learns the boolean
column. `MemberKeyHandler.is_local_identity` (and so `hs.is_mine_identity`) now
answers for member keys by looking the key up in its room, instead of raising.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Rust: `RoomVersion.msc4430_member_keys` and `MSC4430V12`
("org.matrix.msc4430.12" = `MSC4242V12` + the flag); redaction of
`m.room.member` keeps `content.domain` and drops `content.localpart` in such
rooms, so the reference hash, the event ID and the member key signature cover
the claimed domain but not the claimed localpart;
`membership_field::{DOMAIN,LOCALPART}`; `org.matrix.msc4430` in `/versions`;
`.pyi` updated. The version is defined but *not* registered in
`KNOWN_ROOM_VERSIONS`, not even when the flag is on.
Python: `MemberKey` in `synapse.types` next to the other identifier types, with
the grammar `^@[A-Za-z0-9_-]{43}$` (anything with a `:` is a user ID),
`is_member_key` (syntactic, regex only, for hot paths) and
`MemberKey.from_string` (strict: rejects non-canonical base64, so each public
key has exactly one member key string). `MemberKeys` constants
(`SIGNATURE_ENTITY = "sender"`, key ID `ed25519:1`) and
`EventContentFields.MSC4430_{LOCALPART,DOMAIN}`. `supports_msc4430_member_keys`
is a `TypeIs[MSC4430Event]` modelled on `supports_msc4242_state_dag`;
`MSC4430Event` subclasses `MSC4242Event`, since every member key room is a state
DAG room, so one predicate narrows to both.
Signing: `add_hashes_and_signatures_from_signers(room_version, event_dict,
signers, *, server_name)` hashes once and then applies `(entity, key)` signers
in order; `compute_member_key_room_signature(room_version, event_dict, signer,
*, server_name)` adds one signature to an already hashed event (the authoriser's
signature on `/send_join`, our co-signature on an inbound `/invite`) without
rehashing a PDU we have not checked. The three signer kinds are: the sender's
member key under `sender`; our own server name co-signing linkage information;
the authorising member key of a restricted join under the member key string
itself. Together these are the one choke point for signing in member key rooms,
and they refuse with `MemberKeySigningRefusedError` (500) any signer the MSC
does not allow: a `sender` signature by a key other than the one `sender` names,
a key ID other than `ed25519:1`, a server co-signature under any name but ours
or on anything but an `m.room.member` whose `content.domain` is us, or an
authorising signature on anything but a join that names that key in
`join_authorised_via_users_server`. A bug in choosing signers therefore cannot
make us sign as a key we do not hold or vouch for linkage information that is
not ours. An empty signer list hashes only (`/make_join` templates).
Loudness: `get_domain_from_id` and `get_localpart_from_id` raise
`MemberKeyHasNoDomainError` (a `SynapseError`, 500: it is a server bug) when
given a member key, on the no-colon branch only, so user IDs and aliases are
untouched. Because the HTTP layer logs every `SynapseError` at INFO without a
traceback, and the error may be caught before it gets there, both MSC4430 errors
log themselves at ERROR with the stack of the raise site: the point of them is
to name the unadapted code path. `hs.is_mine_id` is left alone: it is called on
raw client strings (state keys, sliding-sync `required_state`,
`/users/{id}/report`, module API) and on remote-controlled room content
(`/upgrade` moving a canonical alias), where "False if malformed" is relied upon
and a member-key-shaped string in a legacy room is just malformed.
`hs.is_mine_identity(room_id, identity)` is the sanctioned question for room
identities; it answers for user IDs and raises `MemberKeyHasNoDomainError` for
member keys. The Prometheus counter
`synapse_msc4430_unsupported_path_total{server_name,path}` and
`record_unsupported_path(server_name, path)` (counts always, warns once per path
per process) are for branches that deliberately skip member keys.
The identity module is `synapse/handlers/member_keys.py` (`MemberKeyHandler`,
`hs.get_member_key_handler()`). The `MemberKey` type and `is_member_key` live in
`synapse.types` because `get_domain_from_id` needs them and the module cannot
import handlers.
Config: `msc4430_enabled` implies `msc4428_enabled`; the Complement image sets
it.
None of this is reachable: no identity in any existing room can satisfy
`is_member_key`, and the room version cannot be used. Not gated, by design: the
`is_member_key`-guarded raises in `get_domain_from_id`/`get_localpart_from_id`,
which only fire for member-key-shaped strings and turn a 400 "Invalid ID" into a
500. The one remote-reachable path that was a clean 400 before,
`/make_join|leave|knock/{roomId}/{userId}` with a member-key-shaped path
parameter (the handlers compare `get_domain_from_id(user_id)` with the origin),
now validates the parameter as a user ID at the servlet (400 `M_INVALID_PARAM`),
so a malformed request cannot make the Synapse federation client fail over
across every resident on a 5xx. A PDU whose sender is member-key-shaped in a
legacy room already fails at `_check_sigs_on_pdu` and is logged at ERROR with a
traceback on `/send`; only the status code changes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Changes to a member's `member_info` entry that no event in the room anchors
(erasure, and MSC4430 link verification) were only delivered when the member
next appeared in a response. Give them a stream so that syncing clients are
woken and told.
* `StreamToken.member_info_key`: a trailing `MultiWriterStreamToken`
component (`StreamKeyType.MEMBER_INFO`); tokens without it still parse.
* `msc4428_member_info_changes` (delta 95/01) behind a multi-writer id
generator with no fixed writer list, as for cache invalidations: any
worker may write (erasure runs wherever deactivation is handled).
`MemberInfoChangesStream` replicates rows; workers wake room members.
* Retention: rows older than 30 days are pruned hourly on the background
worker and the prune point recorded; a since token older than that is
sent the full map per room instead of deltas.
* Producer: `MemberInfoHandler.on_user_erased`, from `deactivate_account`,
one row per room the user has any membership in.
* Consumers: sync v2 reads the changes for all joined rooms once per sync
and includes a room with an empty timeline when it has any; sliding sync
includes such rooms when choosing what to send and resends the full map
for rooms it last sent at an older position.
The stream exists always, like every stream, but nothing writes to it unless
`msc4428_enabled` is set.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add `org.matrix.msc4428.member_info` to `/sync` (joined and left rooms),
sliding sync, `/members`, and `user_id` to `/joined_members` entries. A new
`MemberInfoHandler` computes the map; entries are only produced where the
mapping differs from the stable ID itself, so it only names erased users
(`user_id: null`, via `are_users_erased`).
A room's map covers the stable IDs the response mentions: senders of its
timeline, state and sticky events, targets of its membership events, and
the room's heroes. On an initial sync without lazy-loaded members that is
every member; a lazy-loading client gets the entries for members it loads
later from `/members`, which always carries the map. On sync the map is
omitted when empty, and `member_info` counts towards whether a room is
included in the response. Changes with no event anchor (the erasure
itself) are delivered by the next event from or about that member, or on
the next full sync; a stream for them is A3.
With `msc4428_enabled` off the responses are unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add the `experimental_features.msc4428_enabled` flag. When set, `/versions`
advertises `org.matrix.msc4428` and the Rust event serializer adds
`unsigned.org.matrix.msc4428.stable_id` (the sender's user ID) to every
event served to clients, including the events embedded in `unsigned`
(`redacted_because`, bundled edits, thread summaries). The field is subject
to the client's field allowlist like any other unsigned key.
`SerializeEventConfig.__init__` gains a required `msc4428_enabled` argument
(not module API).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This PR only handles /send_join and connected events in /send,
so everything else should return early with coherent error messages.
The join event must be connected to the main state DAG but we previously
didn't ensure this.
The cascade rejection only worked on in-flight events, not seen events.
We now overwrite events in event_map with rejected events (thus setting .rejected_reason)
to ensure we continue to cascade rejections correctly.
Complement test asserts this.
- Enable MSC4242 complement tests
- Fix an issue which caused cascade rejections to not be applied to events in the /send_join response becau
se we didn't remember in-memory which events were rejected.
- Accept rejected events in the /send_join response, it gives us a bit of robustness at no safety cost.
- Recalculate room state if backfilled events caused the state DAG fwd extrems to change.
- Don't filter out already seen events from event_map, so the statement "the event_map is the complete stat
e DAG" still holds.
- Add validation to length of prev_state_events
- Don't validate auth_events length for state DAG events as it is done too early, before we've had time to
calculate the auth events (and validation is moot anyway given it isn't attacker controlled anymore)
- Let backfilled events update state dag fwd extrems
- Add test for edge case with backfilled state events
