Mjolnir can now be run as an application service,
meaning it will host multiple independent mjolnirs that can be requested by users.
If the user is on the same homeserver as the appservice is deployed on,
then they can provision a mjolnir via a widget https://github.com/matrix-org/mjolnir-widget.
Otherwise they can invite the appservice bot to a room they want to protect.
This will create them a mjolnir, a management room and a policy list.
The appservice shares the same docker image as the bot,
but is started slightly differently by specifying "appservice"
as the first argument to docker run (this s managed by `mjolnir-entrypoint.sh`.
We could have used another Dockerfile for the appservice,
extending the existing one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
A list of followup issues can be found here https://github.com/issues?q=is%3Aopen+is%3Aissue+author%3AGnuxie+archived%3Afalse+label%3AA-Appservice.
Somewhat relevant and squashed commit messages(regrettably squashing because frankly these won't make sense in isolation):
* draft widget backend
* add `managementRoomId` to `provisionNewMjolnir`
* remove ratelimits from appservice mjolnirs
* add /join endpoint to api backend
* tighter guard around room type in PolicyList
matrix-bot-sdk imporved the types for this
* enable esModuleInterop
* launch and use postgres in a container whilst using mx-tester
* limited access control
policy list used for access control
* Redesign initialization API of many mjolnir.
It's much harder to forget to initialize the components now that you have to in order to construct them in the first place.
* Ammend config not to clash with existing CI
this means that the appsrvice bot is now called 'mjolnir-bot' by default
which was easier than going through old code base and renaming
* Change entrypoint in Dockerfile so that we can start the appservice.
We could have used another Dockerfile for the appservice,
extending the exising one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
This solution is much simpler, backwards compatible, and conscious about the future.
Co-authored-by: gnuxie <gnuxie@element.io>
The combination of `resyncJoinedRooms`, `unprotectedWatchedListRooms`,
`explicitlyProtectedRoomIds`, `protectedJoinedRoomIds` was incomprehensible.
https://github.com/matrix-org/mjolnir/issues/370
Separating out the management of `explicitlyProtectedRoomIds`, then
making sure all policy lists have to be explicitly protected
(in either setting of `config.protectAllJoinedRooms`) will make
this code much much simpler.
We will later change the `status` command to explicitly show
which lists are watched and which are watched and protected.
The ACL unit allows you to combine an policy lists and conveniently test users and servers against them.
The main motivation for this work is provide access control on who can provision and continue to use mjolnir instances in the appservice component.
We include a new recommendation type org.matrix.mjolnir.allow which can be used with user and server entity types to create allow lists.
We have also replaced the destructing of policy lists in applyServerACL and applyMemberBans (in ProtectedRooms.ts) with calls to the AccessControlUnit.
Adding commands to add/remove allowed entities is not something i want to do at the moment.
* src/commands/CommandHandler.ts:
- fix small typo that's bugging me in `!status` response.
change "projection" -> "protection"
Signed-off-by: Than Harrison <infosecvoid@proton.me>
Signed-off-by: Than Harrison <infosecvoid@proton.me>
* Attempt to factor out protected rooms from Mjolnir.
This is useful to the appservice because it means we don't
have to wrap a Mjolnir that is designed to sync.
It's also useful if we later on want to have specific
settings per space.
It's also just a nice seperation between Mjolnir's needs while
syncing via client-server and the behaviour of syncing policy rooms.
### Things that have changed
- `ErrorCache` no longer a static class (phew), gets used by `ProtectedRooms`.
- `ManagementRoomOutput` class gets created to handle logging back to the management room.
- Responsibilities for syncing member bans and server ACL are handled by `ProtectedRooms`.
- Responsibilities for watched lists should be moved to `ProtectedRooms` if they haven't been.
- `EventRedactionQueue` is moved to `ProtectedRooms` since this needs to happen after
member bans.
- ApplyServerAcls moved to `ProtectedRooms`
- ApplyMemberBans move to `ProtectedRooms`
- `logMessage` and `replaceRoomIdsWithPills` moved to `ManagementRoomOutput`.
- `resyncJoinedRooms` has been made a little more clear, though I am concerned about how often it does run because it does seem expensive.
* ProtectedRooms is not supposed to track joined rooms.
The reason is because it is supposed to represent a specific
set of rooms to protect, not do horrible logic
for working out what rooms mjolnir is supposed to protect.
* Rework the banning and unbanning of entities in PolicyLists.
1. We keep track of the event that created a list rule so that we
can remove the rule by having a way to determine the original state key for the rule.
This is because the state key of rules can be anything and should not be
relied on by Mjolnir to unban things (which it was doing).
2. The old scheme for producing a state key was causing for some entities to escape bans
https://github.com/matrix-org/mjolnir/issues/322.
We could have used a hash or something similar, but we know that
the reason for the `rule:${entity}` scheme existed was for ease of debugging
and finding rules in devtools. So instead we have followed a scheme simalar to
bridges where the first character of an mxid is replaced with an underscore.
Everything else just gets put into the state key. Since domains can't have '@'
and room ids, aliases can't either.
3. We have stopped the need for Mjolnir to wait for the next response from sync after banning,
unbanning an entity so that we can apply ACL's sooner.
* Use PolicyList's `banEntity` method to create imported rules.
Replace acceptInvitesFromGroup with acceptInvitesFromSpace.
https://github.com/matrix-org/mjolnir/issues/125https://github.com/matrix-org/mjolnir/issues/99
acceptInvitesFromGroup was implemented with an experimental api
that was a precursor to spaces which was refereed to
as either communities or groups.
Support for communities/groups ended in Synapse 1.61.0
https://github.com/matrix-org/synapse/releases/tag/v1.61.0.
To test we just edit the config dynamically which changes how the join room listener functions
though idk, shouldn't we have just made a new mjolnir instance
for this test, or changed the config before the test started somehow?
Co-authored-by: jesopo <github@lolnerd.net>
* Stop the config being global (in almost all contexts).
* make sure unit test has a config
* Make failing word list more visible
* Only use Healthz from index.ts
Not really sure how useful it is anyways?
* banListTest would applyACL before rules appeared in `/state`.
Mjolnir will call applyServerAcls several times while a policy list is being updated, sometimes concurrently. This means a request to set a server ACL in a room which has old data can finish after a more recent recent request with the correct ACL. This means that the old ACL gets applied to the rooms (for a moment).
This is a follow up from https://github.com/matrix-org/mjolnir/pull/314/commits/551065815e65e6117d7e8105afd6f5da9b87c8af
* Only allow one invocation of applyServerAcls at a time as to not conflict with each other by using a promise chain.
We don't use the throttle queue because we don't want to be blocked by other background tasks.
We don't make another throttle queue because we don't want throttling and we don't want to delay the ACL application, which can happen even with throttle time of 0.
Towards opinions in PolicyLists.
This changeset is part of an ongoing effort to implement "opinions"
within policy lists, as per MSC3847.
For the time being:
- we rename BanList into PolicyList;
- we cleanup a little dead code;
- we replace a few `string`s with `enum`;
- `ListRule` becomes an abstract class with two concrete subclasses `ListRuleBan` and `ListRuleOpinion`.
They have been in the spec for ~3 years now and most mjolnirs should be
able to handle them. Let's use the stable endpoint now, so that other
moderation tools don't need to implement the legacy identifier to handle
new bans at some point.
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* Remove debug leftovers from a test.
This is really terrible and has meant whenever anyone has run `yarn test:integration` they have only been running this test.
💀💀💀https://www.youtube.com/watch?v=jmX-tzSOFE0
* Set a default timeout for integration tests that is 5 minutes long.
Seriously, I don't think there is much to gain by making people guess
a reasnoble time for a test to complete in all the time, especially
with how much Synapse changes in response time and all of the machines
involved in running these tests.
* Warn when giving up on being throttled
* For some reason it takes longer for events to appear in /state
no i am not going to track down why yet.
* Rate limiting got a lot more aggresive.
https://github.com/matrix-org/synapse/pull/13018
Rate limiting in Synapse used to reset the burst count and remove
the backoff when you were spamming continuously, now it doesn't.
Ideally we'd rewrite the rate limiting logic to back off for longer
than suggested so we could get burst again, but for now
lets just unblock CI by reducing the number of events we send in these
tests.
This pull requests adds for glob support in the `!mjolnir kick` command.
## Example
```
!mjolnir kick @*:domain.tld <reason> --force
```
This command will kick every user having a mxid matching `domain.tld`.
You can also still kick a particular user:
```
!mjolnir kick @user:domain.tld <reason>
```
## Tests:
Tested on the Furry Tech room (`vRGLvqJYlFvzpThbxI:matrix.org`) after a spam wave.
It kicked over 13k bots in a matter of hours without putting too much strain on the homeserver.
For instance, this command was matching `@spam*`:
Signed-off-by: Jae Lo Presti <me@jae.fi>
* Remove the need to call `/initialSync` in `getMessagesByUserIn`.
At the moment we call `/initialSync` to give a `from` token to `/messages`.
In this PR we instead do not provide a `from` token when calling `/messages`,
which has recently been permitted in the spec
Technically this is still unstable in the spec
https://spec.matrix.org/unstable/client-server-api/#get_matrixclientv3roomsroomidmessageshttps://github.com/matrix-org/matrix-spec/pull/1002
Synapse has supported this for over 2 years and Element web depends on it for threads.
https://github.com/matrix-org/matrix-js-sdk/pull/2065
Given that redactions are super heavy in Mjolnir already and have been reported
as barely functional on matrix.org I believe we should also adopt this approach as
if for some reason the spec did change before the next release (1.3) (extremely unlikely) we can revert this commit.
* Have the callback work on the packed event Obj.
And also explicitly pack the Mjolnir object inside the event.
* Add VS ignores.
* Commit suggested review changes.
- Add bind to callback
- Add type cast for event packed object
* Fix missing parenthesis.
* Don't pass Mjolnir obj to handleReport().
Co-authored-by: David Teller <davidt@element.io>
There is no reason to call process.exit() from `index.ts` or in `Mjolnir.start()` because
https://nodejs.org/api/process.html#warning-using-uncaughtexception-correctly
>The 'uncaughtException' event is emitted when an uncaught JavaScript exception bubbles all the way back to the event loop. By default, Node.js handles such exceptions by printing the stack trace to stderr and exiting with code 1, overriding any previously set process.exitCode. Adding a handler for the 'uncaughtException' event overrides this default behaviour.
* standard protection consequences
* add integration test to make sure good users aren't banned
* the less far `event` propagates, the better
* better document consequence.ts
* improve innocent user integration test
* switch to room.event emit
Gnuxie
