We've done this to make the implementation a lot cleaner and
managable. However, we've taken the opportunity to simplify
all of the client code that would use this method.
Some of these simplifications might come at a small cost.
Updating server ACL, member bans and checking power levels
would all be clearly titled as seperate checks.
However, it should still be obvious what has gone wrong,
since any error would have to give a more detailed explanation.
And not everything is going to fail all at once (and if it does,
there are bigger issues).
a lot cleaner
https://github.com/Gnuxie/Draupnir/pull/85
A bizarre contraption.
The ErrorCache was seemingly introduced to reduce the number of errors
in the management room.
https://github.com/matrix-org/mjolnir/commit/82214c6cd88d83abed05fec4a871a874e6e0265b
It makes sense why it was added if you consider that many admins
will run Draupnir without giving it the permission to manage server
ACL in its rooms. Though, I'm not sure why then you would add the error
cache rather than properly supporting that use case.
So perhaps there were other reasons.
Either way, what is drawing the line for me is that the ErrorCache
will suppress any error within rooms that is not a permission error,
if there was any error that was not a permission error within a
15 minute window.
Considering the typical Draupnir will not even sync
for hours at a time, even in large communities.
It does present a problem for rooms with a lot of join/leave events.
I think that's probably why the error cache was added.
Ahh, well, fuck.
Well what is the real solution to this?
The real solution when the kind is acl is to allow the bot to run
without applying ACLs.
Ok fine but, hey wait a minute.
Why would there be any other kind of persistent error
when banning someone that would be unimportant enough to silently
hide in an ErrorCache??
IDK let's just add an opttion to disable server ACL,
since they might want to use another tool for that anyhow.
Out of scope for the current work though.
https://github.com/Gnuxie/Draupnir/pull/85
This is actually really bad.
For multiple reasons.
The best way for this to be avoided is to drop everything
and reload it when the account data for watched lists is changed.
Then there isn't a situation where you have to inform anyone
about a change in what lists are being watched.
Canonicalise the existence of the "admin room" for managing the appservice and Draupnir instances
* Add utilities for managing users in the admin room
* Merge the appservice admin room and access control list.
The majority of admins will need to use the draupnir admin commands
to manage the list.
* Utility methods for creating generic rules in PolicyLists.
* Commands for managing appservice users.
* basic ban conversion, but i have better ideas
* Still very WIP on CLIM prompt-for-accept semantics.
* Introduce promotable streams.
This allows parameters to specify details to prompt for missing
arguments
and allow for interactive commands.
* Changes that were made before PolicyListManager that no longer make sense
We don't want the default list anymore since we're just going to prompt
with the lists that they can choose from.
* Fix semantics of TagDynamicEnvironment.
Bind and write were wrong and bind was binding to the node name
instead of the variable name.
* The JSX factory can render presentation types to DocumentNodes, unsure if this is the right
move yet but it works
* Attributes for anchor nodes now render properly
* Ban command prompts are working!!!!
* Stub AppserviceBotEmitter.
There isn't much we can do right now until there is time to work on
https://github.com/Gnuxie/Draupnir/issues/13.
* Combine ban/unban syntax.
* Remove old UnbanBanCommands.
WARNING: There is a major difference in that the ban command no longer supports
globs, I don't think?
* Activate new unban command.
* The presentation type boolean will have to be just a string for now.
I don't think it makes sense to read them into actual booleans.
* configurable defaults for ban reason.
So as a history lesson.
The Matrix Bot SDK uses the npm library "requests".
When there was a http error, matrix-bot-sdk
would literally throw the response object.
This would be a horrible 1000+line long thing to accidentally
be logged to the console via node's inspect.
Though it was inevitable since you can't be sure every catch
was handling the error correctly. Irregardless, the solution
developed at Element was to create an error object
that had concise details.
This was great, however, within the matrix-bot-sdk there is
[this](https://github.com/Half-Shot/matrix-bot-sdk/blob/f58d7ea6e24d1db8b9b8009dea4cd97cbff54d0c/src/http.ts#L66)
awful line of code which logs every http error as error using the
matrix-bot-sdk logger.
This wastes so much log space and causes alarm fatigue,
rather than muting the module, the action instead taken
was to redact stack traces from http errors.
This was not a good idea.
Eventually matrix-bot-sdk was updated to have a MatrixError type
when a request was performed via the client-server api that had an
error.
matrix-appservice-bridge depends upon this and so Mjolnir now needs
to be updated to throw MatrixError's.
We have gone a step further in this commit and also muted
the matrix-bot-sdk http module, to stop the alarm fatigue problem
https://github.com/turt2live/matrix-bot-sdk/pull/158
This will want splitting out for other matrix bots to use eventually.
MatrixInterfaceCommand needs decoupling from Mjolnir and then it will
be added too.
This started out as just a way to find out why mjolnir was syncing with lists several times for each update to a policy list.
The main changes are
- Verbosity was irrelevant to the sync command but for some reason was an option.
Unfortunately all this did was suppress whether to tell you when it had finished, meaning it wouldn't
when verbose logging was disabled. Historically this was probably a parameter that got passed through
to applyServerAcl/applyUserBans, which can be horribly verbose, but they access the config directly.
- Stop emitting `'PolicyList.update'` when there are no changes.
- Include a revision ID for the `'PolicyList.update'`method and event.
- Use the revision ID in the `ProtectedRoomsSet` so that we don't unnecessarily resynchronize all rooms when the `'PolicyList.update'` event is received. Though not when the `sync` command is used. Since this is supposed to `sync` in the case when there is a state reset or otherwise or the user has changed some room settings.
- insert an await lock around the `PolicyList.update` method to avoid a race condition where a call can be started and finished within the extent of an existing call (via another task, this can happen if the server is slow with handling one request). `PolicyList.udpate` now has a helper that is synchronous to be called directly after requesting the room state. The reason for this is to enforce that no one `await`s while updating the policy list's cache of rules. Which is important because it is one of the biggest methods that I tolerate and visually checking for `await` is impossible.
- The revision ID uses a ULID, but this is unnecessary and could have just been a "dumb counter".
closes https://github.com/matrix-org/mjolnir/issues/447
For some reason we were relying on a mjolnir listening to
`'PolicyList.batch'` to update policy lists.
This was exposing an implementation detail to Mjolnir
and including it as part of the implementation of
`PolicyList.updateForEvent()` which is supposed to cause
the `PolicyList` to update (eventually).
I am confident this was because of a need before batching was
introduced to get the changes to a policy list directly
from the method call to `PolicyList.update()`, whereas
now you can just listen to `PolicyList.update`.
The `'PolicyList.batch'` event has now been removed
and the PolicyList event batcher (`UpdateBatcher`)
now calls `PolicyList.update()` internally.
* Refactor Matrix event listener in Mjolnir and ManagedMjolnir.
closes https://github.com/matrix-org/mjolnir/issues/411.
Issue #411 says that we have to be careful about room.join,
but this was before we figured how to make matrix-appservice-bridge
echo events sent by its own intents.
* Remove MatrixClientListener since it isn't actually needed.
* Protect which config values can be used for ManagedMjolnirs.
* Introduce MatrixSendClient
so listeners aren't accidentally added to a MatrixClient instead
of MatrixEmitter.
* doc
* Move provisioned mjolnir config to src/config.
This just aids maintance so whenever someone goes to change the config
of the bot they will see this and update it.
* doc for matrix intent listener.
Mjolnir can now be run as an application service,
meaning it will host multiple independent mjolnirs that can be requested by users.
If the user is on the same homeserver as the appservice is deployed on,
then they can provision a mjolnir via a widget https://github.com/matrix-org/mjolnir-widget.
Otherwise they can invite the appservice bot to a room they want to protect.
This will create them a mjolnir, a management room and a policy list.
The appservice shares the same docker image as the bot,
but is started slightly differently by specifying "appservice"
as the first argument to docker run (this s managed by `mjolnir-entrypoint.sh`.
We could have used another Dockerfile for the appservice,
extending the existing one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
A list of followup issues can be found here https://github.com/issues?q=is%3Aopen+is%3Aissue+author%3AGnuxie+archived%3Afalse+label%3AA-Appservice.
Somewhat relevant and squashed commit messages(regrettably squashing because frankly these won't make sense in isolation):
* draft widget backend
* add `managementRoomId` to `provisionNewMjolnir`
* remove ratelimits from appservice mjolnirs
* add /join endpoint to api backend
* tighter guard around room type in PolicyList
matrix-bot-sdk imporved the types for this
* enable esModuleInterop
* launch and use postgres in a container whilst using mx-tester
* limited access control
policy list used for access control
* Redesign initialization API of many mjolnir.
It's much harder to forget to initialize the components now that you have to in order to construct them in the first place.
* Ammend config not to clash with existing CI
this means that the appsrvice bot is now called 'mjolnir-bot' by default
which was easier than going through old code base and renaming
* Change entrypoint in Dockerfile so that we can start the appservice.
We could have used another Dockerfile for the appservice,
extending the exising one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
This solution is much simpler, backwards compatible, and conscious about the future.
Co-authored-by: gnuxie <gnuxie@element.io>
The ACL unit allows you to combine an policy lists and conveniently test users and servers against them.
The main motivation for this work is provide access control on who can provision and continue to use mjolnir instances in the appservice component.
We include a new recommendation type org.matrix.mjolnir.allow which can be used with user and server entity types to create allow lists.
We have also replaced the destructing of policy lists in applyServerACL and applyMemberBans (in ProtectedRooms.ts) with calls to the AccessControlUnit.
Adding commands to add/remove allowed entities is not something i want to do at the moment.
* Rework the banning and unbanning of entities in PolicyLists.
1. We keep track of the event that created a list rule so that we
can remove the rule by having a way to determine the original state key for the rule.
This is because the state key of rules can be anything and should not be
relied on by Mjolnir to unban things (which it was doing).
2. The old scheme for producing a state key was causing for some entities to escape bans
https://github.com/matrix-org/mjolnir/issues/322.
We could have used a hash or something similar, but we know that
the reason for the `rule:${entity}` scheme existed was for ease of debugging
and finding rules in devtools. So instead we have followed a scheme simalar to
bridges where the first character of an mxid is replaced with an underscore.
Everything else just gets put into the state key. Since domains can't have '@'
and room ids, aliases can't either.
3. We have stopped the need for Mjolnir to wait for the next response from sync after banning,
unbanning an entity so that we can apply ACL's sooner.
* Use PolicyList's `banEntity` method to create imported rules.
Towards opinions in PolicyLists.
This changeset is part of an ongoing effort to implement "opinions"
within policy lists, as per MSC3847.
For the time being:
- we rename BanList into PolicyList;
- we cleanup a little dead code;
- we replace a few `string`s with `enum`;
- `ListRule` becomes an abstract class with two concrete subclasses `ListRuleBan` and `ListRuleOpinion`.
They have been in the spec for ~3 years now and most mjolnirs should be
able to handle them. Let's use the stable endpoint now, so that other
moderation tools don't need to implement the legacy identifier to handle
new bans at some point.
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* Test for batching ACL.
* Batch events from sync within BanList.
* Introduce the BanList.batch event to the BanList emitter to let Mjolnir sync after new events have been added from sync.
Fixes#203
This is an experimental ruleserver that will serve the combined rules from
the active policy lists to a Synapse module over a web api.
This makes it easier to communicate changes in policy lists to Synapse workers
that do not have an immediate view over all of the policy rooms at
the same time.
This also allows us to express moderation actions to the homeserver
beyond what is currently expressible via MSC2313 policy
lists.
We do this so that there is only one source of truth for which rules
are active and it simplifies de-duplicating rules of conflicting event
types (e.g. m.room.rule.user vs m.policy.rule.user).
The reason for doing this is because otherwise there may be duplicate
rules under different state types for the same entity.
This simplifies the process of modifying or invalidating rules affecting
an entity because the rule with the most recent type will always be
preferred.
Only shows changes to lists made by other accounts (than the one used by Mjolnir).
Displays when rules are added, removed and modified by either replacing the state event or redacting them.
gnuxie