Commit Graph
17471 Commits
Author SHA1 Message Date
17fa4c7ca7 Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-04 15:06:51 +01:00
reivilibreandGitHub 58f61f10f7 Catch-up after Federation Outage (split, 1) (#8230)
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-09-04 12:22:23 +01:00
e351298444 Fix type signature in simple_select_one_onecol and friends (#8241)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-09-04 12:02:29 +01:00
Patrick ClokeandGitHub c619253db8 Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan AbolivierandGitHub 9f8abdcc38 Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Erik JohnstonandGitHub be16ee59a8 Add type hints to more handlers (#8244) 2020-09-03 22:02:29 +01:00
4535e849d7 Remove obsolete order field in send_new_transaction (#8245)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-03 19:23:07 +01:00
Richard van der HoffandGitHub f97f9485ee Split fetching device keys and signatures into two transactions (#8233)
I think this is simpler (and moves stuff out of the db threads)
2020-09-03 18:27:26 +01:00
Erik JohnstonandGitHub 208e1d3eb3 Fix typing for @cached wrapped functions (#8240)
This requires adding a mypy plugin to fiddle with the type signatures a bit.
2020-09-03 15:38:32 +01:00
Patrick Cloke 15c35c250c Remove useless changelog about reverting a #8239. 2020-09-03 09:47:41 -04:00
Patrick ClokeandGitHub 2aa127c207 Revert pinning of setuptools (#8239) 2020-09-03 09:45:36 -04:00
Erik JohnstonandGitHub 5bfc79486d Fix typing for SyncHandler (#8237) 2020-09-03 12:54:10 +01:00
Richard van der HoffandGitHub 6f6f371a87 wrap _get_e2e_device_keys_and_signatures_txn in a non-txn method (#8231)
We have three things which all call `_get_e2e_device_keys_and_signatures_txn`
with their own `runInteraction`. Factor out the common code.
2020-09-03 11:50:49 +01:00
Patrick ClokeandGitHub c8758cb72f Add an overload for simple_select_one_onecol_txn. (#8235) 2020-09-02 15:03:12 -04:00
Patrick ClokeandGitHub 912e024913 Convert runInteraction to async/await (#8156) 2020-09-02 13:11:02 -04:00
Erik JohnstonandGitHub 112266eafd Add StreamStore to mypy (#8232) 2020-09-02 17:52:38 +01:00
Brendan AbolivierandGitHub 5a1dd297c3 Re-implement unread counts (again) (#8059) 2020-09-02 17:19:37 +01:00
Richard van der HoffandGitHub 0d4f614fda Refactor _get_e2e_device_keys_for_federation_query_txn (#8225)
We can use the existing `_get_e2e_device_keys_and_signatures_txn` instead of
creating our own txn function
2020-09-02 15:53:26 +01:00
Erik JohnstonandGitHub 82c1ee1c22 Add experimental support for sharding event persister. (#8170)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
b257c788c0 Add /user/{user_id}/shared_rooms/ api (#7785)
* Add shared_rooms api

* Add changelog

* Add .

* Wrap response in {"rooms": }

* linting

* Add unstable_features key

* Remove options from isort that aren't part of 5.x

`-y` and `-rc` are now default behaviour and no longer exist.

`dont-skip` is no longer required

https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020

* Update imports to make isort happy

* Add changelog

* Update tox.ini file with correct invocation

* fix linting again for isort

* Vendor prefix unstable API

* Fix to match spec

* import Codes

* import Codes

* Use FORBIDDEN

* Update changelog.d/7785.feature

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Implement get_shared_rooms_for_users

* a comma

* trailing whitespace

* Handle the easy feedback

* Switch to using runInteraction

* Add tests

* Feedback

* Seperate unstable endpoint from v2

* Add upgrade node

* a line

* Fix style by adding a blank line at EOF.

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Tulir Asokan <tulir@maunium.net>

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update UPGRADE.rst

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Fix UPGRADE/CHANGELOG unstable paths

unstable unstable unstable

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
2020-09-02 13:18:40 +01:00
Patrick ClokeandGitHub 9356656e67 Do not try to store invalid data in the stats table (#8226) 2020-09-02 07:59:39 -04:00
Patrick ClokeandGitHub d250521cf5 Convert the main methods run by the reactor to async. (#8213) 2020-09-02 07:44:50 -04:00
Richard van der HoffandGitHub abeab964d5 Make _get_e2e_device_keys_and_signatures_txn return an attrs (#8224)
this makes it a bit clearer what's going on.
2020-09-02 11:47:26 +01:00
Patrick ClokeandGitHub b939251c37 Fix errors when updating the user directory with invalid data (#8223) 2020-09-01 13:02:41 -04:00
Brendan AbolivierandGitHub b5133dd97f Explain better what GDPR-erased means (#8189)
Fixes https://github.com/matrix-org/synapse/issues/8185
2020-09-01 16:31:59 +01:00
Patrick ClokeandGitHub 37db6252b7 Convert additional databases to async/await part 3 (#8201) 2020-09-01 11:04:17 -04:00
Patrick ClokeandGitHub 7d103a594e Convert appservice code to async/await. (#8207) 2020-09-01 11:03:49 -04:00
Richard van der HoffandGitHub 5615eb5cb4 Rename _get_e2e_device_keys_txn (#8222)
... to `_get_e2e_device_keys_and_signatures_txn`, to better reflect what it
does.
2020-09-01 16:02:17 +01:00
Patrick ClokeandGitHub 54f8d73c00 Convert additional databases to async/await (#8199) 2020-09-01 09:21:48 -04:00
Patrick ClokeandGitHub 5bf8e5f55b Convert the well known resolver to async (#8214) 2020-09-01 09:15:22 -04:00
Patrick ClokeandGitHub da77520cd1 Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
Erik JohnstonandGitHub bbb3c8641c Make MultiWriterIDGenerator work for streams that use negative stream IDs (#8203)
This is so that we can use it for the backfill events stream.
2020-09-01 13:36:25 +01:00
Patrick ClokeandGitHub 318245eaa6 Do not install setuptools 50.0. (#8212)
This is due to compatibility issues with old Python versions.
2020-09-01 08:16:58 -04:00
Richard van der HoffandGitHub aa07c37cf0 Move and rename get_devices_with_keys_by_user (#8204)
* Move `get_devices_with_keys_by_user` to `EndToEndKeyWorkerStore`

this seems a better fit for it.

This commit simply moves the existing code: no other changes at all.

* Rename `get_devices_with_keys_by_user`

to better reflect what it does.

* get_device_stream_token abstract method

To avoid referencing fields which are declared in the derived classes, make
`get_device_stream_token` abstract, and define that in the classes which define
`_device_list_id_gen`.
2020-09-01 12:41:21 +01:00
Richard van der HoffandGitHub 45e8f7726f Rename get_e2e_device_keys to better reflect its purpose (#8205)
... and to show that it does something slightly different to
`_get_e2e_device_keys_txn`.

`include_all_devices` and `include_deleted_devices` were never used (and
`include_deleted_devices` was broken, since that would cause `None`s in the
result which were not handled in the loop below.

Add some typing too.
2020-08-29 00:14:17 +01:00
Richard van der Hoff 8027166dd5 Add a comment about _LimitedHostnameResolver 2020-08-29 00:06:00 +01:00
Patrick ClokeandGitHub d2ac767de2 Convert ReadWriteLock to async/await. (#8202) 2020-08-28 16:47:11 -04:00
Andrew Morgan b4826d6eb1 Fix incorrect return signature 2020-08-28 17:39:48 +01:00
Erik JohnstonandGitHub 3b4556cf87 Fix wait_for_stream_position for multiple waiters. (#8196)
This fixes a bug where having multiple callers waiting on the same
stream and position will cause it to try and compare two deferreds,
which fails (due to the sorted list having an entry of `Tuple[int,
Deferred]`).
2020-08-28 17:12:45 +01:00
Patrick ClokeandGitHub d58fda99ff Convert event_push_actions, registration, and roommember datastores to async (#8197) 2020-08-28 11:34:50 -04:00
Richard van der HoffandGitHub 22b926c284 Only return devices with keys from /federation/v1/user/devices/ (#8198)
There's not much point in returning all the others, and some people have a
silly number of devices.
2020-08-28 15:59:28 +01:00
Patrick ClokeandGitHub aec7085179 Convert state and stream stores and related code to async (#8194) 2020-08-28 09:37:55 -04:00
Patrick ClokeandGitHub b055dc9322 Ensure that the OpenID Connect remote ID is a string. (#8190) 2020-08-28 08:56:36 -04:00
Patrick ClokeandGitHub 5c03134d0f Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
Patrick ClokeandGitHub d5e73cb6aa Define StateMap as immutable and add a MutableStateMap type. (#8183) 2020-08-28 07:28:53 -04:00
Andrew MorganandGitHub 2c2e649be2 Move and refactor LoginRestServlet helper methods (#8182)
This is split out from https://github.com/matrix-org/synapse/pull/7438, which had gotten rather large.

`LoginRestServlet` has a couple helper methods, `login_submission_legacy_convert` and `login_id_thirdparty_from_phone`. They're primarily used for converting legacy user login submissions to "identifier" dicts ([see spec](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-login)). Identifying information such as usernames or 3PID information used to be top-level in the login body. They're now supposed to be put inside an [identifier](https://matrix.org/docs/spec/client_server/r0.6.1#identifier-types) parameter instead.

#7438's purpose is to allow using the new identifier parameter during User-Interactive Authentication, which is currently handled in AuthHandler. That's why I've moved these helper methods there. I also moved the refactoring of these method from #7438 as they're relevant.
2020-08-28 09:58:17 +01:00
Patrick ClokeandGitHub e00816ad98 Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Patrick ClokeandGitHub b49a5b9307 Convert stats and related calls to async/await (#8192) 2020-08-27 17:24:37 -04:00
Patrick ClokeandGitHub b71d4a094c Convert simple_delete to async/await. (#8191) 2020-08-27 14:16:41 -04:00
Patrick ClokeandGitHub 9b7ac03af3 Convert calls of async database methods to async (#8166) 2020-08-27 13:38:41 -04:00