Commit Graph
141 Commits
Author SHA1 Message Date
Eric EastwoodandGitHub 6a909aade2 Consolidate SSO redirects through /_matrix/client/v3/login/sso/redirect(/{idpId}) (#17972)
Consolidate SSO redirects through
`/_matrix/client/v3/login/sso/redirect(/{idpId})`

Spawning from
https://github.com/element-hq/sbg/pull/421#discussion_r1859497330 where
we have a proxy that intercepts responses to
`/_matrix/client/v3/login/sso/redirect(/{idpId})` in order to upgrade
them to use OAuth 2.0 Pushed Authorization Requests (PAR). Instead of
needing to intercept multiple endpoints that redirect to the
authorization endpoint, it seems better to just have Synapse consolidate
to a single flow.


### Testing strategy

1. Create a new OAuth application. I'll be using GitHub for example but
there are [many
options](https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/openid.md).
Visit https://github.com/settings/developers -> **New OAuth App**
    - Application name: `Synapse local testing`
    - Homepage URL: `http://localhost:8008`
- Authorization callback URL:
`http://localhost:8008/_synapse/client/oidc/callback`
 1. Update your Synapse `homeserver.yaml`
    ```yaml
    server_name: "my.synapse.server"
    public_baseurl: http://localhost:8008/
    listeners:
      - port: 8008
        bind_addresses: [
          #'::1',
          '127.0.0.1'
        ]
        tls: false
        type: http
        x_forwarded: true
        resources:
          - names: [client, federation, metrics]
            compress: false
    
    # SSO login testing
    oidc_providers:
      - idp_id: github
        idp_name: Github
        idp_brand: "github"  # optional: styling hint for clients
        discover: false
        issuer: "https://github.com/"
        client_id: "xxx" # TO BE FILLED
        client_secret: "xxx" # TO BE FILLED
authorization_endpoint: "https://github.com/login/oauth/authorize"
        token_endpoint: "https://github.com/login/oauth/access_token"
        userinfo_endpoint: "https://api.github.com/user"
        scopes: ["read:user"]
        user_mapping_provider:
          config:
            subject_claim: "id"
            localpart_template: "{{ user.login }}"
            display_name_template: "{{ user.name }}"
    ```
1. Start Synapse: `poetry run synapse_homeserver --config-path
homeserver.yaml`
1. Visit
`http://localhost:8008/_synapse/client/pick_idp?redirectUrl=http%3A%2F%2Fexample.com`
 1. Choose GitHub
1. Notice that you're redirected to GitHub to sign in
(`https://github.com/login/oauth/authorize?...`)

Tested locally and works:

1.
`http://localhost:8008/_synapse/client/pick_idp?idp=oidc-github&redirectUrl=http%3A//example.com`
->
1.
`http://localhost:8008/_matrix/client/v3/login/sso/redirect/oidc-github?redirectUrl=http://example.com`
->
1.
`https://github.com/login/oauth/authorize?response_type=code&client_id=xxx&redirect_uri=http%3A%2F%2Flocalhost%3A8008%2F_synapse%2Fclient%2Foidc%2Fcallback&scope=read%3Auser&state=xxx&nonce=xxx`
2024-11-29 11:26:37 -06:00
Erik JohnstonandGitHub 52af16c561 Add a short sleep if the request is rate-limited (#17210)
This helps prevent clients from "tight-looping" retrying their request.
2024-05-18 12:03:30 +01:00
V02460andGitHub 05489d89c6 Specify IP subnet literals in canonical form (#16953)
This is needed, because the netaddr package removed support for the
implicit prefix form in version 1.0.0:
https://github.com/netaddr/netaddr/pull/360
2024-03-19 17:19:12 +00:00
Patrick ClokeandGitHub 696cc9e802 Stabilize support for Retry-After header (MSC4014) (#16947) 2024-03-08 09:33:46 +00:00
Erik JohnstonandGitHub 23740eaa3d Correctly mention previous copyright (#16820)
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
2024-01-23 11:26:48 +00:00
Patrick Cloke 8e1e62c9e0 Update license headers 2023-11-21 15:29:58 -05:00
Erik JohnstonandGitHub 954921736b Refactor get_user_by_id (#16316) 2023-09-14 12:46:30 +01:00
David RobertsonandGitHub 62a1a9be52 Describe which rate limiter was hit in logs (#16135) 2023-08-30 00:39:39 +01:00
Patrick ClokeandGitHub a8a46b1336 Replace simple_async_mock with AsyncMock (#16180)
Python 3.8 has a native AsyncMock, use it instead of a custom
implementation.
2023-08-25 09:27:21 -04:00
Will HuntandGitHub 0538e3e2db Add Retry-After to M_LIMIT_EXCEEDED error responses (#16136)
Implements MSC4041 behind an experimental configuration flag.
2023-08-24 10:40:26 -04:00
Mathieu VeltenandGitHub dac97642e4 Implements admin API to lock an user (MSC3939) (#15870) 2023-08-10 09:10:55 +00:00
ShayandGitHub d0c4257f14 N + 3: Read from column full_user_id rather than user_id of tables profiles and user_filters (#15649) 2023-06-02 17:24:13 -07:00
Quentin GliechandPatrick Cloke c5cf1b421d Save the scopes in the requester 2023-05-30 09:43:06 -04:00
Quentin GliechandPatrick Cloke e2c8458bba Make the api.auth.Auth a Protocol 2023-05-30 09:43:06 -04:00
Patrick ClokeandGitHub c5d1e6d414 Properly parse event_fields in filters (#15607)
The event_fields property in filters should use the proper
escape rules, namely backslashes can be escaped with
an additional backslash.

This adds tests (adapted from matrix-js-sdk) and implements
the logic to properly split the event_fields strings.
2023-05-22 11:31:22 -04:00
ShayandGitHub 301b4156d5 Add column full_user_id to tables profiles and user_filters. (#15458) 2023-04-26 16:03:26 -07:00
3b0083c92a Use immutabledict instead of frozendict (#15113)
Additionally:

* Consistently use `freeze()` in test

---------

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-22 17:15:34 +00:00
David RobertsonandGitHub 6e6edea6c1 Properly typecheck tests.api (#14983) 2023-02-03 20:03:23 +00:00
Andrew MorganandGitHub 8d90e5f200 Add type hints to TestRatelimiter (#14885) 2023-01-21 15:59:15 +00:00
Sean QuahandGitHub e980982b59 Do not reject /sync requests with unrecognised filter fields (#14369)
For forward compatibility, Synapse needs to ignore fields it does not
recognise instead of raising an error.

Fixes #14365.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-07 13:49:31 +00:00
Patrick ClokeandGitHub 4283bd1cf9 Support filtering the /messages API by relation type (MSC3874). (#14148)
Gated behind an experimental configuration flag.
2022-10-17 11:32:11 -04:00
Quentin GliechandGitHub 3dd175b628 synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID (#13024)
Part of #13019

This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.

It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
599c403d99 Allow rate limiters to passively record actions they cannot limit (#13253)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-07-13 19:09:42 +00:00
reivilibreandGitHub 0dbdc39940 Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. (#13018) 2022-06-15 14:11:55 +00:00
Quentin GliechandGitHub fe1daad672 Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. (#12986)
This simplifies the access token verification logic by removing the `rights`
parameter which was only ever used for the unsubscribe link in email
notifications. The latter has been moved under the `/_synapse` namespace,
since it is not a standard API.

This also makes the email verification link more secure, by embedding the
app_id and pushkey in the macaroon and verifying it. This prevents the user
from tampering the query parameters of that unsubscribe link.

Macaroon generation is refactored:

- Centralised all macaroon generation and verification logic to the
  `MacaroonGenerator`
- Moved to `synapse.utils`
- Changed the constructor to require only a `Clock`, hostname, and a secret key
  (instead of a full `Homeserver`).
- Added tests for all methods.
2022-06-14 09:12:08 -04:00
Quentin GliechandGitHub 92103cb2c8 Decouple synapse.api.auth_blocking.AuthBlocking from synapse.api.auth.Auth. (#13021) 2022-06-14 09:51:15 +01:00
Patrick ClokeandGitHub 7bc08f3201 Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Patrick ClokeandGitHub c52abc1cfd Additional constants for EDU types. (#12884)
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
Patrick ClokeandGitHub 7fbf42499d Use getClientAddress instead of getClientIP. (#12599)
getClientIP was deprecated in Twisted 18.4.0, which also added
getClientAddress. The Synapse minimum version for Twisted is
currently 18.9.0, so all supported versions have the new API.
2022-05-04 14:11:21 -04:00
Patrick ClokeandGitHub 86cf6a3a17 Remove references to unstable identifiers from MSC3440. (#12382)
Removes references to unstable thread relation, unstable
identifiers for filtering parameters, and the experimental
config flag.
2022-04-12 08:42:03 -04:00
Richard van der HoffandGitHub 6c0b44a3d7 Fix PushRuleEvaluator and Filter to work on frozendicts (#12100)
* Fix `PushRuleEvaluator` to work on frozendicts

frozendicts do not (necessarily) inherit from dict, so this needs to handle
them correctly.

* Fix event filtering for frozen events

Looks like this one was introduced by #11194.
2022-02-28 17:40:24 +00:00
Patrick ClokeandGitHub 02d708568b Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
Richard van der HoffandGitHub e24ff8ebe3 Remove HomeServer.get_datastore() (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Jason RobinsonandGitHub 2560b1b6b2 Allow tracking puppeted users for MAU (#11561)
Currently when puppeting another user, the user doing the puppeting is
tracked for client IPs and MAU (if configured).

When tracking MAU is important, it becomes necessary to be possible to
also track the client IPs and MAU of puppeted users. As an example a
client that manages user creation and creation of tokens via the Synapse
admin API, passing those tokens for the client to use.

This PR adds optional configuration to enable tracking of puppeted users
into monthly active users. The default behaviour stays the same.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2022-01-12 16:09:36 +00:00
reivilibreandGitHub 17886d2603 Add experimental support for MSC3202: allowing application services to masquerade as specific devices. (#11538) 2021-12-15 10:40:52 +00:00
Patrick ClokeandGitHub a19d01c3d9 Support filtering by relations per MSC3440 (#11236)
Adds experimental support for `relation_types` and `relation_senders`
fields for filters.
2021-11-09 08:10:58 -05:00
Patrick ClokeandGitHub 94b620a5ed Use direct references for configuration variables (part 6). (#10916) 2021-09-29 06:44:15 -04:00
Jason RobinsonandGitHub fa74536384 Fix AuthBlocking check when requester is appservice (#10881)
If the MAU count had been reached, Synapse incorrectly blocked appservice users even though they've been explicitly configured not to be tracked (the default). This was due to bypassing the relevant if as it was chained behind another earlier hit if as an elif.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-09-24 10:41:18 +01:00
Patrick ClokeandGitHub 47854c71e9 Use direct references for configuration variables (part 4). (#10893) 2021-09-23 12:03:01 -04:00
Quentin GliechandGitHub bd4919fb72 MSC2918 Refresh tokens implementation (#9450)
This implements refresh tokens, as defined by MSC2918

This MSC has been implemented client side in Hydrogen Web: vector-im/hydrogen-web#235

The basics of the MSC works: requesting refresh tokens on login, having the access tokens expire, and using the refresh token to get a new one.

Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
2021-06-24 14:33:20 +01:00
Brendan AbolivierandGitHub a683028d81 Correctly ratelimit invites when creating a room (#9968)
* Correctly ratelimit invites when creating a room

Also allow ratelimiting for more than one action at a time.
2021-05-12 16:05:28 +02:00
Richard van der HoffandGitHub 7562d887e1 Change the format of access tokens away from macaroons (#5588) 2021-05-12 15:04:51 +01:00
Jonathan de JongandGitHub 4b965c862d Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick ClokeandGitHub 0b3112123d Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Erik JohnstonandGitHub 963f4309fe Make RateLimiter class check for ratelimit overrides (#9711)
This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited.

We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits.

Fixes #9663
2021-03-30 12:06:09 +01:00
Eric EastwoodandGitHub 0a00b7ff14 Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick ClokeandGitHub 74af356baf Convert additional test-cases to homeserver test case. (#9396)
And convert some inlineDeferreds to async-friendly functions.
2021-02-16 08:04:15 -05:00
Richard van der Hoff 90cf1eec44 Remove redundant mocking 2020-12-02 17:53:38 +00:00
Patrick ClokeandGitHub 30fba62108 Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Erik JohnstonandGitHub f737368a26 Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00