Travis Ralston
fe742653ff
Add an API to list changes to quarantine state of media ( #19558 )
...
Fixes https://github.com/element-hq/synapse/issues/19352
(See issue for history of this feature and previous PRs)
> First, a [naive
implementation](https://github.com/element-hq/synapse/pull/19268 ) of the
endpoint was introduced, but it quickly ran into [performance issues on
query](https://github.com/element-hq/synapse/pull/19312 ) and [long
startup times](https://github.com/element-hq/synapse/issues/19349 ),
leading to its
[removal](https://github.com/element-hq/synapse/pull/19351 ). It also
didn't actually work, and would fail to expose media when it was
"unquarantined", so a [partial
fix](https://github.com/element-hq/synapse/pull/19308 ) was attempted,
where the suggested direction is to use a
[stream](https://element-hq.github.io/synapse/latest/development/synapse_architecture/streams.html#cheatsheet-for-creating-a-new-stream )
instead of a timestamp column.
This PR re-introduces the API building on the previous feedback:
* Adds a stream which tracks when media becomes (un)quarantined.
* Runs a background update to capture already-quarantined media.
* Adds a new admin API to return rows from the stream table.
We track both quarantine and unquarantine actions in the stream to allow
downstream consumers to process the records appropriately. Namely, to
allow our Synapse exchange in HMA to remove hashes for unquarantined
media (use case further explained in the
[issue](https://github.com/element-hq/synapse/issues/19352 )).
**Note**: This knowingly does not capture all cases of media being
quarantined. Other call sites are lower priority for T&S, and can be
addressed in a future PR. ~~An issue will be created after this PR is
merged to track those sites.~~
https://github.com/element-hq/synapse/issues/19672
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct (run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: turt2live <1190097+turt2live@users.noreply.github.com >
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com >
Co-authored-by: Eric Eastwood <erice@element.io >
2026-04-09 11:41:31 -06:00
Eric Eastwood
5a9ca1e3d9
Introduce Clock.call_when_running(...) to include logcontext by default ( #18944 )
...
Introduce `Clock.call_when_running(...)` to wrap startup code in a
logcontext, ensuring we can identify which server generated the logs.
Background:
> Ideally, nothing from the Synapse homeserver would be logged against the `sentinel`
> logcontext as we want to know which server the logs came from. In practice, this is not
> always the case yet especially outside of request handling.
>
> Global things outside of Synapse (e.g. Twisted reactor code) should run in the
> `sentinel` logcontext. It's only when it calls into application code that a logcontext
> gets activated. This means the reactor should be started in the `sentinel` logcontext,
> and any time an awaitable yields control back to the reactor, it should reset the
> logcontext to be the `sentinel` logcontext. This is important to avoid leaking the
> current logcontext to the reactor (which would then get picked up and associated with
> the next thing the reactor does).
>
> *-- `docs/log_contexts.md`
Also adds a lint to prefer `Clock.call_when_running(...)` over
`reactor.callWhenRunning(...)`
Part of https://github.com/element-hq/synapse/issues/18905
2025-09-22 10:27:59 -05:00
reivilibre
a31d53b28f
Use twisted.internet.testing module in tests instead of deprecated twisted.test.proto_helpers. ( #18728 )
...
Follows: #18727
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org >
2025-07-30 12:32:10 +01:00
Erik Johnston
23740eaa3d
Correctly mention previous copyright ( #16820 )
...
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
2024-01-23 11:26:48 +00:00
Patrick Cloke
8e1e62c9e0
Update license headers
2023-11-21 15:29:58 -05:00
Patrick Cloke
ab3f1b3b53
Convert simple_select_one_txn and simple_select_one to return tuples. ( #16612 )
2023-11-09 11:13:31 -05:00
Patrick Cloke
cfb6d38c47
Remove remaining usage of cursor_to_dict. ( #16564 )
2023-10-31 13:13:28 -04:00
V02460
84f441f88f
Prepare unit tests for Python 3.12 ( #16099 )
2023-08-25 15:05:10 -04:00
Patrick Cloke
3ac412b4e2
Require types in tests.storage. ( #14646 )
...
Adds missing type hints to `tests.storage` package
and does not allow untyped definitions.
2022-12-09 12:36:32 -05:00
Petr Vaněk
bb20113c8f
Remove obsolete RoomEventsStoreTestCase ( #13200 )
...
All tests are prefixed with `STALE_` and therefore they are silently
skipped. They were moved to `STALE_` in version `v0.5.0` in commit
2fcce3b3c5arkamar@atlas.cz >
2022-07-07 13:47:26 +01:00
Erik Johnston
e3163e2e11
Reduce the amount of state we pull from the DB ( #12811 )
2022-06-06 09:24:12 +01:00
Erik Johnston
1e453053cb
Rename storage classes ( #12913 )
2022-05-31 12:17:50 +00:00
Patrick Cloke
02d708568b
Replace assertEquals and friends with non-deprecated versions. ( #12092 )
2022-02-28 07:12:29 -05:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Jonathan de Jong
89cfc3dd98
[pyupgrade] tests/ ( #10347 )
2021-07-13 11:43:15 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl >`
2021-04-14 15:34:27 +01:00
Patrick Cloke
e7b769aea1
Convert storage test cases to HomeserverTestCase. ( #9736 )
2021-04-06 07:21:02 -04:00
Patrick Cloke
4c6c56dc58
Convert simple_select_one and simple_select_one_onecol to async ( #8162 )
2020-08-26 07:19:32 -04:00
Patrick Cloke
b3a97d6dac
Convert some of the data store to async. ( #7976 )
2020-07-30 07:20:41 -04:00
Patrick Cloke
3345c166a4
Convert storage layer to async/await. ( #7963 )
2020-07-28 16:09:53 -04:00
Patrick Cloke
b975fa2e99
Convert state resolution to async/await ( #7942 )
2020-07-24 10:59:51 -04:00
Dirk Klimpel
491f0dab1b
Add delete room admin endpoint ( #7613 )
...
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md )" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md )" API.
Fixes : #6425
It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)#5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-14 12:36:23 +01:00
Manuel Stahl
a4a5ec4096
Add room details admin endpoint ( #7317 )
2020-05-07 15:33:07 -04:00
Erik Johnston
8df862e45d
Add rooms.room_version column ( #6729 )
...
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
Erik Johnston
3ca4c7c516
Use new EventPersistenceStore
2019-10-23 16:15:03 +01:00
Amber Brown
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
Amber Brown
99dd975dae
Run tests under PostgreSQL ( #3423 )
2018-08-13 16:47:46 +10:00
black
8b3d9b6b19
Run black.
2018-08-10 23:54:09 +10:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Mark Haines
700487a7c7
Fix flake8 warnings for tests
2016-02-19 15:34:38 +00:00
Erik Johnston
f8aae79a72
Simplify get_rooms
2016-02-03 13:24:35 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
Daniel Wagner-Hall
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Erik Johnston
7e282a53a5
Tidy up _simple_... methods
2015-03-20 15:05:10 +00:00
Mark Haines
896253e085
Factor out some of the common homeserver setup code into a
...
setup_test_homeserver function in utils.
2015-02-11 11:37:30 +00:00
Mark Haines
ada711504e
Replace hs.parse_roomalias with RoomAlias.from_string
2015-01-23 13:21:58 +00:00
Mark Haines
1c06c48ce2
Replace hs.parse_roomid with RoomID.from_string
2015-01-23 11:55:12 +00:00
Mark Haines
5759bec43c
Replace hs.parse_userid with UserID.from_string
2015-01-23 11:47:15 +00:00
Mark Haines
3891597eb3
Remove unused functions
2015-01-13 15:57:26 +00:00
Erik Johnston
3c77d13aa5
Kill off synapse.api.events.*
2014-12-16 11:29:05 +00:00
Erik Johnston
2fcce3b3c5
Remove stale tests
2014-11-05 13:43:36 +00:00
Paul "LeoNerd" Evans
bcf5121937
Neaten more of the storage layer tests with assertObjectHasAttributes; more standardisation on test layout
2014-09-17 16:58:59 +01:00
Paul "LeoNerd" Evans
b588ce920d
Unit tests for (some) room events via the RoomStore
2014-09-17 16:31:11 +01:00
Paul "LeoNerd" Evans
ba41ca45fa
Use new assertObjectHasAttributes() in tests/storage/test_room.py
2014-09-17 16:04:05 +01:00
Paul "LeoNerd" Evans
de14853237
More RoomStore tests
2014-09-17 15:33:10 +01:00
Paul "LeoNerd" Evans
e32cfed1d8
Initial pass at a RoomStore test
2014-09-15 18:41:24 +01:00
