Commit Graph

163 Commits

Author SHA1 Message Date
Patrick Cloke a0bc9d387e
Use the proper Request in type hints. (#9515)
This also pins the Twisted version in the mypy job for CI until
proper type hints are fixed throughout Synapse.
2021-03-01 12:23:46 -05:00
Dirk Klimpel 71c9f8de6d
Add an `order_by` field to list users' media admin API. (#8978) 2021-02-22 14:38:51 -05:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke 7950aa8a27 Fix some typos. 2021-02-12 11:14:12 -05:00
David Teller b0b2cac057
Merge pull request #9150 from Yoric/develop-context
New API /_synapse/admin/rooms/{roomId}/context/{eventId}
2021-02-08 15:53:44 +01:00
Travis Ralston b60bb28bbc
Add an admin API to get the current room state (#9168)
This could arguably replace the existing admin API for `/members`, however that is out of scope of this change.

This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
2021-02-02 11:16:29 +00:00
David Teller 31d072aea0 FIXUP: linter 2021-01-28 16:53:40 +01:00
David Teller 93f84e0373 FIXUP: Making get_event_context a bit more paranoid 2021-01-28 12:31:07 +01:00
David Teller a764869623 FIXUP: Doc 2021-01-28 12:31:07 +01:00
David Teller b859919acc FIXUP: Now testing that the user is admin! 2021-01-28 12:31:07 +01:00
David Teller 10332c175c New API /_synapse/admin/rooms/{roomId}/context/{eventId}
Signed-off-by: David Teller <davidt@element.io>
2021-01-28 12:29:49 +01:00
Pankaj Yadav 2e537a0280
Check if a user is in the room before sending a PowerLevel event on their behalf (#9235) 2021-01-27 17:38:08 +00:00
Jason Robinson cee4010f94 Merge branch 'develop' into jaywink/admin-forward-extremities
# Conflicts:
#	synapse/rest/admin/__init__.py
2021-01-26 10:15:32 +02:00
Patrick Cloke 4a55d267ee
Add an admin API for shadow-banning users. (#9209)
This expands the current shadow-banning feature to be usable via
the admin API and adds documentation for it.

A shadow-banned users receives successful responses to their
client-server API requests, but the events are not propagated into rooms.

Shadow-banning a user should be used as a tool of last resort and may lead
to confusing or broken behaviour for the client.
2021-01-25 14:49:39 -05:00
Jason Robinson 8965b6cfec Merge branch 'develop' into jaywink/admin-forward-extremities 2021-01-23 21:41:35 +02:00
Dirk Klimpel c55e62548c
Add tests for List Users Admin API (#9045) 2021-01-21 09:18:46 -05:00
Patrick Cloke 3e4cdfe5d9
Add an admin API endpoint to protect media. (#9086)
Protecting media stops it from being quarantined when
e.g. all media in a room is quarantined. This is useful
for sticker packs and other media that is uploaded by
server administrators, but used by many people.
2021-01-15 11:18:09 -05:00
Dirk Klimpel 7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Jason Robinson da16d06301 Address pr feedback
* docs updates
* prettify SQL
* add missing copyright
* cursor_to_dict
* update touched files copyright years

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:43:58 +02:00
David Teller b161528fcc
Also support remote users on the joined_rooms admin API. (#8948)
For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.
2021-01-11 14:32:17 -05:00
Jason Robinson 90ad4d443a Implement clearing cache after deleting forward extremities
Also run linter.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-09 21:57:41 +02:00
Jason Robinson 85c0999bfb Add Rooms admin forward extremities DELETE endpoint
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-08 00:12:23 +02:00
Jason Robinson c91045f56c Move unknown room ID error into resolve_room_id
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-07 23:03:54 +02:00
Jason Robinson b849e46139 Add forward extremities endpoint to rooms admin API
GET /_synapse/admin/v1/rooms/<identifier>/forward_extremities now gets forward extremities for a room, returning count and the list of extremities.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-07 23:01:59 +02:00
Erik Johnston d781a81e69
Allow server admin to get admin bit in rooms where local user is an admin (#8756)
This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power.

Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2020-12-18 15:37:19 +00:00
Patrick Cloke bd30cfe86a
Convert internal pusher dicts to attrs classes. (#8940)
This improves type hinting and should use less memory.
2020-12-16 11:25:30 -05:00
Dirk Klimpel 0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Will Hunt ff1f0ee094
Call set_avatar_url with target_user, not user_id (#8872)
* Call set_avatar_url with target_user, not user_id

Fixes https://github.com/matrix-org/synapse/issues/8871

* Create 8872.bugfix

* Update synapse/rest/admin/users.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Testing

* Update changelog.d/8872.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-12-07 19:13:07 +00:00
David Florness c4675e1b24
Add additional validation for the admin register endpoint. (#8837)
Raise a proper 400 error if the `mac` field is missing.
2020-12-02 10:01:15 -05:00
Richard van der Hoff a090b86209
Add `force_purge` option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Dirk Klimpel 3f0ff53158
Remove deprecated `/_matrix/client/*/admin` endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Dirk Klimpel c3119d1536
Add an admin API for users' media statistics (#8700)
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-05 18:59:12 +00:00
Dirk Klimpel e4676bd877
Add `displayname` to Shared-Secret Registration for admins (#8722)
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
2020-11-05 13:55:45 +00:00
Dirk Klimpel 2239813278
Add an admin APIs to allow server admins to list users' pushers (#8610)
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers
2020-10-28 15:02:42 +00:00
Dirk Klimpel 9b7c28283a
Add admin API to list users' local media (#8647)
Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
2020-10-27 14:12:31 +00:00
Dirk Klimpel 66e6801c3e
Split admin API for reported events into a detail and a list view (#8539)
Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-10-26 18:16:37 +00:00
Dirk Klimpel 49d72dea2a
Add an admin api to delete local media. (#8519)
Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.
2020-10-26 17:02:28 +00:00
Dirk Klimpel 4ac3a8c5dc
Fix a bug in the joined_rooms admin API (#8643)
If the user was not in any rooms then the API returned the same error
as if the user did not exist.
2020-10-26 12:25:48 -04:00
Dirk Klimpel 913f8a06e4
Add field `total` to device list in admin API (#8644) 2020-10-26 14:07:51 +00:00
Patrick Cloke c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff f31f8e6319
Remove stream ordering from Metadata dict (#8452)
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.

This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Erik Johnston 7941372ec8
Make token serializing/deserializing async (#8427)
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Erik Johnston ea70f1c362
Various clean ups to room stream tokens. (#8423) 2020-09-29 21:48:33 +01:00
Dirk Klimpel 4da01f9c61
Admin API for reported events (#8217)
Add an admin API to read entries of table `event_reports`. API: `GET /_synapse/admin/v1/event_reports`
2020-09-22 18:15:04 +01:00
Dirk Klimpel d688b4bafc
Admin API for querying rooms where a user is a member (#8306)
Add a new admin API `GET /_synapse/admin/v1/users/<user_id>/joined_rooms` to
list all rooms where a user is a member.
2020-09-18 15:26:36 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke c3c9732c53
Use admin_patterns for all admin APIs. (#8331)
This reduces duplication of the admin prefix in regular expressions.
2020-09-17 07:04:15 -04:00
Manuel Stahl 97962ad17b
Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
* Search in columns 'name' and 'displayname' in the admin users endpoint

Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-08-25 14:18:14 +01:00
Patrick Cloke e259d63f73
Stop shadow-banned users from sending invites. (#8095) 2020-08-20 15:07:42 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Dirk Klimpel e866e3b896
Add an option to disable purge in delete room admin API (#7964)
Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete```
Fixes: #3761

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-28 20:08:23 +01:00
Michael Albert fff483ea96
Add admin endpoint to get members in a room. (#7842) 2020-07-16 16:43:23 -04:00
Patrick Cloke 8c7d0f163d
Allow accounts to be re-activated from the admin APIs. (#7847) 2020-07-15 11:00:21 -04:00
Dirk Klimpel 491f0dab1b
Add delete room admin endpoint (#7613)
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API.

Fixes: #6425 

It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105)

Related to:
- #5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-07-14 12:36:23 +01:00
Patrick Cloke 71cccf1593
Additional configuration options for auto-join rooms (#7763) 2020-06-30 15:41:36 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff eea124370b
Fix type information on `assert_*_is_admin` methods (#7645)
These things don't return Deferreds.
2020-06-05 14:33:49 +01:00
Dirk Klimpel 908f9e2d24
Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263) 2020-06-05 13:08:49 +01:00
Dirk Klimpel 2970ce8367
Add device management to admin API (#7481)
- Admin is able to
  - change displaynames
  - delete devices
  - list devices
  - get device informations

Fixes #7330
2020-06-05 13:07:22 +01:00
Dirk Klimpel 901b1fa561
Email notifications for new users when creating via the Admin API. (#7267) 2020-06-01 15:34:33 +01:00
Erik Johnston e5c67d04db
Add option to move event persistence off master (#7517) 2020-05-22 16:11:35 +01:00
Erik Johnston 1531b214fc
Add ability to wait for replication streams (#7542)
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).

Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.

People probably want to look at this commit by commit.
2020-05-22 14:21:54 +01:00
Patrick Cloke 9dc6f3075a
Hash passwords earlier in the password reset process (#7538)
This now matches the logic of the registration process as modified in
56db0b1365 / #7523.
2020-05-20 09:48:03 -04:00
Patrick Cloke ee421e5244 Synapse 1.13.0rc3 (2020-05-18)
Bugfixes:
 
 - Hash passwords as early as possible during registration. #7523
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl7CpGYACgkQM/xY9qcR
 MEhVixAAk2hDWVXxbGzUk2LmfiIsFA2eV55sw+VqEw0eRfe1d/mP6aH75VmTt3pw
 IymZUVxDXdbTnPNPw+ldyGhzu9C6JJjXnNRBZnIkR5vcSbWsV0mPl/qHFu/4FnZI
 m4Nj1Sx3sG0CyNDpWjVrzTW6SbDX9J68DXbLwnNTSX3KPa7gNn6TUmFfKzlrNI23
 pPmD+EITYMn/H9HOhxhTzq//Ja7UOViAKQ0q4N2I4GxmLP6ufx9P3s5FG/oJqA+H
 Pka2+9JnfHq2Ze22CoDcg8q5f5MgVkQzGeir0ZsGJwJqOYjeTmbCvD3T/RYWO5g+
 ZghON3tsMQmdzUQqGRxcn/YLOZY9ZqrX2kBf5E6Wapwj9MfKg2ToLZM4yrWN0+RX
 KDuWaKXYtkSQCo1nDS2KooVMWjGNZautWWnHzZ0KNQCIkxVpGC234JYI685grKXb
 dg7R41kdXI7NJzqS4iM1fxXoLx64fpoREa/pbLF6VeLaYXBlzMjfhiIx2pQBN3L/
 q/y3ftev9VCp+2wPxiKUkiC4Sh7dgWUzNuyHU+4lsPUbI1H/MN5dN2ryObdEGWc/
 5YU3tv2MTQJ7jECHR+/fastnG+5d2kVm/FK+zVhG17JvA2VmDaLnSde+mzGbsO1N
 gIUx5VrTEP7y0tC8C/VgbS3c2KqCSOopqd3j2slLLrtQlXM71VE=
 =lpDI
 -----END PGP SIGNATURE-----

Merge tag 'v1.13.0rc3' into develop

Synapse 1.13.0rc3 (2020-05-18)

Bugfixes:

- Hash passwords as early as possible during registration. #7523
2020-05-18 11:10:04 -04:00
Patrick Cloke 56db0b1365
Hash passwords earlier in the registration process (#7523) 2020-05-18 09:46:18 -04:00
Manuel Stahl a4a5ec4096
Add room details admin endpoint (#7317) 2020-05-07 15:33:07 -04:00
Manuel Stahl 04dd7d182d
Return total number of users and profile attributes in admin users endpoint (#6881)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-04-28 18:19:36 +01:00
Dirk Klimpel 6b6685db9f
Extend room admin api with additional attributes (#7225) 2020-04-22 13:38:41 +01:00
Dirk Klimpel fb69690761
Admin API to join users to a room. (#7051) 2020-03-27 19:16:43 +00:00
Patrick Cloke 88b41986db
Add an option to the set password API to choose whether to logout other devices. (#7085) 2020-03-18 07:50:00 -04:00
Dirk Klimpel 9b06d8f8a6
Fixed set a user as an admin with the new API (#6928)
Fix #6910
2020-02-28 09:58:05 +00:00
Andrew Morgan 8c75b621bf
Ensure 'deactivated' parameter is a boolean on user admin API, Fix error handling of call to deactivate user (#6990) 2020-02-26 12:22:55 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Andrew Morgan d8994942f2
Return a 404 for admin api user lookup if user not found (#6901) 2020-02-12 18:14:10 +00:00
Dirk Klimpel 56ca93ef59
Admin api to add an email address (#6789) 2020-02-07 10:29:36 +00:00
Andrew Morgan 370080531e
Allow URL-encoded user IDs on user admin api paths (#6825) 2020-02-03 13:18:42 +00:00
Richard van der Hoff 5bd3cb7260
Minor fixes to user admin api (#6761)
* don't insist on a password (this is valid if you have an SSO login)
* fix reference to undefined `requester`
2020-01-23 12:03:58 +00:00
Andrew Morgan ce84dd9e20
Remove unnecessary abstractions in admin handler (#6751) 2020-01-22 15:09:57 +00:00
Andrew Morgan 90a28fb475
Admin API to list, filter and sort rooms (#6720) 2020-01-22 13:36:43 +00:00
Andrew Morgan aa9b00fb2f
Fix and add test to deprecated quarantine media admin api (#6756) 2020-01-22 11:05:50 +00:00
Erik Johnston b0a66ab83c
Fixup synapse.rest to pass mypy (#6732) 2020-01-20 17:38:21 +00:00
Erik Johnston ceecedc68b
Fix changing password via user admin API. (#6730) 2020-01-20 17:23:59 +00:00
Brendan Abolivier 842c2cfbf1
Remove get_room_event_after_stream_ordering entirely 2020-01-16 20:24:17 +00:00
Andrew Morgan 1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Manuel Stahl d2906fe666 Allow admin users to create or modify users without a shared secret (#6495)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Manuel Stahl 649b6bc088 Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925) 2019-12-05 18:12:23 +00:00
Manuel Stahl 4f5ca455bf Move admin endpoints into separate files (#6308) 2019-11-20 11:49:11 +00:00
Andrew Morgan 54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston 2e97a4c197 Port synapse/rest/client/_base.py to async/await 2019-10-11 15:29:26 +01:00
Erik Johnston 3c2d6c708c Add maybe_awaitable and fix __init__ bugs 2019-10-11 15:26:09 +01:00
Erik Johnston dfbb62c28d Port synaps/rest/client/users.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston f95325e22a Port synaps/rest/client/server_notice_servlet.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston 281f887090 Port synaps/rest/client/media.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston fca3a541e7 Port rest/admin/__init__.py to async/await 2019-10-11 12:17:52 +01:00
Travis Ralston 2f416fc997
Ensure the list media admin API is always available (#5966)
* Ensure the list media admin API is always available

This API is required for some external media repo implementations to operate (mostly for doing quarantine operations on a room).

* changelog
2019-09-03 13:35:20 -06:00
Travis Ralston 0eac7077c9
Ensure an auth instance is available to ListMediaInRoom (#5967)
* Ensure an auth instance is available to ListMediaInRoom

Fixes https://github.com/matrix-org/synapse/issues/5737

* Changelog
2019-09-03 09:01:30 -06:00
Andrew Morgan 4548d1f87e
Remove unnecessary parentheses around return statements (#5931)
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Olivier Wilkinson (reivilibre) c88a119259 Add GET method to admin API /users/@user:dom/admin
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-27 13:12:27 +01:00
reivilibre 1a7e6eb633
Add Admin API capability to set adminship of a user (#5878)
Admin API: Set adminship of a user
2019-08-27 10:14:00 +01:00