Commit Graph

233 Commits

Author SHA1 Message Date
David Robertson 32a59a6495
Keep track of `user_ips` and `monthly_active_users` when delegating auth (#16672)
* Describe `insert_client_ip`
* Pull out client_ips and MAU tracking to BaseAuth
* Define HAS_AUTHLIB once in tests

sick of copypasting

* Track ips and token usage when delegating auth
* Test that we track MAU and user_ips
* Don't track `__oidc_admin`
2023-11-23 12:35:37 +00:00
David Robertson 43d1aa75e8
Add an Admin API to temporarily grant the ability to update an existing cross-signing key without UIA (#16634) 2023-11-15 17:28:10 +00:00
Patrick Cloke f2f2c7c1f0
Use full GitHub links instead of bare issue numbers. (#16637) 2023-11-15 08:02:11 -05:00
Patrick Cloke ab3f1b3b53
Convert simple_select_one_txn and simple_select_one to return tuples. (#16612) 2023-11-09 11:13:31 -05:00
Patrick Cloke ff716b483b
Return attrs for more media repo APIs. (#16611) 2023-11-09 11:00:30 -05:00
Patrick Cloke 2bf9341406
Ensure local invited & knocking users leave before purge. (#16559)
This is mostly useful for federated rooms where some users
would get stuck in the invite or knock state when the room
was purged from their homeserver.
2023-10-27 12:50:50 -04:00
Patrick Cloke 26b960b08b
Register media servlets via regex. (#16419)
This converts the media servlet URLs in the same way as
(most) of the rest of Synapse. This will give more flexibility
in the versions each endpoint exists under.
2023-10-06 07:22:55 -04:00
MomentQYC 5725712d47
Remove Python version from `/_synapse/admin/v1/server_version` (#16380)
There's no reason to expose the full Python version over what is
frequently a public API.
2023-10-02 09:07:53 -04:00
Hanadi eef2b9e344
Filter locked users in the admin API (#16328)
Co-authored-by: Hanadi Tamimi <hanadi.tamimi@sdui.de>
2023-09-18 15:37:51 +01:00
Mathieu Velten dd44ee00b6
Add automatic purge after all users forget a room (#15488)
Also add restore of purge/shutdown rooms after a synapse restart.

Co-authored-by:  Eric Eastwood <erice@matrix.org>
Co-authored-by: Erik Johnston <erikj@matrix.org>
2023-09-15 15:37:44 +02:00
Patrick Cloke aa483cb4c9
Update ruff config (#16283)
Enable additional checks & clean-up unneeded configuration.
2023-09-08 11:24:36 -04:00
Mathieu Velten dcb2778341
Add last_seen_ts to the admin users API (#16218) 2023-09-04 18:13:28 +02:00
Patrick Cloke 9ec3da06da
Bump mypy-zope & mypy. (#16188) 2023-08-29 10:38:56 -04:00
Patrick Cloke daf11e26ef
Replace make_awaitable with AsyncMock (#16179)
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
2023-08-24 19:38:46 -04:00
Alexander Fechler 54317d34b7
Allow filtering for admins in the list accounts admin API (#16114) 2023-08-18 12:26:38 +01:00
Mathieu Velten dac97642e4
Implements admin API to lock an user (MSC3939) (#15870) 2023-08-10 09:10:55 +00:00
Shay f08d05dd2c
Actually stop reading from column `user_id` of tables `profiles` (#15955) 2023-07-23 16:30:54 -07:00
Mathieu Velten 8eb7bb975e
Mark get_user_in_directory private since only used in tests (#15884) 2023-07-12 11:09:13 +02:00
Michael Weimann c8e81898b6
Add not_user_type param to the list accounts admin API (#15844)
Signed-off-by: Michael Weimann <michaelw@element.io>
2023-07-04 15:03:20 -07:00
Hugh Nimmo-Smith bad1f2cd35 Tests for JWKS endpoint 2023-05-30 09:43:06 -04:00
Shay 0e8aa2a1b2
Remove references to supporting per-user flag for msc2654 (#15522) 2023-05-02 14:21:36 -07:00
Shay 89f6fb0d5a
Add an admin API endpoint to support per-user feature flags (#15344) 2023-04-28 11:33:45 -07:00
Shay 301b4156d5
Add column `full_user_id` to tables `profiles` and `user_filters`. (#15458) 2023-04-26 16:03:26 -07:00
Quentin Gliech 5b70f240cf
Make cleaning up pushers depend on the device_id instead of the token_id (#15280)
This makes it so that we rely on the `device_id` to delete pushers on logout,
instead of relying on the `access_token_id`. This ensures we're not removing
pushers on token refresh, and prepares for a world without access token IDs
(also known as the OIDC).

This actually runs the `set_device_id_for_pushers` background update, which
was forgotten in #13831.

Note that for backwards compatibility it still deletes pushers based on the
`access_token` until the background update finishes.
2023-03-24 11:09:39 -04:00
Erik Johnston 827f198177
Fix error when sending message into deleted room. (#15235)
When a room is deleted in Synapse we remove the event forward
extremities in the room, so if (say a bot) tries to send a message into
the room we error out due to not being able to calculate prev events for
the new event *before* we check if the sender is in the room.

Fixes #8094
2023-03-21 09:13:43 +00:00
Dirk Klimpel 93f7955eba
Admin API endpoint to delete a reported event (#15116)
* Admin api to delete event report

* lint +  tests

* newsfile

* Apply suggestions from code review

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* revert changes - move to WorkerStore

* update unit test

* Note that timestamp is in millseconds

---------

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2023-02-28 12:09:10 +00:00
Patrick Cloke 4fc8875876
Refactor media modules. (#15146)
* Removes the `v1` directory from `test.rest.media.v1`.
* Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`.
* Flatten the `v1` directory from `synapse.rest.media`,  but leave compatiblity
  with 3rd party media repositories and spam checkers.
2023-02-27 08:26:05 -05:00
dependabot[bot] 9bb2eac719
Bump black from 22.12.0 to 23.1.0 (#15103) 2023-02-22 15:29:09 -05:00
Patrick Cloke 42aea0d8af
Add final type hint to tests.unittest. (#15072)
Adds a return type to HomeServerTestCase.make_homeserver and deal
with any variables which are no longer Any.
2023-02-14 14:03:35 -05:00
Sean Quah d0c713cc85
Return read-only collections from `@cached` methods (#13755)
It's important that collections returned from `@cached` methods are not
modified, otherwise future retrievals from the cache will return the
modified collection.

This applies to the return values from `@cached` methods and the values
inside the dictionaries returned by `@cachedList` methods. It's not
necessary for the dictionaries returned by `@cachedList` methods
themselves to be read-only.

Signed-off-by: Sean Quah <seanq@matrix.org>
Co-authored-by: David Robertson <davidr@element.io>
2023-02-10 23:29:00 +00:00
Shay 03bccd542b
Add a class UnpersistedEventContext to allow for the batching up of storing state groups (#14675)
* add class UnpersistedEventContext

* modify create new client event to create unpersistedeventcontexts

* persist event contexts after creation

* fix tests to persist unpersisted event contexts

* cleanup

* misc lints + cleanup

* changelog + fix comments

* lints

* fix batch insertion?

* reduce redundant calculation

* add unpersisted event classes

* rework compute_event_context, split into function that returns unpersisted event context and then persists it

* use calculate_context_info to create unpersisted event contexts

* update typing

* $%#^&*

* black

* fix comments and consolidate classes, use attr.s for class

* requested changes

* lint

* requested changes

* requested changes

* refactor to be stupidly explicit

* clearer renaming and flow

* make partial state non-optional

* update docstrings

---------

Co-authored-by: Erik Johnston <erik@matrix.org>
2023-02-09 13:05:02 -08:00
Patrick Cloke d22c1c862c
Respond correctly to unknown methods on known endpoints (#14605)
Respond with a 405 error if a request is received on a known endpoint,
but to an unknown method, per MSC3743.
2023-02-09 13:04:24 -05:00
Patrick Cloke 1182ae5063
Add helper to parse an enum from query args & use it. (#14956)
The `parse_enum` helper pulls an enum value from the query string
(by delegating down to the parse_string helper with values generated
from the enum).

This is used to pull out "f" and "b" in most places and then we thread
the resulting Direction enum throughout more code.
2023-02-01 21:35:24 +00:00
David Robertson 80d44060c9
Faster joins: omit partial rooms from eager syncs until the resync completes (#14870)
* Allow `AbstractSet` in `StrCollection`

Or else frozensets are excluded. This will be useful in an upcoming
commit where I plan to change a function that accepts `List[str]` to
accept `StrCollection` instead.

* `rooms_to_exclude` -> `rooms_to_exclude_globally`

I am about to make use of this exclusion mechanism to exclude rooms for
a specific user and a specific sync. This rename helps to clarify the
distinction between the global config and the rooms to exclude for a
specific sync.

* Better function names for internal sync methods

* Track a list of excluded rooms on SyncResultBuilder

I plan to feed a list of partially stated rooms for this sync to ignore

* Exclude partial state rooms during eager sync

using the mechanism established in the previous commit

* Track un-partial-state stream in sync tokens

So that we can work out which rooms have become fully-stated during a
given sync period.

* Fix mutation of `@cached` return value

This was fouling up a complement test added alongside this PR.
Excluding a room would mean the set of forgotten rooms in the cache
would be extended. This means that room could be erroneously considered
forgotten in the future.

Introduced in #12310, Synapse 1.57.0. I don't think this had any
user-visible side effects (until now).

* SyncResultBuilder: track rooms to force as newly joined

Similar plan as before. We've omitted rooms from certain sync responses;
now we establish the mechanism to reintroduce them into future syncs.

* Read new field, to present rooms as newly joined

* Force un-partial-stated rooms to be newly-joined

for eager incremental syncs only, provided they're still fully stated

* Notify user stream listeners to wake up long polling syncs

* Changelog

* Typo fix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Unnecessary list cast

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Rephrase comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Another comment

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Fixup merge(?)

* Poke notifier when receiving un-partial-stated msg over replication

* Fixup merge whoops

Thanks MV :)

Co-authored-by: Mathieu Velen <mathieuv@matrix.org>

Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-23 15:44:39 +00:00
Patrick Cloke 9d8a3234ba
Respond with proper error responses on unknown paths. (#14621)
Returns a proper 404 with an errcode of M_RECOGNIZED for
unknown endpoints per MSC3743.
2022-12-08 11:37:05 -05:00
Benjamin Kampmann f6c74d1cb2
Implement message forward pagination from start when no from is given, fixes #12383 (#14149)
Fixes https://github.com/matrix-org/synapse/issues/12383
2022-11-24 09:10:51 +00:00
Patrick Cloke 6d47b7e325
Add a type hint for `get_device_handler()` and fix incorrect types. (#14055)
This was the last untyped handler from the HomeServer object. Since
it was being treated as Any (and thus unchecked) it was being used
incorrectly in a few places.
2022-11-22 14:08:04 -05:00
Ashish Kumar a3623af74e
Add an Admin API endpoint for looking up users based on 3PID (#14405) 2022-11-11 15:38:17 +00:00
Tadeusz Sośnierz 1433b5d5b6
Show erasure status when listing users in the Admin API (#14205)
* Show erasure status when listing users in the Admin API

* Use USING when joining erased_users

* Add changelog entry

* Revert "Use USING when joining erased_users"

This reverts commit 30bd2bf106.

* Make the erased check work on postgres

* Add a testcase for showing erased user status

* Appease the style linter

* Explicitly convert `erased` to bool to make SQLite consistent with Postgres

This also adds us an easy way in to fix the other accidentally integered columns.

* Move erasure status test to UsersListTestCase

* Include user erased status when fetching user info via the admin API

* Document the erase status in user_admin_api

* Appease the linter and mypy

* Signpost comments in tests

Co-authored-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-10-21 13:52:44 +01:00
Brendan Abolivier be76cd8200
Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556) 2022-09-29 15:23:24 +02:00
Brendan Abolivier 8ae42ab8fa
Support enabling/disabling pushers (from MSC3881) (#13799)
Partial implementation of MSC3881
2022-09-21 14:39:01 +00:00
Quentin Gliech 74f60cec92
Add an admin API endpoint to find a user based on its external ID in an auth provider. (#13810) 2022-09-16 12:29:03 +00:00
Dirk Klimpel f799eac7ea
Add timestamp to user's consent (#13741)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-09-08 15:41:48 +00:00
Connor Davis bb5b47b62a
Add Admin API to Fetch Messages Within a Particular Window (#13672)
This adds two new admin APIs that allow us to fetch messages from a room within a particular time.
2022-09-07 10:54:44 +01:00
David Robertson a160406d24
Fix admin List Room API return type on sqlite (#13509) 2022-08-31 10:38:16 +00:00
Dirk Klimpel 37f329c9ad
Fix that sending server notices fail if avatar is `None` (#13566)
Indroduced in #11846.
2022-08-23 09:48:35 +01:00
Andrew Morgan f9f03426de
Implement MSC3852: Expose `last_seen_user_agent` to users for their own devices; also expose to Admin API (#13549) 2022-08-19 16:17:10 +00:00
reivilibre 8bdf2bd31e
Fix a bug in the `/event_reports` Admin API which meant that the total count could be larger than the number of results you can actually query for. (#13525)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-08-17 18:08:23 +00:00
Dirk Klimpel d75512d19e
Add forgotten status to Room Details API (#13503) 2022-08-17 09:42:01 +00:00
Dirk Klimpel 2281427175
Use literals in place of `HTTPStatus` constants in tests (#13488)
* Use literals in place of `HTTPStatus` constants in tests

* newsfile

* code style

* code style
2022-08-10 11:01:12 -07:00