Commit Graph

313 Commits

Author SHA1 Message Date
David Robertson 62a1a9be52
Describe which rate limiter was hit in logs (#16135) 2023-08-30 00:39:39 +01:00
Mathieu Velten 358896e1b8
Implements a task scheduler for resumable potentially long running tasks (#15891) 2023-08-21 14:17:13 +02:00
Patrick Cloke d98a43d922
Stabilize support for MSC3970: updated transaction semantics (scope to `device_id`) (#15629)
For now this maintains compatible with old Synapses by falling back
to using transaction semantics on a per-access token. A future version
of Synapse will drop support for this.
2023-08-04 07:47:18 -04:00
Erik Johnston ae55cc1e6b
Add ability to wait for locks and add locks to purge history / room deletion (#15791)
c.f. #13476
2023-07-31 10:58:03 +01:00
Eric Eastwood 0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748)
Context for why we're removing the implementation:

 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734

Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737

Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
Hugh Nimmo-Smith 249f4a338d Refactor config to be an experimental feature
Also enforce you can't combine it with incompatible config options
2023-05-30 09:43:06 -04:00
Quentin Gliech 765244faee Initial MSC3964 support: delegation of auth to OIDC server 2023-05-30 09:43:06 -04:00
Quentin Gliech e2c8458bba Make the api.auth.Auth a Protocol 2023-05-30 09:43:06 -04:00
Jason Little c835befd10
Add Unix socket support for Redis connections (#15644)
Adds a new configuration setting to connect to Redis via a Unix
socket instead of over TCP. Disabled by default.
2023-05-26 15:28:39 -04:00
Patrick Cloke 1e89976b26
Rename blacklist/whitelist internally. (#15620)
Avoid renaming configuration settings for now and rename internal code
to use blocklist and allowlist instead.
2023-05-19 12:25:25 +00:00
Patrick Cloke 89a23c9406
Do not allow deactivated users to login with JWT. (#15624)
To improve the organization of this code it moves the JWT login
checks to a separate handler and then fixes the bug (and a
deprecation warning).
2023-05-19 08:06:54 -04:00
Jason Little d3bd03559b
HTTP Replication Client (#15470)
Separate out a HTTP client for replication in preparation for
also supporting using UNIX sockets. The major difference from
the base class is that this does not use treq to handle HTTP
requests.
2023-05-09 14:25:20 -04:00
Sean Quah e46d5f3586
Factor out an `is_mine_server_name` method (#15542)
Add an `is_mine_server_name` method, similar to `is_mine_id`.

Ideally we would use this consistently, instead of sometimes comparing
against `hs.hostname` and other times reaching into
`hs.config.server.server_name`.

Also fix a bug in the tests where `hs.hostname` would sometimes differ
from `hs.config.server.server_name`.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-05 15:06:22 +01:00
Andrew Morgan 2e59e97ebd
Move ThirdPartyEventRules into module_api/callbacks (#15535) 2023-05-04 14:18:22 +00:00
Sean Quah 8aee823393 Merge branch 'release-v1.83' into develop 2023-05-03 15:23:16 +01:00
Sean Quah 3b837d856c
Revert "Reduce the size of the HTTP connection pool for non-pushers" (#15530)
#15514 introduced a regression where Synapse would encounter
`PartialDownloadError`s when fetching OpenID metadata for certain
providers on startup. Due to #8088, this prevents Synapse from starting
entirely.

Revert the change while we decide what to do about the regression.
2023-05-03 13:09:20 +01:00
Sean Quah 04e79e6a18
Add config option to forget rooms automatically when users leave them (#15224)
This is largely based off the stats and user directory updater code.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-03 12:27:33 +01:00
Patrick Cloke 6aca4e7cb8
Reduce the size of the HTTP connection pool for non-pushers. (#15514)
Pushers tend to make many connections to the same HTTP host
(e.g. a new event comes in, causes events to be pushed, and then
the homeserver connects to the same host many times). Due to this
the per-host HTTP connection pool size was increased, but this does
not make sense for other SimpleHttpClients.

Add a parameter for the connection pool and override it for pushers
(making a separate SimpleHttpClient for pushers with the increased
configuration).

This returns the HTTP connection pool settings to the default Twisted
ones for non-pusher HTTP clients.
2023-05-02 09:29:40 -04:00
Quentin Gliech 8b3a502996
Experimental support for MSC3970: per-device transaction IDs (#15318) 2023-04-25 09:37:09 +01:00
Andrew Morgan aec639e3e3
Move Spam Checker callbacks to a dedicated file (#15453) 2023-04-18 00:57:40 +00:00
Andrew Morgan 4953cd71df
Move Account Validity callbacks to a dedicated file (#15237) 2023-03-16 10:35:31 +00:00
David Robertson ce54477f6f
Give PyCharm some help with `@cache_in_self` (#15238)
* Give PyCharm some help with `@cache_in_self`

* Changelog

* Fix import for old python versions
2023-03-09 19:12:09 +00:00
Patrick Cloke 05e0a4089a
Stop applying edits to event contents (MSC3925). (#15193)
Enables MSC3925 support by default, which:

* Includes the full edit event in the bundled aggregations of an
  edited event.
* Stops modifying the original event's content to return the new
  content from the edit event.

This is a backwards-incompatible change that is considered to be
"correct" by the spec.
2023-03-06 09:43:01 -05:00
Patrick Cloke 4fc8875876
Refactor media modules. (#15146)
* Removes the `v1` directory from `test.rest.media.v1`.
* Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`.
* Flatten the `v1` directory from `synapse.rest.media`,  but leave compatiblity
  with 3rd party media repositories and spam checkers.
2023-02-27 08:26:05 -05:00
999lakhisidhu 27a3a72a50
Support for selecting the Redis logical database. (#15034)
Note that this is only used for key-value store (cached values)
and not for the pub/sub replication used by Synapse.
2023-02-15 07:39:31 -05:00
Patrick Cloke 733531ee3e
Add final type hint to synapse.server. (#15035) 2023-02-09 09:49:04 -05:00
Erik Johnston 65d0386693
Always notify replication when a stream advances (#14877)
This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.
2023-01-20 18:02:18 +00:00
Sean Quah a302d3ecf7
Remove unnecessary reactor reference from `_PerHostRatelimiter` (#14842)
Fix up #14812 to avoid introducing a reference to the reactor.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-16 13:16:19 +00:00
Sean Quah 772e8c2385
Fix stack overflow in `_PerHostRatelimiter` due to synchronous requests (#14812)
When there are many synchronous requests waiting on a
`_PerHostRatelimiter`, each request will be started recursively just
after the previous request has completed. Under the right conditions,
this leads to stack exhaustion.

A common way for requests to become synchronous is when the remote
client disconnects early, because the homeserver is overloaded and slow
to respond.

Avoid stack exhaustion under these conditions by deferring subsequent
requests until the next reactor tick.

Fixes #14480.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-13 00:16:21 +00:00
Richard van der Hoff 06ab64f201
Implement MSC3925: changes to bundling of edits (#14811)
Two parts to this:

 * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag.
 * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.
2023-01-10 16:31:28 +00:00
Patrick Cloke 6d47b7e325
Add a type hint for `get_device_handler()` and fix incorrect types. (#14055)
This was the last untyped handler from the HomeServer object. Since
it was being treated as Any (and thus unchecked) it was being used
incorrectly in a few places.
2022-11-22 14:08:04 -05:00
David Robertson 258b5285b6
Fix typechecking errors introduced in #14128 (#14455)
* Fix typechecking errors introduced in #14128

* Changelog

* Correct annotations

so that context_factory works if you don't use TLS
2022-11-15 16:36:43 +00:00
dependabot[bot] 8bcdd712b8
Bump flake8-bugbear from 22.9.23 to 22.10.27 (#14329)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-11-04 18:43:14 +00:00
Sean Quah 8ef0c8ff14
Fix error in `is_mine_id` when encountering a malformed ID (#13746)
Previously, `is_mine_id` would raise an exception when passed an ID with
no colons. Return `False` instead.

Fixes #13040.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:54:36 +01:00
Brendan Abolivier 898fef2789
Share some metrics between the Prometheus exporter and the phone home stats (#13671) 2022-09-05 10:26:43 +00:00
Eric Eastwood 1eea73b413
Fix rate limit metrics registering twice and misreporting (#13649)
* Fix rate limit metrics registering twice and misreporting

Fix https://github.com/matrix-org/synapse/issues/13641

* Fix lints

* Add changelog

* Document `metrics_name=None`.
2022-08-30 12:08:29 +01:00
Quentin Gliech fe1daad672
Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. (#12986)
This simplifies the access token verification logic by removing the `rights`
parameter which was only ever used for the unsubscribe link in email
notifications. The latter has been moved under the `/_synapse` namespace,
since it is not a standard API.

This also makes the email verification link more secure, by embedding the
app_id and pushkey in the macaroon and verifying it. This prevents the user
from tampering the query parameters of that unsubscribe link.

Macaroon generation is refactored:

- Centralised all macaroon generation and verification logic to the
  `MacaroonGenerator`
- Moved to `synapse.utils`
- Changed the constructor to require only a `Clock`, hostname, and a secret key
  (instead of a full `Homeserver`).
- Added tests for all methods.
2022-06-14 09:12:08 -04:00
Quentin Gliech 92103cb2c8
Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`. (#13021) 2022-06-14 09:51:15 +01:00
Erik Johnston 1e453053cb
Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
Patrick Cloke 49f06866e4
Remove backing code for groups/communities (#12558)
Including handlers, configuration code, appservice support, and
the GroupID construct.
2022-05-26 09:04:34 -04:00
Jess Porter 39bed28b28
SpamChecker metrics (#12513)
* add Measure blocks all over SpamChecker

Signed-off-by: jesopo <github@lolnerd.net>

* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck

* better changelog entry
2022-05-13 12:17:38 +01:00
Patrick Cloke a4c75918b3
Remove unneeded `ActionGenerator` class. (#12691)
It simply passes through to `BulkPushRuleEvaluator`, which can be
called directly instead.
2022-05-11 07:15:21 -04:00
Brendan Abolivier 5ef673de4f
Add a module API to allow modules to edit push rule actions (#12406)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-04-27 13:55:33 +00:00
Patrick Cloke 86cf6a3a17
Remove references to unstable identifiers from MSC3440. (#12382)
Removes references to unstable thread relation, unstable
identifiers for filtering parameters, and the experimental
config flag.
2022-04-12 08:42:03 -04:00
Patrick Cloke fc9bd620ce
Add a relations handler to avoid duplication. (#12227)
Adds a handler layer between the REST and datastore layers for relations.
2022-03-16 10:39:15 -04:00
Patrick Cloke bc9dff1d95
Remove unnecessary pass statements. (#12206) 2022-03-11 07:06:21 -05:00
Patrick Cloke ea27528b5d
Support stable identifiers for MSC3440: Threading (#12151)
The unstable identifiers are still supported if the experimental configuration
flag is enabled. The unstable identifiers will be removed in a future release.
2022-03-10 15:36:13 +00:00
Patrick Cloke 3e4af36bc8
Rename get_tcp_replication to get_replication_command_handler. (#12192)
Since the object it returns is a ReplicationCommandHandler.

This is clean-up from adding support to Redis where the command handler
was added as an additional layer of abstraction from the TCP protocol.
2022-03-10 13:01:56 +00:00
Patrick Cloke d8bab6793c
Fix incorrect type hints for txredis. (#12042)
Some properties were marked as RedisProtocol instead of ConnectionHandler,
which wraps RedisProtocol instance(s).
2022-03-08 07:26:05 -05:00
Richard van der Hoff e24ff8ebe3
Remove `HomeServer.get_datastore()` (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00