Commit Graph

18503 Commits

Author SHA1 Message Date
Patrick Cloke cb7fc7523e
Add a basic test for purging rooms. (#9541)
Unfortunately this doesn't test re-joining the room since
that requires having another homeserver to query over
federation, which isn't easily doable in unit tests.
2021-03-08 09:21:36 -05:00
Erik Johnston b988b07bb0 Merge branch 'master' into develop 2021-03-08 14:06:35 +00:00
Erik Johnston 4de1c35728 Fixup changelog 2021-03-08 13:59:17 +00:00
Erik Johnston 15c788e22d 1.29.0 2021-03-08 13:52:13 +00:00
Patrick Cloke 58114f8a17
Create a SynapseReactor type which incorporates the necessary reactor interfaces. (#9528)
This helps fix some type hints when running with Twisted 21.2.0.
2021-03-08 08:25:43 -05:00
Leo Bärring 0fc4eb103a
Update reverse proxy to add OpenBSD relayd example configuration. (#9508)
Update reverse proxy to add OpenBSD relayd example configuration.

Signed-off-by: Leo Bärring <leo.barring@protonmail.com>
2021-03-06 11:49:19 +00:00
Ben Banfield-Zanin e5da770cce
Add additional SAML2 upgrade notes (#9550) 2021-03-05 12:07:50 +00:00
Richard van der Hoff 8a4b3738f3
Replace `last_*_pdu_age` metrics with timestamps (#9540)
Following the advice at
https://prometheus.io/docs/practices/instrumentation/#timestamps-not-time-since,
it's preferable to export unix timestamps, not ages.

There doesn't seem to be any particular naming convention for timestamp
metrics.
2021-03-04 16:40:18 +00:00
Richard van der Hoff df425c2c63
Prometheus metrics for logins and registrations (#9511)
Add prom metrics for number of users successfully registering and logging in, by SSO provider.
2021-03-04 16:39:27 +00:00
Richard van der Hoff 7eb6e39a8f
Record the SSO Auth Provider in the login token (#9510)
This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)
2021-03-04 14:44:22 +00:00
Erik Johnston a6333b8d42 Fix link in UPGRADES 2021-03-04 10:32:44 +00:00
Erik Johnston ea0a3aaf0a Fix changelog 2021-03-04 10:29:43 +00:00
Erik Johnston 3f49d80dcf 1.29.0rc1 2021-03-04 10:12:53 +00:00
Patrick Cloke 33a02f0f52
Fix additional type hints from Twisted upgrade. (#9518) 2021-03-03 15:47:38 -05:00
Richard van der Hoff 4db07f9aef
Set X-Forwarded-Proto header when frontend-proxy proxies a request (#9539)
Should fix some remaining warnings
2021-03-03 18:49:08 +00:00
Erik Johnston a4fa044c00
Fix 'rejected_events_metadata' background update (#9537)
Turns out matrix.org has an event that has duplicate auth events (which really isn't supposed to happen, but here we are). This caused the background update to fail due to `UniqueViolation`.
2021-03-03 16:04:24 +00:00
Patrick Cloke 922788c604
Purge chain cover tables when purging events. (#9498) 2021-03-03 11:04:08 -05:00
Dirk Klimpel d790d0d314
Add type hints to user admin API. (#9521) 2021-03-03 08:09:39 -05:00
Patrick Cloke 0c330423bc
Bump the mypy and mypy-zope versions. (#9529) 2021-03-03 07:19:19 -05:00
Erik Johnston 16f9f93eb7
Make deleting stale pushers a background update (#9536) 2021-03-03 12:08:16 +00:00
Richard van der Hoff a5daae2a5f
Update nginx reverse-proxy docs (#9512)
Turns out nginx overwrites the Host header by default.
2021-03-03 11:08:11 +00:00
Aaron Raimist 0279e0e086
Prevent presence background jobs from running when presence is disabled (#9530)
Prevent presence background jobs from running when presence is disabled

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-03-03 10:21:46 +00:00
Patrick Cloke aee10768d8 Revert "Fix #8518 (sync requests being cached wrongly on timeout) (#9358)"
This reverts commit f5c93fc993.

This is being backed out due to a regression (#9507) and additional
review feedback being provided.
2021-03-02 09:43:34 -05:00
Erik Johnston 7f5d753d06
Re-run rejected metadata background update. (#9503)
It landed in schema version 58 after 59 had been created, causing some
servers to not run it. The main effect of was that not all rooms had
their chain cover calculated correctly. After the BG updates complete
the chain covers will get fixed when a new state event in the affected
rooms is received.
2021-03-02 14:31:23 +00:00
Erik Johnston 16108c579d
Fix SQL delta file taking a long time to run (#9516)
Fixes #9504
2021-03-02 14:05:01 +00:00
Dirk Klimpel f00c4e7af0
Add type hints to device and event report admin API (#9519) 2021-03-02 09:31:12 +00:00
Patrick Cloke ad8589d392
Fix a bug when a room alias is given to the admin join endpoint (#9506) 2021-03-01 13:59:01 -05:00
Patrick Cloke 16ec8c3272
(Hopefully) stop leaking file descriptors in media repo. (#9497)
By consuming the response if the headers imply that the
content is too large.
2021-03-01 12:45:00 -05:00
Patrick Cloke a0bc9d387e
Use the proper Request in type hints. (#9515)
This also pins the Twisted version in the mypy job for CI until
proper type hints are fixed throughout Synapse.
2021-03-01 12:23:46 -05:00
Jonathan de Jong e12077a78a
Allow bytecode again (#9502)
In #75, bytecode was disabled (from a bit of FUD back in `python<2.4` days, according to dev chat), I think it's safe enough to enable it again.

Added in `__pycache__/` and `.pyc`/`.pyd` to `.gitignore`, to extra-insure compiled files don't get committed.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-02-26 18:30:54 +00:00
Tim Leung ddb240293a
Add support for no_proxy and case insensitive env variables (#9372)
### Changes proposed in this PR

- Add support for the `no_proxy` and `NO_PROXY` environment variables
  - Internally rely on urllib's [`proxy_bypass_environment`](bdb941be42/Lib/urllib/request.py (L2519))
- Extract env variables using urllib's `getproxies`/[`getproxies_environment`](bdb941be42/Lib/urllib/request.py (L2488)) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment

This does contain behaviour changes for consumers so making sure these are called out:
- `no_proxy`/`NO_PROXY` is now respected
- lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY`

Related to #9306 which also uses `ProxyAgent`

Signed-off-by: Timothy Leung tim95@hotmail.co.uk
2021-02-26 17:37:57 +00:00
Richard van der Hoff 15090de850
SSO: redirect to public URL before setting cookies (#9436)
... otherwise, we don't get the cookie back.
2021-02-26 14:02:06 +00:00
Richard van der Hoff e53f11bd62
Call out the need for an X-Forwarded-Proto in the upgrade notes (#9501) 2021-02-26 13:24:54 +00:00
Erik Johnston 2566dc57ce
Test that we require validated email for email pushers (#9496) 2021-02-25 15:35:14 +00:00
Erik Johnston 1e62d9ee8c
Ensure pushers are deleted for deactivated accounts (#9285) 2021-02-25 13:56:55 +00:00
Erik Johnston 1efdcc3e87 Merge branch 'master' into develop 2021-02-25 10:53:31 +00:00
Erik Johnston 2756517f7a Fixup changelog 2021-02-25 10:47:19 +00:00
Erik Johnston 0f9f30b32b Fixup changelog 2021-02-25 10:27:22 +00:00
Erik Johnston b5c4fe1971 1.28.0 2021-02-25 10:22:07 +00:00
Richard van der Hoff d8e95e5452
Add support for X-Forwarded-Proto (#9472)
rewrite XForwardedForRequest to set `isSecure()` based on
`X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.
2021-02-24 18:11:33 +00:00
Andrew Morgan 00bf80cb8e Fix typo in spam checker documentation 2021-02-24 17:51:52 +00:00
Erik Johnston 7cc571510b
Add SQL delta for deleting stale pushers (#9479) 2021-02-24 17:21:10 +00:00
Jonathan de Jong f5c93fc993
Fix #8518 (sync requests being cached wrongly on timeout) (#9358)
This fixes #8518 by adding a conditional check on `SyncResult` in a function when `prev_stream_token == current_stream_token`, as a sanity check. In `CachedResponse.set.<remove>()`, the result is immediately popped from the cache if the conditional function returns "false".

This prevents the caching of a timed-out `SyncResult` (that has `next_key` as the stream key that produced that `SyncResult`). The cache is prevented from returning a `SyncResult` that makes the client request the same stream key over and over again, effectively making it stuck in a loop of requesting and getting a response immediately for as long as the cache keeps those values.

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-02-24 13:57:00 +00:00
Erik Johnston 2927921942
Clean up `ShardedWorkerHandlingConfig` (#9466)
* Split ShardedWorkerHandlingConfig

This is so that we have a type level understanding of when it is safe to
call `get_instance(..)` (as opposed to `should_handle(..)`).

* Remove special cases in ShardedWorkerHandlingConfig.

`ShardedWorkerHandlingConfig` tried to handle the various different ways
it was possible to configure federation senders and pushers. This led to
special cases that weren't hit during testing.

To fix this the handling of the different cases is moved from there and
`generic_worker` into the worker config class. This allows us to have
the logic in one place and allows the rest of the code to ignore the
different cases.
2021-02-24 13:23:18 +00:00
Erik Johnston 0b5c967813
Refactor to ensure we call check_consistency (#9470)
The idea here is to stop people forgetting to call `check_consistency`. Folks can still just pass in `None` to the new args in `build_sequence_generator`, but hopefully they won't.
2021-02-24 10:13:53 +00:00
Patrick Cloke 7292b7c0eb
Add back the deprecated SAML endpoint. (#9474) 2021-02-23 12:57:37 -05:00
Richard van der Hoff 713145d3de
Add a comment about systemd-python. (#9464)
This confused me for a while.
2021-02-23 13:42:36 +00:00
Patrick Cloke 65a9eb8994
Include newly added sequences in the port DB script. (#9449)
And ensure the consistency of `event_auth_chain_id`.
2021-02-23 07:33:24 -05:00
Erik Johnston 66f4949e7f
Fix deleting pushers when using sharded pushers. (#9465) 2021-02-22 21:14:42 +00:00
Richard van der Hoff 1b2d6d55c5
Remove vestiges of uploads_path config (#9462)
`uploads_path` was a thing that was never used; most of it was removed in #6628
but a few vestiges remained.
2021-02-22 19:54:49 +00:00