Commit Graph

19474 Commits

Author SHA1 Message Date
Sean f8bf83b811
Skip the final GC on shutdown to improve restart times (#10712)
Use `gc.freeze()` on exit to exclude all existing objects from the final GC.
In testing, this sped up shutdown by up to a few seconds.

`gc.freeze()` runs in constant time, so there is little chance of performance
regression.

Signed-off-by: Sean Quah <seanq@element.io>
2021-09-01 11:55:31 +01:00
Olivier Wilkinson (reivilibre) 6b2aca473a 1.42.0rc1 2021-09-01 11:47:24 +01:00
Andrew Morgan 3693ea61f5
Fix iteration in _remove_deleted_email_pushers background job. (#10734) 2021-09-01 09:13:01 +00:00
Patrick Cloke e2481dbe93
Allow configuration of the oEmbed URLs. (#10714)
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
2021-08-31 18:37:07 -04:00
Patrick Cloke 287918e2d4
Additional type hints for the client REST servlets (part 3). (#10707) 2021-08-31 17:22:29 +00:00
Patrick Cloke 78e590d473
Move the sessions delta to the latest schema version. (#10725)
This was erroneously put under schema version 62 instead of 63.
2021-08-31 16:38:43 +00:00
Richard van der Hoff 5d9e7e0c71 Merge branch 'master' into develop 2021-08-31 14:09:48 +01:00
Richard van der Hoff a4c8a2f08b 1.41.1 2021-08-31 13:43:28 +01:00
Richard van der Hoff 8c26f16c76
Fix up unit tests (#10723)
These were broken in an incorrect merge of GHSA-jj53-8fmw-f2w2 (cb35df9)
2021-08-31 12:56:22 +01:00
David Robertson 46ff99ef95
Advertise matrix-org.github.io/synapse docs (#10595)
Point to the book where possible, and use hyperlinks to github to refer to files not included in the book.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-08-31 11:29:27 +01:00
reivilibre cb35df940a
Merge pull request from GHSA-jj53-8fmw-f2w2 2021-08-31 11:24:09 +01:00
reivilibre 52c7a51cfc
Merge pull request from GHSA-3x4c-pq33-4w3q
* Add some tests to characterise the problem

Some failing. Current states:

  RoomsMemberListTestCase
test_get_member_list ...
[OK]
test_get_member_list_mixed_memberships ...
[OK]
test_get_member_list_no_permission ...
[OK]
test_get_member_list_no_permission_former_member ...
[OK]
test_get_member_list_no_permission_former_member_with_at_token ...
[FAIL]
test_get_member_list_no_room ...
[OK]
test_get_member_list_no_permission_with_at_token ...
[FAIL]

* Correct the tests

* Check user is/was member before divulging room membership

* Pull out only the 1 membership event we want.

* Update tests/rest/client/v1/test_rooms.py

Co-authored-by: Erik Johnston <erik@matrix.org>

* Fixup tests (following apply review suggestion)

Co-authored-by: Erik Johnston <erik@matrix.org>
2021-08-31 10:09:58 +01:00
Richard van der Hoff e3abc0a5cc Merge remote-tracking branch 'origin/release-v1.41' into develop 2021-08-27 16:35:02 +01:00
Richard van der Hoff 8f98260552
Fix incompatibility with Twisted < 21. (#10713)
Turns out that the functionality added in #10546 to skip TLS was incompatible
with older Twisted versions, so we need to be a bit more inventive.

Also, add a test to (hopefully) not break this in future. Sadly, testing TLS is
really hard.
2021-08-27 16:33:41 +01:00
Azrenbeth 54aa7047eb
Removed page summaries from the top of installation and contributing doc pages (#10711)
- Removed page summaries from CONTRIBUTING and installation pages as 
this information was already in the table of contents on the right hand side
- Fixed some broken links in CONTRIBUTING
- Added margin-right tag for when table of contents is being shown
(otherwise the text in the page sometimes overlaps with it)
2021-08-27 14:19:17 +00:00
Richard van der Hoff 051ddac53b
Clarifications to reverse_proxy.md (#10708)
* Update reverse_proxy.md

* Create 10708.doc
2021-08-27 12:54:21 +01:00
Patrick Cloke 029b7ad7b9
Remove unused `compare_digest` function. (#10706) 2021-08-27 07:08:02 -04:00
Dirk Klimpel e62cdbef1a
Improve ServerNoticeServlet to avoid duplicate requests (#10679)
Fixes: #9544
2021-08-27 09:16:40 +00:00
Erik Johnston c4fa4f37cb
Fix perf of fetching the same events many times. (#10703)
The code to deduplicate repeated fetches of the same set of events was
N^2 (over the number of events requested), which could lead to a process
being completely wedged.

The main fix is to deduplicate the returned deferreds so we only await
on a deferred once rather than many times. Seperately, when handling the
returned events from the defrered we only add the events we care about
to the event map to be returned (so that we don't pay the price of
inserting extraneous events into the dict).
2021-08-27 09:15:50 +00:00
Richard van der Hoff 1800aabfc2
Split `FederationHandler` in half (#10692)
The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.
2021-08-26 21:41:44 +01:00
Richard van der Hoff 96715d7633
Make `backfill` and `get_missing_events` use the same codepath (#10645)
Given that backfill and get_missing_events are basically the same thing, it's somewhat crazy that we have entirely separate code paths for them. This makes backfill use the existing get_missing_events code, and then clears up all the unused code.
2021-08-26 18:34:57 +01:00
Aaron Raimist 40f619eaa5
Validate new m.room.power_levels events (#10232)
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-08-26 17:07:58 +01:00
Azrenbeth ad17fbd20e
Remove pushers when deleting 3pid from account (#10581)
When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)
2021-08-26 13:53:57 +01:00
Patrick Cloke 1aa0dad021
Additional type hints for REST servlets (part 2). (#10674)
Applies the changes from #10665 to additional modules.
2021-08-26 11:53:52 +00:00
Patrick Cloke 5548fe0978
Cache the result of fetching the room hierarchy over federation. (#10647) 2021-08-26 07:16:53 -04:00
Andrew Morgan b45cc1530b
Make a note to leave a summary when one is bumping the schema version (#10621)
I found this easy to miss (and evidently, it looks like it was missed for schema version 62).
2021-08-25 17:00:44 +01:00
Patrick Cloke 882539e423
Ensure the base Docker image is rebuilt when running complement with workers. (#10686)
We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.
2021-08-25 10:18:23 -04:00
Sean 7367473f96
Fix error when selecting between thumbnails with the same quality (#10684)
Fixes #10318
2021-08-25 09:51:08 +00:00
Erik Johnston cd22fb568a Merge branch 'master' into develop 2021-08-24 17:13:29 +01:00
Erik Johnston f03cafb50c Update changelog 2021-08-24 16:06:33 +01:00
Erik Johnston 6f77a3d433 1.41.0 2021-08-24 15:31:55 +01:00
Patrick Cloke d12ba52f17
Persist room hierarchy pagination sessions to the database. (#10613) 2021-08-24 08:14:03 -04:00
Richard van der Hoff 15db8b7c7f
Correctly initialise the `synapse_user_logins` metric. (#10677)
Fix a bug where the prometheus metrics for SSO logins wouldn't be initialised
until the first user logged in with a given auth provider.
2021-08-24 09:17:51 +00:00
Hugo DELVAL 86415f162d
doc: add django-oauth-toolkit to oidc doc (#10192)
Signed-off-by: Hugo Delval <hugo.delval@gmail.com>
2021-08-23 17:12:36 +00:00
Azrenbeth 0c1d6f65d7
Enforce the max length for per-room display names / avatar URLs. (#10654)
To match the maximum lengths allowed for profile data.
2021-08-23 11:25:33 -04:00
Andrew Morgan 3e83f97154
Fix the titles in the OIDC documentation (#10639)
* Fix the titles in the OIDC documentation

Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.

* Changelog

* Add link to google idp docs
2021-08-23 14:58:31 +01:00
Dan Callahan 2efc838f05
Avoid duplicate issues from Twisted trunk failures (#10672)
Setting `update_existing: true` in the `create-an-issue` GitHub Action
will avoid opening duplicate issues if an open issue already exists with
an identical title.

If no open issues match the title, then a new issue will be created.

This helps avoid spamming our issue tracker should there be a failure
when testing against Twisted's trunk.

This PR also pins the SHA of the `create-an-issue` action to mitigate
the risk of a malicious actor gaining access to JasonEtco's account.

See GitHub's page on security hardening third party actions for more:
https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions

Signed-off-by: Dan Callahan <danc@element.io>
2021-08-23 13:06:49 +00:00
Patrick Cloke bd7d398b05
Additional type hints for the sync REST servlet. (#10666) 2021-08-23 08:14:42 -04:00
Patrick Cloke 2af6d31b78
Addtional type hints for the REST servlets. (#10665) 2021-08-23 08:14:17 -04:00
Patrick Cloke 31dac7ffee
Do not include stack traces for known exceptions when trying multiple federation destinations. (#10662) 2021-08-23 08:00:25 -04:00
Richard van der Hoff 4db65f911a
Run a nightly CI build against Twisted trunk. (#10651)
This creates a GHA workflow which runs at 8am every day, and runs mypy, trial and sytest against Twisted's current trunk. If any of the jobs fail, it opens an issue.
2021-08-23 11:12:45 +01:00
Callum Brown 947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142)
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
2021-08-21 22:14:43 +01:00
David Robertson ecd823d766
Flatten tests/rest/client/{v1,v2_alpha} too (#10667) 2021-08-20 17:50:44 +01:00
Dirk Klimpel f499dc38bc
Simplify tests for the device admin rest API. (#10664)
By replacing duplicated code with parameterized tests and
avoiding unnecessary dumping of JSON data.
2021-08-20 15:43:26 +00:00
David Robertson 7862d704fd Follow-up: format changelog, add licence (#10593)
Merged before approval; these comments from @clokep on that PR.
2021-08-20 16:33:52 +01:00
David Robertson ee3b2ac59a
Validate device_keys for C-S /keys/query requests (#10593)
* Validate device_keys for C-S /keys/query requests

Closes #10354

A small, not particularly critical fix. I'm interested in seeing if we
can find a more systematic approach though. #8445 is the place for any discussion.
2021-08-20 15:47:03 +01:00
Patrick Cloke 5cda75fede
Set room version 8 as preferred for restricted rooms. (#10571) 2021-08-20 07:17:50 -04:00
Richard van der Hoff e81d62009e
Split `on_receive_pdu` in half (#10640)
Here we split on_receive_pdu into two functions (on_receive_pdu and process_pulled_event), rather than having both cases in the same method. There's a tiny bit of overlap, but not that much.
2021-08-19 17:05:12 +00:00
Richard van der Hoff 50af1efe4b
Extract `_resolve_state_at_missing_prevs` (#10624)
This is a follow-up to #10615: it takes the code that constructs the state at a backwards extremity, and extracts it to a separate method.
2021-08-19 17:31:40 +01:00
Patrick Cloke 000aa89be6
Do not include rooms with an unknown room version in a sync response. (#10644)
A user will still see this room if it is in a local cache, but it will
not reappear if clearing the cache and reloading.
2021-08-19 11:12:55 -04:00