Commit Graph

20634 Commits

Author SHA1 Message Date
Andrew Morgan f26e390a40
Use Python 3.9 in Synapse dockerfiles by default (#12112) 2022-03-01 13:55:18 +00:00
Sean Quah 91bc15c772
Add `stop_cancellation` utility function (#12106) 2022-03-01 13:51:03 +00:00
reivilibre c893632319
Order in-flight state group queries in biggest-first order (#11610)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-01 13:41:57 +00:00
Richard van der Hoff e2e1d90a5e
Faster joins: persist to database (#12012)
When we get a partial_state response from send_join, store information in the
database about it:
 * store a record about the room as a whole having partial state, and stash the
   list of member servers too.
 * flag the join event itself as having partial state
 * also, for any new events whose prev-events are partial-stated, note that
   they will *also* be partial-stated.

We don't yet make any attempt to interpret this data, so API calls (and a bunch
of other things) are just going to get incorrect data.
2022-03-01 12:49:54 +00:00
Andrew Morgan 4ccc2d09aa
Advertise Python 3.10 support in setup.py (#12111) 2022-03-01 12:35:32 +00:00
Richard van der Hoff 5458eb8551
Fix 'Unhandled error in Deferred' (#12089)
* Fix 'Unhandled error in Deferred'

Fixes a CRITICAL "Unhandled error in Deferred" log message which happened when
a function wrapped with `@cachedList` failed

* Minor optimisation to cachedListDescriptor

we can avoid re-using `missing`, which saves looking up entries in
`deferreds_map`, and means we don't need to copy it.

* Improve type annotation on CachedListDescriptor
2022-03-01 09:51:38 +00:00
Richard van der Hoff 9d11fee8f2
Improve exception handling for concurrent execution (#12109)
* fix incorrect unwrapFirstError import

this was being imported from the wrong place

* Refactor `concurrently_execute` to use `yieldable_gather_results`

* Improve exception handling in `yieldable_gather_results`

Try to avoid swallowing so many stack traces.

* mark unwrapFirstError deprecated

* changelog
2022-03-01 09:34:30 +00:00
Dirk Klimpel 952efd0bca
Add type hints to `tests/rest/client` (#12094)
* Add type hints to `tests/rest/client`

* update `mypy.ini`

* newsfile

* add `test_register.py`
2022-02-28 18:59:00 +00:00
Patrick Cloke 7754af24ab
Remove the unstable `/spaces` endpoint. (#12073)
...and various code supporting it.

The /spaces endpoint was from an old version of MSC2946 and included
both a Client-Server and Server-Server API. Note that the unstable
/hierarchy endpoint (from the final version of MSC2946) is not yet
removed.
2022-02-28 18:33:00 +00:00
Patrick Cloke 1866fb39d7
Move experimental support for MSC3440 to /versions. (#12099)
Instead of being part of /capabilities, this matches a change to
MSC3440 to properly use these endpoints.
2022-02-28 18:29:09 +00:00
Dirk Klimpel 1901cb1d4a
Add type hints to `tests/rest/client` (#12084) 2022-02-28 17:47:37 +00:00
Richard van der Hoff 6c0b44a3d7
Fix `PushRuleEvaluator` and `Filter` to work on frozendicts (#12100)
* Fix `PushRuleEvaluator` to work on frozendicts

frozendicts do not (necessarily) inherit from dict, so this needs to handle
them correctly.

* Fix event filtering for frozen events

Looks like this one was introduced by #11194.
2022-02-28 17:40:24 +00:00
David Robertson 5565f454e1
Actually fix bad debug logging rejecting device list & signing key transactions (#12098) 2022-02-28 14:10:36 +00:00
Patrick Cloke 9e83521af8
Properly failover for unknown endpoints from Conduit/Dendrite. (#12077)
Before this fix, a legitimate 404 from a federation endpoint (e.g. due
to an unknown room) would be treated as an unknown endpoint. This
could cause unnecessary federation traffic.
2022-02-28 07:52:44 -05:00
Patrick Cloke 02d708568b
Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
lukasdenk ab3ef49059
synctl: print warning if synctl_cache_factor is set in config (#11865)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-02-28 11:42:13 +00:00
Sean Quah b43c3ef8e2
Ensure that `get_datastores().main` is typed (#12070)
Signed-off-by: Sean Quah <seanq@element.io>
2022-02-25 10:20:40 +00:00
Sean Quah f3fd8558cd
Minor typing fixes for `synapse/storage/persist_events.py` (#12069)
Signed-off-by: Sean Quah <seanq@element.io>
2022-02-25 10:19:49 +00:00
Dirk Klimpel 54e74cc15f
Add type hints to `tests/rest/client` (#12072) 2022-02-24 13:56:38 -05:00
reivilibre 2cc5ea933d
Add support for MSC3202: sending one-time key counts and fallback key usage states to Application Services. (#11617)
Co-authored-by: Erik Johnston <erik@matrix.org>
2022-02-24 17:55:45 +00:00
Sean Quah 41cf4c2cf6
Fix non-strings in the `event_search` table (#12037)
Don't attempt to add non-string `value`s to `event_search` and add a
background update to clear out bad rows from `event_search` when
using sqlite.

Signed-off-by: Sean Quah <seanq@element.io>
2022-02-24 11:52:28 +00:00
Patrick Cloke c56bfb08bc
Add documentation for missing worker types. (#11599)
And clean-up the endpoints which should be routed to workers.
2022-02-23 22:49:04 +00:00
Erik Johnston a711ae78a8
Add logging to `/sync` for debugging #11916 (#12068) 2022-02-23 14:22:22 +00:00
Dirk Klimpel 64c73c6ac8
Add type hints to `tests/rest/client` (#12066) 2022-02-23 08:33:19 -05:00
Richard van der Hoff 5b2b36809f
Remove more references to `get_datastore` (#12067)
These have snuck in since #12031 was started.

Also a couple of other cleanups while we're in the area.
2022-02-23 12:35:53 +00:00
Richard van der Hoff e24ff8ebe3
Remove `HomeServer.get_datastore()` (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Nicolas Werner c1ac2a8135
Rename default branch of complement.sh to main (#12063)
The complement.sh script relies on the name of the ref matching the name
of the unpacked folder. The branch redirect from renaming the default
branch breaks that assumption.

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
2022-02-23 10:06:18 +00:00
Travis Ralston e3fe6347be
Remove excess condition on `knock->leave` check (#11900) 2022-02-22 18:35:01 +00:00
Erik Johnston 6d14b3dabf
Better error message when failing to request from another process (#12060) 2022-02-22 15:52:08 +00:00
Brendan Abolivier 250104d357
Implement account status endpoints (MSC3720) (#12001)
See matrix-org/matrix-doc#3720

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-02-22 15:10:10 +00:00
David Robertson 94a396e7c4
Prune setup.cfg some more (#12059)
* Remove `trial` section from setup.cfg

This was added in the initial commit from 2014. I can't see that it does
anything. Maybe it's there so that you can run `trial` without any extra
args, but if I do that then I just get the `--help` message.

* Move flake8's config to its own file
2022-02-22 14:52:56 +00:00
reivilibre dcb6a37837
Cap the number of in-flight requests for state from a single group (#11608) 2022-02-22 14:24:31 +00:00
Patrick Cloke 7bcc28f82f
Use room version 9 as the default room version (per MSC3589). (#12058) 2022-02-22 14:09:40 +00:00
Richard van der Hoff 81364db49b
Run `_handle_queued_pdus` as a background process (#12041)
... to ensure it gets a proper log context, mostly.
2022-02-22 13:33:22 +00:00
reivilibre 235d2916ce
Fix slow performance of `/logout` in some cases where refresh tokens are in use. The slowness existed since the initial implementation of refresh tokens. (#12056) 2022-02-22 13:29:04 +00:00
Erik Johnston 6a1bad511d Merge remote-tracking branch 'origin/master' into develop 2022-02-22 12:26:49 +00:00
Erik Johnston 07f82ac29b Merge branch 'release-v1.53' 2022-02-22 12:26:17 +00:00
Richard van der Hoff 7273011f60
Faster joins: Support for calling `/federation/v1/state` (#12013)
This is an endpoint that we have server-side support for, but no client-side support. It's going to be useful for resyncing partial-stated rooms, so let's introduce it.
2022-02-22 12:17:10 +00:00
AndrewRyanChama 066171643b
Fetch images when previewing Twitter URLs. (#11985)
By including "bot" in the User-Agent, which some sites use
to decide whether to include additional Open Graph information.
2022-02-22 07:11:39 -05:00
Erik Johnston 79c18e0709 Merge remote-tracking branch 'origin/master' into develop 2022-02-22 12:05:48 +00:00
Erik Johnston 1bf9cbbf75 Update changelog 2022-02-22 12:00:46 +00:00
Erik Johnston 45e2c04f78 Update changelog 2022-02-22 12:00:05 +00:00
reivilibre 546b9c9e64
Add more tests for in-flight state query duplication. (#12033) 2022-02-22 11:44:11 +00:00
David Robertson af2c1e3d2a
Tidy the building of sdists and wheels (#12051)
* Don't build distribution pkgs in tests.yml
* Run `release-artifacts` on release branches
* Use backend-meta workflow for packaging
2022-02-22 11:33:37 +00:00
Erik Johnston 551dd8c9f8 1.53.0 2022-02-22 11:32:11 +00:00
David Robertson 1ae492c8c0
Move isort config to `pyproject.toml` (#12052) 2022-02-22 11:30:19 +00:00
Travis Ralston d7cb0dcbaa
Use v3 endpoints for fallback auth (Matrix 1.1) (#12019) 2022-02-22 11:20:45 +00:00
Richard van der Hoff 3070af4809
remote join processing: get create event from state, not auth_chain (#12039)
A follow-up to #12005, in which I apparently missed that there are a bunch of other places that assume the create event is in the auth chain.
2022-02-21 19:27:35 +00:00
Richard van der Hoff a85dde3445
Minor typing fixes (#12034)
These started failing in
https://github.com/matrix-org/synapse/pull/12031... I'm a bit mystified by how
they ever worked.
2022-02-21 18:37:04 +00:00
Dirk Klimpel 7c82da27aa
Add type hints to `synapse/storage/databases/main` (#11984) 2022-02-21 16:03:06 +00:00