Commit Graph

19697 Commits

Author SHA1 Message Date
Sean Quah 91f2bd0907 Prevent the media store from writing outside of the configured directory
Also tighten validation of server names by forbidding invalid characters
in IPv6 addresses and empty domain labels.
2021-11-19 13:39:15 +00:00
David Robertson 9f9d82aa84
1.47.0 2021-11-17 13:10:12 +00:00
Andrew Morgan 7baa671dc8 fix up changelog language 2021-11-16 14:42:21 +00:00
Andrew Morgan 729acd82c8 mark the migration file migration as a bug 2021-11-16 14:41:21 +00:00
Andrew Morgan edcdc5fd82 1.47.0rc3 2021-11-16 14:34:46 +00:00
Andrew Morgan 6e084b62b8
Rename `remove_deleted_devices_from_device_inbox` to ensure it is always run (#11353)
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-16 13:16:43 +00:00
Andrew Morgan 9c59e117db
Run _upgrade_existing_database on workers if at current schema_version (#11346)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-15 17:34:15 +00:00
Dirk Klimpel b596a1eb80 Move sql file for `remove_deleted_devices_from_device_inbox` into v65 (#11303) 2021-11-15 11:47:30 +00:00
Olivier Wilkinson (reivilibre) 595f28529c Changelog tweak from feedback 2021-11-10 09:54:34 +00:00
Olivier Wilkinson (reivilibre) ef7f9286d1 Move Debian changelog entries to rc2 since rc1 was not published 2021-11-10 09:48:50 +00:00
Olivier Wilkinson (reivilibre) 82e62b488a 1.47.0rc2 2021-11-10 09:44:38 +00:00
Olivier Wilkinson (reivilibre) af6374905a Correct the Debian changelog 2021-11-10 09:37:48 +00:00
Olivier Wilkinson (reivilibre) dc5f524974 Update __init__.py 2021-11-09 13:51:08 +00:00
Olivier Wilkinson (reivilibre) a754510f28 Changelog tweaks from review 2021-11-09 13:22:36 +00:00
Olivier Wilkinson (reivilibre) b67a7c62a2 Make Deprecations and Removals more prominent 2021-11-09 12:32:05 +00:00
Olivier Wilkinson (reivilibre) 1a4f10045f Changelog tweaks 2021-11-09 12:30:15 +00:00
Olivier Wilkinson (reivilibre) 01f61da77f 1.47.0rc1 2021-11-09 12:17:35 +00:00
Erik Johnston af784644c3
Include cross-signing signatures when syncing remote devices for the first time (#11234)
When fetching remote devices for the first time, we did not correctly include the cross signing keys in the returned results.

c.f. #11159
2021-11-09 11:45:36 +00:00
rogersheu 820337e6a4
Require body for read receipts with user-agent exceptions (#11157)
Co-authored-by: reivilibre <olivier@librepush.net>
2021-11-09 10:26:07 +00:00
Eric Eastwood 84f235aea4
Rename to more clear `get_insertion_event_id_by_batch_id` (MSC2716) (#11244)
`get_insertion_event_by_batch_id` -> `get_insertion_event_id_by_batch_id`

Split out from https://github.com/matrix-org/synapse/pull/11114
2021-11-08 21:21:10 -06:00
Erik Johnston 4ee71b9637
Add some background update admin APIs (#11263)
Fixes #11259
2021-11-08 16:08:02 +00:00
Patrick Cloke 0c82d4aabe
Fix typo in comment from #11255. (#11276) 2021-11-08 14:36:49 +00:00
Richard van der Hoff 86a497efaa
Default value for `public_baseurl` (#11210)
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
2021-11-08 14:13:10 +00:00
Dan Callahan 556a488209
Address review feedback from #11269 (#11273)
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-08 11:57:37 +00:00
Erik Johnston a55e1ec9af
Blacklist new sytest validation test (#11270) 2021-11-08 10:37:43 +00:00
Erik Johnston 98c8fc6ce8
Handle federation inbound instances being killed more gracefully (#11262)
* Make lock better handle process being killed

If the process gets killed and restarted (so that it didn't have a
chance to drop its locks gracefully) then there may still be locks in
the DB that are for the same instance that haven't yet timed out but are
safe to delete.

We handle this case by a) checking if the current instance already has
taken out the lock, and b) if not then ignoring locks that are for the
same instance.

* Periodically check for old staged events

This is to protect against other instances dying and their locks timing
out.
2021-11-08 09:54:47 +00:00
Dan Callahan 9799c569bb
Minor cleanup to Debian packaging (#11269)
* Remove unused Vagrant scripts

* Change package Architecture to any

* Preinstall the wheel package when building venvs.

Addresses the following warnings during Debian builds:

    Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.

* Allow /etc/default/matrix-synapse to be missing

Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":

> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.

Signed-off-by: Dan Callahan <danc@element.io>
2021-11-07 21:18:33 +00:00
Julian 09cb441a04
Add doc to integrate synapse with LemonLDAP OIDC (#11257)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>
2021-11-05 12:08:02 +00:00
Erik Johnston a37df1b091
Fix rolling back when using workers (#11255)
Fixes #11252
2021-11-05 11:12:10 +00:00
reivilibre 499c44d696
Make minor correction to type of auth_checkers callbacks (#11253) 2021-11-04 17:10:11 +00:00
Richard van der Hoff f36434590c
Additional test for `cachedList` (#11246)
I was trying to understand how `cachedList` works, and ended up writing this
extra test. I figure we may as well keep it.
2021-11-04 14:45:34 +00:00
Sean Quah 8eec25a1d9
Track ongoing event fetches correctly in the presence of failure (#11240)
When an event fetcher aborts due to an exception, `_event_fetch_ongoing`
must be decremented, otherwise the event fetcher would never be
replaced. If enough event fetchers were to fail, no more events would be
fetched and requests would get stuck waiting for events.
2021-11-04 10:33:53 +00:00
Nick Barrett a271e233e9
Add a linearizer on (appservice, stream) when handling ephemeral events. (#11207)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-03 16:51:00 +00:00
Nick Barrett af54167516
Enable passing typing stream writers as a list. (#11237)
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
2021-11-03 14:25:47 +00:00
Patrick Cloke 2735b3e6f2
Remove a debug statement from tests. (#11239) 2021-11-03 13:11:16 +00:00
Erik Johnston bcc115c28d
Add twine and towncrier as dev dependencies (#11233)
We don't pin them as we execute them as commands, rather than use them
as libs.
2021-11-03 11:10:25 +00:00
Andrew Morgan d688a6dee5 fix a small typo in the delete room api docs 2021-11-03 11:09:00 +00:00
Eric Eastwood da0040785e
Support sending no `state_events_at_start` in the MSC2716 `/batch_send` endpoint (#11188)
As brought up by @tulir, https://matrix.to/#/!SBYNQlpqkwJzFIdzxI:nevarro.space/$Gwnb2ZvXHc3poYXuBhho0cmoYq4KJ11Jh3m5s8kjNOM?via=nevarro.space&via=beeper.com&via=matrix.org

This use case only works if the user is already joined in the current room state
at the given `?prev_event_id`
2021-11-03 03:13:51 -05:00
Erik Johnston 6250b95efe
Add index to `local_group_updates.stream_id` (#11231)
This should speed up startup times and generally increase performance of
groups.
2021-11-02 15:46:48 +00:00
Erik Johnston 237f7eb87a Merge remote-tracking branch 'origin/master' into develop 2021-11-02 14:28:27 +00:00
Patrick Cloke c01bc5f43d
Add remaining type hints to `synapse.events`. (#11098) 2021-11-02 09:55:52 -04:00
Erik Johnston 2d44ee6868 Update changelog 2021-11-02 13:25:42 +00:00
Erik Johnston df84ad602b 1.46.0 2021-11-02 13:23:01 +00:00
Dirk Klimpel 4535532526
Delete messages for hidden devices from `device_inbox` (#11199) 2021-11-02 13:18:30 +00:00
Andrew Morgan c9c3aea9b1
Fix providing a `RoomStreamToken` instance to `_notify_app_services_ephemeral` (#11137)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-02 10:39:02 +00:00
Dirk Klimpel 7537201840
Add search by room ID and room alias to List Room admin API (#11099)
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-02 10:01:13 +00:00
Richard van der Hoff 46d0937447
ObservableDeferred: run observers in order (#11229) 2021-11-02 00:17:35 +00:00
Dirk Klimpel 93aa670642
Update outdated links in `PULL_REQUEST_TEMPLATE.md` (#11225) 2021-11-01 17:29:51 +00:00
Shay f5c6a80886
Handle missing Content-Type header when accessing remote media (#11200)
* add code to handle missing content-type header and a test to verify that it works

* add handling for missing content-type in the /upload endpoint as well

* slightly refactor test code to put private method in approriate place

* handle possible null value for content-type when pulling from the local db

* add changelog

* refactor test and add code to handle missing content-type in cached remote media

* requested changes

* Update changelog.d/11200.bugfix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01 10:26:02 -07:00
Shay e81fa92648
Add `use_float=true` to ijson calls in Synapse (#11217)
* add use_float=true to ijson calls

* lints

* add changelog

* Update changelog.d/11217.bugfix

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01 09:28:04 -07:00