7d3f639844
Fix compatibility issue with older Twisted in tests.
...
Older Twisted (18.4.0) returns TimeoutError instead of
ConnectingCancelledError when connection times out.
This change allows tests to be compatible with this behaviour.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-09-25 02:51:18 +10:00
04eed80a73
Merge branch 'master' into develop
2018-09-24 23:42:25 +10:00
829213523e
Features
...
--------
- Python 3.5 and 3.6 support is now in beta.
([\#3576](https://github.com/matrix-org/synapse/issues/3576 ))
- Implement `event_format` filter param in `/sync`
([\#3790](https://github.com/matrix-org/synapse/issues/3790 ))
- Add synapse_admin_mau:registered_reserved_users metric to expose
number of real reaserved users
([\#3846](https://github.com/matrix-org/synapse/issues/3846 ))
Bugfixes
--------
- Remove connection ID for replication prometheus metrics, as it creates
a large number of new series.
([\#3788](https://github.com/matrix-org/synapse/issues/3788 ))
- guest users should not be part of mau total
([\#3800](https://github.com/matrix-org/synapse/issues/3800 ))
- Bump dependency on pyopenssl 16.x, to avoid incompatibility with
recent Twisted.
([\#3804](https://github.com/matrix-org/synapse/issues/3804 ))
- Fix existing room tags not coming down sync when joining a room
([\#3810](https://github.com/matrix-org/synapse/issues/3810 ))
- Fix jwt import check
([\#3824](https://github.com/matrix-org/synapse/issues/3824 ))
- fix VOIP crashes under Python 3 (#3821 )
([\#3835](https://github.com/matrix-org/synapse/issues/3835 ))
- Fix manhole so that it works with latest openssh clients
([\#3841](https://github.com/matrix-org/synapse/issues/3841 ))
- Fix outbound requests occasionally wedging, which can result in
federation breaking between servers.
([\#3845](https://github.com/matrix-org/synapse/issues/3845 ))
- Show heroes if room name/canonical alias has been deleted
([\#3851](https://github.com/matrix-org/synapse/issues/3851 ))
- Fix handling of redacted events from federation
([\#3859](https://github.com/matrix-org/synapse/issues/3859 ))
- ([\#3874](https://github.com/matrix-org/synapse/issues/3874 ))
- Mitigate outbound federation randomly becoming wedged
([\#3875](https://github.com/matrix-org/synapse/issues/3875 ))
Internal Changes
----------------
- CircleCI tests now run on the potential merge of a PR.
([\#3704](https://github.com/matrix-org/synapse/issues/3704 ))
- http/ is now ported to Python 3.
([\#3771](https://github.com/matrix-org/synapse/issues/3771 ))
- Improve human readable error messages for threepid
registration/account update
([\#3789](https://github.com/matrix-org/synapse/issues/3789 ))
- Make /sync slightly faster by avoiding needless copies
([\#3795](https://github.com/matrix-org/synapse/issues/3795 ))
- handlers/ is now ported to Python 3.
([\#3803](https://github.com/matrix-org/synapse/issues/3803 ))
- Limit the number of PDUs/EDUs per federation transaction
([\#3805](https://github.com/matrix-org/synapse/issues/3805 ))
- Only start postgres instance for postgres tests on Travis CI
([\#3806](https://github.com/matrix-org/synapse/issues/3806 ))
- tests/ is now ported to Python 3.
([\#3808](https://github.com/matrix-org/synapse/issues/3808 ))
- crypto/ is now ported to Python 3.
([\#3822](https://github.com/matrix-org/synapse/issues/3822 ))
- rest/ is now ported to Python 3.
([\#3823](https://github.com/matrix-org/synapse/issues/3823 ))
- add some logging for the keyring queue
([\#3826](https://github.com/matrix-org/synapse/issues/3826 ))
- speed up lazy loading by 2-3x
([\#3827](https://github.com/matrix-org/synapse/issues/3827 ))
- Improved Dockerfile to remove build requirements after building
reducing the image size.
([\#3834](https://github.com/matrix-org/synapse/issues/3834 ))
- Disable lazy loading for incremental syncs for now
([\#3840](https://github.com/matrix-org/synapse/issues/3840 ))
- federation/ is now ported to Python 3.
([\#3847](https://github.com/matrix-org/synapse/issues/3847 ))
- Log when we retry outbound requests
([\#3853](https://github.com/matrix-org/synapse/issues/3853 ))
- Removed some excess logging messages.
([\#3855](https://github.com/matrix-org/synapse/issues/3855 ))
- Speed up purge history for rooms that have been previously purged
([\#3856](https://github.com/matrix-org/synapse/issues/3856 ))
- Refactor some HTTP timeout code.
([\#3857](https://github.com/matrix-org/synapse/issues/3857 ))
- Fix running merged builds on CircleCI
([\#3858](https://github.com/matrix-org/synapse/issues/3858 ))
- Fix typo in replication stream exception.
([\#3860](https://github.com/matrix-org/synapse/issues/3860 ))
- Add in flight real time metrics for Measure blocks
([\#3871](https://github.com/matrix-org/synapse/issues/3871 ))
- Disable buffering and automatic retrying in treq requests to prevent
timeouts. ([\#3872](https://github.com/matrix-org/synapse/issues/3872 ))
- mention jemalloc in the README
([\#3877](https://github.com/matrix-org/synapse/issues/3877 ))
- Remove unmaintained "nuke-room-from-db.sh" script
([\#3888](https://github.com/matrix-org/synapse/issues/3888 ))
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEIQBQJ4l+yK4dlKkFIwi0edOSShEFAluo6WIACgkQIwi0edOS
ShHFIwf/ZURGbih+6q4jy8voc4wUbgtuNGabmFLQeXGhIXIOYltvKOrB8wcCEi/4
jODztPqewikZoI7NFA4M44GToQKacuIEOYK/M513l3t/mEpBQYDdvLGDccM+6XGc
RcrWqQEfJYkiNyWNmLx5TJ4bYaNelrpfUyNbbONph0wLO7o6f7ANoGyqqVKm/7pD
ufUwwBLqzd2U2vuvNCbQLSW0LkognP91vYytHMN5NTkoAR4BSYfDxQCShUSoNSlJ
098CEYjXZaDcY9ypGA6wlflbNfl3NC2KcTajLOCHdKe2U+FbEMc/9Ou690GbDRJ9
6L+Bonfwx0tsF04LHsteTdyTIOVp+Q==
=Dv5r
-----END PGP SIGNATURE-----
Merge tag 'v0.33.5'
Features
--------
- Python 3.5 and 3.6 support is now in beta.
([\#3576](https://github.com/matrix-org/synapse/issues/3576 ))
- Implement `event_format` filter param in `/sync`
([\#3790](https://github.com/matrix-org/synapse/issues/3790 ))
- Add synapse_admin_mau:registered_reserved_users metric to expose
number of real reaserved users
([\#3846](https://github.com/matrix-org/synapse/issues/3846 ))
Bugfixes
--------
- Remove connection ID for replication prometheus metrics, as it creates
a large number of new series.
([\#3788](https://github.com/matrix-org/synapse/issues/3788 ))
- guest users should not be part of mau total
([\#3800](https://github.com/matrix-org/synapse/issues/3800 ))
- Bump dependency on pyopenssl 16.x, to avoid incompatibility with
recent Twisted.
([\#3804](https://github.com/matrix-org/synapse/issues/3804 ))
- Fix existing room tags not coming down sync when joining a room
([\#3810](https://github.com/matrix-org/synapse/issues/3810 ))
- Fix jwt import check
([\#3824](https://github.com/matrix-org/synapse/issues/3824 ))
- fix VOIP crashes under Python 3 (#3821 )
([\#3835](https://github.com/matrix-org/synapse/issues/3835 ))
- Fix manhole so that it works with latest openssh clients
([\#3841](https://github.com/matrix-org/synapse/issues/3841 ))
- Fix outbound requests occasionally wedging, which can result in
federation breaking between servers.
([\#3845](https://github.com/matrix-org/synapse/issues/3845 ))
- Show heroes if room name/canonical alias has been deleted
([\#3851](https://github.com/matrix-org/synapse/issues/3851 ))
- Fix handling of redacted events from federation
([\#3859](https://github.com/matrix-org/synapse/issues/3859 ))
- ([\#3874](https://github.com/matrix-org/synapse/issues/3874 ))
- Mitigate outbound federation randomly becoming wedged
([\#3875](https://github.com/matrix-org/synapse/issues/3875 ))
Internal Changes
----------------
- CircleCI tests now run on the potential merge of a PR.
([\#3704](https://github.com/matrix-org/synapse/issues/3704 ))
- http/ is now ported to Python 3.
([\#3771](https://github.com/matrix-org/synapse/issues/3771 ))
- Improve human readable error messages for threepid
registration/account update
([\#3789](https://github.com/matrix-org/synapse/issues/3789 ))
- Make /sync slightly faster by avoiding needless copies
([\#3795](https://github.com/matrix-org/synapse/issues/3795 ))
- handlers/ is now ported to Python 3.
([\#3803](https://github.com/matrix-org/synapse/issues/3803 ))
- Limit the number of PDUs/EDUs per federation transaction
([\#3805](https://github.com/matrix-org/synapse/issues/3805 ))
- Only start postgres instance for postgres tests on Travis CI
([\#3806](https://github.com/matrix-org/synapse/issues/3806 ))
- tests/ is now ported to Python 3.
([\#3808](https://github.com/matrix-org/synapse/issues/3808 ))
- crypto/ is now ported to Python 3.
([\#3822](https://github.com/matrix-org/synapse/issues/3822 ))
- rest/ is now ported to Python 3.
([\#3823](https://github.com/matrix-org/synapse/issues/3823 ))
- add some logging for the keyring queue
([\#3826](https://github.com/matrix-org/synapse/issues/3826 ))
- speed up lazy loading by 2-3x
([\#3827](https://github.com/matrix-org/synapse/issues/3827 ))
- Improved Dockerfile to remove build requirements after building
reducing the image size.
([\#3834](https://github.com/matrix-org/synapse/issues/3834 ))
- Disable lazy loading for incremental syncs for now
([\#3840](https://github.com/matrix-org/synapse/issues/3840 ))
- federation/ is now ported to Python 3.
([\#3847](https://github.com/matrix-org/synapse/issues/3847 ))
- Log when we retry outbound requests
([\#3853](https://github.com/matrix-org/synapse/issues/3853 ))
- Removed some excess logging messages.
([\#3855](https://github.com/matrix-org/synapse/issues/3855 ))
- Speed up purge history for rooms that have been previously purged
([\#3856](https://github.com/matrix-org/synapse/issues/3856 ))
- Refactor some HTTP timeout code.
([\#3857](https://github.com/matrix-org/synapse/issues/3857 ))
- Fix running merged builds on CircleCI
([\#3858](https://github.com/matrix-org/synapse/issues/3858 ))
- Fix typo in replication stream exception.
([\#3860](https://github.com/matrix-org/synapse/issues/3860 ))
- Add in flight real time metrics for Measure blocks
([\#3871](https://github.com/matrix-org/synapse/issues/3871 ))
- Disable buffering and automatic retrying in treq requests to prevent
timeouts. ([\#3872](https://github.com/matrix-org/synapse/issues/3872 ))
- mention jemalloc in the README
([\#3877](https://github.com/matrix-org/synapse/issues/3877 ))
- Remove unmaintained "nuke-room-from-db.sh" script
([\#3888](https://github.com/matrix-org/synapse/issues/3888 ))
2018-09-24 23:41:35 +10:00
e3aa2c0b75
towncrier
2018-09-24 23:40:27 +10:00
e302f40e20
update version
2018-09-24 23:40:05 +10:00
19dc676d1a
Fix ExpiringCache.__len__ to be accurate
...
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
2018-09-21 16:25:42 +01:00
5230bc1471
Newsfile
2018-09-21 15:05:37 +01:00
fdd1a62e8d
Add a five minute cache to get_destination_retry_timings
...
Hopefully helps with #3931
2018-09-21 14:56:12 +01:00
79eded1ae4
Make ExpiringCache slightly more performant
2018-09-21 14:52:21 +01:00
d3f80cbc9c
Newsfile
2018-09-21 14:24:39 +01:00
8601c24287
Fix some instances of ExpiringCache not expiring cache items
...
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
2018-09-21 14:19:46 +01:00
ad53a5497d
Merge pull request #3927 from matrix-org/erikj/handle_background_errors
...
Handle exceptions thrown by background tasks
2018-09-21 09:26:30 +01:00
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients ( #3916 )
...
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
94ae1dea3c
Add missing logger
2018-09-20 17:05:34 +01:00
168491c412
Newsfile
2018-09-20 16:16:52 +01:00
9ea408441f
Handle exceptions thrown by background tasks
...
Fixes #3921
2018-09-20 16:15:21 +01:00
3af7a95a9d
add changelog
2018-09-20 14:55:11 +02:00
8dfb33d325
make python 3 work in the docker container
2018-09-20 14:55:11 +02:00
13f6f1624b
Newsfile
2018-09-20 13:52:09 +01:00
b28a7ed503
Fix spurious exceptions when client closes conncetion
...
If a HTTP handler throws an exception while processing a request we
automatically write a JSON error response. If the handler had already
started writing a response twisted throws an exception.
We should check for this case and simple abort the connection if there
was an error after the response had started being written.
2018-09-20 13:44:20 +01:00
23b53b4ef8
Merge pull request #3868 from matrix-org/neilj/fix_room_invite_mail_links
...
Neilj/fix room invite mail links
2018-09-20 13:32:38 +01:00
6bb9a4b1d6
changelog
2018-09-20 13:15:20 +01:00
703de4ec13
Comments and interface cleanup for on_receive_pdu
...
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
2018-09-20 13:06:55 +01:00
1f3f5fcf52
Fix client IPs being broken on Python 3 ( #3908 )
2018-09-20 20:14:34 +10:00
3fd68d533b
Merge pull request #3914 from matrix-org/erikj/remove_retry_cache
...
Remove get_destination_retry_timings cache
2018-09-20 10:54:49 +01:00
741571cf22
Add a way to run tests in PostgreSQL in Docker ( #3699 )
2018-09-20 18:12:45 +10:00
aeca5a5ed5
Add a regression test for logging on failed connections ( #3912 )
2018-09-20 16:28:18 +10:00
642199570c
Improve the logging when handling a federation transaction ( #3904 )
...
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:
- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
them into other, more cryptic, exceptions.
2018-09-19 17:28:18 +01:00
ce846bb620
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing
2018-09-19 15:08:36 +01:00
1a24d4effa
Newsfile
2018-09-19 15:03:05 +01:00
bbab6ebfd9
Fix up changelog and remove spurious comment
2018-09-19 14:45:14 +01:00
392a54128c
pep8
2018-09-19 14:37:49 +01:00
83ee5592a5
Newsfile
2018-09-19 14:22:59 +01:00
b9158ac2bf
Remove get_destination_retry_timings cache
...
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.
Fixes #3798
2018-09-19 14:22:57 +01:00
cb016baa37
Merge pull request #3910 from matrix-org/erikj/update_timeout
...
Update to use new timeout function everywhere.
2018-09-19 11:45:11 +01:00
80d2d50f47
Fixup
2018-09-19 11:19:47 +01:00
9407bcf37a
Replace custom DeferredTimeoutError with defer.TimeoutError
2018-09-19 11:07:29 +01:00
6c48aa0256
Run canceller first to allow it to generate correct error
2018-09-19 11:07:27 +01:00
6bbe3d5732
Newsfile
2018-09-19 10:43:13 +01:00
a334e1cace
Update to use new timeout function everywhere.
...
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842 .
The new helper function doesn't suffer from this problem.
2018-09-19 10:39:40 +01:00
05b9937cd7
update changelog for #3909
2018-09-19 09:17:54 +01:00
47c02e6332
Merge pull request #3909 from turt2live/travis/fix-logging-1
...
Fix matrixfederationclient.py logging: Destination is a string
2018-09-19 18:14:47 +10:00
3f0d8e6b09
Remove documentation referencing Cygwin ( #3873 )
2018-09-19 18:14:30 +10:00
3d6b24fb1b
Merge pull request #3907 from matrix-org/rav/set_sni_to_server_name
...
Set SNI to the server_name, not whatever was in the SRV record
2018-09-19 17:59:33 +10:00
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout
...
Bump timeout on get_missing_events request
2018-09-19 17:57:48 +10:00
13b31a9baf
Changelog
2018-09-18 15:30:25 -06:00
35aec19f0a
Destination is a string
2018-09-18 15:29:30 -06:00
38ead946a9
Merge remote-tracking branch 'origin/develop' into neilj/fix_room_invite_mail_links
2018-09-18 19:02:45 +01:00
a219ce8726
Use directory server for room joins ( #3899 )
...
When we do a join, always try the server we used for the alias lookup first.
Fixes #2418
2018-09-18 18:27:37 +01:00
31c15dcb80
Refactor matrixfederationclient to fix logging ( #3906 )
...
We want to wait until we have read the response body before we log the request
as complete, otherwise a confusing thing happens where the request appears to
have completed, but we later fail it.
To do this, we factor the salient details of a request out to a separate
object, which can then keep track of the txn_id, so that it can be logged.
2018-09-18 18:17:15 +01:00
Oleg Girko