Commit Graph

754 Commits

Author SHA1 Message Date
Aaron Raimist 44dc4c365b
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:10:21 -06:00
Neil Johnson 7039ece8fb Neilj/fix autojoin (#4223)
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Amber Brown 8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Neil Johnson f9b136a886 Neilj/fix mau initial reserved users (#4211)
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
2018-11-28 20:33:41 +11:00
Richard van der Hoff 944d524f18 Support m.login.sso (#4220)
* Clean up the CSS for the fallback login form

I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.

The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.

* Support m.login.sso

MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).

* changelog
2018-11-27 18:51:52 +11:00
Richard van der Hoff a44c0a096f Check logcontexts before and after each test (#4190)
* Add better diagnostics to flakey keyring test

* fix interpolation fail

* Check logcontexts before and after each test

* update changelog

* update changelog
2018-11-27 13:47:18 +11:00
Richard van der Hoff 80527b568d Fix more logcontext leaks in tests (#4209) 2018-11-27 13:01:04 +11:00
Richard van der Hoff de8772a655 Do a GC after each test to fix logcontext leaks (#4227)
* Some words about garbage collections and logcontexts

* Do a GC after each test to fix logcontext leaks

This feels like an awful hack, but...

* changelog
2018-11-27 13:00:33 +11:00
Richard van der Hoff 6c18cc4b50 Ignore __pycache__ directories in schema delta dir
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
2018-11-20 22:52:34 +00:00
Neil Johnson 78ba0e7ab8 Remove riot.im from the list of trusted Identity Servers in the default configuration (#4207) 2018-11-20 12:29:25 +01:00
Richard van der Hoff 416c671474
Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes
Fix some logcontext leaks
2018-11-20 12:19:19 +01:00
Amber Brown 31425d82a3 Merge remote-tracking branch 'origin/master' into develop 2018-11-19 12:55:25 -06:00
Amber Brown 80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Richard van der Hoff 0c05da2e2e changelog 2018-11-19 17:07:42 +00:00
Richard van der Hoff 10cdf519aa
Merge pull request #4182 from aaronraimist/update-issue-template
Add a pull request template and add multiple issue templates
2018-11-19 14:24:30 +01:00
Aaron Raimist cc2cf2da97
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-11-18 12:42:08 -06:00
Amber Brown 4285c818ec
Merge pull request #4193 from kivikakk/add-openbsd-prereq
add jpeg to OpenBSD prereq list
2018-11-17 14:27:53 -06:00
Ashe Connor ceca3b2f30
add changelog.d entry 2018-11-17 15:01:02 +11:00
Travis Ralston d75db3df59 Changelog 2018-11-15 20:44:57 -07:00
Amber Brown 8b1affe7d5
Fix Content-Disposition in media repository (#4176) 2018-11-15 15:55:58 -06:00
Travis Ralston 835779f7fb Add option to track MAU stats (but not limit people) (#3830) 2018-11-15 18:08:27 +00:00
Amber Brown df758e155d
Use <meta> tags to discover the per-page encoding of html previews (#4183) 2018-11-15 11:05:08 -06:00
Amber Brown a51288e5d6
Add a coveragerc (#4180) 2018-11-15 10:50:08 -06:00
Richard van der Hoff 4f8bb633c7 Update README for #1491 fix 2018-11-15 10:03:36 +00:00
Richard van der Hoff 0c4dc6fd76 changelog 2018-11-14 10:48:08 +00:00
David Baker 0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers
Make e2e backup versions numeric in the DB
2018-11-14 07:55:48 +00:00
Aaron Raimist 9ca1215582
Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-11-13 21:46:48 -06:00
David Baker bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Erik Johnston db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events
Drop incoming events from federation for unknown rooms
2018-11-09 17:59:34 +00:00
Erik Johnston 7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison
Fix noop checks when updating device keys
2018-11-08 14:37:20 +00:00
Erik Johnston 94896d7ffe Newsfile 2018-11-08 12:30:25 +00:00
Erik Johnston 06c3d8050f Newsfile 2018-11-08 12:18:41 +00:00
Richard van der Hoff de6223836e changelog 2018-11-08 11:06:28 +00:00
Amber Brown 264cb14402
Port hash_password to Python 3 (#4161)
* port hash_password

* changelog
2018-11-08 04:57:28 +11:00
Amber Brown b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157) 2018-11-08 01:37:43 +11:00
Richard van der Hoff c8ba79327b
Merge pull request #4155 from rubo77/purge-api
add purge_history.sh and purge_remote_media.sh scripts
2018-11-07 14:06:41 +00:00
rubo77 2904d133f3 add purge_history.sh and purge_remote_media.sh scripts to contrib/purge_api/
Signed-off-by: Ruben Barkow <github@r.z11.de>
2018-11-07 14:02:41 +01:00
Amber Brown e62f7f17b3
Remove some boilerplate in tests (#4156) 2018-11-07 03:00:00 +11:00
Travis Ralston 0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142)
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi f1087106cf
handle empty backups according to latest spec proposal (#4123)
fixes #4056
2018-11-05 17:59:29 -05:00
Amber Brown efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Amber Brown 5a63589e80
Add some tests for the HTTP pusher (#4149) 2018-11-06 05:53:24 +11:00
Erik Johnston bc80b3f454 Add helpers for getting prev and auth events (#4139)
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown 0467384d2f
Set the encoding to UTF8 in the default logconfig (#4138) 2018-11-03 02:28:07 +11:00
Erik Johnston 90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events
Clean up event accesses and tests
2018-11-02 14:12:49 +00:00
Erik Johnston 76cd7de108 Newsfile 2018-11-02 13:45:56 +00:00
Amber Brown cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Richard van der Hoff efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking
Fix locked upsert on device_lists_remote_cache
2018-11-02 10:50:53 +00:00
Richard van der Hoff 00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth
Fix logic error that prevented guests from seeing the privacy policy
2018-11-02 10:50:43 +00:00
Erik Johnston b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none
Fix None exception in state res v2
2018-11-02 10:45:57 +00:00
Richard van der Hoff 1cc6671ec4 changelog 2018-11-02 10:36:13 +00:00
Erik Johnston f05d97e283 Newsfile 2018-11-02 10:32:06 +00:00
Travis Ralston 552f090f62 Changelog 2018-11-01 16:51:11 -06:00
Richard van der Hoff 3149d55b7d
Merge pull request #3778 from z3ntu/patch-1
Fix build of Docker image with docker-compose
2018-11-01 17:34:56 +00:00
Travis Ralston c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms
Add m.login.terms to the registration flow
2018-11-01 11:03:38 -06:00
Erik Johnston 1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version
Add STATE_V2_TEST room version
2018-11-01 13:17:57 +00:00
Erik Johnston 62d683161e Newsfile 2018-11-01 11:44:44 +00:00
Amber Brown 073d400b84 Merge branch 'master' into develop 2018-11-01 21:32:12 +11:00
Travis Ralston d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff 1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff 94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff 9b827c40ca
Log some bits about event creation (#4121)
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Richard van der Hoff 60f128a401
Merge pull request #4124 from matrix-org/rav/fix_tox
Attempt to fix tox installs
2018-10-31 15:41:55 +00:00
Amber Brown 916efc8249
Remove fetching keys via the deprecated v1 kex method (#4120) 2018-10-31 23:14:39 +11:00
Amber Brown f79f454485
Remove deprecated v1 key exchange endpoint (#4119) 2018-10-31 22:29:02 +11:00
Richard van der Hoff a2d8bff0dc changelog 2018-10-30 21:21:05 +00:00
Amber Brown e615e95590 changelog 2018-10-31 06:28:11 +11:00
Amber Brown 086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown 3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown 2e223a8c22
Remove the unused /pull federation API (#4118) 2018-10-31 04:24:59 +11:00
Erik Johnston 0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups
Delete unreferenced state groups during purge
2018-10-30 16:58:22 +00:00
Amber Brown 0dce9e1379
Write some tests for the email pusher (#4095) 2018-10-30 23:55:43 +11:00
David Baker 4eacf0f200 news fragment 2018-10-30 10:05:51 +00:00
Erik Johnston 169851b412
Merge pull request #4109 from matrix-org/erikj/repl_devices
A couple of replication fixes for device lists
2018-10-29 18:16:48 +00:00
Erik Johnston 4f0fa7a120 Newsfile 2018-10-29 18:15:42 +00:00
Erik Johnston 39f419868f Newsfile 2018-10-29 17:38:09 +00:00
Amber Brown 4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff 7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff 56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls
Better handling of odd PLs during room upgrades
2018-10-29 12:48:51 +00:00
Richard van der Hoff bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades
Room version upgrade support
2018-10-29 12:47:20 +00:00
Amber Brown c4b3698a80
Make the replication logger quieter (#4108) 2018-10-29 22:59:44 +11:00
Amber Brown 3b0a85fc8e changelog 2018-10-29 21:54:01 +11:00
Richard van der Hoff db24d7f15e Better handling of odd PLs during room upgrades
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff 5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00
Erik Johnston 03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Amber Brown 77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff 6cb2e2448a
Merge pull request #4089 from dekonnection/master
Make Docker image listen on ipv6 as well as ipv4
2018-10-25 19:23:37 +01:00
Richard van der Hoff 68c0ce62d8 changelog 2018-10-25 19:18:25 +01:00
Cédric Laudrel 379376e5e6 Make Docker image listening on ipv6 as well as ipv4
Signed-off-by: Cédric Laudrel <dek@iono.me>
2018-10-25 20:03:47 +02:00
Erik Johnston cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011)
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Richard van der Hoff e5da60d75d Merge remote-tracking branch 'origin/master' into develop 2018-10-25 17:08:09 +01:00
Erik Johnston c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list
Add config option to control alias creation
2018-10-25 17:04:59 +01:00
Neil Johnson 95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init
fix race condiftion in calling initialise_reserved_users
2018-10-25 16:33:40 +01:00
Erik Johnston b94a43d5b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list 2018-10-25 15:25:31 +01:00
Neil Johnson f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Travis Ralston 54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Neil Johnson ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson 07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff 9f72c209ee
Update changelog.d/3975.feature
Co-Authored-By: neilisfragile <neil@matrix.org>
2018-10-24 14:37:36 +01:00
Richard van der Hoff 3ad359e5be Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation 2018-10-24 11:23:49 +01:00
Richard van der Hoff 7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8
Fix a number of flake8 errors
2018-10-24 11:23:35 +01:00
Erik Johnston 3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
Richard van der Hoff 7e07d25ed6 Allow backslashes in event field filters
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.

The intention here is to allow single-backslashes, but disallow
double-backslashes.
2018-10-24 11:11:24 +01:00
Richard van der Hoff ef771cc4c2 Fix a number of flake8 errors
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Richard van der Hoff e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
Erik Johnston cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
Clean up room alias creation
2018-10-24 09:40:27 +01:00
Neil Johnson a67d8ace9b remove errant exception and style 2018-10-23 17:44:39 +01:00
Travis Ralston 43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
Neil Johnson 329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson 6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Richard van der Hoff b3f6dddad2
Give some more things logcontexts (#4077) 2018-10-23 14:29:17 +01:00
Richard van der Hoff 5c445114d3
Correctly account for cpu usage by background threads (#4074)
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
2018-10-23 13:12:32 +01:00
Richard van der Hoff 1fe6bbb555
Merge pull request #3698 from spantaleev/add-matrix-docker-ansible-deploy
Add information about the matrix-docker-ansible-deploy playbook
2018-10-23 10:11:58 +01:00
Richard van der Hoff abd9914683 Changelog 2018-10-22 16:12:11 +01:00
Richard van der Hoff 7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts
Run MAU queries as background processes
2018-10-22 14:46:59 +01:00
Richard van der Hoff 911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil
Make psutil an explicit dependency
2018-10-22 12:33:21 +01:00
Richard van der Hoff a6f421e812 Run MAU queries as background processes
Fixes #3820
2018-10-20 02:14:35 +01:00
Amber Brown e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
steamport b85fe45f46
Add CL 2018-10-19 21:55:38 +00:00
Richard van der Hoff e5b52d0f94 Make psutil an explicit dependency
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.

Fixes #4062.
2018-10-19 21:51:15 +01:00
Richard van der Hoff f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065
Calculate absolute path for email templates
2018-10-19 16:33:54 +01:00
Erik Johnston 056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Richard van der Hoff cc325c7069 Calculate absolute path for email templates 2018-10-19 14:01:59 +01:00
Amber Brown e404ba9aac
Fix manhole on py3 (pt 2) (#4067) 2018-10-19 22:26:00 +11:00
Amber Brown b69216f768
Make the metrics less racy (#4061) 2018-10-19 21:45:45 +11:00
Erik Johnston f9d6c677ea Newsfile 2018-10-19 10:22:45 +01:00
Erik Johnston 0d31109ed5 Newsfile 2018-10-19 10:14:29 +01:00
Amber Brown 1d17fc52ae changelog 2018-10-19 09:27:10 +11:00
Richard van der Hoff c69026a758 Use the right python when starting workers
We should use the same python to start the workers as we do for the main
synapse (ie, the same one used to run synctl.)
2018-10-18 21:06:30 +01:00
Travis Ralston 49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Richard van der Hoff 03287c350e remove redundant changelog file
this change has been released
2018-10-18 15:09:34 +01:00
Richard van der Hoff c632bc8654 Merge branch 'master' into develop 2018-10-18 15:07:03 +01:00
Erik Johnston e77f24d80a
Merge pull request #4049 from matrix-org/erikj/synctl_colour
Only colourise synctl output when attached to tty
2018-10-18 10:51:14 +01:00
Richard van der Hoff 52e3d3813b prep changelog 2018-10-17 17:40:01 +01:00
Richard van der Hoff 0fd2321629 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
Richard van der Hoff 1519572961 Ship the email templates as package_data
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.

In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
Erik Johnston 3a5d8d5891 Newsfile 2018-10-17 16:12:45 +01:00
Erik Johnston 1af16acd4c Newsfile 2018-10-17 13:48:09 +01:00
Will Hunt d6a7797dd1 Fix roomlist since tokens on Python 3 (#4046)
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
Richard van der Hoff 6ec9d8ba0a
Merge pull request #4045 from matrix-org/rav/fix_get_missing_events
Fix incorrect truncation in get_missing_events
2018-10-17 11:40:03 +01:00
Neil Johnson c6584f4b5f clean up config error logic and imports 2018-10-17 11:36:41 +01:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Richard van der Hoff 017eb9d17a changelog 2018-10-16 16:31:47 +01:00
Erik Johnston fc954960e9 Newsfile 2018-10-16 16:28:42 +01:00
Richard van der Hoff b8a5b0097c
Various cleanups in the federation client code (#4031)
- Improve logging: log things in the right order, include destination and txids
  in all log lines, don't log successful responses twice

- Fix the docstring on TransportLayerClient.send_transaction

- Don't use treq.request, which is overcomplicated for our purposes: just use a
  twisted.web.client.Agent.

- simplify the logic for setting up the bodyProducer

- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
Travis Ralston f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Amber Brown 24bc15eab4 update changelog 2018-10-15 22:23:13 +11:00
Amber Brown 4e50fe3edb update changelog 2018-10-15 22:22:49 +11:00
David Baker 03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
E2E backups
2018-10-15 10:19:43 +01:00
Slavi Pantaleev c187638ee9 Add information about the matrix-docker-ansible-deploy playbook
Signed-off-by: Slavi Pantaleev <slavi@devture.com>
2018-10-14 21:50:18 +03:00