Commit Graph

10927 Commits

Author SHA1 Message Date
Neil Johnson 644aac5f73 Tighter filtering for user_daily_visits 2018-05-18 17:10:35 +01:00
Neil Johnson 08462620bf
Merge pull request #3241 from matrix-org/fix_user_visits_insertion
fix psql compatability bug
2018-05-18 15:01:01 +00:00
Neil Johnson ef466b3a13 fix psql compatability bug 2018-05-18 15:51:21 +01:00
Paul Tötterman 861f8a9b21 Add lxml to docker image for web previews
Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>
2018-05-18 16:20:08 +03:00
Neil Johnson 2725223f08 Merge branch 'master' into develop 2018-05-18 14:07:50 +01:00
Neil Johnson ab5e888927 Changes in synapse v0.29.1 (2018-05-17)
==========================================
 Changes:
 
 * Update docker documentation (PR #3222)
 
 Changes in synapse v0.29.0 (2018-05-16)
 ===========================================
 Not changes since v0.29.0-rc1
 
 Changes in synapse v0.29.0-rc1 (2018-05-14)
 ===========================================
 
 Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a
 closed spec bug in the Client Server API. Additionally further prep for Python 3
 migration.
 
 Potentially breaking change:
 
 * Make Client-Server API return 401 for invalid token (PR #3161).
 
   This changes the Client-server spec to return a 401 error code instead of 403
   when the access token is unrecognised. This is the behaviour required by the
   specification, but some clients may be relying on the old, incorrect
   behaviour.
 
   Thanks to @NotAFile for fixing this.
 
 Features:
 
 * Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou!
 
 Changes - General:
 
 * nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77!
 * Part user from rooms on account deactivate (PR #3201)
 * Make 'unexpected logging context' into warnings (PR #3007)
 * Set Server header in SynapseRequest (PR #3208)
 * remove duplicates from groups tables (PR #3129)
 * Improve exception handling for background processes (PR #3138)
 * Add missing consumeErrors to improve exception handling (PR #3139)
 * reraise exceptions more carefully (PR #3142)
 * Remove redundant call to preserve_fn (PR #3143)
 * Trap exceptions thrown within run_in_background (PR #3144)
 
 Changes - Refactors:
 
 * Refactor /context to reuse pagination storage functions (PR #3193)
 * Refactor recent events func to use pagination func (PR #3195)
 * Refactor pagination DB API to return concrete type (PR #3196)
 * Refactor get_recent_events_for_room return type (PR #3198)
 * Refactor sync APIs to reuse pagination API (PR #3199)
 * Remove unused code path from member change DB func (PR #3200)
 * Refactor request handling wrappers (PR #3203)
 * transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov!
 * Refactor event storage to prepare for changes in state calculations (PR #3141)
 * Set Server header in SynapseRequest (PR #3208)
 * Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178)
 * Use run_in_background in preference to preserve_fn (PR #3140)
 
 Changes - Python 3 migration:
 
 * Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile!
 * run config tests on py3 (PR #3159) Thanks to @NotAFile!
 * Open certificate files as bytes (PR #3084) Thanks to @NotAFile!
 * Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile!
 * Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile!
 * Use six.moves.urlparse (PR #3108) Thanks to @NotAFile!
 * Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile!
 * Don't yield in list comprehensions (PR #3150) Thanks to @NotAFile!
 * Move more xrange to six (PR #3151) Thanks to @NotAFile!
 * make imports local (PR #3152) Thanks to @NotAFile!
 * move httplib import to six (PR #3153) Thanks to @NotAFile!
 * Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile!
 * more bytes strings (PR #3155) Thanks to @NotAFile!
 
 Bug Fixes:
 
 * synapse fails to start under Twisted >= 18.4 (PR #3157)
 * Fix a class of logcontext leaks (PR #3170)
 * Fix a couple of logcontext leaks in unit tests (PR #3172)
 * Fix logcontext leak in media repo (PR #3174)
 * Escape label values in prometheus metrics (PR #3175, #3186)
 * Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot!
 * Fix logcontext leaks in rate limiter (PR #3183)
 * notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx!
 * nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77!
 * add guard for None on purge_history api (PR #3160) Thanks to @krombel!
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlr+zv4ACgkQCTxDAAxP
 S/TKxQ/+OMQQJzk7c398zX+wFav0tJO7wSNEtsQ0lkBJq9Yy1Geaz+cXjxoLgPOT
 hthXhLZLVilONx13pnqsJCWrxufiBzWxbP9e7vnvWhP+v1uZca6kp1Pw4fpjGfl6
 DHgCXc3E2/UVDzk0aSIBdufs39qjp/WaUdcT7QuL16uCVwfX0K79nfZtGG+l2KeQ
 fi0SVwK58uZDcSwTrAB39Q3YITZrJzW8Pz1CRkmvG5efWH8KaRtnC/3UZvDb9kBl
 5xGsEkzj7A826PdMXp3ZXPc9H682NYpSD37545XU0ZqddR+RFNofZj0rRH464R+v
 lGVjHYqiiKMeX7y5VF7jZkMsb1j917txP5bcSwS+r2JtymG+RSxBQzUHNiu5gCaC
 MclIY6LbFGT9zJEzTQYBzOYzYA9RvjvlfqbdLV4D5R1Deb5z2VsjRJW7gqpWTj7c
 VpqkF7FP14JqjojMmciHa5b02GC5DPQ/ZSbL8Gp76DKvErbltYEC8GEgFx3eRJLT
 Dru/xcUQcSLUrx4DCk9ViHzpd9RBOGwERmdWilFNW/4HzXb/SK8CjEpWPNSENo+9
 ks3lRSa4J39U0Y/03c3wZMTKQFAQdIcqvAI3bJrlARa5OotvKbN1If5yf2nn1ZmM
 +JaVuV7KLqNMO0Z/O6R5DyuX6j70WAprkddExwGsb7Ha9Uq8Jig=
 =Fpbo
 -----END PGP SIGNATURE-----

Merge tag 'v0.29.1'

Changes in synapse v0.29.1 (2018-05-17)
==========================================
Changes:

* Update docker documentation (PR #3222)

Changes in synapse v0.29.0 (2018-05-16)
===========================================
Not changes since v0.29.0-rc1

Changes in synapse v0.29.0-rc1 (2018-05-14)
===========================================

Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a
closed spec bug in the Client Server API. Additionally further prep for Python 3
migration.

Potentially breaking change:

* Make Client-Server API return 401 for invalid token (PR #3161).

  This changes the Client-server spec to return a 401 error code instead of 403
  when the access token is unrecognised. This is the behaviour required by the
  specification, but some clients may be relying on the old, incorrect
  behaviour.

  Thanks to @NotAFile for fixing this.

Features:

* Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou!

Changes - General:

* nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77!
* Part user from rooms on account deactivate (PR #3201)
* Make 'unexpected logging context' into warnings (PR #3007)
* Set Server header in SynapseRequest (PR #3208)
* remove duplicates from groups tables (PR #3129)
* Improve exception handling for background processes (PR #3138)
* Add missing consumeErrors to improve exception handling (PR #3139)
* reraise exceptions more carefully (PR #3142)
* Remove redundant call to preserve_fn (PR #3143)
* Trap exceptions thrown within run_in_background (PR #3144)

Changes - Refactors:

* Refactor /context to reuse pagination storage functions (PR #3193)
* Refactor recent events func to use pagination func (PR #3195)
* Refactor pagination DB API to return concrete type (PR #3196)
* Refactor get_recent_events_for_room return type (PR #3198)
* Refactor sync APIs to reuse pagination API (PR #3199)
* Remove unused code path from member change DB func (PR #3200)
* Refactor request handling wrappers (PR #3203)
* transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov!
* Refactor event storage to prepare for changes in state calculations (PR #3141)
* Set Server header in SynapseRequest (PR #3208)
* Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178)
* Use run_in_background in preference to preserve_fn (PR #3140)

Changes - Python 3 migration:

* Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile!
* run config tests on py3 (PR #3159) Thanks to @NotAFile!
* Open certificate files as bytes (PR #3084) Thanks to @NotAFile!
* Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile!
* Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile!
* Use six.moves.urlparse (PR #3108) Thanks to @NotAFile!
* Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile!
* Don't yield in list comprehensions (PR #3150) Thanks to @NotAFile!
* Move more xrange to six (PR #3151) Thanks to @NotAFile!
* make imports local (PR #3152) Thanks to @NotAFile!
* move httplib import to six (PR #3153) Thanks to @NotAFile!
* Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile!
* more bytes strings (PR #3155) Thanks to @NotAFile!

Bug Fixes:

* synapse fails to start under Twisted >= 18.4 (PR #3157)
* Fix a class of logcontext leaks (PR #3170)
* Fix a couple of logcontext leaks in unit tests (PR #3172)
* Fix logcontext leak in media repo (PR #3174)
* Escape label values in prometheus metrics (PR #3175, #3186)
* Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot!
* Fix logcontext leaks in rate limiter (PR #3183)
* notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx!
* nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77!
* add guard for None on purge_history api (PR #3160) Thanks to @krombel!
2018-05-18 14:06:34 +01:00
Neil Johnson f3d9dca975 Merge branch 'release-v0.29.0' of https://github.com/matrix-org/synapse into release-v0.29.0 2018-05-18 13:55:19 +01:00
Erik Johnston 0a325e5385
Merge pull request #3226 from matrix-org/erikj/chunk_base
Begin adding implementing room chunks
2018-05-18 13:54:34 +01:00
Erik Johnston b725e128f8 Comments 2018-05-18 13:43:01 +01:00
Richard van der Hoff 6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
Richard van der Hoff 67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff 011e1f4010 Better docstrings 2018-05-18 11:22:12 +01:00
Richard van der Hoff 26305788fe Make sure we reject attempts to invite the notices user 2018-05-18 11:18:39 +01:00
Richard van der Hoff d10707c810 Replace inline docstrings with "Attributes" in class docstring 2018-05-18 11:00:55 +01:00
Erik Johnston fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff 8b1c856d81 Fix error in handling receipts
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):

> TypeError: argument of type 'ObservableDeferred' is not iterable

fixes #3234
2018-05-18 09:15:35 +01:00
Neil Johnson 6958459b50 bump version, change log 2018-05-17 21:35:07 +01:00
Richard van der Hoff 88d3405332 fix missing yield for server_notices_room 2018-05-17 18:33:45 +01:00
Richard van der Hoff d43d480d86 Remove unused `update_external_syncs`
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Neil Johnson a2da6de40e light grammar changes 2018-05-17 18:12:00 +01:00
Michael Kaye 450f500d0c Note that secrets need to be retained. 2018-05-17 18:12:00 +01:00
Michael Kaye 82b0361f02 Document macaroon env var correctly 2018-05-17 18:12:00 +01:00
Michael Kaye 1b1b47aec6 Reference synapse docker image and docker-compose 2018-05-17 18:12:00 +01:00
Richard van der Hoff fed62e21ad Infrastructure for a server notices room
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.

(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler
Move RoomCreationHandler out of synapse.handlers.Handlers
2018-05-17 17:56:42 +01:00
Erik Johnston 0504d809fd More comments 2018-05-17 17:08:36 +01:00
Erik Johnston 12fd6d7688 Document case of unconnected chunks 2018-05-17 16:07:20 +01:00
Erik Johnston a638649254 Make insert_* functions internal and reorder funcs
This makes it clearer what the public interface is vs what subclasses
need to implement.
2018-05-17 15:10:23 +01:00
Erik Johnston d4e4a7344f Increase range of rebalance interval
This both simplifies the code, and ensures that the target node is
roughly in the center of the range rather than at an end.
2018-05-17 15:09:31 +01:00
Erik Johnston c771c124d5 Improve documentation and comments 2018-05-17 15:09:10 +01:00
Erik Johnston 3369354b56 Add note about index in changelog 2018-05-17 14:00:54 +01:00
Erik Johnston 3b505a80dc Merge branch 'develop' of github.com:matrix-org/synapse into erikj/chunk_base 2018-05-17 14:00:41 +01:00
Erik Johnston 943f1029d6 Begin adding implementing room chunks
This commit adds the necessary tables and columns, as well as an
implementation of an online topological sorting algorithm to maintain an
absolute ordering of the room chunks.
2018-05-17 12:05:22 +01:00
Erik Johnston f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream
Use stream rather depth ordering for push actions
2018-05-17 12:01:21 +01:00
Richard van der Hoff ae53c71d90
Merge pull request #1756 from rubo77/patch-4
Add instructions how to setup the postgres user and clarify the final step
2018-05-17 10:52:54 +01:00
rubo77 616da9eb1d postgres.rst: Add instructions how to setup the postgres user and clarify the final step 2018-05-17 11:48:56 +02:00
Richard van der Hoff c46367d0d7 Move RoomCreationHandler out of synapse.handlers.Handlers
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.

Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Neil Johnson 85b8acdeb4 Merge branch 'master' into develop 2018-05-16 16:48:10 +01:00
Neil Johnson 3c099219e0 bump version and changelog for 0.29.0 2018-05-16 15:44:59 +01:00
Erik Johnston 680530cc7f Clarify comment 2018-05-16 11:47:29 +01:00
Erik Johnston 43e6e82c4d Comments 2018-05-16 11:13:31 +01:00
Neil Johnson dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
user visit data
2018-05-16 10:09:24 +00:00
Erik Johnston c945af8799 Move and rename variable 2018-05-16 10:52:06 +01:00
Neil Johnson be11a02c4f remove empty line 2018-05-16 10:45:40 +01:00
Neil Johnson a2204cc9cc remove unused method recurring_user_daily_visit_stats 2018-05-16 09:47:20 +01:00
Neil Johnson 31c2502ca8 style and further contraining query 2018-05-16 09:46:43 +01:00
Richard van der Hoff 8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Neil Johnson c92a8aa578 pep8 2018-05-15 17:31:11 +01:00
Neil Johnson 05ac15ae82 Limit query load of generate_user_daily_visits
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00