Commit Graph

184 Commits

Author SHA1 Message Date
Richard van der Hoff d42455cdc9
Merge pull request #3616 from matrix-org/travis/event_id_send_leave
Update the send_leave path to be an event_id
2018-07-26 23:25:17 +01:00
Richard van der Hoff 65c8dee900
Merge pull request #3614 from matrix-org/rav/stop_populating_event_content
Stop populating events.content
2018-07-26 22:54:08 +01:00
Matthew Hodgson a75231b507
Deduplicate redundant lazy-loaded members (#3331)
* attempt at deduplicating lazy-loaded members

as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again.  For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up).  For now we hardcode X to 30.

* add include_redundant_members filter option & make it work

* remove stale todo

* add tests for _get_some_state_from_cache

* incorporate review
2018-07-26 22:51:30 +01:00
Richard van der Hoff 9e68b1bd2d
Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_id
Remove some redundant joins on event_edges.room_id
2018-07-26 22:31:01 +01:00
Travis Ralston 49254d43a6
Create 3616.misc 2018-07-26 14:45:48 -06:00
Richard van der Hoff 85531a06a2 changelog 2018-07-26 14:55:29 +01:00
Richard van der Hoff cf78eaebad changelog 2018-07-26 13:22:40 +01:00
Richard van der Hoff a15ed52267 changelog 2018-07-26 12:53:18 +01:00
Richard van der Hoff 03751a6420 Fix some looping_call calls which were broken in #3604
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.

Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Matthew Hodgson 1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members
Implement the lazy_load_members room state filter parameter
2018-07-26 00:03:01 +01:00
Travis Ralston 6185650f9c
Create 3609.misc 2018-07-25 15:46:56 -06:00
Matthew Hodgson 2565804030 Merge branch 'develop' into matthew/filter_members 2018-07-25 17:27:49 +01:00
Erik Johnston 3849f7f69f
Merge pull request #3603 from matrix-org/erikj/handle_outliers
Correctly handle outliers during persist events
2018-07-25 13:24:04 +01:00
Richard van der Hoff cee1ae1b72
Merge pull request #3606 from matrix-org/rav/logcontext_fixes_once_more
Fix another logcontext leak in _persist_events
2018-07-25 11:56:00 +01:00
Richard van der Hoff 32b30e15f2
Merge pull request #3607 from matrix-org/rav/fix_persist_events_integrity_error
Fix occasional 'tuple index out of range' error
2018-07-25 11:55:45 +01:00
Richard van der Hoff 4081bd1560
Merge pull request #3605 from matrix-org/rav/fix_update_remote_profile_cache
Fix updating of cached remote profiles
2018-07-25 11:54:46 +01:00
Richard van der Hoff 1be94440d3 Fix occasional 'tuple index out of range' error
This fixes a bug in _delete_existing_rows_txn which was introduced in #3435
(though it's been on matrix-org-hotfixes for *years*). This code is only called
when there is some sort of conflict the first time we try to persist an event,
so it only happens rarely. Still, the exceptions are annoying.
2018-07-25 11:05:58 +01:00
Richard van der Hoff 07defd5fe6 Fix another logcontext leak in _persist_events
We need to run the errback in the sentinel context to avoid losing our own
context.

Also: add logging to runInteraction to help identify where "Starting db
connection from sentinel context" warnings are coming from
2018-07-25 10:53:23 +01:00
Richard van der Hoff 9c237a7ab5 changelog 2018-07-25 10:46:01 +01:00
Richard van der Hoff 3f11d84534 Changelog 2018-07-25 09:43:25 +01:00
Erik Johnston 0b300d323a Newsfile 2018-07-25 09:39:45 +01:00
Michael Telatynski 38eaa5280d
add changelog entry for PR#3391
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-07-24 17:25:42 +01:00
Erik Johnston 1674a85238 Move newsfile 2018-07-24 17:20:27 +01:00
Erik Johnston f14c866e37 Newsfile 2018-07-24 16:51:19 +01:00
Erik Johnston 60a1d147a7
Merge pull request #3595 from matrix-org/erikj/use_deltas
Use deltas to calculate current state deltas
2018-07-24 15:41:18 +01:00
Erik Johnston f33c596533 Newsfile 2018-07-24 14:28:04 +01:00
Richard van der Hoff 81946db9cf
Merge pull request #3587 from matrix-org/rav/better_exception_logging
Improve logging for exceptions when handling PDUs
2018-07-24 14:12:12 +01:00
Richard van der Hoff a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups
Fixes and optimisations for resolve_state_groups
2018-07-24 14:11:45 +01:00
Richard van der Hoff 93b0722c50
Merge pull request #3583 from matrix-org/rav/remove_who_forgot_in_room
Remove redundant checks on room forgottenness
2018-07-24 14:11:11 +01:00
Matthew Hodgson 454f59b7ad Merge branch 'develop' into matthew/filter_members 2018-07-24 14:03:37 +01:00
Matthew Hodgson d19fba3655 Merge branch 'develop' into matthew/filter_members 2018-07-24 12:39:54 +01:00
Richard van der Hoff 30bfed5aa5 Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room 2018-07-24 11:46:09 +01:00
Erik Johnston 97acd385a3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/speed_up_calculate_state_delta 2018-07-24 11:32:13 +01:00
Erik Johnston 2581eb3e1d Newsfile 2018-07-24 11:14:28 +01:00
Richard van der Hoff a678145010
Merge branch 'develop' into rav/logcontext_fixes 2018-07-24 10:43:30 +01:00
Erik Johnston d436ad332c
Merge pull request #3555 from matrix-org/erikj/client_apis_move
Make client_reader support some more read only APIs
2018-07-24 10:42:28 +01:00
Erik Johnston 536bc63a4e
Merge branch 'develop' into erikj/client_apis_move 2018-07-24 09:57:05 +01:00
Richard van der Hoff c6e66821a9 Changelog 2018-07-24 00:38:39 +01:00
Richard van der Hoff 30957a941a changelog 2018-07-24 00:24:38 +01:00
Matthew Hodgson 004a83b43a changelog 2018-07-23 22:32:35 +01:00
Richard van der Hoff d8709df739 changelog 2018-07-23 22:16:22 +01:00
Richard van der Hoff cc99256e90 newsfile 2018-07-23 19:10:50 +01:00
Erik Johnston 2d5bba151b Newsfile 2018-07-23 17:29:32 +01:00
Richard van der Hoff 4f5cc8e4e7 Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room 2018-07-23 17:15:12 +01:00
Richard van der Hoff dae6dc1e77 Remove redundant checks on room forgottenness
Fixes #3550
2018-07-23 17:13:34 +01:00
Erik Johnston 9f41ad491d Newsfile 2018-07-23 16:31:46 +01:00
Erik Johnston a4d24781bf Newsfile 2018-07-23 15:28:51 +01:00
Erik Johnston 0b0b24cb82 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move 2018-07-23 13:21:15 +01:00
Erik Johnston f3182bb1d0 Newsfile 2018-07-23 13:19:24 +01:00
Erik Johnston 959f4b9074 Newsfile 2018-07-23 12:22:59 +01:00
Matthew Hodgson 9b34f3ea3a
Merge branch 'develop' into matthew/sync_deleted_devices 2018-07-23 10:03:28 +01:00
Amber Brown 3132b89f12
Make the rest of the .iterwhatever go away (#3562) 2018-07-21 15:47:18 +10:00
Richard van der Hoff 5c30cb709a Changelog 2018-07-20 14:01:36 +01:00
Richard van der Hoff 4f67623674
Merge pull request #3571 from matrix-org/rav/limiter_fixes
A set of improvements to the Limiter
2018-07-20 13:53:03 +01:00
Amber Brown e1a237eaab
Admin API for creating new users (#3415) 2018-07-20 22:41:13 +10:00
Richard van der Hoff 683f4058c1 changelogs 2018-07-20 13:16:39 +01:00
Amber Brown 7cf76c9a09 changelog 2018-07-19 21:26:30 +10:00
Amber Brown 9a8acdc720 Merge branch 'master' into develop 2018-07-19 21:16:44 +10:00
Amber Brown d69decd5c7 0.33.0 final changelog 2018-07-19 21:12:15 +10:00
Amber Brown 13d501c773 update changelogs 2018-07-19 21:11:24 +10:00
Amber Brown ce0545eca1 Revert "0.33.0rc1 changelog"
This reverts commit 21d3b87943.
2018-07-19 21:03:15 +10:00
Amber Brown 95ccb6e2ec
Don't spew errors because we can't save metrics (#3563) 2018-07-19 20:58:18 +10:00
Richard van der Hoff 1ffb7bec20 Merge remote-tracking branch 'origin/release-v0.33.0' into develop 2018-07-19 11:12:33 +01:00
Richard van der Hoff c754e006f4
Merge pull request #3556 from matrix-org/rav/background_processes
Run things as background processes
2018-07-19 11:04:18 +01:00
Amber Brown a97c845271
Move v1-only APIs into their own module & isolate deprecated ones (#3460) 2018-07-19 20:03:33 +10:00
Richard van der Hoff 18a2b2c0b4 changelog 2018-07-19 10:54:39 +01:00
Benedikt Heine f1dd89fe86 [Docker] Use sorted multiline package lists
This matches docker best practices.

Signed-off-by: Benedikt Heine <bebe@bebehei.de>
2018-07-19 11:38:58 +02:00
Erik Johnston 6f62a6ef21
Merge pull request #3554 from matrix-org/erikj/response_metrics_code
Add response code to response timer metrics
2018-07-19 10:25:18 +01:00
Richard van der Hoff eed24893fa changelog 2018-07-18 22:12:19 +01:00
Richard van der Hoff 08436c556a changelog 2018-07-18 20:55:05 +01:00
Erik Johnston 3a993a660d Newsfile 2018-07-18 15:39:49 +01:00
Richard van der Hoff 65d6a0e477
Merge pull request #3553 from matrix-org/rav/background_process_tracking
Resource tracking for background processes
2018-07-18 14:27:50 +01:00
Erik Johnston 00845c49d2 Newsfile 2018-07-18 14:03:16 +01:00
Richard van der Hoff dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks
Remove event re-signing hacks
2018-07-18 12:46:27 +01:00
David Baker c91a44572e
Merge pull request #3514 from matrix-org/dbkr/turn_dont_add_defaults
Comment dummy TURN parameters in default config
2018-07-18 11:56:18 +01:00
Richard van der Hoff 92aecd557b changelog 2018-07-18 11:29:48 +01:00
Amber Brown 21d3b87943 0.33.0rc1 changelog 2018-07-18 12:53:32 +10:00
Richard van der Hoff 4a11df5b64 changelog 2018-07-17 14:24:29 +01:00
Richard van der Hoff 9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
Amber Brown bc006b3c9d
Refactor REST API tests to use explicit reactors (#3351) 2018-07-17 20:43:18 +10:00
Erik Johnston 9952d18e4d Newsfile 2018-07-17 10:31:51 +01:00
Amber Brown 3fe0938b76
Merge pull request #3530 from matrix-org/erikj/stream_cache
Don't return unknown entities in get_entities_changed
2018-07-17 13:44:46 +10:00
Richard van der Hoff 2731bf7ac3 Changelog 2018-07-16 14:12:25 +01:00
Krombel 4a27000548 check isort by travis 2018-07-16 13:57:33 +02:00
Amber Brown 8a4f05fefb
Fix develop because I broke it :( (#3535) 2018-07-14 09:51:00 +10:00
Amber Brown 8532953c04
Merge pull request #3534 from krombel/use_parse_and_asserts_from_servlet
Use parse and asserts from http.servlet
2018-07-14 09:09:19 +10:00
Amber Brown 33b60c01b5
Make auth & transactions more testable (#3499) 2018-07-14 07:34:49 +10:00
Krombel 516f960ad8 add changelog 2018-07-13 22:19:19 +02:00
Richard van der Hoff 08546be40f better changelog 2018-07-13 16:47:13 +01:00
Richard van der Hoff b03a0e14db changelog 2018-07-13 16:28:28 +01:00
Erik Johnston 5f263b607e Newsfile 2018-07-13 15:47:25 +01:00
Richard van der Hoff 6dff49b8a9
Merge pull request #3521 from matrix-org/rav/optimise_stream_change_cache
Reduce set building in get_entities_changed
2018-07-12 12:08:49 +01:00
Matthew Hodgson 37c4fba0ac changelog 2018-07-12 11:45:33 +01:00
Richard van der Hoff 38c5fa7ee4 changelog 2018-07-12 11:45:28 +01:00
Richard van der Hoff c969754a91 changelog 2018-07-12 10:00:57 +01:00
Erik Johnston 0456e05977
Merge pull request #3505 from matrix-org/erikj/receipts_cahce
Use stream cache in get_linearized_receipts_for_room
2018-07-12 09:46:29 +01:00
David Baker 1b5425527c I failed to correctly guess the PR number 2018-07-11 16:02:29 +01:00
David Baker 36f4fd3e1e Comment dummy TURN parameters in default config
This default config is parsed and used a base before the actual
config is overlaid, so with these values not commented out, the
code to detect when no turn params were set and refuse to generate
credentials was never firing because the dummy default was always set.
2018-07-11 15:49:29 +01:00
Amber Brown 129ffd7b88
Merge pull request #3498 from OlegGirko/fix_attrs_syntax
* Use more portable syntax using attrs package.

Newer syntax

    attr.ib(factory=dict)

is just a syntactic sugar for

    attr.ib(default=attr.Factory(dict))

It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.

We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.

Signed-off-by: Oleg Girko <ol@infoserver.lv>
2018-07-11 04:22:46 +10:00
Amber Brown 85354bb18e changelog entry 2018-07-11 03:27:03 +10:00