Commit Graph

4672 Commits

Author SHA1 Message Date
Erik Johnston cffe46408f Don't rely on options when inserting event into cache 2016-06-03 18:25:21 +01:00
Mark Haines ac9716f154 Fix spelling 2016-06-03 18:10:00 +01:00
Mark Haines 8f79084bd4 Add get_presence_list_accepted to the broken caches in synchrotron 2016-06-03 18:03:40 +01:00
Erik Johnston 10ea3f46ba Change the way we cache events 2016-06-03 17:57:50 +01:00
Erik Johnston f6be734be9 Merge pull request #835 from matrix-org/erikj/get_event_txn
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
Erik Johnston 05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
David Baker 72c4d482e9 3rd time lucky: we'd already calculated it above 2016-06-03 16:39:50 +01:00
David Baker fbf608decb Oops, we're using the dict form 2016-06-03 16:38:39 +01:00
David Baker 06d40c8b98 Add substitutions to email notif From 2016-06-03 16:31:23 +01:00
Erik Johnston 5f88549f4a Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop 2016-06-03 16:27:24 +01:00
Mark Haines ca457f594e Merge pull request #831 from matrix-org/markjh/synchrotronII
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
Erik Johnston 21961c93c7 Bump changelog and version 2016-06-03 15:33:14 +01:00
Mark Haines 48340e4f13 Clear the list of ongoing syncs on shutdown 2016-06-03 15:02:27 +01:00
Erik Johnston fcbc282f56 Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0 2016-06-03 15:02:04 +01:00
Erik Johnston 51773bcbaf Merge pull request #832 from matrix-org/erikj/presence_coount
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
Mark Haines da491e75b2 Appease flake8 2016-06-03 14:56:36 +01:00
Mark Haines 0b3c80a234 Use ClientIpStore to record client ips 2016-06-03 14:55:01 +01:00
Mark Haines dd6f62ed99 Merge branch 'develop' into markjh/synchrotronII 2016-06-03 14:51:33 +01:00
Mark Haines eef541a291 Move insert_client_ip to a separate class 2016-06-03 14:42:35 +01:00
Mark Haines 80aade3805 Send updates to the syncing users every ten seconds or immediately if they've just come online 2016-06-03 14:24:19 +01:00
Erik Johnston ab116bdb0c Fix typo 2016-06-03 14:03:42 +01:00
Erik Johnston 4ce84a1acd Change metric style 2016-06-03 13:49:16 +01:00
Erik Johnston a7ff5a1770 Presence metrics. Change def of small delta 2016-06-03 13:40:55 +01:00
Matthew Hodgson 8d740132f4 Merge branch 'develop' into matthew/brand-from-header 2016-06-03 12:14:18 +01:00
Erik Johnston 3b096c5f5c Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop 2016-06-03 12:00:33 +01:00
Erik Johnston 43b7f371f5 Merge pull request #830 from matrix-org/erikj/metrics_perf
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
Mark Haines abb151f3c9 Add a separate process that can handle /sync requests 2016-06-03 11:57:26 +01:00
Erik Johnston 4982b28868 Merge pull request #829 from matrix-org/erikj/poke_notifier
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
Erik Johnston d06f2a229e Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston 58a224a651 Pull out update_results_dict 2016-06-03 11:47:07 +01:00
Mark Haines 20eccd84d4 Merge pull request #827 from matrix-org/markjh/more_slaved_methods
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
Erik Johnston 722472b48c Merge pull request #825 from matrix-org/erikj/cache_push_rules
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Erik Johnston 73c7112433 Change CacheMetrics to be quicker
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines b09f348530 Merge pull request #824 from matrix-org/markjh/slaved_presence_store
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston 4c04222fa5 Poke notifier on next reactor tick 2016-06-03 11:24:16 +01:00
Erik Johnston ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Mark Haines 81cf449daa Add methods to events, account data and receipt slaves
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Erik Johnston e043ede4a2 Small optimisation to CacheListDescriptor 2016-06-03 11:19:22 +01:00
Mark Haines b821c839dc Merge pull request #823 from matrix-org/markjh/more_slaved_stores
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
Erik Johnston 597013caa5 Make cachedList go a bit faster 2016-06-03 11:13:29 +01:00
Erik Johnston 6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
Mark Haines 3ae915b27e Add a slaved store for presence 2016-06-03 11:05:53 +01:00
Erik Johnston 59f2d73522 Remove unnecessary sets 2016-06-03 11:05:45 +01:00
Erik Johnston 9c26b390a2 Only get local users 2016-06-03 11:04:31 +01:00
Mark Haines f88d747f79 Add a comment explaining why the filter cache doesn't need exipiring 2016-06-03 11:03:10 +01:00
Erik Johnston 065e739d6e Merge pull request #811 from matrix-org/erikj/state_users_in_room
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston 696d7c5937 Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines 0eae075723 Add slaved stores for filters, tokens, and push rules 2016-06-03 10:58:03 +01:00
Matthew Hodgson 79d1f072f4 brand the email from header 2016-06-02 21:34:40 +01:00
David Baker 6bb9aacf9d Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
David Baker 745ddb4dd0 peppate 2016-06-02 17:38:41 +01:00
David Baker 7a5a5f2df2 Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
David Baker 1f31cc37f8 Working unsubscribe links going straight to the HS
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
Matthew Hodgson 2675c1e40e add some branding debugging 2016-06-02 17:21:12 +01:00
David Baker c71177f285 Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe 2016-06-02 17:20:56 +01:00
David Baker 07a5559916 Fix error in email notification string formatting 2016-06-02 17:17:16 +01:00
Mark Haines 56d15a0530 Store the typing users as user_id strings. (#819)
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
David Baker 812b5de0fe Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe 2016-06-02 15:33:28 +01:00
Mark Haines 80f34d7b57 Fix setting the _clock in SQLBaseStore 2016-06-02 15:23:56 +01:00
Mark Haines 661a540dd1 Deduplicate presence entries in sync (#818) 2016-06-02 15:20:28 +01:00
Mark Haines 70599ce925 Allow external processes to mark a user as syncing. (#812)
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
2016-06-02 15:20:15 +01:00
David Baker fb2193cc63 Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
Split out the auth handler
2016-06-02 14:31:35 +01:00
Erik Johnston 356f13c069 Disable INCLUDE_ALL_UNREAD_NOTIFS 2016-06-02 14:07:38 +01:00
Erik Johnston 02ac463dbf Merge pull request #800 from matrix-org/erikj/sync_refactor
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
Matthew Hodgson c5af1b6b00 Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00
David Baker 3a3fb2f6f9 Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe 2016-06-02 13:35:25 +01:00
David Baker 4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
Matthew Hodgson f84b89f0c6 if an email pusher specifies a brand param, use it 2016-06-02 13:29:48 +01:00
David Baker a15ad60849 Email unsubscribing that may in theory, work
Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.
2016-06-02 11:44:15 +01:00
David Baker e793866398 Use user_id in email greeting if display name is null 2016-06-02 09:41:13 +01:00
Matthew Hodgson aaa70e26a2 special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208 2016-06-01 22:13:47 +01:00
Erik Johnston a04a2d043c Merge pull request #807 from matrix-org/erikj/push_rules_cache
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston 0f06b496d1 Merge pull request #806 from matrix-org/erikj/hash_cache
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
David Baker e0deeff23e Fix room list spidering 2016-06-01 17:58:58 +01:00
David Baker 991af8b0d6 WIP on unsubscribing email notifs without logging in 2016-06-01 17:40:52 +01:00
David Baker 00c487a8db Merge pull request #808 from matrix-org/dbkr/room_list_spider
Add secondary_directory_servers option to fetch room list from other servers
2016-06-01 15:32:52 +01:00
Erik Johnston c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
David Baker d60eed0710 Limit number of notifications in an email notification 2016-06-01 11:45:43 +01:00
Erik Johnston 43db0d9f6a Add get_users_with_read_receipts_in_room cache 2016-06-01 10:54:32 +01:00
David Baker 8e539f13c0 Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider 2016-06-01 09:54:36 +01:00
David Baker 6ecb2ca4ec pep8 2016-06-01 09:48:55 +01:00
David Baker 2a449fec4d Add cache to remote room lists
Poll for updates from remote servers, waiting for the poll if there's no cache entry.
2016-05-31 18:27:23 +01:00
David Baker 6ca4d3ae9a Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution 2016-05-31 17:24:50 +01:00
Erik Johnston dea9f20f8c Force boolean 2016-05-31 17:24:30 +01:00
David Baker 963e3ed282 Apparently I am not permitted to have two blank lines here 2016-05-31 17:22:53 +01:00
David Baker d240796ded Basic, un-cached support for secondary_directory_servers 2016-05-31 17:20:07 +01:00
Mark Haines c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston c9ca285d33 Merge pull request #805 from matrix-org/erikj/push_rules_cache
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston 1d4ee854e2 Fix typo 2016-05-31 15:45:53 +01:00
Erik Johnston cca0093fa9 Change fix 2016-05-31 15:44:08 +01:00
Erik Johnston 4efa389299 Fix GET /push_rules 2016-05-31 15:37:53 +01:00
Erik Johnston aefd2d1cbc Cache get_event_reference_hashes 2016-05-31 15:32:32 +01:00
Erik Johnston 10de8c2631 Merge pull request #804 from matrix-org/erikj/push_rules_cache
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Mark Haines c626fc576a Move the AS handler out of the Handlers object.
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Erik Johnston e5b0bbcd33 Add caches to bulk_get_push_rules* 2016-05-31 13:46:58 +01:00
David Baker 70ecb415f5 Fix c+p fail 2016-05-31 12:00:54 +01:00
David Baker e1625d62a8 Add federation room list servlet 2016-05-31 11:55:57 +01:00
David Baker 887c6e6f05 Split out the room list handler
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
Erik Johnston 85b992f621 Fix to allow start with postgres 2016-05-27 10:44:44 +01:00
Erik Johnston cc84f7cb8e Send down correct error response if user not found 2016-05-27 10:35:15 +01:00
David Baker 209ba4d024 Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month
Only delete push actions after 30 days
2016-05-24 16:22:13 +01:00
David Baker b007ee4606 Check for presence of 'avatar_url' key 2016-05-24 15:12:05 +01:00
David Baker cbf8d146ac Merge pull request #799 from matrix-org/matthew/quieter-email-notifs
Tune email notifs to make them quieter:
2016-05-24 14:30:51 +01:00
Erik Johnston faad233ea6 Change short circuit path 2016-05-24 14:27:19 +01:00
Erik Johnston 6900303997 Don't send down all ephemeral events 2016-05-24 11:44:55 +01:00
Erik Johnston 1c5ed2a19b Only work out newly_joined_users for incremental sync 2016-05-24 11:21:34 +01:00
Erik Johnston b08ad0389e Only include non-offline presence in initial sync 2016-05-24 11:15:05 +01:00
Erik Johnston be2c677386 Spell builder correctly 2016-05-24 10:53:03 +01:00
Erik Johnston 79bea8ab9a Inline function. Make load_filtered_recents private 2016-05-24 10:22:24 +01:00
Erik Johnston 84f94e4cbb Add comments 2016-05-24 10:14:53 +01:00
Erik Johnston 137e6a4557 Shuffle things room 2016-05-24 09:50:55 +01:00
Matthew Hodgson 680f1d9387 catch thinko in presentable names 2016-05-23 22:55:11 +01:00
Matthew Hodgson cb8a321bdd fix NPE in room ordering 2016-05-23 22:54:56 +01:00
Matthew Hodgson 88ea5ab2c3 consistency is the better part of valour 2016-05-23 19:33:45 +01:00
Matthew Hodgson 989bdc9e56 Tune email notifs to make them quieter:
* After initial 10 minute window, only alert every 24h for room notifs
 * Reset room state after 6h of idleness
 * Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up'
 * Fix the email subjects to say what triggered the notification
 * Order the rooms in reverse activity order in the email, so the 'reason' room should always come first
2016-05-23 19:24:11 +01:00
Negi Fazeli 6fe04ffef2 Fix set profile error with Requester.
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
Erik Johnston c0c79ef444 Add back concurrently_execute 2016-05-23 18:21:27 +01:00
Erik Johnston b5605dfecc Refactor SyncHandler 2016-05-23 18:08:18 +01:00
David Baker 31b5395ab6 Remove debug logging 2016-05-23 16:32:01 +01:00
David Baker c2da3406fc Oops, missing comma 2016-05-20 18:03:31 +01:00
David Baker ccffb0965d Remove stale line 2016-05-20 17:59:10 +01:00
David Baker 18d68bfee4 Handle empty events table 2016-05-20 17:58:09 +01:00
David Baker d4503e25ed Make deleting push actions more efficient
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker 149fa411e2 Only delete push actions after 30 days 2016-05-20 15:25:12 +01:00
Kegan Dougal 332d7e9b97 Allow clients to specify a server_name to avoid 'No known servers'
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
Matthew Hodgson 6fb51eaf7b Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo
increment badge count per missed convo, not per msg
2016-05-18 13:56:38 +01:00
Matthew Hodgson e837df6adb increment badge count per missed convo, not per msg 2016-05-18 11:53:25 +01:00
Erik Johnston 42368ea8db Add desc to get_presence_for_users 2016-05-18 11:38:10 +01:00
Mark Haines 0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines 6a30a0bfd3 Move the functions for parsing app service config 2016-05-17 11:28:58 +01:00
Mark Haines 523d5bcd0b Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler 2016-05-17 10:43:58 +01:00
Matthew Hodgson 43e1e0489c Merge pull request #786 from matrix-org/matthew/email_notifs_tuning
tune email notifs, fix CSS a bit, and add debugging details
2016-05-17 10:43:24 +01:00
Mark Haines 526bf8126f Remove unused get_joined_rooms_for_user 2016-05-17 10:20:51 +01:00
Mark Haines 425e6b4983 Merge branch 'develop' into markjh/member_cleanup 2016-05-17 10:13:16 +01:00
Mark Haines b153f5b150 Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines 3fd8a07fca Merge pull request #788 from matrix-org/markjh/domian
Spell "domain" correctly
2016-05-17 10:09:34 +01:00
Mark Haines f68eea808a Move SyncHandler out of the Handlers object 2016-05-16 20:19:26 +01:00
Mark Haines 53e171f345 Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler 2016-05-16 20:08:32 +01:00
Mark Haines 80cb9becd8 Remove get_joined_rooms_for_user from RoomMemberHandler 2016-05-16 20:06:55 +01:00
Mark Haines 816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines 821306120a Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room 2016-05-16 19:48:07 +01:00
Mark Haines 1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines e168abbcff Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler 2016-05-16 19:08:40 +01:00
Matthew Hodgson cbd2adc95e tune email notifs, fix CSS a bit, and add debugging details 2016-05-16 18:58:38 +01:00
Mark Haines 3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
David Baker 647781ca56 Fix emailpusher import
Try importing at the root level rather than conditionally importing, as per comment
2016-05-16 18:41:32 +01:00
Erik Johnston c39f305067 os.environ requires a string 2016-05-16 17:21:30 +01:00
Erik Johnston 678c8a7f1e Merge pull request #785 from matrix-org/erikj/cache_factor_synctly
Make synctl read a cache factor from config file
2016-05-16 17:07:35 +01:00
Erik Johnston c5c5a7403b Make synctl read a cache factor from config file 2016-05-16 17:01:57 +01:00
Mark Haines eb79110beb Clean up the blacklist/whitelist handling.
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00