Commit Graph

2239 Commits

Author SHA1 Message Date
Erik Johnston 069ae2df12 Fix initial sync 2017-09-20 10:52:12 +01:00
Richard van der Hoff e76d1135dd Invalidate signing key cache when we gat an update
This might make the cache slightly more efficient.
2017-09-20 01:32:42 +01:00
Erik Johnston 4a9b1cf253 Add user profiles to summary from group server 2017-08-25 16:23:58 +01:00
Erik Johnston 27ebc5c8f2 Add remote profile cache 2017-08-25 11:25:47 +01:00
Erik Johnston 97c544f91f Add _simple_update 2017-08-25 11:11:37 +01:00
Erik Johnston 2800983f3e Merge pull request #2410 from matrix-org/erikj/groups_publicise
Add ability to publicise group membership
2017-08-21 16:51:56 +01:00
Erik Johnston 8b50fe5330 Use BOOLEAN rather than TEXT type 2017-08-21 16:37:29 +01:00
Erik Johnston 175a01f56c Groups: Fix mising json.load in initial sync 2017-08-21 14:45:56 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston b880ff190a Allow update group publicity 2017-08-08 14:19:41 +01:00
Erik Johnston 05e21285aa Store whether the user wants to publicise their membership of a group 2017-08-08 13:01:46 +01:00
Erik Johnston 966a70f1fa Update comment 2017-07-24 17:49:39 +01:00
Erik Johnston 629cdfb124 Use join rather than joined, etc. 2017-07-24 14:54:05 +01:00
Erik Johnston ed666d3969 Fix all the typos 2017-07-24 14:05:09 +01:00
Erik Johnston b76ef6ccb8 Include users membership in group in summary API 2017-07-24 13:55:39 +01:00
Erik Johnston 851aeae7c7 Check users/rooms are in group before adding to summary 2017-07-24 13:40:56 +01:00
Erik Johnston 96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston 0401604222 Merge pull request #2377 from matrix-org/erikj/group_profile_update
Add update group profile API
2017-07-20 17:53:39 +01:00
Erik Johnston 2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 8209b5f033 Fix a storage desc 2017-07-20 16:22:22 +01:00
Erik Johnston 57826d645b Fix typo 2017-07-20 13:15:22 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 3e703eb04e Comment 2017-07-18 10:17:25 +01:00
Erik Johnston 508460f240 Remove sync stuff 2017-07-18 09:55:46 +01:00
Erik Johnston 4540730111 Remove unused tables 2017-07-18 09:38:15 +01:00
Erik Johnston e96ee95a7e Remove sync stuff 2017-07-18 09:38:08 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston 85fda57208 Add DEFAULT_ROLE_ID 2017-07-14 14:03:54 +01:00
Erik Johnston 4b203bdba5 Correctly increment orders 2017-07-14 14:02:31 +01:00
Erik Johnston 3b0470dba5 Remove unused functions 2017-07-13 13:54:01 +01:00
Erik Johnston 8575e3160f Comments 2017-07-13 13:52:41 +01:00
Erik Johnston 26451a09eb Comments 2017-07-12 14:47:18 +01:00
Erik Johnston a62406aaa5 Add group summary APIs 2017-07-12 11:36:15 +01:00
Erik Johnston 6d586dc05c Comment 2017-07-12 09:58:37 +01:00
Erik Johnston 410b4e14a1 Move comment 2017-07-11 15:44:18 +01:00
Erik Johnston e52c391cd4 Rename column to attestation_json 2017-07-11 14:25:46 +01:00
Erik Johnston 0aac30d53b Comments 2017-07-11 14:23:50 +01:00
Erik Johnston 83936293eb Comments 2017-07-11 11:42:25 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston f502b0dea1 Perf: Don't filter events for push
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
Erik Johnston ed9a7f5436 Merge pull request #2309 from matrix-org/erikj/user_ip_repl
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
Erik Johnston b5e8d529e6 Define CACHE_SIZE_FACTOR once 2017-07-04 09:56:44 +01:00
Erik Johnston 6ff14ddd2e Make into list 2017-06-29 15:47:37 +01:00
Erik Johnston 5946aa0877 Prefill forward extrems and event to state groups 2017-06-29 15:38:48 +01:00
Erik Johnston 8c23221666 Fix up 2017-06-27 15:53:45 +01:00
Erik Johnston a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston ed3d0170d9 Batch upsert user ips 2017-06-27 13:37:04 +01:00
Erik Johnston 8abdd7b553 Fix up indices for users_who_share_rooms 2017-06-26 14:01:30 +01:00
Erik Johnston 7fe8ed1787 Store URL cache preview downloads seperately
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston e0004aa28a Add desc 2017-06-22 10:03:48 +01:00
Erik Johnston b668112320 Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar
Don't work out users who share room with appservice users
2017-06-21 14:50:24 +01:00
Erik Johnston dae9a00a28 Initialise exclusive_user_regex 2017-06-21 14:19:33 +01:00
Krombel 4202fba82a Merge branch 'develop' into avoid_duplicate_filters 2017-06-21 14:48:21 +02:00
Krombel 812c030e87 replaced json.dumps with encode_canonical_json 2017-06-21 14:48:12 +02:00
Erik Johnston 385dcb7c60 Handle thumbnail urls 2017-06-19 17:48:28 +01:00
Erik Johnston b8b936a6ea Add API to quarantine media 2017-06-19 17:39:21 +01:00
Erik Johnston d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston dfeca6cf40 Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston d8f47d2efa Merge pull request #2280 from matrix-org/erikj/share_room_user_dir
Include users who you share a room with in user directory
2017-06-16 11:06:00 +01:00
Erik Johnston 052c5d19d5 Merge pull request #2281 from matrix-org/erikj/phone_home_stats
Fix phone home stats
2017-06-15 12:46:23 +01:00
Erik Johnston a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston 4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston 72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston ebcd55d641 Add DB schema for tracking users who share rooms 2017-06-15 09:45:48 +01:00
Erik Johnston 4b461a6931 Add some more stats 2017-06-15 09:39:39 +01:00
Erik Johnston 617304b2cf Fix phone home stats 2017-06-14 19:47:15 +01:00
Erik Johnston d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston 505e7e8b9d Fix up sql 2017-06-13 11:19:18 +01:00
Erik Johnston 6fd7e6db3d Fix user dir to not assume existence of user 2017-06-13 11:11:26 +01:00
Erik Johnston fdca6e36ee Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined
Add cache for is_host_joined
2017-06-13 10:58:53 +01:00
Erik Johnston b58e24cc3c Tweak the ranking of PG user dir search 2017-06-13 10:16:31 +01:00
Erik Johnston d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston 8060974344 Fix replication 2017-06-09 16:40:52 +01:00
Erik Johnston b0d975e216 Comments 2017-06-09 16:25:42 +01:00
Erik Johnston e54d7d536e Cache state deltas 2017-06-09 16:24:00 +01:00
Erik Johnston 935e588799 Tweak SQL 2017-06-09 13:01:23 +01:00
Erik Johnston 0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston ea11ee09f3 Ensure we don't use unpersisted state group as prev group 2017-06-08 11:59:57 +01:00
Erik Johnston c62c480dc6 Merge pull request #2259 from matrix-org/erikj/fix_state_woes
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston 197bd126f0 Fix bug where state_group tables got corrupted
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.

We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston 64ed74c01e When pruning, delete from device_lists_outbound_last_success 2017-06-07 11:20:47 +01:00
Erik Johnston 1a81a1898e Keep pruning background task 2017-06-07 11:16:56 +01:00
Erik Johnston 6ba21bf2b8 Comments 2017-06-07 11:08:36 +01:00
Erik Johnston 09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston 6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston 65f0513a33 Split up device_lists_outbound_pokes table for faster updates. 2017-06-07 11:02:38 +01:00
Erik Johnston 6f83c4537c Increase size of IP cache 2017-06-07 10:18:44 +01:00
Erik Johnston 1a01af079e Handle profile updates in user directory 2017-06-01 15:39:51 +01:00
Erik Johnston 4d039aa2ca Fix sqlite 2017-06-01 14:58:48 +01:00
Erik Johnston 21e255a8f1 Split the table in two 2017-06-01 14:50:46 +01:00
Erik Johnston d5477c7afd Tweak search query 2017-06-01 13:28:01 +01:00
Erik Johnston 02a6108235 Tweak search query 2017-06-01 13:16:40 +01:00
Erik Johnston 59dbb47065 Remove spurious inlineCallbacks 2017-06-01 11:41:29 +01:00
Erik Johnston 0fe6f3c521 Bug fixes and logging
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston 036362ede6 Order by if they have profile info 2017-06-01 09:41:08 +01:00
Erik Johnston a757dd4863 Use prefix matching 2017-06-01 09:40:37 +01:00
Erik Johnston f5cc22bdc6 Comment on why arbitrary comments 2017-05-31 17:30:26 +01:00
Erik Johnston 5dd1b2c525 Use unique indices 2017-05-31 17:29:12 +01:00
Erik Johnston f1378aef91 Convert to int 2017-05-31 17:03:08 +01:00
Erik Johnston f091061711 Fix tests 2017-05-31 16:34:40 +01:00
Erik Johnston 4abcff0177 Fix typo 2017-05-31 16:22:36 +01:00
Erik Johnston 63c58c2a3f Limit number of things we fetch out of the db 2017-05-31 16:17:58 +01:00
Erik Johnston 304880d185 Add stream change cache 2017-05-31 15:46:36 +01:00
Erik Johnston 5d79d728f5 Split out directory and search tables 2017-05-31 15:23:49 +01:00
Erik Johnston dc51af3d03 Pull max id from correct table 2017-05-31 15:13:49 +01:00
Erik Johnston 350622a107 Handle the server leaving a public room 2017-05-31 15:11:36 +01:00
Erik Johnston 63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston 293ef29655 Weight differently 2017-05-31 14:29:32 +01:00
Erik Johnston 3b5f22ca40 Add search 2017-05-31 14:00:01 +01:00
Erik Johnston b5db4ed5f6 Update room column when room becomes unpublic 2017-05-31 13:40:28 +01:00
Erik Johnston eeb2f9e546 Add user_directory to database 2017-05-31 11:51:01 +01:00
Erik Johnston dd48f7204c Add comment 2017-05-30 15:01:22 +01:00
Erik Johnston 04095f7581 Add clobbered event_id 2017-05-30 14:53:01 +01:00
Erik Johnston a584a81b3e Add current_state_delta_stream table 2017-05-30 14:44:09 +01:00
Erik Johnston 619e8ecd0c Handle None state group correctly 2017-05-26 10:46:03 +01:00
Erik Johnston dfbda5e025 Faster cache for get_joined_hosts 2017-05-25 17:24:44 +01:00
Erik Johnston 11f139a647 Merge pull request #2247 from matrix-org/erikj/auth_event
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
Erik Johnston 6e614e9e10 Add background task to clear out old event_auth 2017-05-24 15:23:34 +01:00
Erik Johnston c049472b8a Only store event_auth for state events 2017-05-24 15:23:31 +01:00
Erik Johnston 8cf9f0a3e7 Remove redundant invalidation 2017-05-23 09:46:59 +01:00
Erik Johnston e6618ece2d Missed an invalidation 2017-05-23 09:36:52 +01:00
Erik Johnston 74bf4ee7bf Stream count_e2e_one_time_keys cache invalidation 2017-05-22 16:19:22 +01:00
Erik Johnston 58ebb96cce Fix invalidation of get_users_with_read_receipts_in_room 2017-05-19 14:38:50 +01:00
Erik Johnston 107ac7ac96 Increase size of push rule caches 2017-05-18 17:17:53 +01:00
Erik Johnston 760625acba Make get_if_app_services_interested_in_user faster 2017-05-18 16:34:44 +01:00
Erik Johnston d9e3a4b5db Merge pull request #2230 from matrix-org/erikj/speed_up_get_state
Make get_state_groups_from_groups faster.
2017-05-17 17:23:04 +01:00
Erik Johnston ace23463c5 Merge pull request #2216 from slipeer/app_services_interested_in_user
Fix users claimed non-exclusively by an app service don't get notific…
2017-05-17 16:28:50 +01:00
Erik Johnston bbfe4e996c Make get_state_groups_from_groups faster.
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.

By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
Erik Johnston 85e8092cca Comment 2017-05-17 10:03:09 +01:00
Erik Johnston ad53fc3cf4 Short circuit when we have delta ids 2017-05-17 09:57:34 +01:00
Erik Johnston 13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston b8492b6c2f Merge pull request #2224 from matrix-org/erikj/prefill_state
Prefill state caches
2017-05-16 15:50:11 +01:00
Erik Johnston 331570ea6f Remove spurious merge artifacts 2017-05-16 15:33:07 +01:00
Krombel 55af207321 Merge branch 'develop' into avoid_duplicate_filters 2017-05-16 15:29:59 +02:00
Erik Johnston 608b5a6317 Take a copy before prefilling, as it may be a frozendict 2017-05-16 12:55:29 +01:00
Krombel 64953c8ed2 avoid access-error if no filter_id matches 2017-05-15 18:36:37 +02:00
Erik Johnston e4435b014e Update comment 2017-05-15 15:11:30 +01:00
Erik Johnston 871605f4e2 Comments 2017-05-15 15:11:30 +01:00
Erik Johnston e0d2f6d5b0 Add more granular event send metrics 2017-05-15 15:11:30 +01:00
Erik Johnston bfbc907cec Prefill state caches 2017-05-15 15:11:13 +01:00
Krombel eb7cbf27bc insert whitespace to fix travis build 2017-05-12 12:09:42 +02:00
Krombel 6b95e35e96 add check to only add a new filter if the same filter does not exist previously
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-05-11 16:05:30 +02:00
Richard van der Hoff ff3d810ea8 Add a comment to old delta 2017-05-11 12:48:50 +01:00
Richard van der Hoff 34194aaff7 Don't create event_search index on sqlite
... because the table is virtual
2017-05-11 12:46:55 +01:00
Richard van der Hoff 114f290947 Add more logging for purging
Log the number of events we will be deleting at info.
2017-05-11 12:08:47 +01:00