Commit Graph

2444 Commits

Author SHA1 Message Date
Adrian Tschira 94f4d7f49e move httplib import to six 2018-04-28 13:43:34 +02:00
Adrian Tschira cdb4647a80 Don't yield in list comprehensions
I've tried to grep for more of this with no success.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:36:30 +02:00
Richard van der Hoff 453adf00b6 pep8; remove spurious import 2018-04-27 14:32:08 +01:00
Richard van der Hoff fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff 2a13af23bc Use run_in_background in preference to preserve_fn
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff 41d4b07a53
Merge pull request #3142 from matrix-org/rav/reraise
reraise exceptions more carefully
2018-04-27 12:16:19 +01:00
Richard van der Hoff 9c3da24561
Merge pull request #3138 from matrix-org/rav/catch_unhandled_exceptions
Improve exception handling for background processes
2018-04-27 11:47:49 +01:00
Richard van der Hoff 6493b22b42 reraise exceptions more carefully
We need to be careful (under python 2, at least) that when we reraise an
exception after doing some error handling, we actually reraise the original
exception rather than anything that might have been raised (and handled) during
the error handling.
2018-04-27 11:40:06 +01:00
Richard van der Hoff 605defb9e4 Add missing consumeErrors
In general we want defer.gatherResults to consumeErrors, rather than having
exceptions hanging around and getting logged as CRITICAL unhandled errors.
2018-04-27 11:16:28 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Richard van der Hoff 77ebef9d43
Merge pull request #3118 from matrix-org/rav/reject_prev_events
Reject events which have lots of prev_events
2018-04-23 17:51:38 +01:00
Richard van der Hoff 9b9c38373c Remove spurious param 2018-04-23 12:00:06 +01:00
Richard van der Hoff 8dc4a6144b
Merge pull request #3107 from NotAFile/py3-bool-nonzero
add __bool__ alias to __nonzero__ methods
2018-04-20 15:43:39 +01:00
Richard van der Hoff 3de7d9fe99 accept stupid events over backfill 2018-04-20 11:41:03 +01:00
Richard van der Hoff 11a67b7c9d
Merge pull request #3093 from matrix-org/rav/response_cache_wrap
Refactor ResponseCache usage
2018-04-20 11:31:17 +01:00
Richard van der Hoff bc381d5798
Merge pull request #3117 from matrix-org/rav/refactor_have_events
Refactor store.have_events
2018-04-20 10:26:12 +01:00
Richard van der Hoff b1dfbc3c40 Refactor store.have_events
It turns out that most of the time we were calling have_events, we were only
using half of the result. Replace have_events with have_seen_events and
get_rejection_reasons, so that we can see what's going on a bit more clearly.
2018-04-20 10:25:56 +01:00
Richard van der Hoff 1f4b498b73 Add some comments 2018-04-18 00:15:36 +01:00
Richard van der Hoff e585228860 Check events on backfill too 2018-04-18 00:06:42 +01:00
Richard van der Hoff 9b7794262f Reject events which have too many auth_events or prev_events
... this should protect us from being dossed by people making silly events
(deliberately or otherwise)
2018-04-18 00:06:42 +01:00
Richard van der Hoff 639480e14a Avoid creating events with huge numbers of prev_events
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
2018-04-16 18:41:37 +01:00
Adrian Tschira f63ff73c7f add __bool__ alias to __nonzero__ methods
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
Richard van der Hoff b78395b7fe Refactor ResponseCache usage
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.

This will be largely non-functional, but does include the following functional
changes:

* federation_server.on_context_state_request: drops use of _server_linearizer
  which looked redundant and could cause incorrect cache misses by yielding
  between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
  on production.
2018-04-12 13:02:15 +01:00
Richard van der Hoff d5c74b9f6c
Merge pull request #3092 from matrix-org/rav/response_cache_metrics
Add metrics for ResponseCache
2018-04-12 12:59:36 +01:00
Erik Johnston 19ceb4851f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/processed_event_lag 2018-04-12 11:36:07 +01:00
Erik Johnston f67e906e18 Set all metrics at the same time 2018-04-12 11:18:19 +01:00
Erik Johnston 971059a733
Merge pull request #3088 from matrix-org/erikj/as_parallel
Send events to ASes concurrently
2018-04-12 10:42:36 +01:00
Erik Johnston 4dae4a97ed Track last processed event received_ts 2018-04-11 14:27:09 +01:00
Erik Johnston 92e34615c5 Track where event stream processing have gotten up to 2018-04-11 12:13:40 +01:00
Erik Johnston 121591568b Send events to ASes concurrently 2018-04-11 09:56:00 +01:00
Richard van der Hoff b3384232a0 Add metrics for ResponseCache 2018-04-10 23:14:47 +01:00
Erik Johnston eaa2ebf20b
Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends
Limit concurrent event sends for a room
2018-04-10 16:43:58 +01:00
Erik Johnston f8e8ec013b Note why we're limiting concurrent event sends 2018-04-10 14:00:46 +01:00
Richard van der Hoff a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms
move handling of auto_join_rooms to RegisterHandler
2018-04-10 01:11:00 +01:00
Richard van der Hoff d2c6f4d626
Merge pull request #3080 from matrix-org/rav/fix_500_on_rejoin
Return a 404 rather than a 500 on rejoining empty rooms
2018-04-09 17:32:36 +01:00
Richard van der Hoff f3ef60662f Return a 404 rather than a 500 on rejoining empty rooms
Filter ourselves out of the server list before checking for an empty remote
host list, to fix 500 error

Fixes #2141
2018-04-09 12:56:22 +01:00
Erik Johnston e5082494eb Limit concurrent event sends for a room 2018-04-09 12:07:39 +01:00
David Baker b370fe61c0 Implement group join API 2018-04-06 15:43:27 +01:00
Luke Barnard e089100c62
Merge pull request #3045 from matrix-org/dbkr/group_joinable
Add joinability for groups
2018-04-05 15:57:49 +01:00
Richard van der Hoff 725a72ec5a
Merge pull request #3000 from NotAFile/change-except-style
Replace old style error catching with 'as' keyword
2018-04-04 10:45:22 +01:00
Luke Barnard eb8d8d6f57 Use join_policy API instead of joinable
The API is now under
 /groups/$group_id/setting/m.join_policy

and expects a JSON blob of the shape

```json
{
  "m.join_policy": {
    "type": "invite"
  }
}
```

where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
Richard van der Hoff 8da39ad98f
Merge pull request #3049 from matrix-org/rav/use_staticjson
Use static JSONEncoders
2018-04-03 15:18:32 +01:00
Richard van der Hoff 05630758f2 Use static JSONEncoders
using json.dumps with custom options requires us to create a new JSONEncoder on
each call. It's more efficient to create one upfront and reuse it.
2018-03-29 23:13:33 +01:00
Richard van der Hoff fcfe7f6ad3 Use simplejson throughout
Let's use simplejson rather than json, for consistency.
2018-03-29 22:45:52 +01:00
Richard van der Hoff 9a0db062af
Merge pull request #3034 from matrix-org/rav/fix_key_claim_errors
Fix error when claiming e2e keys from offline servers
2018-03-28 14:50:38 +01:00
David Baker 79452edeee Add joinability for groups
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
Krombel 6152e253d8 Merge branch 'develop' of into allow_auto_join_rooms 2018-03-28 14:45:28 +02:00
Richard van der Hoff a134c572a6 Stringify exceptions for keys/{query,claim}
Make sure we stringify any exceptions we return from keys/query and keys/claim,
to avoid a 'not JSON serializable' error later

Fixes #3010
2018-03-27 17:15:06 +01:00
Richard van der Hoff c2a5cf2fe3 factor out exception handling for keys/claim and keys/query
this stuff is badly c&p'ed
2018-03-27 17:11:23 +01:00
Erik Johnston 3e0c0660b3 Also do check inside linearizer 2018-03-27 13:01:34 +01:00
Erik Johnston 44cd6e1358 PEP8 2018-03-26 12:06:48 +01:00
Erik Johnston a052aa42e7 Linearize calls to _generate_user_id 2018-03-26 12:02:20 +01:00
Erik Johnston fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Matthew Hodgson b2f2282947 make lazy_load_members configurable in filters 2018-03-19 01:15:13 +00:00
Matthew Hodgson 478af0f720 reshuffle todo & comments 2018-03-19 01:00:12 +00:00
Matthew Hodgson 366f730bf6 only get member state IDs for incremental syncs if we're filtering 2018-03-18 21:40:43 +00:00
Matthew Hodgson fc5397fdf5 remove debug 2018-03-16 01:44:55 +00:00
Matthew Hodgson 4f0493c850 fix tsm search again 2018-03-16 01:43:37 +00:00
Matthew Hodgson f7dcc404f2 add state_ids for timeline entries 2018-03-16 01:37:53 +00:00
Matthew Hodgson 5b3b3aada8 simplify timeline_start_members 2018-03-16 01:17:34 +00:00
Matthew Hodgson 3bc5bd2d22 make incr syncs work 2018-03-16 00:52:04 +00:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
NotAFile 2cc9f76bc3 replace old style error catching with 'as' keyword
This is both easier to read and compatible with python3 (not that that
matters)

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-03-15 16:11:17 +01:00
Krombel 91ea0202e6 move handling of auto_join_rooms to RegisterHandler
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work

This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.

This is related to issue #2725

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-14 16:45:37 +01:00
Matthew Hodgson 056a6df546
Merge branch 'develop' into matthew/filter_members 2018-03-14 15:38:05 +00:00
Erik Johnston 57db62e554
Merge pull request #2992 from matrix-org/erikj/implement_member_workre
Implement RoomMemberWorkerHandler
2018-03-14 14:29:33 +00:00
Erik Johnston 62ad701326 s/join/joined/ in notify_user_membership_change 2018-03-14 14:17:43 +00:00
Erik Johnston 3f0f06cb31 Split RoomMemberWorkerHandler to separate file 2018-03-14 11:41:45 +00:00
Erik Johnston 3e839e0548
Merge pull request #2989 from matrix-org/erikj/profile_cache_master
Only update remote profile cache on master
2018-03-14 09:42:27 +00:00
Matthew Hodgson 4d0cfef6ee add copyright to nudge CI 2018-03-14 00:02:20 +00:00
Matthew Hodgson c9d72e4571 oops 2018-03-13 23:46:45 +00:00
Matthew Hodgson ccca02846d make it work 2018-03-13 22:31:41 +00:00
Matthew Hodgson 12350e3f9a merge proper fix to bug 2969 2018-03-13 22:11:58 +00:00
Matthew Hodgson 14a9d2f73d ensure we always include the members for a given timeline block 2018-03-13 22:03:42 +00:00
Erik Johnston b27320b550 Implement RoomMemberWorkerHandler 2018-03-13 18:26:00 +00:00
Erik Johnston 350331d466 _remote_join and co take a requester 2018-03-13 17:50:39 +00:00
Erik Johnston 1a69c6d590
Merge pull request #2987 from matrix-org/erikj/split_room_member_handler
Split RoomMemberHandler into base and master class
2018-03-13 17:40:00 +00:00
Erik Johnston df8ff682a7 Only update remote profile cache on master 2018-03-13 17:38:21 +00:00
Erik Johnston 6dbebef141 Add missing param to docstrings 2018-03-13 17:15:32 +00:00
Erik Johnston 16adb11cc0 Correct import order 2018-03-13 16:57:07 +00:00
Erik Johnston 82f16faa78 Move user_*_room distributor stuff to master class
I added yields when calling user_left_room, but they shouldn't matter on
the master process as they always return None anyway.
2018-03-13 16:38:15 +00:00
Erik Johnston b78717b87b Split RoomMemberHandler into base and master class
The intention here is to split the class into the bits that can be done
on workers and the bits that have to be done on the master.

In future there will also be a class that can be run on the worker,
which will delegate work to the master when necessary.
2018-03-13 16:37:41 +00:00
Erik Johnston 95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer
Remove ReplicationLayer and user Client/Server directly
2018-03-13 15:45:08 +00:00
Erik Johnston 5d8476d8ff
Merge pull request #2981 from matrix-org/erikj/factor_remote_leave
Factor out _remote_reject_invite in RoomMember
2018-03-13 15:44:56 +00:00
Erik Johnston 56e709857c
Merge pull request #2979 from matrix-org/erikj/no_handlers
Don't build handlers on workers unnecessarily
2018-03-13 13:46:38 +00:00
Erik Johnston cb9f8e527c s/replication_client/federation_client/ 2018-03-13 13:26:52 +00:00
Erik Johnston bf8e97bd3c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/factor_remote_leave 2018-03-13 13:17:08 +00:00
Erik Johnston ea3442c15c Add docstring 2018-03-13 13:16:21 +00:00
Erik Johnston 16469a4f15
Merge pull request #2980 from matrix-org/erikj/rm_priv
Make RoomMemberHandler functions private that can be
2018-03-13 13:11:11 +00:00
Erik Johnston c82111a55f
Merge pull request #2982 from matrix-org/erikj/fix_extra_users
extra_users is actually a list of UserIDs
2018-03-13 13:11:04 +00:00
Erik Johnston da87791975
Merge pull request #2983 from matrix-org/erikj/rename_register_3pid
Refactor get_or_register_3pid_guest
2018-03-13 13:10:52 +00:00
Erik Johnston 8b3573a8b2 Refactor get_or_register_3pid_guest 2018-03-13 12:08:58 +00:00
Erik Johnston 0f942f68c1 Factor out _remote_reject_invite in RoomMember 2018-03-13 11:22:45 +00:00
Erik Johnston d0fcc48f9d extra_users is actually a list of UserIDs 2018-03-13 11:20:06 +00:00
Erik Johnston 31becf4ac3 Make functions private that can be 2018-03-13 11:15:16 +00:00
Erik Johnston 265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Erik Johnston c3f79c9da5 Split out edu/query registration to a separate class 2018-03-13 10:24:27 +00:00
Richard van der Hoff e48c7aac4d Add transactional API to history purge
Make the purge request return quickly, and allow scripts to poll for updates.
2018-03-12 16:22:55 +00:00
Richard van der Hoff 1708412f56 Return an error when doing two purges on a room
Queuing up purges doesn't sound like a good thing.
2018-03-12 16:22:54 +00:00
Matthew Hodgson 8713365265 typos 2018-03-11 20:10:25 +00:00
Matthew Hodgson 9b334b3f97 WIP experiment in lazyloading room members 2018-03-11 20:01:41 +00:00
Erik Johnston a56d54dcb7 Fix up log message 2018-03-07 11:55:31 +00:00
Erik Johnston 02a1296ad6 Fix typo 2018-03-07 11:55:31 +00:00
Erik Johnston 8cb44da4aa Fix race in sync when joining room
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.

This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
Richard van der Hoff 8ffaacbee3
Merge pull request #2949 from krombel/use_bcrypt_checkpw
use bcrypt.checkpw
2018-03-06 11:56:06 +00:00
Richard van der Hoff b2932107bb
Merge pull request #2946 from matrix-org/rav/timestamp_to_purge
Implement purge_history by timestamp
2018-03-06 11:20:23 +00:00
Krombel ed9b5eced4 use bcrypt.checkpw
in bcrypt 3.1.0 checkpw got introduced (already 2 years ago)
This makes use of that with enhancements which might get introduced
by that

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-05 18:02:59 +01:00
Erik Johnston d4ffe61d4f Remove ability for AS users to call /events and /sync
This functionality has been deprecated for a while as well as being
broken for a while. Instead of fixing it lets just remove it entirely.

See: https://github.com/matrix-org/matrix-doc/issues/1144
2018-03-05 15:44:46 +00:00
Richard van der Hoff f8bfcd7e0d Provide a means to pass a timestamp to purge_history 2018-03-05 14:37:23 +00:00
Erik Johnston d960d23830 Add missing yield during 3pid signature checks 2018-03-02 11:03:18 +00:00
Erik Johnston 9e08a93a7b
Merge pull request #2927 from matrix-org/erikj/read_marker_caches
Improve caching for read_marker API
2018-03-01 17:12:34 +00:00
Erik Johnston a83c514d1f Improve caching for read_marker API
We add a new storage function to get a paritcular type of room account
data. This allows us to prefill the cache when updating that acount
data.
2018-03-01 17:08:17 +00:00
Erik Johnston 529c026ac1 Move back to hs.is_mine 2018-03-01 16:49:12 +00:00
Erik Johnston 784f036306 Move RoomMemberHandler out of Handlers 2018-03-01 14:36:50 +00:00
Erik Johnston f381d63813 Check event auth on the worker 2018-03-01 10:18:37 +00:00
Erik Johnston 6b8604239f Correctly send ratelimit and extra_users params 2018-03-01 10:08:39 +00:00
Erik Johnston f756f961ea Fixup comments 2018-03-01 10:05:27 +00:00
Erik Johnston 28e973ac11 Calculate push actions on worker 2018-02-28 18:02:30 +00:00
Erik Johnston d874d4f2d7 Delete from push_actions_staging in federation too 2018-02-20 11:37:52 +00:00
Erik Johnston b96278d6fe Ensure that we delete staging push actions on errors 2018-02-15 15:47:06 +00:00
Erik Johnston c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker
Create a worker for event creation
2018-02-13 18:07:10 +00:00
Richard van der Hoff 10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff 74fcbf741b delete_local_events for purge_history
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff 671540dccf rename delete_old_state -> purge_history
(beacause it deletes more than state)
2018-02-09 13:07:41 +00:00
Erik Johnston 50fe92cd26 Move presence handling into handle_new_client_event
As we want to have it run on the main synapse instance
2018-02-07 10:34:09 +00:00
Erik Johnston 24dd73028a Add replication http endpoint for event sending 2018-02-07 10:32:32 +00:00
Erik Johnston e3624fad5f Remove pointless ratelimit check
The intention was for the check to be called as early as possible in the
request, but actually was called just before the main ratelimit check,
so was fairly pointless.
2018-02-07 10:30:25 +00:00
Erik Johnston 617199d73d
Merge pull request #2847 from matrix-org/erikj/separate_event_creation
Split event creation into a separate handler
2018-02-06 17:01:17 +00:00
Erik Johnston 3e1e69ccaf Update copyright 2018-02-06 16:40:38 +00:00
Erik Johnston 770b2252ca s/_create_new_client_event/create_new_client_event/ 2018-02-06 16:40:30 +00:00
Erik Johnston 3d33eef6fc
Store state groups separately from events (#2784)
* Split state group persist into seperate storage func

* Add per database engine code for state group id gen

* Move store_state_group to StateReadStore

This allows other workers to use it, and so resolve state.

* Hook up store_state_group

* Fix tests

* Rename _store_mult_state_groups_txn

* Rename StateGroupReadStore

* Remove redundant _have_persisted_state_group_txn

* Update comments

* Comment compute_event_context

* Set start val for state_group_id_seq

... otherwise we try to recreate old state groups

* Update comments

* Don't store state for outliers

* Update comment

* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
Erik Johnston 3fa362502c Update places where we create events 2018-02-05 16:01:48 +00:00
Erik Johnston 5ff3d23564 Split event creation into a separate handler 2018-02-05 16:01:48 +00:00
Richard van der Hoff 0cbda53819 Rename resolve_state_groups -> resolve_state_groups_for_events
(to make way for a method that actually just does the state group resolution)
2018-02-01 12:40:00 +00:00
Neil Johnson 73560237d6 add white space line 2018-01-26 00:15:10 +00:00
Neil Johnson 86c4f49a31 rather than try reconstruct the results object, better to guard against the xrange step argument being 0 2018-01-26 00:12:02 +00:00
Neil Johnson f632083576 fix return type, should be a dict 2018-01-25 23:52:17 +00:00
Neil Johnson 6c6e197b0a fix PEP8 violation 2018-01-25 23:47:46 +00:00
Neil Johnson d02e43b15f remove white space 2018-01-25 23:29:46 +00:00
Neil Johnson 349c739966 synapse 500s on a call to publicRooms in the case where the number of public rooms is zero, the specific cause is due to xrange trying to use a step value of zero, but if the total room number really is zero then it makes sense to just bail and save the extra processing 2018-01-25 23:28:44 +00:00
Matthew Hodgson ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Matthew Hodgson 447f4f0d5f rewrite based on PR feedback:
* [ ] split config options into allowed_local_3pids and registrations_require_3pid
 * [ ] simplify and comment logic for picking registration flows
 * [ ] fix docstring and move check_3pid_allowed into a new util module
 * [ ] use check_3pid_allowed everywhere

@erikjohnston PTAL
2018-01-19 15:33:55 +00:00
Matthew Hodgson 9d332e0f79 fix up v1, and improve errors 2018-01-19 00:53:58 +00:00
Richard van der Hoff a5213df1f7 Sanity checking for user ids
Check the user_id passed to a couple of APIs for validity, to avoid
"IndexError: list index out of range" exception which looks scary and results
in a 500 rather than a more useful error.

Fixes #1432, among other things
2018-01-17 14:28:54 +00:00
Richard van der Hoff a027c2af8d Metrics for events processed in appservice and fed sender
More metrics I wished I'd had
2018-01-15 18:23:24 +00:00
Erik Johnston 825a07a974
Merge pull request #2773 from matrix-org/erikj/hash_bg
Do bcrypt hashing in a background thread
2018-01-10 18:11:41 +00:00
Erik Johnston f8e1ab5fee Do bcrypt hashing in a background thread 2018-01-10 18:01:28 +00:00
Michael Telatynski 5f07f5694c fix order of operations derp and also use `.get` to default to {}
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:11:35 +00:00
Michael Telatynski 8c9d5b4873 Fix publicised groups API (singular) over federation
which was missing its fed client API, since there is no other API
it might as well reuse the bulk one and unwrap it

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:04:51 +00:00
Erik Johnston d8a6c734fa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/createroom_content 2017-12-07 14:24:01 +00:00
Erik Johnston ef045dcd71 Copy dict in update_membership too 2017-12-07 14:17:15 +00:00
Matthew Hodgson 33cb7ef0b7
Merge pull request #2723 from matrix-org/matthew/search-all-local-users
Add all local users to the user_directory and optionally search them
2017-12-05 11:09:47 +00:00
Richard van der Hoff da1010c83a support custom login types for validating users
Wire the custom login type support from password providers into the UI-auth
user-validation flows.
2017-12-05 09:43:30 +00:00
Richard van der Hoff d7ea8c4800 Factor out a validate_user_via_ui_auth method
Collect together all the places that validate a logged-in user via UI auth.
2017-12-05 09:42:30 +00:00
Richard van der Hoff d5f9fb06b0 Refactor UI auth implementation
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
Matthew Hodgson c22e73293a speed up the rate of initial spam for users 2017-12-04 18:05:28 +00:00
Matthew Hodgson 74e0cc74ce fix pep8 and tests 2017-12-04 15:11:38 +00:00
Matthew Hodgson 1bd40ca73e switch to a simpler 'search_all_users' button as per review feedback 2017-12-04 14:58:39 +00:00
Matthew Hodgson f397153dfc Merge branch 'develop' into matthew/search-all-local-users 2017-11-30 01:51:38 +00:00
Matthew Hodgson a4bb133b68 fix thinkos galore 2017-11-30 01:17:15 +00:00
Matthew Hodgson cd3697e8b7 kick the user_directory index when new users register 2017-11-29 18:33:34 +00:00
Matthew Hodgson 3241c7aac3 untested WIP but might actually work 2017-11-29 18:27:05 +00:00
Matthew Hodgson 47d99a20d5 Add user_directory_include_pattern config param to expand search results to additional users
Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive.
It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users...
...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too.

Committing it here as a WIP
2017-11-29 16:46:45 +00:00
Richard van der Hoff ad7e570d07 Delete devices in various logout situations
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:

 * /logout
 * /logout_all
 * change password
 * deactivate account (by the user or by an admin)
 * invalidate access token from a dynamic module

Fixes #2672.
2017-11-29 16:44:35 +00:00
Richard van der Hoff ae31f8ce45 Move set_password into its own handler
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff 7ca5c68233 Move deactivate_account into its own handler
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff 2c6d63922a Remove pushers when deleting access tokens
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
Erik Johnston dfbc45302e
PEP8 2017-11-28 15:23:26 +00:00
Erik Johnston c4c1d170af
Fix wrong avatars when inviting multiple users when creating room
We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user
2017-11-28 15:19:15 +00:00
Luke Barnard cdf6fc15b0
Merge pull request #2686 from matrix-org/luke/as-flair
Add automagical AS Publicised Group(s)
2017-11-17 10:13:46 +00:00
Jurek eded7084d2 Fix auth handler #2678 2017-11-17 10:07:27 +00:00
Matthew Hodgson 34c3d0a386 typo 2017-11-17 01:54:02 +00:00
Luke Barnard 5b48eec4a1 Make sure we check AS groups for lookup on bulk 2017-11-16 17:55:15 +00:00
Jurek 624a8bbd67 Fix auth handler #2678 2017-11-16 17:19:02 +00:00
Luke Barnard 97bd18af4e Add automagical AS Publicised Group(s)
via registration file "users" namespace:

```YAML
...
namespaces:
  users:
    - exclusive: true
      regex: '.*luke.*'
      group_id: '+all_the_lukes:hsdomain'
...
```

This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.
2017-11-16 16:44:55 +00:00
Richard van der Hoff 1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
Richard van der Hoff 7e6fa29cb5 Remove preserve_context_over_{fn, deferred}
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
Richard van der Hoff 44a1bfd6a6 Reshuffle room list request code
I'm not entirely sure if this will actually help anything, but it simplifies
the code and might give further clues about why room list search requests are
blowing out the get_current_state_ids caches.
2017-11-14 10:29:58 +00:00
Richard van der Hoff 1fc66c7460 Add a load of logging to the room_list handler
So we can see what it gets up to.
2017-11-14 10:23:47 +00:00
Richard van der Hoff 9b803ccc98 Revert "Allow upper-case characters in mxids"
This reverts commit b70b646903.
2017-11-09 21:57:24 +00:00
Richard van der Hoff b70b646903 Allow upper-case characters in mxids
Because we're never going to be able to fix this :'(
2017-11-09 19:36:13 +00:00
Erik Johnston e8814410ef Have an explicit API to update room config 2017-11-08 16:13:27 +00:00
Erik Johnston 94ff2cda73
Revert "Modify group room association API to allow modification of is_public" 2017-11-08 15:43:34 +00:00
Erik Johnston d305987b40
Merge pull request #2631 from xyzz/fix_appservice_event_backlog
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-08 11:54:10 +00:00
Richard van der Hoff 02a9a93bde
Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res
Fix bug in state group storage
2017-11-08 09:22:13 +00:00
Richard van der Hoff e148438e97 s/items/iteritems/ 2017-11-08 09:21:41 +00:00
Ilya Zhuravlev d46386d57e Remove useless assignment in notify_interested_services 2017-11-07 22:23:22 +03:00
Matthew Hodgson 228ccf1fe3
Merge pull request #2643 from matrix-org/matthew/user_dir_typos
Fix various embarrassing typos around user_directory and add some doc.
2017-11-07 17:31:11 +00:00
Richard van der Hoff 780dbb378f Update deltas when doing auth resolution
Fixes a bug where the persisted state groups were different to those actually
being used after auth resolution.
2017-11-07 16:43:00 +00:00
Richard van der Hoff 1ca4288135 factor out _update_context_for_auth_events
This is duplicated, so let's factor it out before fixing it
2017-11-07 16:43:00 +00:00
Erik Johnston 5ef5e14ecc
Merge pull request #2636 from farialima/me-master
Fix for #2635: correctly update rooms avatar/display name when modified by admin
2017-11-07 13:49:27 +00:00
Matthew Hodgson d1622e080f s/intial/initial/ 2017-11-04 19:35:14 +00:00
Francois Granade f103b91ffa removed unused import flagged by flake8a 2017-11-03 18:45:49 +01:00
Francois Granade fa4f337b49 Fix for issue 2635: correctly update rooms avatar/display name when modified by admin 2017-11-03 18:25:04 +01:00
Ilya Zhuravlev 45fbe4ff67 Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services 2017-11-02 22:49:43 +03:00
Richard van der Hoff 1189be43a2 Factor _AccountHandler proxy out to ModuleApi
We're going to need to use this from places that aren't password auth, so let's
move it to a proper class.
2017-11-02 14:36:11 +00:00
David Baker b19d9e2174
Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout
Notify auth providers on logout
2017-11-02 10:55:17 +00:00
David Baker 1f080a6c97
Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers
Allow password_auth_providers to return a callback
2017-11-02 10:49:03 +00:00
David Baker 04897c9dc1
Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers
Let auth providers get to the database
2017-11-02 10:41:25 +00:00
Richard van der Hoff 979eed4362 Fix user-interactive password auth
this got broken in the previous commit
2017-11-01 17:03:20 +00:00
Richard van der Hoff bc8a5c0330 Notify auth providers on logout
Provide a hook by which auth providers can be notified of logouts.
2017-11-01 16:51:51 +00:00
Richard van der Hoff 4c8f94ac94 Allow password_auth_providers to return a callback
... so that they have a way to record access tokens.
2017-11-01 16:51:03 +00:00
Richard van der Hoff 846a94fbc9
Merge pull request #2620 from matrix-org/rav/auth_non_password
Let password auth providers handle arbitrary login types
2017-11-01 16:45:33 +00:00
Richard van der Hoff 3cd6b22c7b Let password auth providers handle arbitrary login types
Provide a hook where password auth providers can say they know about other
login types, and get passed the relevant parameters
2017-11-01 16:43:57 +00:00
David Baker 4f0488b307 Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete 2017-11-01 16:20:19 +00:00
David Baker e5e930aec3
Merge pull request #2615 from matrix-org/rav/break_auth_device_dep
Break dependency of auth_handler on device_handler
2017-11-01 16:06:31 +00:00
David Baker fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens
Remove the last vestiges of refresh_tokens
2017-11-01 15:57:35 +00:00
Richard van der Hoff dd13310fb8 Move access token deletion into auth handler
Also move duplicated deactivation code into the auth handler.

I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
2017-11-01 15:46:22 +00:00
Richard van der Hoff 74c56f794c Break dependency of auth_handler on device_handler
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.

It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
2017-11-01 10:27:06 +00:00
Richard van der Hoff 356bcafc44 Remove the last vestiges of refresh_tokens 2017-10-31 20:35:58 +00:00
Richard van der Hoff 3e0aaad190 Let auth providers get to the database
Somewhat open to abuse, but also somewhat unavoidable :/
2017-10-31 17:22:29 +00:00
Luke Barnard 20fe347906 Modify group room association API to allow modification of is_public
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Richard van der Hoff 1b65ae00ac Refactor some logic from LoginRestServlet into AuthHandler
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.

In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.

This commit *should* be non-functional.
2017-10-31 10:48:41 +00:00
Richard van der Hoff 110b373e9c Merge pull request #2589 from matrix-org/rav/as_deactivate_account
Allow ASes to deactivate their own users
2017-10-27 12:29:32 +01:00
Richard van der Hoff 785bd7fd75 Allow ASes to deactivate their own users 2017-10-27 00:01:00 +01:00
Erik Johnston 566e21eac8 Update room_list.py 2017-10-26 11:39:54 +01:00
Erik Johnston 2a7e9faeec Do logcontexts outside ResponseCache 2017-10-25 15:21:08 +01:00
Erik Johnston 33a9026cdf Add logging and fix log contexts for publicRooms 2017-10-25 10:26:06 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff 3267b81b81 Merge pull request #2561 from matrix-org/rav/id_checking
Updates to ID checking
2017-10-23 14:39:20 +01:00
Richard van der Hoff 29812c628b Allow = in mxids and groupids
... because the spec says we should.
2017-10-20 23:42:53 +01:00
Richard van der Hoff 58fbbe0f1d Disallow capital letters in userids
Factor out a common function for checking user ids and group ids, which forbids
capitals.
2017-10-20 23:37:22 +01:00
Richard van der Hoff 631d7b87b5 Remove pointless create() method
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
Luke Barnard a3ac4f6b0a _create_rererouter for get_invited_users_in_group 2017-10-16 15:41:03 +01:00
Luke Barnard 2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Luke Barnard 6079d0027a Log a warning when no profile for invited member
And return empty profile
2017-10-16 14:20:45 +01:00
David Baker b8d8ed1ba9 Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
Erik Johnston b752507b48 Fix fetching remote summaries 2017-10-11 16:59:18 +01:00
Richard van der Hoff 68fd82e840 Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
David Baker b78bae2d51 fix isinstance 2017-10-11 14:49:09 +01:00
David Baker c3b7a45e84 Allow error strings from spam checker 2017-10-11 14:39:22 +01:00
Richard van der Hoff c3e190ce67 fix a logcontext leak in read receipt handling 2017-10-11 14:37:20 +01:00
Erik Johnston 4ce4379235 Fix attestations to check correct server name 2017-10-11 14:11:43 +01:00
Erik Johnston 535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston bc7f3eb32f Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
Erik Johnston 84e27a592d Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
Richard van der Hoff c8f568ddf9 Fix up deferred handling in federation.py
* Avoid preserve_context_over_deferred, which is broken

* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
  failure" erros
2017-10-06 22:14:24 +01:00
David Baker f878e6f8af Spam checking: add the invitee to user_may_invite 2017-10-05 14:02:28 +01:00
David Baker d8ce68b09b spam check room publishing 2017-10-04 14:29:33 +01:00
David Baker 78d4ced829 un-double indent 2017-10-04 12:44:27 +01:00
David Baker 197c14dbcf Add room creation checks to spam checker
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
David Baker 1e375468de pass room id too 2017-10-03 17:13:14 +01:00
David Baker c2c188b699 Federation was passing strings anyway
so pass string everywhere
2017-10-03 15:46:19 +01:00
David Baker c46a0d7eb4 this shouldn't be debug 2017-10-03 15:20:14 +01:00
David Baker bd769a81e1 better logging 2017-10-03 15:16:40 +01:00
David Baker 41fd9989a2 Skip spam check for admin users 2017-10-03 14:17:44 +01:00
Erik Johnston e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
David Baker 2a7ed700d5 Fix param name & lint 2017-10-03 14:04:10 +01:00
David Baker 84716d267c Allow spam checker to reject invites too 2017-10-03 13:56:43 +01:00
Erik Johnston f2da6df568 Remove spurious line feed 2017-10-03 11:31:06 +01:00
Erik Johnston 30848c0fcd Ignore incoming events for rooms that we have left
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Jeremy Cline cafb8de132
Unfreeze event before serializing with ujson
In newer versions of https://github.com/esnme/ultrajson, ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487. This causes the serialization to fail if the
distribution-provided package is used.

This runs the event through the unfreeze utility before serializing it.

Thanks to @ignatenkobrain for tracking down the root cause.

fixes #2351

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
2017-09-30 11:22:37 -04:00
Erik Johnston 8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
David Baker 6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston 95298783bb Add is_publicised to group summary 2017-09-26 11:04:37 +01:00
Erik Johnston e3edca3b5d Refactor to speed up incremental syncs 2017-09-25 17:35:39 +01:00
Erik Johnston 3166ed55b2 Fix device list when rejoining room (#2461) 2017-09-22 14:44:17 +01:00
Richard van der Hoff aa620d09a0 Add a config option to block all room invites (#2457)
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
Richard van der Hoff 2eabdf3f98 add some comments to on_exchange_third_party_invite_request 2017-09-19 12:20:36 +01:00
Richard van der Hoff 5ed109d59f PoC for filtering spammy events (#2456)
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston 93e504d04e Ensure that creator of group sees group down /sync 2017-09-19 11:08:16 +01:00
Erik Johnston b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff 3f405b34e9 Fix overzealous kicking of guest users (#2453)
We should only kick guest users if the guest access event is authorised.
2017-09-19 08:52:52 +01:00
Erik Johnston 2d1b7955ae Don't filter out current state events from timeline 2017-09-18 17:13:03 +01:00
Erik Johnston d6dadd95ac Correctly handle leaving room in /key/changes 2017-09-18 15:38:22 +01:00
Erik Johnston 4a94eb3ea4 Fix typo 2017-09-15 09:56:54 +01:00
Erik Johnston 3a0cee28d6 Actually hook leave notifs up 2017-09-14 11:49:37 +01:00
Erik Johnston 4f845a0713 Handle joining/leaving rooms in /keys/changes 2017-09-13 16:28:08 +01:00
Erik Johnston 473700f016 Get left rooms 2017-09-13 15:13:41 +01:00
Erik Johnston 9ce866ed4f In sync handle device lists for newly joined/left rooms 2017-09-12 16:44:26 +01:00
Erik Johnston 69ef4987a6 Add left section to /keys/changes 2017-09-08 14:44:36 +01:00
Erik Johnston 53cc8ad35a Send down device list change notif when member leaves/rejoins room 2017-09-07 15:08:39 +01:00
Erik Johnston 4a9b1cf253 Add user profiles to summary from group server 2017-08-25 16:23:58 +01:00
Erik Johnston 258409ef61 Fix typos and reinherit 2017-08-25 14:45:20 +01:00
Erik Johnston bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston 27ebc5c8f2 Add remote profile cache 2017-08-25 11:25:47 +01:00
Erik Johnston ba3ff7918b Fixup 2017-08-11 13:42:42 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston 05e21285aa Store whether the user wants to publicise their membership of a group 2017-08-08 13:01:46 +01:00
Erik Johnston a1e67bcb97 Remove stale TODO comments 2017-08-04 10:07:10 +01:00
Erik Johnston d5e32c843f Correctly add joins to correct segment 2017-07-24 13:31:26 +01:00
Erik Johnston 96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston 960dae3340 Add notifier 2017-07-20 17:14:44 +01:00
Erik Johnston 2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston 139fe30f47 Remember to cast to bool 2017-07-20 16:47:35 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 332839f6ea Update federation client pokes 2017-07-18 14:45:37 +01:00
Erik Johnston e5ea6dd021 Add client apis 2017-07-18 14:37:06 +01:00
Erik Johnston cccfcfa7b9 Comments 2017-07-18 10:35:18 +01:00
Erik Johnston 68f34e85ce Use transport client directly 2017-07-18 10:29:57 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston e9aec001f4 Use less DB for device list handling in sync 2017-07-12 10:30:10 +01:00
Erik Johnston 28e8c46f29 Merge pull request #2352 from matrix-org/erikj/group_server_split
Initial Group Server
2017-07-12 10:26:29 +01:00
Erik Johnston 9a6fd3ef29 Don't compute push actions for backfilled events 2017-07-11 10:02:21 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston 5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Mark Haines 3e279411fe Improve the error handling for bad invites received over federation 2017-06-30 16:20:30 +01:00
Erik Johnston c72058bcc6 Use an ExpiringCache for storing registration sessions
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
Erik Johnston 816605a137 Merge pull request #2307 from matrix-org/erikj/user_ip_batch
Batch upsert user ips
2017-06-27 15:08:32 +01:00
Erik Johnston a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston 1bce3e6b35 Remove unused variables 2017-06-26 14:03:27 +01:00
Erik Johnston e3cbec10c1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip 2017-06-26 14:02:44 +01:00
Erik Johnston 44c722931b Make some more params configurable 2017-06-22 14:59:52 +01:00
Erik Johnston 2d520a9826 Typo. ARGH. 2017-06-22 14:42:39 +01:00
Erik Johnston 24d894e2e2 Fix thinko in unhandled user spam 2017-06-22 14:39:05 +01:00
Erik Johnston 1217c7da91 Don't work out users who share room with appservice users 2017-06-21 12:00:41 +01:00
Erik Johnston e5ae386ea4 Handle all cases of sending membership events 2017-06-19 16:07:54 +01:00
Erik Johnston b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston fcf01dd88e Reject local events that don't round trip the DB 2017-06-19 11:33:40 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston 4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston 72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston 6ae8373d40 Don't assume existance of events when updating user directory 2017-06-13 10:19:26 +01:00
Erik Johnston a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston 1664948e41 Comment 2017-06-09 13:05:05 +01:00
Erik Johnston 0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston 7132e5cdff Mark remote invite rejections as outliers 2017-06-09 10:08:18 +01:00
Erik Johnston f45f07ab86 Merge pull request #2258 from matrix-org/erikj/user_dir
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
Erik Johnston a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston ecdd2a3658 Don't start user_directory handling on workers 2017-06-07 12:02:53 +01:00
Erik Johnston 2f34ad31ac Add some logging to user directory 2017-06-07 11:50:44 +01:00
Erik Johnston 09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston 1a01af079e Handle profile updates in user directory 2017-06-01 15:39:51 +01:00
Erik Johnston 21e255a8f1 Split the table in two 2017-06-01 14:50:46 +01:00
Erik Johnston 7233341eac Comments 2017-06-01 13:11:38 +01:00
Erik Johnston 8be6fd95a3 Check if host is still in room 2017-06-01 13:05:39 +01:00
Erik Johnston 9c7db2491b Fix removing users 2017-06-01 11:36:50 +01:00
Erik Johnston 0fe6f3c521 Bug fixes and logging
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston cc7609aa9f Comment briefly on how we keep user_directory up to date 2017-05-31 17:11:18 +01:00
Erik Johnston b2d8d07109 Lifts things into separate function 2017-05-31 17:00:24 +01:00
Erik Johnston f9791498ae Typos 2017-05-31 16:50:57 +01:00
Erik Johnston f091061711 Fix tests 2017-05-31 16:34:40 +01:00
Erik Johnston dc51af3d03 Pull max id from correct table 2017-05-31 15:13:49 +01:00
Erik Johnston 350622a107 Handle the server leaving a public room 2017-05-31 15:11:36 +01:00
Erik Johnston 63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston 3b5f22ca40 Add search 2017-05-31 14:00:01 +01:00
Erik Johnston b5db4ed5f6 Update room column when room becomes unpublic 2017-05-31 13:40:28 +01:00
Erik Johnston 168524543f Add call later 2017-05-31 11:59:36 +01:00
Erik Johnston 3e123b8497 Start later 2017-05-31 11:56:27 +01:00
Erik Johnston 42137efde7 Don't go round in circles 2017-05-31 11:55:13 +01:00
Erik Johnston eeb2f9e546 Add user_directory to database 2017-05-31 11:51:01 +01:00