Commit Graph

3995 Commits

Author SHA1 Message Date
Daniel Wagner-Hall edca2d9891 Idempotent-ise schema update script
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
David Baker c00f4e48ba Merge pull request #630 from matrix-org/dbkr/post_urlencoded_encode_params
Encode unicode objects given to post_urlencode*
2016-03-08 13:51:49 +00:00
Mark Haines 7076082ae6 Fix relative imports so they work in both py3 and py27 2016-03-08 11:45:50 +00:00
Mark Haines ea72bd9600 Merge pull request #631 from matrix-org/markjh/py3v1
Use syntax that's valid on both py2.7 and py3
2016-03-08 11:08:20 +00:00
David Baker 9a3c80a348 pep8 2016-03-08 10:09:07 +00:00
David Baker 7bcee4733a Encode unicode objects given to post_urlencode* otherwise urllib.urlencode chokes. 2016-03-08 10:04:38 +00:00
Mark Haines 239badea9b Use syntax that works on both py2.7 and py3 2016-03-07 20:13:10 +00:00
David Baker 874fd43257 Send the user ID matching the guest access token, since there is no Matrix API to discover what user ID an access token is for. 2016-03-07 17:13:56 +00:00
Erik Johnston 80916e6884 Merge pull request #626 from matrix-org/erikj/visibility
Send history visibility on boundary changes
2016-03-07 11:56:07 +00:00
Erik Johnston 2ab0bf4b97 Send history visibility on boundary changes 2016-03-04 16:54:32 +00:00
Mark Haines b7a3be693b Merge pull request #618 from matrix-org/markjh/pushrule_stream
Add a stream for push rule updates
2016-03-04 16:35:08 +00:00
Erik Johnston beebc0a40f Merge pull request #625 from matrix-org/erikj/kick_ban_sync
Always include kicks and bans in full /sync
2016-03-04 16:33:52 +00:00
Mark Haines 9848b54cac Prefill from the correct stream 2016-03-04 16:20:22 +00:00
Mark Haines deda48068c prefill the push rules stream change cache 2016-03-04 16:19:42 +00:00
Mark Haines ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines 7e9fc9b6af /FNARG/d 2016-03-04 15:54:09 +00:00
Erik Johnston 125f674eae Always include kicks and bans in full /sync 2016-03-04 15:27:55 +00:00
Erik Johnston 13cbd31040 Spelling 2016-03-04 15:22:39 +00:00
Mark Haines ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Mark Haines 1b4f4a936f Hook up the push rules stream to account_data in /sync 2016-03-04 14:44:01 +00:00
Erik Johnston ed61a49169 Add profile information to invites 2016-03-04 14:35:02 +00:00
Erik Johnston 389d558a3b Merge pull request #598 from Rugvip/invite-state
config,handlers/_base: added homeserver config for what state is included in a room invite
2016-03-04 09:58:52 +00:00
Erik Johnston 44b084a75e Merge pull request #596 from Rugvip/create
handlers/_base: don't allow room create event to be changed
2016-03-04 09:43:20 +00:00
Patrik Oldsberg 5fc59f009c config,handlers/_base: added homeserver config for what state is included in a room invite
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-03-04 10:43:17 +01:00
Erik Johnston ce82b9e48f Merge pull request #610 from Rugvip/unique-user
handlers/register: make sure another user id is generated when a collision occurs
2016-03-04 09:40:20 +00:00
Richard van der Hoff 09b1d98070 Merge pull request #623 from matrix-org/rav/fix_createroom_race
Make sure we add all invited members before returning from createRoom
2016-03-04 09:29:15 +00:00
Erik Johnston dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
Richard van der Hoff a92b4ea76f Make sure we add all invited members before returning from createRoom
add a missing yield.
2016-03-04 00:06:03 +00:00
Richard van der Hoff a85179aff3 Merge remote-tracking branch 'origin/develop' into rav/SYN-642 2016-03-03 19:05:54 +00:00
Daniel Wagner-Hall 0b3083c75b Merge pull request #621 from matrix-org/daniel/ratelimiting
Pass whole requester to ratelimiting
2016-03-03 17:02:48 +00:00
Daniel Wagner-Hall b4022cc487 Pass whole requester to ratelimiting
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
Erik Johnston 5c90451ea0 Merge pull request #619 from matrix-org/dbkr/dont_use_checkpw
Stop using checkpw as it seems to have vanished from bcrypt.
2016-03-03 16:25:41 +00:00
Mark Haines 3406eba4ef Move the code for formatting push rules into a separate function 2016-03-03 16:11:59 +00:00
Mark Haines ddf9e7b302 Hook up the push rules to the notifier 2016-03-03 14:57:45 +00:00
Daniel Wagner-Hall 95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Richard van der Hoff 79f34bdbc2 Merge branch 'develop' into rav/SYN-642 2016-03-03 11:39:25 +00:00
Richard van der Hoff 74cd80e530 Fix typo 2016-03-03 10:28:58 +00:00
David Baker ff8b87118d Stop using checkpw as it seems to have vanished from bcrypt. Use `bcrypt.hashpw(password, hashed) == hashed` as per the bcrypt README. 2016-03-02 18:06:45 +00:00
Mark Haines 2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Richard van der Hoff fc1f932cc0 Move arg default to the start of the function
Also don't overwrite the list that gets passed in.
2016-03-02 16:44:14 +00:00
Matthew Hodgson c0147f86a1 Merge pull request #616 from matrix-org/matthew/800x600
add 800x600 thumbnails to make vector look prettier (and anyone else …
2016-03-02 16:02:26 +00:00
Matthew Hodgson 47c361d2f8 add 800x600 thumbnails to make vector look prettier (and anyone else who likes big thumbnails) 2016-03-02 15:57:54 +00:00
Richard van der Hoff 863d3f26b3 fix pyflakes quibble 2016-03-02 15:52:50 +00:00
Richard van der Hoff 9ff940a0ef Address review comments 2016-03-02 15:40:30 +00:00
Erik Johnston 2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
Mark Haines a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Richard van der Hoff 05ea111c47 Fix pyflakes warning 2016-03-01 17:45:24 +00:00
Richard van der Hoff 8a1d3b86af Handle rejections of invites from local users locally
Slightly hacky fix to SYN-642, which avoids the federation codepath when trying
to reject invites from local users.
2016-03-01 17:27:22 +00:00
Mark Haines a612ce6659 Merge pull request #489 from matrix-org/markjh/replication
Add a /replication API for extracting the updates that happened on synapse.
2016-03-01 15:08:24 +00:00
Mark Haines d50ca1b1ed Merge pull request #613 from matrix-org/markjh/yield
Load the current id in the IdGenerator constructor
2016-03-01 14:54:29 +00:00
Mark Haines 60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Mark Haines 54172924c8 Load the current id in the IdGenerator constructor
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Erik Johnston 374f9b2f07 Limit stream change cache size too 2016-03-01 13:30:15 +00:00
Erik Johnston ce2cdced61 Move cache size fiddling to descriptors only. Fix tests 2016-03-01 13:21:46 +00:00
Erik Johnston 910fc0f28f Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 2016-03-01 12:56:39 +00:00
Erik Johnston 72165e5b77 Reraise exception 2016-03-01 11:00:10 +00:00
Erik Johnston ff2d7551c7 Correct cache miss detection 2016-03-01 10:59:17 +00:00
Erik Johnston 903fb34b39 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/expiring_cache_size 2016-03-01 09:43:27 +00:00
Patrik Oldsberg 9c48f1ed22 handlers/register: make sure another user id is generated when a collision occurs
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-29 23:12:37 +01:00
David Baker bfdcc7b9b6 Merge pull request #607 from matrix-org/dbkr/send_inviter_member_event
Send the invier's member event in room invite state
2016-02-26 14:32:08 +00:00
Mark Haines de27f7fc79 Add support for changing the actions for default rules
See matrix-org/matrix-doc#283

Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``
2016-02-26 14:28:19 +00:00
David Baker 413e36b17a Merge remote-tracking branch 'origin/develop' into dbkr/send_inviter_member_event 2016-02-26 13:40:25 +00:00
Gergely Polonkai 87acd8fb07 Fix to appease the PEP8 dragon 2016-02-26 12:05:38 +01:00
Gergely Polonkai a53774721a Add error codes for malformed/bad JSON in /login
Signed-off-by: Gergely Polonkai <gergely@polonkaieu>
2016-02-26 10:22:35 +01:00
David Baker 0f0b011440 Send the invier's member event in room invite state so the invitee has their display name and avatar. 2016-02-25 18:12:09 +00:00
Mark Haines 15c2ac2cac Make sure we return a JSON object when returning the values of specif…
…ic keys from a push rule
2016-02-25 15:13:07 +00:00
Mark Haines 4ecfbac85f Merge branch 'develop' into markjh/dead_code 2016-02-24 16:37:46 +00:00
Mark Haines 9892d017b2 Remove unused get_rule_attr method 2016-02-24 16:31:07 +00:00
Daniel Wagner-Hall 33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Daniel Wagner-Hall 869580206d Ignore invalid POST bodies when joining rooms 2016-02-24 08:50:28 +00:00
Erik Johnston 278d6c0527 Report size of ExpiringCache 2016-02-23 16:46:21 +00:00
Erik Johnston e7ab0e0f9f Merge pull request #603 from matrix-org/erikj/presence
Create a new stream_id per presence update
2016-02-23 16:14:01 +00:00
Erik Johnston 6451fcd085 Create a new stream_id per presence update 2016-02-23 15:51:39 +00:00
Erik Johnston b5f77eb12a Check presence token interval is less than 100, rather than the token itself 2016-02-23 15:47:37 +00:00
Erik Johnston e3e0ac6ec7 Merge pull request #602 from matrix-org/erikj/presence
Change the way we figure out presence updates for small deltas
2016-02-23 15:18:41 +00:00
Daniel Wagner-Hall 577951b032 Allow third_party_signed to be specified on /join 2016-02-23 15:11:25 +00:00
Erik Johnston 13f86c3489 Handle get_all_entities_changed returning None 2016-02-23 15:05:37 +00:00
Erik Johnston 6e0209112b Add comments 2016-02-23 14:57:45 +00:00
Erik Johnston c77dae7a1a Change the way we figure out presence updates for small deltas 2016-02-23 14:54:40 +00:00
Erik Johnston a7b2ce32f7 Merge pull request #600 from matrix-org/erikj/presence
Measure PresenceEventSource.get_new_events
2016-02-23 14:54:26 +00:00
Erik Johnston 02e928cf9b Don't include defer.returnValue in Measure block 2016-02-23 14:52:16 +00:00
Erik Johnston 56a94ccd9e Measure PresenceEventSource.get_new_events 2016-02-23 14:52:16 +00:00
Patrik Oldsberg baf056bae8 handlers/sync: style fix, line too long
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 15:49:36 +01:00
Erik Johnston 138c405974 Pick up currently_active across federation 2016-02-23 10:40:11 +00:00
Patrik Oldsberg 210b7d8e00 handlers/_base: don't allow room create event to be changed
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 00:22:41 +01:00
Patrik Oldsberg 1dcfb201c4 handlers/sync: fix SyncResult not counting account_data change when converting to bool
This fixes account_data events not triggering an immediate /sync response

Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 00:21:59 +01:00
Richard van der Hoff f7e3de02ef Merge pull request #587 from matrix-org/rav/guest_access_after_room_join
Allow guest users access to messages in rooms they have joined
2016-02-22 16:36:06 +00:00
Richard van der Hoff 5be3944730 address review comments
drop commented-out special casing for historyvisibility event
s/he/they/ for users
2016-02-22 15:27:44 +00:00
Erik Johnston 9e696bd6a3 Remove superfluous call to get_state_at when we already have an event for that stream position 2016-02-22 13:54:46 +00:00
Richard van der Hoff 6c5b147a39 Interpret unknown visibilities the same as shared 2016-02-19 17:11:11 +00:00
Erik Johnston 3dbaeef58c Correctly filter states 2016-02-19 12:27:35 +00:00
Erik Johnston 42ac5f0c1a Only send presence updates to remote hosts if user is ours 2016-02-19 12:19:56 +00:00
Richard van der Hoff 05aee12652 Merge branch 'develop' into rav/guest_access_after_room_join 2016-02-19 12:00:16 +00:00
Erik Johnston 24d9f2c140 Add Measures to presence 2016-02-19 11:50:48 +00:00
Richard van der Hoff b71ca2b014 Allow guest users access to messages in rooms they have joined
There should be no difference between guest users and non-guest users in terms
of access to messages. Define the semantics of the is_peeking argument to
filter_events_for_clients (slightly) better; interpret it appropriately, and
set it correctly from /sync.
2016-02-19 11:41:02 +00:00
Erik Johnston 4a95eb0a12 Add presence metric 2016-02-19 11:32:04 +00:00
Erik Johnston be799453aa Remove spurious extra metrics 2016-02-19 11:29:33 +00:00
Erik Johnston ea7786e8ca Merge pull request #586 from matrix-org/erikj/presence
Fix presence `currently_active`. Add presence metrics.
2016-02-19 11:26:32 +00:00
Erik Johnston 929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston 5f4eca3816 More metrics 2016-02-19 10:21:41 +00:00
Erik Johnston 5614b4dafb Add presence metrics 2016-02-19 09:50:54 +00:00
Erik Johnston e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
Erik Johnston e12ec335a5 "You are not..." 2016-02-18 17:01:53 +00:00
Erik Johnston 220231d8e3 Merge pull request #573 from matrix-org/erikj/sync_fix
Mitigate against incorrect old state in /sync.
2016-02-18 16:40:58 +00:00
Erik Johnston e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston 42109a62a4 Remove unused param from get_max_token 2016-02-18 16:37:28 +00:00
Erik Johnston b8cdec92c7 WheelTimer: Don't scan list, use index. 2016-02-18 16:33:07 +00:00
Mark Haines b9977ea667 Remove dead code for setting device specific rules.
It wasn't possible to hit the code from the API because of a typo
in parsing the request path. Since no-one was using the feature
we might as well remove the dead code.
2016-02-18 16:05:13 +00:00
Erik Johnston 48b652bcbe Remove invalid arg. 2016-02-18 14:57:09 +00:00
Erik Johnston b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Daniel Wagner-Hall f8d21e1431 Review comments 2016-02-18 11:02:14 +00:00
Erik Johnston fe95f2217c Add stuff pulled from the DB to the cache 2016-02-18 10:26:24 +00:00
Erik Johnston 8351538873 PEP8 2016-02-18 10:12:12 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston b31ec214a5 Remove status_msg when going offline. Don't offline -> online if you send a message 2016-02-18 09:54:08 +00:00
Erik Johnston 114b929f8b Check presence state is a valid one 2016-02-18 09:16:32 +00:00
Erik Johnston ddca9c56fc Move if statement 2016-02-18 09:11:53 +00:00
Erik Johnston 58371fa263 Comment 2016-02-18 09:09:50 +00:00
Daniel Wagner-Hall 7e90fb6a57 Merge branch 'develop' into daniel/roomcleanupincremental
Conflicts:
	synapse/rest/client/v1/room.py
2016-02-17 15:53:59 +00:00
Daniel Wagner-Hall 591af2d074 Some cleanup
I'm not particularly happy with the "action" switching, but there's no
convenient way to defer the work that needs to happen after it, so... :(
2016-02-17 15:50:13 +00:00
Erik Johnston c229c87398 Remove spurious comment 2016-02-17 15:48:29 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall a4e278bfe7 Respond to federated invite with non-empty context
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
2016-02-17 15:25:12 +00:00
Erik Johnston 9e7900da1e Add wheeltimer impl 2016-02-17 14:29:28 +00:00
Erik Johnston 200de16440 Merge pull request #580 from Rugvip/develop
api/filtering: don't assume that event content will always be a dict
2016-02-17 12:51:02 +00:00
Patrik Oldsberg 536f949a1a api/filtering: don't assume that event content will always be a dict
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-17 12:59:41 +01:00
Patrik Oldsberg 71d5d2c669 client/v1/room: include event_id in response to state event PUT, in accordance with the spec
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-17 11:53:43 +01:00
Daniel Wagner-Hall 6605adf669 Some cleanup, some TODOs, more to do 2016-02-16 19:05:02 +00:00
Mark Haines 458782bf67 Fix typo in request validation for adding push rules. 2016-02-16 18:00:30 +00:00
Mark Haines a9c9868957 Make adding push rules idempotent
Also remove the **kwargs from the add_push_rule method.

Fixes https://matrix.org/jira/browse/SYN-391
2016-02-16 15:53:38 +00:00
Daniel Wagner-Hall d1fb790818 Some cleanup 2016-02-16 14:25:23 +00:00
Daniel Wagner-Hall 1f403325ac Tidy? up room creation event sending 2016-02-16 12:00:50 +00:00
Daniel Wagner-Hall 04686df17a Add comment 2016-02-16 11:52:46 +00:00
Daniel Wagner-Hall feedaa37fa Merge branch 'develop' into daniel/roomcleanupincremental
Conflicts:
	synapse/handlers/room.py
2016-02-16 11:34:48 +00:00
Daniel Wagner-Hall 4bfb32f685 Branch off member and non member sends
Unclean, needs tidy-up, but works
2016-02-15 18:21:30 +00:00
Daniel Wagner-Hall 1a2197d7bf Simplify room creation code 2016-02-15 18:19:01 +00:00
Daniel Wagner-Hall e560045cfd Simplify room creation code 2016-02-15 18:18:39 +00:00
Daniel Wagner-Hall 8168341e9b Use update_membership for profile updates 2016-02-15 17:14:34 +00:00
Daniel Wagner-Hall 1bbb67c452 Use update_membership to kick guests 2016-02-15 16:40:22 +00:00
Daniel Wagner-Hall 150fcde0dc Reuse update_membership from /join 2016-02-15 16:16:03 +00:00
Daniel Wagner-Hall 73e616df2a Inline _do_local_membership_update 2016-02-15 16:02:22 +00:00
Daniel Wagner-Hall f318d4f2a4 Inline _do_join as it now only has one caller
Also, consistently apply rate limiting.

Again, ugly, but a step in the right direction.
2016-02-15 15:57:10 +00:00
Daniel Wagner-Hall e71095801f Merge implementation of /join by alias or ID
This code is kind of rough (passing the remote servers down a long
chain), but is a step towards improvement.
2016-02-15 15:39:16 +00:00
Daniel Wagner-Hall dbeed36dec Merge some room joining codepaths
Force joining by alias to go through the send_membership_event checks,
rather than bypassing them straight into _do_join. This is the first of
many stages of cleanup.
2016-02-15 14:38:27 +00:00
Daniel Wagner-Hall 4de08a4672 Revert "Merge two of the room join codepaths"
This reverts commit cf81375b94.

It subtly violates a guest joining auth check
2016-02-12 16:17:24 +00:00
Daniel Wagner-Hall cf81375b94 Merge two of the room join codepaths
There's at least one more to merge in.

Side-effects:
 * Stop reporting None as displayname and avatar_url in some cases
 * Joining a room by alias populates guest-ness in join event
 * Remove unspec'd PUT version of /join/<room_id_or_alias> which has not
   been called on matrix.org according to logs
 * Stop recording access_token_id on /join/room_id - currently we don't
   record it on /join/room_alias; I can try to thread it through at some
   point.
2016-02-12 15:11:49 +00:00
Mark Haines 58c9f20692 Catch the exceptions thrown by twisted when you write to a closed connection 2016-02-12 13:46:59 +00:00
Erik Johnston ec0f3836ff Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-02-12 11:19:37 +00:00
Erik Johnston 4d54d87c3e Mitigate against incorrect old state in /sync.
There have been reports from the field that servers occasionally have
incorrect notions of what the old state of a room is. This proves
problematic as /sync relies on a correct old state.
This patch makes /sync specifically include in the 'state' key any
current state events that haven't been correctly included.
2016-02-12 11:13:06 +00:00