Commit Graph

1293 Commits

Author SHA1 Message Date
Erik Johnston 8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Erik Johnston 69ef4987a6 Add left section to /keys/changes 2017-09-08 14:44:36 +01:00
Erik Johnston 53cc8ad35a Send down device list change notif when member leaves/rejoins room 2017-09-07 15:08:39 +01:00
Erik Johnston bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston ba3ff7918b Fixup 2017-08-11 13:42:42 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston b880ff190a Allow update group publicity 2017-08-08 14:19:41 +01:00
hera eae04f1952 fix english 2017-08-04 23:56:42 +01:00
hera 5699b05072 typo 2017-08-04 23:44:37 +01:00
Erik Johnston 09552f9d9c Reduce spammy log line in synchrotrons 2017-08-02 17:29:51 +01:00
Erik Johnston 96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston b238cf7f6b Remove spurious content param 2017-07-20 17:49:55 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston 3431ec55dc Comments 2017-07-18 17:23:50 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Krombel 85b9f76f1d split out reducing stuff; just make encode_* static 2017-07-11 13:14:35 +02:00
Krombel 2f82de18ee fix test 2017-07-10 17:34:58 +02:00
Krombel 6e16aca8b0 encode sync-response statically; omit empty objects from sync-response 2017-07-10 16:42:17 +02:00
Erik Johnston 5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Erik Johnston 7fe8ed1787 Store URL cache preview downloads seperately
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston b8b936a6ea Add API to quarantine media 2017-06-19 17:39:21 +01:00
Erik Johnston 36e51aad3c Remove unused import 2017-06-19 14:42:21 +01:00
Erik Johnston b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston 5db7070dd1 Forget room 2017-06-19 12:40:29 +01:00
Erik Johnston d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston 48d2949416 Throw exception when not retrying when downloading media 2017-06-13 10:23:14 +01:00
Erik Johnston 63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston 535c99f157 Use POST 2017-05-31 14:15:45 +01:00
Erik Johnston 45a5df5914 Add REST API 2017-05-31 14:11:55 +01:00
Erik Johnston 9a804b2812 Merge pull request #2243 from matrix-org/matthew/fix-url-preview-length-again
actually trim oversize og:description meta
2017-05-23 13:26:28 +01:00
Matthew Hodgson 836d5c44b6 actually trim oversize og:description meta 2017-05-22 21:14:20 +01:00
Erik Johnston b4f59c7e27 Add count of one time keys to sync stream 2017-05-19 15:47:55 +01:00
Erik Johnston 2c9475b58e Merge pull request #2221 from psaavedra/sync_timeline_limit_filter_by_name
Configurable maximum number of events requested by /sync and /messages
2017-05-15 16:08:46 +01:00
Erik Johnston d12ae7fd1c Don't log exceptions for NotRetryingDestination 2017-05-15 15:42:18 +01:00
Pablo Saavedra 224137fcf9 Fixed syntax nits 2017-05-15 16:21:02 +02:00
Pablo Saavedra 627e6ea2b0 Fixed implementation errors
* Added HS as property in SyncRestServlet
* Fixed set_timeline_upper_limit function implementat¡ion
2017-05-15 14:51:43 +02:00
Pablo Saavedra 9da4316ca5 Configurable maximum number of events requested by /sync and /messages (#2220)
Set the limit on the returned events in the timeline in the get and sync
operations. The default value is -1, means no upper limit.

For example, using `filter_timeline_limit: 5000`:

POST /_matrix/client/r0/user/user:id/filter
{
room: {
    timeline: {
      limit: 1000000000000000000
    }
}
}

GET /_matrix/client/r0/user/user:id/filter/filter:id

{
room: {
    timeline: {
      limit: 5000
    }
}
}

The server cuts down the room.timeline.limit.
2017-05-13 18:17:54 +02:00
Luke Barnard 369195caa5 Modify register/available to be GET with query param
- GET is now the method for register/available
- a query parameter "username" is now used

Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
Luke Barnard f7278e612e Change register/available to POST (from GET) 2017-05-10 11:40:18 +01:00
Luke Barnard 6aa423a1a8 Merge pull request #2183 from matrix-org/luke/username-availability
Implement username availability checker
2017-05-04 09:58:40 +01:00
Luke Barnard 3669065466 Appease the flake8 gods 2017-05-03 18:05:49 +01:00
Luke Barnard 34ed4f4206 Implement username availability checker
Outlined here: https://github.com/vector-im/riot-web/issues/3605#issuecomment-298679388

```HTTP
GET /_matrix/.../register/available
{
    "username": "desiredlocalpart123"
}
```

If available, the response looks like
```HTTP
HTTP/1.1 200 OK
{
    "available": true
}
```

Otherwise,
```HTTP
HTTP/1.1 429
{
    "errcode": "M_LIMIT_EXCEEDED",
    "error": "Too Many Requests",
    "retry_after_ms": 2000
}
```
or
```HTTP
HTTP/1.1 400
{
    "errcode": "M_USER_IN_USE",
    "error": "User ID already taken."
}

```
or
```HTTP
HTTP/1.1 400
{
    "errcode": "M_INVALID_USERNAME",
    "error": "Some reason for username being invalid"
}
```
2017-05-03 12:04:12 +01:00
Erik Johnston 3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston 25a96e0c63 Merge pull request #2163 from matrix-org/erikj/fix_invite_state
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
Erik Johnston 46826bb078 Comment and remove spurious logging 2017-04-27 17:25:44 +01:00
Erik Johnston d3c150411c Merge pull request #2130 from APwhitehat/roomexists
Check that requested room_id exists
2017-04-27 09:20:26 +01:00
Erik Johnston 34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
Erik Johnston d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Mark Haines e6e2627636 Fix code for reporting old verify keys in synapse 2017-04-24 18:51:25 +01:00
Luke Barnard 3fb8784c92 m.read_marker -> m.fully_read (#2128)
Also:
 - change the REST endpoint to have a "S" on the end (so it's now /read_markers)
 - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
Anant Prakash 574d573ac2 Check that requested room_id exists 2017-04-14 23:50:59 +05:30
Luke Barnard cf6121e3da More null-guard changes 2017-04-12 14:02:03 +01:00
Luke Barnard c0aba0a23e Remove Unused ref to hs 2017-04-12 10:52:11 +01:00
Luke Barnard b9676a75f6 Move a space 2017-04-12 10:51:17 +01:00
Luke Barnard 0127423027 flake8 2017-04-11 17:07:07 +01:00
Luke Barnard d892079844 Finish implementing RM endpoint
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
lukebarnard e263c26690 Initial commit of RM server-side impl
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
2017-04-11 11:55:30 +01:00
Erik Johnston 142b6b4abf Merge pull request #2011 from matrix-org/matthew/turn_allow_guests
add setting (on by default) to support TURN for guests
2017-03-31 14:37:09 +01:00
Erik Johnston 2a06b44be2 Merge pull request #1986 from matrix-org/matthew/enable_guest_3p
enable guest access for the 3pl/3pid APIs
2017-03-31 14:36:03 +01:00
Matthew Hodgson 9ee397b440 switch to allow_guest=True for authing 3Ps as per PR feedback 2017-03-31 13:54:26 +01:00
Richard van der Hoff 01fc847f7f Merge pull request #2057 from matrix-org/rav/missing_yield_2
Add another missing yield on check_device_registered
2017-03-24 12:46:43 +00:00
Richard van der Hoff 13c8749ac9 Add another missing yield on check_device_registered 2017-03-23 22:18:53 +00:00
David Baker a2dfab12c5 Fix token request for addition of phone numbers 2017-03-23 18:46:17 +00:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston a297155a97 Remove unused import 2017-03-15 14:49:25 +00:00
Erik Johnston 6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
David Baker 0ad44acb5a Merge pull request #1997 from matrix-org/dbkr/cas_partialdownload
Handle PartialDownloadError in CAS login
2017-03-15 13:52:34 +00:00
Matthew Hodgson e0ff66251f add setting (on by default) to support TURN for guests 2017-03-15 12:22:18 +00:00
Erik Johnston 7827251daf Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
Richard van der Hoff f2ed64eaaf Merge pull request #1992 from matrix-org/rav/fix_media_loop
Fix routing loop when fetching remote media
2017-03-14 23:40:35 +00:00
Richard van der Hoff 1d09586599 Address review comments
- don't blindly proxy all HTTPRequestExceptions
- log unexpected exceptions at error
- avoid `isinstance`
- improve docs on `from_http_response_exception`
2017-03-14 14:15:37 +00:00
David Baker 1ece06273e Handle PartialDownloadError in CAS login 2017-03-14 13:37:36 +00:00
David Baker 7b6ed9871e Use extend instead of += 2017-03-14 10:49:55 +00:00
David Baker d79a687d85 Oops, remove print 2017-03-14 10:40:20 +00:00
Luke Barnard bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
2017-03-13 17:53:23 +00:00
David Baker 0a9945220e Fix registration for broken clients
Only offer msisdn flows if the x_show_msisdn option is given.
2017-03-13 17:29:38 +00:00
David Baker 73a5f06652 Support registration / login with phone number
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
Luke Barnard c077c3277b Flake 2017-03-13 16:45:38 +00:00
Luke Barnard c81f33f73d Implement delete_devices API
This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.
2017-03-13 16:33:51 +00:00
Richard van der Hoff 170ccc9de5 Fix routing loop when fetching remote media
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.

This should fix a routing loop where the server keeps forwarding back to
itself.

Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
Erik Johnston 7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Matthew Hodgson a61dd408ed enable guest access for the 3pl/3pid APIs 2017-03-12 19:30:45 +00:00
David Baker ece7e00048 Comment when our 3pids would be incomplete 2017-03-08 19:07:18 +00:00
David Baker 727124a762 Not any more, it doesn't 2017-03-08 19:00:23 +00:00
David Baker d4d3629aaf Better error message 2017-03-08 17:01:26 +00:00
David Baker 1c99934b28 pep8 2017-03-08 11:58:20 +00:00
David Baker 85bb322333 Pull out datastore in initialiser 2017-03-08 11:51:25 +00:00
David Baker 65d43f3ca5 Minor fixes from PR feedback 2017-03-08 11:48:43 +00:00
David Baker 82c5e7de25 Typos 2017-03-08 11:42:44 +00:00
David Baker 2e27339add Refector out assert_params_in_request
and replace requestEmailToken where we meant requestMsisdnToken
2017-03-08 11:37:34 +00:00
David Baker 88df6c0c9a Factor out msisdn canonicalisation
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
David Baker 402a7bf63d Fix pep8 2017-03-08 09:33:40 +00:00
David Baker 00466e2feb Support new login format
https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#
2017-03-07 16:37:23 +00:00
David Baker b0effa2160 Add msisdns as 3pids during registration
and support binding them with the bind_msisdn param
2017-03-03 18:34:39 +00:00
David Baker 3557cf34dc Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin 2017-03-01 17:20:37 +00:00
Erik Johnston acf6d4d2e3 Merge pull request #1945 from jkolo/fix_dynamic_thumbnails_aspect
Fix #1677 (dynamic thumbnails aspect)
2017-02-27 09:51:52 +00:00
Jurek aea5461488 Fix dynamic thumbnails aspect 2017-02-24 22:43:27 +01:00
Erik Johnston 17673404fb Remove unused param 2017-02-20 15:02:01 +00:00
David Baker fa467e62a9 Merge pull request #1917 from matrix-org/dbkr/make_ban_reasons_work
Make kick & ban reasons work
2017-02-14 16:10:06 +00:00
David Baker 355d62c499 Make kick & ban reasons work
We somehow specced APIs with reason strings, preserve the content
in the events  and even have the clients display them, but failed
to actually pass the parameter through to the event content.
2017-02-14 15:10:55 +00:00
David Baker ce3e583d94 WIP support for msisdn 3pid proxy methods 2017-02-14 15:05:55 +00:00
Kegan Dougal d0497425f8 Ordering is important on errbacks so add the cleanup func before creating an ObservableDeferred 2017-02-13 13:49:44 +00:00
Kegan Dougal 808ddf0ae7 Pop the txn from the map in case it has already been deleted somehow 2017-02-13 13:36:15 +00:00
Kegan Dougal feb15dc99f Don't cache errors at all 2017-02-13 13:33:12 +00:00
Kegan Dougal ecd7e36047 http txns: Do not cache error responses
Previously we did. This meant that, amongst other errors, rate-limiting errors
would be cached and prevent messages with that txn ID being sent.
2017-02-13 13:16:48 +00:00
Erik Johnston af6da6db2d Merge pull request #1784 from morteza-araby/user-admin
Administration functionalities
2017-02-06 16:21:10 +01:00
Erik Johnston 6826593b81 sets aren't JSON serializable 2017-02-02 14:55:54 +00:00
Morteza Araby 2849d3f29d admin,storage: added more administrator functionalities
administrators can now:
 - Set displayname of users
 - Update user avatars
 - Search for users by user_id
 - Browse all users in a paginated API
 - Reset user passwords
 - Deactivate users

Helpers for doing paginated queries has also been added to storage

Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
Erik Johnston 51adaac953 Fix email push in pusher worker
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
Erik Johnston 7e919bdbd0 Include newly joined users in /keys/changes API 2017-02-01 17:33:16 +00:00
Erik Johnston 73d676dc8b Comment 2017-02-01 13:17:17 +00:00
Erik Johnston acb501c46d Comment 2017-02-01 10:32:49 +00:00
Erik Johnston 97479d0c54 Implement /keys/changes 2017-02-01 10:30:03 +00:00
Erik Johnston 2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Marvin Steadfast 1e38be3a7a Added username and password for turn server
It makes it possible to use a turn server that needs a username and
password instead of a token.
2017-01-19 14:08:20 +01:00
David Baker 8c5009b628 Lowercase all email addresses before querying db
Since we store all emails in the DB in lowercase
(https://github.com/matrix-org/synapse/pull/1170)
2017-01-18 13:25:56 +00:00
Erik Johnston ebf94aff8d Fix spurious Unhandled Error log lines 2017-01-12 17:19:47 +00:00
Mark Haines 32019c9897 Log which files we saved attachments to in the media_repository 2017-01-10 14:19:50 +00:00
Erik Johnston 7d0981b312 Merge pull request #1787 from matrix-org/erikj/linearize_member
Linearize updates to membership via PUT /state/
2017-01-10 14:04:54 +00:00
Erik Johnston 6823fe5241 Linearize updates to membership via PUT /state/ 2017-01-09 18:25:13 +00:00
Erik Johnston f7085ac84f Name linearizer's for better logs 2017-01-09 17:17:10 +00:00
David Baker 0c88ab1844 Add /account/3pid/delete endpoint
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
Marcin Bachry 24c16fc349 Fix crash in url preview when html tag has no text
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
Erik Johnston 1574b839e0 Merge pull request #1676 from matrix-org/erikj/room_list
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
Richard van der Hoff abed247182 Remove unspecced GET endpoints for e2e keys
GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws
500 errors if you call it without all the right params.

GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to
stop people relying on unspecced APIs.
2016-12-12 12:31:40 +00:00
Erik Johnston 52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston de796f27e6 Add joined_rooms servlet 2016-12-08 11:39:03 +00:00
Erik Johnston f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Johannes Löthberg 32c8b5507c preview_url_resource: Ellipsis must be in unicode string
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-01 13:12:13 +01:00
Richard van der Hoff 6841d8ff55 Fix doc-string
Remove refresh_token reference
2016-12-01 11:42:17 +00:00
Richard van der Hoff aa09d6b8f0 Rip out more refresh_token code
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.

Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
Richard van der Hoff dc4b23e1a1 Merge branch 'develop' into rav/no_more_refresh_tokens 2016-11-30 17:10:04 +00:00
Richard van der Hoff 321fe5c44c Merge pull request #1656 from matrix-org/rav/remove_time_caveat
Stop putting a time caveat on access tokens
2016-11-30 16:53:20 +00:00
Richard van der Hoff 77eca2487c Merge pull request #1653 from matrix-org/rav/guest_e2e
Implement E2E for guests
2016-11-29 17:41:35 +00:00
Richard van der Hoff 1c4f05db41 Stop putting a time caveat on access tokens
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.

Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
Richard van der Hoff 5c4edc83b5 Stop generating refresh tokens
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
Richard van der Hoff f62b69e32a Allow guest access to endpoints for E2E
Expose /devices, /keys, and /sendToDevice to guest users, so that they can use
E2E.
2016-11-25 15:26:34 +00:00
Richard van der Hoff 7f02e4d008 Give guest users a device_id
We need to create devices for guests so that they can use e2e, but we don't
have anywhere to store it, so just use a fixed one.
2016-11-25 15:25:30 +00:00
Erik Johnston feec718265 Shuffle receipt handler around so that worker apps don't need to load it 2016-11-23 15:14:24 +00:00
Kegsay d4a459f7cb Merge pull request #1638 from matrix-org/kegan/sync-event-fields
Implement "event_fields" in filters
2016-11-22 14:02:38 +00:00
Kegan Dougal cea4e4e7b2 Glue only_event_fields into the sync rest servlet 2016-11-22 10:14:05 +00:00
Mark Haines a289150943 Fix flake8 2016-11-18 17:15:02 +00:00
David Baker 544722bad2 Work around client replacing reg params
Works around https://github.com/vector-im/vector-android/issues/715
and equivalent for iOS
2016-11-18 17:07:35 +00:00
Kegan Dougal 3991b4cbdb Clean transactions based on time. Add HttpTransactionCache tests. 2016-11-14 11:19:24 +00:00
Kegan Dougal af4a1bac50 Move .observe() up to the cache to make things neater 2016-11-14 09:52:41 +00:00
Kegan Dougal 8ecaff51a1 Review comments 2016-11-11 17:47:03 +00:00
Kegan Dougal f6c48802f5 More flake8 2016-11-11 15:08:24 +00:00
Kegan Dougal a88bc67f88 Flake8 and fix whoopsie 2016-11-11 15:02:29 +00:00
Kegan Dougal 42c43cfafd Use ObservableDeferreds instead of Deferreds as they behave as intended 2016-11-11 14:54:10 +00:00
Kegan Dougal c7daf3136c Use observable deferreds because they are sane 2016-11-11 14:13:32 +00:00
Kegan Dougal 8a8ad46f48 Flake8 2016-11-10 15:22:11 +00:00
Kegan Dougal 2771447c29 Store Promise<Response> instead of Response for HTTP API transactions
This fixes a race whereby:
 - User hits an endpoint.
 - No cached transaction so executes main code.
 - User hits same endpoint.
 - No cache transaction so executes main code.
 - Main code finishes executing and caches response and returns.
 - Main code finishes executing and caches response and returns.

 This race is common in the wild when Synapse is struggling under load.
 This commit fixes the race by:
  - User hits an endpoint.
  - Caches the promise to execute the main code and executes main code.
  - User hits same endpoint.
  - Yields on the same promise as the first request.
  - Main code finishes executing and returns, unblocking both requests.
2016-11-10 14:49:26 +00:00
Mark Haines b1c27975d0 Set CORs headers on responses from the media repo 2016-11-02 11:29:25 +00:00
Erik Johnston 78c083f159 Merge pull request #1164 from pik/error-codes
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
pik e8b1d2a452 Refactor test_filter to use real DataStore
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Alexander Maznev d9350b0db8 Error codes for filters
* add tests

Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
Richard van der Hoff f7aed3d7a2 Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete
User-interactive auth on delete device
2016-10-13 09:38:41 +01:00
Richard van der Hoff 9009143fb9 Handle delete device requests with no body
We should probably return a 401 rather than a 400 for existing clients that
don't know they have to do the UIA dance to delete a device.
2016-10-12 18:47:28 +01:00
Richard van der Hoff fbd3866bc6 User-interactive auth on delete device 2016-10-12 16:16:31 +01:00
Mark Haines 9e18e0b1cb Merge pull request #1167 from matrix-org/markjh/fingerprints
Add config option for adding additional TLS fingerprints
2016-10-12 15:27:44 +01:00
Mark Haines 0af6213019 Improve comment formatting 2016-10-12 14:45:13 +01:00
Mark Haines 6e9f3ab415 Add config option for adding additional TLS fingerprints 2016-10-11 19:14:46 +01:00
Erik Johnston a2f2516199 Merge pull request #1157 from Rugvip/nolimit
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
Patrik Oldsberg 7b5546d077 rest/client/v1/register: use the correct requester in createUser
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
Richard van der Hoff f382117852 window.postmessage for Interactive Auth fallback
If you're a webapp running the fallback in an iframe, you can't set set a
window.onAuthDone function. Let's post a message back to window.opener instead.
2016-10-06 18:16:59 +01:00
Patrik Oldsberg 9bfc617791 storage/appservice: make appservice methods only relying on the cache synchronous 2016-10-06 15:24:59 +02:00
Erik Johnston 22578545a0 Time out typing over federation 2016-09-23 14:00:52 +01:00
Erik Johnston 87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston ddc89df89d Enable guest access to POST /publicRooms 2016-09-17 15:55:24 +01:00
Erik Johnston 2e67cabd7f Make POST /publicRooms require auth 2016-09-16 11:32:51 +01:00
Erik Johnston c33b25fd8d Change the way we calculate new_limit in /publicRooms and add POST API 2016-09-15 17:35:20 +01:00
Erik Johnston 68c1ed4d1a Remove default public rooms limit 2016-09-15 13:56:20 +01:00
Erik Johnston 1d98cf26be By default limit /publicRooms to 100 entries 2016-09-15 13:18:35 +01:00
Erik Johnston 5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Erik Johnston f3eead0660 Allow paginating both forwards and backwards 2016-09-15 10:15:37 +01:00
Erik Johnston 4131381123 Remove support for aggregate room lists 2016-09-15 09:28:15 +01:00
Mark Haines 18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines 949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Shell Turner 29205e9596 Conform better to the CAS protocol specification
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.

Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
Mark Haines dbff7e9436 Merge pull request #1096 from matrix-org/markjh/get_access_token
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
Mark Haines 8aee5aa068 Add helper function for getting access_tokens from requests
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
Paul Evans 56f38d1776 Merge pull request #1091 from matrix-org/paul/third-party-lookup
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
Paul "LeoNerd" Evans d271383e63 Filter returned events for client-facing format 2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans 0fc0a3bdff Allow clients to specify the format a room state event is returned in 2016-09-09 14:34:29 +01:00
Paul "LeoNerd" Evans bdbcfc2a80 appease pep8 2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans 6eb0c8a2e4 Python isn't JavaScript; have to quote dict keys 2016-09-09 13:31:17 +01:00
Paul "LeoNerd" Evans 25eb769b26 Efficiency fix for lookups of a single protocol 2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans 3328428d05 Allow lookup of a single 3PE protocol query metadata 2016-09-09 13:19:04 +01:00
Matthew Hodgson b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Erik Johnston 257025ac89 Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
Erik Johnston 3f9889bfd6 Use parse_string 2016-09-08 15:51:10 +01:00
Mark Haines fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
David Baker 4ef222ab61 Implement `only=highlight` on `/notifications` 2016-09-08 13:43:35 +01:00
Erik Johnston 791658b576 Add server param to /publicRooms 2016-09-08 11:53:05 +01:00
Mark Haines d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Erik Johnston d51b8a1674 Add quotes and be explicity about script-src 2016-09-05 17:35:01 +01:00
Erik Johnston 662b031a30 Allow PDF to be rendered from media repo 2016-09-05 17:25:26 +01:00
Mark Haines 7ed5acacf4 Fix up the calls to the notifier for device messages 2016-09-01 18:08:40 +01:00
Mark Haines 1aa3e1d287 Add a replication stream for direct to device messages 2016-08-31 10:38:58 +01:00
Mark Haines 4bbef62124 Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device 2016-08-26 14:35:31 +01:00
Paul "LeoNerd" Evans 1294d4a329 Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant 2016-08-25 18:34:47 +01:00
Mark Haines ab34fdecb7 Merge branch 'develop' into markjh/direct_to_device 2016-08-25 18:34:46 +01:00
Mark Haines b162cb2e41 Add some TODOs 2016-08-25 18:18:53 +01:00
Mark Haines 641efb6a39 Fix the deduplication of incoming direct-to-device messages 2016-08-25 18:14:02 +01:00
Mark Haines e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Paul Evans d9dcb2ba3a Merge pull request #1041 from matrix-org/paul/third-party-lookup
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans 8e1ed09dff Move static knowledge of protocol metadata into AS handler; cache the result 2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans 965f33c901 Declare 'gitter' known protocol, with user lookup 2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans 9899824b85 Initial hack at the 3PN protocols metadata lookup API 2016-08-24 12:33:01 +01:00
Erik Johnston 9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans 63c19e1df9 Move 3PU/3PL lookup APIs into /thirdparty containing entity 2016-08-24 11:55:57 +01:00
Kegan Dougal 4b31426a02 Pass through user-supplied content in /join/$room_id
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.

If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Matthew Hodgson 6e80c03d45 Merge branch 'develop' into dbkr/notifications_api 2016-08-20 00:16:18 +01:00
David Baker 0acdd0f1ea Use tuple comparison
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
David Baker 602c84cd9c Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans 105ff162d4 Authenticate 3PE lookup requests 2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans 06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans f3afd6ef1a Remove TODO note about request fields being strings - they're always strings 2016-08-18 15:53:01 +01:00
Paul "LeoNerd" Evans d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans 3856582741 Ensure that 3PU lookup request fields actually get passed in 2016-08-18 14:06:02 +01:00
Erik Johnston 0af9e1a637 Set `Content-Security-Policy` on media repo
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
Erik Johnston f90b3d83a3 Add None check to _iterate_over_text 2016-08-17 15:17:17 +01:00
Paul "LeoNerd" Evans e3e3fbc23a Initial empty implementation that just registers an API endpoint handler 2016-08-17 12:46:49 +01:00
Erik Johnston 109a560905 Flake8 2016-08-16 14:57:21 +01:00
Erik Johnston 48b5829aea Fix up preview URL API. Add tests.
This includes:

- Splitting out methods of a class into stand alone functions, to make
  them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston 4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Erik Johnston 866a5320de Dont invoke get_handlers fromClientV1RestServlet
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
David Baker b4ecf0b886 Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-11 14:09:13 +01:00
Will Hunt 2510db3e76 Don't change status_msg on /sync 2016-08-10 12:59:59 +01:00
Erik Johnston fa1ce4d8ad Don't print stack traces when failing to get remote keys 2016-08-10 10:44:37 +01:00
David Baker cd41c6ece2 Merge pull request #995 from matrix-org/rav/clean_up_cas_login
Clean up CAS login code
2016-08-09 10:21:56 +01:00
Richard van der Hoff 65666fedd5 Clean up CAS login code
Remove some apparently unused code.

Clean up parse_cas_response, mostly to catch the exception if the CAS response
isn't valid XML.
2016-08-08 17:17:25 +01:00
Richard van der Hoff 0682ca04b3 Fix CAS login
Attempting to log in with CAS was giving a 500 error.
2016-08-08 17:01:30 +01:00
Erik Johnston 5bcccfde6c Don't include html comments in description 2016-08-05 14:45:11 +01:00
Erik Johnston b5525c76d1 Typo 2016-08-04 16:10:08 +01:00
Erik Johnston e97648c4e2 Test summarization 2016-08-04 16:09:09 +01:00
Erik Johnston b3682df2ca Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix 2016-08-04 15:29:45 +01:00
Richard van der Hoff 530bc862dc Merge branch 'rav/null_default_device_displayname' into develop 2016-08-03 14:30:32 +01:00
Richard van der Hoff 4fec5e57be Default device_display_name to null
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Mark Haines 921f17f938 Merge branch 'develop' into rav/refactor_device_query 2016-08-03 11:12:47 +01:00
Erik Johnston 58c9653c6b Don't infer paragrahs from newlines 2016-08-02 18:50:24 +01:00
Erik Johnston 6b58ade2f0 Comment on why we clone 2016-08-02 18:41:22 +01:00
Erik Johnston 9e66c58ceb Spelling. 2016-08-02 18:37:31 +01:00
Erik Johnston f83f5fbce8 Make it actually compile 2016-08-02 18:32:42 +01:00
Erik Johnston aecaec3e10 Change the way we summarize URLs
Using XPath is slow on some machines (for unknown reasons), so use a
different approach to get a list of text nodes.

Try to generate a summary that respect paragraph and then word
boundaries, adding ellipses when appropriate.
2016-08-02 18:25:53 +01:00
Richard van der Hoff 986615b0b2 Move e2e query logic into a handler 2016-08-01 18:02:07 +01:00
David Baker 271d3e7865 Fix adding emails on registration
Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.
2016-07-29 15:25:24 +01:00
Richard van der Hoff f6f8f81a48 Add r0.1.0 to the "supported versions" list 2016-07-28 10:14:07 +01:00
Richard van der Hoff fda078f995 Add r0.2.0 to the "supported versions" list 2016-07-28 09:14:21 +01:00
Richard van der Hoff ccec25e2c6 key upload tweaks
1. Add v2_alpha URL back in, since things seem to be using it.

2. Don't reject the request if the device_id in the upload request fails to
   match that in the access_token.
2016-07-27 16:41:06 +01:00
Richard van der Hoff d47115ff8b Delete e2e keys on device delete 2016-07-27 12:24:52 +01:00
Richard van der Hoff 2e3d90d67c Make the device id on e2e key upload optional
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Richard van der Hoff eb359eced4 Add `create_requester` function
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Richard van der Hoff 012b4c1913 Implement updating devices
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff 436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
David Baker 2c28e25bda Merge pull request #943 from matrix-org/rav/get_device_api
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
David Baker 1e6e370b76 Merge pull request #942 from matrix-org/rav/fix_register_deviceid
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
Richard van der Hoff 406f7aa0f6 Implement GET /device/{deviceId} 2016-07-21 12:00:29 +01:00
Richard van der Hoff 1a64dffb00 Preserve device_id from first call to /register
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
Richard van der Hoff 7314bf4682 Merge branch 'develop' into rav/get_devices_api
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff b97a1356b1 Register a device_id in the /v2/register flow.
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.

Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
Richard van der Hoff c68518dfbb Merge pull request #933 from matrix-org/rav/type_annotations
Type annotations
2016-07-20 12:26:32 +01:00
David Baker e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston 1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff 3413f1e284 Type annotations
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
Richard van der Hoff 40cbffb2d2 Further registration refactoring
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker b9e997f561 Merge pull request #931 from matrix-org/rav/refactor_register
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff 8f6281ab0c Don't bind email unless threepid contains expected fields 2016-07-19 15:50:01 +01:00
Richard van der Hoff 0da0d0a29d rest/client/v2_alpha/register.py: Refactor flow somewhat.
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.

The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Richard van der Hoff 93efcb8526 Merge pull request #928 from matrix-org/rav/refactor_login
Refactor login flow
2016-07-18 16:12:35 +01:00
Richard van der Hoff dcfd71aa4c Refactor login flow
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.

Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
Will Hunt 511a52afc8 Use body.get to check for 'user' 2016-07-16 18:44:08 +01:00
Will Hunt e885e2a623 Fall back to 'username' if 'user' is not given for appservice reg. 2016-07-16 18:33:48 +01:00
Erik Johnston a3036ac37e Merge pull request #921 from matrix-org/erikj/account_deactivate
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
Erik Johnston a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Erik Johnston 209e04fa11 Merge pull request #918 from negzi/bugfix_for_token_expiry
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
Erik Johnston 848d3bf2e1 Add hs object 2016-07-14 10:25:52 +01:00
Erik Johnston b55c770271 Only accept password auth 2016-07-14 10:00:38 +01:00
Erik Johnston d543b72562 Add an /account/deactivate endpoint 2016-07-14 09:56:53 +01:00
Negar Fazeli 0136a522b1 Bug fix: expire invalid access tokens 2016-07-13 15:00:37 +02:00
David Baker c55ad2e375 be more pythonic 2016-07-12 14:15:10 +01:00
David Baker aaa9d9f0e1 on_OPTIONS isn't neccessary 2016-07-12 14:13:14 +01:00
David Baker 75fa7f6b3c Remove other debug logging 2016-07-12 14:08:57 +01:00
David Baker a5db0026ed Separate out requestTokens to separate handlers 2016-07-11 09:57:07 +01:00
David Baker 9c491366c5 Oops, remove debug logging 2016-07-11 09:07:40 +01:00
David Baker 385aec4010 Implement https://github.com/matrix-org/matrix-doc/pull/346/files 2016-07-08 17:42:48 +01:00
Erik Johnston 67f2c901ea Add rest servlet. Fix SQL. 2016-07-06 15:56:59 +01:00
Erik Johnston a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston 76b18df3d9 Check that there are no null bytes in user and passsword 2016-07-06 11:17:53 +01:00
Erik Johnston 0da24cac8b Add null separator to hmac 2016-07-06 11:05:16 +01:00