Commit Graph

5087 Commits

Author SHA1 Message Date
Matthew Hodgson 6e80c03d45 Merge branch 'develop' into dbkr/notifications_api 2016-08-20 00:16:18 +01:00
Erik Johnston 27e0178da9 Add a top level measure 2016-08-19 18:49:37 +01:00
Erik Johnston 6d1a94d218 Remove redundant measure 2016-08-19 18:40:31 +01:00
Erik Johnston 8731197e54 Only abort Measure on Exceptions 2016-08-19 18:23:45 +01:00
Erik Johnston afbf6b33fc defer.returnValue must not be called within Measure 2016-08-19 18:23:44 +01:00
Erik Johnston 37adde32dc Move defer.returnValue out of Measure 2016-08-19 18:23:44 +01:00
Erik Johnston 04fc8bbcb0 Update keyring Measure 2016-08-19 18:23:44 +01:00
Erik Johnston 39b900b316 Measure http.server render 2016-08-19 18:23:44 +01:00
Erik Johnston 47dd8f02a1 Measure _get_event_from_row 2016-08-19 18:23:44 +01:00
Erik Johnston 2426c2f21a Measure keyrings 2016-08-19 18:23:44 +01:00
Erik Johnston 39242090e3 Add measure blocks to notifier 2016-08-19 18:23:44 +01:00
Erik Johnston e6784daf07 Merge pull request #1030 from matrix-org/erikj/cache_contexts
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston 45fd2c8942 Ensure invalidation list does not grow unboundedly 2016-08-19 16:09:16 +01:00
Erik Johnston c0d7d9d642 Rename to on_invalidate 2016-08-19 15:13:58 +01:00
Erik Johnston dc76a3e909 Make cache_context an explicit option 2016-08-19 15:02:38 +01:00
Erik Johnston f164fd9220 Move _bulk_get_push_rules_for_room to storage layer 2016-08-19 14:29:20 +01:00
Erik Johnston ba214a5e32 Remove lru option 2016-08-19 14:17:11 +01:00
Erik Johnston 4161ff2fc4 Add concept of cache contexts 2016-08-19 14:17:07 +01:00
Erik Johnston b770435389 Make get_new_events_for_appservice use indices 2016-08-19 10:28:42 +01:00
Paul Evans 5674ea3e6c Merge pull request #1026 from matrix-org/paul/thirdpartylookup
3rd party entity lookup
2016-08-18 20:52:50 +01:00
David Baker 1e4217c90c Explicit join 2016-08-18 17:53:44 +01:00
David Baker 0acdd0f1ea Use tuple comparison
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans 65201631a4 Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater 2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans 697872cf08 More warnings about invalid results from AS 3PE query 2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
David Baker 602c84cd9c Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans 2a91799fcc Minor syntax neatenings 2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans fcf1dec809 Appease pep8 2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans 105ff162d4 Authenticate 3PE lookup requests 2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans 06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans f3afd6ef1a Remove TODO note about request fields being strings - they're always strings 2016-08-18 15:53:01 +01:00
Erik Johnston bcbd74dc5b Remove log lines 2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans d7b42afc74 Log a warning if an AS yields an invalid 3PU lookup result 2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans 80f4740c8f Scattergather the call out to ASes; validate received results 2016-08-18 15:40:41 +01:00
Erik Johnston 07229bbdae Add appservice worker 2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans 434bbf2cb5 Filter 3PU lookups by only ASes that declare knowledge of that protocol 2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans 718ffcf8bb Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions 2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans 3856582741 Ensure that 3PU lookup request fields actually get passed in 2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans f0c73a1e7a Extend individual list results into the main return list, don't append 2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans b3511adb92 Don't catch the return-value-as-exception that @defer.inlineCallbacks will use 2016-08-18 13:45:18 +01:00
Erik Johnston 9da84a9a1e Make AppserviceHandler stream events from database
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
2016-08-18 11:54:41 +01:00
Mark Haines 403ecd8a2c Missed a s/federation reader/media repository/ in a log message 2016-08-18 10:26:15 +01:00
Mark Haines 47fbff7a05 Merge pull request #1024 from matrix-org/markjh/media_repository
Add a media repository worker
2016-08-18 10:12:17 +01:00
Mark Haines 396624864a Add a media repository worker 2016-08-18 09:38:42 +01:00
Erik Johnston e73dcb66da Merge pull request #1022 from matrix-org/erikj/as_notify_perf
Make notify_interested_services faster
2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans 3ec10dffd6 Actually make 3PU lookup calls out to ASes 2016-08-18 00:39:09 +01:00
Erik Johnston 732cf72b86 Fix push_display_name_rename schema update 2016-08-17 18:12:21 +01:00
Erik Johnston abcb9aee5b Make push Measure finer grained 2016-08-17 18:00:18 +01:00
Erik Johnston 2a0d8f8219 Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy
Set `Content-Security-Policy` on media repo
2016-08-17 17:22:43 +01:00
Erik Johnston 320dfe523c Make notify_interested_services faster 2016-08-17 17:20:50 +01:00
Erik Johnston 0af9e1a637 Set `Content-Security-Policy` on media repo
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
David Baker 0d7cef0943 Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston f90b3d83a3 Add None check to _iterate_over_text 2016-08-17 15:17:17 +01:00
Matrix f743471380 Change name of metric 2016-08-17 15:05:50 +01:00
Erik Johnston b9e888858c Move Measure block inside loop 2016-08-17 14:52:26 +01:00
Erik Johnston 973d67a033 Merge pull request #1019 from matrix-org/erikj/appservice_clean
Clean up _ServiceQueuer
2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans e3e3fbc23a Initial empty implementation that just registers an API endpoint handler 2016-08-17 12:46:49 +01:00
Erik Johnston 7321f45457 Clean up _ServiceQueuer 2016-08-17 12:03:04 +01:00
Erik Johnston b9abf3e4e3 Remove dead appservice code 2016-08-17 11:48:23 +01:00
Erik Johnston 62c5245c87 Measure notify_interested_services 2016-08-17 11:12:29 +01:00
Erik Johnston 949629291c Do it in storage function 2016-08-16 17:05:34 +01:00
David Baker ad42322257 Add migration script
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
David Baker 0bba2799b6 Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override 2016-08-16 16:46:37 +01:00
Erik Johnston 64a2acb161 Don't update caches replication stream if tokens haven't advanced 2016-08-16 16:44:19 +01:00
David Baker 1594eba29e s/underride/override/ in the rule_id too 2016-08-16 16:44:07 +01:00
Erik Johnston 109a560905 Flake8 2016-08-16 14:57:21 +01:00
Erik Johnston 48b5829aea Fix up preview URL API. Add tests.
This includes:

- Splitting out methods of a class into stand alone functions, to make
  them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston 7c6f4f9427 Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri
Limit number of extremeties in backfill request
2016-08-16 12:55:42 +01:00
Erik Johnston 25c2332071 Merge pull request #1010 from matrix-org/erikj/refactor_deletions
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston 2ee1bd124c Limit number of extremeties in backfill request
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
2016-08-16 11:34:36 +01:00
Erik Johnston a2427981b7 Use cached get_user_by_access_token in slaves 2016-08-16 11:24:32 +01:00
Erik Johnston 6cbd1b495e Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry 2016-08-16 10:50:24 +01:00
David Baker 1c7c317df1 Move display name rule
As per https://github.com/matrix-org/matrix-doc/pull/373 and comment
2016-08-15 18:34:53 +01:00
Erik Johnston dc3a00f24f Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves. 2016-08-15 17:04:39 +01:00
Erik Johnston 75299af4fc Merge pull request #1009 from matrix-org/erikj/event_split
Split out /events to synchrotron
2016-08-15 15:39:05 +01:00
Erik Johnston 89e786bd85 Doc get_next() context manager usage 2016-08-15 13:45:26 +01:00
Erik Johnston d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston 784a2d4f2c Remove broken cache stuff 2016-08-15 11:25:48 +01:00
Erik Johnston 0be963472b Use cached version of get_aliases_for_room 2016-08-15 11:24:12 +01:00
Erik Johnston 64e7e11853 Implement cache replication stream 2016-08-15 11:16:45 +01:00
Erik Johnston 4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Erik Johnston 99bbd90b0d Always run txn.after_callbacks 2016-08-15 09:45:44 +01:00
Benjamin Saunders 8a57cc3123 Add missing database corruption recovery case
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
Daniel Ehlers dfaf0fee31
Log the value which is observed in the first place.
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:49:05 +02:00
Daniel Ehlers e380538b59
Fix AttributeError when bind_dn is not defined.
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:48:33 +02:00
Erik Johnston 4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Erik Johnston 866a5320de Dont invoke get_handlers fromClientV1RestServlet
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
Erik Johnston 448ac6cf0d Only process one local membership event per room at a time 2016-08-12 09:32:19 +01:00
Erik Johnston 832799dbff Merge pull request #997 from Half-Shot/develop
Don't change status_msg on /sync
2016-08-11 14:10:55 +01:00
David Baker b4ecf0b886 Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-11 14:09:13 +01:00
Will Hunt 5b5148b7ec Synced up synchrotron set_state with PresenceHandler set_state 2016-08-11 11:48:30 +01:00
Erik Johnston 1d40373c9d Include prev_content in redacted state events 2016-08-11 10:24:41 +01:00
Erik Johnston c315922b5f PEP8 2016-08-10 16:34:10 +01:00
Erik Johnston ca8abfbf30 Clean up TransactionQueue 2016-08-10 16:24:16 +01:00
Erik Johnston 5aeadb7414 Merge pull request #999 from matrix-org/erikj/measure_more
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
Erik Johnston 487bc49bf8 Don't stop on 4xx series errors 2016-08-10 13:39:12 +01:00
Erik Johnston 739ea29d1e Also check if server is in the room 2016-08-10 13:32:23 +01:00
Erik Johnston ea8c4094db Also pull out rejected events 2016-08-10 13:26:13 +01:00