Commit Graph

412 Commits

Author SHA1 Message Date
Erik Johnston 13e6262659 Add metrics to pushers 2016-02-08 14:26:45 +00:00
Erik Johnston d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston e70165039c If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed 2016-01-29 16:41:32 +00:00
Erik Johnston 18579534ea Prefill stream change caches 2016-01-29 14:37:59 +00:00
Erik Johnston b18114e19e Merge pull request #536 from matrix-org/erikj/sync
Make /sync "better".
2016-01-29 13:04:51 +00:00
Erik Johnston fb7299800f Directly set self.value 2016-01-29 11:29:14 +00:00
Erik Johnston c046630c33 Remove spurious self.size 2016-01-29 11:17:54 +00:00
Erik Johnston a30364c1f9 Correctly bookkeep the size of TreeCache 2016-01-29 10:44:46 +00:00
Erik Johnston 766526e114 Make TreeCache keep track of its own size. 2016-01-29 10:11:21 +00:00
Erik Johnston 50e18938a9 Reset size on clear 2016-01-29 10:00:45 +00:00
Erik Johnston 3f5dd18bd4 If the same as the earliest key, assume nothing has changed. 2016-01-28 18:11:41 +00:00
Erik Johnston 40431251cb Correctly update _entity_to_key 2016-01-28 18:05:43 +00:00
Erik Johnston 82cf3a8043 Fix inequalities 2016-01-28 17:44:04 +00:00
Erik Johnston 0663c5bd52 Include cache hits with has_entity_changed 2016-01-28 17:27:28 +00:00
Erik Johnston 45cf827c8f Change name and doc has_entity_changed 2016-01-28 16:39:18 +00:00
Erik Johnston 00cb3eb24b Cache tags and account data 2016-01-28 16:37:41 +00:00
Erik Johnston c23a8c7833 Ensure keys to RoomStreamChangeCache are ints 2016-01-28 15:55:26 +00:00
Erik Johnston e1941442d4 Invalidate caches properly. Remove unused arg 2016-01-28 15:02:41 +00:00
Erik Johnston b97f6626b6 Add cache to room stream 2016-01-27 17:33:26 +00:00
David Baker 7cd418d38e Don't add the member functiopn if we're not using treecache 2016-01-22 13:40:37 +00:00
David Baker cd80019eec docs 2016-01-22 12:21:13 +00:00
David Baker d552861346 Revert all the bits changing keys of eeverything that used LRUCaches to tuples 2016-01-22 12:18:14 +00:00
David Baker 10f76dc5da Make LRU cache not default to treecache & add options to use it 2016-01-22 12:10:33 +00:00
David Baker 5b142788d2 Add __contains__ 2016-01-22 11:49:59 +00:00
David Baker eaa836e8ca Docs for treecache 2016-01-22 11:47:22 +00:00
David Baker 8acc5cb60f Add invalidate_many here too 2016-01-22 11:22:32 +00:00
David Baker 330be18ec5 peppate 2016-01-21 19:17:32 +00:00
David Baker f1f8122120 Change LRUCache to be tree-based so we can delete subtrees. 2016-01-21 19:16:25 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Mark Haines d12c00bdc3 Add some docstring explaining the snapshot cache does 2015-12-23 15:18:11 +00:00
Mark Haines 7fa71e3267 Add a unit test for the snapshot cache 2015-12-23 11:48:03 +00:00
Mark Haines 9ac417fa88 Add a cache for initialSync responses that expires after 5 minutes 2015-12-22 18:27:56 +00:00
Mark Haines 6a5ff5f223 Track the time spent in the database per request.
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Mark Haines 3dd1630848 Add a setter for the current log context.
Move the resource tracking inside that setter so that it is easier
to make sure that the resource tracking isn't double counting the
resource usage.
2015-12-07 10:51:18 +00:00
Mark Haines 5231737369 Add comments to explain why we are hardcoding RUSAGE_THREAD 2015-12-04 11:53:38 +00:00
Mark Haines d6059bdd2a Fix warnings 2015-12-04 11:34:23 +00:00
Mark Haines 48a2526d62 Track the cpu used in the main thread by each logging context 2015-12-03 21:03:01 +00:00
Mark Haines a412b9a465 Run the background updates when starting synapse. 2015-11-10 15:50:58 +00:00
Daniel Wagner-Hall 2cebe53545 Exchange 3pid invites for m.room.member invites 2015-11-05 16:43:19 +00:00
Erik Johnston 5bc690408d Merge pull request #340 from matrix-org/erikj/server_retries
Retry dead servers a lot less often
2015-11-05 16:15:50 +00:00
Mark Haines c452dabc3d Remove the LockManager class because it wasn't being used 2015-11-04 14:08:15 +00:00
Erik Johnston eacb068ac2 Retry dead servers a lot less often 2015-11-02 16:56:30 +00:00
Daniel Wagner-Hall 9261ef3a15 Merge pull request #312 from matrix-org/daniel/3pidinvites
Stuff signed data in a standalone object
2015-10-19 15:52:34 +01:00
Mark Haines 9371a35e89 Merge pull request #306 from matrix-org/markjh/unused_methods
Remove some login classes from synapse.
2015-10-16 18:18:41 +01:00
Daniel Wagner-Hall 0e5239ffc3 Stuff signed data in a standalone object
Makes both generating it in sydent, and verifying it here, simpler at
the cost of some repetition
2015-10-16 17:45:48 +01:00
Daniel Wagner-Hall c225d63e9e Add signing host and keyname to signatures 2015-10-16 15:07:56 +01:00
Daniel Wagner-Hall b8dd5b1a2d Verify third party ID server certificates 2015-10-16 14:54:54 +01:00
Daniel Wagner-Hall 643b5fcdc8 Look for keys on the right objects 2015-10-15 13:10:30 +01:00
Daniel Wagner-Hall f38df51e8d Merge branch 'develop' into daniel/3pidinvites 2015-10-15 11:51:55 +01:00
Mark Haines 5338220d3a synapse.util.emailutils was unused 2015-10-15 10:39:33 +01:00
Daniel Wagner-Hall d82c5f7b5c Use more descriptive error code 2015-10-13 18:02:00 +01:00
Daniel Wagner-Hall 0c38e8637f Remove unnecessary class-wrapping 2015-10-13 18:00:38 +01:00
Mark Haines 32d66738b0 Fix pep8 warnings. 2015-10-13 17:18:29 +01:00
Mark Haines 7639c3d9e5 Bounce all deferreds through the reactor to make debugging easier.
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
2015-10-13 17:13:04 +01:00
Daniel Wagner-Hall 17dffef5ec Move event contents into third_party_layout field 2015-10-13 15:48:12 +01:00
Daniel Wagner-Hall 5b3e9713dd Implement third party identifier invites 2015-10-01 17:49:52 +01:00
Mark Haines 372ac60375 synapse/util/__init__.py:unwrap_deferred was unused 2015-09-22 18:16:07 +01:00
Mark Haines e85c7873dc Allow non-ascii filenames for attachments 2015-08-26 16:26:37 +01:00
Erik Johnston 8e254862f4 Don't assume @cachedList function returns keys for everything 2015-08-18 11:11:33 +01:00
Erik Johnston cfa62007a3 Docstring 2015-08-12 16:42:46 +01:00
Erik Johnston d7451e0f22 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-12 10:30:30 +01:00
Erik Johnston 4807616e16 Wire up the dictionarycache to the metrics 2015-08-12 10:13:35 +01:00
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Erik Johnston dcefac3b06 Comments 2015-08-10 14:16:24 +01:00
Erik Johnston f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston 7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston 07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
Erik Johnston c67ba143fa Move DictionaryCache 2015-08-04 15:58:28 +01:00
Erik Johnston 9beaedd164 Enforce ascii filenames for uploads 2015-06-30 10:31:59 +01:00
Erik Johnston 7fa1363fb0 Merge pull request #192 from matrix-org/erikj/fix_log_context
Fix log context when sending requests
2015-06-19 16:21:40 +01:00
Erik Johnston a68abc79fd Add comment on cancellation of observers 2015-06-19 11:48:55 +01:00
Erik Johnston 653533a3da Fix log context when sending requests 2015-06-19 11:46:49 +01:00
Erik Johnston 22049ea700 Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races. 2015-06-18 15:49:24 +01:00
Erik Johnston dd74436ffd Unused import 2015-05-28 15:47:20 +01:00
Erik Johnston 31ade3b3e9 Remove a deep copy 2015-05-28 13:45:23 +01:00
Erik Johnston 4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Mark Haines d61ce3f670 Add a cache for get_current_state with state_key 2015-05-21 16:41:39 +01:00
Erik Johnston 7cd6a6f6cf Awful idea for speeding up fetching of events 2015-05-14 15:34:02 +01:00
Erik Johnston cdb3757942 Refactor _get_events 2015-05-14 13:31:55 +01:00
Mark Haines ec07dba29e Merge pull request #143 from matrix-org/erikj/SYN-375
SYN-375 - Lots of unhandled deferred exceptions.
2015-05-12 15:25:54 +01:00
Erik Johnston 8022b27fc2 Make distributer.fire work as it did 2015-05-12 13:14:48 +01:00
Erik Johnston 95dedb866f Unwrap defer.gatherResults failures 2015-05-12 13:14:29 +01:00
Erik Johnston 2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston da1aa07db5 Add some docs 2015-05-08 16:52:49 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston fca28d243e Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors' 2015-05-08 16:28:08 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston 1c82fbd2eb Implement create_observer.
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
2015-04-27 13:59:37 +01:00
Erik Johnston 806f380a8b Make LruCache thread safe, as its used for event cache 2015-04-15 16:07:59 +01:00
Erik Johnston a43b40449b Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 16:46:56 +01:00
Erik Johnston 9cef051ce2 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-04-08 16:43:17 +01:00
Erik Johnston ccda401dbf SYN-338: Fix typo that caused the cache to throw an exception in some instances 2015-04-08 16:34:23 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Paul "LeoNerd" Evans d6b3ea75d4 Implement the 'key in dict' test for LruCache() 2015-03-25 19:04:59 +00:00
Erik Johnston ed4d44d833 Merge pull request #109 from matrix-org/default_registration
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Erik Johnston 69135f59aa Implement registering with shared secret. 2015-03-13 15:23:37 +00:00
Paul "LeoNerd" Evans e1a7e3564f Delete a couple of TODO markers of monitoring stats now done 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 027fd1242c Give LruCache a __len__, so that len(cache) works 2015-03-04 17:32:28 +00:00
Erik Johnston 3077cb2915 Use contextlib.contextmanager instead of a custom class 2015-03-02 13:32:44 +00:00
Erik Johnston 0554d07082 Move federation rate limiting out of transport layer 2015-02-27 15:41:52 +00:00
Erik Johnston cc3d3babb0 Remove unused import 2015-02-18 12:01:41 +00:00
Erik Johnston 36e144091b Remove spurious comma. Remove temp run_on_reactor 2015-02-18 11:25:20 +00:00
Erik Johnston b17bd31da0 Temporarily add a run_on_reactor() call 2015-02-18 11:17:26 +00:00
Erik Johnston 859fbd4423 s/self._clock/self.clock/ 2015-02-18 10:39:14 +00:00
Erik Johnston 1be67eca8a Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop 2015-02-18 10:34:40 +00:00
Erik Johnston fe672a04f7 Merge pull request #77 from matrix-org/failures
Failures
2015-02-18 10:29:29 +00:00
Erik Johnston 4fd176a41d More docs 2015-02-18 10:11:24 +00:00
Erik Johnston d77912ff44 Docs. 2015-02-18 10:09:54 +00:00
Erik Johnston 9371019133 Try to only back off if we think we failed to connect to the remote 2015-02-17 18:13:34 +00:00
Erik Johnston c8436b38a0 Only update destination_retry_timings if we have succeeded when retrying 2015-02-17 17:38:38 +00:00
Erik Johnston f91263b1e0 Remove spurious self 2015-02-17 17:37:51 +00:00
Erik Johnston 2b8f1a956c Add per server retry limiting.
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
Erik Johnston 964bb43fbe Fix typo in function name 2015-02-17 15:44:41 +00:00
Erik Johnston e7e20417ca ExpiringCache: purge every 1/2 interval 2015-02-17 15:44:26 +00:00
Erik Johnston 472734a8cc Consume errors in time_bound_deferred 2015-02-17 15:13:50 +00:00
Erik Johnston 2674aeb96a Factor out ExpiringCache from StateHandler 2015-02-16 16:16:47 +00:00
Erik Johnston c52e8d395b Merge pull request #61 from matrix-org/timeout-federation-requests
Timeout federation requests
2015-02-11 17:10:33 +00:00
Erik Johnston 42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Erik Johnston ef276e8770 Fix so timing out connections to actually work. 2015-02-11 16:48:05 +00:00
Erik Johnston 41a9a76a99 Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 16:12:59 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Mark Haines d8324d5a2b Add a lru cache class 2015-02-11 14:52:23 +00:00
Erik Johnston fb233dc40b Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 10:33:19 +00:00
Erik Johnston dcf52469e8 Move time_bound_deferred into Clock 2015-02-11 10:25:06 +00:00
Erik Johnston 0e6b3e4e40 Time out HTTP federation requests 2015-02-10 18:17:27 +00:00
Erik Johnston ef995e6946 Add looping_call to Clock 2015-02-09 14:47:59 +00:00
Erik Johnston dfa05f0cd6 Optimize FrozenEvent creation 2015-01-06 18:51:03 +00:00
Erik Johnston d5ae67e67d Fix typo where we used wrong var. 2015-01-06 16:05:01 +00:00
Erik Johnston 03a501456c Time how long calls to _get_destination_retry_timings take 2015-01-06 15:22:28 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Mark Haines 041ac476a5 Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them 2014-12-18 18:47:13 +00:00
Mark Haines dbe77ec79a Replace distributor deferred list, with a simple for loop until I understand why the former breaks and the latter doesn't 2014-12-18 17:47:00 +00:00
Erik Johnston 5a465b67ba Fix pyflakes 2014-12-16 13:41:43 +00:00
Erik Johnston 58168498b0 Remove FrozenEncoder 2014-12-16 13:38:38 +00:00
Erik Johnston bc6564bac0 Add PEP8 newlines 2014-12-09 11:01:44 +00:00
Erik Johnston 721482c83e Add forgotten file 2014-12-08 09:10:12 +00:00
Mark Haines 15099fade5 Drop log level for incorrect logging contexts to WARN if the context is wrong and DEBUG if the context is missing 2014-11-25 10:57:31 +00:00
Mark Haines db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
Mark Haines 32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
Mark Haines ca91bb2f7f Sometimes there isn't a current logging context 2014-11-19 17:18:55 +00:00
Mark Haines 97c7c34f6f Preserve logging context in a few more places, drop the logging context after it has been stashed to reduce potential for confusion 2014-11-19 16:37:43 +00:00
Mark Haines e903c941cb Merge branch 'develop' into request_logging
Conflicts:
	setup.py
	synapse/storage/_base.py
	synapse/util/async.py
2014-11-14 11:16:50 +00:00
Erik Johnston 4317c8e583 Implement new replace_state and changed prev_state
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
Erik Johnston ef9c4476a0 Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-10-30 11:18:28 +00:00
Mark Haines 7c06399512 Merge branch 'develop' into request_logging
Conflicts:
	synapse/config/logger.py
2014-10-30 11:13:58 +00:00
Mark Haines 7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Mark Haines fa955cc2a4 Pep8 and a few doc strings 2014-10-30 10:13:46 +00:00
Mark Haines b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00
Erik Johnston 53216a500d Add a run_on_reactor function 2014-10-29 17:02:22 +00:00
Kegan Dougal cc83b06cd1 Added support for the HS to send emails. Use it to send password resets. Added email_smtp_server and email_from_address config args. Added emailutils. 2014-09-16 12:36:39 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans 967b45bc1a Allow optional non-suppression of exceptions through the Distributor 2014-09-03 10:40:21 +01:00
Mark Haines c6eafdfbaf Add copyright notices and fix pyflakes errors 2014-09-03 09:43:11 +01:00
Erik Johnston 54d0a75573 Merge branch 'develop' of github.com:matrix-org/synapse into presence_logging
Conflicts:
	synapse/handlers/presence.py
2014-08-28 16:52:46 +01:00
Mark Haines bddc1d9fff use @wraps to set the __name__ __module__ and __doc__ correctly for logged functions 2014-08-28 15:32:38 +01:00
Erik Johnston b8b52ca09d Add logging to try and figure out what is going on with the presence stuff 2014-08-28 14:58:51 +01:00
Paul "LeoNerd" Evans 657ab9ba9d Put some DEBUG logging in lockutils.py so we can debug roomlocks 2014-08-14 16:06:05 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00