754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
4ee4450d66
fix recursion error
2018-05-24 21:44:10 +02:00
dd068ca979
remaining isintance fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:55:08 +02:00
36501068d8
Merge pull request #3247 from NotAFile/py3-misc
...
Misc Python3 fixes
2018-05-24 12:58:37 -05:00
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter
...
Add batch_iter to utils
2018-05-24 12:54:12 -05:00
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
7948ecf234
Comment
2018-05-22 11:39:43 +01:00
020377a550
Fix logcontext resource usage tracking
2018-05-22 11:16:07 +01:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
45b55e23d3
Add batch_iter to utils
...
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
73cbdef5f7
fix py3 intern and remove unnecessary py3 encode
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:31 +02:00
093d8c415a
Merge remote-tracking branch 'origin/develop' into rav/warn_on_logcontext_fail
2018-05-03 14:59:29 +01:00
a7fe62f0cb
Fix logcontext leaks in rate limiter
2018-05-03 12:31:59 +01:00
415c6b672e
Merge branch 'develop' into rav/more_logcontext_leaks
2018-05-02 16:16:01 +01:00
f22e7cda2c
Fix a class of logcontext leaks
...
So, it turns out that if you have a first `Deferred` `D1`, you can add a
callback which returns another `Deferred` `D2`, and `D2` must then complete
before any further callbacks on `D1` will execute (and later callbacks on `D1`
get the *result* of `D2` rather than `D2` itself).
So, `D1` might have `called=True` (as in, it has started running its
callbacks), but any new callbacks added to `D1` won't get run until `D2`
completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield`
will 'block'.
In conclusion: some of our assumptions in `logcontext` were invalid. We need to
make sure that we don't optimise out the logcontext juggling when this
situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
2018-05-02 11:58:00 +01:00
e482f8cd85
Fix incorrect reference to StringIO
...
This was introduced in 4f2f5171
2018-05-02 09:12:26 +01:00
fdb6849b81
Merge pull request #3144 from matrix-org/rav/run_in_background_exception_handling
...
Trap exceptions thrown within run_in_background
2018-04-30 10:23:02 +01:00
db75c86e84
Merge branch 'develop' into py3-xrange-1
2018-04-30 01:02:25 +01:00
049b0b5af2
Merge pull request #3154 from NotAFile/py3-stringio
...
Replace stringIO imports with six
2018-04-30 00:59:04 +01:00
dbf6f28d64
Merge pull request #3155 from NotAFile/py3-bytes-1
...
more bytes strings
2018-04-30 00:38:21 +01:00
aab2e4da60
Merge pull request #3140 from matrix-org/rav/use_run_in_background
...
Use run_in_background in preference to preserve_fn
2018-04-30 00:34:28 +01:00
e9143b6593
more bytes strings
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-29 00:13:57 +02:00
d82b6ea9e6
Move more xrange to six
...
plus a bonus next()
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:57:00 +02:00
4f2f5171b7
replace stringIO imports
2018-04-28 13:46:23 +02:00
fc149b4eeb
Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background
2018-04-27 14:31:23 +01:00
6146332387
Merge remote-tracking branch 'origin/develop' into rav/deferred_timeout
2018-04-27 14:18:00 +01:00
2a13af23bc
Use run_in_background in preference to preserve_fn
...
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
9d2c1b8429
Backport deferred.addTimeout
...
Twisted 16.0 doesn't have addTimeout, so let's backport it.
2018-04-27 12:52:30 +01:00
13843f771e
Trap exceptions thrown within run_in_background
...
Turn any exceptions that get thrown synchronously within run_in_background into
Failures instead.
2018-04-27 12:17:13 +01:00
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
1ea904b9f0
Use deferred.addTimeout instead of time_bound_deferred
...
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
8dc4a6144b
Merge pull request #3107 from NotAFile/py3-bool-nonzero
...
add __bool__ alias to __nonzero__ methods
2018-04-20 15:43:39 +01:00
c09a6daf09
Merge pull request #3110 from NotAFile/py3-six-queue
...
Replace Queue with six.moves.queue
2018-04-20 15:35:00 +01:00
11a67b7c9d
Merge pull request #3093 from matrix-org/rav/response_cache_wrap
...
Refactor ResponseCache usage
2018-04-20 11:31:17 +01:00
878995e660
Replace Queue with six.moves.queue
...
and a six.range change which I missed the last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-16 00:46:21 +02:00
f63ff73c7f
add __bool__ alias to __nonzero__ methods
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
d3347ad485
Revert "Use sortedcontainers instead of blist"
...
This reverts commit 9fbe70a7dc
2018-04-13 11:16:43 +01:00
60f6014bb7
ResponseCache: fix handling of completed results
...
Turns out that ObservableDeferred.observe doesn't return a deferred if the
result is already completed. Fix handling and improve documentation.
2018-04-13 07:32:29 +01:00
b78395b7fe
Refactor ResponseCache usage
...
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.
This will be largely non-functional, but does include the following functional
changes:
* federation_server.on_context_state_request: drops use of _server_linearizer
which looked redundant and could cause incorrect cache misses by yielding
between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
on production.
2018-04-12 13:02:15 +01:00
d5c74b9f6c
Merge pull request #3092 from matrix-org/rav/response_cache_metrics
...
Add metrics for ResponseCache
2018-04-12 12:59:36 +01:00
261124396e
Merge pull request #3059 from matrix-org/rav/doc_response_cache
...
Document the behaviour of ResponseCache
2018-04-12 11:22:30 +01:00
b3384232a0
Add metrics for ResponseCache
2018-04-10 23:14:47 +01:00
9fbe70a7dc
Use sortedcontainers instead of blist
...
This commit drop-in replaces blist with SortedContainers. They are
written in pure python so work with pypy, but perform as good as
native implementations, at least in a couple benchmarks:
http://www.grantjenks.com/docs/sortedcontainers/performance.html
2018-04-10 11:29:51 +02:00
13decdbf96
Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics"
...
We aren't ready to release this yet, so I'm reverting it for now.
This reverts commit d1679a4ed7e089100c62
2018-04-09 12:59:12 +01:00
3449da3bc7
Merge pull request #3068 from matrix-org/rav/fix_cache_invalidation
...
Improve database cache performance
2018-04-05 17:21:44 +01:00
01afc563c3
Fix overzealous cache invalidation
...
Fixes an issue where a cache invalidation would invalidate *all* pending
entries, rather than just the entry that we intended to invalidate.
2018-04-05 16:24:04 +01:00
518f6de088
Remove redundant metrics which were deprecated in 0.27.0.
2018-04-04 19:46:28 +01:00
a9a74101a4
Document the behaviour of ResponseCache
...
it looks like everything that uses ResponseCache expects to have to
`make_deferred_yieldable` its results. It's debatable whether that is the best
approach, but let's document it for now to avoid further confusion.
2018-04-04 09:06:22 +01:00
05630758f2
Use static JSONEncoders
...
using json.dumps with custom options requires us to create a new JSONEncoder on
each call. It's more efficient to create one upfront and reuse it.
2018-03-29 23:13:33 +01:00
8cbbfaefc1
404 correctly on missing paths via NoResource
...
fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
2018-03-23 10:32:50 +00:00
9a0d783c11
Add comments
2018-03-19 11:35:53 +00:00
5a6e54264d
Make 'unexpected logging context' into warnings
...
I think we've now fixed enough of these that the rest can be logged at
warning.
2018-03-15 18:40:38 +00:00
7c7706f42b
Fix bug where state cache used lots of memory
...
The state cache bases its size on the sum of the size of entries. The
size of the entry is calculated once on insertion, so it is important
that the size of entries does not change.
The DictionaryCache modified the entries size, which caused the state
cache to incorrectly think it was smaller than it actually was.
2018-03-15 15:46:54 +00:00
20f40348d4
Factor run_in_background out from preserve_fn
...
It annoys me that we create temporary function objects when there's really no
need for it. Let's factor the gubbins out of preserve_fn and start using it.
2018-03-08 11:50:11 +00:00
3a75de923b
Rewrite make_deferred_yieldable avoiding inlineCallbacks
...
... because (a) it's actually simpler (b) it might be marginally more
performant?
2018-03-01 12:40:05 +00:00
bc496df192
report metrics on number of cache evictions
2018-02-05 15:34:01 +00:00
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
d84f65255e
Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid
...
add registrations_require_3pid and allow_local_3pids
2018-01-22 13:57:22 +00:00
8fe253f19b
fix PR nitpicking
2018-01-19 18:23:45 +00:00
447f4f0d5f
rewrite based on PR feedback:
...
* [ ] split config options into allowed_local_3pids and registrations_require_3pid
* [ ] simplify and comment logic for picking registration flows
* [ ] fix docstring and move check_3pid_allowed into a new util module
* [ ] use check_3pid_allowed everywhere
@erikjohnston PTAL
2018-01-19 15:33:55 +00:00
b6dc7044a9
Merge pull request #2804 from matrix-org/erikj/file_consumer
...
Add decent impl of a FileConsumer
2018-01-18 16:31:33 +00:00
d57765fc8a
Fix bugs in block metrics
...
... which I introduced in #2785
2018-01-18 12:24:42 +00:00
be0dfcd4a2
Do logcontexts correctly
2018-01-18 11:57:57 +00:00
1432f7ccd5
Move test stuff to tests
2018-01-18 11:57:57 +00:00
2f18a2647b
Make all fields private
2018-01-18 11:57:54 +00:00
dc519602ac
Ensure we registerProducer isn't called twice
2018-01-18 11:07:17 +00:00
17b54389fe
Fix _notify_empty typo
2018-01-18 11:05:34 +00:00
28b338ed9b
Move definition of paused_producer to __init__
2018-01-18 11:04:41 +00:00
a177325b49
Fix comments
2018-01-18 11:02:43 +00:00
bc67e7d260
Add decent impl of a FileConsumer
...
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.
Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
2018-01-17 16:43:03 +00:00
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
6324b65f08
Track db txn time in millisecs
...
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
44a498418c
Optimise LoggingContext creation and copying
...
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
2018-01-16 15:49:42 +00:00
39f4e29d01
Reorganise request and block metrics
...
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.
The following are all deprecated, and replaced with synapse_util_metrics_block_count:
synapse_util_metrics_block_timer:count
synapse_util_metrics_block_ru_utime:count
synapse_util_metrics_block_ru_stime:count
synapse_util_metrics_block_db_txn_count:count
synapse_util_metrics_block_db_txn_duration:count
The following are all deprecated, and replaced with synapse_http_server_response_count:
synapse_http_server_requests
synapse_http_server_response_time:count
synapse_http_server_response_ru_utime:count
synapse_http_server_response_ru_stime:count
synapse_http_server_response_db_txn_count:count
synapse_http_server_response_db_txn_duration:count
The following are renamed (the old metrics are kept for now, but deprecated):
synapse_util_metrics_block_timer:total ->
synapse_util_metrics_block_time_seconds
synapse_util_metrics_block_ru_utime:total ->
synapse_util_metrics_block_ru_utime_seconds
synapse_util_metrics_block_ru_stime:total ->
synapse_util_metrics_block_ru_stime_seconds
synapse_util_metrics_block_db_txn_count:total ->
synapse_util_metrics_block_db_txn_count
synapse_util_metrics_block_db_txn_duration:total ->
synapse_util_metrics_block_db_txn_duration_seconds
synapse_http_server_response_time:total ->
synapse_http_server_response_time_seconds
synapse_http_server_response_ru_utime:total ->
synapse_http_server_response_ru_utime_seconds
synapse_http_server_response_ru_stime:total ->
synapse_http_server_response_ru_stime_seconds
synapse_http_server_response_db_txn_count:total ->
synapse_http_server_response_db_txn_count
synapse_http_server_response_db_txn_duration:total
synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
b2cd6accf5
Remove __PreservingContextDeferred too
2017-11-14 23:00:10 +00:00
7e6fa29cb5
Remove preserve_context_over_{fn, deferred}
...
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
bf993db11c
Logging and logcontext fixes for Limiter
...
Add some logging to the Limiter in a similar spirit to the Linearizer, to help
debug issues.
Also fix a logcontext leak.
Also refactor slightly to avoid throwing exceptions.
2017-11-07 00:48:57 +00:00
0be99858f3
fix vars named `l`
...
E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".
2017-10-23 15:56:38 +01:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
2e9f5ea31a
Fix logcontext handling for persist_events
...
* don't use preserve_context_over_deferred, which is known broken.
* remove a redundant preserve_fn.
* add/improve some comments
2017-10-17 10:59:30 +01:00
cc794d60e7
Merge pull request #2532 from matrix-org/rav/fix_linearizer
...
Fix stackoverflow and logcontexts from linearizer
2017-10-11 17:29:32 +01:00
f30c4ed2bc
logformatter: fix AttributeError
...
make sure we have the relevant fields before we try to log them.
2017-10-11 17:26:17 +01:00
4fad8efbfb
Fix stackoverflow and logcontexts from linearizer
...
1. make it not blow out the stack when there are more than 50 things waiting
for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505 .
2. Make it not mess up the log contexts.
2017-10-11 15:05:05 +01:00
3cc852d339
Fancy logformatter to format exceptions better
...
This is a bit of an experimental change at this point; the idea is to see if it
helps us track down where our stack overflows are coming from by logging the
stack when the exception was caught and turned into a Failure. (We'll also need
edf2704420
2017-10-09 17:44:42 +01:00
148428ce76
Fix logcontext handling for concurrently_execute
...
Avoid preserve_context_over_deferred, which is broken.
2017-10-06 22:24:28 +01:00
8ad5f34908
pep8
2017-09-26 19:21:41 +01:00
9fd086e506
unnecessary parens
2017-09-26 17:59:46 +01:00
0b03a97708
Add module_loader.py
2017-09-26 17:56:41 +01:00
495f075b41
Increase default cache factor size.
2017-07-04 09:58:32 +01:00
b5e8d529e6
Define CACHE_SIZE_FACTOR once
2017-07-04 09:56:44 +01:00
c72058bcc6
Use an ExpiringCache for storing registration sessions
...
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
efc2b7db95
Rewrite conditional
2017-06-09 13:35:15 +01:00
eed59dcc1e
Fix has_any_entity_changed
...
Occaisonally has_any_entity_changed would throw the error: "Set changed
size during iteration" when taking the max of the `sorteddict`. While
its uncertain how that happens, its quite inefficient to iterate over
the entire dict anyway so we change to using the more traditional
`bisect_*` functions.
2017-06-09 11:44:01 +01:00
304880d185
Add stream change cache
2017-05-31 15:46:36 +01:00
bd7bb5df71
Pull out if statement from for loop
2017-05-22 15:12:19 +01:00
e3417a06e2
Update list cache to handle one arg case
...
We update the normal cache descriptors to handle caches with a single
argument specially so that the key wasn't a 1-tuple. We need to update
the cache list to be aware of this.
2017-05-22 15:04:42 +01:00
bbfe4e996c
Make get_state_groups_from_groups faster.
...
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
ffad4fe35b
Don't update event cache hit ratio from get_joined_users
...
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
d2d8ed4884
Optimise caches with single key
2017-05-04 14:18:46 +01:00
2e996271fe
Instantiate DeferredTimedOutError correctly
...
Call `super` correctly, so that we correctly initialise the `errcode` field.
Fixes https://github.com/matrix-org/synapse/issues/2179 .
2017-05-02 13:26:17 +01:00
d9aa645f86
Reduce size of joined_user cache
...
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.
We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
efab1dadde
Remove DEBUG_CACHES
2017-04-25 10:54:09 +01:00
119cb9bbcf
Reduce cache size by not storing deferreds
...
Currently the cache descriptors store deferreds rather than raw values,
this is a simple way of triggering only one database hit and sharing the
result if two callers attempt to get the same value.
However, there are a few caches that simply store a mapping from string
to string (or int). These caches can have a large number of entries,
under the assumption that each entry is small. However, the size of a
deferred (specifically the size of ObservableDeferred) is signigicantly
larger than that of the raw value, 2kb vs 32b.
This PR therefore changes the cache descriptors to store the raw values
rather than the deferreds.
As a side effect cached storage function now either return a deferred or
the actual value, as the cached list decriptor already does. This is
fine as we always end up just yield'ing on the returned value
eventually, which handles that case correctly.
2017-04-25 10:23:11 +01:00
d134d0935e
Only intern ascii strings
2017-04-24 14:07:48 +01:00
e2eebf1696
Fix fixme in preserve_fn
...
`preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a
fixme therein.
2017-04-03 15:38:02 +01:00
4d17add8de
Remove unused instance variable
2017-03-31 09:38:27 +01:00
5b5b171f3e
Docs
2017-03-30 17:05:53 +01:00
b282fe7170
Revert log context change
2017-03-30 17:03:59 +01:00
6194a64ae9
Doc new instance variables
2017-03-30 14:19:10 +01:00
014fee93b3
Manually calculate cache key as getcallargs is expensive
...
This is because getcallargs recomputes the getargspec, amongst other
things, which we don't need to do as its already been done
2017-03-30 14:14:46 +01:00
86780a8bc3
Don't convert to deferreds when not necessary
2017-03-30 14:14:36 +01:00
f9b4bb05e0
Fix the logcontext handling in the cache wrappers ( #2077 )
...
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
2017-03-30 13:22:24 +01:00
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
06ce7335e9
Merge pull request #2052 from matrix-org/rav/time_bound_deferred
...
Fix time_bound_deferred to throw the right exception
2017-03-23 22:22:54 +00:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
b88a323ffb
Fix time_bound_deferred to throw the right exception
...
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
2017-03-23 12:07:11 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
19b9366d73
Fix a couple of logcontext leaks
...
Use preserve_fn to correctly manage the logcontexts around things we don't want
to yield on.
2017-03-23 00:17:46 +00:00
95f21c7a66
Fix caching of remote servers' signature keys
...
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
2017-03-22 15:11:30 +00:00
bd08ee7a46
Merge pull request #2026 from matrix-org/rav/logcontext_docs
...
Logcontext docs
2017-03-20 12:05:21 +00:00
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
d2d146a314
Logcontext docs
2017-03-17 23:59:28 +00:00
2abe85d50e
Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join
...
Queue up federation PDUs while a room join is in progress
2017-03-17 11:32:44 +00:00
29ed09e80a
Fix assertion to stop transaction queue getting wedged
...
... and update some docstrings to correctly reflect the types being used.
get_new_device_msgs_for_remote can return a long under some circumstances,
which was being stored in last_device_list_stream_id_by_dest, and was then
upsetting things on the next loop.
2017-03-15 12:16:55 +00:00
b5d1c68beb
Implement reset_context_after_deferred
...
to correctly reset the context when we fire off a deferred we aren't going to
wait for.
2017-03-15 02:21:07 +00:00
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
3545e17f43
Add setdefault key to ExpiringCache
2017-03-10 10:30:49 +00:00
9d0d40fc15
Docs
2017-03-08 19:05:29 +00:00
3edc57296d
Incorrectly copied copyright
...
This file post-dates OM
2017-03-08 19:00:51 +00:00
1c99934b28
pep8
2017-03-08 11:58:20 +00:00
a9e2b9ec16
Add msisdn util file
2017-03-08 11:53:36 +00:00
6b61060b51
Comment
2017-02-02 14:47:15 +00:00
9efcc3f3be
Comment
2017-02-02 13:50:22 +00:00
df4ecff5a9
Correctly raise exceptions for ratelimitng. Ratelimit on 401
2017-02-01 15:42:19 +00:00
fe08db2713
Remove explicit < 400 check as apparently this is confusing
2017-01-31 15:21:32 +00:00
4c0ec15bdc
Comment
2017-01-31 13:53:46 +00:00
85c590105f
Comment
2017-01-31 13:46:38 +00:00
ae7a132f38
Better handle 404 response for federation /send/
2017-01-31 13:40:09 +00:00
c430111d0e
Update LruCache size estimate on clear
2017-01-18 14:55:23 +00:00
380dba1020
Measure metrics of string_cache
2017-01-17 17:04:46 +00:00
37b4c7d8a9
Fix typo in return type
2017-01-17 14:43:32 +00:00
d6c75cb7c2
Rename and comment tree_to_leaves_iterator
2017-01-17 11:47:03 +00:00
1ccd5676e3
Remove needless call to evict()
2017-01-17 11:42:26 +00:00
f85b6ca494
Speed up cache size calculation
...
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
6d00213e80
Use OrderedDict in ExpiringCache
2017-01-16 15:33:22 +00:00
46aebbbcbf
Add support for 'iterable' to ExpiringCache
2017-01-16 14:57:23 +00:00
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
bf5c9706d9
Remove full_twisted_stacktraces option
...
The debug 'full_twisted_stacktraces' flag caused synapse to rewrite
twisted deferreds to always fire the callback on the next reactor tick.
This was to force the deferred to always store the stacktraces on
exceptions, and thus be more likely to have a full stacktrace when it
reaches the final error handlers and gets printed to the logs.
Dynamically rewriting things is generally bad, and in particular this
change violates assumptions of various bits of Twisted. This wouldn't
necessarily be so bad, but it turns out this option has been turned on
on some production servers.
Turning the option can cause e.g. #1778 .
For now, lets just entirely nuke this option.
2017-01-12 10:32:52 +00:00
f477370c0c
Add paranoia exception catch in Linearizer
2017-01-10 14:04:13 +00:00
dd3df11c55
More logging for the linearizer and for get_events
2017-01-05 12:32:47 +00:00
62ce3034f3
s/aquire/acquire/g
2016-12-30 20:04:44 +00:00
0aff09f6c9
Add more useful logging when we block fetching events
2016-12-30 20:00:44 +00:00
d53a80af25
Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
...
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
fbaf868f62
Correctly handle timeout errors
2016-12-09 16:30:29 +00:00
11bfe438a2
Use correct var
2016-11-24 15:26:53 +00:00
aaecffba3a
Correctly handle 500's and 429 on federation
2016-11-24 15:04:49 +00:00
90565d015e
Invalidate retry cache in both directions
2016-11-22 17:45:44 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
d56c39cf24
Use external ldap auth pacakge
2016-11-15 13:03:19 +00:00
3991b4cbdb
Clean transactions based on time. Add HttpTransactionCache tests.
2016-11-14 11:19:24 +00:00
64038b806c
Comments
2016-11-11 10:42:08 +00:00
d073cb7ead
Add Limiter: limit concurrent access to resource
2016-11-10 16:29:51 +00:00
27d3f2e7ab
Explicitly set authentication mode in ldap3
...
This only makes a difference for versions of ldap3 before 1.0, but a)
its best to be explicit and b) there are distributions that package
ancient versions for ldap3 (e.g. debian).
2016-11-08 14:35:25 +00:00
850b103b36
Implement pluggable password auth
...
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
955f34d23e
Change get_pos_of_last_change to return upper bound
2016-09-15 15:12:07 +01:00
cb3edec6af
Use stream_change cache to make get_forward_extremeties_for_room cache more effective
2016-09-15 14:28:13 +01:00
7356d52e73
Fix up push to use get_current_state_ids
2016-08-25 18:35:49 +01:00
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
e65bc7d315
Merge pull request #1031 from matrix-org/erikj/measure_notifier
...
Add more Measure blocks
2016-08-22 12:13:07 +01:00
8731197e54
Only abort Measure on Exceptions
2016-08-19 18:23:45 +01:00
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
3bc9629be5
Measure federation send transaction resources
2016-08-10 10:56:38 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
7b0f6293f2
Merge pull request #940 from matrix-org/erikj/fed_state_cache
...
Cache federation state responses
2016-08-02 15:21:37 +01:00
bf81e38d36
Fix retry utils to check if the exception is a subclass of CME
2016-07-28 10:47:02 +01:00
389c890f14
Don't include name of room for invites in push
...
Avoids insane pushes like, "Bob invited you to invite from Bob"
2016-07-28 10:20:47 +01:00
242c52d607
typo
2016-07-26 10:09:25 +02:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
7335f0adda
Add ReadWriteLock
2016-07-05 15:23:17 +01:00
870c45913e
Use similar naming we use in email notifs for push
...
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
9f1800fba8
Remove registered_users from the distributor.
...
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.
This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
3b096c5f5c
Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop
2016-06-03 12:00:33 +01:00
58a224a651
Pull out update_results_dict
2016-06-03 11:47:07 +01:00
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
e043ede4a2
Small optimisation to CacheListDescriptor
2016-06-03 11:19:22 +01:00
597013caa5
Make cachedList go a bit faster
2016-06-03 11:13:29 +01:00
680f1d9387
catch thinko in presentable names
2016-05-23 22:55:11 +01:00
c7c75e87dc
Docstring
2016-04-29 19:47:35 +01:00
ec9cbe847d
pep8 newline
2016-04-29 10:07:30 +01:00
acded821c4
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 10:05:20 +01:00
ebbabc4986
Handle room invites in email notifs
2016-04-28 11:49:36 +01:00
7b4715bad7
More variable calculation for email notifs
...
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
2016-04-25 18:27:04 +01:00
72e2fafa20
Add a metrics listener and a ssh listener to the pusher
2016-04-25 17:34:25 +01:00
f22f46f4f9
Move the listenTCP call outside the manhole function
2016-04-25 14:59:21 +01:00
290f125a13
Typo
2016-04-25 14:42:59 +01:00
05e49ffbdf
No we don't: it's just the display name
2016-04-22 18:44:17 +01:00
c5b3c6e101
Sort member events
...
So names of people in a room are given in order
2016-04-22 18:33:36 +01:00
e8701e64b9
Implement group-of-people names
2016-04-22 17:28:42 +01:00
5905f36f05
Split out setting up the manhole to a separate file
2016-04-22 17:09:15 +01:00
9e7aa98c22
Split out create_resource_tree to a separate file
2016-04-22 15:40:51 +01:00
c10ed26c30
Flesh out email templating
...
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
2016-04-21 19:19:07 +01:00
eb8619e256
Create log context in Measure if one doesn't exist
2016-04-18 16:08:32 +01:00
cb9c465707
Use SynapseError 504 for Timeout errors
2016-04-15 10:21:32 +01:00
82d7eea7e3
Move the versionstring code out of app.homeserver into util
2016-04-11 14:57:09 +01:00
7e2f971c08
Remove some unused functions ( #711 )
...
* Remove some unused functions
* get_room_events_stream is only used in tests
* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
b9ee5650b0
Move all the wrapper functions for distributor.fire
...
Move the functions inside the distributor and import them
where needed. This reduces duplication and makes it possible
for flake8 to detect when the functions aren't used in a
given file.
2016-04-08 11:01:38 +01:00
95ac3078da
Rename things
2016-04-07 16:07:16 +01:00
ee5aef6c72
Log contexts and squash things together
2016-04-07 15:34:21 +01:00
639cd07d6d
Add comment
2016-04-07 14:24:12 +01:00
af03ecf352
Deduplicate joins
2016-04-07 14:19:02 +01:00
87f2dec8d4
Make the cache objects be per instance rather than being global
2016-04-06 13:08:05 +01:00
3f4eb4c924
Comment
2016-04-01 14:15:27 +01:00
8d73cd502b
Add concurrently_execute function
2016-04-01 14:06:00 +01:00
77cba688ed
Fix typo
2016-03-24 18:02:37 +00:00
191c7bef6b
Deduplicate identical /sync requests
2016-03-24 17:47:31 +00:00
8122ad7bab
Simplify intern_dict
2016-03-23 16:41:54 +00:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
75daede92f
String intern
2016-03-23 14:53:53 +00:00
c4a8cbd15a
Make LruCache use a dedicated _Node class
2016-03-22 16:06:21 +00:00
a547e2df85
Return list, not generator.
2016-03-14 15:30:19 +00:00
239badea9b
Use syntax that works on both py2.7 and py3
2016-03-07 20:13:10 +00:00
374f9b2f07
Limit stream change cache size too
2016-03-01 13:30:15 +00:00
ce2cdced61
Move cache size fiddling to descriptors only. Fix tests
2016-03-01 13:21:46 +00:00
910fc0f28f
Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-01 12:56:39 +00:00
72165e5b77
Reraise exception
2016-03-01 11:00:10 +00:00
ff2d7551c7
Correct cache miss detection
2016-03-01 10:59:17 +00:00
278d6c0527
Report size of ExpiringCache
2016-02-23 16:46:21 +00:00
c77dae7a1a
Change the way we figure out presence updates for small deltas
2016-02-23 14:54:40 +00:00
5614b4dafb
Add presence metrics
2016-02-19 09:50:54 +00:00
b8cdec92c7
WheelTimer: Don't scan list, use index.
2016-02-18 16:33:07 +00:00
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
9e7900da1e
Add wheeltimer impl
2016-02-17 14:29:28 +00:00
e557dc80b8
Merge pull request #566 from matrix-org/erikj/logcontext
...
Don't bother copying records on parent context
2016-02-10 11:41:45 +00:00
4eb8f9ca8a
Remove comment
2016-02-10 11:29:21 +00:00
00c9ad49df
s/parent_context/previous_context/
2016-02-10 11:25:19 +00:00
9777c5f49a
Set parent context on instansiation
2016-02-10 11:23:32 +00:00
82631c5f94
Fix unit tests
2016-02-09 13:50:37 +00:00
9daa4e2a85
Don't create new logging context
2016-02-09 11:06:19 +00:00
3e2fcd67b2
Don't bother copying records on parent context
2016-02-09 10:50:31 +00:00
241b71852e
Fix bug in util.metrics.Measure
2016-02-09 10:28:13 +00:00
97294ef2fd
Create new context when measuring
2016-02-09 10:12:00 +00:00
c486b7b41c
Change logcontext warns to debug
2016-02-09 09:20:38 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
13e6262659
Add metrics to pushers
2016-02-08 14:26:45 +00:00
d4f72a5bfb
Allowing tagging log contexts
2016-02-03 13:52:27 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
e70165039c
If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed
2016-01-29 16:41:32 +00:00
18579534ea
Prefill stream change caches
2016-01-29 14:37:59 +00:00
b18114e19e
Merge pull request #536 from matrix-org/erikj/sync
...
Make /sync "better".
2016-01-29 13:04:51 +00:00
fb7299800f
Directly set self.value
2016-01-29 11:29:14 +00:00
c046630c33
Remove spurious self.size
2016-01-29 11:17:54 +00:00
a30364c1f9
Correctly bookkeep the size of TreeCache
2016-01-29 10:44:46 +00:00
766526e114
Make TreeCache keep track of its own size.
2016-01-29 10:11:21 +00:00
50e18938a9
Reset size on clear
2016-01-29 10:00:45 +00:00
3f5dd18bd4
If the same as the earliest key, assume nothing has changed.
2016-01-28 18:11:41 +00:00
40431251cb
Correctly update _entity_to_key
2016-01-28 18:05:43 +00:00
82cf3a8043
Fix inequalities
2016-01-28 17:44:04 +00:00
0663c5bd52
Include cache hits with has_entity_changed
2016-01-28 17:27:28 +00:00
45cf827c8f
Change name and doc has_entity_changed
2016-01-28 16:39:18 +00:00
00cb3eb24b
Cache tags and account data
2016-01-28 16:37:41 +00:00
c23a8c7833
Ensure keys to RoomStreamChangeCache are ints
2016-01-28 15:55:26 +00:00
e1941442d4
Invalidate caches properly. Remove unused arg
2016-01-28 15:02:41 +00:00
b97f6626b6
Add cache to room stream
2016-01-27 17:33:26 +00:00
7cd418d38e
Don't add the member functiopn if we're not using treecache
2016-01-22 13:40:37 +00:00
cd80019eec
docs
2016-01-22 12:21:13 +00:00
d552861346
Revert all the bits changing keys of eeverything that used LRUCaches to tuples
2016-01-22 12:18:14 +00:00
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
5b142788d2
Add __contains__
2016-01-22 11:49:59 +00:00
eaa836e8ca
Docs for treecache
2016-01-22 11:47:22 +00:00
8acc5cb60f
Add invalidate_many here too
2016-01-22 11:22:32 +00:00
330be18ec5
peppate
2016-01-21 19:17:32 +00:00
f1f8122120
Change LRUCache to be tree-based so we can delete subtrees.
2016-01-21 19:16:25 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
d12c00bdc3
Add some docstring explaining the snapshot cache does
2015-12-23 15:18:11 +00:00
7fa71e3267
Add a unit test for the snapshot cache
2015-12-23 11:48:03 +00:00
9ac417fa88
Add a cache for initialSync responses that expires after 5 minutes
2015-12-22 18:27:56 +00:00
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
3dd1630848
Add a setter for the current log context.
...
Move the resource tracking inside that setter so that it is easier
to make sure that the resource tracking isn't double counting the
resource usage.
2015-12-07 10:51:18 +00:00
5231737369
Add comments to explain why we are hardcoding RUSAGE_THREAD
2015-12-04 11:53:38 +00:00
d6059bdd2a
Fix warnings
2015-12-04 11:34:23 +00:00
48a2526d62
Track the cpu used in the main thread by each logging context
2015-12-03 21:03:01 +00:00
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
2cebe53545
Exchange 3pid invites for m.room.member invites
2015-11-05 16:43:19 +00:00
5bc690408d
Merge pull request #340 from matrix-org/erikj/server_retries
...
Retry dead servers a lot less often
2015-11-05 16:15:50 +00:00
c452dabc3d
Remove the LockManager class because it wasn't being used
2015-11-04 14:08:15 +00:00
eacb068ac2
Retry dead servers a lot less often
2015-11-02 16:56:30 +00:00
9261ef3a15
Merge pull request #312 from matrix-org/daniel/3pidinvites
...
Stuff signed data in a standalone object
2015-10-19 15:52:34 +01:00
9371a35e89
Merge pull request #306 from matrix-org/markjh/unused_methods
...
Remove some login classes from synapse.
2015-10-16 18:18:41 +01:00
0e5239ffc3
Stuff signed data in a standalone object
...
Makes both generating it in sydent, and verifying it here, simpler at
the cost of some repetition
2015-10-16 17:45:48 +01:00
c225d63e9e
Add signing host and keyname to signatures
2015-10-16 15:07:56 +01:00
b8dd5b1a2d
Verify third party ID server certificates
2015-10-16 14:54:54 +01:00
643b5fcdc8
Look for keys on the right objects
2015-10-15 13:10:30 +01:00
f38df51e8d
Merge branch 'develop' into daniel/3pidinvites
2015-10-15 11:51:55 +01:00
5338220d3a
synapse.util.emailutils was unused
2015-10-15 10:39:33 +01:00
d82c5f7b5c
Use more descriptive error code
2015-10-13 18:02:00 +01:00
0c38e8637f
Remove unnecessary class-wrapping
2015-10-13 18:00:38 +01:00
32d66738b0
Fix pep8 warnings.
2015-10-13 17:18:29 +01:00
7639c3d9e5
Bounce all deferreds through the reactor to make debugging easier.
...
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
2015-10-13 17:13:04 +01:00
17dffef5ec
Move event contents into third_party_layout field
2015-10-13 15:48:12 +01:00
5b3e9713dd
Implement third party identifier invites
2015-10-01 17:49:52 +01:00
372ac60375
synapse/util/__init__.py:unwrap_deferred was unused
2015-09-22 18:16:07 +01:00
e85c7873dc
Allow non-ascii filenames for attachments
2015-08-26 16:26:37 +01:00
8e254862f4
Don't assume @cachedList function returns keys for everything
2015-08-18 11:11:33 +01:00
cfa62007a3
Docstring
2015-08-12 16:42:46 +01:00
d7451e0f22
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-12 10:30:30 +01:00
4807616e16
Wire up the dictionarycache to the metrics
2015-08-12 10:13:35 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
dcefac3b06
Comments
2015-08-10 14:16:24 +01:00
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
9beaedd164
Enforce ascii filenames for uploads
2015-06-30 10:31:59 +01:00
7fa1363fb0
Merge pull request #192 from matrix-org/erikj/fix_log_context
...
Fix log context when sending requests
2015-06-19 16:21:40 +01:00
a68abc79fd
Add comment on cancellation of observers
2015-06-19 11:48:55 +01:00
653533a3da
Fix log context when sending requests
2015-06-19 11:46:49 +01:00
22049ea700
Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races.
2015-06-18 15:49:24 +01:00
dd74436ffd
Unused import
2015-05-28 15:47:20 +01:00
31ade3b3e9
Remove a deep copy
2015-05-28 13:45:23 +01:00
4429e720ae
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-05-22 10:33:00 +01:00
d61ce3f670
Add a cache for get_current_state with state_key
2015-05-21 16:41:39 +01:00
7cd6a6f6cf
Awful idea for speeding up fetching of events
2015-05-14 15:34:02 +01:00
cdb3757942
Refactor _get_events
2015-05-14 13:31:55 +01:00
ec07dba29e
Merge pull request #143 from matrix-org/erikj/SYN-375
...
SYN-375 - Lots of unhandled deferred exceptions.
2015-05-12 15:25:54 +01:00
8022b27fc2
Make distributer.fire work as it did
2015-05-12 13:14:48 +01:00
95dedb866f
Unwrap defer.gatherResults failures
2015-05-12 13:14:29 +01:00
2236ef6c92
Fix up leak. Add warnings.
2015-05-08 19:53:34 +01:00
da1aa07db5
Add some docs
2015-05-08 16:52:49 +01:00
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
fca28d243e
Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors'
2015-05-08 16:28:08 +01:00
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
1c82fbd2eb
Implement create_observer.
...
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
2015-04-27 13:59:37 +01:00
806f380a8b
Make LruCache thread safe, as its used for event cache
2015-04-15 16:07:59 +01:00
a43b40449b
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 16:46:56 +01:00
9cef051ce2
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-04-08 16:43:17 +01:00
ccda401dbf
SYN-338: Fix typo that caused the cache to throw an exception in some instances
2015-04-08 16:34:23 +01:00
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
d6b3ea75d4
Implement the 'key in dict' test for LruCache()
2015-03-25 19:04:59 +00:00
ed4d44d833
Merge pull request #109 from matrix-org/default_registration
...
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
69135f59aa
Implement registering with shared secret.
2015-03-13 15:23:37 +00:00
e1a7e3564f
Delete a couple of TODO markers of monitoring stats now done
2015-03-12 16:24:50 +00:00
027fd1242c
Give LruCache a __len__, so that len(cache) works
2015-03-04 17:32:28 +00:00
3077cb2915
Use contextlib.contextmanager instead of a custom class
2015-03-02 13:32:44 +00:00
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
cc3d3babb0
Remove unused import
2015-02-18 12:01:41 +00:00
36e144091b
Remove spurious comma. Remove temp run_on_reactor
2015-02-18 11:25:20 +00:00
b17bd31da0
Temporarily add a run_on_reactor() call
2015-02-18 11:17:26 +00:00
859fbd4423
s/self._clock/self.clock/
2015-02-18 10:39:14 +00:00
1be67eca8a
Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop
2015-02-18 10:34:40 +00:00
fe672a04f7
Merge pull request #77 from matrix-org/failures
...
Failures
2015-02-18 10:29:29 +00:00
4fd176a41d
More docs
2015-02-18 10:11:24 +00:00
d77912ff44
Docs.
2015-02-18 10:09:54 +00:00
9371019133
Try to only back off if we think we failed to connect to the remote
2015-02-17 18:13:34 +00:00
c8436b38a0
Only update destination_retry_timings if we have succeeded when retrying
2015-02-17 17:38:38 +00:00
f91263b1e0
Remove spurious self
2015-02-17 17:37:51 +00:00
2b8f1a956c
Add per server retry limiting.
...
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
964bb43fbe
Fix typo in function name
2015-02-17 15:44:41 +00:00
e7e20417ca
ExpiringCache: purge every 1/2 interval
2015-02-17 15:44:26 +00:00
472734a8cc
Consume errors in time_bound_deferred
2015-02-17 15:13:50 +00:00
2674aeb96a
Factor out ExpiringCache from StateHandler
2015-02-16 16:16:47 +00:00
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
42bc56dad3
Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson
2015-02-11 17:01:38 +00:00
ef276e8770
Fix so timing out connections to actually work.
2015-02-11 16:48:05 +00:00
41a9a76a99
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 16:12:59 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
d8324d5a2b
Add a lru cache class
2015-02-11 14:52:23 +00:00
fb233dc40b
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 10:33:19 +00:00
dcf52469e8
Move time_bound_deferred into Clock
2015-02-11 10:25:06 +00:00
0e6b3e4e40
Time out HTTP federation requests
2015-02-10 18:17:27 +00:00
ef995e6946
Add looping_call to Clock
2015-02-09 14:47:59 +00:00
dfa05f0cd6
Optimize FrozenEvent creation
2015-01-06 18:51:03 +00:00
d5ae67e67d
Fix typo where we used wrong var.
2015-01-06 16:05:01 +00:00
03a501456c
Time how long calls to _get_destination_retry_timings take
2015-01-06 15:22:28 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
041ac476a5
Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them
2014-12-18 18:47:13 +00:00
dbe77ec79a
Replace distributor deferred list, with a simple for loop until I understand why the former breaks and the latter doesn't
2014-12-18 17:47:00 +00:00
5a465b67ba
Fix pyflakes
2014-12-16 13:41:43 +00:00
58168498b0
Remove FrozenEncoder
2014-12-16 13:38:38 +00:00
bc6564bac0
Add PEP8 newlines
2014-12-09 11:01:44 +00:00
721482c83e
Add forgotten file
2014-12-08 09:10:12 +00:00
15099fade5
Drop log level for incorrect logging contexts to WARN if the context is wrong and DEBUG if the context is missing
2014-11-25 10:57:31 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
32090aee16
Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts
2014-11-20 16:24:00 +00:00
ca91bb2f7f
Sometimes there isn't a current logging context
2014-11-19 17:18:55 +00:00
97c7c34f6f
Preserve logging context in a few more places, drop the logging context after it has been stashed to reduce potential for confusion
2014-11-19 16:37:43 +00:00
e903c941cb
Merge branch 'develop' into request_logging
...
Conflicts:
setup.py
synapse/storage/_base.py
synapse/util/async.py
2014-11-14 11:16:50 +00:00
4317c8e583
Implement new replace_state and changed prev_state
...
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
ef9c4476a0
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
2014-10-30 11:18:28 +00:00
7c06399512
Merge branch 'develop' into request_logging
...
Conflicts:
synapse/config/logger.py
2014-10-30 11:13:58 +00:00
7d709542ca
Fix pep8 warnings
2014-10-30 11:10:17 +00:00
fa955cc2a4
Pep8 and a few doc strings
2014-10-30 10:13:46 +00:00
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
53216a500d
Add a run_on_reactor function
2014-10-29 17:02:22 +00:00
cc83b06cd1
Added support for the HS to send emails. Use it to send password resets. Added email_smtp_server and email_from_address config args. Added emailutils.
2014-09-16 12:36:39 +01:00
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
967b45bc1a
Allow optional non-suppression of exceptions through the Distributor
2014-09-03 10:40:21 +01:00
c6eafdfbaf
Add copyright notices and fix pyflakes errors
2014-09-03 09:43:11 +01:00
54d0a75573
Merge branch 'develop' of github.com:matrix-org/synapse into presence_logging
...
Conflicts:
synapse/handlers/presence.py
2014-08-28 16:52:46 +01:00
bddc1d9fff
use @wraps to set the __name__ __module__ and __doc__ correctly for logged functions
2014-08-28 15:32:38 +01:00
b8b52ca09d
Add logging to try and figure out what is going on with the presence stuff
2014-08-28 14:58:51 +01:00
657ab9ba9d
Put some DEBUG logging in lockutils.py so we can debug roomlocks
2014-08-14 16:06:05 +01:00
f98e6380f1
add in whitespace after copyright statements to improve legibility
2014-08-13 03:14:34 +01:00
4f475c7697
Reference Matrix Home Server
2014-08-12 15:10:52 +01:00
Amber Brown