Commit Graph

10080 Commits

Author SHA1 Message Date
Erik Johnston cd871a3057 Fix storage provider bug introduced when renamed to store_local 2018-01-18 18:37:59 +00:00
Erik Johnston 8ff6726c0d
Merge pull request #2812 from matrix-org/erikj/media_storage_provider_config
Make storage providers configurable
2018-01-18 18:33:57 +00:00
Erik Johnston d69768348f Fix passing wrong config to provider constructor 2018-01-18 17:14:05 +00:00
Erik Johnston 8e85220373 Remove duplicate directory test 2018-01-18 17:12:35 +00:00
Erik Johnston 3fe2bae857 Missing staticmethod 2018-01-18 17:11:45 +00:00
Erik Johnston aae77da73f Fixup comments 2018-01-18 17:11:29 +00:00
Erik Johnston ce4f66133e Add unit tests 2018-01-18 16:32:11 +00:00
Erik Johnston b6dc7044a9
Merge pull request #2804 from matrix-org/erikj/file_consumer
Add decent impl of a FileConsumer
2018-01-18 16:31:33 +00:00
Erik Johnston 9a89dae8c5 Fix typo in thumbnail resource causing access times to be incorrect 2018-01-18 15:06:24 +00:00
Erik Johnston 0af5dc63a8 Make storage providers more configurable 2018-01-18 14:07:21 +00:00
Richard van der Hoff 5a4da21d58
Merge pull request #2810 from matrix-org/rav/metrics_fixes
Fix bugs in block metrics
2018-01-18 13:35:28 +00:00
Richard van der Hoff d57765fc8a Fix bugs in block metrics
... which I introduced in #2785
2018-01-18 12:24:42 +00:00
Erik Johnston 2cf6a7bc20 Use better file consumer 2018-01-18 12:00:46 +00:00
Erik Johnston 4a53f3a3e8 Ensure media is in local cache before thumbnailing 2018-01-18 12:00:46 +00:00
Erik Johnston be0dfcd4a2 Do logcontexts correctly 2018-01-18 11:57:57 +00:00
Erik Johnston 1432f7ccd5 Move test stuff to tests 2018-01-18 11:57:57 +00:00
Erik Johnston 2f18a2647b Make all fields private 2018-01-18 11:57:54 +00:00
Richard van der Hoff d6af5512bb
Merge pull request #2809 from matrix-org/rav/metrics_errors
better exception logging in callbackmetrics
2018-01-18 11:46:37 +00:00
Richard van der Hoff ce236f8ac8 better exception logging in callbackmetrics
when we fail to render a metric, give a clue as to which metric it was
2018-01-18 11:30:49 +00:00
Erik Johnston dc519602ac Ensure we registerProducer isn't called twice 2018-01-18 11:07:17 +00:00
Erik Johnston 17b54389fe Fix _notify_empty typo 2018-01-18 11:05:34 +00:00
Erik Johnston 28b338ed9b Move definition of paused_producer to __init__ 2018-01-18 11:04:41 +00:00
Erik Johnston a177325b49 Fix comments 2018-01-18 11:02:43 +00:00
Richard van der Hoff 36da256cc6
Merge pull request #2805 from matrix-org/rav/log_state_res
Log room when doing state resolution
2018-01-17 18:05:04 +00:00
Richard van der Hoff 1224612a79 Log room when doing state resolution
Mostly because it helps figure out what is prompting the resolution
2018-01-17 17:11:59 +00:00
Erik Johnston bc67e7d260 Add decent impl of a FileConsumer
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.

Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
2018-01-17 16:43:03 +00:00
Erik Johnston a87006f9c7
Merge pull request #2783 from matrix-org/erikj/media_last_accessed
Keep track of last access time for local media
2018-01-17 16:39:02 +00:00
Matthew Hodgson 06db5c4b76
Merge pull request #2803 from matrix-org/matthew/fix-userdir-sql
fix SQL when searching all users
2018-01-17 16:27:54 +00:00
Richard van der Hoff 8716eb4920
Merge pull request #2802 from matrix-org/rav/clean_up_resolve_events
Split resolve_events into two functions
2018-01-17 16:01:33 +00:00
Matthew Hodgson 2d9ab533f9 fix SQL when searching all users 2018-01-17 15:58:52 +00:00
Richard van der Hoff 390093d45e Split resolve_events into two functions
... so that the return type doesn't depend on the arg types
2018-01-17 15:44:31 +00:00
Erik Johnston 2fb3a28c98
Remove lost comment 2018-01-17 14:59:44 +00:00
Richard van der Hoff a7e4ff9cca
Merge pull request #2795 from matrix-org/rav/track_db_scheduling
Track DB scheduling delay per-request
2018-01-17 14:29:37 +00:00
Richard van der Hoff f884cfffb9
Merge pull request #2797 from matrix-org/rav/user_id_checking
Sanity checking for user ids
2018-01-17 14:29:12 +00:00
Richard van der Hoff a5213df1f7 Sanity checking for user ids
Check the user_id passed to a couple of APIs for validity, to avoid
"IndexError: list index out of range" exception which looks scary and results
in a 500 rather than a more useful error.

Fixes #1432, among other things
2018-01-17 14:28:54 +00:00
Richard van der Hoff e8f7541d3f Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling 2018-01-17 14:01:57 +00:00
Richard van der Hoff fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
Richard van der Hoff 1954e867b4
Merge pull request #2796 from matrix-org/rav/fix_closed_connection_errors
Fix 'NoneType' object has no attribute 'writeHeaders'
2018-01-17 13:52:24 +00:00
Richard van der Hoff f23b4078c0
Merge pull request #2794 from matrix-org/rav/rework_run_interaction
rework runInteraction in terms of runConnection
2018-01-17 13:50:42 +00:00
Richard van der Hoff 11ab2f56f5 Merge branch 'develop' into rav/fix_closed_connection_errors 2018-01-17 11:25:11 +00:00
Richard van der Hoff 0486a7814a Merge branch 'develop' into rav/rework_run_interaction 2018-01-17 11:24:38 +00:00
Richard van der Hoff 90c14da992 Merge branch 'develop' into rav/db_txn_time_in_millis 2018-01-17 11:19:55 +00:00
Richard van der Hoff 1067b96364
Merge pull request #2792 from matrix-org/rav/optimise_logging_context
Optimise LoggingContext creation and copying
2018-01-17 11:19:28 +00:00
Richard van der Hoff 38506773eb Merge remote-tracking branch 'origin/develop' into rav/optimise_logging_context 2018-01-17 10:57:13 +00:00
Erik Johnston 300edc2348 Update last access time when thumbnails are viewed 2018-01-17 10:24:43 +00:00
Erik Johnston 05f98a2224 Keep track of last access time for local media 2018-01-17 10:24:43 +00:00
Erik Johnston 3cb2dabaad
Merge pull request #2789 from matrix-org/erikj/fix_thumbnails
Fix thumbnailing remote files
2018-01-17 10:22:20 +00:00
Erik Johnston d728c47142 Add docstring 2018-01-17 10:06:14 +00:00
Richard van der Hoff 936482d507 Fix 'NoneType' object has no attribute 'writeHeaders'
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.

This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
Richard van der Hoff 3d12d97415 Track DB scheduling delay per-request
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00