Commit Graph

10270 Commits

Author SHA1 Message Date
Richard van der Hoff ee6fb4cf85 Remove redundant return value from _calculate_state_delta
we already have the state from _get_new_state_after_events, so returning it
from _calculate_state_delta is just confusing.
2018-02-05 16:23:20 +00:00
Richard van der Hoff 3c7b480ba3 Factor out common code for search insert
we can reuse the same code as is used for event insert, for doing the
background index population.
2018-02-05 16:12:14 +00:00
Erik Johnston 25c0a020f4 Updates tests 2018-02-05 16:01:48 +00:00
Erik Johnston 3fa362502c Update places where we create events 2018-02-05 16:01:48 +00:00
Erik Johnston 5ff3d23564 Split event creation into a separate handler 2018-02-05 16:01:48 +00:00
Richard van der Hoff c46e75d3d8 Move store_event_search_txn to SearchStore
... as a precursor to making event storing and doing the bg update share some
code.
2018-02-05 15:43:22 +00:00
Richard van der Hoff db91e72ade
Merge pull request #2844 from matrix-org/rav/evicted_metrics
montoring metrics for number of cache evictions
2018-02-05 16:34:36 +01:00
Richard van der Hoff bc496df192 report metrics on number of cache evictions 2018-02-05 15:34:01 +00:00
Erik Johnston a1beca0e25 Fix broken unit test for media storage 2018-02-05 12:44:03 +00:00
Erik Johnston b5049d2e5c Add .vscode to gitignore 2018-02-05 12:06:46 +00:00
Erik Johnston 1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor
Ensure media is in local cache before thumbnailing
2018-02-05 11:28:52 +00:00
Richard van der Hoff 80b8a28100 Factor out common code for search insert
we can reuse the same code as is used for event insert, for doing the
background index population.
2018-02-04 00:23:06 +00:00
Richard van der Hoff bd25f9cf36 Clean up work_mem handling
Add some comments and improve exception handling when twiddling work_mem for
the search update
2018-02-03 23:05:41 +00:00
Richard van der Hoff 4eeae7ad65 Move store_event_search_txn to SearchStore
... as a precursor to making event storing and doing the bg update share some
code.
2018-02-03 22:59:45 +00:00
Richard van der Hoff bb9f0f3cdb Merge branch 'develop' into matthew/gin_work_mem 2018-02-03 22:40:28 +00:00
Richard van der Hoff 6b02fc80d1 Reinstate event_search_postgres_gist handler
People may have queued updates for this, so we can't just delete it.
2018-02-02 14:32:51 +00:00
Richard van der Hoff 9c9356512e
Merge pull request #2845 from matrix-org/rav/urlcache_error_handling
Handle url_previews with no content-type
2018-02-02 15:27:52 +01:00
Richard van der Hoff 18eae413af
Merge pull request #2842 from matrix-org/rav/state_resolution_handler
Factor out resolve_state_groups to a separate handler
2018-02-02 15:27:35 +01:00
Richard van der Hoff 78d6ddba86
Merge pull request #2841 from matrix-org/rav/refactor_calc_state_delta
factor _get_new_state_after_events out of _calculate_state_delta
2018-02-02 15:27:15 +01:00
Richard van der Hoff 9dcd667ac2
Merge pull request #2836 from matrix-org/rav/resolve_state_events_docstring
Docstring fixes
2018-02-02 15:26:49 +01:00
Richard van der Hoff 33cac3dc29
Merge pull request #2818 from turt2live/travis/admin-list-media
Add an admin route to get all the media in a room
2018-02-02 11:41:03 +01:00
Travis Ralston 6e87b34f7b
Merge branch 'develop' into travis/admin-list-media 2018-02-01 18:05:47 -07:00
Richard van der Hoff d5352cbba8 Handle url_previews with no content-type
avoid failing with an exception if the remote server doesn't give us a
Content-Type header.

Also, clean up the exception handling a bit.
2018-02-02 00:53:46 +00:00
Richard van der Hoff 14737ba495 doc arg types for _seperate 2018-02-01 12:41:34 +00:00
Richard van der Hoff e15d4ea248 More docstring fixes
Fix a couple of errors in docstrings
2018-02-01 12:41:34 +00:00
Richard van der Hoff a18828c129 Fix docstring for StateHandler.resolve_state_groups
The return type was a complete lie, so fix it
2018-02-01 12:41:34 +00:00
Richard van der Hoff 6da4c4d3bd Factor out resolve_state_groups to a separate handler
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
2018-02-01 12:40:04 +00:00
Richard van der Hoff 0cbda53819 Rename resolve_state_groups -> resolve_state_groups_for_events
(to make way for a method that actually just does the state group resolution)
2018-02-01 12:40:00 +00:00
Richard van der Hoff 77c0629ebc
Merge pull request #2837 from matrix-org/rav/fix_quarantine_media
Fix sql error in quarantine_media
2018-02-01 11:45:58 +01:00
Travis Ralston e16e45b1b4 pep8
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-01-31 15:30:38 -07:00
Richard van der Hoff e1e4ec9f9d factor _get_new_state_after_events out of _calculate_state_delta
This reduces the scope of a bunch of variables
2018-01-31 21:32:09 +00:00
Richard van der Hoff 78e7e05188
Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_fail
Remove spurious log argument
2018-01-31 22:18:46 +01:00
Richard van der Hoff ad48dfe73d
Merge pull request #2835 from matrix-org/rav/remove_event_type_param
Remove unused "event_type" param on state.get_current_state_ids
2018-01-31 22:17:57 +01:00
Richard van der Hoff 518a74586c
Merge pull request #2834 from matrix-org/rav/better_persist_event_exception_handling
Improve exception handling in persist_event
2018-01-31 22:17:38 +01:00
Richard van der Hoff d1fe4db882
Merge pull request #2833 from matrix-org/rav/pusher_hacks
Logging and metrics for the http pusher
2018-01-31 22:16:59 +01:00
Richard van der Hoff 421d68ca8c
Merge pull request #2817 from matrix-org/rav/http_conn_pool
Use a connection pool for the SimpleHttpClient
2018-01-31 22:14:22 +01:00
Richard van der Hoff 326189c25a Script to move remote media to another media store 2018-01-31 18:45:10 +00:00
Travis Ralston 3af53c183a Add admin api documentation for list media endpoint
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-01-31 08:15:59 -07:00
Travis Ralston 63c4383927 Documentation and naming
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-01-31 08:07:52 -07:00
Richard van der Hoff af19f5e9aa Remove spurious log argument
... which would cause scary-looking and unhelpful errors in the log on dns fail
2018-01-30 17:52:03 +00:00
Richard van der Hoff 773f0eed1e Fix sql error in quarantine_media 2018-01-30 15:02:51 +00:00
Richard van der Hoff adfc0c9539 docstring for get_current_state_ids 2018-01-29 17:39:55 +00:00
Richard van der Hoff d413a2ba98 Remove unused "event_type" param on state.get_current_state_ids
this param doesn't seem to be used, and is a bit pointless anyway because it
can easily be replicated by the caller. It is also horrible, because it changes
the return type of the method.
2018-01-29 17:06:57 +00:00
Richard van der Hoff b387ee17b6 Improve exception handling in persist_event
1. use `deferred.errback()` instead of `deferred.errback(e)`, which means that
a Failure object will be constructed using the current exception state,
*including* its stack trace - so the stack trace is saved in the Failure,
leading to better exception reports.

2. Set `consumeErrors=True` on the ObservableDeferred, because we know that
there will always be at least one observer - which avoids a spurious "CRITICAL:
unhandled exception in Deferred" error in the logs
2018-01-29 17:05:33 +00:00
Richard van der Hoff 03dd745fe2 Better logging when pushes fail 2018-01-29 15:49:06 +00:00
Richard van der Hoff e051abd20b add appid/device_display_name to to pusher logging 2018-01-29 15:04:16 +00:00
Richard van der Hoff 02ba118f81 Increase http conn pool size 2018-01-29 14:30:15 +00:00
Richard van der Hoff 4c65b98e4a
Merge pull request #2831 from matrix-org/rav/fix-userdir-search-again
Fix SQL for user search
2018-01-27 22:58:24 +01:00
Richard van der Hoff d1f3490e75 Add tests for user directory search 2018-01-27 17:21:57 +00:00
Richard van der Hoff 46022025ea Fix SQL for user search
fix some syntax errors for user search when search_all_users is enabled

fixes #2801, hopefully
2018-01-27 17:21:57 +00:00