Commit Graph

70 Commits

Author SHA1 Message Date
Erik Johnston 7258d081a5 Fix bug when invalidating destination retry timings 2018-10-02 15:47:57 +01:00
Erik Johnston fdd1a62e8d Add a five minute cache to get_destination_retry_timings
Hopefully helps with #3931
2018-09-21 14:56:12 +01:00
Erik Johnston bbab6ebfd9 Fix up changelog and remove spurious comment 2018-09-19 14:45:14 +01:00
Erik Johnston 392a54128c pep8 2018-09-19 14:37:49 +01:00
Erik Johnston b9158ac2bf Remove get_destination_retry_timings cache
Currently we rely on the master to invalidate this cache promptly.
However, after having moved most federation endpoints off of master this
no longer happens, causing outbound fedeariont to get blackholed.

Fixes #3798
2018-09-19 14:22:57 +01:00
Amber Brown 14e4d4f4bf
Port storage/ to Python 3 (#3725) 2018-08-31 00:19:58 +10:00
Richard van der Hoff 03751a6420 Fix some looping_call calls which were broken in #3604
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.

Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Richard van der Hoff 371da42ae4 Wrap a number of things that run in the background
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Adrian Tschira 4b9d0cde97 add remaining memoryview changes 2018-05-29 17:42:43 +02:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Richard van der Hoff 6cfee09be9 Make __init__ consitstent across Store heirarchy
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Erik Johnston 54fed21c04 Fix tests and flake8 2016-11-22 18:18:31 +00:00
Erik Johnston 90565d015e Invalidate retry cache in both directions 2016-11-22 17:45:44 +00:00
Erik Johnston 760469c812 Continue to clean up received_transactions 2016-11-01 11:42:08 +00:00
Erik Johnston f09d2b692f Removed unused stuff 2016-10-31 17:10:56 +00:00
Erik Johnston 4c3eb14d68 Increase batching of sent transaction inserts
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
Erik Johnston cce957e254 Bump max_entries on get_destination_retry_timings 2016-09-02 14:08:33 +01:00
Erik Johnston ef0cc648cf Clean up old sent transactions 2016-08-31 11:12:02 +01:00
Erik Johnston aad8a1a825 Delete old received_transactions 2016-08-22 16:29:46 +01:00
Mark Haines 87ffd21b29 Fix a couple of bugs in the transaction and keyring code 2016-07-26 19:19:08 +01:00
Erik Johnston d13459636f Pull prev txn from in memory 2016-05-06 11:30:55 +01:00
Erik Johnston 1d275dba69 Don't needlessly enter transaction 2016-05-06 11:25:58 +01:00
Erik Johnston 56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Mark Haines 54172924c8 Load the current id in the IdGenerator constructor
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Mark Haines c0a279e808 Delete the table objects from TransactionStore 2016-01-13 11:15:20 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Erik Johnston 39de87869c Fix bug where assumed dict was namedtuple 2015-11-12 14:47:48 +00:00
Erik Johnston 7ce264ce5f Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should 2015-11-03 16:24:03 +00:00
Mark Haines 78323ccdb3 Remove syutil dependency in favour of smaller single-purpose libraries 2015-08-24 16:17:38 +01:00
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Erik Johnston 0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston 977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston 25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston 0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston 304111afd0 Don't use AUTOINCREMENT, use an in memory version 2015-04-07 12:05:36 +01:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
Erik Johnston 1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
Erik Johnston d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston 278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Mark Haines 74048bdd41 Remove unused import 2015-02-23 18:17:43 +00:00
Paul "LeoNerd" Evans 044d813ef7 Use the @cached decorator to implement the destination_retry_timings cache 2015-02-23 16:04:40 +00:00