Commit Graph

40 Commits

Author SHA1 Message Date
Patrick Cloke 2c6a7dfcbf
Use attempt_to_set_autocommit everywhere. (#16615)
To avoid asserting the type of the database connection.
2023-11-09 16:19:42 -05:00
Patrick Cloke 679c691f6f
Remove more usages of cursor_to_dict. (#16551)
Mostly to improve type safety.
2023-10-26 15:12:28 -04:00
Patrick Cloke 9407d5ba78
Convert simple_select_list and simple_select_list_txn to return lists of tuples (#16505)
This should use fewer allocations and improves type hints.
2023-10-26 13:01:36 -04:00
Patrick Cloke a4904dcb04
Convert simple_select_many_batch, simple_select_many_txn to tuples. (#16444) 2023-10-11 13:24:56 -04:00
Patrick Cloke 2763c49eca
Improve comments in StateGroupBackgroundUpdateStore. (#16383) 2023-09-25 18:50:47 +00:00
reivilibre 11e15d79b8
Fix a performance issue introduced in Synapse v1.83.0 which meant that purging rooms was very slow and database-intensive. (#15693)
* Add indices required to efficiently validate new foreign key constraints on stream_ordering

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-05-31 14:59:56 +01:00
Eric Eastwood 703a8f9c67
Instrument `state` and `state_group` storage related things (tracing) (#15610)
Instrument `state` and `state_group` storage related things (tracing) so it's a little more clear where these database transactions are coming from as there is a lot of wires crossing in these functions.

Part of `/messages` performance investigation: https://github.com/matrix-org/synapse/issues/13356
2023-05-19 12:26:58 -05:00
Patrick Cloke 88efc75bab
Include the room ID in more purge room log lines. (#15222) 2023-03-08 20:08:56 +00:00
Shay 1c95ddd09b
Batch up storing state groups when creating new room (#14918) 2023-02-24 13:15:29 -08:00
dependabot[bot] 9bb2eac719
Bump black from 22.12.0 to 23.1.0 (#15103) 2023-02-22 15:29:09 -05:00
David Robertson b5b5f66084
Move `StateFilter` to `synapse.types` (#14668)
* Move `StateFilter` to `synapse.types`

* Changelog
2022-12-12 16:19:30 +00:00
Eric Eastwood 7f78b383ca
Optimize `filter_events_for_client` for faster `/messages` - v2 (#14527)
Fix #14108
2022-11-22 21:56:28 +00:00
reivilibre d3d9ca156e
Cancel the processing of key query requests when they time out. (#13680) 2022-09-07 12:03:32 +01:00
Erik Johnston 0b87eb8e0c
Make DictionaryCache have better expiry properties (#13292) 2022-07-21 17:13:44 +01:00
Erik Johnston de70b25e84
Reduce memory usage of state group cache (#13323) 2022-07-19 14:40:37 +01:00
Erik Johnston 0731e0829c
Don't pull out the full state when storing state (#13274) 2022-07-15 12:59:45 +00:00
reivilibre 66a5f6c400
Add a unique index to `state_group_edges` to prevent duplicates being accidentally introduced and the consequential impact to performance. (#12687) 2022-05-19 14:16:49 +01:00
Shay 19d79b6ebe
Refactor `resolve_state_groups_for_events` to not pull out full state when no state resolution happens. (#12775) 2022-05-18 10:15:52 -07:00
reivilibre c7b2f1ccdc
Back out in-flight state caching changes. (#12126) 2022-03-02 10:37:04 +00:00
reivilibre c893632319
Order in-flight state group queries in biggest-first order (#11610)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-01 13:41:57 +00:00
reivilibre dcb6a37837
Cap the number of in-flight requests for state from a single group (#11608) 2022-02-22 14:24:31 +00:00
reivilibre 284ea2025a
Track and deduplicate in-flight requests to `_get_state_for_groups`. (#10870)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-18 17:23:31 +00:00
Patrick Cloke 3e0536cd2a
Replace uses of simple_insert_many with simple_insert_many_values. (#11742)
This should be (slightly) more efficient and it is simpler
to have a single method for inserting multiple values.
2022-01-13 19:44:18 -05:00
reivilibre 437961744c
Fix remove_stale_pushers job on SQLite. (#10843) 2021-09-20 10:26:13 +01:00
Patrick Cloke 3eba047d38
Add type hints to state database module. (#10823) 2021-09-15 09:54:13 -04:00
Patrick Cloke b7186c6e8d
Add type hints to state handler. (#10482) 2021-07-26 12:49:53 -04:00
Richard van der Hoff 25f43faa70
Reorganise the database schema directories (#9932)
The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate.

It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
2021-05-07 10:22:05 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Jonathan de Jong 2ca4e349e9
Bugbear: Add Mutable Parameter fixes (#9682)
Part of #9366

Adds in fixes for B006 and B008, both relating to mutable parameter lint errors.

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-04-08 22:38:54 +01:00
Patrick Cloke 01dd90b0f0
Add type hints to DictionaryCache and TTLCache. (#9442) 2021-03-29 12:15:33 -04:00
Erik Johnston 0b5c967813
Refactor to ensure we call check_consistency (#9470)
The idea here is to stop people forgetting to call `check_consistency`. Folks can still just pass in `None` to the new args in `build_sequence_generator`, but hopefully they won't.
2021-02-24 10:13:53 +00:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke 7950aa8a27 Fix some typos. 2021-02-12 11:14:12 -05:00
Erik Johnston 7a43482f19
Use execute_batch in more places (#9188)
* Use execute_batch in more places

* Newsfile
2021-01-21 14:44:12 +00:00
Richard van der Hoff 2649d545a5
Mypy fixes for `synapse.handlers.federation` (#8422)
For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
2020-09-29 15:57:36 +01:00
Erik Johnston bd380d942f
Add checks for postgres sequence consistency (#8402) 2020-09-28 18:00:30 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke aec7085179
Convert state and stream stores and related code to async (#8194) 2020-08-28 09:37:55 -04:00
Patrick Cloke a0acdfa9e9
Converts event_federation and registration databases to async/await (#8061) 2020-08-11 17:21:13 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00