Commit Graph

2835 Commits

Author SHA1 Message Date
Ben Parsons b5ac0ffa0a add more detail to logging regarding "More than one row matched" error (#4234) 2018-12-04 11:57:39 +01:00
Richard van der Hoff ecc23188f4
Fix UnicodeDecodeError when postgres is not configured in english (#4253)
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
2018-12-04 11:55:52 +01:00
Aaron Raimist 704c5298f0
Drop sent_transactions
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:07:35 -06:00
Neil Johnson f9b136a886 Neilj/fix mau initial reserved users (#4211)
* fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail

* towncrier

* white space
2018-11-28 20:33:41 +11:00
Richard van der Hoff 6c18cc4b50 Ignore __pycache__ directories in schema delta dir
Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
2018-11-20 22:52:34 +00:00
Travis Ralston 835779f7fb Add option to track MAU stats (but not limit people) (#3830) 2018-11-15 18:08:27 +00:00
David Baker bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
David Baker d44dea0223 pep8 2018-11-09 14:38:31 +00:00
David Baker 4f93abd62d add docs 2018-11-09 13:25:38 +00:00
Erik Johnston 5ebed18692 Lets convert bytes to unicode instead 2018-11-08 12:33:13 +00:00
Erik Johnston b1a22b24ab Fix noop checks when updating device keys
Clients often reupload their device keys (for some reason) so its
important for the server to check for no-ops before sending out device
list update notifications.

The check is broken in python 3 due to the fact comparing bytes and
unicode always fails, and that we write bytes to the DB but get unicode
when we read.
2018-11-08 12:18:38 +00:00
Erik Johnston bc80b3f454 Add helpers for getting prev and auth events (#4139)
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Richard van der Hoff 350f654e7b Add unique indexes to a couple of tables
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
2018-11-02 10:36:13 +00:00
Richard van der Hoff 50e328d1e7 Remove redundant database locks for device list updates
We can rely on the application-level per-user linearizer.
2018-11-02 10:36:13 +00:00
Erik Johnston 0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups
Delete unreferenced state groups during purge
2018-10-30 16:58:22 +00:00
David Baker e0934acdbb Cast to int here too 2018-10-30 11:12:23 +00:00
David Baker 12941f5f8b Cast bacjup version to int when querying 2018-10-30 11:01:07 +00:00
David Baker 2f0f911c52 Convert version back to a string 2018-10-30 10:35:18 +00:00
David Baker 64fa557f80 Try & make it work on postgres 2018-10-30 09:51:04 +00:00
David Baker 563f9b61b1 Make e2e backup versions numeric in the DB
We were doing max(version) which does not do what we wanted
on a column of type TEXT.
2018-10-29 21:01:22 +00:00
Erik Johnston 88e5ffe6fe Deduplicate device updates sent over replication
We currently send several kHz of device list updates over replication
occisonally, which often causes the replications streams to lag and then
get dropped.

A lot of those updates will actually be duplicates, since we don't send
e.g. device_ids across replication, so let's deduplicate it when we pull
them out of the database.
2018-10-29 17:34:34 +00:00
Erik Johnston ad88460e0d Move _find_unreferenced_groups 2018-10-29 14:24:19 +00:00
Erik Johnston 664b192a3b Fix set operations thinko 2018-10-29 14:21:43 +00:00
Erik Johnston f4f223aa44 Don't make temporary list 2018-10-29 14:01:49 +00:00
Erik Johnston b2399f6281 Make SQL a bit cleaner 2018-10-29 14:01:11 +00:00
Erik Johnston 03e634dad4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-26 16:22:45 +01:00
Richard van der Hoff 4cda300058 preserve room visibility 2018-10-25 19:10:24 +01:00
Erik Johnston cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011)
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Neil Johnson 95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init
fix race condiftion in calling initialise_reserved_users
2018-10-25 16:33:40 +01:00
Neil Johnson fcbd488e9a add new line 2018-10-25 16:13:43 +01:00
Neil Johnson f8fe98812b improve comments 2018-10-25 14:58:59 +01:00
Neil Johnson f7f487e14c Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin 2018-10-25 14:40:06 +01:00
Neil Johnson ea69a84bbb fix style inconsistencies 2018-10-24 17:18:08 +01:00
Neil Johnson 07126e43a4 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init 2018-10-24 16:25:39 +01:00
Richard van der Hoff 7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8
Fix a number of flake8 errors
2018-10-24 11:23:35 +01:00
Erik Johnston 3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
Richard van der Hoff ef771cc4c2 Fix a number of flake8 errors
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Neil Johnson 329d18b39c remove white space 2018-10-23 15:27:20 +01:00
Neil Johnson 6105c6101f fix race condiftion in calling initialise_reserved_users 2018-10-23 15:24:58 +01:00
Amber Brown e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
Erik Johnston 67f7b9cb50 pep8 2018-10-19 16:06:59 +01:00
Erik Johnston 056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
Erik Johnston 47a9da28ca Batch process handling state groups 2018-10-19 15:48:15 +01:00
Richard van der Hoff fc0f13dd03 Fix incorrect truncation in get_missing_events
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.

We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Erik Johnston 4a28d3d36f Update event_auth table for rejected events 2018-10-16 16:28:42 +01:00
Erik Johnston e238013c44 Add v2 state res algorithm.
We hook this up to the vdh test room version.
2018-10-16 16:16:13 +01:00
David Baker 03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
E2E backups
2018-10-15 10:19:43 +01:00
Erik Johnston 67a1e315cc Fix up comments 2018-10-12 13:49:48 +01:00
David Baker 306361b31b Misc PR feedback bits 2018-10-12 11:48:56 +01:00
Richard van der Hoff 8ddd0f273c Comments on get_all_new_events_stream
just some docstrings to clarify the behaviour here
2018-10-12 09:55:41 +01:00