Commit Graph

21486 Commits

Author SHA1 Message Date
Eric Eastwood f694bb71b7
Strip number suffix from instance name to consolidate services that traces are spread over (#13729)
The problem with many services is that it makes it hard to find which service has the trace you want, see https://github.com/jaegertracing/jaeger-ui/issues/985

Previously, we split traces out into services based on their instance name like `matrix.org client_reader-1`, etc but there are many worker instances of the same `client_reader` so there is a lot to click through.

With this PR, all of the traces are just collected under the worker type like `client_reader`, `event_persister` 😇

Note: A Synapse worker instance name is an opaque string with the number convention only being our own thing for the `matrix.org` deployment. But seems pretty sensible to group things this way.
2022-09-09 11:30:06 -05:00
Patrick Cloke 3d9f82efcb
Use an upsert for `receipts_graph`. (#13752)
Instead of a delete, then insert.

This was previously done for `receipts_linearized` in
2dc430d36e (#7607).
2022-09-09 07:08:41 -04:00
Erik Johnston c85c5ace52
Add rust to CI (#13763) 2022-09-09 11:29:04 +01:00
David Robertson f2d2481e56
Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
Sean Quah 69fa29700e
Re-type hint some collections in `/sync` code as read-only (#13754)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 20:13:39 +01:00
reivilibre 5261d2e2e8
Remove unused Prometheus recording rules from `synapse-v2.rules` and add comments describing where the rest are used. (#13756) 2022-09-08 17:50:15 +00:00
Dirk Klimpel f799eac7ea
Add timestamp to user's consent (#13741)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-09-08 15:41:48 +00:00
Sean Quah 906cead9ca
Update docstrings to explain the impact of partial state (#13750)
Update the docstrings for `get_users_in_room` and
`get_current_hosts_in_room` to explain the impact of partial state.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:55:29 +01:00
Sean Quah 89e8b98b65
Avoid raising errors due to malformed IDs in `get_current_hosts_in_room` (#13748)
Handle malformed user IDs with no colons in `get_current_hosts_in_room`.
It's not currently possible for a malformed user ID to join a room, so
this error would never be hit.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:55:03 +01:00
Sean Quah 8ef0c8ff14
Fix error in `is_mine_id` when encountering a malformed ID (#13746)
Previously, `is_mine_id` would raise an exception when passed an ID with
no colons. Return `False` instead.

Fixes #13040.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-09-08 15:54:36 +01:00
reivilibre cf11919ddd
Fix cache metrics not being updated when not using the legacy exposition module. (#13717) 2022-09-08 15:30:48 +01:00
reivilibre 526f84bc2e
Fix Prometheus recording rules to not use legacy metric names. (#13718) 2022-09-08 15:01:42 +01:00
Erik Johnston 1cc729c177
Fix latest deps (#13743) 2022-09-08 13:58:31 +01:00
David Robertson 9d11842562
Also cite upgrade notes 2022-09-08 12:04:29 +01:00
David Robertson a7c71686ca
Add minimum version bump for sqlite to the release notes (#13742)
* Notify that SQLite min version will be bumped
* Mention in upgrade notes

Co-authored-by: reivilibre <oliverw@matrix.org>
2022-09-08 12:00:03 +01:00
reivilibre b7e4bfd005
Fix a bug where Synapse fails to start if a signing key file contains an empty line. (#13738) 2022-09-08 11:18:03 +01:00
Eric Eastwood d4d3249ded
Instrument `get_metadata_for_events` for tracing (#13730)
When backfilling, `_get_state_ids_after_missing_prev_event` calls [`get_metadata_for_events`](26bc26586b/synapse/handlers/federation_event.py (L1133)). For `#matrix:matrix.org`, it's called with 77k `state_events` which means 77 calls to the database and takes 28 seconds.
2022-09-07 11:41:52 -05:00
Erik Johnston 8d7fcf9b76
Fix latest deps CI (#13734) 2022-09-07 14:07:06 +00:00
Erik Johnston dc0e896b68
Add some rust caching to CI (#13735) 2022-09-07 13:56:59 +00:00
David Robertson c46fecd1f2
Correct out-of-date doc for `event_cache_size` (#13726) 2022-09-07 14:46:11 +01:00
David Robertson 77f3986451
Define SQLite compat policy (#13728) 2022-09-07 12:07:42 +00:00
David Robertson b58386e37e
A second batch of Pydantic models for rest/client/account.py (#13687) 2022-09-07 12:16:10 +01:00
reivilibre d3d9ca156e
Cancel the processing of key query requests when they time out. (#13680) 2022-09-07 12:03:32 +01:00
reivilibre c2fe48a6ff
Rename the `EventFormatVersions` enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
Connor Davis bb5b47b62a
Add Admin API to Fetch Messages Within a Particular Window (#13672)
This adds two new admin APIs that allow us to fetch messages from a room within a particular time.
2022-09-07 10:54:44 +01:00
reivilibre 26bc26586b
Remove the unspecced room_id field in the /hierarchy response. (#13506)
This is a re-do of 57d334a13d (#13365),
which was backed out in 12abd72497 (#13501).

The `room_id` field represented the parent space for each room
and was made redundant by changes in the API shape where the
`children_state` is now nested underneath each `room`.

The room ID of each child is in the `state_key` field and is still
available.
2022-09-06 15:28:44 -04:00
Erik Johnston c9b7e97355
Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston 3d20115115
Fix trial-olddeps (#13725) 2022-09-06 14:21:55 +00:00
David Robertson a4ecb8e353
Actually fix typechecking with latest types-jsonschema (#13724) 2022-09-06 14:29:16 +01:00
Erik Johnston ec2fe7bb53 Fixup changelog 2022-09-06 13:04:22 +01:00
Erik Johnston b5effc7201
Update trial old deps CI to use poetry 1.2.0 (#13707) 2022-09-06 11:43:04 +00:00
reivilibre b455c2a5ec
Update Grafana dashboard to not use legacy metric names. (#13714) 2022-09-06 12:21:21 +01:00
Erik Johnston 571f565c1f Update changelog 2022-09-06 09:25:32 +01:00
Erik Johnston cdf7fb737b 1.67.0rc1 2022-09-06 09:01:26 +01:00
Patrick Cloke 32fc3b7ba4
Remove configuration options for direct TCP replication. (#13647)
Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
2022-09-06 07:50:02 +00:00
Erik Johnston 8edf3f66d5
Reduce number of CI jobs run on PRs (#13713)
* Reduce number of CI jobs run on PRs

* Newsfile

* Also limit sytest jobs

* Fix typo

* Fix up

* Fixup
2022-09-05 13:31:42 +01:00
reivilibre c7b18d9d44
Extend the release script to wait for GitHub Actions to finish and to be usable as a guide for the whole process. (#13483) 2022-09-05 12:16:59 +01:00
David Robertson 8cb9261598
Fix typechecking with latest `types-jsonschema` (#13712) 2022-09-05 11:13:13 +00:00
Brendan Abolivier 898fef2789
Share some metrics between the Prometheus exporter and the phone home stats (#13671) 2022-09-05 10:26:43 +00:00
Matthew Hodgson ad7fc8e92f
fix grammar 2022-09-03 11:08:28 +01:00
Eric Eastwood 877bdfa889
Clarify `(room_id, event_id)` global uniqueness (#13701)
Summarized from @richvdh's reply at https://github.com/matrix-org/synapse/pull/13589#discussion_r961116999
2022-09-02 14:05:39 -05:00
Eric Eastwood 36b184b782
Update docs to make enabling metrics more clear (#13678)
It was really easy to miss the `enable_metrics: True` step with the previous language.
2022-09-02 11:29:51 -05:00
Erik Johnston 4fee4a339d
Update lock file for Poetry v1.2.0 (#13689) 2022-09-02 13:20:03 +00:00
Erik Johnston 0fdb685c2b
Only run trial CI on all python versions on non-PRs (#13698) 2022-09-02 13:12:47 +01:00
Dirk Klimpel 044900af6c
Fix two typos with colon in headlines (#13665) 2022-09-01 20:38:37 +00:00
Patrick Cloke 48a5c47a9f
Add a schema delta to drop unstable private read receipts. (#13692)
Otherwise they'll be leaked due to the filtering code only respecting
the stable identifiers for private read receipts.
2022-09-01 14:57:47 -04:00
Patrick Cloke 390b7ce946
Disable calculating unread counts unless the config flag is enabled. (#13694)
This avoids doing work that will never be used (since the
resulting unread counts will never be sent in a /sync
response).

The negative of doing this is that unread counts will be
incorrect when the feature is initially enabled.
2022-09-01 16:52:03 +00:00
reivilibre f48f4dd59e
Update the Grafana dashboard that is included with Synapse in the `contrib` directory. (#13697)
* Add missing graph to contrib

* Update with minor but plausible changes, including positioning changes

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-09-01 16:27:06 +01:00
Erik Johnston 9d2823ab70
Cache `is_partial_state_room` (#13693)
Fixes #13613.
2022-09-01 16:07:01 +01:00
Will Hunt c913e440c0
Add monthly active users documentation (#13617)
* Add monthly active users documentation

* changelog

* Tidy up notes

* more tidyup

* Rewrite #1

* link back to mau docs

* fix links

* s/appservice|AS/application service

* further review

* a newline

* Remove bit about shadow banned users.

I think talking about them is confusing, and the current text doesn't imply they get any special treatment.

* Update docs/usage/administration/monthly_active_users.md

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Update docs/usage/administration/monthly_active_users.md

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-09-01 14:48:43 +00:00