Commit Graph

22664 Commits

Author SHA1 Message Date
Jason Little d3bd03559b
HTTP Replication Client (#15470)
Separate out a HTTP client for replication in preparation for
also supporting using UNIX sockets. The major difference from
the base class is that this does not use treq to handle HTTP
requests.
2023-05-09 14:25:20 -04:00
Travis Ralston ab4535b608
Add config option to prevent media downloads from listed domains. (#15197)
This stops media (and thumbnails) from being accessed from the
listed domains. It does not delete any already locally cached media,
but will prevent accessing it.

Note that admin APIs are unaffected by this change.
2023-05-09 14:08:51 -04:00
Mathieu Velten 266d287165 Merge branch 'master' into develop 2023-05-09 19:42:57 +02:00
Mathieu Velten 64a11fb61f 1.83.0 2023-05-09 18:13:48 +02:00
Patrick Cloke 4b4e0dc3ce
Error if attempting to set m.push_rules account data, per MSC4010. (#15555)
m.push_rules, like m.fully_read, is a special account data type that cannot
be set using the normal /account_data endpoint. Return an error instead
of allowing data that will not be used to be stored.
2023-05-09 10:34:10 -04:00
Patrick Cloke 2bfe3f0b81
Use account data constants in more places. (#15554) 2023-05-09 07:23:27 -04:00
dependabot[bot] 6b7da31221
Bump hiredis from 2.2.2 to 2.2.3 (#15552) 2023-05-08 07:57:40 -04:00
dependabot[bot] 58a07f0c3f
Bump serde from 1.0.160 to 1.0.162 (#15548) 2023-05-08 07:56:14 -04:00
dependabot[bot] 058c6269f3
Bump serde from 1.0.160 to 1.0.162 (#15548) 2023-05-08 07:29:55 -04:00
dependabot[bot] 0a18aa236d
Bump sentry-sdk from 1.19.1 to 1.22.1 (#15550) 2023-05-08 07:15:22 -04:00
dependabot[bot] db093df5eb
Bump types-setuptools from 67.6.0.5 to 67.7.0.1 (#15549) 2023-05-08 07:15:10 -04:00
dependabot[bot] 245d34bdcc
Bump ruff from 0.0.259 to 0.0.265 (#15551) 2023-05-08 07:14:52 -04:00
dependabot[bot] 9a87895b59
Bump types-requests from 2.29.0.0 to 2.30.0.0 (#15553) 2023-05-08 07:13:47 -04:00
Patrick Cloke 28bceef84e
Check appservices for devices during a /user/devices query. (#15539)
MSC3984 proxies /keys/query requests to appservices, but servers will
can also requests devices / keys from the /user/devices endpoint.

The formats are close enough that we can "proxy" that /user/devices to
appservices (by calling /keys/query) and then change the format of the
returned data before returning it over federation.
2023-05-05 15:18:47 -04:00
Patrick Cloke 36df9c5e36
Implement MSC4009 to widen the allowed Matrix ID grammar (#15536)
Behind a configuration flag this adds + to the list of allowed
characters in Matrix IDs. The main feature this enables is
using full E.164 phone numbers as Matrix IDs.
2023-05-05 12:13:50 -04:00
Zdziszek a0f53afd62
Handle `DNSNotImplementedError` in SRV resolver (#15523)
Signed-off-by: Zdzichu <zdzichu.rks@protonmail.com>
2023-05-05 15:54:32 +01:00
Andrew Morgan ad141efb47
Add `mdbook` to flake.nix (#15545) 2023-05-05 15:51:58 +01:00
Andrew Morgan 7c95b65873
Clean up and clarify "Create or modify Account" Admin API documentation (#15544) 2023-05-05 15:51:46 +01:00
Sean Quah e46d5f3586
Factor out an `is_mine_server_name` method (#15542)
Add an `is_mine_server_name` method, similar to `is_mine_id`.

Ideally we would use this consistently, instead of sometimes comparing
against `hs.hostname` and other times reaching into
`hs.config.server.server_name`.

Also fix a bug in the tests where `hs.hostname` would sometimes differ
from `hs.config.server.server_name`.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-05 15:06:22 +01:00
Andrew Morgan 83e7fa5eee
Allow running Complement integration tests via podman (#15543) 2023-05-05 13:27:51 +01:00
Andrew Morgan 2e59e97ebd
Move ThirdPartyEventRules into module_api/callbacks (#15535) 2023-05-04 14:18:22 +00:00
Patrick Cloke ded8f3d349
Update the base rules to remove the dont_notify action. (MSC3987) (#15534)
A dont_notify action is a no-op (and coalesce is undefined). These are
both considered no-ops by the spec, per MSC3987 and the predefined
push rules were updated to remove dont_notify from the list of actions.
2023-05-04 11:54:13 +00:00
dependabot[bot] cc872eaf16
Bump pyicu from 2.10.2 to 2.11 (#15509)
* Bump pyicu from 2.10.2 to 2.11

Bumps [pyicu](https://gitlab.pyicu.org/main/pyicu) from 2.10.2 to 2.11.

---
updated-dependencies:
- dependency-name: pyicu
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-05-03 23:45:42 +01:00
Sandro 5f8822854d
Use oEmbed for YouTube Shorts (#15025)
It seems that YouTube Short previews do not work in some
regions, but the oEmbed information for those areas is still
valid.

This causes YouTube Shorts to always use (only) the oEmbed
endpoint which is a minor regression for regions where the URL
preview was already working -- some of the additional video
metadata is lost. It is not likely that clients are using this today
and it is more beneficial to have a limited preview working everywhere
than unused metadata in the Open Graph response.
2023-05-03 12:54:42 -04:00
Andrew Morgan 1d6140ec8a
Add `xmlsec` package to flake.nix (#15532) 2023-05-03 16:26:15 +01:00
Andrew Morgan 7be05df0b1
Switch back to upstream devenv flake (#15533) 2023-05-03 14:40:10 +00:00
Sean Quah 8aee823393 Merge branch 'release-v1.83' into develop 2023-05-03 15:23:16 +01:00
Erik Johnston 28ac1a1a91
Speed up deleting of old rows in `event_push_actions` (#15531)
Enforce that we use index scans (rather than seq scans), which we also do for state queries. The reason to enforce this is that we can't correctly get PostgreSQL to understand the distribution of `stream_ordering` depends on `highlight`, and so it always defaults (on matrix.org) to sequential scans.
2023-05-03 13:42:43 +00:00
Erik Johnston fc3a878220
Speed up rebuilding of the user directory for local users (#15529)
The idea here is to batch up the work.
2023-05-03 13:41:37 +00:00
Sean Quah 3b837d856c
Revert "Reduce the size of the HTTP connection pool for non-pushers" (#15530)
#15514 introduced a regression where Synapse would encounter
`PartialDownloadError`s when fetching OpenID metadata for certain
providers on startup. Due to #8088, this prevents Synapse from starting
entirely.

Revert the change while we decide what to do about the regression.
2023-05-03 13:09:20 +01:00
Andrew Morgan 9890f23469
Suppress the trusted key server warning for matrix.org in the demo scripts (#15527) 2023-05-03 12:07:49 +00:00
Patrick Cloke a7b3e9ce65
Set thread_id column to non-null for event_push_{actions,actions_staging,summary} (#15437)
Updates the database schema to require a thread_id (by adding a
constraint that the column is non-null) for event_push_actions,
event_push_actions_staging, and event_push_actions_summary.

For PostgreSQL we add the constraint as NOT VALID, then
VALIDATE the constraint a background job to avoid locking
the table during an upgrade.

For SQLite we simply rebuild the table & copy the data.
2023-05-03 07:49:03 -04:00
Sean Quah 04e79e6a18
Add config option to forget rooms automatically when users leave them (#15224)
This is largely based off the stats and user directory updater code.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-03 12:27:33 +01:00
Shay 0e8aa2a1b2
Remove references to supporting per-user flag for msc2654 (#15522) 2023-05-02 14:21:36 -07:00
Erik Johnston 4de271a7fc
Allow adding random delay to push (#15516)
This is to discourage timing based profiling on the push gateways.
2023-05-02 16:45:44 +00:00
Sean Quah 1c0e98717b Update CHANGES.md 2023-05-02 16:14:20 +01:00
Sean Quah ca6bda2f57 Update CHANGES.md 2023-05-02 16:13:29 +01:00
Sean Quah 60d59af300 Fix up docs summary to include new experimental features admin API docs 2023-05-02 16:12:50 +01:00
Sean Quah 7b41966be9 1.83.0rc1 2023-05-02 15:56:50 +01:00
Patrick Cloke 6aca4e7cb8
Reduce the size of the HTTP connection pool for non-pushers. (#15514)
Pushers tend to make many connections to the same HTTP host
(e.g. a new event comes in, causes events to be pushed, and then
the homeserver connects to the same host many times). Due to this
the per-host HTTP connection pool size was increased, but this does
not make sense for other SimpleHttpClients.

Add a parameter for the connection pool and override it for pushers
(making a separate SimpleHttpClient for pushers with the increased
configuration).

This returns the HTTP connection pool settings to the default Twisted
ones for non-pusher HTTP clients.
2023-05-02 09:29:40 -04:00
Patrick Cloke 07b1c70d6b
Initial implementation of MSC3981: recursive relations API (#15315)
Adds an optional keyword argument to the /relations API which
will recurse a limited number of event relationships.

This will cause the API to return not just the events related to the
parent event, but also events related to those related to the parent
event, etc.

This is disabled by default behind an experimental configuration
flag and is currently implemented using prefixed parameters.
2023-05-02 07:59:55 -04:00
dependabot[bot] 3b853b18b5
Bump anyhow from 1.0.70 to 1.0.71 (#15507) 2023-05-01 11:34:01 -04:00
dependabot[bot] 0da7cceae9
Bump setuptools-rust from 1.5.2 to 1.6.0 (#15512) 2023-05-01 08:27:29 -04:00
dependabot[bot] fcc943d552
Bump packaging from 23.0 to 23.1 (#15510) 2023-05-01 08:01:46 -04:00
dependabot[bot] 229fe1d197
Bump types-requests from 2.28.11.16 to 2.29.0.0 (#15511) 2023-05-01 07:52:38 -04:00
dependabot[bot] 1046184f35
Bump types-pillow from 9.4.0.19 to 9.5.0.2 (#15508)
* Bump types-pillow from 9.4.0.19 to 9.5.0.2

Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.19 to 9.5.0.2.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)
2023-05-01 07:26:00 -04:00
Shay 89f6fb0d5a
Add an admin API endpoint to support per-user feature flags (#15344) 2023-04-28 11:33:45 -07:00
Shay eb6f8dc215
Update development docs referencing outdated versions of sqlite we no longer support (#15498) 2023-04-28 10:59:00 -07:00
Patrick Cloke 57aeeb308b
Add support for claiming multiple OTKs at once. (#15468)
MSC3983 provides a way to request multiple OTKs at once from appservices,
this extends this concept to the Client-Server API.

Note that this will likely be spit out into a separate MSC, but is currently part of
MSC3983.
2023-04-27 12:57:46 -04:00
Patrick Cloke 6efa674004
Add type hints to schema deltas (#15497)
Cleans-up the schema delta files:

* Removes no-op functions.
* Adds missing type hints to function parameters.
* Fixes any issues with type hints.

This also renames one (very old) schema delta to avoid a conflict
that mypy complains about.
2023-04-27 12:44:53 +00:00