Commit Graph

19798 Commits

Author SHA1 Message Date
Nicolai Søborg e2dabec996
Docs: Quote wildcard `federation_certificate_verification_whitelist` (#11381)
Otherwise I get this beautiful stacktrace:

```
python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/synapse/synapse/app/homeserver.py", line 455, in <module>
    main()
  File "/root/synapse/synapse/app/homeserver.py", line 445, in main
    hs = setup(sys.argv[1:])
  File "/root/synapse/synapse/app/homeserver.py", line 345, in setup
    config = HomeServerConfig.load_or_generate_config(
  File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config
    config_dict = read_config_files(config_files)
  File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files
    yaml_config = yaml.safe_load(file_stream)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node
    while not self.check_event(SequenceEndEvent):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry
    return self.parse_block_sequence_entry()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
    if not self.check_token(BlockEntryToken, BlockEndToken):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/matrix/homeserver.yaml", line 614, column 5
expected alphabetic or numeric character, but found '.'
  in "/etc/matrix/homeserver.yaml", line 614, column 6
```

Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
2021-11-18 12:24:40 +00:00
Sean Quah 84fac0f814
Add type annotations to `synapse.metrics` (#10847) 2021-11-17 19:07:02 +00:00
Aaron R d993c3bb1e
Add support for `/_matrix/media/v3` APIs (#11371)
* Add support for `/_matrix/media/v3` APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Update `workers.md` to use v3 client and media APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-17 15:30:24 +00:00
David Robertson b76337fdf8
Merge branch 'master' into develop 2021-11-17 14:19:56 +00:00
David Robertson 077b74929f
Merge remote-tracking branch 'origin/release-v1.47' 2021-11-17 14:19:27 +00:00
reivilibre 0d86f6334a
Rename `get_access_token_for_user_id` method to `create_access_token_for_user_id` (#11369) 2021-11-17 14:10:57 +00:00
Patrick Cloke 60ecb6b4d4
Fix running complement.sh script. (#11368)
By reverting changes from #11166 in this script. Specifically commit
13f084eb58.
2021-11-17 09:04:50 -05:00
David Robertson 9f9d82aa84
1.47.0 2021-11-17 13:10:12 +00:00
Patrick Cloke 319dcb955e
Fix incorrect return value in tests. (#11359) 2021-11-16 16:36:46 +00:00
David Robertson 0caf20883c Synapse 1.47.0rc3 (2021-11-16)
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. ([\#11346](https://github.com/matrix-org/synapse/issues/11346))
 - Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from `device_inbox` column' background process from running when updating from a recent Synapse version. ([\#11303](https://github.com/matrix-org/synapse/issues/11303), [\#11353](https://github.com/matrix-org/synapse/issues/11353))
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCAAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAmGTw7wTHGFuZHJld0Bh
 bW9yZ2FuLnh5egAKCRCIhIgNLv5f9FUdD/9EgimhmOUuwcbNQS+zJJo6qhbnxngA
 uOIWGhuoBq9vyZjx3s1mwRO/sCuBc6KRs2JYzhntfHwV6FCgkC0QJXPkrYO7mkbg
 L/QsET3QYrWwClMN1y/j2/rstOqGtgrtg6pPgd9LiQH+LjdM+vzAyrgxJRyhk9Sx
 7184Y9QpEt/6ApqFIbXZewP+zH1z1QyVeZ1WiuHsqhjcrrcpv1t8Rq0fv4FxgIVP
 Tmy3T2uw0FeVoQIL8BnNVmRGbLjlSgpDfiXZTwuXGS/uLtTe1tnHf0QsKEK7XZxY
 yUsRKQVDy76GW9AJuWKFXa9Voy5gkjLWnvMwHvr+B1WcJi/5cjz9lP2A0Bsm6vhQ
 ivrktlhlw7O3x0EvrL1r4z2gRY4GXxyZKOBLRsY4AAIMblNR/e94SmK0rLoKIQn8
 Rp50VV2B4cf3WLxoqHBpJh7CDFisth1nXMnETU3y0VgAYMg2+cxqVDawTSTH9v7Q
 QeGI10QCMmeFwGoL6lyVJ+qhyUqFeFOqTrNqzigsiB1qule+w1fEEl7cvI+Om/QM
 Wjkvw6fpOvUGBN1OOARL5Qnm4rjxa0Ld2vw5vugkGiDTV8P3TMJP1JFQ44IX2GKP
 yGJH92ac0qy6vQ2za1gL/w/U3miNblHUIvAmpkaIIlvovQJH9dU5dMsxnvi+QZU+
 Auyqh9rwiWiizA==
 =iNww
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmGT0mcACgkQkD7OEIo5
 3t0sSA/+KjLlnYe3xO9Pbeeq7/E6Qs3VTMbwERcVVj0WZQU8Hik/EH+p0zsnG9jU
 8t9UVs0xnu1r5ZMNsrMnfOU5SR9o7RM8UBCHnIYde8yFf1N6FI0YgGJ8nOP96gSG
 dKZQOCKA9psQqd0pvhwCem4ITmBvVz2CrnUZYkKp9rHjUZeBYwE7cWmHL5W/WzT+
 Sv0zafhvYi76PxMKdXu08MArMZ9JpCLbZnlzaoW/sY+xvnQBRfQviqYkj+qZhQjr
 tPnJlQN5FS1pO1ZHd9o0mVcdbDBwLqeQlqC1toFuZXbed6e558KtCzvizM5VAfF8
 peRGVFarbZpD/QcvoFljoH9qIECrcAYIN1HeE8aX7pIedh2AVROyINumdnbmsMJH
 F4OyX/aLb3KbbecHJaVLQ2c/KyDnSnxr6Fs3HlEslD7DRdA0TS7XsI/BVER0pscp
 j2Q5nLOthpV2d1SVekj4Ge/Hr++AuTnriHldEx9OlGI3/74ldrNL2c/L5gM3ue1W
 qnPQfr5ehtIitqa/ROiexxSoWS5OC953UujTCwBgil9mAD9gC7mhoEFOAFVuMHfN
 zBjsunMGBRYsGcw2umTPGPd7D3Gi1FXQrduN1xUoV8g8vmJld4GDjqJsitiaR3lv
 NUsQ5JzttakDUwAJU7qijOo2Y/HtSs5E2nF66bMCSmwHvl9AkUc=
 =c7gi
 -----END PGP SIGNATURE-----

Merge tag 'v1.47.0rc3' into develop

Synapse 1.47.0rc3 (2021-11-16)
==============================

Bugfixes
--------

- Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. ([\#11346](https://github.com/matrix-org/synapse/issues/11346))
- Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from `device_inbox` column' background process from running when updating from a recent Synapse version. ([\#11303](https://github.com/matrix-org/synapse/issues/11303), [\#11353](https://github.com/matrix-org/synapse/issues/11353))
2021-11-16 15:46:45 +00:00
Sean Quah 88375beeaa
Avoid sharing room hierarchy responses between users (#11355)
Different users may be allowed to see different rooms within a space,
so sharing responses between users is inadvisable.
2021-11-16 15:40:47 +00:00
Andrew Morgan 7baa671dc8 fix up changelog language 2021-11-16 14:42:21 +00:00
Andrew Morgan 729acd82c8 mark the migration file migration as a bug 2021-11-16 14:41:21 +00:00
Andrew Morgan edcdc5fd82 1.47.0rc3 2021-11-16 14:34:46 +00:00
Aaron R dfa536490e
Add support for `/_matrix/client/v3` APIs (#11318)
This is one of the changes required to support Matrix 1.1

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-16 14:47:58 +01:00
Patrick Cloke 7468723697
Add most missing type hints to synapse.util (#11328) 2021-11-16 08:47:36 -05:00
Andrew Morgan 6e084b62b8
Rename `remove_deleted_devices_from_device_inbox` to ensure it is always run (#11353)
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-16 13:16:43 +00:00
reivilibre 3a1462f7e0
Properly register all callback hooks for legacy password authentication providers (#11340)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-16 12:53:31 +00:00
Patrick Cloke 24b61f379a
Add ability to un-shadow-ban via the admin API. (#11347) 2021-11-16 12:43:53 +00:00
David Robertson 0dda1a7968
Misc typing fixes for tests, part 2 of N (#11330) 2021-11-16 10:41:35 +00:00
Ashwin Nair e72135b9d3
change 'Home Server' to one word 'homeserver' (#11320)
Signed-off-by: Ashwin S. Nair <58840757+Ashwin-exe@users.noreply.github.com>
2021-11-16 10:21:01 +00:00
Andrew Morgan 9c59e117db
Run _upgrade_existing_database on workers if at current schema_version (#11346)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-15 17:34:15 +00:00
David Robertson e605e4b8f2
Database storage profile passes mypy (#11342)
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
2021-11-15 12:59:33 +00:00
David Robertson 5562ce6a53
Get directory db file to pass mypy (#11339) 2021-11-15 12:59:05 +00:00
Dirk Klimpel b596a1eb80 Move sql file for `remove_deleted_devices_from_device_inbox` into v65 (#11303) 2021-11-15 11:47:30 +00:00
Tulir Asokan 6f862c5c28
Add support for the stable version of MSC2778 (#11335)
* Add support for the stable version of MSC2778

Signed-off-by: Tulir Asokan <tulir@maunium.net>

* Expect m.login.application_service in login and password provider tests

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2021-11-15 10:31:22 +00:00
Shay 605921bc6b
Remove unused tables `room_stats_historical` and `user_stats_historical` (#11280)
* remove unused tables room_stats_historical and user_stats_historical

* update changelog number

* Bump schema compat version comment

* make linter happy

* Update comment to give more info

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-12 16:47:56 -08:00
David Robertson fe58672546
Annotations for state_deltas.py (#11316)
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
2021-11-12 20:24:12 +00:00
Patrick Cloke 3fad4e3fe5
Rollback #11322 due to wrong syntax in mypy.ini. (#11332)
This was only checking the __init__ files in modules instead of
all files in a module, which don't pass yet.
2021-11-12 20:10:03 +00:00
David Robertson bea815cec8
Test room alias deletion (#11327)
* Prefer `HTTPStatus` over plain `int`

This is an Opinion that no-one has seemed to object to yet.

* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
2021-11-12 19:56:00 +00:00
Shay 0bcae8ad56
Change display names/avatar URLs to None if they contain null bytes before storing in DB (#11230)
* change display names/avatar URLS to None if they contain null bytes

* add changelog

* add POC test, requested changes

* add a saner test and remove old one

* update test to verify that display name has been changed to None

* make test less fragile
2021-11-12 10:38:24 -08:00
Patrick Cloke 9b90b9454b
Add type hints to media repository storage module (#11311) 2021-11-12 11:05:26 -05:00
David Robertson 6f8f3d4bc5
Attempt to annotate events_forward_extremities (#11314)
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore

the former implicitly inherits from the latter, so they should be
ordered like this when used.
2021-11-12 15:58:17 +00:00
David Robertson 4c96ce396e
Misc typing fixes for `tests`, part 1 of N (#11323)
* Annotate HomeserverTestCase.servlets
* Correct annotation of federation_auth_origin
* Use AnyStr custom_headers instead of a Union

This allows (str, str) and (bytes, bytes).
This disallows (str, bytes) and (bytes, str)

* DomainSpecificString.SIGIL is a ClassVar
2021-11-12 15:50:54 +00:00
Patrick Cloke 95547e5300
Generalize the disallowed_untyped_defs in mypy.ini (#11322) 2021-11-12 14:27:45 +00:00
Patrick Cloke b64b6d12d4
Add more type hints to synapse.util. (#11321) 2021-11-12 13:43:06 +00:00
reivilibre 2fffcb24d8
Suggest using `/etc/matrix-synapse/conf.d/` for configuration with Debian packages (#11281) 2021-11-12 13:17:06 +00:00
reivilibre 4ad5ee9996
Correct target of link to the modules page from the Password Auth Providers page (#11309) 2021-11-12 12:58:39 +00:00
Dirk Klimpel 8840a7b7f1
Convert delete room admin API to async endpoint (#11223)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-12 12:35:31 +00:00
David Robertson c99da2d079
Annotations for user_erasure_store (#11313)
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
2021-11-11 19:22:19 +00:00
David Robertson 6a605f4a77
Get db signatures file to pass mypy (#11312) 2021-11-11 17:04:44 +00:00
David Robertson 8dc666f785
Correct type hint for room_batch.py (#11310)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11 16:49:28 +00:00
Dirk Klimpel 48278a0d09
Move sql file for `remove_deleted_devices_from_device_inbox` into v65 (#11303) 2021-11-11 15:01:13 +00:00
Patrick Cloke 64ef25391d
Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
Neeeflix 6ce19b94e8
Fix error in thumbnail generation (#11288)
Signed-off-by: Jonas Zeunert <jonas@zeunert.org>
2021-11-10 20:49:43 +00:00
Patrick Cloke 5cace20bf1
Add missing type hints to `synapse.app`. (#11287) 2021-11-10 15:06:54 -05:00
Patrick Cloke 66c4b774fd
Add type hints to synapse._scripts (#11297) 2021-11-10 17:55:32 +00:00
Andrew Morgan 5f277ffe89
Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298) 2021-11-10 17:54:56 +00:00
Richard van der Hoff 73cbb284b9
Remove redundant parameters on `_check_event_auth` (#11292)
as of #11012, these parameters are unused.
2021-11-10 14:16:06 +00:00
Olivier Wilkinson (reivilibre) 68c258a604 Synapse 1.47.0rc2 (2021-11-10)
==============================
 
 This fixes an issue with publishing the Debian packages for 1.47.0rc1.
 It is otherwise identical to 1.47.0rc1.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmGLl4sACgkQLS76LzL7
 4EdpXxAArEBqEWCUCy6wSNRfexzI+qAITzhhqR5BiDDtkt33GzXLWDN83lT3kCS/
 xNZyhiwwUbtt/TqkZ0/Tqu2afI5JZCizpP/kXLVS2WA03jY8+l+eQbKEQR5vEsEV
 752J8OVJ9GUunewOI4Uo4xRqndzvOMQBKaoPzyq44PcFd6lS2rkAnJYstVnow+rB
 JvRIXFjOocaOzpemal5Mh8ToH5Y6yfe7MEE8B0s3PjX1FMAv87475X1oLaK7GeKn
 3yf8XJ1mJcvJfkHuopqX8PxW+pFGorb4N1AFk2BikxB4XV0nCo5VHUejSWaEP4oH
 uHtTYDV5RHwHhZyHYMbBOyPD2bSIKZ6wXqgn/o5io7mfsUS77SdwuGhRDqnupCi/
 Vg+4F5e1nuSPaLEJd5Qfb1NM2xErGm8gfYN3DlRBzpuQ+Jx/034YWds42ZuNv2IO
 qdAOeztiOMdrPkdPTL+XlNNXV80waMsQFt2EaycYkYmPVtgvlKvr/c/Wg06jyD7u
 dll33KlE8d+jM3JbOf6D/Ze5ECQlf0gGtCukXTMIEOorrCdQuLH4R8hb49YqmiLq
 nL8aPXCv1pZOrMbHGbcYfHoZIV+qMhUK04PN7jMLF/+nyBkTI1wRODN0lvBh8gy6
 dNWTzM/dMk5IvFY0FDipkR7Cv5c2U2Wu1xCy8026VHCu/DIzVRw=
 =Baxl
 -----END PGP SIGNATURE-----

Merge tag 'v1.47.0rc2' into develop

Synapse 1.47.0rc2 (2021-11-10)
==============================

This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.
2021-11-10 13:01:08 +00:00