Commit Graph

5069 Commits

Author SHA1 Message Date
Daniel Wagner-Hall 728d07c8c1 Merge pull request #256 from matrix-org/auth
Attempt to validate macaroons
2015-09-14 18:09:33 +01:00
Erik Johnston d59acb8c5b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable 2015-09-14 18:05:31 +01:00
Erik Johnston 91cb3b630d Merge pull request #265 from matrix-org/erikj/check_room_exists
Check room exists when authenticating an event
2015-09-14 17:56:18 +01:00
Erik Johnston dffc9c4ae0 Drop unused index 2015-09-14 14:41:37 +01:00
Mark Haines e2054ce21a Allow users to GET individual state events for rooms that they have left 2015-09-10 15:06:47 +01:00
Erik Johnston 49ae42bbe1 Bundle in some room state in the unsigned bit of the invite when sending to invited servers 2015-09-10 14:25:54 +01:00
David Baker ca32c7a065 Fix adding threepids to an existing account 2015-09-10 10:44:56 +01:00
David Baker 184a5c81f0 Merge pull request #274 from matrix-org/add_threepid_fix
Fix adding threepids to an existing account
2015-09-10 10:36:58 +01:00
David Baker 30768dcf40 Fix adding threepids to an existing account 2015-09-10 10:33:48 +01:00
Erik Johnston 4ae73d16a9 Merge pull request #270 from matrix-org/markjh/fix_metrics
Fix the size reported by maxrss.
2015-09-10 10:32:10 +01:00
Erik Johnston 3f60481655 Bump version and change log 2015-09-10 09:58:32 +01:00
Erik Johnston e1eb1f3fb9 Various bug fixes to crypto.keyring 2015-09-10 09:48:12 +01:00
Mark Haines 09cb5c7d33 Allow users that have left a room to get the messages that happend in the room before they left 2015-09-09 17:31:09 +01:00
Erik Johnston dd0867f5ba Various bug fixes to crypto.keyring 2015-09-09 17:02:39 +01:00
Mark Haines 3c166a24c5 Remove undocumented and unimplemented 'feedback' parameter from the Client-Server API 2015-09-09 16:05:09 +01:00
Mark Haines bc8b25eb56 Allow users that have left the room to view the member list from the point they left 2015-09-09 15:42:16 +01:00
Daniel Wagner-Hall 2c746382e0 Merge branch 'daniel/insecureclient' into develop 2015-09-09 14:27:30 +01:00
Mark Haines 1d579df664 Allow rooms/{roomId}/state for a room that has been left 2015-09-09 14:12:24 +01:00
Daniel Wagner-Hall ddfe30ba83 Better document the intent of the insecure SSL setting 2015-09-09 13:26:23 +01:00
Mark Haines 89ae0166de Allow room initialSync for users that have left the room, returning a snapshot of how the room was when they left it 2015-09-09 13:25:22 +01:00
Daniel Wagner-Hall 6485f03d91 Fix random formatting 2015-09-09 13:05:00 +01:00
Daniel Wagner-Hall 81a93ddcc8 Allow configuration to ignore invalid SSL certs
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
2015-09-09 12:02:07 +01:00
Erik Johnston e530208e68 Change default history visibility for private rooms 2015-09-09 09:57:49 +01:00
Mark Haines dd42bb78d0 Include rooms that a user has left in an initialSync. Include the state and messages at the point they left the room 2015-09-08 18:16:09 +01:00
Mark Haines 417485eefa Include the event_id and stream_ordering of membership events when looking up which rooms a user is in 2015-09-08 18:14:54 +01:00
Erik Johnston 2ff439cff7 Bump version/changelog 2015-09-08 11:01:48 +01:00
Mark Haines 709ba99afd Check that /proc/self/fd exists before listing it 2015-09-07 16:45:55 +01:00
Mark Haines 9e4dacd5e7 The maxrss reported by getrusage is in kilobytes, not pages 2015-09-07 16:45:48 +01:00
Mark Haines 73e4ad4b8b Merge branch 'master' into develop
Conflicts:
	setup.py
2015-09-07 15:06:46 +01:00
Erik Johnston 1002bbd732 Change log level to info 2015-09-03 09:51:01 +01:00
Erik Johnston 9ad38c9807 Bump version and changelog 2015-09-03 09:49:54 +01:00
Erik Johnston fd0a919af3 Lists use 'append' 2015-09-02 17:27:59 +01:00
Erik Johnston e90f32646f Bump version and changelog 2015-09-02 17:17:40 +01:00
Daniel Wagner-Hall 77580addc3 Merge pull request #262 from matrix-org/redactyoself
Allow users to redact their own events
2015-09-02 10:02:36 +01:00
Mark Haines 8bab7abddd Add nacl.bindings to the list of modules checked. Re-arrange import order to check packages after the packages they depend on 2015-09-01 16:51:10 +01:00
Mark Haines 3cdfd37d95 Print an example "pip install" line for a missing requirement 2015-09-01 16:47:26 +01:00
Erik Johnston 9b05ef6f39 Also check the domains for membership state_keys 2015-09-01 16:17:25 +01:00
Erik Johnston 187320b019 Merge branch 'erikj/check_room_exists' into erikj/unfederatable 2015-09-01 15:58:10 +01:00
Erik Johnston b345853918 Check against sender rather than event_id 2015-09-01 15:57:35 +01:00
Erik Johnston a88e16152f Add flag which disables federation of the room 2015-09-01 15:47:30 +01:00
Erik Johnston 00149c063b Fix tests 2015-09-01 15:42:03 +01:00
Erik Johnston ab9e01809d Check room exists when authenticating an event, by asserting they reference a creation event 2015-09-01 15:21:24 +01:00
Mark Haines 57df6fffa7 Use the version of "matrix-angular-sdk" hosted on pypi 2015-09-01 14:47:57 +01:00
Erik Johnston b62c1395d6 Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop 2015-09-01 13:11:55 +01:00
Daniel Wagner-Hall e255c2c32f s/user_id/user/g for consistency 2015-09-01 12:41:16 +01:00
Erik Johnston 9c8eb4a809 Merge pull request #261 from matrix-org/erikj/scripts_clean
Clean up scripts/
2015-09-01 11:55:26 +01:00
Daniel Wagner-Hall b854a375b0 Check domain of events properly
Federated servers still need to delegate authority to owning servers
2015-09-01 11:53:31 +01:00
Erik Johnston cd800ad99a Lower size of 'stateGroupCache' now that we have data from matrix.org to support doing so 2015-09-01 10:09:03 +01:00
Matthew Hodgson d71af2ee12 don't log the whole DB config (including postgres password...) 2015-08-29 22:23:21 +01:00
Daniel Wagner-Hall b143641b20 Merge pull request #258 from matrix-org/slowtestsmakemesad
Swap out bcrypt for md5 in tests
2015-08-28 15:42:25 +01:00
Daniel Wagner-Hall 4d1ea40008 Merge branch 'develop' into redactyoself
Conflicts:
	synapse/handlers/_base.py
2015-08-28 15:35:39 +01:00
Daniel Wagner-Hall 8256a8ece7 Allow users to redact their own events 2015-08-28 15:31:49 +01:00
Mark Haines a7122692d9 Merge branch 'release-v0.10.0' into develop
Conflicts:
	synapse/handlers/auth.py
	synapse/python_dependencies.py
	synapse/rest/client/v1/login.py
2015-08-28 11:15:27 +01:00
Erik Johnston b442217d91 Actually add config path 2015-08-28 10:37:17 +01:00
Erik Johnston 5371c2a1f7 Bump version and changelog 2015-08-27 11:21:11 +01:00
Erik Johnston ddf4d2bd98 Consistency 2015-08-27 10:50:49 +01:00
Erik Johnston 66ec6cf9b8 Check for an internationalised filename first 2015-08-27 10:48:58 +01:00
Erik Johnston 53c2eed862 None check the correct variable 2015-08-27 10:38:22 +01:00
Erik Johnston f02532baad Check for None 2015-08-27 10:37:02 +01:00
Erik Johnston 25b32b63ae Bump changelog and version 2015-08-27 10:09:32 +01:00
Mark Haines c9cb354b58 Give a sensible error message if the filename is invalid UTF-8 2015-08-26 17:27:23 +01:00
Mark Haines 5a9e0c3682 Handle unicode filenames given when downloading or received over federation 2015-08-26 17:08:47 +01:00
Mark Haines e85c7873dc Allow non-ascii filenames for attachments 2015-08-26 16:26:37 +01:00
Daniel Wagner-Hall 86fac9c95e Remove unused import 2015-08-26 16:03:17 +01:00
Daniel Wagner-Hall 3063383547 Swap out bcrypt for md5 in tests
This reduces our ~8 second sequential test time down to ~7 seconds
2015-08-26 15:59:32 +01:00
Daniel Wagner-Hall 37f0ddca5f Merge branch 'mergeeriksmadness' into auth 2015-08-26 13:45:06 +01:00
Daniel Wagner-Hall d3c0e48859 Merge erikj/user_dedup to develop 2015-08-26 13:42:45 +01:00
Daniel Wagner-Hall 6a4b650d8a Attempt to validate macaroons
A couple of weird caveats:
 * If we can't validate your macaroon, we fall back to checking that
   your access token is in the DB, and ignoring the failure
 * Even if we can validate your macaroon, we still have to hit the DB to
   get the access token ID, which we pretend is a device ID all over the
   codebase.

This mostly adds the interesting code, and points out the two pieces we
need to delete (and necessary conditions) in order to fix the above
caveats.
2015-08-26 13:22:23 +01:00
Erik Johnston da51acf0e7 Remove needless existence checks 2015-08-26 11:08:23 +01:00
Erik Johnston f4d552589e Don't loop over all rooms ever in typing.get_new_events_for_user 2015-08-26 10:51:08 +01:00
Erik Johnston 90fde4b8d7 Bump changelog and version 2015-08-25 17:49:58 +01:00
Erik Johnston 0de2aad061 Merge pull request #250 from matrix-org/erikj/generated_directory
Add config option to specify where generated files should be dumped
2015-08-25 17:40:19 +01:00
Erik Johnston 3f6f74686a Update config doc 2015-08-25 17:37:21 +01:00
Erik Johnston 82145912c3 s/--generated-directory/--keys-directory/ 2015-08-25 17:31:22 +01:00
Daniel Wagner-Hall a2355fae7e Merge pull request #251 from matrix-org/removeadmin
Stop looking up "admin", which we never read
2015-08-25 17:23:05 +01:00
Daniel Wagner-Hall ee3fa1a99c Merge pull request #248 from matrix-org/deviceid
Remove completely unused concepts from codebase
2015-08-25 17:19:06 +01:00
Erik Johnston 3e1029fe80 Warn if we encounter unexpected files in config directories 2015-08-25 17:08:23 +01:00
Erik Johnston af7c1397d1 Add config option to specify where generated files should be dumped 2015-08-25 16:58:01 +01:00
Daniel Wagner-Hall 460cad7c11 Merge branch 'deviceid' into removeadmin 2015-08-25 16:37:59 +01:00
Daniel Wagner-Hall 825f0875bc Fix up one more reference 2015-08-25 16:37:37 +01:00
Daniel Wagner-Hall a9d8bd95e7 Stop looking up "admin", which we never read 2015-08-25 16:29:39 +01:00
Erik Johnston bfb66773a4 Allow specifying directories as config files 2015-08-25 16:25:54 +01:00
Daniel Wagner-Hall 57619d6058 Re-wrap line 2015-08-25 16:25:46 +01:00
Daniel Wagner-Hall a0b181bd17 Remove completely unused concepts from codebase
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Erik Johnston 747535f20f Merge pull request #245 from matrix-org/erikj/configurable_client_location
Allow specifying a directory to host a web client from
2015-08-25 15:50:25 +01:00
Erik Johnston 1d1c303b9b Fix typo when using sys.stderr.write 2015-08-25 15:39:16 +01:00
Erik Johnston d33f31d741 Print the correct pip install line when failing due to lack of matrix-angular-sdk 2015-08-25 15:33:23 +01:00
Erik Johnston f63208a1c0 Add utility to parse config and print out a key
Usage:

```
$ python -m synapse.config read server_name -c homeserver.yaml
localhost
```
2015-08-25 15:16:31 +01:00
Erik Johnston 37403ab06c Update the log message 2015-08-25 14:19:09 +01:00
Erik Johnston 8b52fe48b5 Revert previous commit. Instead, always download matrix-angular-sdk as a requirement, but don't complain (when we do check_requirements) if we don't have it when we start synapse. 2015-08-25 14:10:31 +01:00
Erik Johnston d9088c923f Remove dependency on matrix-angular-sdk 2015-08-25 13:34:50 +01:00
Erik Johnston 86cef6a91b Allow specifying a directory to host a web client from 2015-08-25 12:01:23 +01:00
Mark Haines 1c847af28a Merge pull request #243 from matrix-org/markjh/remove_syutil
Replace syutil dependency with smaller, single-purpose libraries
2015-08-25 10:52:16 +01:00
Mark Haines cf8c04948f Fix typo in module imports and package dependencies 2015-08-25 10:42:59 +01:00
Mark Haines 037481a033 Remove autoincrement since we incrementing the ID in the storage layer 2015-08-24 17:48:57 +01:00
Mark Haines 01fc3943f1 Fix indent 2015-08-24 17:18:58 +01:00
Erik Johnston 571ac105e6 Bump version and changelog 2015-08-24 17:10:45 +01:00
Erik Johnston 51c53369a3 Do auth checks *before* persisting the event 2015-08-24 16:38:20 +01:00
Erik Johnston f8f3d72e2b Don't make pushers handle presence/typing events 2015-08-24 16:19:43 +01:00
Mark Haines 78323ccdb3 Remove syutil dependency in favour of smaller single-purpose libraries 2015-08-24 16:17:38 +01:00
Erik Johnston 457970c724 Don't insert events into 'event_*_extremeties' tables if they're outliers 2015-08-23 13:44:23 +01:00
Erik Johnston 1bd1a43073 Actually check if event_id isn't returned by _get_state_groups 2015-08-21 14:30:34 +01:00
Erik Johnston fd5ad0f00e Doc string 2015-08-21 11:45:43 +01:00
Erik Johnston 745b72660a Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop 2015-08-21 11:39:38 +01:00
Erik Johnston 42f12ad92f When logging in fetch user by user_id case insensitively, *unless* there are multiple case insensitive matches, in which case require the exact user_id 2015-08-21 11:38:44 +01:00
Erik Johnston aa3c9c7bd0 Don't allow people to register user ids which only differ by case to an existing one 2015-08-21 10:57:47 +01:00
Erik Johnston 1f7642efa9 Fix bug where we didn't correctly serialize the redacted_because key over federation 2015-08-21 09:36:07 +01:00
Erik Johnston 3e9ee62db0 Add missing param in store.get_state_groups invocation 2015-08-21 09:15:13 +01:00
David Baker 21b71b6d7c Return fully qualified user_id as per spec 2015-08-20 21:54:53 +01:00
Daniel Wagner-Hall c7788685b0 Fix bad merge 2015-08-20 17:43:12 +01:00
Daniel Wagner-Hall 8c74bd8960 Fix indentation 2015-08-20 17:26:52 +01:00
Daniel Wagner-Hall ea570ffaeb Fix flake8 warnings 2015-08-20 17:22:41 +01:00
Daniel Wagner-Hall d5a825edee Merge branch 'auth' into refresh
Conflicts:
	synapse/handlers/register.py
2015-08-20 17:13:33 +01:00
Daniel Wagner-Hall 225c244aba Remove incorrect whitespace 2015-08-20 17:10:10 +01:00
Daniel Wagner-Hall 4e706ec82c Merge branch 'develop' into auth 2015-08-20 16:59:41 +01:00
Daniel Wagner-Hall f90ea3dc73 Correct pynacl version to 0.3.0
0.0.3 was a typo
2015-08-20 16:42:17 +01:00
Daniel Wagner-Hall ce2a7ed6e4 Merge branch 'develop' into auth 2015-08-20 16:28:36 +01:00
Daniel Wagner-Hall e8cf77fa49 Merge branch 'develop' into refresh
Conflicts:
	synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Erik Johnston 3befc9ccc3 Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop 2015-08-20 16:09:27 +01:00
Erik Johnston c141d47a28 Merge pull request #235 from matrix-org/erikj/room_avatars
Add m.room.avatar to default power levels.
2015-08-20 16:07:49 +01:00
Daniel Wagner-Hall 13a6517d89 s/by_token/by_access_token/g
We're about to have two kinds of token, access and refresh
2015-08-20 16:01:29 +01:00
David Baker f764f92647 Remove spurious extra arg to set_password 2015-08-20 15:35:54 +01:00
David Baker ca0d28ef34 Another use of check_password that got missed in the yield fix 2015-08-20 15:35:14 +01:00
Erik Johnston 9b63def388 Add m.room.avatar to default power levels. Change default required power levels of such events to 50 2015-08-20 14:35:40 +01:00
Erik Johnston 9d720223f2 Bump version and changelog 2015-08-20 14:12:01 +01:00
Daniel Wagner-Hall 617501dd2a Move token generation to auth handler
I prefer the auth handler to worry about all auth, and register to call
into it as needed, than to smatter auth logic between the two.
2015-08-20 11:35:56 +01:00
Erik Johnston 099ce4bc38 Merge pull request #231 from matrix-org/erikj/pushers_store_last_token
Push: store the 'last_token' in the db, even if we processed no events
2015-08-20 11:27:31 +01:00
Mark Haines 22346a0ee7 Merge pull request #206 from matrix-org/erikj/generate_presice_thumbnails
Always return a thumbnail of the requested size.
2015-08-20 11:27:15 +01:00
Erik Johnston cbd053bb8f Merge pull request #233 from matrix-org/erikj/canonical_alias
Add server side support for canonical aliases
2015-08-20 11:26:09 +01:00
Daniel Wagner-Hall ade5342752 Merge branch 'auth' into refresh 2015-08-20 11:03:47 +01:00
David Baker 4cf302de5b Comma comma comma comma comma chameleon 2015-08-20 10:31:18 +01:00
David Baker c50ad14bae Merge branch 'develop' into email_login 2015-08-20 10:16:01 +01:00
Mark Haines a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use
Changes for unique emails
2015-08-20 10:04:04 +01:00
Erik Johnston aadb2238c9 Check that the canonical room alias actually points to the room 2015-08-20 09:55:04 +01:00
Daniel Wagner-Hall f9e7493ac2 Merge branch 'develop' into auth 2015-08-19 15:20:09 +01:00
Daniel Wagner-Hall 70e265e695 Re-add whitespace around caveat operators 2015-08-19 14:30:31 +01:00
Erik Johnston daa01842f8 Don't get apservice interested rooms in RoomHandler.get_joined_rooms_for_users 2015-08-19 13:46:03 +01:00
Daniel Wagner-Hall 7f08ebb772 Switch to pymacaroons-pynacl 2015-08-19 13:21:36 +01:00
Erik Johnston d7272f8d9d Add canonical alias to the default power levels 2015-08-19 12:03:09 +01:00
Erik Johnston 78fa346b07 Store the 'last_token' in the db, even if we processed no events 2015-08-19 10:08:31 +01:00
Erik Johnston a45ec7c651 Block on storing the current last_tokens 2015-08-19 10:08:12 +01:00
Erik Johnston 40da1f200d Remove an access token log line 2015-08-19 09:41:07 +01:00
Erik Johnston abc6986a24 Fix regression where we incorrectly responded with a 200 to /login 2015-08-19 09:31:11 +01:00
Daniel Wagner-Hall ce832c38d4 Remove padding space around caveat operators 2015-08-18 17:39:26 +01:00
Erik Johnston e624cdec64 Merge pull request #228 from matrix-org/erikj/_get_state_for_groups
Ensure we never return a None event from _get_state_for_groups
2015-08-18 16:30:17 +01:00
Erik Johnston c3dd2ecd5e Merge pull request #230 from matrix-org/erikj/appservice_auth_entity
Set request.authenticated_entity for application services
2015-08-18 16:30:11 +01:00
Erik Johnston 38a965b816 Merge pull request #227 from matrix-org/erikj/receipts_take2
Re-enable receipts API.
2015-08-18 16:30:04 +01:00
Erik Johnston a82938416d Remove newline because vertical whitespace makes mjark sad 2015-08-18 16:28:13 +01:00
Erik Johnston 0bfdaf1f4f Rejig the code to make it nicer 2015-08-18 16:26:07 +01:00
Erik Johnston a5cbd20001 Merge pull request #225 from matrix-org/erikj/reactor_metrics
Fix pending_calls metric to not lie
2015-08-18 16:21:11 +01:00
Erik Johnston 128ed32e6b Bump size of get_presence_state cache 2015-08-18 15:51:23 +01:00
Daniel Wagner-Hall 3e6fdfda00 Fix some formatting to use tuples 2015-08-18 15:18:50 +01:00
Erik Johnston ee59af9ac0 Set request.authenticated_entity for application services 2015-08-18 15:17:47 +01:00
Daniel Wagner-Hall 1469141023 Merge branch 'develop' into auth 2015-08-18 14:43:44 +01:00
Daniel Wagner-Hall cacdb529ab Remove accidentally added file 2015-08-18 14:27:23 +01:00
Daniel Wagner-Hall 2d3462714e Issue macaroons as opaque auth tokens
This just replaces random bytes with macaroons. The macaroons are not
inspected by the client or server.

In particular, they claim to have an expiry time, but nothing verifies
that they have not expired.

Follow-up commits will actually enforce the expiration, and allow for
token refresh.

See https://bit.ly/matrix-auth for more information
2015-08-18 14:22:02 +01:00
Erik Johnston f704c10f29 Rename unhelpful variable name 2015-08-18 11:54:03 +01:00
Erik Johnston 6e7d36a72c Also check for presence of 'threadCallQueue' in reactor 2015-08-18 11:51:08 +01:00
Erik Johnston d3da63f766 Use more helpful variable names 2015-08-18 11:47:00 +01:00
Erik Johnston 8199475ce0 Ensure we never return a None event from _get_state_for_groups 2015-08-18 11:44:10 +01:00
Erik Johnston 0d4abf7777 Typo 2015-08-18 11:19:08 +01:00
Erik Johnston e55291ce5e None check 2015-08-18 11:17:37 +01:00
Erik Johnston 8e254862f4 Don't assume @cachedList function returns keys for everything 2015-08-18 11:11:33 +01:00
Erik Johnston 85d0bc3bdc Reduce cache size from obscenely large to quite large 2015-08-18 11:00:38 +01:00
Erik Johnston cfc503681f Comments 2015-08-18 10:49:23 +01:00
Erik Johnston dc2a105fca Merge pull request #226 from matrix-org/erikj/room_presence
Add and use cached batched storage.get_state function.
2015-08-18 10:43:50 +01:00
Erik Johnston 83eb627b5a More helpful variable names 2015-08-18 10:33:11 +01:00
Erik Johnston 776ee6d92b Doc strings 2015-08-18 10:30:07 +01:00
Erik Johnston f72ed6c6a3 Remove debug try/catch 2015-08-18 10:29:49 +01:00
Mark Haines 8899df13bf Merge pull request #208 from matrix-org/markjh/end-to-end-key-federation
Federation for end-to-end key requests.
2015-08-18 09:12:54 +01:00
Erik Johnston 8f4165628b Add index receipts_linearized_room_stream 2015-08-17 14:43:54 +01:00
Erik Johnston d3d582bc1c Remove unused import 2015-08-17 13:38:09 +01:00
Erik Johnston 4d8e1e1f9e Remove added unused methods 2015-08-17 13:36:07 +01:00
Erik Johnston afef6f5d16 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2 2015-08-17 13:23:44 +01:00
Erik Johnston 2d97e65558 Remember to invalidate caches 2015-08-17 10:46:55 +01:00
Erik Johnston 1a9510bb84 Implement a batched presence_handler.get_state and use it 2015-08-17 10:40:23 +01:00
Erik Johnston 47abebfd6d Add batched version of store.get_presence_state 2015-08-17 09:50:50 +01:00
Erik Johnston f9d4da7f45 Fix bug where we were leaking None into state event lists 2015-08-17 09:39:45 +01:00
Daniel Wagner-Hall 30883d8409 Merge pull request #221 from matrix-org/auth
Simplify LoginHander and AuthHandler
2015-08-14 17:02:22 +01:00
Erik Johnston 891dfd90bd Fix pending_calls metric to not lie 2015-08-14 15:43:11 +01:00
Erik Johnston 68b255c5a1 Batch _get_linearized_receipts_for_rooms 2015-08-14 15:06:22 +01:00
Mark Haines 95b0f5449d Fix flake8 warning 2015-08-13 17:34:22 +01:00
Erik Johnston 129ee4e149 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2 2015-08-13 17:28:43 +01:00
Mark Haines c5966b2a97 Merge remote-tracking branch 'origin/develop' into markjh/end-to-end-key-federation 2015-08-13 17:27:53 +01:00
Mark Haines 0cceb2ac92 Add a few strategic new lines to break up the on_query_client_keys and on_claim_client_keys methods in federation_server.py 2015-08-13 17:27:46 +01:00
Mark Haines b16cd18a86 Merge remote-tracking branch 'origin/develop' into erikj/generate_presice_thumbnails 2015-08-13 17:23:39 +01:00
Erik Johnston 9f7f228ec2 Remove pointless map 2015-08-13 17:20:59 +01:00
Erik Johnston 2bb2c02571 Remove some vertical space 2015-08-13 17:11:30 +01:00
Mark Haines 3d1cdda762 Merge branch 'develop' into erikj/reactor_metrics 2015-08-13 17:03:58 +01:00
Erik Johnston 57877b01d7 Replace list comprehension 2015-08-13 17:00:17 +01:00
Erik Johnston 5db5677969 Add metrics to the receipts cache 2015-08-13 16:58:23 +01:00
Erik Johnston 7e77a82c5f Re-enable receipts 2015-08-13 16:58:10 +01:00
Erik Johnston 06750140f6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-13 11:55:20 +01:00
Erik Johnston adbd720fab PEP8 2015-08-13 11:47:38 +01:00
Erik Johnston 8b7ce2945b Merge branch 'erikj/reactor_metrics' into erikj/dictionary_cache 2015-08-13 11:42:22 +01:00
Erik Johnston a6c27de1aa Don't time getDelayedCalls 2015-08-13 11:41:57 +01:00
Erik Johnston c044aca1fd Merge branch 'erikj/reactor_metrics' into erikj/dictionary_cache 2015-08-13 11:39:38 +01:00
Erik Johnston ba5d34a832 Add some metrics about the reactor 2015-08-13 11:38:59 +01:00
Mark Haines 21ac8be5f7 Depend on Twisted>=15.1 rather than pining to a particular version 2015-08-12 17:25:13 +01:00
Erik Johnston 0fbed2a8fa Comment 2015-08-12 17:22:54 +01:00
Mark Haines 998a72d4d9 Merge branch 'develop' into markjh/twisted-15
Conflicts:
	synapse/http/matrixfederationclient.py
2015-08-12 17:21:14 +01:00
Erik Johnston c10ac7806e Explain why we're prefilling dict with Nones 2015-08-12 17:16:30 +01:00
Erik Johnston 101ee3fd00 Better variable name 2015-08-12 17:08:05 +01:00
Erik Johnston df361d08f7 Split _get_state_for_group_from_cache into two 2015-08-12 17:06:21 +01:00
Erik Johnston 7b0e797080 Fix _filter_events_for_client 2015-08-12 17:05:24 +01:00
Erik Johnston cfa62007a3 Docstring 2015-08-12 16:42:46 +01:00
Daniel Wagner-Hall 5ce903e2f7 Merge password checking implementations 2015-08-12 16:09:19 +01:00
Erik Johnston a7eeb34c64 Simplify staggered deferred lists 2015-08-12 16:02:05 +01:00
Erik Johnston f7e2f981ea Use list comprehension instead of filter 2015-08-12 16:01:10 +01:00
Daniel Wagner-Hall bcc1d34d35 Merge branch 'develop' into auth 2015-08-12 15:58:52 +01:00
Daniel Wagner-Hall 415c2f0549 Simplify LoginHander and AuthHandler
* Merge LoginHandler -> AuthHandler
 * Add a bunch of documentation
 * Improve some naming
 * Remove unused branches

I will start merging the actual logic of the two handlers shortly
2015-08-12 15:49:37 +01:00
David Baker f43041aacd Check absent before trying to access keys 2015-08-12 15:44:08 +01:00
David Baker 73605f8070 Just leaving off the $ is fine. r* == registerrrrrrrrr 2015-08-12 15:40:54 +01:00
Mark Haines de3b7b55d6 Doc-string for config ultility function 2015-08-12 14:29:17 +01:00
Erik Johnston d46208c12c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-12 14:28:43 +01:00
Erik Johnston 4f11a5b2b5 Merge pull request #220 from matrix-org/markjh/generate_keys
Fix the --generate-keys option.
2015-08-12 14:23:54 +01:00
Mark Haines 7bbaab9432 Fix the --generate-keys option. Make it do the same thing as --generate-config does when the config file exists, but without printing a warning 2015-08-12 11:57:37 +01:00
Mark Haines fdb724cb70 Add config option for setting the list of thumbnail sizes to precalculate 2015-08-12 10:55:27 +01:00
Mark Haines 7e3d1c7d92 Make a config option for whether to generate new thumbnail sizes dynamically 2015-08-12 10:54:38 +01:00
Erik Johnston d7451e0f22 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-12 10:30:30 +01:00
Erik Johnston 4807616e16 Wire up the dictionarycache to the metrics 2015-08-12 10:13:35 +01:00
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Daniel Wagner-Hall 45610305ea Add missing space because linter 2015-08-11 16:43:27 +01:00
Daniel Wagner-Hall 88e03da39f Minor docs cleanup 2015-08-11 16:35:28 +01:00
Daniel Wagner-Hall 9dba813234 Remove redundant if-guard
The startswith("@") does the job
2015-08-11 16:34:17 +01:00
Erik Johnston 53a817518b Comments 2015-08-11 11:40:40 +01:00
Erik Johnston 6eaa116867 Comment 2015-08-11 11:35:24 +01:00
Erik Johnston 4762c276cb Docs 2015-08-11 11:33:41 +01:00
Erik Johnston dc8399ee00 Remove debug loggers 2015-08-11 11:30:59 +01:00
Erik Johnston 1b994a97dd Fix application of ACLs 2015-08-11 10:41:40 +01:00
Erik Johnston 10b874067b Fix state cache 2015-08-11 09:12:41 +01:00
Erik Johnston 017b798e4f Clean up StateStore 2015-08-10 15:01:06 +01:00
Erik Johnston 2c019eea11 Remove unused function 2015-08-10 14:44:41 +01:00
Erik Johnston bb0a475c30 Comments 2015-08-10 14:27:38 +01:00
Erik Johnston dcefac3b06 Comments 2015-08-10 14:16:24 +01:00
Mark Haines 559c51debc Use TypeError instead of ValueError and give a nicer error mesasge
when someone calls Cache.invalidate with the wrong type.
2015-08-10 14:07:17 +01:00
Erik Johnston 7ce71f2ffc Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-10 13:47:51 +01:00
Erik Johnston 8c3a62b5c7 Merge pull request #215 from matrix-org/erikj/cache_varargs_interface
Change Cache to not use *args in its interface
2015-08-10 13:47:45 +01:00
Erik Johnston 86eaaa885b Rename keyargs to args in CacheDescriptor 2015-08-10 13:44:44 +01:00
Erik Johnston e0b6e49466 Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-10 10:39:22 +01:00
Erik Johnston 2cd6cb9f65 Rename keyargs to args in Cache 2015-08-10 10:38:47 +01:00
Erik Johnston aa88582e00 Do bounds check 2015-08-10 10:08:15 +01:00
Erik Johnston 5119e416e8 Line length 2015-08-10 10:05:30 +01:00
Erik Johnston 8f04b6fa7a Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-07 19:30:25 +01:00
Erik Johnston 7dec0b2bee Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-07 19:28:39 +01:00
Erik Johnston 06218ab125 Merge pull request #212 from matrix-org/erikj/cache_deferreds
Make CacheDescriptor cache deferreds rather than the deferreds' values
2015-08-07 19:28:05 +01:00
Erik Johnston 2352974aab Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface 2015-08-07 19:26:54 +01:00
Erik Johnston 9c5385b53a s/observed/observer/ 2015-08-07 19:26:38 +01:00
Erik Johnston ffab798a38 Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface 2015-08-07 19:18:47 +01:00
Erik Johnston 62126c996c Propogate stale cache errors to calling functions 2015-08-07 19:17:58 +01:00
Erik Johnston 3213ff630c Remove unnecessary cache 2015-08-07 19:14:05 +01:00
Erik Johnston 20addfa358 Change Cache to not use *args in its interface 2015-08-07 18:32:47 +01:00
Erik Johnston 9eb5b23d3a Batch up various DB requests for event -> state 2015-08-07 18:16:02 +01:00
Erik Johnston 0211890134 Implement a CacheListDescriptor 2015-08-07 18:14:49 +01:00
Erik Johnston ffdb8c3828 Don't be too enthusiatic with defer.gatherResults 2015-08-07 18:13:48 +01:00
Paul "LeoNerd" Evans 0db40d3e93 Don't complain about extra .pyc files we find while hunting for database schemas 2015-08-07 17:22:11 +01:00
Paul "LeoNerd" Evans e3c8e2c13c Add a --generate-keys option 2015-08-07 16:42:27 +01:00
Paul "LeoNerd" Evans efe60d5e8c Only print the pidfile path on startup if requested by a commandline flag 2015-08-07 16:36:42 +01:00
Erik Johnston b2c7bd4b09 Cache get_recent_events_for_room 2015-08-07 14:42:34 +01:00
Erik Johnston b3768ec10a Remove unncessary cache 2015-08-07 13:41:05 +01:00
Erik Johnston b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston fe994e728f Store absence of state in cache 2015-08-07 10:17:38 +01:00
Erik Johnston 1d08bf7c17 Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 14:03:15 +01:00
Erik Johnston 63b1eaf32c Docs 2015-08-06 14:02:50 +01:00
Erik Johnston b811c98574 Remove failed deferreds from cache 2015-08-06 14:01:27 +01:00
Erik Johnston 433314cc34 Re-implement DEBUG_CACHES flag 2015-08-06 14:01:05 +01:00
Erik Johnston f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston 2efb93af52 Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds 2015-08-06 13:35:28 +01:00
Erik Johnston 953dbd28a7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args 2015-08-06 13:35:03 +01:00
Erik Johnston 7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston 3e1b77efc2 Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 16:45:56 +01:00
Erik Johnston b52b4a84ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 15:41:20 +01:00
Erik Johnston 1e62a3d3a9 Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room' 2015-08-05 15:40:40 +01:00
Erik Johnston a89559d797 Use LRU cache by default 2015-08-05 15:39:47 +01:00
Erik Johnston 07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
David Baker 185ac7ee6c Allow sign in using email address 2015-08-04 16:29:54 +01:00
David Baker a0dea6eaed Remember to yield: not much point testing is a deferred is not None 2015-08-04 16:18:17 +01:00
Erik Johnston c67ba143fa Move DictionaryCache 2015-08-04 15:58:28 +01:00
Erik Johnston e7768e77f5 Add basic dictionary cache 2015-08-04 15:56:56 +01:00
David Baker 883aabe423 splt long line 2015-08-04 15:20:35 +01:00
David Baker e124128542 Bump schema version 2015-08-04 14:50:31 +01:00
David Baker c77048e12f Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server. 2015-08-04 14:37:09 +01:00
Erik Johnston 2e35a733cc Merge branch 'develop' of github.com:matrix-org/synapse into erikj/acl_perf 2015-08-04 13:00:52 +01:00
Erik Johnston 413a4c289b Add comment 2015-08-04 11:08:07 +01:00
Erik Johnston 4d6cb8814e Speed up event filtering (for ACL) logic 2015-08-04 09:32:23 +01:00
David Baker 7148aaf5d0 Don't try & check the username if we don't have one (which we won't if it's been saved in the auth layer) 2015-08-03 17:03:27 +01:00
David Baker 28d07a02e4 Add vector.im as trusted ID server 2015-08-03 15:31:21 +01:00
Kegan Dougal 11b0a34074 Use the same reg paths as register v1 for ASes.
Namely this means using registration_handler.appservice_register.
2015-07-29 10:00:54 +01:00
Kegan Dougal a4d62ba36a Fix v2_alpha registration. Add unit tests.
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
2015-07-28 17:34:12 +01:00
Erik Johnston 39e21ea51c Add support for using keyword arguments with cached functions 2015-07-27 13:57:29 +01:00
Mark Haines 2da3b1e60b Get the end-to-end key federation working 2015-07-24 18:26:46 +01:00
Mark Haines 62c010283d Add federation support for end-to-end key requests 2015-07-23 16:03:38 +01:00
Erik Johnston 459085184c Factor out thumbnail() 2015-07-23 15:59:53 +01:00
Erik Johnston 2b4f47db9c Generate local thumbnails on a thread 2015-07-23 14:52:29 +01:00
Erik Johnston 33d83f3615 Fix remote thumbnailing 2015-07-23 14:24:21 +01:00
Erik Johnston ff7c2e41de Always return a thumbnail of the requested size.
Before, we returned a thumbnail that was at least as big (if possible)
as the requested size. Now, if we don't have a thumbnail of the given
size we generate (and persist) one of that size.
2015-07-23 14:12:49 +01:00
Erik Johnston 103e1c2431 Pick larger than desired thumbnail for 'crop' 2015-07-23 11:12:49 +01:00
Matrix 4e2e67fd50 Disable receipts for now 2015-07-22 16:13:46 +01:00
David Baker a56eccbbfc Query for all the ones we were asked about, not just the last... 2015-07-21 16:38:16 -07:00
David Baker 20c0324e9c Dodesn't seem to make any difference: guess it does work with the object reference 2015-07-21 16:21:37 -07:00
David Baker cf7a40b08a I think this was what was intended... 2015-07-21 16:08:00 -07:00
Erik Johnston 90dbd71c13 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-07-21 09:25:30 +01:00
Mark Haines 3b5823c74d s/take/claim/ for end to end key APIs 2015-07-20 18:23:54 +01:00
Daniel Wagner-Hall 53d1174aa9 Improve naming 2015-07-20 06:32:12 -07:00
Kegan Dougal ddef5ea126 Remove semicolon. 2015-07-20 14:02:36 +01:00
Kegan Dougal b6ee0585bd Parse the ID given to /invite|ban|kick to make sure it looks like a user ID. 2015-07-20 13:55:19 +01:00
Matrix 4f973eb657 Up default cache size for _RoomStreamChangeCache 2015-07-18 19:07:33 +01:00
Matrix 4cab2cfa34 Don't do any database hits in receipt handling if from_key == to_key 2015-07-18 19:07:12 +01:00
Erik Johnston b6d4a4c6d8 Merge pull request #199 from matrix-org/erikj/receipts
Implement read receipts.
2015-07-16 18:18:36 +01:00
Erik Johnston d155b318d2 Merge pull request #203 from matrix-org/erikj/room_creation_presets
Implement presets at room creation
2015-07-16 18:18:11 +01:00
Erik Johnston a2ed7f437c Merge pull request #202 from matrix-org/erikj/power_level_sanity
Change power level semantics.
2015-07-16 18:18:04 +01:00
Erik Johnston c456d17daf Implement specifying custom initial state for /createRoom 2015-07-16 15:25:29 +01:00
David Baker 09489499e7 pep8 + debug line 2015-07-15 19:39:18 +01:00
David Baker 4da05fa0ae Add back in support for remembering parameters submitted to a user-interactive auth call. 2015-07-15 19:28:57 +01:00
Matthew Hodgson 8cedf3ce95 bump up image quality a bit more as it looks crap 2015-07-14 23:53:13 +01:00
Erik Johnston baa55fb69e Merge pull request #193 from matrix-org/erikj/bulk_persist_event
Add bulk insert events API
2015-07-14 10:49:24 +01:00
Erik Johnston 002a44ac1a s/everyone_ops/original_invitees_have_ops/ 2015-07-14 10:37:42 +01:00
David Baker 62b4b72fe4 Close, but no cigar. 2015-07-14 10:33:25 +01:00
Erik Johnston b49a30a972 Capitalize contants 2015-07-14 10:20:31 +01:00
Erik Johnston 4624d6035e Docs 2015-07-14 10:19:07 +01:00
Erik Johnston d5cc794598 Implement presets at room creation 2015-07-13 16:56:08 +01:00
Erik Johnston 5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston 016c089f13 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/power_level_sanity 2015-07-13 13:48:13 +01:00
Erik Johnston e5991af629 Comments 2015-07-13 13:30:43 +01:00
Erik Johnston 17bb9a7eb9 Remove commented out code 2015-07-10 14:07:57 +01:00
Erik Johnston a5ea22d468 Sanitize power level checks 2015-07-10 14:05:38 +01:00
Erik Johnston 7e3b14fe78 You shouldn't be able to ban/kick users with higher power levels 2015-07-10 14:05:38 +01:00
Erik Johnston 532fcc997a Merge pull request #196 from matrix-org/erikj/room_history
Add ability to restrict room history.
2015-07-10 13:47:04 +01:00
Erik Johnston 0b3389bcd2 Merge pull request #194 from matrix-org/erikj/bulk_verify_sigs
Implement bulk verify_signed_json API
2015-07-10 13:46:53 +01:00
Erik Johnston 0d7f0febf4 Uniquely name unique constraint 2015-07-10 13:43:03 +01:00
Erik Johnston b7cb37b189 Merge pull request #198 from matrix-org/markjh/client-end-to-end-key-management
Client end to end key management API
2015-07-10 13:36:17 +01:00
Mark Haines a01097d60b Assume that each device for a user has only one of each type of key 2015-07-10 13:26:18 +01:00
Erik Johnston f3049d0b81 Small tweaks to SAML2 configuration.
- Add saml2 config docs to default config.
- Use existence of saml2 config to indicate if saml2 should be enabled.
2015-07-10 10:50:14 +01:00
Erik Johnston 9158ad1abb Merge pull request #201 from EricssonResearch/msba/saml2-develop
Integrate SAML2 basic authentication - uses pysaml2
2015-07-10 10:25:56 +01:00
Erik Johnston b5f0d73ea3 Add comment 2015-07-09 17:09:26 +01:00
Erik Johnston ed88720952 Handle error slightly better 2015-07-09 16:14:46 +01:00
Erik Johnston f0979afdb0 Remove spurious comment 2015-07-09 16:02:07 +01:00
Mark Haines bf0d59ed30 Don't bother with a timeout for one time keys on the server. 2015-07-09 14:04:03 +01:00
Erik Johnston c2d08ca62a Integer timestamps 2015-07-09 13:15:34 +01:00
Erik Johnston 4019b48aaa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-09 11:55:52 +01:00
Erik Johnston 294dbd712f We don't want semicolons. 2015-07-09 11:47:24 +01:00
Erik Johnston 1af188209a Change format of receipts to allow inclusion of data 2015-07-09 11:39:30 +01:00
Muthu Subramanian 8cd34dfe95 Make SAML2 optional and add some references/comments 2015-07-09 13:34:47 +05:30
Muthu Subramanian d2caa5351a code beautify 2015-07-09 12:58:15 +05:30
Matthew Hodgson fb8d2862c1 remove the tls_certificate_chain_path param and simply support tls_certificate_path pointing to a file containing a chain of certificates 2015-07-09 00:45:41 +01:00
Matthew Hodgson 8ad2d2d1cb document tls_certificate_chain_path more clearly 2015-07-09 00:06:01 +01:00
Matthew Hodgson f26a3df1bf oops, context.tls_certificate_chain_file() expects a file, not a certificate. 2015-07-08 21:33:02 +01:00
Matthew Hodgson 19fa3731ae typo 2015-07-08 18:53:41 +01:00
Matthew Hodgson 465acb0c6a *cough* 2015-07-08 18:30:59 +01:00
Matthew Hodgson 64afbe6ccd add new optional config for tls_certificate_chain_path for folks with intermediary SSL certs 2015-07-08 18:20:02 +01:00
Matthew Hodgson 04192ee05b typo 2015-07-08 17:49:15 +01:00
Mark Haines 8fb79eeea4 Only remove one time keys when new one time keys are added 2015-07-08 17:04:29 +01:00
Erik Johnston ce9e2f84ad Add blist to dependencies 2015-07-08 15:41:59 +01:00
Erik Johnston 304343f4d7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-08 15:37:33 +01:00
Erik Johnston af812b68dd Add a cache to fetching of receipt streams 2015-07-08 15:35:00 +01:00
Erik Johnston d85ce8d89b Split receipt events up into one per room 2015-07-08 11:36:05 +01:00
Muthu Subramanian f53bae0c19 code beautify 2015-07-08 16:05:46 +05:30
Muthu Subramanian 77c5db5977 code beautify 2015-07-08 16:05:20 +05:30
Muthu Subramanian 81682d0f82 Integrate SAML2 basic authentication - uses pysaml2 2015-07-08 15:36:54 +05:30
Erik Johnston 87311d1b8c Hook up receipts to v1 initialSync 2015-07-08 11:02:04 +01:00
Erik Johnston ca041d5526 Wire together receipts and the notifer/federation 2015-07-07 15:25:30 +01:00
Erik Johnston 716e426933 Fix various typos 2015-07-07 10:55:31 +01:00
Erik Johnston e8b2f6f8a1 Add a ReceiptServlet 2015-07-07 10:55:22 +01:00
Mark Haines 28ef344077 Merge branch 'mjark/missing_regex_group' into markjh/client-end-to-end-key-management 2015-07-06 18:48:27 +01:00
Mark Haines 2ef182ee93 Add client API for uploading and querying keys for end to end encryption 2015-07-06 18:47:57 +01:00
Mark Haines b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Mark Haines a7dcbfe430 Don't 500 if a group is missing from the regex 2015-07-06 16:47:17 +01:00
Erik Johnston 1a3255b507 Add m.room.history_visibility to newly created rooms' m.room.power_levels 2015-07-06 13:25:35 +01:00
Erik Johnston fb47c3cfbe Rename key and values for m.room.history_visibility. Support 'invited' value 2015-07-06 13:05:52 +01:00
Erik Johnston 65e69dec8b Don't explode if we don't recognize one of the event_ids in the backfill request 2015-07-06 09:33:03 +01:00
Erik Johnston c3e2600c67 Filter and redact events that the other server doesn't have permission to see during backfill 2015-07-03 17:52:57 +01:00
Erik Johnston 400894616d Respect m.room.history_visibility in v2_alpha sync API 2015-07-03 14:51:01 +01:00
Erik Johnston 12b83f1a0d If user supplies filename in URL when downloading from media repo, use that name in Content Disposition 2015-07-03 11:24:55 +01:00
Erik Johnston 00ab882ed6 Add m.room.history_visibility to list of auth events 2015-07-03 10:31:24 +01:00
Erik Johnston 41938afed8 Make v1 initial syncs respect room history ACL 2015-07-02 17:12:35 +01:00
Erik Johnston 1a60545626 Add basic impl for room history ACL on GET /messages client API 2015-07-02 16:20:10 +01:00
Erik Johnston ac78e60de6 Add stream_id index 2015-07-02 13:18:41 +01:00
Erik Johnston bd1236c0ee Consolidate duplicate code in notifier 2015-07-02 11:46:05 +01:00
Erik Johnston ddf7979531 Add receipts_key to StreamToken 2015-07-02 11:45:44 +01:00
Erik Johnston 0862fed2a8 Add basic ReceiptHandler 2015-07-01 17:19:31 +01:00
Erik Johnston 80a61330ee Add basic storage functions for handling of receipts 2015-07-01 17:19:12 +01:00
Erik Johnston 480d720388 Bump changelog and version to v0.9.3 2015-07-01 15:12:32 +01:00
Erik Johnston 901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston 9beaedd164 Enforce ascii filenames for uploads 2015-06-30 10:31:59 +01:00
Erik Johnston 2124f668db Add Content-Disposition headers to media repo v1 downloads 2015-06-30 09:35:44 +01:00
Erik Johnston f0dd568e16 Wait for previous attempts at fetching keys for a given server before trying to fetch more 2015-06-26 11:25:00 +01:00
Erik Johnston b5f55a1d85 Implement bulk verify_signed_json API 2015-06-26 10:39:34 +01:00
Erik Johnston 5130d80d79 Add bulk insert events API 2015-06-25 17:29:34 +01:00
David Baker 6825eef955 Oops: underride rule had an identifier with override in it. 2015-06-23 14:26:14 +01:00
Erik Johnston 6924852592 Batch SELECTs in _get_auth_chain_ids_txn 2015-06-23 11:01:04 +01:00
Erik Johnston 9c72011fd7 Bumb version 2015-06-23 10:12:19 +01:00
Erik Johnston 2f556e0c55 Fix typo 2015-06-19 16:22:53 +01:00
Erik Johnston 7fa1363fb0 Merge pull request #192 from matrix-org/erikj/fix_log_context
Fix log context when sending requests
2015-06-19 16:21:40 +01:00
Erik Johnston 275dab6b55 Merge pull request #190 from matrix-org/erikj/syn-412
Fix notifier leak
2015-06-19 11:49:58 +01:00
Erik Johnston a68abc79fd Add comment on cancellation of observers 2015-06-19 11:48:55 +01:00
Erik Johnston 653533a3da Fix log context when sending requests 2015-06-19 11:46:49 +01:00
Erik Johnston 9bf61ef97b Merge pull request #189 from matrix-org/erikj/room_init_sync
Improve room init sync speed.
2015-06-19 11:36:06 +01:00
Erik Johnston 0e58d19163 Merge pull request #187 from matrix-org/erikj/sanitize_logging
Sanitize logging
2015-06-19 11:35:59 +01:00
Erik Johnston 18968efa0a Remove stale debug lines 2015-06-19 10:18:02 +01:00
Erik Johnston eb928c9f52 Add site_tag to logger 2015-06-19 10:16:48 +01:00
Erik Johnston 9d112f4440 Add IDs to outbound transactions 2015-06-19 10:13:03 +01:00
Eric Myhre 9e5a353663 Make upload dir a configurable path.
Fixes SYN-425.

Signed-off-by: Eric Myhre <hash@exultant.us>
2015-06-18 23:38:20 -05:00
Erik Johnston 6f6ebd216d PEP8 2015-06-18 17:00:32 +01:00
Erik Johnston 73513ececc Documentation 2015-06-18 16:15:10 +01:00
Erik Johnston 1f24c2e589 Don't bother proxying lookups on _NotificationListener to underlying deferred 2015-06-18 16:09:53 +01:00
Erik Johnston 22049ea700 Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races. 2015-06-18 15:49:24 +01:00
Erik Johnston 050ebccf30 Fix notifier leak 2015-06-18 11:36:26 +01:00
Kegan Dougal d88e20cdb9 Fix bug where synapse was sending AS user queries incorrectly.
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
2015-06-17 17:26:03 +01:00
Erik Johnston eceb554a2f Use another deferred list 2015-06-16 17:12:27 +01:00
Erik Johnston b849a64f8d Use DeferredList 2015-06-16 17:03:24 +01:00
Erik Johnston 0460406298 Don't do unecessary db ops in presence.get_state 2015-06-16 16:59:38 +01:00
Paul "LeoNerd" Evans 9a3cd1c00d Correct -H SERVER_NAME in config-missing complaint message 2015-06-16 16:03:35 +01:00
Erik Johnston fb7def3344 Remove access_token from synapse.rest.client.v1.transactions {get,store}_response logging 2015-06-16 10:09:43 +01:00
Erik Johnston f13890ddce Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sanitize_logging 2015-06-15 18:26:24 +01:00
Erik Johnston aaa749d366 Disable twisted access logging. Move access logging to SynapseRequest object 2015-06-15 18:18:05 +01:00
Erik Johnston bc42ca121f Merge pull request #185 from matrix-org/erikj/listeners_config
Change listener config.
2015-06-15 18:05:58 +01:00
Erik Johnston cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston b5209c5744 Create SynapseRequest that overrides __repr__ to not print access_token 2015-06-15 16:37:04 +01:00
Erik Johnston 44c9102e7a Merge branch 'erikj/listeners_config' into erikj/sanitize_logging 2015-06-15 14:45:52 +01:00
Erik Johnston 7f7ec84d6f Bump version 2015-06-15 14:16:29 +01:00
Erik Johnston f00f8346f1 Make http.server request logging more verbose, but redact access_tokens 2015-06-15 13:37:58 +01:00
Erik Johnston 83f119a84a Log requests and responses sent via http.client 2015-06-15 13:14:12 +01:00
Erik Johnston 9d0326baa6 Remove redundant newline 2015-06-15 11:27:29 +01:00
Erik Johnston 186f61a3ac Document listener config. Remove deprecated config options 2015-06-15 11:25:53 +01:00
Erik Johnston 6c01ceb8d0 Bump version 2015-06-13 12:22:14 +01:00
Erik Johnston 2eda996a63 Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py 2015-06-13 12:21:58 +01:00
Erik Johnston a005b7269a Add backwards compat support for metrics, manhole and webclient config options 2015-06-12 17:44:23 +01:00
Erik Johnston 942e39e87c PEP8 2015-06-12 17:13:54 +01:00
Erik Johnston 9c5fc81c2d Correctly handle x_forwaded listener option 2015-06-12 17:13:23 +01:00
Erik Johnston fd2c07bfed Use config.listeners 2015-06-12 15:33:07 +01:00
Erik Johnston c42ed47660 Fix up create_resource_tree 2015-06-12 11:52:52 +01:00
Erik Johnston 1e365e88bd Bump schema version 2015-06-11 15:50:39 +01:00
Erik Johnston 2d41dc0069 Bump version 2015-06-11 15:49:19 +01:00
Erik Johnston f7f07dc517 Begin changing the config format 2015-06-11 15:48:52 +01:00
David Baker b8690dd840 Catch any exceptions in the pusher loop. Use a lower timeout for pushers so we can see if they're actually still running. 2015-06-05 11:40:22 +01:00
David Baker da84946de4 pep8 2015-06-04 16:43:45 +01:00
David Baker 63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston 5730b20c6d Merge pull request #175 from matrix-org/erikj/thumbnail_thread
Thumbnail images on a seperate thread
2015-06-03 17:26:56 +01:00
Erik Johnston 8047fd2434 Merge pull request #176 from matrix-org/erikj/backfill_auth
Improve backfill.
2015-06-03 17:25:37 +01:00
Erik Johnston 3bbd0d0e09 Merge pull request #180 from matrix-org/erikj/prev_state_context
Don't needlessly compute prev_state
2015-06-03 17:20:56 +01:00
Erik Johnston 9dda396baa Merge pull request #179 from matrix-org/erikj/state_group_outliers
Don't compute EventContext for outliers.
2015-06-03 17:20:40 +01:00
Erik Johnston 13ed3b9985 Merge pull request #178 from matrix-org/erikj/cache_state_groups
Add cache to get_state_groups.
2015-06-03 17:20:33 +01:00
Erik Johnston bd2cf9d4bf Merge pull request #177 from matrix-org/erikj/content_repo_http_client
SYN-403: Make content repository use its own http client.
2015-06-03 17:20:27 +01:00
Erik Johnston d4902a7ad0 Merge pull request #174 from matrix-org/erikj/compress_option
Add config option to disable compression of http responses
2015-06-03 17:18:17 +01:00
Erik Johnston 55bf90b9e4 Don't needlessly compute prev_state 2015-06-03 16:44:24 +01:00
Erik Johnston 53f0bf85d7 Comment 2015-06-03 16:43:40 +01:00
Erik Johnston 1c3d844e73 Don't needlessly compute context 2015-06-03 16:41:51 +01:00
Erik Johnston 0d7d9c37b6 Add cache to get_state_groups 2015-06-03 14:45:55 +01:00
Erik Johnston d8866d7277 Caches should be bound to instances.
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
Erik Johnston 2ef2f6d593 SYN-403: Make content repository use its own http client. 2015-06-03 10:17:37 +01:00
Erik Johnston 3483b78d1a Log where a request came from in federation 2015-06-02 18:15:13 +01:00
Erik Johnston d3ded420b1 Rephrase log line 2015-06-02 16:30:52 +01:00
Erik Johnston 22716774d5 Don't about JSON when warning about content tampering 2015-06-02 16:30:52 +01:00
Erik Johnston 5044e6c544 Thumbnail images on a seperate thread 2015-06-02 15:39:08 +01:00
Erik Johnston 09e23334de Add a timeout 2015-06-02 11:00:37 +01:00
Erik Johnston 02410e9239 Handle the fact we might be missing auth events 2015-06-02 10:58:35 +01:00
Erik Johnston e552b78d50 Add some logging 2015-06-02 10:28:14 +01:00
Erik Johnston fde0da6f19 Correctly look up auth_events 2015-06-02 10:19:38 +01:00
Erik Johnston 3f04a08a0c Don't process events we've already processed. Remember to process state events 2015-06-02 10:11:32 +01:00
Erik Johnston 4bbfbf898e Correctly pass in auth_events 2015-06-01 17:02:23 +01:00
Erik Johnston 6e17463228 Don't explode if we don't have the event 2015-06-01 16:39:43 +01:00
Erik Johnston 522f285f9b Add config option to disable compression of http responses 2015-06-01 13:36:30 +01:00
Mark Haines b8d49be5a1 Merge branch 'develop' into markjh/twisted-15
Conflicts:
	synapse/python_dependencies.py
2015-06-01 10:56:05 +01:00
Mark Haines 90abdaf3bc Use Twisted-15.2.1, Use Agent.usingEndpointFactory rather than implement our own Agent 2015-06-01 10:51:50 +01:00
Mark Haines d70c847b4f Merge pull request #170 from matrix-org/markjh/SYT-8-recaptcha
Allow endpoint for verifying recaptcha to be configured
2015-05-29 15:32:54 +01:00
Erik Johnston d15f166093 Remove log line 2015-05-29 15:03:24 +01:00
Erik Johnston ca580ef862 Don't copy twice 2015-05-29 15:02:55 +01:00
Mark Haines 784aaa53df Merge branch 'develop' into markjh/SYT-8-recaptcha
Conflicts:
	synapse/handlers/auth.py
2015-05-29 13:49:44 +01:00
Erik Johnston 8355b4d074 Bump syutil version 2015-05-29 13:08:43 +01:00
Erik Johnston a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Mark Haines d94590ed48 Add config for setting the recaptcha verify api endpoint, so we can test it in sytest 2015-05-29 12:11:40 +01:00
Erik Johnston afbd3b2fc4 SYN-395: Fix CAPTCHA, don't double decode json 2015-05-28 18:05:00 +01:00
Erik Johnston 79e37a7ecb Correctly pass connection pool parameter 2015-05-28 16:48:53 +01:00
Erik Johnston 0f118e55db Merge pull request #168 from matrix-org/erikj/conn_pool
Make HTTP clients use connection pools.
2015-05-28 16:03:56 +01:00
Erik Johnston dd74436ffd Unused import 2015-05-28 15:47:20 +01:00
Erik Johnston 11f51e6ded Up maxPersistentPerHost count 2015-05-28 15:45:46 +01:00
Erik Johnston 086df80790 Add connection pooling to SimpleHttpClient 2015-05-28 15:43:21 +01:00
Erik Johnston 291e942332 Use connection pool for federation connections 2015-05-28 15:43:21 +01:00
Erik Johnston 31ade3b3e9 Remove a deep copy 2015-05-28 13:45:23 +01:00
Erik Johnston 36b3b75b21 Registration should be disabled by default 2015-05-28 11:01:34 +01:00
Mark Haines 554c63ca60 Iterate over the user_streams not the user_ids 2015-05-26 15:03:49 +01:00
Mark Haines fff7905409 Merge branch 'bugs/SYN-390' into release-v0.9.1 2015-05-26 14:58:49 +01:00
Mark Haines 00dd207f60 Take a dict of the rule, not the rule list 2015-05-26 14:57:48 +01:00
Erik Johnston 2651fd5e24 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.9.1 2015-05-26 11:05:50 +01:00
Mark Haines e7b25a649c Merge pull request #166 from matrix-org/bugs/SYN-390
SYN-390: Don't modify the dictionary returned from the database here either
2015-05-26 10:40:50 +01:00
Mark Haines 804b732aab SYN-390: Don't modify the dictionary returned from the database here either 2015-05-26 10:35:08 +01:00
Erik Johnston 45fffe8cbe Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.9.1 2015-05-26 10:22:41 +01:00
Erik Johnston 9ba3c1ede4 Merge pull request #165 from matrix-org/bugs/SYN-390
SYN-390: Don't modify the dictionary returned from the data store
2015-05-26 10:20:36 +01:00
Mark Haines a0bebeda8b SYN-390: Don't modify the dictionary returned from the data store 2015-05-26 10:14:15 +01:00
Erik Johnston 27e093cbc1 Bump version 2015-05-22 17:03:37 +01:00
Mark Haines d9f60e8dc8 Merge pull request #163 from matrix-org/markjh/presence_list_cache
Add a cache for the presence list
2015-05-22 17:02:23 +01:00
Mark Haines 0e42dfbe22 Merge pull request #164 from matrix-org/markjh/pusher_performance_2
Add a cache for get_push rules for user, fix cache invalidation
2015-05-22 17:01:56 +01:00
Erik Johnston 6eadbfbea0 Remove redundant for loop 2015-05-22 16:12:20 +01:00
Mark Haines 1a9a9abcc7 Add a cache for getting the presence list for a user 2015-05-22 16:11:17 +01:00
Erik Johnston 74b7de83ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-22 16:10:42 +01:00
Mark Haines 36317f3dad Merge pull request #156 from matrix-org/erikj/join_perf
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
Mark Haines 052ac0c8d0 Merge pull request #159 from matrix-org/erikj/metrics_interface_config
Enable changing the interface the metrics listener binds to
2015-05-22 16:09:33 +01:00
Mark Haines 49a2c10279 Merge pull request #157 from matrix-org/markjh/presence_performance
Improve presence performance in loadtest
2015-05-22 16:04:40 +01:00
Mark Haines 5d53c14342 Merge pull request #160 from matrix-org/markjh/appservice_performance
Make the appservice use 'users_in_room' rather than get_room_members …
2015-05-22 16:04:22 +01:00
Mark Haines 106a3051b8 Remove spurious TODO comment 2015-05-22 15:53:03 +01:00
Erik Johnston 284f55a7fb Add doc strings 2015-05-22 15:18:04 +01:00
Erik Johnston 1ce1509989 s/metric_interface/metric_bind_host/ 2015-05-22 14:51:22 +01:00
Erik Johnston 8bb85c8c5a Update log line 2015-05-22 14:48:06 +01:00
Mark Haines c8135f808b Remove unused import 2015-05-22 14:45:46 +01:00
Erik Johnston b21d015c55 Log origin and stats of incoming transactions 2015-05-22 14:44:25 +01:00
Erik Johnston e70e8e053e Add txn_id to some log lines 2015-05-22 14:44:02 +01:00
Erik Johnston 1b446a5d85 Log less lines at INFO level, but include more helpful information 2015-05-22 14:29:57 +01:00
Erik Johnston 59a0682f3e Enable changing the interface the metrics listener binds to 2015-05-22 13:13:07 +01:00
Mark Haines b6adfc59f5 Invalidate the get_latest_event_ids_in_room cache when deleting from event_forward_extremities 2015-05-22 13:01:03 +01:00
Mark Haines f43544eecc Make the appservice use 'users_in_room' rather than get_room_members since it is cached 2015-05-22 11:01:28 +01:00
Mark Haines a04cde613e Add a cache for get_push rules for user, fix cache invalidation 2015-05-22 10:39:45 +01:00
Erik Johnston 4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Erik Johnston f8c2cd129d Bump version 2015-05-21 17:03:30 +01:00
Mark Haines 2043527b9b Don't try to use a txn when not in one, remove spurious debug logging 2015-05-21 16:53:03 +01:00
Mark Haines 53447e9cd3 Add caches for things requested by the pushers 2015-05-21 16:41:39 +01:00
Mark Haines d61ce3f670 Add a cache for get_current_state with state_key 2015-05-21 16:41:39 +01:00
Erik Johnston a910984b58 Actually return something from lambda 2015-05-21 15:58:41 +01:00
Erik Johnston e309b1045d Sort backfill events 2015-05-21 15:57:35 +01:00
Erik Johnston 0180bfe4aa Remove dead code 2015-05-21 15:53:41 +01:00
Erik Johnston 1f3d1d85a9 Only get non-state 2015-05-21 15:52:29 +01:00
Erik Johnston 39a3340f73 Skip events we've already seen 2015-05-21 15:48:56 +01:00
Erik Johnston ae3bff3491 Correctly prepopulate queue 2015-05-21 15:46:07 +01:00
Erik Johnston dc085ddf8c Don't prepopulate event_results 2015-05-21 15:44:05 +01:00
Erik Johnston 73d23c6ae8 Don't readd things that are already in event_results 2015-05-21 15:40:22 +01:00
Erik Johnston 6189d8e54d PriorityQueue gives lowest first 2015-05-21 15:38:08 +01:00
Erik Johnston 115ef3ddac Correctly capture Queue.Empty exception 2015-05-21 15:37:43 +01:00
Erik Johnston 4fb858d90a Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-21 15:25:54 +01:00
Mark Haines 88f1ea36ce Oops, get_rooms_for_user returns a namedtuple, not a room_id 2015-05-21 15:23:58 +01:00
Erik Johnston c2633907c5 Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-21 14:58:47 +01:00
Erik Johnston ebfdd2eb5b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-21 14:54:52 +01:00
Erik Johnston a551c5dad7 Merge pull request #155 from matrix-org/erikj/perf
Bulk and batch retrieval of events.
2015-05-21 14:54:40 +01:00
Erik Johnston 27e4b45c06 s/for events/for requests for events/ 2015-05-21 14:52:23 +01:00
Erik Johnston ac5f2bf9db s/for events/for requests for events/ 2015-05-21 14:50:57 +01:00
Erik Johnston 80a167b1f0 Add comments 2015-05-21 11:19:04 +01:00
Mark Haines 7ae8afb7ef Removed unused 'is_visible' method 2015-05-20 14:48:11 +01:00
Erik Johnston 9118a92862 Split up _get_events into defer and txn versions 2015-05-20 13:27:16 +01:00
Mark Haines e01b825cc9 Clean up the presence_list checking logic a bit 2015-05-20 13:21:59 +01:00
Erik Johnston ab45e12d31 Make not return a deferred _get_event_from_row_txn 2015-05-20 13:07:19 +01:00
Erik Johnston f407cbd2f1 PEP8 2015-05-20 13:02:01 +01:00
Erik Johnston 227f8ef031 Split out _get_event_from_row back into defer and _txn version 2015-05-20 13:00:57 +01:00
Erik Johnston 2bc60c55af Fix _get_backfill_events to return events in the correct order 2015-05-20 12:57:00 +01:00
Erik Johnston 20814fabdd Actually fetch state for new backwards extremeties when backfilling. 2015-05-20 11:59:02 +01:00
Erik Johnston 9084cdd70f Ensure event_results is a set 2015-05-19 16:34:31 +01:00
Erik Johnston 3a653515ec Add None check 2015-05-19 15:27:09 +01:00
Erik Johnston aa729349dd Fix event_backwards_extrem insertion to ignore outliers 2015-05-19 15:27:00 +01:00
Erik Johnston 5b1631a4a9 Add a timeout param to get_event 2015-05-19 14:53:32 +01:00
Erik Johnston 291cba284b Handle the case when things return empty but non none things 2015-05-19 14:42:46 +01:00
Erik Johnston 253f76a0a5 Don't always hit get_server_verify_key_v1_direct 2015-05-19 14:42:38 +01:00
Erik Johnston 6837c5edab Handle the case when things return empty but non none things 2015-05-19 14:27:11 +01:00
Erik Johnston 7223129916 Don't apply new room join hack if depth > 5 2015-05-19 14:16:08 +01:00
Erik Johnston 5ae4a84211 Don't always hit get_server_verify_key_v1_direct 2015-05-19 13:43:34 +01:00
Erik Johnston 118a760719 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 13:20:29 +01:00
David Baker 19505e0392 Disable GZip encoding on static file resources as per comment 2015-05-19 13:20:25 +01:00
Erik Johnston d3e09f12d0 SYN-383: Actually, we expect this value to be a dict 2015-05-19 13:12:41 +01:00
Erik Johnston 677be13ffc Revert accidental commit 2015-05-19 13:12:28 +01:00
Erik Johnston 350b88656a SYN-383: Actually, we expect this value to be a dict 2015-05-19 13:01:57 +01:00
Erik Johnston 9de94d5a4d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 12:50:17 +01:00
Erik Johnston 2b7120e233 SYN-383: Handle the fact the server might not have signed things 2015-05-19 12:49:38 +01:00
Erik Johnston 8b256a7296 Don't reuse var names 2015-05-19 11:58:22 +01:00
Erik Johnston 62ccc6d95f Don't reuse var names 2015-05-19 11:58:04 +01:00
Erik Johnston 01858bcbf2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 11:56:35 +01:00
Erik Johnston 2aeee2a905 SYN-383: Fix parsing of verify_keys and catching of _DefGen_Return 2015-05-19 11:56:18 +01:00
Erik Johnston 5e7883ec19 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 10:50:43 +01:00
Mark Haines c6a03c46e6 SYN-383: Extract the response list from 'server_keys' in the response JSON as it might work better than iterating over the top level dict 2015-05-19 10:23:02 +01:00
Mark Haines e4c65b338d Speed up the get_pagination_rows as well 2015-05-18 18:21:06 +01:00
Mark Haines 99914ec9f8 Merge pull request #152 from matrix-org/notifier_performance
Notifier performance
2015-05-18 17:49:59 +01:00
Erik Johnston ef910a0358 Do work in parellel when joining a room 2015-05-18 17:17:04 +01:00
Mark Haines 591c4bf223 Cache the most recent serial for each room 2015-05-18 16:21:51 +01:00
Mark Haines e1150cac4b Move updating the serial and state of the presence cache into a single function 2015-05-18 15:46:37 +01:00
Erik Johnston 165eb2dbe6 Comments and shuffle of functions 2015-05-18 15:18:41 +01:00
Mark Haines 880fb46de0 Merge branch 'notifier_performance' into markjh/presence_performance 2015-05-18 14:33:58 +01:00
Erik Johnston 65878a2319 Remove unused metric 2015-05-18 14:06:30 +01:00
Mark Haines ad31fa3040 Don't bother sorting by the room_stream_ids, it shouldn't matter which order they are notified in 2015-05-18 14:04:58 +01:00
Erik Johnston 4d1b6f4ad1 Remove rejected events if we don't want rejected events 2015-05-18 14:03:46 +01:00
Mark Haines 0b0033c40b Merge branch 'develop' into notifier_performance 2015-05-18 13:50:01 +01:00
Mark Haines 755def8083 Add more doc string, reduce C+P boilerplate for getting room list 2015-05-18 13:46:47 +01:00
Mark Haines 1e90715a3d Make sure the notifier stream token goes forward when it is updated. Sort the pending events by the correct room_stream_id 2015-05-18 13:17:36 +01:00
Erik Johnston 131bdf9bb1 Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf 2015-05-18 10:23:37 +01:00
Erik Johnston 10f1bdb9a2 Move get_events functions to storage.events 2015-05-18 10:21:40 +01:00
Erik Johnston d5cea26d45 Remove pointless newline 2015-05-18 10:16:45 +01:00
Erik Johnston c71176858b Newline, remove debug logging 2015-05-18 10:11:14 +01:00
Erik Johnston f8bd4de87d Remove debug logging 2015-05-18 09:58:03 +01:00
Erik Johnston c3b37abdfd PEP8 2015-05-15 16:59:58 +01:00
Erik Johnston 6c74fd62a0 Revert limiting of fetching, it didn't help perf. 2015-05-15 16:45:35 +01:00
Erik Johnston 9ff7f66a2b init j 2015-05-15 16:36:03 +01:00
Erik Johnston 70f272f71c Don't completely drain the list 2015-05-15 16:34:17 +01:00
Erik Johnston 8763dd80ef Don't fetch prev_content for current_state 2015-05-15 15:33:01 +01:00
Erik Johnston 807229f2f2 Err, defer.gatherResults ftw 2015-05-15 15:20:29 +01:00
Erik Johnston acb12cc811 Make store.get_current_state fetch events asyncly 2015-05-15 15:20:05 +01:00
Erik Johnston d62dee7eae Remove more debug logging 2015-05-15 15:06:37 +01:00
Erik Johnston 0f29cfabc3 Remove debug logging 2015-05-15 14:06:42 +01:00
Erik Johnston e275a9c0d9 preserve log context 2015-05-15 11:54:51 +01:00
Erik Johnston aa32bd38e4 Add a wait 2015-05-15 11:35:04 +01:00
Erik Johnston 372d4c6d7b Srsly. Don't use closures. Baaaaaad 2015-05-15 11:26:00 +01:00
Erik Johnston 575ec91d82 Correctly pass through params 2015-05-15 11:15:10 +01:00
Mark Haines 10be983f2c Merge pull request #153 from matrix-org/markjh/presence_docstring
Add some doc strings for presence.
2015-05-15 11:11:47 +01:00
Mark Haines 415b158ce2 More whitespace 2015-05-15 11:09:47 +01:00
Erik Johnston de01438a57 Sort out error handling 2015-05-15 11:00:50 +01:00
Erik Johnston a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Mark Haines 0a4330cd5d Add some missed argument types, cleanup the whitespace a bit 2015-05-14 17:48:12 +01:00
Mark Haines 47ec693e29 More doc-strings 2015-05-14 17:07:02 +01:00
Erik Johnston 1d566edb81 Remove race condition 2015-05-14 16:54:35 +01:00
David Baker 6e1ad283cf Support gzip encoding for client, client v2 and web client resources (SYN-176). 2015-05-14 16:39:19 +01:00
Erik Johnston ef3d8754f5 Call from right thread 2015-05-14 15:41:55 +01:00
Erik Johnston 142934084a Count and loop 2015-05-14 15:40:21 +01:00
Erik Johnston 96c5b9f87c Don't start up more fetch_events 2015-05-14 15:36:04 +01:00
Erik Johnston 7cd6a6f6cf Awful idea for speeding up fetching of events 2015-05-14 15:34:02 +01:00
Mark Haines c5d1b4986b Remove unused arguments and doc PresenceHandler.push_update_to_clients 2015-05-14 14:59:31 +01:00
Erik Johnston 7f4105a5c9 Turn off preemptive transactions 2015-05-14 14:51:06 +01:00
Erik Johnston f4d58deba1 PEP8 2015-05-14 14:45:42 +01:00
Erik Johnston 386b7330d2 Move from _base to events 2015-05-14 14:45:22 +01:00
Mark Haines 0ad1c67234 Add some doc-strings to notifier 2015-05-14 14:35:07 +01:00
Erik Johnston 7d6a1dae31 Jump out early 2015-05-14 14:27:58 +01:00
Erik Johnston 656223fbd3 Actually, we probably want to run this in a transaction 2015-05-14 14:26:35 +01:00
David Baker 67800f7626 Treat setting your display name to the empty string as removing it (SYN-186). 2015-05-14 14:19:59 +01:00
Erik Johnston 2f7f8e1c2b Preemptively jump into a transaction if we ask for get_prev_content 2015-05-14 14:17:36 +01:00
Mark Haines 4770cec7bc Merge pull request #150 from matrix-org/notifier_unify
Make v1 and v2 client APIs interact with the notifier in the same way.
2015-05-14 14:16:59 +01:00
Erik Johnston e1e9f0c5b2 loop -> gatherResults 2015-05-14 13:58:49 +01:00
Erik Johnston ab78a8926e Err, we probably want a bigger limit 2015-05-14 13:47:16 +01:00
Erik Johnston f6f902d459 Move fetching of events into their own transactions 2015-05-14 13:45:48 +01:00
Erik Johnston cdb3757942 Refactor _get_events 2015-05-14 13:31:55 +01:00
David Baker 92e1c8983d Disallow whitespace in aliases here too 2015-05-14 13:21:55 +01:00
David Baker 0c894e1ebd Throw error when creating room if alias contains whitespace #SYN-335 2015-05-14 13:11:28 +01:00
Mark Haines 084c365c3a Use the current token when timing out a notifier, make sure the user_id is a string in on_new_user_event 2015-05-14 12:03:26 +01:00
David Baker c37a6e151f Make shared secret registration work again 2015-05-14 12:03:13 +01:00
Erik Johnston 36ea26c5c0 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf 2015-05-14 12:01:38 +01:00
David Baker 7c549dd557 Add ID generator for push_rules_enable to #resolve SYN-378 2015-05-14 11:44:03 +01:00
Mark Haines 899d4675dd Merge branch 'notifier_unify' into notifier_performance 2015-05-14 11:36:44 +01:00
Mark Haines 243c56e725 Merge branch 'develop' into notifier_unify 2015-05-14 11:36:23 +01:00
Mark Haines 3edd2d5c93 Fix v2 sync, update the last_notified_ms only if there was an active listener 2015-05-14 11:25:30 +01:00
Erik Johnston 4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines 5e0c533672 Fix metric counter 2015-05-13 17:20:28 +01:00
Erik Johnston 968b01a91a Actually use async method 2015-05-13 17:02:46 +01:00
Erik Johnston 4071f29653 Fetch events from events_id in their own transactions 2015-05-13 16:59:41 +01:00
Mark Haines f1b83d88a3 Discard unused NotifierUserStreams 2015-05-13 16:54:02 +01:00
Erik Johnston a988361aea Typo 2015-05-13 15:44:15 +01:00
Erik Johnston 8888982db3 Don't insert None 2015-05-13 15:43:32 +01:00
Mark Haines 9af432257d Don't set a timer if there's already a result to return 2015-05-13 15:42:13 +01:00
Erik Johnston cf706cc6ef Don't return None 2015-05-13 15:31:25 +01:00
Erik Johnston 5971d240d4 Limit batch size 2015-05-13 15:26:49 +01:00
Erik Johnston ca4f458787 Fetch events in bulk 2015-05-13 15:13:42 +01:00
Mark Haines df6db5c802 Don't bother checking for new events from a source if the stream token hasn't advanced for that source 2015-05-13 15:08:24 +01:00
Erik Johnston 6edff11a88 Don't fetch redaction and rejection stuff for each event, so we can use index only scan 2015-05-13 14:39:05 +01:00
Mark Haines 63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston 02590c3e1d Temp turn off checking for rejections and redactions 2015-05-13 11:31:28 +01:00
Erik Johnston 619a21812b defer.gatherResults loop 2015-05-13 11:29:03 +01:00
Erik Johnston fec4485e28 Batch fetching of events for state groups 2015-05-13 11:22:42 +01:00
Erik Johnston 409bcc76bd Load events for state group seperately 2015-05-13 11:13:31 +01:00
Mark Haines cffe6057fb Merge branch 'notifier_unify' into notifier_performance
Conflicts:
	synapse/notifier.py
2015-05-12 16:37:50 +01:00
Erik Johnston 80fd2b574c Don't talk to yourself when backfilling 2015-05-12 16:19:46 +01:00
Erik Johnston e122685978 You need to call contextmanager 2015-05-12 16:12:37 +01:00
Mark Haines d7b3ac46f8 Revert "Improvement to performance of presence event stream handling" 2015-05-12 15:44:21 +01:00
Mark Haines 4429e4bf24 Merge branch 'develop' into notifier_unify
Conflicts:
	synapse/notifier.py
2015-05-12 15:31:26 +01:00
Mark Haines ec07dba29e Merge pull request #143 from matrix-org/erikj/SYN-375
SYN-375 - Lots of unhandled deferred exceptions.
2015-05-12 15:25:54 +01:00
Mark Haines c167cbc9fd Merge pull request #147 from matrix-org/presence-performance
Improvement to performance of presence event stream handling
2015-05-12 15:24:54 +01:00
Mark Haines a6fb2aa2a5 Merge pull request #144 from matrix-org/erikj/logging_context
Preserving logging contexts
2015-05-12 15:23:50 +01:00
Mark Haines 1fce36b111 Merge pull request #149 from matrix-org/erikj/backfill
Backfill support
2015-05-12 15:20:32 +01:00
Erik Johnston 8b28209c60 Err, delete the right stuff 2015-05-12 15:02:53 +01:00
Erik Johnston 30c72d377e Newlines 2015-05-12 14:47:40 +01:00
Erik Johnston e4eddf9b36 We do actually want to delete rows out of event_backward_extremities 2015-05-12 14:47:23 +01:00
Erik Johnston c1779a79bc Fix up _handle_prev_events to not try to insert duplicate rows 2015-05-12 14:41:50 +01:00
Erik Johnston 74850d7f75 Do state groups persistence /after/ checking if we have already persisted the event 2015-05-12 14:14:58 +01:00
Erik Johnston 07a1223156 s/backfil/backfill/ 2015-05-12 14:09:54 +01:00
Erik Johnston 0d31ad5101 Typos everywhere 2015-05-12 14:02:01 +01:00
Erik Johnston a0dfffb33c And another typo. 2015-05-12 14:00:31 +01:00
Erik Johnston 6e5ac4a28f Err, gatherResults doesn't take a dict... 2015-05-12 13:58:14 +01:00
Erik Johnston 8022b27fc2 Make distributer.fire work as it did 2015-05-12 13:14:48 +01:00
Erik Johnston 95dedb866f Unwrap defer.gatherResults failures 2015-05-12 13:14:29 +01:00
Mark Haines 78672a9fd5 Merge branch 'notifier_unify' into notifier_performance 2015-05-12 13:11:54 +01:00
Erik Johnston da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Mark Haines 2551b6645d Update the end_token correctly, otherwise the token doesn't advance and the client gets duplicate events 2015-05-12 11:54:18 +01:00
Mark Haines 5e4ba463b7 Merge branch 'develop' into notifier_unify 2015-05-12 11:41:53 +01:00
Mark Haines 5002056b16 SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes. 2015-05-12 11:20:40 +01:00
Mark Haines 5c75adff95 Add a NotifierUserStream to hold all the notification listeners for a user 2015-05-12 11:00:37 +01:00
Erik Johnston 367382b575 Handle the case where the other side is unreachable when backfilling 2015-05-12 10:35:45 +01:00
Erik Johnston 4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston 84e6b4001f Initial hack at wiring together pagination and backfill 2015-05-11 18:01:31 +01:00
Erik Johnston 17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Mark Haines e269c511f6 Don't bother passing the events to the notifier since it isn't using them 2015-05-11 15:01:51 +01:00
Mark Haines 5e3b254dc8 Use wait_for_events to implement 'get_events' 2015-05-11 14:37:33 +01:00
Erik Johnston 79b7154454 Merge pull request #146 from matrix-org/erikj/push_rules_fixes
Fix 500 on push rule updates.
2015-05-11 11:33:47 +01:00
Erik Johnston 4ef556f650 Bump version 2015-05-11 11:31:04 +01:00
Erik Johnston b036596b75 Prefer to use _simple_*. 2015-05-11 11:24:01 +01:00
Erik Johnston cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston de87541862 Bump version 2015-05-10 10:51:08 +01:00
Erik Johnston 22d2f498fa Fix push rule bug: can't insert bool into small int column 2015-05-10 10:50:51 +01:00
Erik Johnston 2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston da1aa07db5 Add some docs 2015-05-08 16:52:49 +01:00
Erik Johnston 4ac1941592 PEP8 2015-05-08 16:33:01 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston fca28d243e Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors' 2015-05-08 16:28:08 +01:00
Erik Johnston 0cd1401f8d Bump version 2015-05-08 16:11:51 +01:00
Erik Johnston 724bb1e7d9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-08 16:11:19 +01:00
Mark Haines 1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines 9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines 84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Mark Haines 6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston 126d562576 Bump version 2015-05-08 13:29:37 +01:00
Paul "LeoNerd" Evans 45543028bb Use the presence cachemap ordering to early-abort the iteration loop 2015-05-07 22:40:10 +01:00
Paul "LeoNerd" Evans f683b5de47 Store presence cachemap in an ordered dict, so that the newer serials will be at the end 2015-05-07 21:27:53 +01:00
Erik Johnston db0dca2f6f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-07 19:21:00 +01:00
Erik Johnston a151693a3b Bump syweb version 2015-05-07 18:01:46 +01:00
Mark Haines dfa98f911b revert accidental bcrypt gensalt round reduction from loadtesting 2015-05-07 17:45:42 +01:00
Mark Haines ef8e8ebd91 pynacl-0.3.0 was released so we can finally start using it directly from pypi 2015-05-07 16:46:51 +01:00
David Baker 97a64f3ebe Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-05-07 09:33:42 +01:00
David Baker b850c9fa04 Typo 2015-05-07 09:33:30 +01:00
Mark Haines 4a7a4a5b6c Optional profiling using cProfile 2015-05-06 17:08:00 +01:00
Mark Haines b994fb2b96 Don't read from the config file before checking it exists 2015-05-06 12:56:47 +01:00
Erik Johnston 04c01882fc Bump version 2015-05-06 09:59:13 +01:00
Erik Johnston 0c4ac271ca Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371 2015-05-05 18:21:19 +01:00
Erik Johnston 0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Erik Johnston 977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Mark Haines 31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369
Fix race with cache invalidation. SYN-369
2015-05-05 17:46:13 +01:00
Mark Haines deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines e45b05647e Fix the --help option for synapse 2015-05-05 17:39:59 +01:00
Erik Johnston 3d5a955e08 Missed events are not outliers 2015-05-05 17:36:57 +01:00
Mark Haines d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston 9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines 041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines 63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston 531d7955fd Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely 2015-05-05 16:12:28 +01:00
Mark Haines bfa4a7f8b0 Invalidate the room_member cache if the current state events updates 2015-05-05 15:43:49 +01:00
Mark Haines d0fece8d3c Missing return for when the event was already persisted 2015-05-05 15:39:09 +01:00
Erik Johnston bdcd7693c8 Fix indentation 2015-05-05 15:14:48 +01:00
Erik Johnston 43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston 1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Mark Haines 261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston d9cc5de9e5 Correctly name transaction 2015-05-05 10:24:10 +01:00
Erik Johnston b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
Erik Johnston 1942382246 Don't log enqueue_ 2015-05-01 16:14:25 +01:00
David Baker eb9bd2d949 user_id now in user_threepids 2015-05-01 15:04:37 +01:00
Erik Johnston 2d386d7038 That wasn't a deferred 2015-05-01 14:41:25 +01:00
Erik Johnston 4ac2823b3c Remove inlineCallbacks from non-generator 2015-05-01 14:41:25 +01:00
Erik Johnston 22c7c5eb8f Typo 2015-05-01 14:41:25 +01:00
Erik Johnston 42c12c04f6 Remove some run_on_reactors 2015-05-01 14:41:25 +01:00
Erik Johnston adb5b76ff5 Don't log all auth events every time we call auth.check 2015-05-01 14:41:25 +01:00
Mark Haines 3bcdf3664c Use the daemonize key from the config if it exists 2015-05-01 14:34:55 +01:00
David Baker 9eeb03c0dd Don't use self.execute: it's designed for fetching stuff 2015-05-01 14:21:25 +01:00
Mark Haines 7b50769eb9 Merge pull request #136 from matrix-org/markjh/config_cleanup
Config restructuring.
2015-05-01 14:04:39 +01:00
David Baker 7693f24792 No id field on user 2015-05-01 13:55:42 +01:00
Mark Haines 46a65c282f Allow generate-config to run against an existing config file to generate default keys 2015-05-01 13:54:38 +01:00
David Baker 92b20713d7 More missed get_user_by_id API changes 2015-05-01 13:45:54 +01:00
Erik Johnston da4ed08739 One too many lens 2015-05-01 13:29:38 +01:00
Erik Johnston 9060dc6b59 Change public room list to use defer.gatherResults 2015-05-01 13:28:36 +01:00
David Baker 1fae1b3166 This api now no longer returns an array 2015-05-01 13:26:41 +01:00
Erik Johnston 80b4119279 Don't wait for storage of access_token 2015-05-01 13:14:05 +01:00
Erik Johnston 4011cf1c42 Cache latest_event_ids_in_room 2015-05-01 13:06:26 +01:00
Erik Johnston 657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston 23c639ff32 Split a storage function in two so that we don't have to do extra work. 2015-05-01 10:17:19 +01:00
Erik Johnston 8be5284e91 Remove pointless join 2015-04-30 18:49:26 +01:00
Erik Johnston 503e4d3d52 Fix broken SQL 2015-04-30 18:44:47 +01:00
Erik Johnston 00718ae7a9 Need more yield 2015-04-30 18:43:39 +01:00
Erik Johnston 0465560c1a Add missing param 2015-04-30 18:42:44 +01:00
Erik Johnston 61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston 6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Mark Haines 50c87b8eed Allow "manhole" to be ommited from the config 2015-04-30 18:11:47 +01:00
Mark Haines 345995fcde Remove the ~, comment the lines instead 2015-04-30 18:10:19 +01:00
Mark Haines 62cebee8ee Update key.py 2015-04-30 17:54:01 +01:00
Mark Haines 95cbfee8ae Update metrics.py 2015-04-30 17:52:20 +01:00
Mark Haines 6ea9cf58be missing import 2015-04-30 17:21:21 +01:00
Erik Johnston f383d5a801 Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction 2015-04-30 17:12:52 +01:00
Mark Haines c95480963e read the pid_file from the config file in synctl 2015-04-30 17:12:15 +01:00
Mark Haines 2d4d2bbae4 Merge branch 'develop' into markjh/config_cleanup
Conflicts:
	synapse/config/captcha.py
2015-04-30 16:54:55 +01:00
Mark Haines 2f1348f339 Write a default log_config when generating config 2015-04-30 16:52:57 +01:00
Erik Johnston 69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
David Baker 5b02f33451 Undo changes to logger config, ie. remove the access_log_file option: decision is to support this through log_config rather tan adding an option. 2015-04-30 16:21:39 +01:00
David Baker 054aa0d58c Do access log using python's logging stuff, just under a separate logger name 2015-04-30 16:21:38 +01:00
Erik Johnston 3c4c229788 Don't use sub queries, it makes postgres sad 2015-04-30 16:16:53 +01:00
Mark Haines 74aaacf82a Don't break when sizes or durations are given as integers 2015-04-30 16:04:02 +01:00
Mark Haines 29400b45b9 SYN-367: Use upsert rather than insert_or_replace 2015-04-30 15:21:31 +01:00
Mark Haines c28f1d16f0 Add a random string to the auto generated key id 2015-04-30 15:13:14 +01:00
Mark Haines 265f30bd3f Allow --enable-registration to be passed on the commandline 2015-04-30 15:04:06 +01:00
Mark Haines c9e62927f2 Use disable_registration keys if they are present 2015-04-30 14:34:09 +01:00
Erik Johnston 2366d28780 Don't needlessly join on state_events 2015-04-30 14:02:06 +01:00
David Baker d89a9f7283 Add an access_log
SYN-161 #resolve
2015-04-30 13:58:13 +01:00
Mark Haines 1aa11cf7ce Allow multiple config files, set up a default config before applying the config files 2015-04-30 13:48:15 +01:00
David Baker 0c1b7f843b Unused import 2015-04-30 13:33:30 +01:00
David Baker 4b46fbec5b Doesn't look like this is used anymore 2015-04-30 12:04:08 +01:00
Erik Johnston 1d7702833d Make simple query rather than long one and then throw away half the results 2015-04-30 10:16:12 +01:00
Mark Haines 6b69ddd17a remove duplicate parse_size method 2015-04-30 04:26:29 +01:00
Mark Haines d624e2a638 Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class 2015-04-30 04:24:44 +01:00
Erik Johnston b1ca784aca Correctly decode, for sqlite and postgres, rows from pushers table 2015-04-29 19:41:14 +01:00
Erik Johnston 4a9dc5b2f5 pushkey' are also bytes. 2015-04-29 19:27:02 +01:00
Erik Johnston 0ade2712d1 Typo 2015-04-29 19:17:25 +01:00
Erik Johnston 50f96f256f Also remove yield from within lock in the other generator 2015-04-29 19:17:00 +01:00
Erik Johnston d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston 3e71d13acf Also log when we've started pushers 2015-04-29 18:37:42 +01:00
Erik Johnston e7a6edb0ee Revert previous 2015-04-29 18:37:30 +01:00
Erik Johnston c27d6ad6b5 Only start pushers when synapse has fully started 2015-04-29 18:25:24 +01:00
Erik Johnston 46daf2d200 Start pushers on reactor thread 2015-04-29 18:22:20 +01:00
Erik Johnston 3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston 0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston 09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston 472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
Erik Johnston a6e62cf6d0 Fix off by one in presence token handling 2015-04-29 17:37:11 +01:00
David Baker 12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker 61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker 4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston 0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston 884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
David Baker 9927170787 Accept camelcase + underscores in binding too 2015-04-29 15:57:09 +01:00
David Baker 109c8aafd2 Fix includes 2015-04-29 15:45:44 +01:00
David Baker b7788f80a3 Accept both camelcase and underscore threepid creds for transition 2015-04-29 15:41:29 +01:00
Erik Johnston c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Mark Haines f2d90d5c02 Fix whitespace 2015-04-29 14:53:23 +01:00
Mark Haines 845b0b2c97 Check requirements before doing anything else when running the homeserver 2015-04-29 14:52:42 +01:00
Mark Haines c0036ced54 bump syutil to 0.0.6 2015-04-29 14:16:41 +01:00
Erik Johnston 970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Erik Johnston 64991b0c8b Merge pull request #129 from matrix-org/key_distribution
Key distribution v2
2015-04-29 13:34:38 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines 1319905d7a Use a defer.gatherResults to collect results from the perspective servers 2015-04-29 13:31:14 +01:00
Mark Haines a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines 4ad8b45155 Merge branch 'develop' into key_distribution
Conflicts:
	synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Mark Haines 74874ffda7 Update the query format used by keyring to match current key v2 spec 2015-04-29 12:14:08 +01:00
Erik Johnston cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston 4932a7e2d9 Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__ 2015-04-29 12:12:25 +01:00
Mark Haines 9894da6a29 Merge branch 'develop' into erikj/postgres_charset_check 2015-04-29 11:58:06 +01:00
Mark Haines 46d200a3a1 Implement minimum_valid_until_ts in the remote key resource 2015-04-29 11:57:26 +01:00
Erik Johnston a08bf11138 Appease PEP8 2015-04-29 11:44:48 +01:00
Erik Johnston 204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Mark Haines f4c9ebbc34 Delete ugly commented out log line. 2015-04-29 11:07:13 +01:00
Matthew Hodgson 68c0603946 comment out ugly test logline 2015-04-29 00:14:44 +01:00
Erik Johnston 1783c7ca92 Ensure we never miss any presence updates 2015-04-28 17:24:24 +01:00
Erik Johnston 0126ef7f3c Fix typo 2015-04-28 17:23:53 +01:00
Erik Johnston d98edb548a Ensure the serial returned by presence is always an integer 2015-04-28 17:20:32 +01:00
Erik Johnston 073b891ec1 Remove unused imports 2015-04-28 13:44:23 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines 9182f87664 Merge pull request #126 from matrix-org/csauth
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines 55e1bc8920 And don't bump the schema version unnecessarily 2015-04-28 10:54:15 +01:00
Mark Haines 55fcf62e9c Merge pull request #133 from matrix-org/invite_power_level
Invite power level
2015-04-28 10:50:34 +01:00
Mark Haines b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Mark Haines 252e6f6869 Merge pull request #130 from matrix-org/server_rename_check
Fix for SYN-266
2015-04-28 10:39:31 +01:00
Mark Haines 0bc71103e1 Output vim style mode markers into the yaml config file 2015-04-28 10:17:10 +01:00
Mark Haines f8b865264a Merge branch 'develop' into key_distribution
Conflicts:
	synapse/crypto/keyring.py
2015-04-27 18:29:32 +01:00
Paul "LeoNerd" Evans 38432d8c25 Merge branch 'develop' into invite_power_level 2015-04-27 17:09:25 +01:00
Erik Johnston 42b7139dec Remove unused import 2015-04-27 15:59:56 +01:00
Erik Johnston 1ef66cc3bd Move database configuration into config module 2015-04-27 15:57:43 +01:00
Erik Johnston 416a3e6c4f Ensure check_same_thread is enabled for sqlite3 2015-04-27 15:44:30 +01:00
Erik Johnston 8558e1ec73 Make get_max_token into inlineCallbacks so that the lock works. 2015-04-27 15:19:44 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston 2c70849dc3 Fix newlines 2015-04-27 14:38:29 +01:00
Erik Johnston 0a016b0525 Pull inner function out. 2015-04-27 14:37:24 +01:00
Erik Johnston e701aec2d1 Implement locks using create_observer for fetching media and server keys 2015-04-27 14:20:26 +01:00
David Baker 412ece18e7 Add commentage. 2015-04-27 14:08:45 +01:00
Erik Johnston 1c82fbd2eb Implement create_observer.
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
2015-04-27 13:59:37 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
David Baker 03c4f0ed67 pep8 2015-04-27 12:36:59 +01:00
David Baker f1acb9fd40 logging args 2015-04-27 11:56:34 +01:00
David Baker 8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker df75914791 pep8 2015-04-27 11:48:33 +01:00
David Baker b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker 2f475bd5d5 pep8 2015-04-24 18:15:07 +01:00
David Baker a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Mark Haines 288702170d Add config for setting the perspective servers 2015-04-24 17:01:34 +01:00
David Baker 1bac74b9ae Change to https for ID server communication 2015-04-24 14:48:49 +01:00
David Baker ed83638668 Make one-to-one rule an underride otherwise bings don't work in one-to-one wrooms. Likewise a couple of other rules. 2015-04-24 14:27:17 +01:00
David Baker 7ac8a60c6f More underscores 2015-04-24 11:44:27 +01:00
Mark Haines c253b14f6e Merge branch 'develop' into key_distribution 2015-04-24 11:29:46 +01:00
Mark Haines bdcb23ca25 Fix spelling 2015-04-24 11:29:19 +01:00
Mark Haines b2c2dc8940 Merge branch 'develop' into key_distribution 2015-04-24 11:28:10 +01:00
Mark Haines 869dc94cbb Call the super classes when generating config 2015-04-24 11:27:56 +01:00
David Baker a218619626 Use underscores instead of camelcase for id server stuff 2015-04-24 11:27:38 +01:00
Mark Haines b1e68add19 Add a config file for perspective servers 2015-04-24 11:26:19 +01:00
Mark Haines 31e262e6b4 Copyright notice 2015-04-24 10:36:51 +01:00
Mark Haines eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines 4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
Mark Haines c8c710eca7 Move the key related config parser into a separate file 2015-04-24 10:22:22 +01:00
Mark Haines 149ed9f151 Better help for the old-signing-key option 2015-04-24 10:07:55 +01:00
David Baker f7a79a37be pep8 2015-04-24 09:42:37 +01:00
David Baker 6532b6e607 Merge branch 'develop' into csauth
Conflicts:
	synapse/http/server.py
2015-04-24 09:37:54 +01:00
David Baker 74270defda No commas here, otherwise our error string constants become tuples. 2015-04-24 09:28:57 +01:00
Paul "LeoNerd" Evans e1e5e53127 Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again 2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans b3bda8a75f Don't let the remote offline serial list grow arbitrarily large 2015-04-23 18:40:47 +01:00
Paul "LeoNerd" Evans 8a785c3006 Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261) 2015-04-23 18:40:19 +01:00
Paul "LeoNerd" Evans 191f7f09ce Generate presence event-stream JSON structures directly 2015-04-23 18:27:25 +01:00
David Baker 03eb4adc6e Dedicated error code for failed 3pid auth verification 2015-04-23 18:20:17 +01:00
Mark Haines 4bbf7156ef Update to match the specification for key/v2 2015-04-23 16:39:13 +01:00
David Baker 0eb61a3d16 Remove ultimately unused feature of saving params from the first call in the session: it's probably too open to abuse. 2015-04-23 14:44:12 +01:00
David Baker a2c10d37d7 Add an error code to 'missing token' response. 2015-04-23 13:23:44 +01:00
David Baker 2e0d9219b9 Remove now-redundant email config 2015-04-23 11:45:29 +01:00
Mark Haines f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
Paul "LeoNerd" Evans a16eaa0c33 Neater fetching of user's auth level in a room - squash to int() at access time (SYN-353) 2015-04-22 14:20:04 +01:00
Paul "LeoNerd" Evans f43063158a Appease pep8 2015-04-22 13:12:11 +01:00
Paul "LeoNerd" Evans 2808c040ef Also remember to check 'invite' level for changes 2015-04-21 21:13:14 +01:00
Paul "LeoNerd" Evans 48b6ee2b67 Create an 'invite' powerlevel when making new rooms 2015-04-21 21:07:35 +01:00
Paul "LeoNerd" Evans bc41f0398f Initial implementation of an 'invite' power_level 2015-04-21 20:56:08 +01:00
Paul "LeoNerd" Evans d3309933f5 Much neater fetching of defined powerlevels from m.room.power_levels state event 2015-04-21 20:53:23 +01:00
Paul "LeoNerd" Evans b568c0231c Remove debugging print statement accidentally committed 2015-04-21 20:21:14 +01:00
Paul "LeoNerd" Evans 3a7d7a3f22 Sanitise a user's powerlevel to an int() before numerical comparison, because otherwise Python is "helpful" with it (SYN-351) 2015-04-21 20:18:29 +01:00
Mark Haines 3ba522bb23 Merge branch 'develop' into key_distribution 2015-04-21 17:10:25 +01:00
Mark Haines 6080830bef Bump syutil version to 0.0.5 2015-04-21 17:04:06 +01:00
Mark Haines 8b183781cb Merge pull request #128 from matrix-org/unify_http_wrappers
Unify http wrappers
2015-04-21 17:01:04 +01:00
Mark Haines 812a99100b Set a version_string in BaseMediaResource so that the request_handler wrapper works 2015-04-21 16:43:58 +01:00
Mark Haines 1967650bc4 Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator 2015-04-21 16:35:53 +01:00
Mark Haines 1ebff9736b Split out the JsonResource request logging and error handling into a separate wrapper function 2015-04-21 16:07:20 +01:00
Mark Haines 24d21887ed SYN-350: Don't ratelimit the individual events generated during room creation 2015-04-21 14:14:19 +01:00
Mark Haines db8d4e8dd6 Merge branch 'develop' into key_distribution 2015-04-20 16:24:21 +01:00
Mark Haines 2f9157b427 Implement v2 key lookup 2015-04-20 16:23:47 +01:00
David Baker 91c8f828e1 pep8 2015-04-17 19:56:04 +01:00
David Baker 8db6832db8 Password reset, finally. 2015-04-17 19:53:47 +01:00
David Baker 117f35ac4a Add endpoint to get threepids from server 2015-04-17 17:20:18 +01:00
David Baker 4eea5cf6c2 pep8 2015-04-17 16:46:45 +01:00
David Baker f96ab9d18d make add3pid servlet work 2015-04-17 16:44:49 +01:00
Erik Johnston 865398b4a9 Revert needless change to rest.profile 2015-04-17 16:14:51 +01:00
Erik Johnston e3417bbbe0 Revert needless change to storage.profile 2015-04-17 16:13:14 +01:00
Erik Johnston 2492efb162 Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 16:08:42 +01:00
Erik Johnston 4a5990ff8f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-04-17 16:08:28 +01:00
Erik Johnston 5e7a90316d Update --database-path metavar to SQLITE_DATABASE_PATH 2015-04-17 16:08:18 +01:00
David Baker 0b1a8500a2 just the once would probably be fine 2015-04-17 13:53:54 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
David Baker bf5e54f255 Register the 3pid servlet 2015-04-17 13:44:55 +01:00
David Baker 94e1e58b4d password -> account servlet and add start of an 'add 3pid' endpoint 2015-04-17 13:44:12 +01:00
Erik Johnston ced39d019f Bump version 2015-04-17 13:25:16 +01:00
David Baker 83b554437e Need to yield the username check, otherwise very very weird things happen. 2015-04-17 12:57:25 +01:00
Erik Johnston 6ba2e3df4e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 11:22:31 +01:00
Erik Johnston 0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
David Baker 4cd5fb13a3 Oops, left debugging in. 2015-04-16 20:03:13 +01:00
David Baker ea1776f556 Return user ID in use error straight away 2015-04-16 19:56:44 +01:00
Erik Johnston e1c0970c11 PEP8 2015-04-16 11:18:45 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston bc9e69e160 Move encoding and decoding of JSON into storage layer 2015-04-16 11:01:09 +01:00
Paul "LeoNerd" Evans f2cf37518b Filter typing nofication events to only those rooms the requesting user is a member of (SYN-328) 2015-04-15 23:34:16 +01:00
Paul "LeoNerd" Evans 0268d40281 Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility 2015-04-15 23:09:35 +01:00
Paul "LeoNerd" Evans 399b5add58 Neater implementation of membership change auth checks, ensuring we can't forget to check if the calling user is a member of the room 2015-04-15 18:40:23 +01:00
Paul "LeoNerd" Evans e6e130b9ba Ensure that non-room-members cannot ban others, even if they do have enough powerlevel (SYN-343) 2015-04-15 18:07:33 +01:00
David Baker 766bd8e880 Dummy login so we can do the first POST request to get login flows without it just succeeding 2015-04-15 17:14:25 +01:00
Erik Johnston ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Mark Haines 8d761134c2 Fail quicker for 4xx responses in the key client, optional hit a different API path 2015-04-15 16:57:58 +01:00
Erik Johnston cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston 5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston 6e91f14d09 Add missing yield 2015-04-15 16:25:07 +01:00
Erik Johnston ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston 806f380a8b Make LruCache thread safe, as its used for event cache 2015-04-15 16:07:59 +01:00
David Baker a19b739909 Regstration with email in v2 2015-04-15 15:50:38 +01:00
Erik Johnston a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston e19f794fee Change from exception to warn 2015-04-15 15:12:57 +01:00
Erik Johnston d5ff9effcf Don't wait on federation_handler.handle_new_event 2015-04-15 15:05:57 +01:00
Erik Johnston e845434028 Remove run_on_reactor()s 2015-04-15 15:05:45 +01:00
Erik Johnston 4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston bc6cef823f Do more parellelization for initialSync 2015-04-15 14:21:59 +01:00
Erik Johnston 1ec6fa98c9 Parellelize initial sync 2015-04-15 14:17:16 +01:00
Erik Johnston 25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Erik Johnston 6606f7c659 Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-15 10:27:20 +01:00
Erik Johnston a971fa9d58 Use try..finally in contextlib.contextmanager 2015-04-15 10:25:43 +01:00
Erik Johnston ded4128965 Use True for True rather than 1 2015-04-15 10:24:24 +01:00
Erik Johnston f9e12f79ca Add missing yield in storage func 2015-04-15 10:24:07 +01:00
Erik Johnston c756dfeb14 Correctly identify deadlocks 2015-04-15 10:23:42 +01:00
Mark Haines 32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Mark Haines 88cb06e996 Update syutil version to 0.0.4 2015-04-14 16:18:17 +01:00
Mark Haines d488463fa3 Add a version 2 of the key server api 2015-04-14 16:04:52 +01:00
Erik Johnston 127fad17dd Add postgres database engine 2015-04-14 14:50:29 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Mark Haines 3cbc286d06 Move server key api into rest/key/v1 2015-04-14 13:28:11 +01:00
Erik Johnston 3c741682e5 Correctly increment the _next_id initially 2015-04-14 09:54:44 +01:00
Erik Johnston 86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston 93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston 4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00
Erik Johnston 4d5b098626 Use LONGBLOB and TEXT for arbitary length rows 2015-04-10 18:45:09 +01:00
Erik Johnston 7ed2ec3061 Handle the fact that in sqlite binary data might be stored as unicode or bytes 2015-04-10 13:41:54 +01:00
Erik Johnston ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston 7e863c51e6 Use unsigned bigint 2015-04-10 11:22:04 +01:00
Erik Johnston 0f12772e32 SQLite wants INTEGER and not BIGINT for primary keys 2015-04-10 11:16:09 +01:00
Erik Johnston d5d4281647 Update full_schemas/16 to match delta files. Add delta/16 scripts 2015-04-10 10:59:46 +01:00
Erik Johnston cda4a6f93f Revert non-trivial changes to upgrade scripts 2015-04-10 10:19:50 +01:00
Erik Johnston e2722f58ee Fix schema again 2015-04-10 10:16:29 +01:00
Erik Johnston a1665c5094 Revert non-trivial schema changes and move them to a new schema version. 2015-04-10 10:05:44 +01:00
Erik Johnston 2ded344620 Remove unused import 2015-04-09 13:46:06 +01:00
Erik Johnston 9707acfc40 Remove spurious spaces 2015-04-09 13:45:20 +01:00
Erik Johnston 8bf285e082 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-09 13:13:26 +01:00
Mark Haines 1280a47fc6 Add comment 2015-04-09 11:42:21 +01:00
Mark Haines 23d285ad57 Unset the timer in the timeout callback so that we don't try to cancel it if it has been called 2015-04-09 11:41:50 +01:00
Erik Johnston 8ad0f4912e
Stream ordering and out of order insertions.
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Mark Haines 6f9dea7483 SYN-339: Cancel the notifier timeout when the notifier fires 2015-04-09 11:07:20 +01:00
Erik Johnston 279a547a8b Use generic db exceptions rather than sqlite3 specific ones 2015-04-08 16:53:48 +01:00
Mark Haines 83f5125d52 Merge pull request #114 from matrix-org/improve_get_event_cache
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston a43b40449b Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 16:46:56 +01:00
Erik Johnston 9cef051ce2 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-04-08 16:43:17 +01:00
Erik Johnston 45131b2bca Bump version 2015-04-08 16:35:12 +01:00
Erik Johnston ccda401dbf SYN-338: Fix typo that caused the cache to throw an exception in some instances 2015-04-08 16:34:23 +01:00
Erik Johnston 3887350e47 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 14:11:07 +01:00
Erik Johnston 19234cc6c3 typo 2015-04-08 14:10:06 +01:00
Erik Johnston e8f1521605 Don't yield on notifying all listeners 2015-04-08 14:08:30 +01:00
Mark Haines 605941ee26 Merge pull request #117 from matrix-org/notifier-leak
Fix a notifier leak
2015-04-08 14:02:37 +01:00
Erik Johnston 5bc41fe9f8 Move comment into docstring 2015-04-08 14:01:22 +01:00
Erik Johnston 638be5a6b9 Factor out loops into '_discard_if_notified' 2015-04-08 13:58:32 +01:00
Erik Johnston 830d07db82 Also perform paranoia checks in 'on_new_user_event' 2015-04-08 13:40:20 +01:00
Erik Johnston 65f5e4e3e4 Add paranoia checks to make sure that we evict stale NotificationListeners when we are about to process them 2015-04-08 13:33:38 +01:00
Erik Johnston 07d4041709 Fix bug where we didn't inform the NotificataionListeners about new rooms they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts 2015-04-08 13:33:38 +01:00
Erik Johnston c1b34af441 Move database timer logging to seperate logger 2015-04-08 13:31:19 +01:00
Erik Johnston 9a05795619 Retry transaction, not SQL query 2015-04-08 13:11:28 +01:00
Erik Johnston 24d8134ac1 Fix maria engine to correctly recognize deadlocks 2015-04-08 13:10:54 +01:00
Erik Johnston d5e7e6b9b6 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:17:22 +01:00
Erik Johnston 0775c62469 Fix --enable-registration flag to work if you don't give a value 2015-04-07 18:16:23 +01:00
Erik Johnston 38928c6609 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:08:38 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Kegan Dougal ae8ff92e05 Fix a bug which causes a send event level of 0 to not be honoured.
Caused by a bad if check, which incorrectly executes for both 0 and None,
when None was the original intent.
2015-04-07 15:48:20 +01:00
Erik Johnston 49d6aa1394 Retry on deadlock 2015-04-07 15:28:37 +01:00
Erik Johnston 0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston 6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston 05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston 8574bf62dc Add index to presence table 2015-04-07 12:09:36 +01:00
Erik Johnston 0af5f5efaf Don't use multiple UNIQUE constraints; it will cause deadlocks 2015-04-07 12:08:35 +01:00
Erik Johnston c8d3f6486d Implement or_ignore flag on inserts 2015-04-07 12:06:22 +01:00
Erik Johnston 304111afd0 Don't use AUTOINCREMENT, use an in memory version 2015-04-07 12:05:36 +01:00
Erik Johnston d0e444a648 Explicitly name the __main__ module logger 2015-04-07 12:04:02 +01:00
David Baker 4eb6d66b45 Add app service auth back in to v2 register 2015-04-02 17:51:19 +01:00
David Baker 6b59650753 Throw sensible errors on not-json when allowing empty body 2015-04-02 17:45:16 +01:00
David Baker 41cd778d66 pep8 2015-04-02 17:06:17 +01:00
David Baker 70a84f17f3 Add shared secret auth into register v2 and switch the script over. 2015-04-02 17:01:29 +01:00
Erik Johnston 779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Paul "LeoNerd" Evans ef1e019840 Appease pep8 2015-04-01 19:17:38 +01:00
Paul "LeoNerd" Evans 5583e29513 Report process open filehandles in metrics 2015-04-01 19:15:23 +01:00
David Baker c5bf0343e8 Explain how I justified to myself making JsonResource not always send JSON. 2015-04-01 15:13:14 +01:00
Erik Johnston e24c32e6f3 Fix SQLite support 2015-04-01 15:09:51 +01:00
David Baker e9c908ebc0 Completely replace fallback auth for C/S V2:
* Now only the auth part goes to fallback, not the whole operation
 * Auth fallback is a normal API endpoint, not a static page
 * Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal 813e54bd5b Fix more AS sender ID thinkos.
Specifically, the ASes own user ID wasn't being treated as 'exclusive' so
a human could nab it. Also, the HS would needlessly send user queries to the
AS for its own user ID.
2015-04-01 14:05:24 +01:00
Kegan Dougal 9fa8bda099 Merge branch 'develop' into application-services-registration-script 2015-04-01 10:19:17 +01:00
David Baker f129ee1e18 Make docs a bit more true 2015-03-31 18:25:10 +01:00
Kegan Dougal 09cbff174a Fix thinko whereby events *for the AS specifically* were not passed on.
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
David Baker d18e7779ca Grammar and deduplication 2015-03-31 14:40:02 +01:00
Kegan Dougal 5e88a09a42 Add same user_id char checks as registration. 2015-03-31 14:00:25 +01:00
Kegan Dougal cf1fa59f4b Use a sender localpart instead of a user ID.
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal 3470cb36a8 Pyflakes 2015-03-31 13:03:31 +01:00
Kegan Dougal c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
David Baker 9f642a93ec pep8 2015-03-31 09:50:44 +01:00
Kegan Dougal e7887e37a8 Remove appservice REST servlets 2015-03-31 09:32:40 +01:00
Kegan Dougal af853a4cdb Add AppServiceConfig 2015-03-31 09:22:31 +01:00
David Baker 59bf16eddc New registration for C/S API v2. Only ReCAPTCHA working currently. 2015-03-30 18:13:10 +01:00
Matthew Hodgson 8366fde82f turn --disable-registration into --enable-registration, given the default is for registration to be disabled by default now. this is backwards incompatible by removing the old --disable-registration arg, but makes for a much more intuitive arg 2015-03-30 12:01:09 -04:00
Paul "LeoNerd" Evans 3e420aebd8 Revert "Add another @cached wrapper, this time on get_presence_state()"
This reverts commit ff1fa0fbf8.
2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans ff1fa0fbf8 Add another @cached wrapper, this time on get_presence_state() 2015-03-27 15:57:16 +00:00
David Baker 6f4f7e4e22 pep8 2015-03-26 14:12:06 +00:00
David Baker a32e876ef4 Delete pushers when changing password 2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans a198894bf7 Appease pep8 2015-03-26 11:53:58 +00:00
Kegan Dougal 32206dde3f Fixes from PR comments 2015-03-26 10:11:52 +00:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans 953e40f9dc Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one 2015-03-25 19:12:16 +00:00
David Baker df4c12c762 pep8 blank lines 2015-03-25 19:08:17 +00:00
David Baker c1a256cc4c Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API. 2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans f173d40a32 Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 1b988b051b Store the rejected reason in (Frozen)Event structs 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 033a517feb Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans d6b3ea75d4 Implement the 'key in dict' test for LruCache() 2015-03-25 19:04:59 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
David Baker 9aa0224cdf unused import 2015-03-24 17:25:59 +00:00
David Baker c7023f2155 1) Pushers are now associated with an access token
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston 0ba393924a Escape non printing ascii character 2015-03-24 16:31:52 +00:00
Erik Johnston f488293d96 Don't reinsert into event_edges 2015-03-24 16:20:26 +00:00
Erik Johnston 1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston 5a447098dd Don't use room hosts table 2015-03-24 16:19:24 +00:00
Erik Johnston 9e98f1022a Don't order by rowid 2015-03-24 16:19:01 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
David Baker d19e79ecc9 Make deleting other access tokens when you change your password actually work 2015-03-24 15:33:48 +00:00
Paul "LeoNerd" Evans ed008e85a8 Reduce activity timer granularity to avoid too many quick updates (SYN-247) 2015-03-23 17:25:50 +00:00
Erik Johnston 6e7131f02f Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. 2015-03-23 15:38:56 +00:00
Erik Johnston 9a7f496298 Sanitize RoomMemberStore 2015-03-23 15:29:04 +00:00
David Baker 78adccfaf4 pep8 / pyflakes 2015-03-23 14:23:51 +00:00
David Baker d98660a60d Implement password changing (finally) along with a start on making client/server auth more general. 2015-03-23 14:20:28 +00:00
Erik Johnston d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston 278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans 72d8406409 Put a cache on get_aliases_for_room 2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans 0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans b1022ed8b5 func(*EXPR) is valid Python syntax, really... 2015-03-20 17:47:45 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston 80cd08c190 PEP8 2015-03-20 16:03:25 +00:00
Erik Johnston 9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Erik Johnston dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston 7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans 91cb46191d Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though 2015-03-20 14:59:45 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston 455579ca90 Make database selection configurable 2015-03-20 10:55:55 +00:00
Erik Johnston 0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston 56f2d31676 Bump version 2015-03-19 17:48:33 +00:00
Erik Johnston c178e4e6ca Add missing servlet to list 2015-03-19 17:48:21 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Erik Johnston 58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston 0dcb145c7e Bump version 2015-03-19 11:26:03 +00:00
Erik Johnston 6cc046302f Bump version 2015-03-18 11:41:00 +00:00
Erik Johnston ed4d44d833 Merge pull request #109 from matrix-org/default_registration
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Erik Johnston f88db7ac0b Factor out user id validation checks 2015-03-18 11:34:18 +00:00
Erik Johnston 57976f646f Do more validation of incoming request 2015-03-18 11:30:04 +00:00
Erik Johnston bb24609158 Clean out event_forward_extremities table when the server rejoins the room 2015-03-18 11:19:47 +00:00
Erik Johnston 89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans 93978c5e2b @cached() annotate get_user_by_token() - achieves a minor DB performance improvement 2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans 1489521ee5 Be polite and ensure we use @functools.wraps() when creating a function decorator 2015-03-17 17:19:22 +00:00
David Baker 6d33f97703 pep8 2015-03-17 11:53:55 +00:00
David Baker 7564dac8cb Wire up the webclient option
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
2015-03-17 12:45:37 +01:00
Paul "LeoNerd" Evans 3f7a31d366 Add a DistributionMetric to HTTP request/response processing time in the server 2015-03-16 18:31:29 +00:00
Paul "LeoNerd" Evans be170b1426 Add a metric for the scheduling latency of SQL queries 2015-03-16 17:21:59 +00:00
Erik Johnston cd2539ab2a Merge pull request #110 from matrix-org/fix_ban
Fix ban
2015-03-16 15:36:52 +00:00
Kegan Dougal f0d6f724a2 Set the service ID as soon as it is known. 2015-03-16 15:24:32 +00:00
Erik Johnston f1d2b94e0b Copy dict of context.current_state before changing it. 2015-03-16 15:13:05 +00:00
Erik Johnston 857810d2dd Revert incorrect changes to where we send events 2015-03-16 15:12:47 +00:00
Kegan Dougal d04fa1f712 Implement ServiceQueuer with tests. 2015-03-16 14:03:16 +00:00
Erik Johnston c2c9471cba Don't block waiting on waking up all the listeners when sending an event. 2015-03-16 13:16:37 +00:00
Kegan Dougal 6279285b2a Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer. 2015-03-16 13:15:40 +00:00
Erik Johnston 8bad40701b Comment. 2015-03-16 13:13:07 +00:00
Erik Johnston 250e143084 Use 403 instead of 400 2015-03-16 13:11:42 +00:00
Erik Johnston b2e6ee5b43 Remove concept of context.auth_events, instead use context.current_state 2015-03-16 13:06:23 +00:00
Kegan Dougal c9c444f562 Wrap polling/retry blocks in try/excepts to avoid sending to other ASes breaking permanently should an error occur. 2015-03-16 10:38:02 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Erik Johnston 758d114cbc Send all membership events to the remote homeserver 2015-03-16 00:27:59 +00:00
Erik Johnston ea8590cf66 Make context.auth_events grap auth events from current state. Otherwise auth is wrong. 2015-03-16 00:18:08 +00:00
Erik Johnston ab8229479b Respect ban membership 2015-03-16 00:17:25 +00:00
Matthew Hodgson c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Paul Evans e731d30d90 Merge pull request #108 from matrix-org/metrics
Metrics
2015-03-13 17:31:10 +00:00
Erik Johnston 98a3825614 Allow enabling of registration with --disable-registration false 2015-03-13 16:49:18 +00:00
Erik Johnston 9266cb0a22 PEP8 2015-03-13 15:26:00 +00:00
Erik Johnston dea236e4fa Add missing commas 2015-03-13 15:24:03 +00:00
Erik Johnston 69135f59aa Implement registering with shared secret. 2015-03-13 15:23:37 +00:00
Erik Johnston 58367a9da2 Disable registration by default 2015-03-13 12:59:45 +00:00
Erik Johnston 58247c8b4b Also bump dependency link version 2015-03-13 11:39:57 +00:00
Matthew Hodgson f55bd3f94b bump dep on syweb 0.6.5 2015-03-12 18:56:53 +00:00
Paul "LeoNerd" Evans e90002ca1d Merge remote-tracking branch 'origin/develop' into metrics 2015-03-12 16:55:25 +00:00
David Baker bbb010a30f More sacrifices to the pep8 gods. 2015-03-12 16:53:12 +00:00
Paul "LeoNerd" Evans 05a056a409 Appease pyflakes 2015-03-12 16:45:05 +00:00
Paul "LeoNerd" Evans 0eb7e6b9a8 Delete unused import of NOT_READY_YET 2015-03-12 16:39:52 +00:00
Paul "LeoNerd" Evans 128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b98b4c135d Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans a2cdd11d4a Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening() 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans e0214a263b Build MetricsResource as a specific HomeServer dependency 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans e75fa8bbbf Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans c782e893ec Neater metrics from TransactionQueue 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 89ac1fa8ba Add a counter to track total number of events served by the notifier 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 2e4f0b2bd7 Replace the @metrics.counted annotations in federation with specifically-written counters and distributions 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans c1cdd7954d Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 63cb7ece62 Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 493e3fa0ca Don't forbid '_' in metric basenames any more, to allow things like foo_time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 642f725fd7 Pretend the 'getEvent' cache is just another cache in the set of all the others for metric 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans cbc0406be8 Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 1748605c5d Count incoming HTTP requests per servlet that responds 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 4d661ec0f3 Remember to emit final linefeed from /metrics page, or Prometheus gets upset 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 0e847540c3 Prometheus needs "escaped" label values 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 22b37b75db Kill unused CounterMetric.fetch() method 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 0b96bb793e Have all @metrics.counted use a single metric name vectored on the method name, rather than a brand new scalar counter per counted method 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b3a0179d64 Bugfix to rendering output of vectored TimerMetrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 399689dcc7 Provide some process resource usage metrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans fa319a5786 Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 6d146e15df Put some gauge metrics on the number of notifier listeners, and notified-on objects (users, rooms, appservices) 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 25187ab674 Collect per-SQL-verb timer stats on query execution time 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans f52acf3b12 Neater register_* methods on overall Metrics container 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans a99d6edc05 Neater implementation of metric render methods by pulling out 'render' as a base method that calls self.render_item 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 72625f2f4d Initial hack at a TimerMetric; for storing counts + duration accumulators 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e1a7e3564f Delete a couple of TODO markers of monitoring stats now done 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 094803cf82 Put vector gauges on transaction queue pending PDU and EDU dicts 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e9c4b0d178 Ensure that /_synapse/metrics response is UTF-8 encoded 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 23ab0c68c2 Implement vector CallbackMetrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 849300bc73 Neater introspection methods on BaseMetric so that subclasses don't need to touch self.keys directly 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 8664599af7 Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e02cc249da Ensure that exceptions while rendering individual metrics don't stop others from being rendered anyway - especially useful for CallbackMetric 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 59c448f074 Add a scalar gauge metric on the size of the presence user cachemap 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans d8caa5454d Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans b0cdf097f4 Sprinkle some CacheCounterMetrics around the synapse.storage layer 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans ce8b5769f7 Create the concept of a cachecounter metric; generating two counters specific to caches 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 7d72e44eb9 Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans c53ec53d80 Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 9470412316 Initial attempt at sprinkling some @metrics.counted decorations around the federation code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans a594087f06 Have the MetricsResource actually render metric counters 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 74bc42cfdd An initial implementation of a 'metrics' instance, similar to a 'logger' for keeping counter stats on method calls 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 120b689284 Delete pointless (and unreachable) __init__ method from FederationClient 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e7420a3bef Initial tiny attempt at (vectorable) counter metrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e07fc62833 A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 5b6e11d560 Commandline option to enable metrics system 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 211c14c391 No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans ad5701f50f Expose 'config' as a real HomeServer dependency key 2015-03-12 16:24:50 +00:00
David Baker c92fdf88a3 Log the matching push rule. 2015-03-11 22:17:31 +00:00
Paul "LeoNerd" Evans a7a28f85ae Appease pep8 2015-03-11 17:32:43 +00:00
Paul "LeoNerd" Evans 59a5f012cc Also give _execute() a description 2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans 099e4b88d8 Add a description to storage layer's _execute_and_decode() 2015-03-11 17:08:57 +00:00
David Baker cdb2e045ee Again, underscore, not hyphen 2015-03-11 14:22:35 +00:00
David Baker 465354ffde 'false' is not False 2015-03-11 11:24:50 +00:00
David Baker 83b1e7fb3c PEP8 blank lines 2015-03-11 10:01:17 +00:00
David Baker 04f8478aaa Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default. 2015-03-10 17:26:25 +00:00
David Baker 8916acbc13 These aren't defined for redacted events so don't crash 2015-03-10 11:21:37 +00:00
Erik Johnston abaf47bbb6 Fix bug in logging. 2015-03-10 10:28:29 +00:00
Erik Johnston 045afd6b61 in_thread takes no arguments 2015-03-10 10:19:03 +00:00
Erik Johnston 98b867f7b7 Fix bug in logging. 2015-03-10 10:16:09 +00:00
Kegan Dougal db1fbc6c6f Fix remaining scheduler bugs. Add more informative logging. 2015-03-10 10:04:20 +00:00
Erik Johnston e84fe3599b Merge pull request #105 from matrix-org/erikj-perf
Add a Twisted Service to synapse.app.homeserver
2015-03-10 10:02:26 +00:00
Erik Johnston c37eceeb9e Split out the 'run' from 'setup' 2015-03-10 09:58:33 +00:00
Erik Johnston b8a6692657 Add documentation. When starting via twistd respect soft_file_limit config option. 2015-03-10 09:39:42 +00:00
Kegan Dougal 7e0bba555c Remove unused import 2015-03-09 17:48:37 +00:00
Kegan Dougal 04c9751f24 Bug fixes whilst putting it all together 2015-03-09 17:45:41 +00:00
Erik Johnston 019422ebba Merge pull request #101 from matrix-org/neaten-federation-servlets
Neaten federation servlets
2015-03-09 17:39:06 +00:00
Kegan Dougal b98cd03193 Use event IDs instead of dumping event content in the txns table. 2015-03-09 17:25:20 +00:00
Kegan Dougal 21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Erik Johnston 8e28db5cc9 Change room handlers get_rooms_for_user to get_joined_rooms_for_user. This uses the a storage api that is cached. 2015-03-09 16:43:09 +00:00
Kegan Dougal 0a60bbf4fa Finish appservice txn storage impl and tests. 2015-03-09 15:53:03 +00:00
Kegan Dougal 1ead1caa18 Implement create_appservice_txn with tests. 2015-03-09 13:54:20 +00:00
Erik Johnston f31e65ca8b Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf 2015-03-09 13:29:41 +00:00
Kegan Dougal 1c2dcf762a Partially implement txn store methods with tests. 2015-03-09 13:10:31 +00:00
David Baker 1df3ccf7ee D'oh: underscore, not hyphen 2015-03-09 12:39:56 +00:00
David Baker 118c883429 Call notifications should be override else they'll get clobbered by sender/room rules. 2015-03-06 19:41:36 +00:00
Kegan Dougal 406d32f8b5 Start implementing ApplicationServiceTransactionStore 2015-03-06 17:35:14 +00:00
Kegan Dougal 34ce2ca62f Merge branch 'develop' into application-services-txn-reliability 2015-03-06 17:28:49 +00:00
Kegan Dougal 4a6afa6abf Assign the AS ID from the database; replace old placeholder txn id. 2015-03-06 17:27:55 +00:00
Kegan Dougal 01c099d9ef Add appservice txns sql schema 2015-03-06 17:16:47 +00:00
Kegan Dougal 64345b7559 Upper bound the backoff. 2015-03-06 16:41:19 +00:00
Erik Johnston 5d43eaed61 Merge branch 'develop' into release-v0.8.0 2015-03-06 16:25:19 +00:00
Erik Johnston 9ccccd4874 When setting display name more graciously handle failures to update room state. 2015-03-06 16:24:05 +00:00
Kegan Dougal 2602ddc379 Apply clarity and docstrings 2015-03-06 16:16:14 +00:00
Kegan Dougal 0354659f9d Finish synapse.appservice.scheduler implementation.
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
David Baker be9dafcd37 Dial down logging for failed pushers 2015-03-06 15:32:38 +00:00
Kegan Dougal 7d3491c741 Add some loggers 2015-03-06 15:17:50 +00:00
David Baker 96eda876a4 Specify when we don't want to highlight 2015-03-06 15:12:37 +00:00
Kegan Dougal f260cb72cd Flesh out more stub functions. 2015-03-06 15:12:24 +00:00
David Baker e7d7152c3c Remove the fallback rule - we probably don't want to be notifying for everything even if we don't know what it is. 2015-03-06 15:03:34 +00:00
Kegan Dougal 141ec04d19 Add stub ApplicationServiceTransactionStore. Bootstrap Recoverers. Fill in stub Transaction functions. 2015-03-06 14:53:35 +00:00
David Baker 5ecc768970 Add attribute so push gateways can tell if a member event is about the user in question 2015-03-06 14:41:50 +00:00
Kegan Dougal 0fbfe1b08a Add more tests; fix bugs. 2015-03-06 14:36:52 +00:00
Erik Johnston 369449827d Bump version 2015-03-06 14:24:53 +00:00
Erik Johnston c54773473f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-03-06 14:23:41 +00:00
Erik Johnston b102a87348 Merge pull request #96 from matrix-org/pushrules2
Evolution of push rules
2015-03-06 14:20:04 +00:00
David Baker cf66ddc1b4 Schema change as delta in v14 2015-03-06 14:11:49 +00:00
David Baker c06b45129c Add more server default rules so we have default rules for whether you get notifs for invites / random member events 2015-03-06 11:50:51 +00:00
Kegan Dougal 192e228a98 Start adding some tests 2015-03-06 11:50:27 +00:00
Erik Johnston e49d6b1568 Unused import 2015-03-06 11:37:24 +00:00
David Baker 657a0d2568 Comment typo 2015-03-06 11:34:30 +00:00
Erik Johnston 3ce8540484 Don't look for an TLS private key if we have set --no-tls 2015-03-06 11:34:06 +00:00
David Baker 1487bba226 Suppress notices should trump content/room/sender rules. 2015-03-06 10:27:32 +00:00
David Baker 83d31144eb Add the highlight tweak where messages should be highlighted a different colour in appropriate clients. 2015-03-06 10:26:08 +00:00
Kegan Dougal d516d68b29 Rejig structure given the appservice_handler already filters the correct ASes to use. 2015-03-06 10:25:50 +00:00
Erik Johnston 130df8fb01 Add some randomness to the user specified timeout on event streams to mitigate against thundering herds problems 2015-03-06 10:25:36 +00:00
Paul "LeoNerd" Evans d79d91a4a7 Appease pep8 2015-03-05 20:55:40 +00:00
Paul "LeoNerd" Evans 5eab2549ab Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it 2015-03-05 20:36:05 +00:00
Paul "LeoNerd" Evans 7644cb79b2 Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods 2015-03-05 20:33:16 +00:00
Paul "LeoNerd" Evans ba8ac996f9 Remove the dead 'rate_limit_origin' method from TransportLayerServer 2015-03-05 19:43:17 +00:00
Paul "LeoNerd" Evans a901ed16b5 Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances 2015-03-05 19:10:57 +00:00
Kegan Dougal 0c838f9f5e Minor tweaks 2015-03-05 17:45:52 +00:00
Kegan Dougal 773cb3b688 Add stub architecture for txn reliability. 2015-03-05 17:35:07 +00:00
Erik Johnston 5b5c7a28d6 Log error message when we fail to fetch remote server keys 2015-03-05 17:09:13 +00:00
Erik Johnston 12bcf3d179 Merge pull request #100 from matrix-org/missing_pdu_compat
Handle if get_missing_pdu returns 400 or not all events.
2015-03-05 16:42:15 +00:00
Erik Johnston 9708f49abf Docs 2015-03-05 16:35:16 +00:00
Erik Johnston 96fee64421 Remove unecessary check 2015-03-05 16:31:47 +00:00
Erik Johnston 39aa968a76 Respect min_depth argument 2015-03-05 16:31:32 +00:00
Erik Johnston 6dfd8c73fc Docs. 2015-03-05 16:31:13 +00:00
Kegan Dougal e319071191 Add stub scheduler module for txn reliability 2015-03-05 16:30:33 +00:00
Paul "LeoNerd" Evans 9d9d39536b Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through 2015-03-05 16:24:13 +00:00
Erik Johnston ae702d161a Handle if get_missing_pdu returns 400 or not all events. 2015-03-05 16:08:02 +00:00
Kegan Dougal be09c23ff0 Add txn_id kwarg to push methods 2015-03-05 15:40:07 +00:00
Paul "LeoNerd" Evans dc4b774f1e Rename rooms_to_listeners to room_to_listeners, for consistency with user_ and appservice_* 2015-03-05 14:30:20 +00:00
Paul "LeoNerd" Evans 027fd1242c Give LruCache a __len__, so that len(cache) works 2015-03-04 17:32:28 +00:00
David Baker 590b544f67 Add default rule to suppress notices. 2015-03-04 15:29:02 +00:00
David Baker ed72fc3a50 Merge branch 'develop' into pushrules2
Conflicts:
	synapse/storage/schema/pusher.sql
2015-03-04 15:24:21 +00:00
Erik Johnston d56c01fff4 Note that we don't specify execution order 2015-03-04 15:10:05 +00:00
Erik Johnston 17d319a20d s/schema_deltas/applied_schema_deltas/ 2015-03-04 15:06:22 +00:00
David Baker 92b3dc3219 Merge branch 'develop' into pushrules2 2015-03-04 14:56:41 +00:00
Erik Johnston 5681264faa s/%r/%s/ 2015-03-04 14:21:53 +00:00
Erik Johnston f701197227 Add example directory structures in doc 2015-03-04 14:20:14 +00:00
David Baker 2a45f3d448 Use if not results rather than len, as per feedback. 2015-03-04 14:17:59 +00:00
Erik Johnston 16dd87d848 Don't assume db conn is a Context Manager.
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
Erik Johnston 5eefd1f618 Add unique constraint on schema_version.lock schema. Use conflict clause in sql. 2015-03-04 13:52:18 +00:00
Erik Johnston b4c38738f4 Change to use logger in db upgrade script 2015-03-04 13:43:35 +00:00
Erik Johnston 640e53935d Use context manager with db conn to correctly commit and rollback 2015-03-04 13:43:17 +00:00
Erik Johnston 8c8354e85a Actually add full_schemas dir 2015-03-04 13:34:38 +00:00
Erik Johnston c3530c3fb3 More docs. Rename 'schema/current' to 'schema/full_schemas' 2015-03-04 13:34:11 +00:00
Erik Johnston 811355ccd0 Add some docs and remove unused variables 2015-03-04 13:11:01 +00:00
Erik Johnston 82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
Erik Johnston 527e0c43a5 Bump version 2015-03-03 14:49:34 +00:00
Erik Johnston ede89ae3b4 Also bump version of downloaded syweb 2015-03-03 14:49:19 +00:00
Erik Johnston da877aad15 Bump syweb dependency 2015-03-03 13:31:50 +00:00
Erik Johnston 8d33adfbbb SYN-67: Begin changing the way we handle schema versioning 2015-03-02 18:23:55 +00:00
David Baker 6fab7bd2c1 s/user_name/user/ as per mjark's comment 2015-03-02 18:17:19 +00:00
David Baker 09f9e8493c Oops, missed a replacement. 2015-03-02 17:37:22 +00:00
Erik Johnston 9f03553f48 Add missing comma 2015-03-02 16:38:40 +00:00
Erik Johnston b41dc68773 We purposefully don't have a version 14 delta script. 2015-03-02 16:36:19 +00:00
David Baker 20436cdf75 Blank lines 2015-03-02 15:58:12 +00:00
Kegan Dougal 2de5b14fe0 Fix bug which prevented the HS pushing events to the AS due to FrozenEvents 2015-03-02 15:36:37 +00:00
Erik Johnston 8486910b64 Bump webclient version 2015-03-02 14:57:37 +00:00
Kegsay 8ad024ea80 Merge pull request #93 from matrix-org/application-services-exclusive
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
Erik Johnston 0a036944bd Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-03-02 13:53:30 +00:00
Erik Johnston 3fce185c77 Merge pull request #83 from matrix-org/nofile_limit_config
Add config option to set the soft fd limit on start
2015-03-02 13:52:16 +00:00
Erik Johnston 4195e55ccc Merge branch 'develop' of github.com:matrix-org/synapse into federation_rate_limit 2015-03-02 13:39:22 +00:00
Kegan Dougal c3c01641d2 Run deltas and bump user_version in upgrade script 2015-03-02 13:38:57 +00:00
Erik Johnston 210d3c5d72 Merge pull request #95 from matrix-org/serialize_transaction_processing
Process transactions serially.
2015-03-02 13:33:05 +00:00
Erik Johnston 3077cb2915 Use contextlib.contextmanager instead of a custom class 2015-03-02 13:32:44 +00:00
David Baker 769f8b58e8 Rename the room-with-two-people rule to be more compatible if we have actual one to one rooms. 2015-03-02 13:28:24 +00:00
Kegsay 33f93d389e Merge pull request #92 from matrix-org/application-services-event-stream
Application services event stream support
2015-03-02 12:02:48 +00:00
Erik Johnston 29481690c5 If we're yielding don't add errback 2015-03-02 11:50:43 +00:00
Erik Johnston 23d9bd1d74 Process transactions serially.
Since the events received in a transaction are ordered, later events
might depend on earlier events and so we shouldn't blindly process them
in parellel.
2015-03-02 11:39:57 +00:00
Erik Johnston 9d9b230501 Make the federation server ratelimiting configurable. 2015-03-02 11:33:45 +00:00
Kegan Dougal cb97ea3ec2 PEP8 2015-03-02 11:23:46 +00:00
Kegan Dougal 377ae369c1 Wrap all of get_app_service_rooms in a txn. 2015-03-02 11:20:51 +00:00
Kegan Dougal b216b36892 JOIN state_events rather than parsing unrecognized_keys to pull out member state_keys 2015-03-02 10:41:35 +00:00
Kegan Dougal 3d73383d18 Modify _simple_select_list to allow an empty WHERE clause. Use it for get_all_rooms and get_all_users. 2015-03-02 10:16:24 +00:00
Kegan Dougal ebc4830666 PR tweaks: set earlier on and use 'as json' for compat 2015-03-02 09:53:00 +00:00
David Baker 2a6dedd7cc It's set_tweak now, not set_sound 2015-02-27 18:38:56 +00:00
Erik Johnston 0554d07082 Move federation rate limiting out of transport layer 2015-02-27 15:41:52 +00:00
Erik Johnston 9dc9118e55 Document FederationRateLimiter 2015-02-27 15:16:47 +00:00
Kegan Dougal 58ff066064 Implement exclusive namespace checks. 2015-02-27 13:51:41 +00:00
Kegan Dougal 40c9896705 Add functions to return whether an AS has exclusively claimed a matching namespace. 2015-02-27 11:03:56 +00:00
Kegan Dougal 16b90764ad Convert expected format for AS regex to include exclusivity.
Previously you just specified the regex as a string, now it expects a JSON
object with a 'regex' key and an 'exclusive' boolean, as per spec.
2015-02-27 10:44:32 +00:00
Kegan Dougal 806a6c886a PEP8 2015-02-27 09:48:57 +00:00
Kegan Dougal 1cc77145d4 Notify appservices of invites mid-poll.
This requires the notifier to have knowledge of appservice listeners so it can
do the regex checks on incoming invites to see if the state_key matches. It
isn't enough to just rely on the room listeners and store.get_app_service_rooms
as the room will initially not exist or won't be on the ASes radar due to
having none of its users in the room.
2015-02-27 09:39:12 +00:00
David Baker cfac3b7873 SYN-267 Add a fallback rule as an explicit server default rule and make the default dont-notify so you effectively have a "notify for everything else" switch you can turn on and off. 2015-02-26 18:58:14 +00:00
David Baker 1959088156 Add API for getting/setting enabled-ness of push rules. 2015-02-26 18:07:44 +00:00
Kegan Dougal f0995436e7 Check for membership invite events correctly. 2015-02-26 17:21:17 +00:00
Kegan Dougal dcec7175dc Finish impl to get new events for AS. ASes should now be able to poll /events 2015-02-26 16:23:01 +00:00
Erik Johnston 93d90765c4 Initial implementation of federation server rate limiting 2015-02-26 16:15:26 +00:00
Erik Johnston 59362454dd Must update pending_transactions map before yield'ing 2015-02-26 15:47:35 +00:00
Kegan Dougal 92478e96d6 Finish impl to extract all room IDs an AS may be interested in when polling the event stream. 2015-02-26 14:35:28 +00:00
David Baker 944003021b whitespace 2015-02-26 13:43:05 +00:00
David Baker 94fa334b01 Add enable/disable overlay for push rules (REST API not yet hooked up) 2015-02-25 19:17:07 +00:00
Kegan Dougal 29267cf9d7 PEP8 and pyflakes 2015-02-25 17:42:28 +00:00
Kegan Dougal 978ce87c86 Comment unused variables. 2015-02-25 17:37:48 +00:00
Kegan Dougal 2c79c4dc7f Fix alias query. 2015-02-25 17:37:14 +00:00
Kegan Dougal 2b8ca84296 Add support for extracting matching room_ids and room_aliases for a given AS. 2015-02-25 17:15:25 +00:00
Kegan Dougal 2d20466f9a Add stub functions and work out execution flow to implement AS event stream polling. 2015-02-25 15:00:59 +00:00
David Baker a025055643 SYWEB-278 Don't allow rules with no rule_id. 2015-02-25 14:02:38 +00:00
David Baker 255f989c7b turns uris config options should append since it's a list 2015-02-24 20:57:58 +00:00
David Baker 443ba4eecc %s for strings otherwise you end up sending 'u"foo"' 2015-02-24 15:00:12 +00:00
Erik Johnston c0aaf9fe76 Merge pull request #89 from matrix-org/registration-fallback
Registration fallback
2015-02-24 10:00:33 +00:00
Paul "LeoNerd" Evans 9640510de2 Use OrderedDict for @cached backing store, so we can evict the oldest key unbiased 2015-02-23 18:41:58 +00:00
Paul "LeoNerd" Evans f53fcbce97 Use cache.pop() instead of a separate membership test + del [] 2015-02-23 18:30:45 +00:00
Mark Haines 27080698e7 Fix code style warning 2015-02-23 18:19:13 +00:00
Mark Haines 74048bdd41 Remove unused import 2015-02-23 18:17:43 +00:00
Kegan Dougal 28d8614f48 Trailing comma 2015-02-23 17:36:37 +00:00
Paul "LeoNerd" Evans bd84755e64 Merge remote-tracking branch 'origin/develop' into performance-cache-improvements 2015-02-23 17:16:03 +00:00
Paul "LeoNerd" Evans a09e59a698 Pull the _get_event_cache.setdefault() call out of the try block, as it doesn't need to be there and is confusing 2015-02-23 16:55:57 +00:00
Paul "LeoNerd" Evans 044d813ef7 Use the @cached decorator to implement the destination_retry_timings cache 2015-02-23 16:04:40 +00:00
Paul "LeoNerd" Evans 357fba2c24 RoomMemberStore no longer needs a _user_rooms_cache member 2015-02-23 15:57:41 +00:00
Paul "LeoNerd" Evans e76d485e29 Allow @cached-wrapped functions to have a prefill method for setting entries 2015-02-23 15:41:54 +00:00
Kegan Dougal 0696dfd94b Actually treat this as static content, not random Resources. 2015-02-23 15:35:09 +00:00
Kegan Dougal 22399d3d8f Add RegisterFallbackResource to /_matrix/static/client/register
Try to keep both forms of registration logic (native/fallback) close
together for sanity.
2015-02-23 15:14:56 +00:00
Paul "LeoNerd" Evans 4631b737fd Squash out the now-redundant ApplicationServicesCache object class 2015-02-23 14:38:44 +00:00
Erik Johnston e25e0f4da9 Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-02-23 14:36:00 +00:00
Erik Johnston 42b972bccd Revert get_auth_chain changes 2015-02-23 14:35:23 +00:00
Erik Johnston db215b7e00 Implement and use new batched get missing pdu 2015-02-23 13:58:02 +00:00
Erik Johnston a87c56c673 Bump version 2015-02-21 13:45:07 +00:00
Erik Johnston 1f29fafc95 Don't exit if we can't work out if we're running in a git repo 2015-02-21 13:44:46 +00:00
Erik Johnston 7c56210f20 By default set soft limit to hard limit 2015-02-20 16:09:44 +00:00
Erik Johnston 7367ca42b5 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-20 16:06:28 +00:00
Erik Johnston dc0ee55110 Change version scheme 2015-02-20 15:00:14 +00:00
Erik Johnston 0edfecc904 Bump version 2015-02-20 14:14:28 +00:00
Erik Johnston 2bafeca270 Add missing comma so that it generates a dict and not a set 2015-02-20 14:08:42 +00:00
Erik Johnston 15e2d7e387 Always allow AS to register 2015-02-20 11:39:53 +00:00
Paul "LeoNerd" Evans 55022d6ca5 Remove a TODO note 2015-02-19 18:38:09 +00:00
Paul "LeoNerd" Evans ebc3db295b Take named arguments to @cached() decorator, add a 'max_entries' limit 2015-02-19 18:36:02 +00:00
Paul "LeoNerd" Evans 077d200342 Move @cached decorator out into synapse.storage._base; add minimal docs 2015-02-19 17:29:39 +00:00
Erik Johnston 0ac2a79faa Initial stab at implementing a batched get_missing_pdus request 2015-02-19 17:24:14 +00:00
Paul "LeoNerd" Evans 61959928bb Pull out the 'get_rooms_for_user' cache logic into a reüsable @cached decorator 2015-02-19 14:58:07 +00:00
Erik Johnston 0722f982d3 Disable registration if config option was set. 2015-02-19 14:22:20 +00:00
Erik Johnston 81163f822e Add config option to disable registration. 2015-02-19 14:16:53 +00:00
Erik Johnston 939273c4b0 Rename resource variable so as to not shadow module import 2015-02-19 11:53:13 +00:00
Erik Johnston c3eb7dd9c5 Add config option to set the soft fd limit on start 2015-02-19 11:50:49 +00:00
David Baker b457f1677c Send room ID in http notifications so clients know which room to go to if the user responds to the notification. 2015-02-19 10:06:17 +00:00
Erik Johnston 7025781df8 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1 2015-02-18 17:37:43 +00:00
Erik Johnston 142f1263f6 Merge pull request #82 from matrix-org/git_tag_version
Git tag version
2015-02-18 17:37:19 +00:00
Erik Johnston 6311ae8968 Conform to header spec take two 2015-02-18 17:34:26 +00:00
Erik Johnston 3f1871021e Make /keys/ return correct Server version 2015-02-18 17:32:12 +00:00
Erik Johnston b6771037a6 Make version_string conform to User-Agent and Server spec 2015-02-18 17:31:50 +00:00
Erik Johnston 5b753d472b Bump matrix-angular-sdk version 2015-02-18 17:02:40 +00:00
Erik Johnston 1df8bad63e pyflakes 2015-02-18 16:54:25 +00:00
Erik Johnston 5358966a87 Use git aware version string in User-Agent and Server headers 2015-02-18 16:52:04 +00:00
Erik Johnston aa577df064 When computing git version run git commands in same dir as source files 2015-02-18 16:52:04 +00:00
Erik Johnston d122e215ff Generate a version string that includes git details if run from git checkout 2015-02-18 16:52:04 +00:00
Erik Johnston a7925259a1 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1 2015-02-18 13:57:55 +00:00
Erik Johnston 7d304ae11c Merge pull request #80 from matrix-org/restrict-destinations
Restrict the destinations that synapse can talk to
2015-02-18 13:56:48 +00:00
Erik Johnston 446ef58992 Add errback to all deferreds in transaction_queue 2015-02-18 12:03:26 +00:00
Erik Johnston cc3d3babb0 Remove unused import 2015-02-18 12:01:41 +00:00
Mark Haines 6375bd3e33 SYN-282: Don't log tracebacks for client errors 2015-02-18 12:01:37 +00:00
Mark Haines 2462aacd77 Restrict the destinations that synapse can talk to 2015-02-18 11:52:51 +00:00
Erik Johnston b68e4a729f Discard destination 'localhost' 2015-02-18 11:32:39 +00:00
Erik Johnston 47d3ff4cf8 Don't send failure to self 2015-02-18 11:30:37 +00:00
Erik Johnston 36e144091b Remove spurious comma. Remove temp run_on_reactor 2015-02-18 11:25:20 +00:00
Erik Johnston b17bd31da0 Temporarily add a run_on_reactor() call 2015-02-18 11:17:26 +00:00
Mark Haines 5806d52423 Fix syntax 2015-02-18 11:01:37 +00:00
Mark Haines 87e9aeb914 Move pynacl to the top of the depedency link list so that it is
installed before syutil
2015-02-18 11:00:13 +00:00
Erik Johnston 7e9d59f3b4 Don't convert DNSLookupError to a 4xx SynapseError 2015-02-18 10:58:13 +00:00
Erik Johnston cedad8fbd6 Bump version 2015-02-18 10:54:34 +00:00
Erik Johnston 65ca713ff5 Add .__name__ after type(e) 2015-02-18 10:51:32 +00:00
Erik Johnston 5e24471469 Fix up ResponseNeverReceived to str 2015-02-18 10:50:10 +00:00
Erik Johnston e482541e1d Fix pyflakes 2015-02-18 10:44:22 +00:00
Erik Johnston 0db52d43fa strings.join() expects iterable of strings 2015-02-18 10:41:46 +00:00
Erik Johnston 859fbd4423 s/self._clock/self.clock/ 2015-02-18 10:39:14 +00:00
Erik Johnston 1be67eca8a Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop 2015-02-18 10:34:40 +00:00
Erik Johnston 2635d4e634 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-02-18 10:29:54 +00:00
Erik Johnston fe672a04f7 Merge pull request #77 from matrix-org/failures
Failures
2015-02-18 10:29:29 +00:00
Erik Johnston 08f804208b Merge pull request #79 from matrix-org/get_pdu_limiting
Get pdu limiting
2015-02-18 10:29:10 +00:00
Erik Johnston ec847059f3 Rename _fail_fetch_pdu_cache to _get_pdu_cache 2015-02-18 10:14:10 +00:00
Erik Johnston 4fd176a41d More docs 2015-02-18 10:11:24 +00:00
Erik Johnston d77912ff44 Docs. 2015-02-18 10:09:54 +00:00
Erik Johnston 9371019133 Try to only back off if we think we failed to connect to the remote 2015-02-17 18:13:34 +00:00
Erik Johnston 649dc8a7e2 Merge branch 'develop' of github.com:matrix-org/synapse into failures 2015-02-17 17:43:14 +00:00
Erik Johnston c8436b38a0 Only update destination_retry_timings if we have succeeded when retrying 2015-02-17 17:38:38 +00:00
Erik Johnston f91263b1e0 Remove spurious self 2015-02-17 17:37:51 +00:00
Erik Johnston 1177245e86 Merge branch 'hotfixes-v0.7.0g' of github.com:matrix-org/synapse into develop 2015-02-17 17:30:11 +00:00
Erik Johnston 20e3172f38 Merge pull request #75 from matrix-org/dont_write_bytecode
Don't write bytecode
2015-02-17 17:29:55 +00:00
Erik Johnston 58554fa658 Merge branch 'develop' of github.com:matrix-org/synapse into keyclient_retry_scheme 2015-02-17 17:26:46 +00:00
Erik Johnston 2c29ed3e84 Use absolute path when loading delta sql files 2015-02-17 17:22:24 +00:00
Erik Johnston 2b8f1a956c Add per server retry limiting.
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
Erik Johnston 5025305fb2 Rate limit retries when fetching server keys. 2015-02-17 15:57:42 +00:00
Erik Johnston 1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Erik Johnston 964bb43fbe Fix typo in function name 2015-02-17 15:44:41 +00:00
Erik Johnston e7e20417ca ExpiringCache: purge every 1/2 interval 2015-02-17 15:44:26 +00:00
Erik Johnston 8b919c00f3 Start the get_pdu cache 2015-02-17 15:44:01 +00:00
Erik Johnston 676e8ee78a Remove debug raise 2015-02-17 15:22:45 +00:00
Erik Johnston 08e70231c9 Merge branch 'develop' of github.com:matrix-org/synapse into failures 2015-02-17 15:21:33 +00:00
Erik Johnston 0647e27a41 Remove unused import 2015-02-17 15:19:54 +00:00
Erik Johnston fa6c93bd26 Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop 2015-02-17 15:18:17 +00:00
Erik Johnston c02da58a9d Merge branch 'develop' of github.com:matrix-org/synapse into failures 2015-02-17 15:15:07 +00:00
Erik Johnston 472734a8cc Consume errors in time_bound_deferred 2015-02-17 15:13:50 +00:00
Erik Johnston 4de93001bf Make matrixfederationclient log more nicely 2015-02-17 15:12:06 +00:00
Erik Johnston 659ead082f Format the response of transaction request in a nicer way 2015-02-17 15:11:44 +00:00
Erik Johnston c82e26ad4b Actually respond with JSON to incoming transaction 2015-02-17 13:24:13 +00:00
Erik Johnston 47281f8fa4 Change some debug logging to info 2015-02-17 13:14:11 +00:00
Erik Johnston 02bfa889de Handle recieving failures in transactions 2015-02-17 13:13:14 +00:00
Erik Johnston c2b1dbd84c We do want to consumeError 2015-02-17 11:11:11 +00:00
Erik Johnston ea1d6c16cd Don't write bytecode 2015-02-17 10:54:06 +00:00
Erik Johnston 72a4de2ce6 Use consumeErrors=True on all DeferredLists.
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
Erik Johnston 0194e71e99 Merge branch 'develop' of github.com:matrix-org/synapse into get_pdu_limiting 2015-02-17 09:48:23 +00:00
Erik Johnston baa5b9a975 Cache results of get_pdu. 2015-02-16 18:02:39 +00:00
Erik Johnston 2674aeb96a Factor out ExpiringCache from StateHandler 2015-02-16 16:16:47 +00:00
Erik Johnston 91fc5eef1d Mark old events as outliers.
This is to fix the issue where if a remote server sends an event
that references a really "old" event, then the local server will pull
that in and send to all clients.

We decide if an event is old if its depth is less than the minimum depth
of the room.
2015-02-16 14:27:40 +00:00
Erik Johnston 6138584651 Don't return anything from _handle_new_pdu, since we ignore the return value anyway 2015-02-16 14:08:02 +00:00
Erik Johnston 8a59915d7d Merge branch 'hotfixes-v0.7.0f' of github.com:matrix-org/synapse into develop 2015-02-16 09:47:22 +00:00
Erik Johnston 6dd5c95841 Bump version 2015-02-15 20:38:52 +00:00
Erik Johnston b99a33f283 resolve_events expect lists, not dicts 2015-02-15 20:20:51 +00:00
Erik Johnston 5f68529036 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-13 16:21:30 +00:00
David Baker 64def4f953 Merge branch 'hotfixes-0.7.0e' into develop 2015-02-13 16:18:34 +00:00
Erik Johnston a78838c5ba Bump version 2015-02-13 16:17:54 +00:00
David Baker 8d5cce62ab Update pushers by app id and pushkey, not user id and pushkey 2015-02-13 16:16:16 +00:00
Erik Johnston 650dc7f0f9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-13 15:46:42 +00:00
Erik Johnston be26697b29 Bump version 2015-02-13 15:37:35 +00:00
Erik Johnston b11a6e1c3c Fix wrong variable name 2015-02-13 15:37:18 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Mark Haines fa662b52d0 Merge pull request #72 from matrix-org/in_memory_sqlite_for_testing
Prepare the database whenever a connection is opened from the db_pool so...
2015-02-13 14:42:27 +00:00
Mark Haines 183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston 0546126cc5 Bump version 2015-02-13 14:36:40 +00:00
Erik Johnston e441c10a73 pyflakes 2015-02-13 14:23:39 +00:00
Erik Johnston 8c652a2b5f When we see a difference in current state, actually use state conflict resolution algorithm 2015-02-13 14:20:05 +00:00
Erik Johnston 6375abcdac Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-13 13:34:25 +00:00
Erik Johnston c09493d7aa Bump version 2015-02-13 13:33:37 +00:00
Erik Johnston 74626a8de4 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-13 10:09:55 +00:00
Paul "LeoNerd" Evans 55e0916ffc Reindent code to be less human-readable to keep pep8 from complaining 2015-02-12 21:04:34 +00:00
Paul "LeoNerd" Evans f22646efcc Only attempt to fetch presence state of JOINed members in room initialSync (SYN-202) 2015-02-12 21:01:29 +00:00
Erik Johnston 789251afa7 Fix logging 2015-02-12 19:29:43 +00:00
Erik Johnston 38df10b99e Remove unused function 2015-02-12 19:29:32 +00:00
Paul "LeoNerd" Evans 93d07c87dc Reindent code to be less human-readable to keep pep8 from complaining 2015-02-12 19:19:37 +00:00
Paul "LeoNerd" Evans 5f6e6530d0 Appease pyflakes 2015-02-12 19:15:23 +00:00
Paul "LeoNerd" Evans 29805213d1 Can now remove the FIXME too 2015-02-12 19:13:21 +00:00
Paul "LeoNerd" Evans 860b1b4841 Only attempt to fetch presence state of JOINed members in room initialSync (SYN-202) 2015-02-12 19:13:21 +00:00
Erik Johnston 58d848adc0 Parrellize fetching of events 2015-02-12 18:35:36 +00:00
Erik Johnston 963256638d Correctly handle all the places that can throw exceptions 2015-02-12 18:17:11 +00:00
Erik Johnston 48fbe79f71 Bump version 2015-02-12 14:52:02 +00:00
Erik Johnston 6b186a57ba Merge branch 'fix' of github.com:matrix-org/synapse into hotfixes-v0.7.0a 2015-02-12 14:51:46 +00:00
Erik Johnston 717687e1fc Get an auth query one at a time 2015-02-12 14:39:31 +00:00
Erik Johnston a566ed2f0e Bump webclient version 2015-02-12 10:49:07 +00:00
Erik Johnston 83d41f25d8 Set database schema version in delta 2015-02-12 10:05:47 +00:00
Erik Johnston ff2a2ae56e Bump version 2015-02-12 10:02:06 +00:00
Erik Johnston 8bbdf32849 Convert get_rooms to use runInteraction so the transacion has a more helpful description 2015-02-11 18:56:13 +00:00
Erik Johnston 2bf0e85f3d Use encode_canonical_json for http client 2015-02-11 17:34:23 +00:00
Erik Johnston e9e54449f5 Use encode_canonical_json for pushes 2015-02-11 17:32:42 +00:00
Erik Johnston af89456c3c Update dependency links 2015-02-11 17:15:16 +00:00
Erik Johnston c52e8d395b Merge pull request #61 from matrix-org/timeout-federation-requests
Timeout federation requests
2015-02-11 17:10:33 +00:00
Erik Johnston 021d93db11 Merge pull request #62 from matrix-org/state-chache
State chache
2015-02-11 17:10:28 +00:00
Erik Johnston a793a0b810 Bump syutil version 2015-02-11 17:02:52 +00:00
Erik Johnston 42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Mark Haines 9c24cff6ef Allow newer versions of syutil 2015-02-11 17:00:32 +00:00
Erik Johnston 7eef84a95b pyflakes 2015-02-11 16:52:22 +00:00
Erik Johnston 76935078d1 Remove more debug logging 2015-02-11 16:51:22 +00:00
Erik Johnston ed877d6585 Remove debug logging 2015-02-11 16:50:46 +00:00
Erik Johnston ef276e8770 Fix so timing out connections to actually work. 2015-02-11 16:48:05 +00:00
Kegan Dougal f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Kegan Dougal f518324426 Minor tweaks based on PR feedback. 2015-02-11 16:41:16 +00:00
Paul "LeoNerd" Evans b164e0896c Merge branch 'bugs/SYN-264' into develop 2015-02-11 16:23:30 +00:00
Paul "LeoNerd" Evans 7f47ba7383 Added another TODO note 2015-02-11 16:18:21 +00:00
Erik Johnston 41a9a76a99 Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 16:12:59 +00:00
Paul "LeoNerd" Evans 45b56609ae Cache the result of a get_rooms_for_user query, to make user_rooms_intersect() much lighter in the read-common case 2015-02-11 16:04:08 +00:00
Paul "LeoNerd" Evans 7be0f6594e First step of making user_rooms_intersect() faster - implement in intersection logic in Python code terms of a DB query that is cacheable per user 2015-02-11 15:53:56 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Mark Haines ddc25cf4e2 Invalidate the cache for an event if it is redacted 2015-02-11 15:23:28 +00:00
Mark Haines aff892ce79 Fix formatting 2015-02-11 15:02:35 +00:00
Mark Haines f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Mark Haines d8324d5a2b Add a lru cache class 2015-02-11 14:52:23 +00:00
Erik Johnston 4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Kegan Dougal 14d413752b Fix newline on __init__ 2015-02-11 10:53:47 +00:00
Kegan Dougal fd40d992ad PEP8-ify 2015-02-11 10:41:33 +00:00
Kegan Dougal 8beb613916 Add newline to EOF 2015-02-11 10:36:48 +00:00
Kegan Dougal c7783d6fee Notify ASes for events sent by other users in a room which an AS user is a part of. 2015-02-11 10:36:08 +00:00
Erik Johnston 5758dafb4e Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-11 10:35:13 +00:00
Erik Johnston 6370cffbbf Fix bug where variable was not always defined 2015-02-11 10:34:41 +00:00
Erik Johnston fb233dc40b Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 10:33:19 +00:00
Erik Johnston 05b961d7e3 PEP8 2015-02-11 10:28:46 +00:00
Erik Johnston dcf52469e8 Move time_bound_deferred into Clock 2015-02-11 10:25:06 +00:00
Erik Johnston 8c83cc471b Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-02-11 10:20:43 +00:00
Kegan Dougal 9978c5c103 Merge branch 'develop' into application-services 2015-02-11 10:03:24 +00:00
Erik Johnston 0e6b3e4e40 Time out HTTP federation requests 2015-02-10 18:17:27 +00:00
Mark Haines 5e54365234 Merge branch 'develop' into single_source_version_and_dependencies 2015-02-10 18:13:25 +00:00
Mark Haines 84a769cdb7 Fix code-style 2015-02-10 17:58:36 +00:00
Mark Haines 7ed971d9b2 Single source version and python dependencies, prevent people accidentally installing with easy_install, use scripts rather than entry_points to install synctl 2015-02-10 17:42:36 +00:00
Erik Johnston eae0842bc1 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-10 17:34:51 +00:00
Erik Johnston c8e1da930d Log all the exits from _attempt_new_transaction 2015-02-10 17:30:46 +00:00
Erik Johnston b61a308b27 Bump version 2015-02-10 16:37:12 +00:00
Erik Johnston e8d4a31475 Fix prune_events to work with nested dicts 2015-02-10 16:36:51 +00:00
Mark Haines b085fac735 Code-style fixes 2015-02-10 16:30:48 +00:00
Erik Johnston 093e34e301 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-10 15:46:48 +00:00
Erik Johnston 697ab75a34 Sign auth_chains when returned by /state/ requests 2015-02-10 15:46:24 +00:00
Erik Johnston f8abbae99f Remove unnecessary logging 2015-02-10 15:45:50 +00:00
Erik Johnston 794fe2ca45 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-10 15:23:08 +00:00
Mark Haines f88d3ee8ae Merge pull request #58 from matrix-org/get_event_counters
Add performance counters for different stages of loading events
2015-02-10 15:15:20 +00:00
Mark Haines fda4422bc9 Fix pyflakes 2015-02-10 14:54:07 +00:00
Mark Haines d7c7efb691 Add performance counters for different stages of loading events 2015-02-10 14:50:53 +00:00
Erik Johnston 91f0e41153 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-10 14:30:26 +00:00
Erik Johnston f91345bdb5 yaml.load expects strings to be a yaml rather than file 2015-02-10 13:57:31 +00:00
Erik Johnston 30595b466f Use yaml logging config format because it is much nicer 2015-02-10 13:50:33 +00:00
Erik Johnston c86ebe7673 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache
Conflicts:
	synapse/app/homeserver.py
	synapse/state.py
2015-02-10 11:04:37 +00:00
Erik Johnston 2b042ad67f Oops, we do want to defer.return regardless of whether we are caching or not 2015-02-10 11:03:16 +00:00
Erik Johnston d19e2ed02f Move construction of object within if block 2015-02-10 11:01:15 +00:00
Erik Johnston b90d377af4 Merge branch 'release-v0.6.2' of github.com:matrix-org/synapse into develop 2015-02-10 10:33:35 +00:00
Mark Haines 8ce100c7b4 Convert directory paths to absolute paths before daemonizing 2015-02-09 18:30:00 +00:00
Mark Haines 5c5f5c1f0e Merge pull request #56 from matrix-org/room_initial_sync_perf
During room intial sync, only calculate current state once.
2015-02-09 18:09:30 +00:00
Erik Johnston 375eba6a18 Merge pull request #57 from matrix-org/transaction_counters
Transaction counters
2015-02-09 18:08:08 +00:00
Mark Haines 0c4536da8f Use the transaction 'desc' rather than 'name', increment the txn_ids in
txn names
2015-02-09 18:06:31 +00:00
Mark Haines 347b497db0 Formatting 2015-02-09 17:57:09 +00:00
Mark Haines 3a5ad7dbd5 Performance counters for database transaction names 2015-02-09 17:55:56 +00:00
Erik Johnston d94f682a4c During room intial sync, only calculate current state once. 2015-02-09 17:41:29 +00:00
David Baker 8f616684a3 Need to use re.search if looking for matches not at the start of the string. Also comparisons with None should be 'is'. 2015-02-09 17:01:40 +00:00
Matthew Hodgson 0b725f5c4f oops 2015-02-09 16:48:31 +00:00
Matthew Hodgson bd2373277d oops 2015-02-09 16:48:09 +00:00
Matthew Hodgson a578251b48 only do word-boundary patches on bodies for now 2015-02-09 16:44:47 +00:00
Kegan Dougal 53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00
Kegan Dougal f7cac2f7b6 Fix bugs so lazy room joining works as intended. 2015-02-09 15:01:28 +00:00
Erik Johnston 76c5a5c2f6 Merge pull request #55 from matrix-org/profiling
Profiling
2015-02-09 15:01:26 +00:00
Erik Johnston c4ee4ce93e Fix typo 2015-02-09 15:00:37 +00:00
Erik Johnston ef995e6946 Add looping_call to Clock 2015-02-09 14:47:59 +00:00
Erik Johnston 66fde49f07 Log database time every 10s and log as percentage 2015-02-09 14:45:15 +00:00
Erik Johnston 75656712e3 Time how long we're spending on the database thread 2015-02-09 14:22:52 +00:00
David Baker 784d714a3f Fix server default rule injection (downwards, not upwards!) 2015-02-09 14:17:52 +00:00
Kegan Dougal 5a7dd05818 Modify auth.get_user_by_req for authing appservices directly.
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
Erik Johnston 24cc6979fb Log when we receive a request, when we send a response and how long it took to process it. 2015-02-09 13:46:22 +00:00
Kegan Dougal ac3183caaa Register a user account for the AS when the AS registers. Add 'sender' column to AS table. 2015-02-09 12:03:37 +00:00
Matthew Hodgson ecb0f78063 glob *s should probably be non-greedy 2015-02-08 02:37:35 +00:00
Matthew Hodgson c2afc2ad90 oops 2015-02-08 00:37:03 +00:00
Matthew Hodgson 8be07e0db4 kill off fnmatch in favour of word-boundary based push alerts (untested) 2015-02-08 00:34:34 +00:00
Matthew Hodgson 37b6b880ef don't give up if we can't create default avatars during tests 2015-02-07 21:24:08 +00:00
Matthew Hodgson 582019f870 ...and here's the actual impl. git fail. 2015-02-07 13:32:14 +00:00
Matthew Hodgson e117bc3fc5 thou shalt specify a content-length 2015-02-07 12:56:21 +00:00
Matthew Hodgson 34c39398fa i hate weakly typed languages 2015-02-07 12:55:13 +00:00
Kegan Dougal 73a680b2a8 Add errcodes for appservice registrations. 2015-02-06 17:10:04 +00:00
Erik Johnston af613824e4 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-06 16:59:00 +00:00
Erik Johnston 5bf318e9a6 Bug fixes. 2015-02-06 16:52:22 +00:00
Erik Johnston b4886264a3 Bugfix cache layer 2015-02-06 16:17:05 +00:00
Erik Johnston c4e3029d55 Add cache layer to state group resolution 2015-02-06 16:08:13 +00:00
Mark Haines 20db147ef3 SYN-258: get_recent_events_for_room only accepts stream tokens, convert the topological token to a stream token before passing it to get_recent_events_for_room 2015-02-06 16:01:04 +00:00
Mark Haines 55a186485c SYN-258: get_recent_events_for_room only accepts stream tokens, convert the topological token to a stream token before passing it to get_recent_events_for_room 2015-02-06 15:58:40 +00:00
Erik Johnston cc0532a4bf Explicitly list the RejectedReasons that we can prove 2015-02-06 15:16:26 +00:00
Erik Johnston 0cd66885e3 Move delta/v13.sql to delta/v12.sql 2015-02-06 14:38:04 +00:00
Erik Johnston e890ce223c Don't query auth if the only difference is events that were rejected due to auth. 2015-02-06 14:16:50 +00:00
Erik Johnston c78b5fb1f1 Make seen_ids a set 2015-02-06 13:52:16 +00:00
Kegan Dougal 0995810273 Pyflakes: unused variable. 2015-02-06 11:45:19 +00:00
Kegan Dougal c3ae8def75 Grant ASes the ability to delete aliases in their own namespace. 2015-02-06 11:32:07 +00:00
Kegan Dougal e426df8e10 Grant ASes the ability to create alias in their own namespace.
Add a new errcode type M_EXCLUSIVE when users try to create aliases inside
AS namespaces, and when ASes try to create aliases outside their own
namespace.
2015-02-06 10:57:14 +00:00
Erik Johnston 9f2573eea1 Return body of response in HttpResponseException 2015-02-06 10:55:01 +00:00
Erik Johnston 3737329d9b Handle the fact the list.remove raises if element doesn't exist 2015-02-06 10:53:18 +00:00
Kegan Dougal 0227618d3c Add m.login.application_service registration procedure.
This allows known application services to register any user ID under their
own user namespace(s).
2015-02-05 17:29:27 +00:00
Kegan Dougal 11e6b3d18b Dependency inject ApplicationServiceApi when creating ApplicationServicesHandler. 2015-02-05 17:04:59 +00:00
Kegan Dougal a3c6010718 Add delta sql file. 2015-02-05 16:48:57 +00:00
Kegan Dougal cab4c73088 Prevent user IDs in AS namespaces being created/deleted by humans. 2015-02-05 16:46:56 +00:00
Kegan Dougal e9484d6a95 Prevent aliases in AS namespaces being created/deleted by users. Check with ASes when queried for room aliases via federation. 2015-02-05 16:29:56 +00:00
Kegan Dougal c20281ee33 Merge branch 'develop' into application-services 2015-02-05 16:11:34 +00:00
David Baker a93fa42bce priority class now dealt with in namespaced rule_id 2015-02-05 15:45:16 +00:00
Kegan Dougal fc8bcc809d Merge branch 'develop' into application-services 2015-02-05 15:32:45 +00:00
Kegan Dougal 5b99b471b2 Fix unit tests. 2015-02-05 15:12:36 +00:00
David Baker aaf50bf6f3 Give server default rules the 'default' attribute and fix various brokenness. 2015-02-05 15:11:38 +00:00
Kegan Dougal c163357f38 Add CS extension for masquerading as users within the namespaces specified by the AS. 2015-02-05 15:00:33 +00:00
David Baker 2df41aa138 Server default rules now of all kinds rather than all being at lowest prio. 2015-02-05 14:46:37 +00:00
David Baker f90782a658 namespace rule IDs to be unique within their scope and rule type. 2015-02-05 14:46:37 +00:00
Kegan Dougal 951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Kegan Dougal c71456117d Fix user query checks. HS>AS pushing now works. 2015-02-05 14:17:08 +00:00
Erik Johnston 8f5b858a1b Merge branch 'develop' of github.com:matrix-org/synapse into federation_client_retries 2015-02-05 13:50:28 +00:00
Erik Johnston e9c85a4d5a Connection errors in twisted aren't RuntimeErrors 2015-02-05 13:50:15 +00:00
Erik Johnston e1515c3e91 Pass through list of room hosts from room alias query to federation so that it can retry against different room hosts 2015-02-05 13:44:42 +00:00
Kegan Dougal 0613666d9c Serialize events before sending to ASes 2015-02-05 13:42:35 +00:00
Kegan Dougal 51d63ac329 Glue AS work to general event notifications. Add more exception handling when poking ASes. 2015-02-05 13:19:46 +00:00
Erik Johnston 26a041541b SYN-202: Log as WARN the 404 'Presence information not visible' errors instead of as ERROR since they were spamming the logs 2015-02-05 13:17:05 +00:00
Kegan Dougal b932600653 Add unknown room alias check. Call it from directory_handler.get_association 2015-02-05 11:47:11 +00:00
Kegan Dougal f0c730252f Add unknown user ID check. Use store.get_aliases_for_room(room_id) when searching for services by alias. 2015-02-05 11:25:32 +00:00
Kegan Dougal 27091f146a Add hs_token column and generate a different token f.e application service. 2015-02-05 10:08:12 +00:00
Kegan Dougal a1a4960baf Impl push_bulk function 2015-02-05 09:43:22 +00:00
Erik Johnston 77e5ae22a9 Ver bump 2015-02-04 23:51:34 +00:00
Erik Johnston 19ebdc321d Pull in python_dependencies.py from develop 2015-02-04 23:51:02 +00:00
Kegan Dougal 543e84fe70 Add SimpleHttpClient.put_json with the same semantics as get_json. 2015-02-04 17:39:51 +00:00
Erik Johnston 8046df6efa Merge branch 'develop' of github.com:matrix-org/synapse into federation_client_retries 2015-02-04 17:37:34 +00:00
Kegan Dougal 6d3e4f4d0a Update user/alias query APIs to use new format of SimpleHttpClient.get_json 2015-02-04 17:32:44 +00:00
Kegan Dougal 96d4bf9012 Modify API for SimpleHttpClient.get_json and update usages.
Previously, this would only return the HTTP body as JSON, and discard other
response information (e.g. the HTTP response code). This has now been changed
to throw a CodeMessageException on a non-2xx response, with the response code
and body, which can then be parsed as JSON.

Affected modules include:
 - Registration/Login (when using an email for IS auth)
2015-02-04 17:07:31 +00:00
Kegan Dougal aa8cce58bf Add query_user/alias APIs. 2015-02-04 16:44:53 +00:00
Erik Johnston d45e2302ed Merge branch 'signature_failures' of github.com:matrix-org/synapse into federation_client_retries 2015-02-04 16:30:15 +00:00
Erik Johnston ae46f10fc5 Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations 2015-02-04 16:28:12 +00:00
David Baker 2e77ba637a More s/instance_handle/profile_tag/ 2015-02-04 16:24:15 +00:00
Kegan Dougal ce8bc642ae Merge branch 'develop' into application-services 2015-02-04 15:31:02 +00:00
Kegan Dougal 89f2e8fbdf Fix bug in store defer. Add more unit tests. 2015-02-04 15:21:03 +00:00
Erik Johnston 95e2d2d36d When returning lists of servers from alias lookups, put the current server first in the list 2015-02-04 15:02:23 +00:00
Erik Johnston 650e32d455 Change context.auth_events to what the auth_events would be bases on context.current_state, rather than based on the auth_events from the event. 2015-02-04 14:06:46 +00:00
Erik Johnston ff78eded01 Retry make_join 2015-02-04 13:55:10 +00:00
Kegan Dougal 525a218b2b Begin to add unit tests for appservice glue and regex testing. 2015-02-04 12:24:20 +00:00
Kegan Dougal 17753f0c20 Add stub ApplicationServiceApi and glue it with the handler. 2015-02-04 11:19:18 +00:00
Erik Johnston 03d415a6a2 Brief comment on why we do some things on every call to persist_event and not others 2015-02-04 10:40:59 +00:00
Erik Johnston f275ba49bb Fix state resolution to remember join_rules is a type of auth event. 2015-02-04 10:36:28 +00:00
Erik Johnston c0462dbf15 Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event. 2015-02-04 10:16:51 +00:00
Erik Johnston 02be8da5e1 Add doc to get_event 2015-02-03 17:34:07 +00:00
David Baker dc7bb70f22 s/instance_handle/profile_tag/ 2015-02-03 16:51:07 +00:00
Erik Johnston 3c39f42a05 New line 2015-02-03 16:14:19 +00:00
Erik Johnston 7dd1c5c542 Neaten the handling of state and auth_chain up a bit 2015-02-03 16:12:04 +00:00
David Baker 9a71add1c0 Use set_tweak instead of set_sound 2015-02-03 16:06:31 +00:00
Erik Johnston 9bace3a367 Actually, the old prune_event function was non-deterministic, so no point keeping it around :( 2015-02-03 15:32:17 +00:00
Erik Johnston 8dae5c8108 Remove unused imports 2015-02-03 15:01:12 +00:00
Erik Johnston 7b810e136e Add new FederationBase 2015-02-03 15:00:42 +00:00
Erik Johnston 0dd3aea319 Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers 2015-02-03 14:58:30 +00:00
Kegan Dougal 94a5db9f4d Add appservice package and move ApplicationService into it. 2015-02-03 14:44:16 +00:00
Erik Johnston 6efd4d1649 Don't completely die if get auth_chain or querying auth_chain requests fail 2015-02-03 13:57:54 +00:00
Erik Johnston 77a076bd25 Set combinations is | and not + 2015-02-03 13:35:17 +00:00
Kegan Dougal f2c039bfb9 Implement restricted namespace checks. Begin fleshing out the main hook for notifying application services. 2015-02-03 13:29:27 +00:00
Erik Johnston fed29251d7 Spelling 2015-02-03 13:23:58 +00:00
Kegan Dougal a060b47b13 Add namespace constants. Add restrict_to option to limit namespace checks. 2015-02-03 13:17:28 +00:00
Kegan Dougal 3bd2841fdb Everyone loves SQL typos 2015-02-03 11:37:52 +00:00
Kegan Dougal 197f3ea4ba Implement regex checks for app services.
Expose handler.get_services_for_event which manages the checks for all
services.
2015-02-03 11:26:33 +00:00
Erik Johnston 06c34bfbae Give exception better message 2015-02-03 11:23:44 +00:00
Erik Johnston 4ff2273b30 Add FIXME note. 2015-02-03 11:23:26 +00:00
Erik Johnston 0f48e22ef6 PEP8 2015-02-03 10:43:29 +00:00
Erik Johnston 51969f9e5f Return rejected events if asked for it over federation. 2015-02-03 10:40:14 +00:00
Erik Johnston e7ca813dd4 Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering. 2015-02-03 10:39:41 +00:00
Mark Haines 09601255f5 Merge pull request #46 from matrix-org/identicons
Add a media/v1/identicon resource for generating identicons
2015-02-02 18:56:34 +00:00
Kegan Dougal 9ff349a3cb Add defers in the right places. 2015-02-02 17:42:49 +00:00
Kegan Dougal 1a2de0c5fe Implement txns for AS (un)registration. 2015-02-02 17:39:41 +00:00
Mark Haines a2da04b8ab Add pydenticon to python_dependencies 2015-02-02 17:37:26 +00:00
Mark Haines 4574b5a9e6 Generate a list of dependencies from synapse/python_dependencies.py 2015-02-02 17:23:51 +00:00
Erik Johnston 40c6fe1b81 Don't bother requesting PDUs with bad signatures from the same server 2015-02-02 17:06:37 +00:00
Mark Haines 1bb0528316 Add Cache-Control header to identicon 2015-02-02 16:57:26 +00:00
Erik Johnston 941f59101b Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator. 2015-02-02 16:56:01 +00:00
Mark Haines f2eda123b7 Fix setting identicon width and height 2015-02-02 16:32:33 +00:00
Mark Haines 038f5afb07 Spell height more correctly 2015-02-02 16:29:18 +00:00
Kegan Dougal a006d168c5 Actually merge into develop. 2015-02-02 16:05:34 +00:00
Mark Haines 22c1ffb0a0 Add a media/v1/identicon resource for generating identicons using pydenticon 2015-02-02 16:02:31 +00:00
Kegan Dougal c059c9fea5 Merge branch 'develop' into application-services
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Mark Haines 6e856d7729 Merge master into develop
Conflicts:
	README.rst
	setup.py
2015-02-02 14:05:42 +00:00
Mark Haines d8cf06e525 Bump version to 0.6.1d 2015-02-02 13:18:36 +00:00
Erik Johnston 3b33529dfd Bump version 2015-02-02 13:03:25 +00:00
David Baker 365e007bee Ignore empty strings for display names & room names in notifications 2015-01-31 12:48:06 +00:00
David Baker 0b354fcb84 Again, don't assume all member events have displayname. 2015-01-30 23:10:35 +00:00
David Baker fe10b882b7 Don't assume all member events have a display nme. 2015-01-30 23:06:39 +00:00
Erik Johnston 4c0da49d7c Resign events when we return them via /query_auth/ 2015-01-30 22:53:13 +00:00
Mark Haines 0b1cc7cc0b Return empty list rather than None when there are no emphemeral events for a room 2015-01-30 16:56:13 +00:00
Erik Johnston 2cd29dbdd9 Fix bug where accepting invite over federation didn't work. Add logging. 2015-01-30 16:51:58 +00:00
Erik Johnston 7d897f5bfc Merge pull request #43 from matrix-org/rejections
Rejections
2015-01-30 16:11:14 +00:00
Erik Johnston 776ac820f9 Briefly doc structure of query_auth API. 2015-01-30 15:58:28 +00:00
Erik Johnston b724a809c4 Only auth_events with event if event in event.auth_events 2015-01-30 15:57:53 +00:00
Erik Johnston 7a1e881665 Remove debug logging 2015-01-30 15:56:32 +00:00
David Baker b4b892f4a3 Spit out server default rules too. 2015-01-30 15:54:51 +00:00
Mark Haines 6dc92d3427 Merge pull request #41 from matrix-org/client_v2_sync
Client v2 sync
2015-01-30 15:54:25 +00:00
Mark Haines 017dfaef4c Add doc string for __nonzero__ overrides for sync results, raise not implemented if the client attempts to do a gapless sync 2015-01-30 15:52:05 +00:00
Erik Johnston 1bd540ef79 Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/im.sql
2015-01-30 15:16:38 +00:00
Mark Haines 9ec9d6f2cb Merge pull request #42 from matrix-org/replication_split
Replication split
2015-01-30 15:14:10 +00:00
David Baker 4ffac34a64 Add glob asterisks when running rules.
Means that now you can't do exact matches even in override rules,
but I think we can live with that. Advantage is that you'll now
always get back what was put in to the API.
2015-01-30 15:03:56 +00:00
Erik Johnston 91015ad008 Remove merge conflict 2015-01-30 14:58:54 +00:00
Erik Johnston 4f7fe63b6d Remember to add schema file to list 2015-01-30 14:57:53 +00:00
Erik Johnston fdd2ac495a Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Mark Haines 8bc3066e0b Merge branch 'client_v2_filter' into client_v2_sync 2015-01-30 14:57:04 +00:00
Erik Johnston 471c47441d Merge pull request #37 from matrix-org/client_v2_filter
Client v2 filter
2015-01-30 14:56:08 +00:00
Mark Haines e97f756a05 Use 'in' to test if the key exists, remove unused _filters_for_user 2015-01-30 14:54:06 +00:00
Erik Johnston 2f4cb04f45 Be more specific in naming columns in selects. 2015-01-30 14:48:11 +00:00
Erik Johnston 472cf532b7 Put CREATE rejections into seperate .sql 2015-01-30 14:48:03 +00:00
David Baker 322a047502 Add room member count condition and default rule to make a noise on rooms of only 2 people. 2015-01-30 14:46:45 +00:00
Erik Johnston 3d7026e709 Add a slightly more helpful comment 2015-01-30 14:37:31 +00:00
Erik Johnston c515d37797 Merge branch 'replication_split' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:19:49 +00:00
Erik Johnston 84b78c3b5f Merge branch 'rejections_storage' of github.com:matrix-org/synapse into replication_split 2015-01-30 14:17:47 +00:00
Erik Johnston 2aaedab203 Merge branch 'develop' of github.com:matrix-org/synapse into new_state_resolution 2015-01-30 14:09:32 +00:00
Erik Johnston e0b7c521cb Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Erik Johnston 875a481a1e Merge branch 'new_state_resolution' of github.com:matrix-org/synapse into rejections_storage 2015-01-30 14:04:53 +00:00
Erik Johnston 7a9f6f083e Remove commented line 2015-01-30 13:55:46 +00:00
Erik Johnston 76d7fd39cd Style changes. 2015-01-30 13:52:02 +00:00
Mark Haines 8fe39a0311 Check if the user has joined the room between incremental syncs 2015-01-30 13:38:34 +00:00
Erik Johnston a70a801184 Fix bug where we superfluously asked for current state. Change API of /query_auth/ so that we don't duplicate events in the response. 2015-01-30 13:34:01 +00:00
Mark Haines 4a67834bc8 Pass client info to the sync_config 2015-01-30 11:50:15 +00:00
Mark Haines c562f237f6 Unused import 2015-01-30 11:43:00 +00:00
Mark Haines 8498d348d8 Fix token formatting 2015-01-30 11:42:09 +00:00
Mark Haines e97de6d96a Filter the recent events before applying the limit when doing an initial sync 2015-01-30 11:35:20 +00:00
Mark Haines 22dd1cde2d Filter the recent events before applying the limit when doing an incremental sync with a gap 2015-01-30 11:32:35 +00:00
Erik Johnston 2c9e136d57 Fix bad merge fo python_dependencies.py 2015-01-30 11:14:33 +00:00
David Baker bd03947c05 We do need Twisted 14, not 15: we use internal Twisted things that have been removed in 15. 2015-01-30 11:13:42 +00:00
Erik Johnston 2ebf795c0a Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Erik Johnston 0c2d245fdf Update the current state of an event if we update auth events. 2015-01-30 11:08:52 +00:00
Erik Johnston 823999716e Fix bug in timeout handling in keyclient 2015-01-30 11:08:01 +00:00
Erik Johnston c1d860870b Fix regression where we no longer correctly handled the case of gaps in our event graph 2015-01-30 10:48:47 +00:00
Erik Johnston c1c7b39827 Fix bug where we changes in outlier in metadata dict propogated to other events 2015-01-30 10:30:54 +00:00
David Baker fc946f3b8d Include content in notification pokes 2015-01-29 21:59:17 +00:00
David Baker 0b16886397 Change 'from' in notification pokes to 'sender' to match client API v2. Send sender display names where they exist. 2015-01-29 18:51:22 +00:00
David Baker 1235f7f383 Add default push rules including setting a sound for messages mentioning your username / display name 2015-01-29 18:38:22 +00:00
Mark Haines ece828a7b7 Update todo for the filtering on sync 2015-01-29 18:15:24 +00:00
Mark Haines 365a186729 Add basic filtering support 2015-01-29 18:11:28 +00:00
Mark Haines 7ceda8bf3d Merge branch 'client_v2_filter' into client_v2_sync 2015-01-29 18:04:07 +00:00
Mark Haines 93ed31dda2 Create a separate filter object to do the actual filtering, so that we can
split the storage and management of filters from the actual filter code
and don't have to load a filter from the db each time we filter an event
2015-01-29 17:45:07 +00:00
David Baker 4bdfce30d7 Renumber priority classes so we can use 0 for defaults. 2015-01-29 17:12:11 +00:00
David Baker e0d2c6889b Allow kind to be set to null to delete a pusher. 2015-01-29 17:05:00 +00:00
Erik Johnston 78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Mark Haines 4ad45f2582 Fix indent 2015-01-29 16:41:49 +00:00
Mark Haines 722b65f461 Move typing notifs to an "emphermal" event list on the room object 2015-01-29 16:41:21 +00:00
Mark Haines cc42d3f907 Fix check for empty room update 2015-01-29 16:27:38 +00:00
Mark Haines 4d9dd9bdc0 Fix v2 initial sync 2015-01-29 16:23:03 +00:00
Mark Haines 8e571cbed8 Merge branch 'client_v2_filter' into client_v2_sync 2015-01-29 16:18:59 +00:00
Mark Haines 295322048d Merge branch 'develop' into client_v2_filter 2015-01-29 16:18:34 +00:00
Mark Haines acb68a39e0 Code style fixes. 2015-01-29 16:12:40 +00:00
David Baker 8b1dd9f57f Only send a badge-reset if the user actually has unread notifications. 2015-01-29 16:10:01 +00:00
Mark Haines 9150a0d62e Fix code-style 2015-01-29 16:01:14 +00:00
Mark Haines cf7c54ec93 Merge branch 'client_v2_filter' into client_v2_sync 2015-01-29 15:55:58 +00:00
Mark Haines 33391db5f8 Merge in auth changes from develop 2015-01-29 15:54:54 +00:00
Mark Haines 396a67a09a Merge branch 'client_v2_filter' into client_v2_sync
Conflicts:
	synapse/rest/client/v2_alpha/__init__.py
2015-01-29 14:58:00 +00:00
Mark Haines 9d8f798a3f Merge changes from develop 2015-01-29 14:55:27 +00:00
Mark Haines e4f50fa0aa Move bump schema delta 2015-01-29 14:53:18 +00:00
Mark Haines e016f4043b Use get_room_events_stream to get changes to the rooms if the number of changes is small 2015-01-29 14:40:28 +00:00
Erik Johnston 5a3a15f5c1 Make post_json(...) actually send data. 2015-01-29 13:58:22 +00:00
Erik Johnston c183cec8f6 Add post_json(...) method to federation client 2015-01-29 13:44:52 +00:00
Kegan Dougal 83172487b0 Add basic filtering public API unit tests. Use defers in the right places. 2015-01-29 12:20:59 +00:00
Kegan Dougal 777d9914b5 Implement filter algorithm. Add basic event type unit tests to assert it works. 2015-01-29 11:38:06 +00:00
Kegan Dougal 50de1eaad9 Add filtering public API; outline filtering algorithm. 2015-01-29 10:24:57 +00:00
Kegan Dougal 2a4fda7b88 Add filtering.filter_events function, with stub passes_filter function. 2015-01-29 09:27:16 +00:00
Kegan Dougal 3773759c0f Also edit the filter column on the delta SQL 2015-01-29 09:15:33 +00:00
Mark Haines e3e72b8c5c Remove typing TODO 2015-01-29 03:35:25 +00:00
Mark Haines 3dbce6f4a5 Add typing notifications to sync 2015-01-29 03:33:51 +00:00
Mark Haines b9c442c85c Include transaction ids in unsigned section of events in the sync results for the clients that made those requests 2015-01-29 02:46:00 +00:00
Mark Haines 1b4a164c02 Add support for formatting events in the way a v2 client expects 2015-01-29 02:34:35 +00:00
Mark Haines b0b80074e0 SYN-252: Supply the stream and topological parts in the correct order to the constructor 2015-01-29 01:48:48 +00:00
David Baker d5bdf3c0c7 Allow the push rule delete method to take more specifiers. 2015-01-28 18:06:04 +00:00
David Baker 8552ed8df2 Change uses of get_user_by_req because it returns a tuple now. 2015-01-28 18:04:40 +00:00
Kegan Dougal 11634017f4 s/definition/filter_json/ since definition is now used to mean a component of the filter, rather than the complete json 2015-01-28 17:42:19 +00:00
Mark Haines c81a19552f Add ports back to demo/start.sh 2015-01-28 17:32:49 +00:00
Mark Haines 9c61556504 Merge branch 'develop' into client_v2_sync 2015-01-28 17:29:30 +00:00
Mark Haines 26c8fff19e Merge pull request #36 from matrix-org/device_id_from_access_token
Extract the device id and token id from the access token when autheniticating users
2015-01-28 17:19:28 +00:00
Mark Haines 3cca61e006 Rename ClientID to ClientInfo since it is a pair of IDs rather than a single identifier 2015-01-28 17:16:12 +00:00
Mark Haines c18e551640 Add a : to the doc string after the type of the return value 2015-01-28 17:08:53 +00:00
Mark Haines 388581e087 Extract the id token of the token when authing users, include the token and device_id in the internal meta data for the event along with the transaction id when sending events 2015-01-28 16:58:23 +00:00
Kegan Dougal c23e3db544 Add filter JSON sanity checks. 2015-01-28 16:45:18 +00:00
Erik Johnston 0ef5bfd6a9 Start implementing auth conflict res 2015-01-28 16:16:53 +00:00
David Baker 6840e7cece Merge branch 'master' into develop 2015-01-28 16:03:35 +00:00
David Baker 60b143a52e Move pushers delta to v12 and bump schema version 2015-01-28 15:48:28 +00:00
Mark Haines c59bcabf0b Return the device_id from get_auth_by_req 2015-01-28 15:43:41 +00:00
David Baker e78dd33292 Use %s instead of + 2015-01-28 14:52:58 +00:00
David Baker 93aac9bb7b Newline 2015-01-28 14:51:01 +00:00
David Baker 445ad9941e Redundant parens 2015-01-28 14:49:59 +00:00
David Baker 6d485dd1c7 unnecessary newlines 2015-01-28 14:48:42 +00:00
David Baker fb0928097a More magic commas (including the place I copied it from...) 2015-01-28 14:48:07 +00:00
David Baker 0cbb6b0f52 Google doc style 2015-01-28 14:44:41 +00:00
David Baker 2cfdfee572 spaces 2015-01-28 14:41:51 +00:00
David Baker 289a249874 Unnecessary newlines. 2015-01-28 14:39:03 +00:00
David Baker 3cb5b73c0d Unnecessary newline. 2015-01-28 14:37:55 +00:00
David Baker 8807f4170e Better style 2015-01-28 14:35:00 +00:00
David Baker 032f8d4ed3 Another superfluous newline 2015-01-28 14:33:15 +00:00
David Baker d93ce29a86 Ah, the comma of doom. 2015-01-28 14:27:01 +00:00
David Baker 6741c3dbd9 Brackets are nicer 2015-01-28 14:26:03 +00:00
David Baker 4fbf2328c2 Unnecessary new line 2015-01-28 14:24:28 +00:00
David Baker 30fbba168b Easy on the newlines 2015-01-28 14:23:16 +00:00
David Baker dd3abbd61f 2015 2015-01-28 14:22:39 +00:00
David Baker 6fde707add doc style fix 2015-01-28 14:14:49 +00:00
David Baker 5f2665320f It is 2015 2015-01-28 14:11:45 +00:00
David Baker 20c47383dc Oops, bad merge: needed to change the base class of the rest servlets too. 2015-01-28 14:10:46 +00:00
David Baker 03149ad23a More code style things 2015-01-28 14:01:24 +00:00
David Baker e1ca0f1396 Brackets rather than slashes at end 2015-01-28 13:58:32 +00:00
David Baker 6df6f5e084 Redundant bracketing & missed space 2015-01-28 13:56:35 +00:00
David Baker ca7240a2f0 Update copyright 2015-01-28 13:17:55 +00:00
David Baker fb532d8425 Unused import 2015-01-28 13:06:09 +00:00
David Baker c291a4d522 Merge branch 'develop' into pushers
Conflicts:
	synapse/handlers/events.py
	synapse/server.py
2015-01-28 12:51:05 +00:00
Kegan Dougal 42876969b9 Add basic application_services SQL, and hook up parts of the appservice store to read from it. 2015-01-28 11:59:38 +00:00
David Baker 273b12729b Reset badge count to zero when last active time is bumped 2015-01-28 11:55:49 +00:00
David Baker e32ded7b3e Add matrix.org as a trusted ID server because it's now passed through on ports 80/443 and the web client defaults to that now. Fixes email validation (including signing up with an email address). 2015-01-28 10:09:54 +00:00
Kegan Dougal b46fa8603e Remove unused import 2015-01-28 09:17:48 +00:00
Mark Haines e020574d65 Fix Formatting 2015-01-27 20:19:36 +00:00
Mark Haines b19cf6a105 Wait for events if the incremental sync is empty and a timeout is given 2015-01-27 20:09:52 +00:00
Paul "LeoNerd" Evans 8398f19bce Created schema delta 2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans 06cc147012 Initial stab at real SQL storage implementation of user filter definitions 2015-01-27 18:46:03 +00:00
Paul "LeoNerd" Evans 54e513b4e6 Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests 2015-01-27 17:48:13 +00:00
Kegan Dougal fbeaeb8689 Log when ASes are registered/unregistered. 2015-01-27 17:34:40 +00:00
Kegan Dougal ec3719b583 Use ApplicationService when registering. 2015-01-27 17:15:06 +00:00
Kegan Dougal 92171f9dd1 Add stub methods, TODOs and docstrings for application services. 2015-01-27 16:53:59 +00:00
Mark Haines a56008842b Start implementing incremental initial sync 2015-01-27 16:24:22 +00:00
Kegan Dougal 7331d34839 Add AS specific classes with docstrings. 2015-01-27 16:23:46 +00:00
Paul "LeoNerd" Evans 059651efa1 Have the Filtering API return Deferreds, so we can do the Datastore implementation nicely 2015-01-27 16:17:56 +00:00
David Baker f7c4daa8f9 Oops, remove debugging 2015-01-27 16:08:47 +00:00
David Baker 5eacaeb4a7 or of course we could just return the deferred 2015-01-27 16:05:23 +00:00
David Baker eba89f093f Need a defer.inlineCallbacks here as we yield in it: otherwise nothing in the cb gets executed. 2015-01-27 16:00:07 +00:00
David Baker 1d77969124 Unbreak bad presence merge - don't add these blocks together with an and: they're different things. 2015-01-27 15:58:27 +00:00
Kegan Dougal 51449e0665 Add appservice handler and store. Glue together rest > handler > store. 2015-01-27 15:50:28 +00:00
Kegan Dougal 6efdc11cc8 Parse /register and /unregister request JSON. 2015-01-27 15:03:19 +00:00
Paul "LeoNerd" Evans 05c7cba73a Initial trivial implementation of an actual 'Filtering' object; move storage of user filters into there 2015-01-27 14:28:56 +00:00
Kegan Dougal fa8e6ff900 Add stub application services REST API. 2015-01-27 14:01:51 +00:00
Paul "LeoNerd" Evans 0484d7f6e9 Merge branch 'develop' into client_v2_filter 2015-01-27 13:11:03 +00:00
Mark Haines 436513068d Start implementing the non-incremental sync portion of the v2 /sync API 2015-01-26 18:53:31 +00:00
David Baker b481889117 Support membership events and more camelcase/underscores 2015-01-26 17:27:28 +00:00
David Baker 69a75b7ebe Add brackets to make get room name / alias work 2015-01-26 16:52:47 +00:00
Mark Haines 3186c5bdbc Merge branch 'develop' into client_v2_sync 2015-01-26 16:32:40 +00:00
Mark Haines e5725eb3b9 Remove unused import from server.py 2015-01-26 16:16:50 +00:00
Mark Haines 7f6f3f9d62 Pass the current time to serialize event, rather than passing an
HS and getting a clock from it and calling time_msec on the clock.
Remove the serialize_event method from the HS since it is no longer
needed.
2015-01-26 16:11:28 +00:00
Mark Haines 0cfb4591a7 Add handler for /sync API 2015-01-26 15:46:31 +00:00
David Baker efac71d6ca Pushers should only try & look for rejected devices in something that's a list or tuple. 2015-01-26 14:37:14 +00:00
Paul "LeoNerd" Evans 8d7accb28f Initial minimal attempt at /user/:user_id/filter API - in-memory storage, no actual filter implementation 2015-01-26 14:33:30 +00:00
Erik Johnston c92d64a6c3 Make it the responsibility of the replication layer to check signature and hashes. 2015-01-26 14:33:11 +00:00
Mark Haines 14ff33bd93 Merge branch 'develop' into client_v2_sync
Conflicts:
	synapse/rest/client/v2_alpha/__init__.py
2015-01-26 13:14:59 +00:00
Erik Johnston 7b88619241 Split up replication_layer module into client, server and transaction queue 2015-01-26 10:45:24 +00:00
Mark Haines 7b814d3f7f Add client v2_alpha resource to synapse server resource tree 2015-01-23 18:55:19 +00:00
Mark Haines 2b1799883d Add client v2_alpha resource to synapse server resource tree 2015-01-23 18:49:05 +00:00
Mark Haines e26340cee7 Start implementing the v2_alpha sync API 2015-01-23 18:48:17 +00:00
Paul "LeoNerd" Evans 85419e1257 Stop complaining about Synapse Angular SDK 0.6.1 2015-01-23 18:37:37 +00:00
David Baker 5f84ba8ea1 Add API to delete push rules. 2015-01-23 17:49:37 +00:00
David Baker f21f9fa3c5 Use push settings! 2015-01-23 17:07:06 +00:00
Erik Johnston 9b1e552b51 Merge branch 'develop' of github.com:matrix-org/synapse into rejections 2015-01-23 15:51:48 +00:00
Erik Johnston 3b9cc882a5 Add storage method have_events 2015-01-23 15:42:52 +00:00
Mark Haines e0bf18addf Add RestServlet base class in synapse/http/servlet.py 2015-01-23 14:16:28 +00:00
Mark Haines 4be637cb12 Extract the client v1 base RestServlet to a separate class 2015-01-23 14:09:51 +00:00
David Baker fc7a05c443 more pep8 suggestions 2015-01-23 13:36:01 +00:00
David Baker b3f66ea6fb more pep8 2015-01-23 13:28:00 +00:00
David Baker d3e72b4d87 Make string format tuple an actual tuple 2015-01-23 13:25:58 +00:00
David Baker 98e1080555 redundant parens 2015-01-23 13:25:36 +00:00
David Baker 54c689c819 stray space 2015-01-23 13:25:14 +00:00
Mark Haines c4652d7772 Remove hs.parse_eventid 2015-01-23 13:25:07 +00:00
David Baker 6188c4f69c make per-device rules work 2015-01-23 13:23:10 +00:00
Mark Haines ada711504e Replace hs.parse_roomalias with RoomAlias.from_string 2015-01-23 13:21:58 +00:00
Mark Haines 1c06c48ce2 Replace hs.parse_roomid with RoomID.from_string 2015-01-23 11:55:12 +00:00
Mark Haines 5759bec43c Replace hs.parse_userid with UserID.from_string 2015-01-23 11:47:15 +00:00
David Baker 49fe31792b Add slightly pedantic trailing slash error. 2015-01-23 11:19:02 +00:00
Mark Haines 7256def8e4 Merge rest servlets into the client json resource object 2015-01-23 10:37:38 +00:00
David Baker f87586e661 right super() param 2015-01-23 10:32:40 +00:00
David Baker bcd48b9636 Fix adding rules without before/after & add the rule that we couldn't find to the error 2015-01-23 10:28:25 +00:00
David Baker 6927b6b197 This really serves me right for ever making a map called 'map'. 2015-01-23 10:21:47 +00:00
David Baker 8a850573c9 As yet fairly untested GET API for push rules 2015-01-22 19:32:17 +00:00
David Baker 673773b217 oops, this is not its own schema file 2015-01-22 18:27:07 +00:00
David Baker 7ecb49ef25 Insufficient newlines 2015-01-22 17:53:30 +00:00
David Baker 5c6189ea3e Merge branch 'develop' into pushers
Conflicts:
	synapse/rest/__init__.py
2015-01-22 17:46:16 +00:00
David Baker ede491b4e0 Oops: second part of commit dc938606 2015-01-22 17:38:53 +00:00
David Baker dc93860619 Add rest API & store for creating push rules
Also make unrecognised request error look more like synapse errors
because it makes it easier to throw them from within rest classes.
2015-01-22 17:37:12 +00:00
Erik Johnston ca65a9d03e Split out TransactionQueue from replication layer 2015-01-22 16:37:08 +00:00
Mark Haines 97c68c508d Move rest APIs back under the rest directory 2015-01-22 16:10:07 +00:00
Erik Johnston c2f9768740 Merge branch 'new_state_resolution' of github.com:matrix-org/synapse into rejections 2015-01-22 15:57:26 +00:00
Erik Johnston 73dd81ca62 fix pyflakes 2015-01-22 15:57:08 +00:00
Erik Johnston b1b85753d7 Add support for storing rejected events in EventContext and data stores 2015-01-22 15:50:17 +00:00
Mark Haines 1d2016b4a8 Move client v1 api rest servlets into a "client/v1" directory 2015-01-22 14:59:08 +00:00
Erik Johnston 7f058c5ff7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf
Conflicts:
	synapse/app/homeserver.py
2015-01-22 13:35:34 +00:00
Erik Johnston b390bf39f2 Remove unused function. Add comment. 2015-01-21 16:44:04 +00:00
Erik Johnston 6dcade97be Implement new state resolution algorithm 2015-01-21 16:27:04 +00:00
David Baker 5d5932d493 use underscores everywhere, not camelcase. 2015-01-20 11:52:08 +00:00
David Baker afb714f7be add instance_handles to pushers so we have a way to refer to them even if the push token changes. 2015-01-20 11:49:48 +00:00
Mark Haines dc70d1fef8 Only start the notifier timeout once we've had a chance to check for updates. Otherwise the timeout could fire while we are waiting for the database to return any updates it might have 2015-01-19 16:24:54 +00:00
Mark Haines 42529cbced Fix pyflakes errors 2015-01-19 15:33:04 +00:00
Mark Haines 00e9c08609 Fix syntax 2015-01-19 15:30:48 +00:00
Mark Haines 3e85e52b3f Allow ':memory:' as the database path for sqlite3 2015-01-19 15:26:19 +00:00
Mark Haines 5fed042640 Finish renaming "context" to "room_id" in federation codebase 2015-01-16 19:01:03 +00:00
Mark Haines 2408c4b0a4 Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json 2015-01-16 19:01:03 +00:00
Mark Haines 602684eac5 Split transport layer into client and server parts 2015-01-16 19:01:03 +00:00
Mark Haines 2bdee98269 Remove temporary debug logging that was accidentally committed 2015-01-16 19:00:40 +00:00
David Baker 2d2953cf5f Require device language when adding a pusher.
Because this seems like it might be useful to do sooner rather
than later.
2015-01-16 11:24:10 +00:00
David Baker 2ca2dbc821 Send room name and first alias in notification poke. 2015-01-15 16:56:18 +00:00
David Baker e3e2fc3255 Don't make the pushers' event streams cause people to appear online 2015-01-15 16:17:21 +00:00
David Baker 2cb30767fa Honour the 'rejected' return from push gateways
Add a timestamp to push tokens so we know the last time they we
got them from the device. Send it to the push gateways so it can
determine whether its failure is more recent than the token.
Stop and remove pushers that have been rejected.
2015-01-13 19:48:37 +00:00
Paul "LeoNerd" Evans 34a5fbe2b7 Have /join/:room_id return the room ID in response anyway, for consistency of clients (SYN-234) 2015-01-13 17:29:24 +00:00
Paul "LeoNerd" Evans c2e7c84e58 Don't try to cancel already-expired timers - SYN-230 2015-01-13 16:58:36 +00:00
Mark Haines 3891597eb3 Remove unused functions 2015-01-13 15:57:26 +00:00
Mark Haines fda63064fc get_room_events isn't called anywhere 2015-01-13 14:43:26 +00:00
Mark Haines 895fcb377e Fix stream token ordering 2015-01-13 14:38:53 +00:00
David Baker c06a9063e1 Merge branch 'develop' into pushers 2015-01-13 13:15:51 +00:00
David Baker 70d0a453f3 Split out function to decide whether to notify or a given event 2015-01-13 13:14:41 +00:00
Erik Johnston 38e3241eb7 Merge branch 'hotfixes-v0.6.1b' of github.com:matrix-org/synapse into develop 2015-01-13 10:01:22 +00:00
Erik Johnston 1d3d37937d Bump version 2015-01-13 09:59:47 +00:00
Erik Johnston 39585bf556 Insert 'age' into top level when returning events to clients 2015-01-13 09:57:32 +00:00
Paul "LeoNerd" Evans 02ffbb20d0 Use float rather than integer divisions to turn msec into sec - so timeouts under 1000msec will actually work 2015-01-12 19:09:14 +00:00
Paul "LeoNerd" Evans 67d8305aea Make typing notification timeouts print a (debug) logging message 2015-01-12 18:22:00 +00:00
Kegan Dougal c43d898119 SYN-178: Fix off by one. 2015-01-12 17:38:40 +00:00
Mark Haines 63403aa7a5 Check the existance and versions of necessary modules when starting synapse, log which modules are used 2015-01-08 17:08:57 +00:00
Kegan Dougal 9d0dcf2e3c SYN-142: Rotate logs if logging to file. Fixed to a 4 file rotate with 100MB/file for now. 2015-01-08 15:31:29 +00:00
Matthew Hodgson 7f83613733 make our JPEG thumbnail quality less horrifically ugly 2015-01-08 15:11:22 +00:00
Kegan Dougal b5924cae04 Add raw query param for scrollback. 2015-01-08 14:37:55 +00:00
Erik Johnston 379a653ae3 Add better help message for --server-name config option. 2015-01-08 14:32:53 +00:00
Kegan Dougal edb557b2ad Return the raw federation event rather than adding extra keys for federation data. 2015-01-08 14:28:08 +00:00
Erik Johnston 5940ec993b Add missing continuation indent. 2015-01-08 13:59:29 +00:00
Kegan Dougal 5720ab59e0 Add 'raw' query parameter to expose the event graph and signatures to savvy clients. 2015-01-08 13:57:40 +00:00
Mark Haines 8ac9199f56 Merge branch 'master' into develop 2015-01-08 09:43:48 +00:00
Mark Haines 5a0e687d5c Bump version 2015-01-08 09:42:23 +00:00
Mark Haines c9d2cecac9 SYN-231: User agent header broken 2015-01-08 09:41:11 +00:00
Erik Johnston 42507b0011 Log server version on startup 2015-01-07 17:25:28 +00:00
Kegan Dougal 76e1565200 Change error message for missing pillow libs. 2015-01-07 17:11:19 +00:00
Kegan Dougal 333836ff92 PEP8 and pyflakes warnings 2015-01-07 16:18:12 +00:00
Kegan Dougal 4c68460392 SYN-154: Tweak how the m.room.create check is done.
Don't perform the check in auth.is_host_in_room but instead do it in _do_join
and also assert that there are no m.room.members in the room before doing so.
2015-01-07 16:09:00 +00:00
Kegan Dougal 9cb4f75d53 SYN-154: Better error messages when joining an unknown room by ID.
The simple fix doesn't work here because room creation also involves
unknown room IDs. The check relies on the presence of m.room.create for
rooms being created, whereas bogus room IDs have no state events at all.
2015-01-07 15:21:48 +00:00
Matthew Hodgson 9b8e348b15 *cough* 2015-01-07 15:08:32 +00:00
Erik Johnston 89fc09c3d1 Bump version and changelog 2015-01-07 13:56:56 +00:00
Erik Johnston 82be4457de Add twisted Service interface 2015-01-07 13:46:37 +00:00
Erik Johnston a039e2544c Remove unused import 2015-01-07 09:48:03 +00:00
Erik Johnston 1497e50649 Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf 2015-01-07 09:40:42 +00:00
Mark Haines 5cf45c4319 Merge branch 'master' into develop 2015-01-06 19:48:53 +00:00
Erik Johnston dfa05f0cd6 Optimize FrozenEvent creation 2015-01-06 18:51:03 +00:00
Erik Johnston 36a2a877e2 Use time.time() instead of time.clock() 2015-01-06 16:34:41 +00:00
Erik Johnston d5ae67e67d Fix typo where we used wrong var. 2015-01-06 16:05:01 +00:00
Erik Johnston fd9a8db7ea Only fetch the columns we need. 2015-01-06 15:59:31 +00:00
Erik Johnston 9e5545a6fa RoomsForUser now has sender instead of user_id 2015-01-06 15:53:50 +00:00
Erik Johnston a01416cf21 Add delta and bump DB version 2015-01-06 15:42:18 +00:00
Erik Johnston f6da237c35 Add index on transaction_id to sent_transcations 2015-01-06 15:40:38 +00:00
Erik Johnston 9bd07bed23 Actually time that function 2015-01-06 15:28:56 +00:00
Erik Johnston 03a501456c Time how long calls to _get_destination_retry_timings take 2015-01-06 15:22:28 +00:00
Erik Johnston 52b2c6c9c7 Don't include None's in _get_events_txn 2015-01-06 14:56:57 +00:00
Erik Johnston 8a12df8cf3 Merge branch 'erikj-perf' of github.com:matrix-org/synapse into develop 2015-01-06 14:45:57 +00:00
Erik Johnston 96707ed718 Name 'user_rooms_intersect' transaction 2015-01-06 14:44:27 +00:00
Erik Johnston 76ec154e95 We don't need the full events for get_rooms_for_user_where_membership_is 2015-01-06 14:37:00 +00:00
Mark Haines bc2ec808f4 SYN-32 Use the ANTIALIAS resize method for thumbnailing images 2015-01-06 14:14:17 +00:00
Matrix 0529a7e2e9 Add some logging for when we are sending transactions. 2015-01-06 14:06:25 +00:00
Mark Haines b9f77d1ae1 Increase default maximum attachment size to 10M 2015-01-06 14:04:58 +00:00
Mark Haines 5e23a19204 Merge pull request #28 from matrix-org/erikj-perf
Database performance improvements.
2015-01-06 13:33:40 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston af1c7c7808 PEP8 2015-01-06 13:13:17 +00:00
Erik Johnston 12819d5082 Remove debug lines 2015-01-06 13:12:30 +00:00
Erik Johnston 52d8519008 Don't do batching when getting events. 2015-01-06 13:10:27 +00:00
Mark Haines 773de09774 Set a content-length for JSON responses 2015-01-06 13:05:19 +00:00
Erik Johnston 98933e3db6 Only fetch prev_content when a client is streaming/paginating. Use transactions for event streams. 2015-01-06 13:03:23 +00:00
Kegan Dougal 78edb47cc5 SYN-208/SYN-228: Add runtime checks on startup to enforce that JPEG/PNG support is included when installing pillow. 2015-01-06 11:43:04 +00:00
Mark Haines 3c8c3bf3b7 SYN-229: Include Content-Length when downloading files 2015-01-06 11:32:36 +00:00
Erik Johnston 3e26720e05 Temporarily turn off 'redacted_because' and 'prev_content' keys 2015-01-06 11:26:58 +00:00
Erik Johnston f4ea78e9e2 More debug logging 2015-01-06 11:24:18 +00:00
Erik Johnston 753126b8cc Add some debug logging 2015-01-06 11:18:12 +00:00
Erik Johnston d7e8ea67b3 Reformat 2015-01-06 11:18:02 +00:00
Erik Johnston f0128f9600 Add RoomMemberStore.get_users_in_room, so that we can get the list of joined users without having to retrieve the full events 2015-01-06 10:55:43 +00:00
Erik Johnston 96a5ba41f5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf 2015-01-06 10:53:04 +00:00
Mark Haines 90d60e3fe4 Merge branch 'hotfixes-v0.6.0a' 2014-12-29 14:01:07 +00:00
Mark Haines af61c29527 Return the argument passed to the callback in a deferred callback, otherwise twisted will replace the deferred result with 'None' 2014-12-29 13:54:05 +00:00
Matthew Hodgson 407c299828 improve error msg 2014-12-24 17:50:42 +00:00
Mark Haines 24b5d01853 Include version in User-Agent and Server headers 2014-12-22 10:16:02 +00:00
Erik Johnston 74ee4048c2 Merge branch 'master' of github.com:matrix-org/synapse into erikj-perf 2014-12-21 11:47:45 +00:00
Kegan Dougal 67a406a754 Rate limit display names and avatar urls per request rather than per event. 2014-12-19 17:36:33 +00:00
Erik Johnston d61109f578 Merge branch 'hotfixes-v0.6.0' of github.com:matrix-org/synapse into erikj-perf 2014-12-19 16:37:08 +00:00
Mark Haines efd27ff01b Set a state_key for the topic and room name, otherwise they won't be treated as room state 2014-12-19 15:31:27 +00:00
Mark Haines 9c71d945d6 Look for name, topic in the event content rather than the event itself when persisting room name and topic events 2014-12-19 15:16:48 +00:00
Mark Haines f70e622d59 bump_presence_active_time when sending a message event 2014-12-19 14:30:57 +00:00
Mark Haines a999f0dec3 Don't ratelimit room create events 2014-12-19 14:18:27 +00:00
Mark Haines 1e7f83b91d Set display name when joining via alias 2014-12-19 12:31:46 +00:00
Mark Haines 5dbe820e9a Remove unneeded federation keys from events 2014-12-19 12:16:26 +00:00
Mark Haines 390e48a8b0 SYN-203: Handle requests for thunbnails for images that are small 2014-12-19 12:05:38 +00:00
Mark Haines 5739e6c606 s/user_id/sender/ 2014-12-19 11:43:46 +00:00
Erik Johnston 41ce544abe Merge branch 'release-v0.6.0' of github.com:matrix-org/synapse into erikj-perf 2014-12-18 18:57:21 +00:00
Mark Haines 041ac476a5 Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them 2014-12-18 18:47:13 +00:00
David Baker fead431c18 If we didn't get any events, advance the token or we'll just keep not getting the same events again. 2014-12-18 18:44:33 +00:00
Mark Haines dbe77ec79a Replace distributor deferred list, with a simple for loop until I understand why the former breaks and the latter doesn't 2014-12-18 17:47:00 +00:00
David Baker b56730bb6e Merge branch 'develop' into pushers
Conflicts:
	synapse/api/errors.py
	synapse/server.py
	synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
David Baker afa953a293 schema version is now 10 2014-12-18 15:11:06 +00:00
David Baker 0a6664493a Merge branch 'master' into pushers 2014-12-18 15:06:11 +00:00
David Baker 4c7ad50f6e Thank you, pyflakes 2014-12-18 14:55:04 +00:00
David Baker 173264b656 ...and bump SCHEMA_VERSION 2014-12-18 14:53:10 +00:00
David Baker fc7c5e9cd7 Rename the pusher SQL delta to v9 which the next free one 2014-12-18 14:51:29 +00:00
David Baker 9728c305a3 after a few rethinks, a working implementation of pushers. 2014-12-18 14:49:22 +00:00
Mark Haines dea5d4b03b Don't yield on sending the event accross federation. 2014-12-18 11:29:46 +00:00
Erik Johnston f3788e3c78 Test some ideas that might help performance a bit 2014-12-17 23:37:08 +00:00
Erik Johnston dec5b62339 Use _get_events_txn instead of _parse_events_txn 2014-12-16 19:16:41 +00:00
Erik Johnston 21cab3a7ec Fix where we pulled in event.state_events from hotfixes branch 2014-12-16 19:16:15 +00:00
Erik Johnston 2215faa361 Merge branch 'hotfixes-v0.5.4a' of github.com:matrix-org/synapse into release-v0.6.0 2014-12-16 19:11:13 +00:00
Erik Johnston 3defd5b3ee Add FIXME 2014-12-16 19:07:20 +00:00
Erik Johnston 96779d2490 Fix bug where we did not send the full auth chain to people that joined over federation 2014-12-16 18:57:36 +00:00
Erik Johnston 2d7716d4d0 Make error messages slightly more helpful 2014-12-16 18:41:48 +00:00
Erik Johnston f76269392b Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.6.0
Conflicts:
	synapse/state.py
2014-12-16 18:35:46 +00:00
Erik Johnston 52f99243ab Use is_outlier() so that we don't get AttributeError 2014-12-16 18:33:50 +00:00
Erik Johnston 5b39cfff69 Don't assume an event exists 2014-12-16 18:25:24 +00:00
Mark Haines 56db465047 Merge branch 'release-v0.6.0' into develop 2014-12-16 17:29:49 +00:00
Kegan Dougal 4dcad143dd SYN-142: Use a default log file 'homeserver.log' so people get logging by default. 2014-12-16 17:24:49 +00:00
Erik Johnston f06161a307 Enable rate limiting for all events 2014-12-16 16:10:17 +00:00
Mark Haines 627e4f01d2 Remove send_message since nothing was calling it. Remove Snapshot because only send_message was using it 2014-12-16 16:07:41 +00:00
Erik Johnston 23da4a4051 Fix typo where we thought a list was a dict 2014-12-16 15:59:40 +00:00
Mark Haines c3eae8a88c Construct the EventContext in the state handler rather than constructing one and then immediately calling state_handler.annotate_context_with_state 2014-12-16 15:59:17 +00:00
Mark Haines 3c7857e49b clean up coding style a bit 2014-12-16 15:24:03 +00:00
Erik Johnston 42b725ce52 Fix upgrade script to run all the missing deltas. 2014-12-16 15:13:34 +00:00
Mark Haines 8b8beba194 Remove annotate_event_with_state as nothing was using it. Update state tests to call annotate_context_with_state 2014-12-16 15:08:37 +00:00
Erik Johnston b3c793e362 Do run all deltas up to missing delta 10 2014-12-16 14:44:53 +00:00
Erik Johnston 2e44714214 Make failure to run appropraite upgrade scripts more helpful. 2014-12-16 14:20:32 +00:00
Erik Johnston ef5a141050 Bump database version 2014-12-16 13:57:47 +00:00
Erik Johnston 96cc7c8740 Bump version 2014-12-16 13:57:27 +00:00
Erik Johnston 5a465b67ba Fix pyflakes 2014-12-16 13:41:43 +00:00
Erik Johnston 58168498b0 Remove FrozenEncoder 2014-12-16 13:38:38 +00:00
Erik Johnston 8133cdcc88 Better english in docstrings are helpful. 2014-12-16 13:32:06 +00:00
Erik Johnston 35f4f6b070 Update upgrade script 2014-12-16 13:27:53 +00:00
Erik Johnston 882dc8dcab Persist internal_metadata 2014-12-16 13:17:09 +00:00
Erik Johnston 4afac88390 Add basic docstring to annotate_context_with_state 2014-12-16 13:09:44 +00:00
Erik Johnston 3c77d13aa5 Kill off synapse.api.events.* 2014-12-16 11:29:05 +00:00
Erik Johnston 6a1da99fab Add fixme to raising of AuthError in federation land 2014-12-16 09:35:31 +00:00
Erik Johnston 65b2e49429 Fix pyflakes 2014-12-15 17:35:37 +00:00
Erik Johnston 9c49054f1d Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-15 17:33:23 +00:00
Erik Johnston f280929a12 Use frozenutils 2014-12-15 17:31:36 +00:00
Erik Johnston 009e4b5637 User.is_mine is no longer a thing. Use hs.is_mine instead. 2014-12-15 17:17:51 +00:00
Erik Johnston cf6e5f1dbf Rename MessageHandler.handle_event. Add a few comments. 2014-12-15 17:01:12 +00:00
Kegsay 67c9585656 Update media_repository.py
_ not -
2014-12-15 16:57:53 +00:00
Erik Johnston 670dcdfc14 Remove unused functions 2014-12-15 16:16:58 +00:00
Paul "LeoNerd" Evans 0c1deca574 Remember to hook up the typing event stream to the notifier as well 2014-12-15 16:14:53 +00:00
Erik Johnston b75adaedca Finish up upgrade script 2014-12-15 16:14:34 +00:00
Erik Johnston 65cdf4e724 Get current member state from current_state snapshot. Fix leave test. 2014-12-15 15:03:27 +00:00
Erik Johnston 57e0e619f3 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
Conflicts:
	tests/handlers/test_room.py
2014-12-15 14:45:59 +00:00
Paul "LeoNerd" Evans 20beed9dd4 Still send typing notifications to myself if I'm the only one in the room (it's a lonely life...) 2014-12-15 14:37:12 +00:00
Mark Haines 3610641a62 Update docs in media_repository 2014-12-15 13:56:43 +00:00
Erik Johnston c8dd3314d6 Fix bug where we ignored event_edge_hashes table 2014-12-15 13:55:22 +00:00
Mark Haines 58fa6d3fc6 return an mxc uri rather than a content_token. 2014-12-15 13:54:10 +00:00
Paul "LeoNerd" Evans 0aa8c08478 Merge branch 'develop' into typing_notifications 2014-12-15 11:19:30 +00:00
Erik Johnston 3983c7fb0f Merge branch 'hotfixes-v0.5.4' of github.com:matrix-org/synapse into develop 2014-12-13 18:16:12 +00:00
Erik Johnston eea58b8076 Bump version and change log 2014-12-13 18:04:37 +00:00
Erik Johnston 6380ead2ee Fix bug while generating the error message when a file path specified in the config doesn't exist 2014-12-13 18:03:01 +00:00
Erik Johnston 23c7cb6220 Remove unused imports 2014-12-12 16:31:59 +00:00
Erik Johnston fc409096ac Make auth module use EventTypes constants 2014-12-12 16:31:50 +00:00
Erik Johnston 1fc2a0e33e Fix tests and remove debug logging 2014-12-12 15:08:29 +00:00
Erik Johnston 7b43a503f3 Consistently url decode and decode as utf 8 the URL parts 2014-12-12 15:05:37 +00:00
Erik Johnston c39beb5559 Store json as UTF-8 and not bytes 2014-12-12 14:53:37 +00:00
Erik Johnston 75085bb4d1 Pyflakes 2014-12-12 14:34:34 +00:00
Paul "LeoNerd" Evans b0bb1756a9 Send list of typing user IDs as 'user_ids' list within 'content', so that m.typing stream events have a toplevel content, for consistency with others 2014-12-12 11:59:46 +00:00
Erik Johnston 63810c777d Validate message, topic and name event contents 2014-12-12 11:01:09 +00:00
Erik Johnston fa4b610ae3 Fix stream test. Make sure we add join to auth_events for invitiations 2014-12-12 10:42:27 +00:00
Paul "LeoNerd" Evans 0b70023373 Merge branch 'develop' into typing_notifications 2014-12-11 18:35:05 +00:00
Paul "LeoNerd" Evans 57b5094545 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2014-12-11 18:34:26 +00:00
Paul "LeoNerd" Evans cfb963af03 When users leave rooms mark them as no longer typing in them 2014-12-11 18:33:09 +00:00
Paul "LeoNerd" Evans f25764943c Add a 'user_left_room' distributor signal analogous to 'user_joined_room' 2014-12-11 18:27:01 +00:00
Mark Haines b3e34a5399 Fix typo in media repository doc string 2014-12-11 18:21:08 +00:00
Mark Haines 64bf9f54cc Fix media repository doc string to include server_name 2014-12-11 18:18:58 +00:00
Paul "LeoNerd" Evans 5ebc994f84 Actually auth-check to ensure people can only send typing notifications for rooms they're actually in 2014-12-11 18:11:43 +00:00
Paul "LeoNerd" Evans 966c4b2b04 Add a sprinkling of logger.debug() into typing notification handler 2014-12-11 18:00:15 +00:00
Erik Johnston cde840a82c Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
Conflicts:
	setup.py
2014-12-11 17:48:48 +00:00
Erik Johnston d3eb12c7b8 Fix federation test 2014-12-11 17:01:27 +00:00
Mark Haines 03d9024cbc Allow only one download for a given image at a time, so that we don't end up downloading the same image twice if two clients request a remote image at the same time 2014-12-11 16:48:11 +00:00
Paul "LeoNerd" Evans 3b2cc26053 Initial hack at unit tests of room typing REST API 2014-12-11 16:03:12 +00:00
Erik Johnston 0b04369238 Fix public room joining by making sure replaces_state never points to itself. 2014-12-11 15:56:06 +00:00
Erik Johnston 9191292b0f Fix prev_content 2014-12-11 15:16:55 +00:00
Mark Haines d80d505b1f Limit the size of images that are thumbnailed serverside. Limit the size of file that a server will download from a remote server 2014-12-11 14:19:32 +00:00
Erik Johnston 8cdebce470 Fix redactions. Fix 'age' key 2014-12-11 13:25:19 +00:00
Paul "LeoNerd" Evans 0ca072b3b6 Initial tiny hack at REST API for setting room typing notification status 2014-12-11 10:55:36 +00:00
Mark Haines ead8fc5e38 doc the thumbnail methods 2014-12-11 10:41:43 +00:00
Mark Haines b5eb9124f7 Make sure we pass a tuple to string '%' formatting 2014-12-11 10:08:09 +00:00
Paul "LeoNerd" Evans 5f49914dee Avoid cyclic dependency in handler setup 2014-12-10 21:17:48 +00:00
Paul "LeoNerd" Evans 1a75ff5c23 Hook up the event stream to typing notifications 2014-12-10 21:01:49 +00:00
Paul "LeoNerd" Evans 4006d58335 Store serial numbers per room for typing event stream purposes 2014-12-10 20:48:25 +00:00
Paul "LeoNerd" Evans 9eb819e828 First hack at implementing timeouts in typing notification handler 2014-12-10 19:39:01 +00:00
Erik Johnston 4c682143c8 .from_string() no longer takes a HS 2014-12-10 18:00:49 +00:00
Erik Johnston 02e4c18171 Remove dead code 2014-12-10 18:00:36 +00:00
Erik Johnston b245ee34ed Add some basic event validation 2014-12-10 17:59:47 +00:00
Mark Haines 4f37c0ea9d Merge branch 'develop' into media_repository 2014-12-10 16:55:06 +00:00
Mark Haines 61fc37e467 Merge branch 'develop' into media_repository 2014-12-10 16:14:17 +00:00
Mark Haines 2d265ef3bd import Image as PIL.Image. 2014-12-10 16:09:18 +00:00
Erik Johnston 1d2a0040cf Fix bug where we clobbered old state group values 2014-12-10 15:55:03 +00:00
Mark Haines e5275d856e Get the code actually working 2014-12-10 15:46:18 +00:00
Mark Haines cc84d3ea78 Thumbnail uploaded and cached images 2014-12-10 15:40:52 +00:00
Erik Johnston cabead6194 Actually fix bug when uploading state with empty state_key 2014-12-10 14:49:52 +00:00
Erik Johnston 02db7eb209 Fix bug when uploading state with empty state_key 2014-12-10 14:02:48 +00:00
Erik Johnston aae8a37e63 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-10 13:18:40 +00:00
Erik Johnston 02db1fd2e7 Fix AttributeError 2014-12-10 12:00:05 +00:00
Erik Johnston 018443cb59 Make depth increase. 2014-12-10 11:59:53 +00:00
Erik Johnston 102d2373b4 Add __str__ to FrozenEvent 2014-12-10 11:38:08 +00:00
Erik Johnston 95aa903ffa Try and figure out how and why signatures are being changed. 2014-12-10 11:37:47 +00:00
Matthew Hodgson 0f4dcab238 turn back on per-request transaction retries, so that every time we try to hit a dead server we actually end up hammering 5 times :| 2014-12-10 10:28:27 +00:00
Erik Johnston b8d30899b1 Code style. 2014-12-10 10:16:09 +00:00
Matthew Hodgson 71da2bed55 plateau retries after 1h 2014-12-10 00:18:44 +00:00
Matthew Hodgson faf12b64f8 add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception 2014-12-10 00:12:51 +00:00
Matthew Hodgson 2b1acb7671 squidge to 79 columns as per pep8 2014-12-10 00:03:55 +00:00
Erik Johnston 008303b245 PEP8 2014-12-09 14:49:11 +00:00
Erik Johnston 5eca288d28 Fix joining from an invite 2014-12-09 14:47:27 +00:00
Erik Johnston aa3f66cf7f Change the way we implement get_events to be less sucky 2014-12-09 13:35:26 +00:00
Erik Johnston bc6564bac0 Add PEP8 newlines 2014-12-09 11:01:44 +00:00
Erik Johnston 8c48450682 Add PEP8 newlines 2014-12-09 10:58:31 +00:00
Erik Johnston 1c8ee06877 Remove unused snapshot 2014-12-09 10:53:34 +00:00
Erik Johnston 4e57943cc5 Remove unused import 2014-12-09 10:51:36 +00:00
Matthew Hodgson 8529fba02d fix a million stupid bugs and make it actually work 2014-12-08 19:34:51 +00:00
Erik Johnston 609c31e8df More bug fixes 2014-12-08 17:50:56 +00:00
Matthew Hodgson 0d3fa1ac6e add a write-through cache on the retry schedule 2014-12-08 17:48:57 +00:00
Erik Johnston ee3df06183 More bug fixes 2014-12-08 14:50:48 +00:00
Erik Johnston ba3d1e2fc0 Remove unused import 2014-12-08 12:01:25 +00:00
Erik Johnston e8323b9e34 More bug fixes 2014-12-08 10:16:18 +00:00
Erik Johnston a295a3c691 Fix registration 2014-12-08 09:24:37 +00:00
Erik Johnston 721482c83e Add forgotten file 2014-12-08 09:10:12 +00:00
Erik Johnston d044121168 Various typos and bug fixes. 2014-12-08 09:08:26 +00:00
Matthew Hodgson 9c43b258ec actually reset retry schedule if we can successfuly talk to it 2014-12-08 00:17:12 +00:00
Matthew Hodgson 5cd43d4b9f fix stupid syntax thinkos 2014-12-07 23:44:16 +00:00
Matthew Hodgson aed62a3583 track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover. 2014-12-07 02:26:07 +00:00
Mark Haines a953be097f Add a method field to thumbnail storage 2014-12-05 16:31:56 +00:00
Erik Johnston 6630e1b579 Start making more things use EventContext rather than event.* 2014-12-05 16:20:48 +00:00
Mark Haines 0363820122 Add a class for generating thumbnails using PIL 2014-12-05 16:12:37 +00:00
Erik Johnston c5c32266d8 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-04 15:58:24 +00:00
Erik Johnston c31dba86ec Convert rest and handlers to use new event structure 2014-12-04 15:50:01 +00:00
Mark Haines c01fd5573c Implement download support for media_repository 2014-12-04 14:22:31 +00:00
Erik Johnston 5d7c9ab789 Begin converting things to use the new Event structure 2014-12-04 11:27:59 +00:00
Paul "LeoNerd" Evans f5d2514fc0 @log_function on PresenceStream's get_new_events_for_user() 2014-12-03 19:48:14 +00:00
Paul "LeoNerd" Evans 52f1d3c886 Store any incoming presence push in the local cache anyway, even if there's no interested observers (yet *hint*) (SYN-115) 2014-12-03 19:06:24 +00:00
Erik Johnston 797193283e Bump changes and version 2014-12-03 18:01:47 +00:00
Erik Johnston 75b4329aaa WIP for new way of managing events. 2014-12-03 16:07:21 +00:00
David Baker 88af58d41d Update to app_id / app_instance_id (partially) and mangle to be PEP8 compliant. 2014-12-03 13:37:02 +00:00
Erik Johnston 6941a19715 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-03 11:56:49 +00:00
Erik Johnston d06dfc70b0 Fix bug where did not always resolve all the deferreds in _attempt_new_transaction 2014-12-03 11:50:23 +00:00
Paul "LeoNerd" Evans 66b0596b7a Don't log synapse.storage at DEBUG level when under -v; require -v -v for that 2014-12-02 22:28:18 +00:00
Paul "LeoNerd" Evans 10eb8f070c Workaround for non-uniqueness of room member events in the database confusing HAVING COUNT() test of room membership intersection (with thanks to Tom Molesworth) 2014-12-02 21:36:53 +00:00
Paul "LeoNerd" Evans 274d137b00 Extra verbosity on logging of received presence info 2014-12-02 20:51:28 +00:00
Paul "LeoNerd" Evans 307f94dcbe Squash room_ids list to a set() to remove duplicates - with TODO marker as I have no idea /why/ the dups are happening in the first place 2014-12-02 20:51:12 +00:00
Mark Haines 2f804a7072 Fix pyflakes and pep8 warnings 2014-12-02 19:55:18 +00:00
Mark Haines 5da65085d1 Get uploads working with new media repo 2014-12-02 19:51:47 +00:00
Paul "LeoNerd" Evans afb646dc1e Allow GET /room/:room_id/messages without 'limit' parameter to default to 10 2014-12-02 18:01:16 +00:00
Erik Johnston 23a2c42469 Bump version 2014-12-02 17:45:40 +00:00
Mark Haines 7993e3d10d SYN-187: Set a more sensible default for the content_addr 2014-12-02 17:20:02 +00:00
Mark Haines 279c48c8b4 Write the upload portion of version 1 of the media repository 2014-12-02 17:13:14 +00:00
David Baker 7642d95d5e Merge branch 'develop' into pushers 2014-12-02 13:50:05 +00:00
Erik Johnston c1e66800a9 Begin fleshing out a new Event object 2014-12-02 11:40:22 +00:00
Erik Johnston 9d53228158 Change DomainSpecificString so that it doesn't use a HomeServer object 2014-12-02 10:42:28 +00:00
Erik Johnston ec2b5d8c28 Store full JSON of events in db 2014-12-01 16:22:07 +00:00
Erik Johnston ab74afdd8d Bump version 2014-11-27 17:30:08 +00:00
Erik Johnston cce32f8dc5 Bump version and changelog 2014-11-27 17:15:32 +00:00
Erik Johnston 1505055334 Don't return outliers when we get recent events for rooms. 2014-11-27 16:38:50 +00:00
Erik Johnston 027542e2e5 Fix bugs when joining a remote room that has dodgy event graphs. This should also fix the number of times a HS will trigger a GET /event/ 2014-11-27 16:02:26 +00:00
Erik Johnston 0294fba042 on_receive_pdu takes more args 2014-11-27 14:46:33 +00:00
Erik Johnston 07699b5871 Change the way we get missing auth and state events 2014-11-27 14:31:43 +00:00
Erik Johnston b8849c8cbf Re-sign events when we return them via federation as a temporary hack to work around the problem where we reconstruct events differently than when they were signed 2014-11-27 13:53:31 +00:00
Erik Johnston 00ab5cd6f2 Attempt to fix bug where we 500d an event stream due to trying to cancel a timer twice 2014-11-26 18:04:33 +00:00
Erik Johnston 858e87ab0d Add a workaround for bug where some initial join events don't reference creation events in their auth_events 2014-11-26 16:29:03 +00:00
Erik Johnston 6c485c282d Catch exceptions when trying to add an entry to rooms tables 2014-11-26 16:06:20 +00:00
Erik Johnston 5288a7dc9a Bump version and changelog 2014-11-26 15:19:08 +00:00
Erik Johnston 4e2ffe79a4 Don't delete the entire current_state_events table 2014-11-26 15:17:08 +00:00
Erik Johnston ad13f14432 Bump version numbers and change log 2014-11-26 11:53:12 +00:00
Erik Johnston cb76945688 Add update delta for schema change 2014-11-26 11:17:19 +00:00
Erik Johnston 87538711b6 Update schema to support multiple signatures 2014-11-26 11:14:30 +00:00
Erik Johnston 3598c11c8d Correctly handle the case where we get an event for an unknown room, which turns out we are actually in 2014-11-26 10:41:08 +00:00
Erik Johnston 64fc859dac Fix bugs in invite/join dances.
We now do more implement more of the auth on the events so that we
don't reject valid events.
2014-11-25 17:59:49 +00:00
Kegan Dougal 3536fd7d60 Don't double url-decode state event types. 2014-11-25 11:02:19 +00:00
Mark Haines 15099fade5 Drop log level for incorrect logging contexts to WARN if the context is wrong and DEBUG if the context is missing 2014-11-25 10:57:31 +00:00
Erik Johnston 4961a4fab1 Mark the auth events as possible outlier 2014-11-24 13:55:49 +00:00
Erik Johnston e549aac127 Add missing None check 2014-11-24 13:47:58 +00:00
Erik Johnston 2bca242fdc Ask for any auth events that we don't have 2014-11-24 13:46:41 +00:00
Erik Johnston 4bd0ab76c6 We don't always want to Auth get_persisted_pdu 2014-11-24 12:56:17 +00:00
Erik Johnston a46e5ef621 SYN-163: Add an order by rowid to selects.
This should fix the bug where the edges of the graph get returned in a
different order than they were inserted in, and so no get_event no
longer returned the exact same JSON as was inserted. This meant that
signature checks failed.
2014-11-24 10:56:36 +00:00
Matthew Hodgson ae8ad55cb8 typos 2014-11-24 01:41:12 +00:00
Mark Haines fd40a80a68 Return 404 M_NOT_FOUND when trying to look up a room alias that doesn't exist 2014-11-21 15:11:48 +00:00
David Baker eb6aedf92c More work on pushers. Attempt to do HTTP pokes. Not sure if the actual HTTP pokes work or not yet but the retry semantics are pretty good. 2014-11-21 12:21:00 +00:00
Paul "LeoNerd" Evans 5f19c55731 SYN-58: Allow passing explicit limit=0 to initialSync to request no messages at all; missing still implies default 10 2014-11-20 19:33:45 +00:00
David Baker 58f82e2e54 Merge branch 'develop' into pushers 2014-11-20 18:25:31 +00:00
David Baker 23465a30b6 Merge branch 'develop' into pushers 2014-11-20 18:17:46 +00:00
Mark Haines 610c2ea131 Fix pep8 and pyflakes warnings 2014-11-20 18:00:10 +00:00
David Baker f1c7f8e813 Merge branch 'develop' into http_client_refactor 2014-11-20 17:49:48 +00:00
David Baker e377d33652 Separate out the matrix http client completely because just about all of its code it now separate from the simple case we need for standard HTTP(S) 2014-11-20 17:41:56 +00:00
Mark Haines db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
Mark Haines 32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
David Baker 20326054da Oops, I removed this param. 2014-11-20 15:24:38 +00:00
David Baker ebf6c08a47 Merge branch 'http_client_refactor' into pushers 2014-11-20 14:01:41 +00:00
David Baker dc60eee50e Refactor the HTTP clients a little. 2014-11-20 13:53:34 +00:00
David Baker cf66532ac1 CaptchaServerHttpClient should extend the base, not matrix http client. 2014-11-20 12:48:21 +00:00
Mark Haines b2aeaa2dcc Merge branch 'master' into develop 2014-11-20 10:00:13 +00:00
Mark Haines dcb99e4972 SYN-153: Fix formatting of federation error message 2014-11-20 09:58:23 +00:00
David Baker 051b185811 remove random half-line 2014-11-19 18:37:00 +00:00
David Baker 74c3879760 Start creating a module to do generic notifications (just prints them to stdout currently!) 2014-11-19 18:20:59 +00:00
Erik Johnston 1a9551db82 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.5.0 2014-11-19 18:03:03 +00:00
Erik Johnston 5b46ce579b Bump version, changelog and upgrade.rst 2014-11-19 18:00:57 +00:00
Erik Johnston 415ddf59bb Don't add a 'prev' key to m.room.member messages 2014-11-19 17:59:51 +00:00
Paul "LeoNerd" Evans 03dc63f6c8 Initialise UserPresenceCache instances to always contain a 'presence' key 2014-11-19 17:31:46 +00:00
Erik Johnston 4eada9a908 Fix backfill request 2014-11-19 17:22:37 +00:00
Erik Johnston 512993b57f Only users can set state events which have their own user_id 2014-11-19 17:22:37 +00:00
Mark Haines ca91bb2f7f Sometimes there isn't a current logging context 2014-11-19 17:18:55 +00:00
Mark Haines 8993affdc0 SYN-153: Raise 404 if room alias is not found 2014-11-19 17:14:14 +00:00
Mark Haines 0d1221155e remove unused import 2014-11-19 16:40:01 +00:00
Mark Haines c5eabe3143 replace user_id with sender 2014-11-19 16:38:40 +00:00
Mark Haines 97c7c34f6f Preserve logging context in a few more places, drop the logging context after it has been stashed to reduce potential for confusion 2014-11-19 16:37:43 +00:00
Mark Haines 3e54d70ae2 SYN-141: Encode query params as UTF-8. 2014-11-18 19:43:08 +00:00
Mark Haines 428581dd05 SYN-144: Remove bad keys from pdu json objects, convert age_ts to age
for all pdus sent.
2014-11-18 19:20:25 +00:00
Paul "LeoNerd" Evans 572a1ca42a Remember also to UTF-8 decode bytes in room alias names in directory server URLs 2014-11-18 18:06:35 +00:00
Paul "LeoNerd" Evans 3bfc3dd45b Remember to URL decode the room_id in room initialSync 2014-11-18 17:44:55 +00:00
Mark Haines db7e8b5619 SYN-141: Decode the query params as UTF-8 2014-11-18 17:17:57 +00:00
Mark Haines 54c438d8d3 Remove unused variable 2014-11-18 16:46:12 +00:00
Mark Haines 1731af3f29 SYN-104: When going backwards the end token should be before the last event 2014-11-18 16:45:06 +00:00
Paul Evans 11fd81e398 Merge pull request #17 from matrix-org/room-initial-sync
Room initial sync
2014-11-18 16:44:25 +00:00
Paul "LeoNerd" Evans 88dfa7baa6 Ensure to parse a real pagination config object out of room initialSync request and pass it on 2014-11-18 16:34:43 +00:00
Paul "LeoNerd" Evans 75e95c45a2 Rename message handler's new snapshot_room to room_initial_sync() as that better suits its purpose 2014-11-18 16:02:44 +00:00
Erik Johnston c6ea29d916 Revert accidental commit of bad file 2014-11-18 15:57:00 +00:00
Paul "LeoNerd" Evans e9f587ecba Merge remote-tracking branch 'origin/develop' into room-initial-sync 2014-11-18 15:48:30 +00:00
Mark Haines 3553101eb3 Null check when determining default power levels 2014-11-18 15:43:17 +00:00
Mark Haines b01dd76be1 SYN-149: Enable auth for events added during room creation since they should pass auth checks 2014-11-18 15:42:53 +00:00
Erik Johnston 95614e5220 Fix auth to correctly handle initial creation of rooms 2014-11-18 15:36:41 +00:00
Mark Haines ae9c2ab165 SYN-149: Send join event immediately after the room create event 2014-11-18 15:29:48 +00:00
Paul "LeoNerd" Evans 33d328d967 Include room members' presence in room initialSync 2014-11-18 15:28:58 +00:00
Paul "LeoNerd" Evans 759db7d7d5 Added ability to .get_state() from the PresenceHandler by returning a complete m.presence event 2014-11-18 15:25:55 +00:00
Paul "LeoNerd" Evans 4c18e08036 Don't expect all _user_cachemap entries to definitely contain a "last_active" key 2014-11-18 15:10:11 +00:00
Mark Haines a5b88c489e Split out sending the room alias events from creating the alias so that we can do them in the right point when creating a room 2014-11-18 15:03:13 +00:00
Paul "LeoNerd" Evans 17f977a9de Include 'messages' snapshot in room initialSync 2014-11-18 14:07:51 +00:00
Mark Haines 26fc878944 Stop before starting when restarting 2014-11-17 19:16:15 +00:00
Paul "LeoNerd" Evans 31a049eb69 Merge branch 'develop' into room-initial-sync
Conflicts:
	synapse/handlers/message.py
2014-11-17 16:59:24 +00:00
Mark Haines cf45e57d9c SYN-148: Add the alias after creating the room 2014-11-17 16:37:33 +00:00
Mark Haines 2eaf689f71 These lines aren't doing anything 2014-11-17 10:41:35 +00:00
Mark Haines cb4b6c844a Merge PDUs and Events into one object 2014-11-14 21:25:02 +00:00
Mark Haines 8c2b5ea7c4 Fix PDU and event signatures 2014-11-14 19:11:04 +00:00
Mark Haines de1ec90133 Validate signatures on incoming events 2014-11-14 19:11:04 +00:00
David Baker fe3401e037 Be more helpful and tell the user how to generate a config too. 2014-11-14 13:30:06 +00:00
David Baker 933ce76057 Adding --generate-config will not help if the user has not specified a config file. 2014-11-14 13:24:12 +00:00
Mark Haines e903c941cb Merge branch 'develop' into request_logging
Conflicts:
	setup.py
	synapse/storage/_base.py
	synapse/util/async.py
2014-11-14 11:16:50 +00:00
Mark Haines 8d8a133c89 SYN-103: Remove "origin" and "destination" keys from edus 2014-11-13 15:49:03 +00:00
Mark Haines e7c6d2c9d9 SYN-138: Rewrite synctl in python and include it in the python distribution 2014-11-13 14:39:30 +00:00
Erik Johnston f04b3d5042 Store all signatures on events rather than just dropping them 2014-11-12 17:02:34 +00:00
Erik Johnston b2596c660b Add a few more comments to the federation handler 2014-11-12 16:20:30 +00:00
Erik Johnston e715741abc Update some of the docs in event_federation 2014-11-12 16:20:30 +00:00
Erik Johnston e24d5cb97d Document StateStore and use transactions 2014-11-12 14:33:48 +00:00
Erik Johnston 58c0ef90c9 Add indices to state group tables 2014-11-12 14:33:48 +00:00
Erik Johnston 3db0efa69f Fix pyflake warnings and add a FIXME comment to deal with auth_chains received when joining 2014-11-12 11:27:02 +00:00
Erik Johnston 6fea478d2e Fix bugs with invites/joins across federatiom.
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
Erik Johnston 61ecb13bf0 PEP8ify 2014-11-11 18:00:13 +00:00
Erik Johnston 37900a92db Only allow people in a room to look up room state. 2014-11-11 17:55:32 +00:00
Erik Johnston 997ed151db synapse.state docs. 2014-11-11 17:45:46 +00:00
Erik Johnston 3db2c0d43e Rename annotate_state_groups to annotate_event_with_state 2014-11-11 16:58:53 +00:00
Mark Haines a8ceeec0fd Merge pull request #12 from matrix-org/federation_authorization
Federation authorization
2014-11-11 16:40:50 +00:00
Erik Johnston 092979b8cc Fix bugs which broke federation due to changes in function signatures. 2014-11-11 14:19:13 +00:00
Erik Johnston 5ff0bfb81d Fix bug where we /always/ created a new state group 2014-11-11 14:16:41 +00:00
Erik Johnston 2cdff00788 Fix typo in validator 2014-11-11 10:31:47 +00:00
Erik Johnston 0292d991af Add EventValidator module 2014-11-11 08:09:28 +00:00
Matthew Hodgson 7df8c8c287 apply some cache headers to try to make the content repo less nutso 2014-11-11 05:36:39 +00:00
Paul "LeoNerd" Evans c23afed39a Include room membership in room initialSync 2014-11-10 19:34:47 +00:00
Paul "LeoNerd" Evans 1fd8139138 Put room state in room initialSync output - I guess this is right; I really can't find any other tests similar... 2014-11-10 19:29:58 +00:00
Paul "LeoNerd" Evans 269f80bf8e Have room initialSync return the room's room_id 2014-11-10 19:02:19 +00:00
Erik Johnston a8e565eca8 Add an EventValidator. Fix bugs in auth ++ storage 2014-11-10 18:25:42 +00:00
Erik Johnston cdc1b5d629 Fix regression where we did not return redacted events. 2014-11-10 15:21:30 +00:00
Erik Johnston 5d439b127b PEP8 2014-11-10 13:46:44 +00:00
Erik Johnston c46088405a Remove useless comments 2014-11-10 13:39:33 +00:00
Erik Johnston 003668cfaa Add auth to the various server-server APIs 2014-11-10 13:37:24 +00:00
Erik Johnston 6447db063a Fix backfill to work. Add auth to backfill request 2014-11-10 11:59:51 +00:00
Erik Johnston 65f846ade0 Notify users about invites. 2014-11-10 11:15:02 +00:00
Erik Johnston 407d8a5019 Fix invite auth 2014-11-10 10:35:43 +00:00
Erik Johnston 6cb6cb9e69 Tidy up some of the unused sql tables 2014-11-10 10:31:00 +00:00
Erik Johnston 1c06806f90 Finish redaction algorithm. 2014-11-10 10:21:32 +00:00
Erik Johnston 07286a73b1 Use current state to get room hosts, rather than querying the database 2014-11-07 16:03:31 +00:00
Erik Johnston 02c3b1c9e2 Add '/event_auth/' federation api 2014-11-07 15:35:53 +00:00
Erik Johnston d2fb2b8095 Implement invite part of invite join dance 2014-11-07 13:41:00 +00:00
Erik Johnston 328dab2463 Remove /context/ request 2014-11-07 11:40:38 +00:00
Erik Johnston 97a096b507 Add hash of current state to events 2014-11-07 11:37:06 +00:00
Erik Johnston 3b4dec442d Return auth chain when handling send_join 2014-11-07 11:22:12 +00:00
Erik Johnston 16a0815fac Fix bug in _get_auth_chain_txn 2014-11-07 11:21:20 +00:00
Erik Johnston 3cb678f84c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-11-07 10:55:28 +00:00