Mirrors
/
synapse
mirror of https://github.com/element-hq/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of github.com:matrix-org/synapse

This commit is contained in:
Kegan Dougal 2014-08-13 15:53:44 +01:00
parent 5a59da8a94 ecc299d458
commit 38c7e92331
13 changed files with 160 additions and 146 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
MAP.rst Normal file
View File

@ -0,0 +1,35 @@
Directory Structure
===================
Warning: this may be a bit stale...
::
.
├── cmdclient Basic CLI python Matrix client
├── demo Scripts for running standalone Matrix demos
├── docs All doc, including the draft Matrix API spec
│   ├── client-server The client-server Matrix API spec
│   ├── model Domain-specific elements of the Matrix API spec
│   ├── server-server The server-server model of the Matrix API spec
│   └── sphinx The internal API doc of the Synapse homeserver
├── experiments Early experiments of using Synapse's internal APIs
├── graph Visualisation of Matrix's distributed message store
├── synapse The reference Matrix homeserver implementation
│   ├── api Common building blocks for the APIs
│   │   ├── events Definition of state representation Events
│   │   └── streams Definition of streamable Event objects
│   ├── app The __main__ entry point for the homeserver
│   ├── crypto The PKI client/server used for secure federation
│   │   └── resource PKI helper objects (e.g. keys)
│   ├── federation Server-server state replication logic
│   ├── handlers The main business logic of the homeserver
│   ├── http Wrappers around Twisted's HTTP server & client
│   ├── rest Servlet-style RESTful API
│   ├── storage Persistence subsystem (currently only sqlite3)
│   │   └── schema sqlite persistence schema
│   └── util Synapse-specific utilities
├── tests Unit tests for the Synapse homeserver
└── webclient Basic AngularJS Matrix web client

133
README.rst
View File

@ -1,16 +1,52 @@
About
=====
Quick Start
===========
Matrix is an ambitious new ecosystem for open federated Instant Messaging and
VoIP[1].
VoIP[1]. The basics you need to know to get up and running are:
Matrix specifies a set of pragmatic RESTful HTTP JSON APIs as an open standard,
providing:
- Chatrooms are distributed and do not exist on any single server. Rooms
can be found using names like ``#matrix:matrix.org`` or
``#test:localhost:8080`` or they can be ephemeral.
- Matrix user IDs look like ``@matthew:matrix.org`` (although in the future
you will normally refer to yourself and others using a 3PID: email
address, phone number, etc rather than manipulating matrix user IDs)
The overall architecture is::
client <----> homeserver <=================> homeserver <-----> client
e.g. matrix.org:8080 e.g. mydomain.net:8080
To get up and running:
- To simply play with an **existing** homeserver you can
just go straight to http://matrix.org/alpha.
- To run your own **private** homeserver on localhost:8080, install synapse
with ``python setup.py develop --user`` and then run one with
``python synapse/app/homeserver.py``
- To run your own webclient:
``cd webclient; python -m SimpleHTTPServer`` and hit http://localhost:8000
in your web browser (a recent Chrome, Safari or Firefox for now,
please...)
- To make the homeserver **public** and let it exchange messages with
other homeservers and participate in the overall Matrix federation, open
up port 8080 and run ``python synapse/app/homeserver.py --host
machine.my.domain.name``. Then come join ``#matrix:matrix.org`` and
say hi! :)
About Matrix
============
Matrix specifies a set of pragmatic RESTful HTTP JSON APIs for VoIP and IM as an
open standard, providing:
- Creating and managing fully distributed chat rooms with no
single points of control or failure
- Eventually-consistent cryptographically secure[2] synchronisation of room
state across a global open network of federated servers and services
state across a global open network of federated servers and services
- Sending and receiving extensible messages in a room with (optional)
end-to-end encryption[3]
- Inviting, joining, leaving, kicking, banning room members
@ -57,74 +93,6 @@ Thanks for trying Matrix!
[3] End-to-end encryption is currently in development
Quick Start
===========
The basics you need to know about Matrix are:
- Chatrooms look like ``#matrix:matrix.org`` or ``#test:localhost:8080``
- Matrix user IDs look like ``@matthew:matrix.org`` (although in the future
you will normally refer to yourself and others using a 3PID: email
address, phone number, etc rather than manipulating matrix user IDs)
- To simply play with an **existing** homeserver (e.g. matrix.org), you can
just go straight to http://matrix.org/alpha, specify a homeserver
(defaults to matrix.org) and sign up and use it. (Sign-up security is
currently work-in-progress)
- To run your own **private** homeserver on localhost:8080, install synapse
with ``python setup.py develop --user`` and then run one with
``python synapse/app/homeserver.py``
- To run your own webclient:
``cd webclient; python -m SimpleHTTPServer`` and hit http://localhost:8000
in your web browser (a recent Chrome, Safari or Firefox for now,
please...)
- For now, register some accounts like ``@testing:localhost:8080`` from
different browsers, join a room like ``#test:localhost:8080`` and have a
play.
- To quickly run a **public** homeserver that can exchange messages with
other homeservers and participate in the overall Matrix federation, open
up port 8080 and run ``python synapse/app/homeserver.py --host
machine.my.domain.name``. Then come join ``#matrix:matrix.org`` and
say hi! :)
Directory Structure
===================
::
.
├── cmdclient Basic CLI python Matrix client
├── demo Scripts for running standalone Matrix demos
├── docs All doc, including the draft Matrix API spec
│   ├── client-server The client-server Matrix API spec
│   ├── model Domain-specific elements of the Matrix API spec
│   ├── server-server The server-server model of the Matrix API spec
│   └── sphinx The internal API doc of the Synapse homeserver
├── experiments Early experiments of using Synapse's internal APIs
├── graph Visualisation of Matrix's distributed message store
├── synapse The reference Matrix homeserver implementation
│   ├── api Common building blocks for the APIs
│   │   ├── events Definition of state representation Events
│   │   └── streams Definition of streamable Event objects
│   ├── app The __main__ entry point for the homeserver
│   ├── crypto The PKI client/server used for secure federation
│   │   └── resource PKI helper objects (e.g. keys)
│   ├── federation Server-server state replication logic
│   ├── handlers The main business logic of the homeserver
│   ├── http Wrappers around Twisted's HTTP server & client
│   ├── rest Servlet-style RESTful API
│   ├── storage Persistence subsystem (currently only sqlite3)
│   │   └── schema sqlite persistence schema
│   └── util Synapse-specific utilities
├── tests Unit tests for the Synapse homeserver
└── webclient Basic AngularJS Matrix web client
Homeserver Installation
=======================
@ -151,6 +119,12 @@ may need to also run:
$ sudo apt-get install python-pip
$ sudo pip install --upgrade setuptools
If you get errors about ``sodium.h`` being missing, you may also need to
manually install a newer PyNaCl via pip as setuptools installs an old one. Or
you can check PyNaCl out of git directly (https://github.com/pyca/pynacl) and
installing it. Installing PyNaCl using pip may also work (remember to remove any
other versions installed by setuputils in, for example, ~/.local/lib).
This will run a process of downloading and installing into your
user's .local/lib directory all of the required dependencies that are
@ -168,8 +142,8 @@ This should end with a 'PASSED' result::
PASSED (successes=143)
Running The Synapse Homeserver
==============================
Setting up Federation
=====================
In order for other homeservers to send messages to your server, it will need to
be publicly visible on the internet, and they will need to know its host name.
@ -215,7 +189,7 @@ Running a Demo Federation of Homeservers
If you want to get up and running quickly with a trio of homeservers in a
private federation (``localhost:8080``, ``localhost:8081`` and
``localhost:8082``) which you can then point a demo webclient at, simply run::
``localhost:8082``) which you can then access through the webclient running at http://localhost:8080. Simply run::
$ demo/start.sh
@ -279,8 +253,8 @@ as the primary means of identity and E2E encryption is not complete. As such,
we're not yet running an identity server in public.
How does it all work?!
======================
Where's the spec?!
==================
For now, please go spelunking in the ``docs/`` directory to find out.
@ -297,3 +271,4 @@ sphinxcontrib-napoleon::
Building internal API documentation::
$ python setup.py build_sphinx

14
docs/server-server/specification.rst
View File

@ -116,9 +116,13 @@ federation.]]
Protocol URLs
=============
All these URLs are namespaced within a prefix of
/matrix/federation/v1/...
For active pushing of messages representing live activity "as it happens":
PUT /send/:transaction_id/
PUT .../send/:transaction_id/
Body: JSON encoding of a single Transaction
Response: [[TODO(paul): I don't actually understand what
@ -132,7 +136,7 @@ For active pushing of messages representing live activity "as it happens":
To fetch a particular PDU:
GET /pdu/:origin/:pdu_id/
GET .../pdu/:origin/:pdu_id/
Response: JSON encoding of a single Transaction containing one PDU
@ -142,7 +146,7 @@ To fetch a particular PDU:
To fetch all the state of a given context:
GET /state/:context/
GET .../state/:context/
Response: JSON encoding of a single Transaction containing multiple PDUs
@ -153,7 +157,7 @@ To fetch all the state of a given context:
To paginate events on a given context:
GET /paginate/:context/
GET .../paginate/:context/
Query args: v, limit
Response: JSON encoding of a single Transaction containing multiple PDUs
@ -167,7 +171,7 @@ To paginate events on a given context:
To stream events all the events:
GET /pull/
GET .../pull/
Query args: origin, v
Response: JSON encoding of a single Transaction consisting of multiple PDUs

8
synapse/api/constants.py
View File

@ -37,7 +37,7 @@ class Feedback(object):
class PresenceState(object):
"""Represents the presence state of a user."""
OFFLINE = 0
BUSY = 1
ONLINE = 2
FREE_FOR_CHAT = 3
OFFLINE = u"offline"
UNAVAILABLE = u"unavailable"
ONLINE = u"online"
FREE_FOR_CHAT = u"free_for_chat"

5
synapse/app/homeserver.py
View File

@ -88,9 +88,8 @@ def setup_logging(verbosity=0, filename=None, config_path=None):
'%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(message)s'
)
if not verbosity or verbosity == 0:
level = logging.INFO
elif verbosity == 1:
level = logging.INFO
if verbosity:
level = logging.DEBUG
# FIXME: we need a logging.WARN for a -q quiet option

33
synapse/federation/transport.py
View File

@ -33,6 +33,9 @@ import re
logger = logging.getLogger(__name__)
PREFIX = "/matrix/federation/v1"
class TransportLayer(object):
"""This is a basic implementation of the transport layer that translates
transactions and other requests to/from HTTP.
@ -84,9 +87,9 @@ class TransportLayer(object):
logger.debug("get_context_state dest=%s, context=%s",
destination, context)
path = "/state/%s/" % context
subpath = "/state/%s/" % context
return self._do_request_for_transaction(destination, path)
return self._do_request_for_transaction(destination, subpath)
@log_function
def get_pdu(self, destination, pdu_origin, pdu_id):
@ -104,9 +107,9 @@ class TransportLayer(object):
logger.debug("get_pdu dest=%s, pdu_origin=%s, pdu_id=%s",
destination, pdu_origin, pdu_id)
path = "/pdu/%s/%s/" % (pdu_origin, pdu_id)
subpath = "/pdu/%s/%s/" % (pdu_origin, pdu_id)
return self._do_request_for_transaction(destination, path)
return self._do_request_for_transaction(destination, subpath)
@log_function
def paginate(self, dest, context, pdu_tuples, limit):
@ -130,14 +133,14 @@ class TransportLayer(object):
if not pdu_tuples:
return
path = "/paginate/%s/" % context
subpath = "/paginate/%s/" % context
args = {"v": ["%s,%s" % (i, o) for i, o in pdu_tuples]}
args["limit"] = limit
return self._do_request_for_transaction(
dest,
path,
subpath,
args=args,
)
@ -166,7 +169,7 @@ class TransportLayer(object):
code, response = yield self.client.put_json(
transaction.destination,
path="/send/%s/" % transaction.transaction_id,
path=PREFIX + "/send/%s/" % transaction.transaction_id,
data=data
)
@ -189,7 +192,7 @@ class TransportLayer(object):
# This is when someone is trying to send us a bunch of data.
self.server.register_path(
"PUT",
re.compile("^/send/([^/]*)/$"),
re.compile("^" + PREFIX + "/send/([^/]*)/$"),
self._on_send_request
)
@ -207,7 +210,7 @@ class TransportLayer(object):
# This is for when someone asks us for everything since version X
self.server.register_path(
"GET",
re.compile("^/pull/$"),
re.compile("^" + PREFIX + "/pull/$"),
lambda request: handler.on_pull_request(
request.args["origin"][0],
request.args["v"]
@ -218,7 +221,7 @@ class TransportLayer(object):
# data_id pair.
self.server.register_path(
"GET",
re.compile("^/pdu/([^/]*)/([^/]*)/$"),
re.compile("^" + PREFIX + "/pdu/([^/]*)/([^/]*)/$"),
lambda request, pdu_origin, pdu_id: handler.on_pdu_request(
pdu_origin, pdu_id
)
@ -227,7 +230,7 @@ class TransportLayer(object):
# This is when someone asks for all data for a given context.
self.server.register_path(
"GET",
re.compile("^/state/([^/]*)/$"),
re.compile("^" + PREFIX + "/state/([^/]*)/$"),
lambda request, context: handler.on_context_state_request(
context
)
@ -235,7 +238,7 @@ class TransportLayer(object):
self.server.register_path(
"GET",
re.compile("^/paginate/([^/]*)/$"),
re.compile("^" + PREFIX + "/paginate/([^/]*)/$"),
lambda request, context: self._on_paginate_request(
context, request.args["v"],
request.args["limit"]
@ -244,7 +247,7 @@ class TransportLayer(object):
self.server.register_path(
"GET",
re.compile("^/context/([^/]*)/$"),
re.compile("^" + PREFIX + "/context/([^/]*)/$"),
lambda request, context: handler.on_context_pdus_request(context)
)
@ -300,7 +303,7 @@ class TransportLayer(object):
@defer.inlineCallbacks
@log_function
def _do_request_for_transaction(self, destination, path, args={}):
def _do_request_for_transaction(self, destination, subpath, args={}):
"""
Args:
destination (str)
@ -313,7 +316,7 @@ class TransportLayer(object):
data = yield self.client.get_json(
destination,
path=path,
path=PREFIX + subpath,
args=args,
)

15
tests/federation/test_federation.py
View File

@ -96,7 +96,7 @@ class FederationTestCase(unittest.TestCase):
# Empty context initially
(code, response) = yield self.mock_http_server.trigger("GET",
"/state/my-context/", None)
"/matrix/federation/v1/state/my-context/", None)
self.assertEquals(200, code)
self.assertFalse(response["pdus"])
@ -121,7 +121,7 @@ class FederationTestCase(unittest.TestCase):
)
(code, response) = yield self.mock_http_server.trigger("GET",
"/state/my-context/", None)
"/matrix/federation/v1/state/my-context/", None)
self.assertEquals(200, code)
self.assertEquals(1, len(response["pdus"]))
@ -132,7 +132,7 @@ class FederationTestCase(unittest.TestCase):
)
(code, response) = yield self.mock_http_server.trigger("GET",
"/pdu/red/abc123def456/", None)
"/matrix/federation/v1/pdu/red/abc123def456/", None)
self.assertEquals(404, code)
# Now insert such a PDU
@ -151,7 +151,7 @@ class FederationTestCase(unittest.TestCase):
)
(code, response) = yield self.mock_http_server.trigger("GET",
"/pdu/red/abc123def456/", None)
"/matrix/federation/v1/pdu/red/abc123def456/", None)
self.assertEquals(200, code)
self.assertEquals(1, len(response["pdus"]))
self.assertEquals("m.text", response["pdus"][0]["pdu_type"])
@ -177,7 +177,7 @@ class FederationTestCase(unittest.TestCase):
self.mock_http_client.put_json.assert_called_with(
"remote",
path="/send/1000000/",
path="/matrix/federation/v1/send/1000000/",
data={
"ts": 1000000,
"origin": "test",
@ -212,7 +212,7 @@ class FederationTestCase(unittest.TestCase):
# MockClock ensures we can guess these timestamps
self.mock_http_client.put_json.assert_called_with(
"remote",
path="/send/1000000/",
path="/matrix/federation/v1/send/1000000/",
data={
"origin": "test",
"ts": 1000000,
@ -234,7 +234,8 @@ class FederationTestCase(unittest.TestCase):
self.federation.register_edu_handler("m.test", recv_observer)
yield self.mock_http_server.trigger("PUT", "/send/1001000/",
yield self.mock_http_server.trigger("PUT",
"/matrix/federation/v1/send/1001000/",
"""{
"origin": "remote",
"ts": 1001000,

24
tests/handlers/test_presence.py
View File

@ -27,7 +27,7 @@ from synapse.handlers.presence import PresenceHandler, UserPresenceCache
OFFLINE = PresenceState.OFFLINE
BUSY = PresenceState.BUSY
UNAVAILABLE = PresenceState.UNAVAILABLE
ONLINE = PresenceState.ONLINE
@ -149,12 +149,12 @@ class PresenceStateTestCase(unittest.TestCase):
yield self.handler.set_state(
target_user=self.u_apple, auth_user=self.u_apple,
state={"state": BUSY, "status_msg": "Away"})
state={"state": UNAVAILABLE, "status_msg": "Away"})
mocked_set.assert_called_with("apple",
{"state": 1, "status_msg": "Away"})
{"state": UNAVAILABLE, "status_msg": "Away"})
self.mock_start.assert_called_with(self.u_apple,
state={"state": 1, "status_msg": "Away"})
state={"state": UNAVAILABLE, "status_msg": "Away"})
yield self.handler.set_state(
target_user=self.u_apple, auth_user=self.u_apple,
@ -555,7 +555,7 @@ class PresencePushTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@apple:test",
"state": 2},
"state": "online"},
],
}),
call(
@ -564,7 +564,7 @@ class PresencePushTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@apple:test",
"state": 2},
"state": "online"},
],
})
], any_order=True)
@ -582,7 +582,7 @@ class PresencePushTestCase(unittest.TestCase):
"remote", "m.presence", {
"push": [
{"user_id": "@potato:remote",
"state": 2},
"state": "online"},
],
}
)
@ -646,7 +646,7 @@ class PresencePushTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@apple:test",
"state": 2},
"state": "online"},
],
}),
call(
@ -655,7 +655,7 @@ class PresencePushTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@banana:test",
"state": 0},
"state": "offline"},
],
}),
], any_order=True)
@ -666,7 +666,7 @@ class PresencePushTestCase(unittest.TestCase):
self.handler._user_cachemap[self.u_clementine] = UserPresenceCache()
self.handler._user_cachemap[self.u_clementine].update(
{"state": PresenceState.ONLINE}, self.u_clementine)
{"state": ONLINE}, self.u_clementine)
self.room_members.append(self.u_potato)
yield self.distributor.fire("user_joined_room", self.u_clementine,
@ -680,7 +680,7 @@ class PresencePushTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@clementine:test",
"state": 2},
"state": "online"},
],
}),
)
@ -882,7 +882,7 @@ class PresencePollingTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@banana:test",
"state": 0,
"state": "offline",
"status_msg": None},
],
},

12
tests/handlers/test_presencelike.py
View File

@ -29,7 +29,7 @@ from synapse.handlers.profile import ProfileHandler
OFFLINE = PresenceState.OFFLINE
BUSY = PresenceState.BUSY
UNAVAILABLE = PresenceState.UNAVAILABLE
ONLINE = PresenceState.ONLINE
@ -125,12 +125,12 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase):
yield self.handlers.presence_handler.set_state(
target_user=self.u_apple, auth_user=self.u_apple,
state={"state": BUSY, "status_msg": "Away"})
state={"state": UNAVAILABLE, "status_msg": "Away"})
mocked_set.assert_called_with("apple",
{"state": 1, "status_msg": "Away"})
{"state": UNAVAILABLE, "status_msg": "Away"})
self.mock_start.assert_called_with(self.u_apple,
state={"state": 1, "status_msg": "Away",
state={"state": UNAVAILABLE, "status_msg": "Away",
"displayname": "Frank",
"avatar_url": "http://foo"})
@ -220,7 +220,7 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase):
content={
"push": [
{"user_id": "@apple:test",
"state": 2,
"state": "online",
"displayname": "Frank",
"avatar_url": "http://foo"},
],
@ -238,7 +238,7 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase):
"remote", "m.presence", {
"push": [
{"user_id": "@potato:remote",
"state": 2,
"state": "online",
"displayname": "Frank",
"avatar_url": "http://foo"},
],

12
tests/rest/test_presence.py
View File

@ -31,7 +31,7 @@ logging.getLogger().addHandler(logging.NullHandler())
OFFLINE = PresenceState.OFFLINE
BUSY = PresenceState.BUSY
UNAVAILABLE = PresenceState.UNAVAILABLE
ONLINE = PresenceState.ONLINE
@ -69,7 +69,7 @@ class PresenceStateTestCase(unittest.TestCase):
def test_get_my_status(self):
mocked_get = self.mock_handler.get_state
mocked_get.return_value = defer.succeed(
{"state": 2, "status_msg": "Available"})
{"state": ONLINE, "status_msg": "Available"})
(code, response) = yield self.mock_server.trigger("GET",
"/presence/%s/status" % (myid), None)
@ -87,12 +87,12 @@ class PresenceStateTestCase(unittest.TestCase):
(code, response) = yield self.mock_server.trigger("PUT",
"/presence/%s/status" % (myid),
'{"state": 1, "status_msg": "Away"}')
'{"state": "unavailable", "status_msg": "Away"}')
self.assertEquals(200, code)
mocked_set.assert_called_with(target_user=self.u_apple,
auth_user=self.u_apple,
state={"state": 1, "status_msg": "Away"})
state={"state": UNAVAILABLE, "status_msg": "Away"})
class PresenceListTestCase(unittest.TestCase):
@ -234,7 +234,7 @@ class PresenceEventStreamTestCase(unittest.TestCase):
# I'll already get my own presence state change
self.assertEquals({"start": "0", "end": "1", "chunk": [
{"type": "m.presence",
"content": {"user_id": "@apple:test", "state": 2}},
"content": {"user_id": "@apple:test", "state": ONLINE}},
]}, response)
self.mock_datastore.set_presence_state.return_value = defer.succeed(
@ -251,5 +251,5 @@ class PresenceEventStreamTestCase(unittest.TestCase):
self.assertEquals(200, code)
self.assertEquals({"start": "1", "end": "2", "chunk": [
{"type": "m.presence",
"content": {"user_id": "@banana:test", "state": 2}},
"content": {"user_id": "@banana:test", "state": ONLINE}},
]}, response)

2
webclient/app.css
View File

@ -106,7 +106,7 @@ h1 {
background-color: #38AF00;
}
.away {
.unavailable {
background-color: #FFCC00;
}

11
webclient/room/room-controller.js
View File

@ -123,17 +123,14 @@ angular.module('RoomController', [])
var member = $scope.members[chunk.content.user_id];
if ("state" in chunk.content) {
var ONLINE = 2;
var AWAY = 1;
var OFFLINE = 0;
if (chunk.content.state === ONLINE) {
if (chunk.content.state === "online") {
member.presenceState = "online";
}
else if (chunk.content.state === OFFLINE) {
else if (chunk.content.state === "offline") {
member.presenceState = "offline";
}
else if (chunk.content.state === AWAY) {
member.presenceState = "away";
else if (chunk.content.state === "unavailable") {
member.presenceState = "unavailable";
}
}

2
webclient/room/room.html
View File

@ -14,7 +14,7 @@
<img class="userAvatarGradient" src="img/gradient.png" width="80" height="24"/>
<div class="userName">{{ info.displayname || name }}</div>
</td>
<td class="userPresence" ng-class="info.presenceState === 'online' ? 'online' : (info.presenceState === 'away' ? 'away' : '')" />
<td class="userPresence" ng-class="info.presenceState === 'online' ? 'online' : (info.presenceState === 'unavailable' ? 'unavailable' : '')" />
</table>
</div>