Mirrors
/
synapse
mirror of https://github.com/element-hq/synapse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/develop' into room-initial-sync

This commit is contained in:
Paul "LeoNerd" Evans 2014-11-18 15:48:30 +00:00
parent 33d328d967 3553101eb3
commit e9f587ecba
8 changed files with 209 additions and 167 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

173
README.rst
View File

@ -4,9 +4,9 @@ Introduction
Matrix is an ambitious new ecosystem for open federated Instant Messaging and
VoIP. The basics you need to know to get up and running are:
- Chatrooms are distributed and do not exist on any single server. Rooms
can be found using aliases like ``#matrix:matrix.org`` or
``#test:localhost:8008`` or they can be ephemeral.
- Everything in Matrix happens in a room. Rooms are distributed and do not
exist on any single server. Rooms can be located using convenience aliases
like ``#matrix:matrix.org`` or ``#test:localhost:8008``.
- Matrix user IDs look like ``@matthew:matrix.org`` (although in the future
you will normally refer to yourself and others using a 3PID: email
@ -17,56 +17,12 @@ The overall architecture is::
client <----> homeserver <=====================> homeserver <----> client
https://somewhere.org/_matrix https://elsewhere.net/_matrix
WARNING
=======
**Synapse is currently in a state of rapid development, and not all features
are yet functional. Critically, some security features are still in
development, which means Synapse can *not* be considered secure or reliable at
this point.** For instance:
- **SSL Certificates used by server-server federation are not yet validated.**
- **Room permissions are not yet enforced on traffic received via federation.**
- **Homeservers do not yet cryptographically sign their events to avoid
tampering**
- Default configuration provides open signup to the service from the internet
Despite this, we believe Synapse is more than useful as a way for experimenting
and exploring Synapse, and the missing features will land shortly. **Until
then, please do *NOT* use Synapse for any remotely important or secure
communication.**
Quick Start
===========
System requirements:
- POSIX-compliant system (tested on Linux & OSX)
- Python 2.7
To get up and running:
- To simply play with an **existing** homeserver you can
just go straight to http://matrix.org/alpha.
- To run your own **private** homeserver on localhost:8008, generate a basic
config file: ``./synctl start`` will give you instructions on how to do this.
For this purpose, you can use 'localhost' or your hostname as a server name.
Once you've done so, running ``./synctl start`` again will start your private
home server. You will find a webclient running at http://localhost:8008.
Please use a recent Chrome or Firefox for now (or Safari if you don't need
VoIP support).
- To run a **public** homeserver and let it exchange messages with other
homeservers and participate in the global Matrix federation, you must expose
port 8448 to the internet and edit homeserver.yaml to specify server_name
(the public DNS entry for this server) and then run ``synctl start``. If you
changed the server_name, you may need to move the old database
(homeserver.db) out of the way first. Then come join ``#matrix:matrix.org``
and say hi! :)
For more detailed setup instructions, please see further down this document.
``#matrix:matrix.org`` is the official support room for Matrix, and can be
accessed by the web client at http://matrix.org/alpha or via an IRC bridge at
irc://irc.freenode.net/matrix.
Synapse is currently in rapid development, but as of version 0.5 we believe it
is sufficiently stable to be run as an internet-facing service for real usage!
About Matrix
============
@ -76,10 +32,10 @@ which handle:
- Creating and managing fully distributed chat rooms with no
single points of control or failure
- Eventually-consistent cryptographically secure[1] synchronisation of room
- Eventually-consistent cryptographically secure synchronisation of room
state across a global open network of federated servers and services
- Sending and receiving extensible messages in a room with (optional)
end-to-end encryption[2]
end-to-end encryption[1]
- Inviting, joining, leaving, kicking, banning room members
- Managing user accounts (registration, login, logout)
- Using 3rd Party IDs (3PIDs) such as email addresses, phone numbers,
@ -111,25 +67,31 @@ Synapse ships with two basic demo Matrix clients: webclient (a basic group chat
web client demo implemented in AngularJS) and cmdclient (a basic Python
command line utility which lets you easily see what the JSON APIs are up to).
We'd like to invite you to take a look at the Matrix spec, try to run a
homeserver, and join the existing Matrix chatrooms already out there,
experiment with the APIs and the demo clients, and let us know your thoughts at
https://github.com/matrix-org/synapse/issues or at matrix@matrix.org.
Meanwhile, iOS and Android SDKs and clients are currently in development and available from:
Thanks for trying Matrix!
* https://github.com/matrix-org/matrix-ios-sdk
* https://github.com/matrix-org/matrix-android-sdk
[1] Cryptographic signing of messages isn't turned on yet
We'd like to invite you to join #matrix:matrix.org (via http://matrix.org/alpha), run a homeserver, take a look at the Matrix spec at
http://matrix.org/docs/spec, experiment with the APIs and the demo
clients, and report any bugs via http://matrix.org/jira.
[2] End-to-end encryption is currently in development
Thanks for using Matrix!
[1] End-to-end encryption is currently in development
Homeserver Installation
=======================
System requirements:
- POSIX-compliant system (tested on Linux & OSX)
- Python 2.7
Synapse is written in python but some of the libraries is uses are written in
C. So before we can install synapse itself we need a working C compiler and the
header files for python C extensions.
Installing prerequisites on Ubuntu::
Installing prerequisites on Ubuntu or Debian::
$ sudo apt-get install build-essential python2.7-dev libffi-dev \
python-pip python-setuptools
@ -137,8 +99,62 @@ Installing prerequisites on Ubuntu::
Installing prerequisites on Mac OS X::
$ xcode-select --install
To install the synapse homeserver run::
Synapse uses NaCl (http://nacl.cr.yp.to/) for encryption and digital signatures.
$ pip install --user --process-dependency-links https://github.com/matrix-org/synapse/tarball/master
This installs synapse, along with the libraries it uses, into
``$HOME/.local/lib/`` on Linux or ``$HOME/Library/Python/2.7/lib/`` on OSX.
Troubleshooting Installation
----------------------------
Synapse requires pip 1.7 or later, so if your OS provides too old a version and
you get errors about ``error: no such option: --process-dependency-links`` you
may need to manually upgrade it::
$ sudo pip install --upgrade pip
If pip crashes mid-installation for reason (e.g. lost terminal), pip may
refuse to run until you remove the temporary installation directory it
created. To reset the installation::
$ rm -rf /tmp/pip_install_matrix
pip seems to leak *lots* of memory during installation. For instance, a Linux
host with 512MB of RAM may run out of memory whilst installing Twisted. If this
happens, you will have to individually install the dependencies which are
failing, e.g.:
$ pip install --user twisted
Running Your Homeserver
=======================
To actually run your new homeserver, pick a working directory for Synapse to run
(e.g. ``~/.synapse``), and::
$ mkdir ~/.synapse
$ cd ~/.synapse
$ # on Linux
$ ~/.local/bin/synctl start
$ # on OSX
$ ~/Library/Python/2.7/bin/synctl start
Troubleshooting Running
-----------------------
If ``synctl`` fails with ``pkg_resources.DistributionNotFound`` errors you may
need a newer version of setuptools than that provided by your OS.
$ sudo pip install setuptools --upgrade
If synapse fails with ``missing "sodium.h"`` crypto errors, you may need
to manually upgrade PyNaCL, as synapse uses NaCl (http://nacl.cr.yp.to/) for
encryption and digital signatures.
Unfortunately PyNACL currently has a few issues
(https://github.com/pyca/pynacl/issues/53) and
(https://github.com/pyca/pynacl/issues/79) that mean it may not install
@ -151,21 +167,6 @@ fix try re-installing from PyPI or directly from
$ # Install from github
$ pip install --user https://github.com/pyca/pynacl/tarball/master
On OSX, if you encounter ``clang: error: unknown argument: '-mno-fused-madd'``
you will need to ``export CFLAGS=-Qunused-arguments``.
To install the synapse homeserver run::
$ pip install --user --process-dependency-links https://github.com/matrix-org/synapse/tarball/master
This installs synapse, along with the libraries it uses, into
``$HOME/.local/lib/``.
To actually run your new homeserver, pick a working directory for Synapse to run (e.g. ``~/.synapse``), and::
$ mkdir ~/.synapse
$ cd ~/.synapse
$ synctl start
Homeserver Development
======================
@ -200,8 +201,8 @@ This should end with a 'PASSED' result::
Upgrading an existing homeserver
================================
Before upgrading an existing homeserver to a new version, please refer to
UPGRADE.rst for any additional instructions.
IMPORTANT: Before upgrading an existing homeserver to a new version, please
refer to UPGRADE.rst for any additional instructions.
Setting up Federation
@ -230,10 +231,7 @@ For the first form, simply pass the required hostname (of the machine) as the
--generate-config
$ python -m synapse.app.homeserver --config-path homeserver.config
Alternatively, you can run synapse via synctl - running ``synctl start`` to
generate a homeserver.yaml config file, where you can then edit server-name to
specify machine.my.domain.name, and then set the actual server running again
with synctl start.
Alternatively, you can run ``synctl start`` to guide you through the process.
For the second form, first create your SRV record and publish it in DNS. This
needs to be named _matrix._tcp.YOURDOMAIN, and point at at least one hostname
@ -270,6 +268,8 @@ private federation (``localhost:8080``, ``localhost:8081`` and
http://localhost:8080. Simply run::
$ demo/start.sh
This is mainly useful just for development purposes.
Running The Demo Web Client
===========================
@ -328,13 +328,14 @@ time.
Where's the spec?!
==================
For now, please go spelunking in the ``docs/`` directory to find out.
The source of the matrix spec lives at https://github.com/matrix-org/matrix-doc.
A recent HTML snapshot of this lives at http://matrix.org/docs/spec
Building Internal API Documentation
===================================
Before building internal API documentation install spinx and
Before building internal API documentation install sphinx and
sphinxcontrib-napoleon::
$ pip install sphinx

25
synapse/api/auth.py
View File

@ -21,7 +21,7 @@ from synapse.api.constants import Membership, JoinRules
from synapse.api.errors import AuthError, StoreError, Codes, SynapseError
from synapse.api.events.room import (
RoomMemberEvent, RoomPowerLevelsEvent, RoomRedactionEvent,
RoomJoinRulesEvent, RoomCreateEvent,
RoomJoinRulesEvent, RoomCreateEvent, RoomAliasesEvent,
)
from synapse.util.logutils import log_function
from syutil.base64util import encode_base64
@ -63,6 +63,10 @@ class Auth(object):
# FIXME
return True
# FIXME: Temp hack
if event.type == RoomAliasesEvent.TYPE:
return True
if event.type == RoomMemberEvent.TYPE:
allowed = self.is_membership_change_allowed(event)
if allowed:
@ -144,6 +148,17 @@ class Auth(object):
@log_function
def is_membership_change_allowed(self, event):
membership = event.content["membership"]
# Check if this is the room creator joining:
if len(event.prev_events) == 1 and Membership.JOIN == membership:
# Get room creation event:
key = (RoomCreateEvent.TYPE, "", )
create = event.old_state_events.get(key)
if event.prev_events[0][0] == create.event_id:
if create.content["creator"] == event.state_key:
return True
target_user_id = event.state_key
# get info about the caller
@ -159,8 +174,6 @@ class Auth(object):
target_in_room = target and target.membership == Membership.JOIN
membership = event.content["membership"]
key = (RoomJoinRulesEvent.TYPE, "", )
join_rule_event = event.old_state_events.get(key)
if join_rule_event:
@ -255,6 +268,12 @@ class Auth(object):
level = power_level_event.content.get("users", {}).get(user_id)
if not level:
level = power_level_event.content.get("users_default", 0)
else:
key = (RoomCreateEvent.TYPE, "", )
create_event = event.old_state_events.get(key)
if (create_event is not None and
create_event.content["creator"] == user_id):
return 100
return level

61
synapse/app/homeserver.py
View File

@ -184,15 +184,7 @@ class SynapseHomeServer(HomeServer):
logger.info("Synapse now listening on port %d", unsecure_port)
def setup():
config = HomeServerConfig.load_config(
"Synapse Homeserver",
sys.argv[1:],
generate_section="Homeserver"
)
config.setup_logging()
def setup(config, run_http=True):
logger.info("Server hostname: %s", config.server_name)
if re.search(":[0-9]+$", config.server_name):
@ -212,12 +204,13 @@ def setup():
content_addr=config.content_addr,
)
hs.register_servlets()
if run_http:
hs.register_servlets()
hs.create_resource_tree(
web_client=config.webclient,
redirect_root_to_web_client=True,
)
hs.create_resource_tree(
web_client=config.webclient,
redirect_root_to_web_client=True,
)
db_name = hs.get_db_name()
@ -237,11 +230,18 @@ def setup():
f.namespace['hs'] = hs
reactor.listenTCP(config.manhole, f, interface='127.0.0.1')
bind_port = config.bind_port
if config.no_tls:
bind_port = None
hs.start_listening(bind_port, config.unsecure_port)
if run_http:
bind_port = config.bind_port
if config.no_tls:
bind_port = None
hs.start_listening(bind_port, config.unsecure_port)
hs.config = config
return hs
def run(config):
if config.daemonize:
print config.pid_file
daemon = Daemonize(
@ -257,13 +257,26 @@ def setup():
else:
reactor.run()
def run():
with LoggingContext("run"):
reactor.run()
def main():
def main(args, run_http=True):
with LoggingContext("main"):
setup()
config = HomeServerConfig.load_config(
"Synapse Homeserver",
args,
generate_section="Homeserver"
)
config.setup_logging()
hs = setup(config, run_http=run_http)
def r():
run(config)
hs.run = r
return hs
if __name__ == '__main__':
main()
hs = main(sys.argv[1:])
hs.run()

2
synapse/app/synctl.py
View File

@ -56,8 +56,8 @@ def main():
elif action == "stop":
stop()
elif action == "restart":
start()
stop()
start()
else:
sys.stderr.write("Usage: %s [start|stop|restart]\n" % (sys.argv[0],))
sys.exit(1)

20
synapse/handlers/directory.py
View File

@ -21,8 +21,6 @@ from synapse.api.errors import SynapseError
from synapse.api.events.room import RoomAliasesEvent
import logging
import sqlite3
logger = logging.getLogger(__name__)
@ -56,17 +54,11 @@ class DirectoryHandler(BaseHandler):
if not servers:
raise SynapseError(400, "Failed to get server list")
try:
yield self.store.create_room_alias_association(
room_alias,
room_id,
servers
)
except sqlite3.IntegrityError:
defer.returnValue("Already exists")
# TODO: Send the room event.
yield self._update_room_alias_events(user_id, room_id)
yield self.store.create_room_alias_association(
room_alias,
room_id,
servers
)
@defer.inlineCallbacks
def delete_association(self, user_id, room_alias):
@ -136,7 +128,7 @@ class DirectoryHandler(BaseHandler):
})
@defer.inlineCallbacks
def _update_room_alias_events(self, user_id, room_id):
def send_room_alias_update_event(self, user_id, room_id):
aliases = yield self.store.get_aliases_for_room(room_id)
event = self.event_factory.create_event(

67
synapse/handlers/room.py
View File

@ -106,20 +106,37 @@ class RoomCreationHandler(BaseHandler):
if not room_id:
raise StoreError(500, "Couldn't generate a room ID.")
if room_alias:
directory_handler = self.hs.get_handlers().directory_handler
yield directory_handler.create_association(
user_id=user_id,
room_id=room_id,
room_alias=room_alias,
servers=[self.hs.hostname],
)
user = self.hs.parse_userid(user_id)
creation_events = self._create_events_for_new_room(
user, room_id, is_public=is_public
)
room_member_handler = self.hs.get_handlers().room_member_handler
@defer.inlineCallbacks
def handle_event(event):
snapshot = yield self.store.snapshot_room(event)
logger.debug("Event: %s", event)
yield self._on_new_room_event(
event, snapshot, extra_users=[user], suppress_auth=True
)
if event.type == RoomMemberEvent.TYPE:
yield room_member_handler.change_membership(
event,
do_auth=True
)
else:
yield self._on_new_room_event(
event, snapshot, extra_users=[user], suppress_auth=True
)
for event in creation_events:
yield handle_event(event)
@ -146,16 +163,6 @@ class RoomCreationHandler(BaseHandler):
yield handle_event(topic_event)
content = {"membership": Membership.JOIN}
join_event = self.event_factory.create_event(
etype=RoomMemberEvent.TYPE,
state_key=user_id,
room_id=room_id,
user_id=user_id,
membership=Membership.JOIN,
content=content
)
content = {"membership": Membership.INVITE}
for invitee in invite_list:
invite_event = self.event_factory.create_event(
@ -165,35 +172,22 @@ class RoomCreationHandler(BaseHandler):
user_id=user_id,
content=content
)
yield self.hs.get_handlers().room_member_handler.change_membership(
invite_event,
do_auth=False
)
yield self.hs.get_handlers().room_member_handler.change_membership(
join_event,
do_auth=False
)
yield handle_event(invite_event)
result = {"room_id": room_id}
if room_alias:
result["room_alias"] = room_alias.to_string()
directory_handler = self.hs.get_handlers().directory_handler
yield directory_handler.create_association(
user_id=user_id,
room_id=room_id,
room_alias=room_alias,
servers=[self.hs.hostname],
)
directory_handler.send_room_alias_update_event(user_id, room_id)
defer.returnValue(result)
def _create_events_for_new_room(self, creator, room_id, is_public=False):
creator_id = creator.to_string()
event_keys = {
"room_id": room_id,
"user_id": creator.to_string(),
"user_id": creator_id,
}
def create(etype, **content):
@ -208,6 +202,16 @@ class RoomCreationHandler(BaseHandler):
creator=creator.to_string(),
)
join_event = self.event_factory.create_event(
etype=RoomMemberEvent.TYPE,
state_key=creator_id,
content={
"membership": Membership.JOIN,
},
**event_keys
)
power_levels_event = self.event_factory.create_event(
etype=RoomPowerLevelsEvent.TYPE,
content={
@ -236,6 +240,7 @@ class RoomCreationHandler(BaseHandler):
return [
creation_event,
join_event,
power_levels_event,
join_rules_event,
]

4
synapse/rest/directory.py
View File

@ -70,9 +70,11 @@ class ClientDirectoryServer(RestServlet):
dir_handler = self.handlers.directory_handler
try:
user_id = user.to_string()
yield dir_handler.create_association(
user.to_string(), room_alias, room_id, servers
user_id, room_alias, room_id, servers
)
yield dir_handler.send_room_alias_update_event(user_id, room_id)
except SynapseError as e:
raise e
except:

24
synapse/storage/directory.py
View File

@ -14,10 +14,15 @@
# limitations under the License.
from ._base import SQLBaseStore
from synapse.api.errors import SynapseError
from twisted.internet import defer
from collections import namedtuple
import sqlite3
RoomAliasMapping = namedtuple(
"RoomAliasMapping",
@ -75,13 +80,18 @@ class DirectoryStore(SQLBaseStore):
Returns:
Deferred
"""
yield self._simple_insert(
"room_aliases",
{
"room_alias": room_alias.to_string(),
"room_id": room_id,
},
)
try:
yield self._simple_insert(
"room_aliases",
{
"room_alias": room_alias.to_string(),
"room_id": room_id,
},
)
except sqlite3.IntegrityError:
raise SynapseError(
409, "Room alias %s already exists" % room_alias.to_string()
)
for server in servers:
# TODO(erikj): Fix this to bulk insert