Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor

Conflicts:
	synapse/storage/stream.py

README.rst

@@ -26,8 +26,8 @@ To get up and running:
       with ``python setup.py develop --user`` and then run one with
       ``python synapse/app/homeserver.py``
       
-    - To run your own webclient:
+    - To run your own webclient, add ``-w``:
-      ``cd webclient; python -m SimpleHTTPServer`` and hit http://localhost:8000
+      ``python synapse/app/homeserver.py -w`` and hit http://localhost:8080/matrix/client
       in your web browser (a recent Chrome, Safari or Firefox for now,
       please...)
              
@@ -120,6 +120,10 @@ may need to also run:
     $ sudo apt-get install python-pip
     $ sudo pip install --upgrade setuptools
     
+If you don't have access to github, then you may need to install ``syutil``
+manually by checking it out and running ``python setup.py develop --user`` on it
+too.
+    
 If you get errors about ``sodium.h`` being missing, you may also need to
 manually install a newer PyNaCl via pip as setuptools installs an old one. Or
 you can check PyNaCl out of git directly (https://github.com/pyca/pynacl) and
@@ -189,22 +193,17 @@ Running a Demo Federation of Homeservers
 
 If you want to get up and running quickly with a trio of homeservers in a
 private federation (``localhost:8080``, ``localhost:8081`` and
-``localhost:8082``) which you can then access through the webclient running at http://localhost:8080.  Simply run::
+``localhost:8082``) which you can then access through the webclient running at
+http://localhost:8080. Simply run::
 
     $ demo/start.sh
 
 Running The Demo Web Client
 ===========================
 
-At the present time, the web client is not directly served by the homeserver's
+You can run the web client when you run the homeserver by adding ``-w`` to the
-HTTP server. To serve this in a form the web browser can reach, arrange for the
+command to run ``homeserver.py``. The web client can be accessed via 
-'webclient' sub-directory to be made available by any sort of HTTP server that
+http://localhost:8080/matrix/client
-can serve static files. For example, python's SimpleHTTPServer will suffice::
-
-    $ cd webclient
-    $ python -m SimpleHTTPServer
-
-You can now point your browser at  http://localhost:8000/  to find the client.
 
 If this is the first time you have used the client from that browser (it uses
 HTML5 local storage to remember its config), you will need to log in to your

demo/webserver.py

@@ -2,9 +2,32 @@ import argparse
 import BaseHTTPServer
 import os
 import SimpleHTTPServer
+import cgi, logging
 
 from daemonize import Daemonize
 
+class SimpleHTTPRequestHandlerWithPOST(SimpleHTTPServer.SimpleHTTPRequestHandler):
+    UPLOAD_PATH = "upload"
+
+    """
+    Accept all post request as file upload
+    """
+    def do_POST(self):
+
+        path = os.path.join(self.UPLOAD_PATH, os.path.basename(self.path))
+        length = self.headers['content-length']
+        data = self.rfile.read(int(length))
+
+        with open(path, 'wb') as fh:
+            fh.write(data)
+
+        self.send_response(200)
+        self.send_header('Content-Type', 'application/json')
+        self.end_headers()
+
+        # Return the absolute path of the uploaded file
+        self.wfile.write('{"url":"/%s"}' % path)
+
 
 def setup():
     parser = argparse.ArgumentParser()
@@ -19,7 +42,7 @@ def setup():
 
     httpd = BaseHTTPServer.HTTPServer(
         ('', args.port),
-        SimpleHTTPServer.SimpleHTTPRequestHandler
+        SimpleHTTPRequestHandlerWithPOST
     )
 
     def run():

docs/client-server/specification.rst

@@ -1,6 +1,6 @@
-=========================
+========================
-Synapse Client-Server API
+Matrix Client-Server API
-=========================
+========================
 
 The following specification outlines how a client can send and receive data from 
 a home server.
@@ -262,7 +262,10 @@ the error, but the keys 'error' and 'errcode' will always be present.
 Some standard error codes are below:
 
 M_FORBIDDEN:
-Forbidden access, e.g. bad access token, failed login.
+Forbidden access, e.g. joining a room without permission, failed login.
+
+M_UNKNOWN_TOKEN:
+The access token specified was not recognised.
 
 M_BAD_JSON:
 Request contained valid JSON, but it was malformed in some way, e.g. missing
@@ -411,6 +414,9 @@ The server checks this, finds it is valid, and returns:
 {
   "access_token": "abcdef0123456789"
 }
+The server may optionally return "user_id" to confirm or change the user's ID.
+This is particularly useful if the home server wishes to support localpart entry
+of usernames (e.g. "bob" rather than "@bob:matrix.org").
 
 OAuth2-based
 ------------

docs/server-server/specification.rst

@@ -1,8 +1,8 @@
-============================
+===========================
-Synapse Server-to-Server API
+Matrix Server-to-Server API
-============================
+===========================
 
-A description of the protocol used to communicate between Synapse home servers;
+A description of the protocol used to communicate between Matrix home servers;
 also known as Federation.
 
 
@@ -10,7 +10,7 @@ Overview
 ========
 
 The server-server API is a mechanism by which two home servers can exchange
-Synapse event messages, both as a real-time push of current events, and as a
+Matrix event messages, both as a real-time push of current events, and as a
 historic fetching mechanism to synchronise past history for clients to view. It
 uses HTTP connections between each pair of servers involved as the underlying
 transport. Messages are exchanged between servers in real-time by active pushing
@@ -19,7 +19,7 @@ historic data for the purpose of back-filling scrollback buffers and the like
 can also be performed.
 
 
- { Synapse entities }                            { Synapse entities }
+  { Matrix clients }                              { Matrix clients }
      ^          |                                    ^          |
      |  events  |                                    |  events  |
      |          V                                    |          V
@@ -29,27 +29,53 @@ can also be performed.
  |                  |<--------( HTTP )-----------|                  |
  +------------------+                            +------------------+
 
+There are three main kinds of communication that occur between home servers:
 
-Transactions and PDUs
+ * Queries
-=====================
+   These are single request/response interactions between a given pair of
+   servers, initiated by one side sending an HTTP request to obtain some
+   information, and responded by the other. They are not persisted and contain
+   no long-term significant history. They simply request a snapshot state at the
+   instant the query is made.
 
-The communication between home servers is performed by a bidirectional exchange
+ * EDUs - Ephemeral Data Units
-of messages. These messages are called Transactions, and are encoded as JSON
+   These are notifications of events that are pushed from one home server to
-objects with a dict as the top-level element, passed over HTTP. A Transaction is
+   another. They are not persisted and contain no long-term significant history,
-meaningful only to the pair of home servers that exchanged it; they are not
+   nor does the receiving home server have to reply to them.
-globally-meaningful.
 
-Each transaction has an opaque ID and timestamp (UNIX epoch time in miliseconds)
+ * PDUs - Persisted Data Units
-generated by its origin server, an origin and destination server name, a list of
+   These are notifications of events that are broadcast from one home server to
-"previous IDs", and a list of PDUs - the actual message payload that the
+   any others that are interested in the same "context" (namely, a Room ID).
-Transaction carries.
+   They are persisted to long-term storage and form the record of history for
+   that context.
+
+Where Queries are presented directly across the HTTP connection as GET requests
+to specific URLs, EDUs and PDUs are further wrapped in an envelope called a
+Transaction, which is transferred from the origin to the destination home server
+using a PUT request.
+
+
+Transactions and EDUs/PDUs
+==========================
+
+The transfer of EDUs and PDUs between home servers is performed by an exchange
+of Transaction messages, which are encoded as JSON objects with a dict as the
+top-level element, passed over an HTTP PUT request. A Transaction is meaningful
+only to the pair of home servers that exchanged it; they are not globally-
+meaningful.
+
+Each transaction has an opaque ID and timestamp (UNIX epoch time in
+milliseconds) generated by its origin server, an origin and destination server
+name, a list of "previous IDs", and a list of PDUs - the actual message payload
+that the Transaction carries.
 
  {"transaction_id":"916d630ea616342b42e98a3be0b74113",
   "ts":1404835423000,
   "origin":"red",
   "destination":"blue",
   "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
-  "pdus":[...]}
+  "pdus":[...],
+  "edus":[...]}
 
 The "previous IDs" field will contain a list of previous transaction IDs that
 the origin server has sent to this destination. Its purpose is to act as a
@@ -58,7 +84,9 @@ successfully received that Transaction, or ask for a retransmission if not.
 
 The "pdus" field of a transaction is a list, containing zero or more PDUs.[*]
 Each PDU is itself a dict containing a number of keys, the exact details of
-which will vary depending on the type of PDU.
+which will vary depending on the type of PDU. Similarly, the "edus" field is
+another list containing the EDUs. This key may be entirely absent if there are
+no EDUs to transfer.
 
 (* Normally the PDU list will be non-empty, but the server should cope with
 receiving an "empty" transaction, as this is useful for informing peers of other
@@ -86,7 +114,7 @@ field of a PDU refers to PDUs that any origin server has sent, rather than
 previous IDs that this origin has sent. This list may refer to other PDUs sent
 by the same origin as the current one, or other origins.
 
-Because of the distributed nature of participants in a Synapse conversation, it
+Because of the distributed nature of participants in a Matrix conversation, it
 is impossible to establish a globally-consistent total ordering on the events.
 However, by annotating each outbound PDU at its origin with IDs of other PDUs it
 has received, a partial ordering can be constructed allowing causallity
@@ -112,6 +140,15 @@ so on. This part needs refining. And writing in its own document as the details
 relate to the server/system as a whole, not specifically to server-server
 federation.]]
 
+EDUs, by comparison to PDUs, do not have an ID, a context, or a list of
+"previous" IDs. The only mandatory fields for these are the type, origin and
+destination home server names, and the actual nested content.
+
+ {"edu_type":"m.presence",
+  "origin":"blue",
+  "destination":"orange",
+  "content":...}
+
 
 Protocol URLs
 =============
@@ -179,3 +216,16 @@ To stream events all the events:
   Retrieves all of the transactions later than any version given by the "v"
   arguments. [[TODO(paul): I'm not sure what the "origin" argument does because
   I think at some point in the code it's got swapped around.]]
+
+
+To make a query:
+
+  GET .../query/:query_type
+    Query args: as specified by the individual query types
+
+  Response: JSON encoding of a response object
+
+  Performs a single query request on the receiving home server. The Query Type
+  part of the path specifies the kind of query being made, and its query
+  arguments have a meaning specific to that kind of query. The response is a
+  JSON-encoded object whose meaning also depends on the kind of query.

synapse/api/auth.py

@@ -18,7 +18,7 @@
 from twisted.internet import defer
 
 from synapse.api.constants import Membership
-from synapse.api.errors import AuthError, StoreError
+from synapse.api.errors import AuthError, StoreError, Codes
 from synapse.api.events.room import (RoomTopicEvent, RoomMemberEvent,
                                      MessageEvent, FeedbackEvent)
 
@@ -163,4 +163,5 @@ class Auth(object):
             user_id = yield self.store.get_user_by_token(token=token)
             defer.returnValue(self.hs.parse_userid(user_id))
         except StoreError:
-            raise AuthError(403, "Unrecognised access token.")
+            raise AuthError(403, "Unrecognised access token.",
+                            errcode=Codes.UNKNOWN_TOKEN)

synapse/api/errors.py

@@ -27,6 +27,7 @@ class Codes(object):
     BAD_PAGINATION = "M_BAD_PAGINATION"
     UNKNOWN = "M_UNKNOWN"
     NOT_FOUND = "M_NOT_FOUND"
+    UNKNOWN_TOKEN = "M_UNKNOWN_TOKEN"
 
 
 class CodeMessageException(Exception):
@@ -74,7 +75,10 @@ class AuthError(SynapseError):
 
 class EventStreamError(SynapseError):
     """An error raised when there a problem with the event stream."""
-    pass
+    def __init__(self, *args, **kwargs):
+        if "errcode" not in kwargs:
+            kwargs["errcode"] = Codes.BAD_PAGINATION
+        super(EventStreamError, self).__init__(*args, **kwargs)
 
 
 class LoginError(SynapseError):

synapse/api/notifier.py

@@ -56,6 +56,10 @@ class Notifier(object):
         if (event.type == RoomMemberEvent.TYPE and
                 event.content["membership"] == Membership.INVITE):
             member_list.append(event.target_user_id)
+        # similarly, LEAVEs must be sent to the person leaving
+        if (event.type == RoomMemberEvent.TYPE and
+                event.content["membership"] == Membership.LEAVE):
+            member_list.append(event.target_user_id)
 
         for user_id in member_list:
             if user_id in self.stored_event_listeners:

synapse/api/urls.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+# Copyright 2014 matrix.org
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Contains the URL paths to prefix various aspects of the server with. """
+
+CLIENT_PREFIX = "/matrix/client/api/v1"
+FEDERATION_PREFIX = "/matrix/federation/v1"
+WEB_CLIENT_PREFIX = "/matrix/client"

synapse/app/homeserver.py

@@ -21,8 +21,12 @@ from synapse.server import HomeServer
 from twisted.internet import reactor
 from twisted.enterprise import adbapi
 from twisted.python.log import PythonLoggingObserver
-from synapse.http.server import TwistedHttpServer
+from twisted.web.resource import Resource
+from twisted.web.static import File
+from twisted.web.server import Site
+from synapse.http.server import JsonResource, RootRedirect
 from synapse.http.client import TwistedHttpClient
+from synapse.api.urls import CLIENT_PREFIX, FEDERATION_PREFIX, WEB_CLIENT_PREFIX
 
 from daemonize import Daemonize
 
@@ -36,12 +40,19 @@ logger = logging.getLogger(__name__)
 
 
 class SynapseHomeServer(HomeServer):
-    def build_http_server(self):
-        return TwistedHttpServer()
 
     def build_http_client(self):
         return TwistedHttpClient()
 
+    def build_resource_for_client(self):
+        return JsonResource()
+
+    def build_resource_for_federation(self):
+        return JsonResource()
+
+    def build_resource_for_web_client(self):
+        return File("webclient")  # TODO configurable?
+
     def build_db_pool(self):
         """ Set up all the dbs. Since all the *.sql have IF NOT EXISTS, so we
         don't have to worry about overwriting existing content.
@@ -74,6 +85,98 @@ class SynapseHomeServer(HomeServer):
 
         return pool
 
+    def create_resource_tree(self, web_client, redirect_root_to_web_client):
+        """Create the resource tree for this Home Server.
+
+        This in unduly complicated because Twisted does not support putting
+        child resources more than 1 level deep at a time.
+
+        Args:
+            web_client (bool): True to enable the web client.
+            redirect_root_to_web_client (bool): True to redirect '/' to the
+            location of the web client. This does nothing if web_client is not
+            True.
+        """
+        # list containing (path_str, Resource) e.g:
+        # [ ("/aaa/bbb/cc", Resource1), ("/aaa/dummy", Resource2) ]
+        desired_tree = [
+            (CLIENT_PREFIX, self.get_resource_for_client()),
+            (FEDERATION_PREFIX, self.get_resource_for_federation())
+        ]
+        if web_client:
+            logger.info("Adding the web client.")
+            desired_tree.append((WEB_CLIENT_PREFIX,
+                                self.get_resource_for_web_client()))
+
+        if web_client and redirect_root_to_web_client:
+            self.root_resource = RootRedirect(WEB_CLIENT_PREFIX)
+        else:
+            self.root_resource = Resource()
+
+        # ideally we'd just use getChild and putChild but getChild doesn't work
+        # unless you give it a Request object IN ADDITION to the name :/ So
+        # instead, we'll store a copy of this mapping so we can actually add
+        # extra resources to existing nodes. See self._resource_id for the key.
+        resource_mappings = {}
+        for (full_path, resource) in desired_tree:
+            logging.info("Attaching %s to path %s", resource, full_path)
+            last_resource = self.root_resource
+            for path_seg in full_path.split('/')[1:-1]:
+                if not path_seg in last_resource.listNames():
+                    # resource doesn't exist, so make a "dummy resource"
+                    child_resource = Resource()
+                    last_resource.putChild(path_seg, child_resource)
+                    res_id = self._resource_id(last_resource, path_seg)
+                    resource_mappings[res_id] = child_resource
+                    last_resource = child_resource
+                else:
+                    # we have an existing Resource, use that instead.
+                    res_id = self._resource_id(last_resource, path_seg)
+                    last_resource = resource_mappings[res_id]
+
+            # ===========================
+            # now attach the actual desired resource
+            last_path_seg = full_path.split('/')[-1]
+
+            # if there is already a resource here, thieve its children and
+            # replace it
+            res_id = self._resource_id(last_resource, last_path_seg)
+            if res_id in resource_mappings:
+                # there is a dummy resource at this path already, which needs
+                # to be replaced with the desired resource.
+                existing_dummy_resource = resource_mappings[res_id]
+                for child_name in existing_dummy_resource.listNames():
+                    child_res_id = self._resource_id(existing_dummy_resource,
+                                                     child_name)
+                    child_resource = resource_mappings[child_res_id]
+                    # steal the children
+                    resource.putChild(child_name, child_resource)
+
+            # finally, insert the desired resource in the right place
+            last_resource.putChild(last_path_seg, resource)
+            res_id = self._resource_id(last_resource, last_path_seg)
+            resource_mappings[res_id] = resource
+
+        return self.root_resource
+
+    def _resource_id(self, resource, path_seg):
+        """Construct an arbitrary resource ID so you can retrieve the mapping
+        later.
+
+        If you want to represent resource A putChild resource B with path C,
+        the mapping should looks like _resource_id(A,C) = B.
+
+        Args:
+            resource (Resource): The *parent* Resource
+            path_seg (str): The name of the child Resource to be attached.
+        Returns:
+            str: A unique string which can be a key to the child Resource.
+        """
+        return "%s-%s" % (resource, path_seg)
+
+    def start_listening(self, port):
+        reactor.listenTCP(port, Site(self.root_resource))
+
 
 def setup_logging(verbosity=0, filename=None, config_path=None):
     """ Sets up logging with verbosity levels.
@@ -157,7 +260,10 @@ def setup():
 
     hs.register_servlets()
 
-    hs.get_http_server().start_listening(args.port)
+    hs.create_resource_tree(
+        web_client=args.webclient,
+        redirect_root_to_web_client=True)
+    hs.start_listening(args.port)
 
     hs.build_db_pool()
 

synapse/federation/__init__.py

@@ -23,7 +23,7 @@ from .transport import TransportLayer
 def initialize_http_replication(homeserver):
     transport = TransportLayer(
         homeserver.hostname,
-        server=homeserver.get_http_server(),
+        server=homeserver.get_resource_for_federation(),
         client=homeserver.get_http_client()
     )
 

synapse/federation/transport.py

@@ -23,6 +23,7 @@ over a different (albeit still reliable) protocol.
 
 from twisted.internet import defer
 
+from synapse.api.urls import FEDERATION_PREFIX as PREFIX
 from synapse.util.logutils import log_function
 
 import logging
@@ -33,9 +34,6 @@ import re
 logger = logging.getLogger(__name__)
 
 
-PREFIX = "/matrix/federation/v1"
-
-
 class TransportLayer(object):
     """This is a basic implementation of the transport layer that translates
     transactions and other requests to/from HTTP.

synapse/handlers/directory.py

@@ -20,17 +20,11 @@ from ._base import BaseHandler
 from synapse.api.errors import SynapseError
 
 import logging
-import json
-import urllib
 
 
 logger = logging.getLogger(__name__)
 
 
-# TODO(erikj): This needs to be factored out somewere
-PREFIX = "/matrix/client/api/v1"
-
-
 class DirectoryHandler(BaseHandler):
 
     def __init__(self, hs):

synapse/handlers/login.py

@@ -16,7 +16,7 @@
 from twisted.internet import defer
 
 from ._base import BaseHandler
-from synapse.api.errors import LoginError
+from synapse.api.errors import LoginError, Codes
 
 import bcrypt
 import logging
@@ -51,7 +51,7 @@ class LoginHandler(BaseHandler):
         user_info = yield self.store.get_user_by_id(user_id=user)
         if not user_info:
             logger.warn("Attempted to login as %s but they do not exist.", user)
-            raise LoginError(403, "")
+            raise LoginError(403, "", errcode=Codes.FORBIDDEN)
 
         stored_hash = user_info[0]["password_hash"]
         if bcrypt.checkpw(password, stored_hash):
@@ -62,4 +62,4 @@ class LoginHandler(BaseHandler):
             defer.returnValue(token)
         else:
             logger.warn("Failed password login for user %s", user)
-            raise LoginError(403, "")
+            raise LoginError(403, "", errcode=Codes.FORBIDDEN)

synapse/handlers/presence.py

@@ -177,7 +177,9 @@ class PresenceHandler(BaseHandler):
             state = self._get_or_offline_usercache(target_user).get_state()
 
         if "mtime" in state:
-            state["mtime_age"] = self.clock.time_msec() - state.pop("mtime")
+            state["mtime_age"] = int(
+                self.clock.time_msec() - state.pop("mtime")
+            )
         defer.returnValue(state)
 
     @defer.inlineCallbacks
@@ -367,7 +369,9 @@ class PresenceHandler(BaseHandler):
             p["observed_user"] = observed_user
             p.update(self._get_or_offline_usercache(observed_user).get_state())
             if "mtime" in p:
-                p["mtime_age"] = self.clock.time_msec() - p.pop("mtime")
+                p["mtime_age"] = int(
+                    self.clock.time_msec() - p.pop("mtime")
+                )
 
         defer.returnValue(presence)
 
@@ -560,7 +564,9 @@ class PresenceHandler(BaseHandler):
 
         if "mtime" in state:
             state = dict(state)
-            state["mtime_age"] = self.clock.time_msec() - state.pop("mtime")
+            state["mtime_age"] = int(
+                self.clock.time_msec() - state.pop("mtime")
+            )
 
         yield self.federation.send_edu(
             destination=destination,
@@ -598,7 +604,9 @@ class PresenceHandler(BaseHandler):
             del state["user_id"]
 
             if "mtime_age" in state:
-                state["mtime"] = self.clock.time_msec() - state.pop("mtime_age")
+                state["mtime"] = int(
+                    self.clock.time_msec() - state.pop("mtime_age")
+                )
 
             statuscache = self._get_or_make_usercache(user)
 
@@ -720,6 +728,8 @@ class UserPresenceCache(object):
         content["user_id"] = user.to_string()
 
         if "mtime" in content:
-            content["mtime_age"] = clock.time_msec() - content.pop("mtime")
+            content["mtime_age"] = int(
+                clock.time_msec() - content.pop("mtime")
+            )
 
         return {"type": "m.presence", "content": content}

synapse/handlers/room.py

@@ -94,10 +94,10 @@ class MessageHandler(BaseHandler):
                 event.room_id
             )
 
-            yield self.hs.get_federation().handle_new_event(event)
-
             self.notifier.on_new_room_event(event, store_id)
 
+        yield self.hs.get_federation().handle_new_event(event)
+
     @defer.inlineCallbacks
     def get_messages(self, user_id=None, room_id=None, pagin_config=None,
                      feedback=False):

synapse/http/server.py

@@ -22,6 +22,7 @@ from synapse.api.errors import cs_exception, CodeMessageException
 from twisted.internet import defer, reactor
 from twisted.web import server, resource
 from twisted.web.server import NOT_DONE_YET
+from twisted.web.util import redirectTo
 
 import collections
 import logging
@@ -52,10 +53,9 @@ class HttpServer(object):
         pass
 
 
-# The actual HTTP server impl, using twisted http server
+class JsonResource(HttpServer, resource.Resource):
-class TwistedHttpServer(HttpServer, resource.Resource):
+    """ This implements the HttpServer interface and provides JSON support for
-    """ This wraps the twisted HTTP server, and triggers the correct callbacks
+    Resources.
-    on the transport_layer.
 
     Register callbacks via register_path()
     """
@@ -160,6 +160,22 @@ class TwistedHttpServer(HttpServer, resource.Resource):
         return False
 
 
+class RootRedirect(resource.Resource):
+    """Redirects the root '/' path to another path."""
+
+    def __init__(self, path):
+        resource.Resource.__init__(self)
+        self.url = path
+
+    def render_GET(self, request):
+        return redirectTo(self.url, request)
+
+    def getChild(self, name, request):
+        if len(name) == 0:
+            return self  # select ourselves as the child to render
+        return resource.Resource.getChild(self, name, request)
+
+
 def respond_with_json_bytes(request, code, json_bytes, send_cors=False):
     """Sends encoded JSON in response to the given request.
 

synapse/rest/__init__.py

@@ -15,8 +15,7 @@
 
 
 from . import (
-    room, events, register, login, profile, public, presence, im, directory,
+    room, events, register, login, profile, public, presence, im, directory
-    webclient
 )
 
 
@@ -32,19 +31,15 @@ class RestServletFactory(object):
     """
 
     def __init__(self, hs):
-        http_server = hs.get_http_server()
+        client_resource = hs.get_resource_for_client()
 
         # TODO(erikj): There *must* be a better way of doing this.
-        room.register_servlets(hs, http_server)
+        room.register_servlets(hs, client_resource)
-        events.register_servlets(hs, http_server)
+        events.register_servlets(hs, client_resource)
-        register.register_servlets(hs, http_server)
+        register.register_servlets(hs, client_resource)
-        login.register_servlets(hs, http_server)
+        login.register_servlets(hs, client_resource)
-        profile.register_servlets(hs, http_server)
+        profile.register_servlets(hs, client_resource)
-        public.register_servlets(hs, http_server)
+        public.register_servlets(hs, client_resource)
-        presence.register_servlets(hs, http_server)
+        presence.register_servlets(hs, client_resource)
-        im.register_servlets(hs, http_server)
+        im.register_servlets(hs, client_resource)
-        directory.register_servlets(hs, http_server)
+        directory.register_servlets(hs, client_resource)
-
-    def register_web_client(self, hs):
-        http_server = hs.get_http_server()
-        webclient.register_servlets(hs, http_server)

synapse/rest/base.py

@@ -14,6 +14,7 @@
 # limitations under the License.
 
 """ This module contains base REST classes for constructing REST servlets. """
+from synapse.api.urls import CLIENT_PREFIX
 import re
 
 
@@ -27,7 +28,7 @@ def client_path_pattern(path_regex):
     Returns:
         SRE_Pattern
     """
-    return re.compile("^/matrix/client/api/v1" + path_regex)
+    return re.compile("^" + CLIENT_PREFIX + path_regex)
 
 
 class RestServlet(object):

synapse/rest/login.py

@@ -16,6 +16,7 @@
 from twisted.internet import defer
 
 from synapse.api.errors import SynapseError
+from synapse.types import UserID
 from base import RestServlet, client_path_pattern
 
 import json
@@ -45,12 +46,17 @@ class LoginRestServlet(RestServlet):
 
     @defer.inlineCallbacks
     def do_password_login(self, login_submission):
+        if not login_submission["user"].startswith('@'):
+            login_submission["user"] = UserID.create_local(
+                login_submission["user"], self.hs).to_string()
+
         handler = self.handlers.login_handler
         token = yield handler.login(
             user=login_submission["user"],
             password=login_submission["password"])
 
         result = {
+            "user_id": login_submission["user"],  # may have changed
             "access_token": token,
             "home_server": self.hs.hostname,
         }

synapse/rest/webclient.py

@@ -1,45 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from synapse.rest.base import RestServlet
-
-import logging
-import re
-
-logger = logging.getLogger(__name__)
-
-
-class WebClientRestServlet(RestServlet):
-    # No PATTERN; we have custom dispatch rules here
-
-    def register(self, http_server):
-        http_server.register_path("GET",
-                                  re.compile("^/$"),
-                                  self.on_GET_redirect)
-        http_server.register_path("GET",
-                                  re.compile("^/matrix/client$"),
-                                  self.on_GET)
-
-    def on_GET(self, request):
-        return (200, "not implemented")
-
-    def on_GET_redirect(self, request):
-        request.setHeader("Location", request.uri + "matrix/client")
-        return (302, None)
-
-
-def register_servlets(hs, http_server):
-    logger.info("Registering web client.")
-    WebClientRestServlet(hs).register(http_server)

synapse/server.py

@@ -55,7 +55,6 @@ class BaseHomeServer(object):
 
     DEPENDENCIES = [
         'clock',
-        'http_server',
         'http_client',
         'db_pool',
         'persistence_service',
@@ -70,6 +69,9 @@ class BaseHomeServer(object):
         'room_lock_manager',
         'notifier',
         'distributor',
+        'resource_for_client',
+        'resource_for_federation',
+        'resource_for_web_client',
     ]
 
     def __init__(self, hostname, **kwargs):
@@ -135,7 +137,9 @@ class HomeServer(BaseHomeServer):
     required.
 
     It still requires the following to be specified by the caller:
-        http_server
+        resource_for_client
+        resource_for_web_client
+        resource_for_federation
         http_client
         db_pool
     """
@@ -178,9 +182,6 @@ class HomeServer(BaseHomeServer):
 
     def register_servlets(self):
         """ Register all servlets associated with this HomeServer.
-
-        Args:
-            host_web_client (bool): True to host the web client as well.
         """
         # Simply building the ServletFactory is sufficient to have it register
-        factory = self.get_rest_servlet_factory()
+        self.get_rest_servlet_factory()

synapse/state.py

@@ -157,7 +157,10 @@ class StateHandler(object):
             defer.returnValue(True)
             return
 
-        if new_branch[-1] == current_branch[-1]:
+        n = new_branch[-1]
+        c = current_branch[-1]
+
+        if n.pdu_id == c.pdu_id and n.origin == c.origin:
             # We have all the PDUs we need, so we can just do the conflict
             # resolution.
 
@@ -188,10 +191,18 @@ class StateHandler(object):
                 key=lambda x: x.depth
             )
 
+            pdu_id = missing_prev.prev_state_id
+            origin = missing_prev.prev_state_origin
+
+            is_missing = yield self.store.get_pdu(pdu_id, origin) is None
+
+            if not is_missing:
+                raise Exception("Conflict resolution failed.")
+
             yield self._replication.get_pdu(
                 destination=missing_prev.origin,
-                pdu_origin=missing_prev.prev_state_origin,
+                pdu_origin=origin,
-                pdu_id=missing_prev.prev_state_id,
+                pdu_id=pdu_id,
                 outlier=True
             )
 

synapse/storage/registration.py

@@ -17,7 +17,7 @@ from twisted.internet import defer
 
 from sqlite3 import IntegrityError
 
-from synapse.api.errors import StoreError
+from synapse.api.errors import StoreError, Codes
 
 from ._base import SQLBaseStore
 
@@ -73,7 +73,7 @@ class RegistrationStore(SQLBaseStore):
                         "VALUES (?,?,?)",
                         [user_id, password_hash, now])
         except IntegrityError:
-            raise StoreError(400, "User ID already taken.")
+            raise StoreError(400, "User ID already taken.", errcode=Codes.USER_IN_USE)
 
         # it's possible for this to get a conflict, but only for a single user
         # since tokens are namespaced based on their user ID

synapse/util/lockutils.py

@@ -24,9 +24,10 @@ logger = logging.getLogger(__name__)
 
 class Lock(object):
 
-    def __init__(self, deferred):
+    def __init__(self, deferred, key):
         self._deferred = deferred
         self.released = False
+        self.key = key
 
     def release(self):
         self.released = True
@@ -38,9 +39,10 @@ class Lock(object):
             self.release()
 
     def __enter__(self):
-            return self
+        return self
 
     def __exit__(self, type, value, traceback):
+        logger.debug("Releasing lock for key=%r", self.key)
         self.release()
 
 
@@ -63,6 +65,10 @@ class LockManager(object):
         self._lock_deferreds[key] = new_deferred
 
         if old_deferred:
+            logger.debug("Queueing on lock for key=%r", key)
             yield old_deferred
+            logger.debug("Obtained lock for key=%r", key)
+        else:
+            logger.debug("Entering uncontended lock for key=%r", key)
 
-        defer.returnValue(Lock(new_deferred))
+        defer.returnValue(Lock(new_deferred, key))

tests/federation/test_federation.py

@@ -70,7 +70,7 @@ class FederationTestCase(unittest.TestCase):
         )
         self.clock = MockClock()
         hs = HomeServer("test",
-                http_server=self.mock_http_server,
+                resource_for_federation=self.mock_http_server,
                 http_client=self.mock_http_client,
                 db_pool=None,
                 datastore=self.mock_persistence,

tests/handlers/test_directory.py

@@ -51,7 +51,7 @@ class DirectoryTestCase(unittest.TestCase):
                 "get_association_from_room_alias",
             ]),
             http_client=None,
-            http_server=Mock(),
+            resource_for_federation=Mock(),
             replication_layer=self.mock_federation,
         )
         hs.handlers = DirectoryHandlers(hs)

tests/handlers/test_federation.py

@@ -42,7 +42,7 @@ class FederationTestCase(unittest.TestCase):
                 "persist_event",
                 "store_room",
             ]),
-            http_server=NonCallableMock(),
+            resource_for_federation=NonCallableMock(),
             http_client=NonCallableMock(spec_set=[]),
             notifier=NonCallableMock(spec_set=["on_new_room_event"]),
             handlers=NonCallableMock(spec_set=[

tests/handlers/test_presence.py

@@ -66,7 +66,7 @@ class PresenceStateTestCase(unittest.TestCase):
                     "set_presence_list_accepted",
                 ]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_federation=Mock(),
                 http_client=None,
             )
         hs.handlers = JustPresenceHandlers(hs)
@@ -188,7 +188,7 @@ class PresenceInvitesTestCase(unittest.TestCase):
                     "del_presence_list",
                 ]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_client=Mock(),
                 http_client=None,
                 replication_layer=self.replication
             )
@@ -402,7 +402,7 @@ class PresencePushTestCase(unittest.TestCase):
                     "set_presence_state",
                 ]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_client=Mock(),
                 http_client=None,
                 replication_layer=self.replication,
             )
@@ -727,7 +727,7 @@ class PresencePollingTestCase(unittest.TestCase):
                 db_pool=None,
                 datastore=Mock(spec=[]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_client=Mock(),
                 http_client=None,
                 replication_layer=self.replication,
             )

tests/handlers/test_presencelike.py

@@ -71,7 +71,7 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase):
                     "set_profile_displayname",
                 ]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_federation=Mock(),
                 http_client=None,
                 replication_layer=MockReplication(),
             )

tests/handlers/test_profile.py

@@ -56,7 +56,7 @@ class ProfileTestCase(unittest.TestCase):
                     "set_profile_avatar_url",
                 ]),
                 handlers=None,
-                http_server=Mock(),
+                resource_for_federation=Mock(),
                 replication_layer=self.mock_federation,
             )
         hs.handlers = ProfileHandlers(hs)
@@ -139,7 +139,7 @@ class ProfileTestCase(unittest.TestCase):
         mocked_set = self.datastore.set_profile_avatar_url
         mocked_set.return_value = defer.succeed(())
 
-        yield self.handler.set_avatar_url(self.frank, self.frank, 
+        yield self.handler.set_avatar_url(self.frank, self.frank,
                 "http://my.server/pic.gif")
 
         mocked_set.assert_called_with("1234ABCD", "http://my.server/pic.gif")

tests/handlers/test_room.py

@@ -46,7 +46,7 @@ class RoomMemberHandlerTestCase(unittest.TestCase):
                 "get_room",
                 "store_room",
             ]),
-            http_server=NonCallableMock(),
+            resource_for_federation=NonCallableMock(),
             http_client=NonCallableMock(spec_set=[]),
             notifier=NonCallableMock(spec_set=["on_new_room_event"]),
             handlers=NonCallableMock(spec_set=[
@@ -317,7 +317,6 @@ class RoomCreationTest(unittest.TestCase):
             datastore=NonCallableMock(spec_set=[
                 "store_room",
             ]),
-            http_server=NonCallableMock(),
             http_client=NonCallableMock(spec_set=[]),
             notifier=NonCallableMock(spec_set=["on_new_room_event"]),
             handlers=NonCallableMock(spec_set=[

tests/rest/test_presence.py

@@ -51,7 +51,8 @@ class PresenceStateTestCase(unittest.TestCase):
         hs = HomeServer("test",
             db_pool=None,
             http_client=None,
-            http_server=self.mock_server,
+            resource_for_client=self.mock_server,
+            resource_for_federation=self.mock_server,
         )
 
         def _get_user_by_token(token=None):
@@ -108,7 +109,8 @@ class PresenceListTestCase(unittest.TestCase):
         hs = HomeServer("test",
             db_pool=None,
             http_client=None,
-            http_server=self.mock_server,
+            resource_for_client=self.mock_server,
+            resource_for_federation=self.mock_server
         )
 
         def _get_user_by_token(token=None):
@@ -183,7 +185,8 @@ class PresenceEventStreamTestCase(unittest.TestCase):
         hs = HomeServer("test",
             db_pool=None,
             http_client=None,
-            http_server=self.mock_server,
+            resource_for_client=self.mock_server,
+            resource_for_federation=self.mock_server,
             datastore=Mock(spec=[
                 "set_presence_state",
                 "get_presence_list",

tests/rest/test_profile.py

@@ -43,7 +43,7 @@ class ProfileTestCase(unittest.TestCase):
         hs = HomeServer("test",
             db_pool=None,
             http_client=None,
-            http_server=self.mock_server,
+            resource_for_client=self.mock_server,
             federation=Mock(),
             replication_layer=Mock(),
         )

tests/test_state.py

@@ -37,6 +37,7 @@ class StateTestCase(unittest.TestCase):
             "update_current_state",
             "get_latest_pdus_in_context",
             "get_current_state",
+            "get_pdu",
         ])
         self.replication = Mock(spec=["get_pdu"])
 
@@ -220,6 +221,8 @@ class StateTestCase(unittest.TestCase):
 
         self.replication.get_pdu.side_effect = set_return_tree
 
+        self.persistence.get_pdu.return_value = None
+
         is_new = yield self.state.handle_new_state(new_pdu)
 
         self.assertTrue(is_new)

webclient/app-controller.js

@@ -55,8 +55,14 @@ angular.module('MatrixWebClientController', ['matrixService'])
         
         // And go to the login page
         $location.path("login");
-    };    
+    };
-                          
+
+    // Listen to the event indicating that the access token is no more valid.
+    // In this case, the user needs to log in again.
+    $scope.$on("M_UNKNOWN_TOKEN", function() {
+        console.log("Invalid access token -> log user out");
+        $scope.logout();
+    });
 }]);
 
    

webclient/app.css

@@ -219,6 +219,20 @@ h1 {
     background-color: #fff ! important;
 }
 
+/*** Profile ***/
+
+.profile-avatar {
+    width: 160px;
+    height: 160px;
+    display:table-cell;
+    vertical-align: middle;
+}
+
+.profile-avatar img {
+    max-width: 100%;
+    max-height: 100%;
+}
+
 /******************************/
 
 .header {

webclient/app.js

@@ -23,8 +23,8 @@ var matrixWebClient = angular.module('matrixWebClient', [
     'matrixService'
 ]);
 
-matrixWebClient.config(['$routeProvider',
+matrixWebClient.config(['$routeProvider', '$provide', '$httpProvider',
-    function($routeProvider) {
+    function($routeProvider, $provide, $httpProvider) {
         $routeProvider.
             when('/login', {
                 templateUrl: 'login/login.html',
@@ -41,6 +41,22 @@ matrixWebClient.config(['$routeProvider',
             otherwise({
                 redirectTo: '/rooms'
             });
+            
+        $provide.factory('AccessTokenInterceptor', ['$q', '$rootScope', 
+            function ($q, $rootScope) {
+            return {
+                responseError: function(rejection) {
+                    if (rejection.status === 403 && "data" in rejection && 
+                            "errcode" in rejection.data && 
+                            rejection.data.errcode === "M_UNKNOWN_TOKEN") {
+                        console.log("Got a 403 with an unknown token. Logging out.")
+                        $rootScope.$broadcast("M_UNKNOWN_TOKEN");
+                    }
+                    return $q.reject(rejection);
+                }
+            };
+        }]);
+        $httpProvider.interceptors.push('AccessTokenInterceptor');
     }]);
 
 matrixWebClient.run(['$location', 'matrixService' , function($location, matrixService) {
@@ -75,4 +91,4 @@ matrixWebClient
         return function(text) {
             return $sce.trustAsHtml(text);
         };
-    }]);
+    }]);

webclient/components/fileInput/file-input-directive.js

@@ -0,0 +1,43 @@
+/*
+ Copyright 2014 matrix.org
+ 
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ 
+ http://www.apache.org/licenses/LICENSE-2.0
+ 
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+'use strict';
+
+/*
+ * Transform an element into an image file input button.
+ * Watch to the passed variable change. It will contain the selected HTML5 file object.
+ */
+angular.module('mFileInput', [])
+.directive('mFileInput', function() {
+    return {
+        restrict: 'A',
+        transclude: 'true',
+        template: '<div ng-transclude></div><input ng-hide="true" type="file" accept="image/*"/>',
+        scope: {
+            selectedFile: '=mFileInput'
+        },
+        
+        link: function(scope, element, attrs, ctrl) {
+            element.bind("click", function() {
+                element.find("input")[0].click();
+                element.find("input").bind("change", function(e) {
+                    scope.selectedFile = this.files[0];
+                    scope.$apply();
+                });
+            });
+      }
+    };
+});

webclient/components/fileUpload/file-upload-service.js

@@ -0,0 +1,47 @@
+/*
+ Copyright 2014 matrix.org
+ 
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ 
+ http://www.apache.org/licenses/LICENSE-2.0
+ 
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+'use strict';
+
+/*
+ * Upload an HTML5 file to a server
+ */
+angular.module('mFileUpload', [])
+.service('mFileUpload', ['$http', '$q', function ($http, $q) {
+        
+    /*
+     * Upload an HTML5 file to a server and returned a promise
+     * that will provide the URL of the uploaded file.
+     */
+    this.uploadFile = function(file) {
+        var deferred = $q.defer();
+        
+        // @TODO: This service runs with the do_POST hacky implementation of /synapse/demos/webserver.py.
+        // This is temporary until we have a true file upload service
+        console.log("Uploading " + file.name + "...");
+        $http.post(file.name, file)
+        .success(function(data, status, headers, config) {
+            deferred.resolve(location.origin + data.url);
+            console.log("   -> Successfully uploaded! Available at " + location.origin + data.url);
+        }).
+        error(function(data, status, headers, config) {
+            console.log("   -> Failed to upload"  + file.name);
+            deferred.reject();
+        });
+        
+        return deferred.promise;
+    };
+}]);

webclient/components/matrix/matrix-service.js

@@ -17,7 +17,7 @@ limitations under the License.
 'use strict';
 
 angular.module('matrixService', [])
-.factory('matrixService', ['$http', '$q', function($http, $q) {
+.factory('matrixService', ['$http', '$q', '$rootScope', function($http, $q, $rootScope) {
         
    /* 
     * Permanent storage of user information
@@ -49,28 +49,13 @@ angular.module('matrixService', [])
         if (path.indexOf(prefixPath) !== 0) {
             path = prefixPath + path;
         }
-        // Do not directly return the $http instance but return a promise
+        return $http({
-        // with enriched or cleaned information
-        var deferred = $q.defer();
-        $http({
             method: method,
             url: baseUrl + path,
             params: params,
             data: data,
             headers: headers
         })
-        .success(function(data, status, headers, config) {
-            // @TODO: We could detect a bad access token here and make an automatic logout
-            deferred.resolve(data, status, headers, config);
-        })
-        .error(function(data, status, headers, config) {
-            // Enrich the error callback with an human readable error reason
-            var reason = data.error;
-            if (!data.error) {
-                reason = JSON.stringify(data);
-            }
-            deferred.reject(reason, data, status, headers, config);
-        });
 
         return deferred.promise;
     };
@@ -227,6 +212,17 @@ angular.module('matrixService', [])
             path = path.replace("$room_id", room_id);
             return doRequest("GET", path);
         },
+        
+        paginateBackMessages: function(room_id, from_token, limit) {
+            var path = "/rooms/$room_id/messages/list";
+            path = path.replace("$room_id", room_id);
+            var params = {
+                from: from_token,
+                to: "START",
+                limit: limit
+            };
+            return doRequest("GET", path, params);
+        },
 
         // get a list of public rooms on your home server
         publicRooms: function() {
@@ -301,6 +297,12 @@ angular.module('matrixService', [])
             return doBaseRequest(config.identityServer, "POST", path, {}, data, headers); 
         },
         
+        
+        // 
+        testLogin: function() {
+            
+        },
+        
         /****** Permanent storage of user information ******/
         
         // Returns the current config

webclient/index.html

@@ -14,6 +14,8 @@
     <script src="room/room-controller.js"></script>
     <script src="rooms/rooms-controller.js"></script>
     <script src="components/matrix/matrix-service.js"></script>
+    <script src="components/fileInput/file-input-directive.js"></script>
+    <script src="components/fileUpload/file-upload-service.js"></script>
 </head>
 
 <body>

webclient/login/login-controller.js

@@ -3,8 +3,16 @@ angular.module('LoginController', ['matrixService'])
                                     function($scope, $location, matrixService) {
     'use strict';
     
+    
+    // Assume that this is hosted on the home server, in which case the URL
+    // contains the home server.
+    var hs_url = $location.protocol() + "://" + $location.host();
+    if ($location.port()) {
+        hs_url += ":" + $location.port();
+    }
+    
     $scope.account = {
-        homeserver: "http://localhost:8080",
+        homeserver: hs_url,
         desired_user_name: "",
         user_id: "",
         password: "",
@@ -31,14 +39,13 @@ angular.module('LoginController', ['matrixService'])
         }
 
         matrixService.register($scope.account.desired_user_name, $scope.account.pwd1).then(
-            function(data) {
+            function(response) {
                 $scope.feedback = "Success";
-
                 // Update the current config 
                 var config = matrixService.config();
                 angular.extend(config, {
-                    access_token: data.access_token,
+                    access_token: response.data.access_token,
-                    user_id: data.user_id
+                    user_id: response.data.user_id
                 });
                 matrixService.setConfig(config);
 
@@ -48,8 +55,15 @@ angular.module('LoginController', ['matrixService'])
                  // Go to the user's rooms list page
                 $location.path("rooms");
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failure: " + reason;
+                if (error.data) {
+                    if (error.data.errcode === "M_USER_IN_USE") {
+                        $scope.feedback = "Username already taken.";
+                    }
+                }
+                else if (error.status === 0) {
+                    $scope.feedback = "Unable to talk to the server.";
+                }
             });
     };
 
@@ -61,18 +75,28 @@ angular.module('LoginController', ['matrixService'])
         // try to login
         matrixService.login($scope.account.user_id, $scope.account.password).then(
             function(response) {
-                if ("access_token" in response) {
+                if ("access_token" in response.data) {
                     $scope.feedback = "Login successful.";
                     matrixService.setConfig({
                         homeserver: $scope.account.homeserver,
-                        user_id: $scope.account.user_id,
+                        user_id: response.data.user_id,
-                        access_token: response.access_token
+                        access_token: response.data.access_token
                     });
                     matrixService.saveConfig();
                     $location.path("rooms");
                 }
                 else {
-                    $scope.feedback = "Failed to login: " + JSON.stringify(response);
+                    $scope.feedback = "Failed to login: " + JSON.stringify(response.data);
+                }
+            },
+            function(error) {
+                if (error.data) {
+                    if (error.data.errcode === "M_FORBIDDEN") {
+                        $scope.login_error_msg = "Incorrect username or password.";
+                    }
+                }
+                else if (error.status === 0) {
+                    $scope.login_error_msg = "Unable to talk to the server.";
                 }
             }
         );

webclient/login/login.html

@@ -15,15 +15,16 @@
         <!-- New user registration -->
         <div>
             <br/>
-            <button ng-click="register()" ng-disabled="!account.desired_user_name || !account.homeserver || !account.identityServer || !account.pwd1 || !account.pwd2">Register</button>
+            <button ng-click="register()" ng-disabled="!account.desired_user_name || !account.homeserver || !account.identityServer || !account.pwd1 || !account.pwd2 || account.pwd1 !== account.pwd2">Register</button>
         </div>
     </form>
 
     <h3>Got an account?</h3>
     <form novalidate>
         <!-- Login with an registered user -->
+        <div>{{ login_error_msg }} </div>
         <div>
-            <input id="user_id" size="70" type="text" auto-focus ng-model="account.user_id" placeholder="User ID (ex:@bob:localhost)"/>
+            <input id="user_id" size="70" type="text" auto-focus ng-model="account.user_id" placeholder="User ID (ex:@bob:localhost or bob)"/>
             <br />
             <input id="password" size="70" type="password" ng-model="account.password" placeholder="Password"/><br />
             <br/>

webclient/room/room-controller.js

@@ -18,11 +18,15 @@ angular.module('RoomController', [])
 .controller('RoomController', ['$scope', '$http', '$timeout', '$routeParams', '$location', 'matrixService',
                                function($scope, $http, $timeout, $routeParams, $location, matrixService) {
    'use strict';
+    var MESSAGES_PER_PAGINATION = 10;
     $scope.room_id = $routeParams.room_id;
     $scope.room_alias = matrixService.getRoomIdToAliasMapping($scope.room_id);
     $scope.state = {
         user_id: matrixService.config().user_id,
-        events_from: "START"
+        events_from: "END", // when to start the event stream from.
+        earliest_token: "END", // stores how far back we've paginated.
+        can_paginate: true, // this is toggled off when we run out of items
+        stream_failure: undefined // the response when the stream fails
     };
     $scope.messages = [];
     $scope.members = {};
@@ -30,6 +34,53 @@ angular.module('RoomController', [])
 
     $scope.imageURLToSend = "";
     $scope.userIDToInvite = "";
+    
+    var scrollToBottom = function() {
+        $timeout(function() {
+            var objDiv = document.getElementsByClassName("messageTableWrapper")[0];
+            objDiv.scrollTop = objDiv.scrollHeight;
+        },0);
+    };
+    
+    var parseChunk = function(chunks, appendToStart) {
+        for (var i = 0; i < chunks.length; i++) {
+            var chunk = chunks[i];
+            if (chunk.room_id == $scope.room_id && chunk.type == "m.room.message") {
+                if ("membership_target" in chunk.content) {
+                    chunk.user_id = chunk.content.membership_target;
+                }
+                if (appendToStart) {
+                    $scope.messages.unshift(chunk);
+                }
+                else {
+                    $scope.messages.push(chunk);
+                    scrollToBottom();
+                }
+            }
+            else if (chunk.room_id == $scope.room_id && chunk.type == "m.room.member") {
+                updateMemberList(chunk);
+            }
+            else if (chunk.type === "m.presence") {
+                updatePresence(chunk);
+            }
+        }
+    };
+    
+    var paginate = function(numItems) {
+        matrixService.paginateBackMessages($scope.room_id, $scope.state.earliest_token, numItems).then(
+            function(response) {
+                parseChunk(response.data.chunk, true);
+                $scope.state.earliest_token = response.data.end;
+                if (response.data.chunk.length < MESSAGES_PER_PAGINATION) {
+                    // no more messages to paginate :(
+                    $scope.state.can_paginate = false;
+                }
+            },
+            function(error) {
+                console.log("Failed to paginateBackMessages: " + JSON.stringify(error));
+            }
+        )
+    };
 
     var shortPoll = function() {
         $http.get(matrixService.config().homeserver + matrixService.prefix + "/events", {
@@ -39,30 +90,13 @@ angular.module('RoomController', [])
                 "timeout": 5000
             }})
             .then(function(response) {
+                $scope.state.stream_failure = undefined;
                 console.log("Got response from "+$scope.state.events_from+" to "+response.data.end);
                 $scope.state.events_from = response.data.end;
-
                 $scope.feedback = "";
 
-                for (var i = 0; i < response.data.chunk.length; i++) {
+                parseChunk(response.data.chunk, false);
-                    var chunk = response.data.chunk[i];
+                
-                    if (chunk.room_id == $scope.room_id && chunk.type == "m.room.message") {
-                        if ("membership_target" in chunk.content) {
-                            chunk.user_id = chunk.content.membership_target;
-                        }
-                        $scope.messages.push(chunk);
-                        $timeout(function() {
-                            var objDiv = document.getElementsByClassName("messageTableWrapper")[0];
-                            objDiv.scrollTop = objDiv.scrollHeight;
-                        },0);
-                    }
-                    else if (chunk.room_id == $scope.room_id && chunk.type == "m.room.member") {
-                        updateMemberList(chunk);
-                    }
-                    else if (chunk.type === "m.presence") {
-                        updatePresence(chunk);
-                    }
-                }
                 if ($scope.stopPoll) {
                     console.log("Stopping polling.");
                 }
@@ -70,7 +104,7 @@ angular.module('RoomController', [])
                     $timeout(shortPoll, 0);
                 }
             }, function(response) {
-                $scope.feedback = "Can't stream: " + response.data;
+                $scope.state.stream_failure = response;
 
                 if (response.status == 403) {
                     $scope.stopPoll = true;
@@ -99,8 +133,8 @@ angular.module('RoomController', [])
                     function(response) {
                         var member = $scope.members[chunk.target_user_id];
                         if (member !== undefined) {
-                            console.log("Updated displayname "+chunk.target_user_id+" to " + response.displayname);
+                            console.log("Updated displayname "+chunk.target_user_id+" to " + response.data.displayname);
-                            member.displayname = response.displayname;
+                            member.displayname = response.data.displayname;
                         }
                     }
                 ); 
@@ -108,8 +142,8 @@ angular.module('RoomController', [])
                     function(response) {
                          var member = $scope.members[chunk.target_user_id];
                          if (member !== undefined) {
-                            console.log("Updated image for "+chunk.target_user_id+" to " + response.avatar_url);
+                            console.log("Updated image for "+chunk.target_user_id+" to " + response.data.avatar_url);
-                            member.avatar_url = response.avatar_url;
+                            member.avatar_url = response.data.avatar_url;
                          }
                     }
                 );
@@ -171,8 +205,8 @@ angular.module('RoomController', [])
                 console.log("Sent message");
                 $scope.textInput = "";
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failed to send: " + reason;
+                $scope.feedback = "Failed to send: " + error.data.error;
             });               
     };
 
@@ -183,22 +217,24 @@ angular.module('RoomController', [])
         // Join the room
         matrixService.join($scope.room_id).then(
             function() {
-                console.log("Joined room");
+                console.log("Joined room "+$scope.room_id);
                 // Now start reading from the stream
                 $timeout(shortPoll, 0);
 
                 // Get the current member list
                 matrixService.getMemberList($scope.room_id).then(
                     function(response) {
-                        for (var i = 0; i < response.chunk.length; i++) {
+                        for (var i = 0; i < response.data.chunk.length; i++) {
-                            var chunk = response.chunk[i];
+                            var chunk = response.data.chunk[i];
                             updateMemberList(chunk);
                         }
                     },
-                    function(reason) {
+                    function(error) {
-                        $scope.feedback = "Failed get member list: " + reason;
+                        $scope.feedback = "Failed get member list: " + error.data.error;
                     }
                 );
+                
+                paginate(MESSAGES_PER_PAGINATION);
             },
             function(reason) {
                 $scope.feedback = "Can't join room: " + reason;
@@ -224,8 +260,8 @@ angular.module('RoomController', [])
                 console.log("Left room ");
                 $location.path("rooms");
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failed to leave room: " + reason;
+                $scope.feedback = "Failed to leave room: " + error.data.error;
             });
     };
 
@@ -234,10 +270,14 @@ angular.module('RoomController', [])
             function() {
                 console.log("Image sent");
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failed to send image: " + reason;
+                $scope.feedback = "Failed to send image: " + error.data.error;
             });
     };
+    
+    $scope.loadMoreHistory = function() {
+        paginate(MESSAGES_PER_PAGINATION);
+    };
 
     $scope.$on('$destroy', function(e) {
         console.log("onDestroyed: Stopping poll.");

webclient/room/room.html

@@ -35,7 +35,7 @@
                     <div class="bubble">
                         {{ msg.content.msgtype === "m.emote" ? ("* " + (members[msg.user_id].displayname || msg.user_id) + " " + msg.content.body) : "" }}
                         {{ msg.content.msgtype === "m.text" ? msg.content.body : "" }}
-                        <img class="image" ng-hide='msg.content.msgtype !== "m.image"' src="{{ msg.content.url }}" alt="{{ msg.content.body }}"/>
+                        <img class="image" ng-hide='msg.content.msgtype !== "m.image"' ng-src="{{ msg.content.url }}" alt="{{ msg.content.body }}"/>
                     </div>
                 </td>
                 <td class="rightBlock">
@@ -86,6 +86,10 @@
                     <button ng-click="inviteUser(userIDToInvite)">Invite</button>
             </span>
             <button ng-click="leaveRoom()">Leave</button>
+            <button ng-click="loadMoreHistory()" ng-disabled="!state.can_paginate">Load more history</button>
+            <div ng-hide="!state.stream_failure">
+                {{ state.stream_failure.data.error || "Connection failure" }}
+            </div>
         </div>
     </div>
 

webclient/rooms/rooms-controller.js

@@ -16,9 +16,9 @@ limitations under the License.
 
 'use strict';
 
-angular.module('RoomsController', ['matrixService'])
+angular.module('RoomsController', ['matrixService', 'mFileInput', 'mFileUpload'])
-.controller('RoomsController', ['$scope', '$location', 'matrixService',
+.controller('RoomsController', ['$scope', '$location', 'matrixService', 'mFileUpload',
-                               function($scope, $location, matrixService) {
+                               function($scope, $location, matrixService, mFileUpload) {
                                    
     $scope.rooms = [];
     $scope.public_rooms = [];
@@ -40,7 +40,8 @@ angular.module('RoomsController', ['matrixService'])
 
     $scope.newProfileInfo = {
         name: matrixService.config().displayName,
-        avatar: matrixService.config().avatarUrl
+        avatar: matrixService.config().avatarUrl,
+        avatarFile: undefined
     };
 
     $scope.linkedEmails = {
@@ -74,18 +75,18 @@ angular.module('RoomsController', ['matrixService'])
         // List all rooms joined or been invited to
         $scope.rooms = matrixService.rooms();
         matrixService.rooms().then(
-            function(data) {
+            function(response) {
-                data = assignRoomAliases(data);
+                var data = assignRoomAliases(response.data);
                 $scope.feedback = "Success";
                 $scope.rooms = data;
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failure: " + reason;
+                $scope.feedback = "Failure: " + error.data;
             });
         
         matrixService.publicRooms().then(
-            function(data) {
+            function(response) {
-                $scope.public_rooms = assignRoomAliases(data.chunk);
+                $scope.public_rooms = assignRoomAliases(response.data.chunk);
             }
         );
     };
@@ -100,14 +101,14 @@ angular.module('RoomsController', ['matrixService'])
         matrixService.create(room_id, visibility).then(
             function(response) { 
                 // This room has been created. Refresh the rooms list
-                console.log("Created room " + response.room_alias + " with id: "+
+                console.log("Created room " + response.data.room_alias + " with id: "+
-                response.room_id);
+                response.data.room_id);
                 matrixService.createRoomIdToAliasMapping(
-                    response.room_id, response.room_alias);
+                    response.data.room_id, response.data.room_alias);
                 $scope.refresh();
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Failure: " + reason;
+                $scope.feedback = "Failure: " + error.data;
             });
     };
     
@@ -117,17 +118,17 @@ angular.module('RoomsController', ['matrixService'])
         //$location.path("room/" + room_id);
         matrixService.join(room_id).then(
             function(response) {
-                if (response.hasOwnProperty("room_id")) {
+                if (response.data.hasOwnProperty("room_id")) {
-                    if (response.room_id != room_id) {
+                    if (response.data.room_id != room_id) {
-                        $location.path("room/" + response.room_id);
+                        $location.path("room/" + response.data.room_id);
                         return;
                      }
                 }
 
                 $location.path("room/" + room_id);
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Can't join room: " + reason;
+                $scope.feedback = "Can't join room: " + error.data;
             }
         );
     };
@@ -135,15 +136,15 @@ angular.module('RoomsController', ['matrixService'])
     $scope.joinAlias = function(room_alias) {
         matrixService.joinAlias(room_alias).then(
             function(response) {
-                if (response.hasOwnProperty("room_id")) {
+                if (response.data.hasOwnProperty("room_id")) {
-                    $location.path("room/" + response.room_id);
+                    $location.path("room/" + response.data.room_id);
                     return;
                 } else {
                     // TODO (erikj): Do something here?
                 }
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Can't join room: " + reason;
+                $scope.feedback = "Can't join room: " + error.data;
             }
         );
     };
@@ -157,12 +158,28 @@ angular.module('RoomsController', ['matrixService'])
                 matrixService.setConfig(config);
                 matrixService.saveConfig();
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Can't update display name: " + reason;
+                $scope.feedback = "Can't update display name: " + error.data;
             }
         );
     };
 
+
+    $scope.$watch("newProfileInfo.avatarFile", function(newValue, oldValue) {
+        if ($scope.newProfileInfo.avatarFile) {
+            console.log("Uploading new avatar file...");
+            mFileUpload.uploadFile($scope.newProfileInfo.avatarFile).then(
+                function(url) {
+                    $scope.newProfileInfo.avatar = url;
+                    $scope.setAvatar($scope.newProfileInfo.avatar);
+                },
+                function(error) {
+                    $scope.feedback = "Can't upload image";
+                } 
+            );
+        }
+    });
+
     $scope.setAvatar = function(newUrl) {
         console.log("Updating avatar to "+newUrl);
         matrixService.setProfilePictureUrl(newUrl).then(
@@ -174,8 +191,8 @@ angular.module('RoomsController', ['matrixService'])
                 matrixService.setConfig(config);
                 matrixService.saveConfig();
             },
-            function(reason) {
+            function(error) {
-                $scope.feedback = "Can't update avatar: " + reason;
+                $scope.feedback = "Can't update avatar: " + error.data;
             }
         );
     };
@@ -183,8 +200,8 @@ angular.module('RoomsController', ['matrixService'])
     $scope.linkEmail = function(email) {
         matrixService.linkEmail(email).then(
             function(response) {
-                if (response.success === true) {
+                if (response.data.success === true) {
-                    $scope.linkedEmails.authTokenId = response.tokenId;
+                    $scope.linkedEmails.authTokenId = response.data.tokenId;
                     $scope.emailFeedback = "You have been sent an email.";
                     $scope.linkedEmails.emailBeingAuthed = email;
                 }
@@ -192,8 +209,8 @@ angular.module('RoomsController', ['matrixService'])
                     $scope.emailFeedback = "Failed to send email.";
                 }
             },
-            function(reason) {
+            function(error) {
-                $scope.emailFeedback = "Can't send email: " + reason;
+                $scope.emailFeedback = "Can't send email: " + error.data;
             }
         );
     };
@@ -206,7 +223,7 @@ angular.module('RoomsController', ['matrixService'])
         }
         matrixService.authEmail(matrixService.config().user_id, tokenId, code).then(
             function(response) {
-                if ("success" in response && response.success === false) {
+                if ("success" in response.data && response.data.success === false) {
                     $scope.emailFeedback = "Failed to authenticate email.";
                     return;
                 }

webclient/rooms/rooms.html

@@ -3,18 +3,35 @@
     <div class="page">
     <div class="wrapper">
             
+    <div>
+        <form>
+            <table>
+                <tr>
+                    <td>
+                        <div class="profile-avatar">
+                            <img  ng-src="{{ newProfileInfo.avatar || 'img/default-profile.jpg' }}" m-file-input="newProfileInfo.avatarFile"/>
+                        </div>
+                    </td>
+                    <td>
+                        <button  m-file-input="newProfileInfo.avatarFile">Upload new Avatar</button> 
+                        or use an existing image URL:
+                        <div>
+                            <input size="40" ng-model="newProfileInfo.avatar" ng-enter="setAvatar(newProfileInfo.avatar)" />
+                            <button ng-disabled="!newProfileInfo.avatar" ng-click="setAvatar(newProfileInfo.avatar)">Update Avatar</button>   
+                        </div>
+                    </td>
+                </tr>
+            </table>
+        </form>
+    </div>
+
     <div>
         <form>
             <input size="40" ng-model="newProfileInfo.name" ng-enter="setDisplayName(newProfileInfo.name)" />
             <button ng-disabled="!newProfileInfo.name" ng-click="setDisplayName(newProfileInfo.name)">Update Name</button>    
         </form>
     </div>
-    <div>
+
-        <form>
-            <input size="40" ng-model="newProfileInfo.avatar" ng-enter="setAvatar(newProfileInfo.avatar)" />
-            <button ng-disabled="!newProfileInfo.avatar" ng-click="setAvatar(newProfileInfo.avatar)">Update Avatar</button>    
-        </form>
-    </div>
     <br/>
 
     <div>