Commit Graph

112 Commits

Author SHA1 Message Date
Eric Eastwood d19c08397a
Fixup room directory fetch error message (#108)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/96
2022-10-24 15:49:23 -05:00
Eric Eastwood 2bda335f1c
Clean up spurious logs when SSR rendering Hydrogen (#106)
**Before:**

```
Child printed something to stdout: Mounting Hydrogen...

Child printed something to stderr: Skipping `addedHomservers` read from LocalStorage since LocalStorage is not available

Child printed something to stdout: Completed mounting Hydrogen: 22.188ms

Child printed something to stdout: 2 uncaughtException TypeError: this.dialogNode.close is not a function
    at ModalView.closeModal (C:\Users\MLM\Documents\GitHub\element\matrix-public-archive\shared\views\ModalView.js:115:21)
    at Timeout._onTimeout (C:\Users\MLM\Documents\GitHub\element\matrix-public-archive\shared\views\ModalView.js:87:18)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
```


**After:**

```
Child printed something to stdout: Mounting Hydrogen...

Child printed something to stderr: Skipping `addedHomservers` read from LocalStorage since LocalStorage is not available

Child printed something to stdout: Completed mounting Hydrogen: 14.2ms
```
2022-10-21 17:27:10 -05:00
Eric Eastwood 7c09ac13f9
Fix form <select> value not being up to date by the time we auto-submit the form on selection (#105)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/102#discussion_r1002190782
2022-10-21 16:24:22 -05:00
Eric Eastwood d2d6c5a580
Link rooms with `?via` query paramter from room directory landing page (#104)
Fix https://github.com/matrix-org/matrix-public-archive/issues/91

So we're always able to join the room from our origin homeserver to the remote room we don't know about yet.
2022-10-21 16:06:11 -05:00
Eric Eastwood 3b4fb52bfe
Fix pagination links not accounting for homeserver selection (#103)
Fix https://github.com/matrix-org/matrix-public-archive/issues/93

Previously caused the `?homeserver` query parameter to be lost when paginating
2022-10-21 15:52:07 -05:00
Eric Eastwood 048f03fff7
Stop using `selectEl.value` as crutch over swapping `selected` attribute (#102)
Originally introduced in:

 - https://github.com/matrix-org/matrix-public-archive/pull/87
 - https://github.com/matrix-org/matrix-public-archive/pull/98
2022-10-21 15:35:11 -05:00
Eric Eastwood dcc2f5dd03
Fix typo in room fetch error section (#100) 2022-10-21 02:15:42 -05:00
Eric Eastwood 1e89179f09
Page-load with the correct homeserver selected (#98)
Page-load with the correct homeserver selected (according to `?homeserver`).

Fix https://github.com/matrix-org/matrix-public-archive/issues/92

Also makes sure that the `?homeserver` is always available somewhere in the list; whether that be in the available homeserver list or the added homeserver list depending on it someone cleared it out or never had it because they visited from someone else's link.
2022-10-21 02:09:26 -05:00
Eric Eastwood 75d4a14845
Fix hash and query parameter ordering and repeating (#99)
This would happen after opening the "Add server" modal

Before:
```
/?search=&homeserver=foo.bar#/add-server?search=&homeserver=foo.bar
```

After:
```
/?search=&homeserver=foo.bar#/add-server
```
2022-10-21 01:14:06 -05:00
Eric Eastwood 6bb88b1ecd
Load room directory and show error message when we're unable to fetch rooms (#96)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/84 to address https://github.com/matrix-org/matrix-public-archive/issues/80

Also explains why we show the details of the error message.

Part of https://github.com/matrix-org/internal-config/issues/1342

Related to https://github.com/matrix-org/matrix-public-archive/issues/97
2022-10-20 22:48:00 -05:00
Eric Eastwood b34c1b817d
Add homeserver selector to room directory landing page (#87)
Opting for the simple solution and using `include_all_networks` instead of needing to fetch the information about the third-party networks.

Fix https://github.com/matrix-org/matrix-public-archive/issues/6 (last piece done with this PR)
2022-10-20 02:06:43 -05:00
Eric Eastwood a0089b0fe4
Add `Content-Security-Policy` (CSP) (#81)
Add `Content-Security-Policy` (CSP) that restricts the page to just what it is expected to do.

This helps limit the damage that can be done by any XSS attack.

Fix https://github.com/matrix-org/internal-config/issues/1341
2022-10-19 12:07:39 -05:00
Eric Eastwood 0962075f8d
Improve performance of scrolling on room directory (animating gradient and shadow header) (#90) 2022-10-19 01:50:16 -05:00
Eric Eastwood 27ed76c7a9
Implement `ModalView` with native `<dialog>` element (#88)
The main benefit we're getting from this refactor is semantics, `Escape` to close modal, and focus lock.

We have one existing modal with the Developer Options.

This refactor is happening so it can be used to add a new homeserver in the homeserver selector on the room directory page, https://github.com/matrix-org/matrix-public-archive/pull/87
2022-10-19 01:04:19 -05:00
Eric Eastwood 5142e508a2
Have full room description readable from tooltip (#85)
The `RoomCard`  clips text two lines before ellipsing. This will make the rest of the description readable if you hover over the description.
2022-10-18 18:38:00 -05:00
Eric Eastwood f9e16964e1
Fix off-by-one calendar months with time zones that are greater than UTC+0 (#86)
Fix off-by-one calendar months with time zones that are greater than UTC+0 (GMT+0).

Fix https://github.com/matrix-org/matrix-public-archive/issues/77

Previously, we would calculate `lastDayOfTheMonthDate` in the local timezone because we were using the vanilla `new Date(year, month, 0)` constructor. For any timezone greater than `UTC+0` (like London UTC+1 or Korea UTC +9), this means that the date is a day-behind when we go back to UTC+0.

**Before:**

```
inputDate Fri, 29 Jul 2022 00:00:00 GMT 1659052800000
lastDayOfTheMonthDate Sat, 30 Jul 2022 23:00:00 GMT 1659222000000
lastDayOfTheMonth 30
```

**After**

```
inputDate Fri, 29 Jul 2022 00:00:00 GMT 1659052800000
lastDayOfTheMonthDate Sun, 31 Jul 2022 00:00:00 GMT 1659225600000
lastDayOfTheMonth 31
```
2022-10-18 18:36:13 -05:00
Eric Eastwood df89750401
Throw more understandable error when we fail to fetch from the homeserver room directory (#84)
Fix https://github.com/matrix-org/matrix-public-archive/issues/80

```
RethrownError: Unable to fetch rooms from room directory (homeserver=http://localhost:8008/)
    searchTerm=, paginationToken=undefined, limit=9
    at matrix-public-archive\server\routes\room-directory-routes.js:55:13
    --- Original Error ---
    Error: HTTP Error Response: 500 Internal Server Error: {"errcode":"M_UNKNOWN","error":"Internal server error"}
        URL=http://localhost:8008/_matrix/client/v3/publicRooms?
        at checkResponseStatus (matrix-public-archive\server\lib\fetch-endpoint.js:21:11)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async fetchEndpoint (matrix-public-archive\server\lib\fetch-endpoint.js:38:3)
        at async fetchEndpointAsJson (matrix-public-archive\server\lib\fetch-endpoint.js:63:15)
        at async fetchPublicRooms (matrix-public-archive\server\lib\matrix-utils\fetch-public-rooms.js:26:26)
        at async matrix-public-archive\server\tracing\trace-utilities.js:31:24
        at async matrix-public-archive\server\routes\room-directory-routes.js:45:62
```
2022-10-18 16:42:33 -05:00
Eric Eastwood b8062b16a2
Fix wrong path to Hydrogen styles on timeout error page (#83)
Regressed in https://github.com/matrix-org/matrix-public-archive/pull/61 where we tried to serve this under `/css/hydrogen-styles.css` but it doesn't work because all of the image and font references in the CSS file expect it to be at the domain root so just reverted back to serving at the root `/`.
2022-10-18 03:42:37 -05:00
Eric Eastwood f796afe55e
Sanity check that we are not leaking the access token to the client (#82)
This isn't spawning from any previous security issue. Just adding an extra check to help ensure we don't ever regress this in the future.

```
AssertionError [ERR_ASSERTION]: We should not be leaking the `config.matrixAccessToken` to the Hydrogen render function because this will reach the client!
    at renderHydrogenToString (matrix-public-archive\server\hydrogen-render\render-hydrogen-to-string.js:24:3)
    at renderHydrogenVmRenderScriptToPageHtml (matrix-public-archive\server\hydrogen-render\render-hydrogen-vm-render-script-to-page-html.js:22:36)
    at matrix-public-archive\server\routes\room-directory-routes.js:53:28
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
```
2022-10-18 02:40:40 -05:00
Eric Eastwood 1d77c721d0
Use rainbow Matrix.org gradient (#75)
Another iteration of the design,
https://www.figma.com/file/lpW5CqaEbPsYX2pmfIhzRo/Matrix-Public-Archive

Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-10-18 01:30:26 -05:00
Eric Eastwood 2581f88495
Fix XSS when blatting `window.matrixPublicArchiveContext` to the page (#79)
Fix https://github.com/matrix-org/internal-config/issues/1335
2022-10-13 14:36:04 -05:00
Eric Eastwood ff315141fd
Add domain to tracing service to distinguish different Matrix public archive instances (#76) 2022-10-11 16:03:33 -05:00
Eric Eastwood b39d7caf31
Add linting to CI (#74)
- Prettier
 - ESLint
2022-09-27 22:21:00 -05:00
Eric Eastwood be837515fe
Show surrounding messages for a full screen of content (#71)
1. Add surrounding messages to the given messages so we have a full screen of content to make it feel lively even in quiet rooms
    - As you scroll around the timeline across different days, the date changes in the URL, calendar, etc
 2. Add summary item to the bottom of the timeline that explains if we couldn't find any messages in the specific day requested 
    - Also allows you to the jump to the next activity in the room. Adds `/:roomId/jump?ts=xxx&dir=[f|b]` to facilitate this.
    - Part of https://github.com/matrix-org/matrix-public-archive/issues/46
 3. Add developer options modal which is linked from the bottom of the right-panel
    - Adds an option so you can debug the `IntersectionObserver` and how it's selecting the active day from the top-edge of the scroll viewport.
    - In the future, this will also include a nice little visualization of the backend timing traces
2022-09-20 16:02:09 -05:00
Eric Eastwood 92668996d7
Add search to room directory landing page (#70)
Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-09-15 20:41:55 -05:00
Eric Eastwood f73246768f
Bigger more clickable buttons on mobile (#69)
Bigger more clickable buttons on mobile. Feels a lot better on a phone. The buttons already have invisible margin in their hitbox but the bigger size makes your thumb less cramped to the edge to click them.

Also reduce the container padding so it feels more balanced in the single column card layout.
2022-09-08 22:32:06 -05:00
Eric Eastwood b7597b2749
Add clickjacking prevention middleware (#68)
Fix https://github.com/matrix-org/matrix-public-archive/issues/67
2022-09-08 19:30:20 -05:00
Eric Eastwood 32c77ecffe
Only show `world_readable` or `public` rooms in the archive. Only index `world_readable` (#66)
Only show `world_readable` or `public` rooms in the archive. Only allow `world_readable` rooms to be indexed by search engines.

Related to https://github.com/matrix-org/matrix-public-archive/issues/47
2022-09-08 19:15:07 -05:00
Eric Eastwood 65a371910a
Redirect to last day with message history (#65)
Redirect to last day with message history: `/:roomIdOrAlias` -> `/:roomIdOrAlias/date/:yyyy/:mm/:dd`

Fix https://github.com/matrix-org/matrix-public-archive/issues/60
2022-09-08 02:18:18 -05:00
Eric Eastwood 127d416e6a
Room directory landing page v1 (#61)
Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-09-08 01:30:04 -05:00
Eric Eastwood 02b86a8405
Render pipeline separation of concerns (#64)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/36

Render pipeline separation of concerns:

 1. Run in `child_process`
 2. Hydrogen render
 
It's now just a generic `child_process` runner that runs the Hydrogen render in it. This eliminates the windy path of the 1-4 steps that was only held together by the file names themselves.
2022-09-02 20:49:06 -05:00
Eric Eastwood f6bd581f77
Better `child_process` error handling v2 - timeouts and actually fail process for error in scope (#62)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/51

Better `child_process` error handling for a couple scenarios with the finger pointing at it 👉

Also make sure we handle all of these scenarios:

 1. Child process fork script throws an `uncaughtException` or `unhandledRejection`
    - These are captured and serialized back to the parent and stored in `childErrors` and exposed if we never get a successful rendered HTML response.
 2. Child process fails to startup 
    - Render process is rejected in the `child.on('error', ...` callback
 3. 👉 Child process times out and is aborted
    - Render process is rejected in the `child.on('error', ...` callback and any `childErrors` encountered are logged
 4. 👉 Child process fork script throws an error in scope of in `process.on('message', async (renderOptions) => {`
    - Child exits with code 1 and we reject the render process with the error
 5. Child process exits with code 1 (error)
    - Render process is rejected with any `childError` info
 6. Child process exits with code 0 (success) but never sends back any HTML
    - We have a `returnedData` data check and any child errors encountered are logged
2022-09-02 18:49:45 -05:00
Eric Eastwood fd00fec6f1
Fix `DisabledComposerView` description being overriden by powerlevel changes (#58)
We added the description in https://github.com/matrix-org/matrix-public-archive/pull/54 but then made a fix to one of the after render errors, https://github.com/matrix-org/matrix-public-archive/pull/57, which exposed that the description is then overriden when the powerlevel changes.

We now just override the getters directly instead of the internal fields which are overwritten internally.

---

This also fixes a warning from Hydrogen in the console:

```
disposable not found, did it leak? {kind: 'disabled', description: Array(3)}
```
2022-08-31 00:16:14 -05:00
Eric Eastwood 33d52e8c07
Fix missing `room.observePowerLevels()` throwing error after render (#57)
Fix missing `room.observePowerLevels()` throwing error after render.

```
Uncaught (in promise) TypeError: this._room.observePowerLevels is not a function
    at RoomViewModel2._recreateComposerOnPowerLevelChange (matrix-public-archive.js:23498:53)
    at new RoomViewModel2 (matrix-public-archive.js:23469:14)
    at mountHydrogen (matrix-public-archive.js:27131:25)
    at matrix-public-archive.js:27174:1
```

Doesn't affect main-line rendering but the error is annoying to see. More Hydrogen boilerplate 😞
2022-08-30 23:40:07 -05:00
Eric Eastwood d9c0f66c9d
Use `DisabledComposerView` instead of no composer to add message (#54)
Use `DisabledComposerView` instead of no composer to add message:

> You're viewing an archive of events from 2022-02-08. Use a [Matrix client](https://matrix.to/#/!HBehERstyQBxyJDLfR:my.synapse.server) to start chatting in this room.

Also refactors some of the `views` and `viewmodels` out to their own files
2022-08-30 19:10:57 -05:00
Eric Eastwood eb5dc23d5d
Make the archive responsive (#53)
This also needs a release of `@mlm/hydrogen-view-sdk` with our latest scratch changes in https://github.com/vector-im/hydrogen-web/pull/653 but we can make the dependency update later.
2022-08-30 18:47:03 -05:00
Eric Eastwood b45c31a597
Better assertion error language (#56) 2022-08-30 18:45:10 -05:00
Eric Eastwood 6568fc7102
Add Apache 2.0 license (#55) 2022-08-30 18:35:36 -05:00
Eric Eastwood b81df10c8e
Use JSON5 for configuration files with comments (#52)
Use JSON5 for configuration files with comments. Now we can leave the available config in `config.default.json` without having to add weird instructions to remove the `xxx`, etc

 - https://www.npmjs.com/package/json5
 - https://www.npmjs.com/package/nconf
 - https://github.com/indexzero/nconf/issues/113#issuecomment-69999413
2022-08-29 20:33:02 -05:00
Eric Eastwood 36925cd603
Add test to make sure the archive doesn't fail when event for event relation is missing and not included in list of provided events (#43)
Add test to make sure the archive doesn't fail when event for event relation is missing and not included in list of provided events. Like if someone is replying to an event that was from long ago out of our range.

In the case of missing relations, Hydrogen does `_loadContextEntryNotInTimeline` because it can't find the event locally which throws an `uncaughtException`. Before https://github.com/matrix-org/matrix-public-archive/pull/51, the `uncaughtException` killed the Hydrogen `child_process` before it could pass back the HTML. Now this PR mainly just adds a test to make sure it works.
```
TypeError: Cannot read properties of undefined (reading 'storeNames')
    at TimelineReader.readById (hydrogen-web\target\lib-build\hydrogen.cjs.js:12483:33)
    at Timeline._getEventFromStorage (hydrogen-web\target\lib-build\hydrogen.cjs.js:12762:46)
    at Timeline._loadContextEntryNotInTimeline (hydrogen-web\target\lib-build\hydrogen.cjs.js:12747:35)
    at Timeline._loadContextEntriesWhereNeeded (hydrogen-web\target\lib-build\hydrogen.cjs.js:12741:14)
    at Timeline.addEntries (hydrogen-web\target\lib-build\hydrogen.cjs.js:12699:10)
    at mountHydrogen (4-hydrogen-vm-render-script.js:204:12)
    at 4-hydrogen-vm-render-script.js:353:1
    at Script.runInContext (node:vm:139:12)
    at _renderHydrogenToStringUnsafe (matrix-public-archive\server\hydrogen-render\3-render-hydrogen-to-string-unsafe.js:102:41)
    at async process.<anonymous> (matrix-public-archive\server\hydrogen-render\2-render-hydrogen-to-string-fork-script.js:18:27)
```
2022-08-29 19:42:18 -05:00
Eric Eastwood bdaa98e722
Make the `child_process` error catching more robust (`uncaughtException`) (#51)
Split off from https://github.com/matrix-org/matrix-public-archive/pull/43

Listen to `process.on('uncaughtException', ...)` and handle the async errors ourselves so it no longer fails the child process.

And if the process does exit with status code 1 (error), we have those underlying errors serialized and shown.
2022-08-29 19:13:56 -05:00
Eric Eastwood e9d13db911
Add test for joining a new federated room (#31)
Add test for joining a new federated room and making sure the messages are available (homeserver should backfill).

Synapse changes: https://github.com/matrix-org/synapse/pull/13205, https://github.com/matrix-org/synapse/pull/13320
2022-08-29 18:56:31 -05:00
Eric Eastwood b5b79b94f2
Manually instrument some archive logic (#44) 2022-08-29 14:13:13 -05:00
Eric Eastwood 27886a92d3
Add available Jaeger port (#48) 2022-08-29 14:08:15 -05:00
Eric Eastwood fba827a44e
Add app screenshot (#42) 2022-07-18 13:29:47 -05:00
Eric Eastwood 07bc094890
Enable tracing by config so we can enable from argv, env variable, or config file (#41) 2022-07-14 11:26:53 -05:00
Eric Eastwood ddfe94beab
OpenTelemetry tracing so we can see spans where the app is taking time (#27)
OpenTelemetry tracing so we can see spans where the app is taking time.
For the user, we specifically show the spans for the external API HTTP requests
that are slow (so we know when the Matrix API is being slow).

Enable tracing:

 - `npm run start -- --tracing`
 - `npm run start-dev -- --tracing`

What does this PR change:

 - Adds OpenTelemetry tracing with some of the automatic instrumentation (includes HTTP and express)
    - We ignore traces for serving static assets (just noise)
 - Adds `X-Trace-Id` to the response headers
 - Adds `window.tracingSpansForRequest` which includes the external HTTP API requests made during the request
 - Adds a fancy 504 timeout page that includes trace details and lists the slow HTTP requests
 - Adds `jaegerTracesEndpoint` configuration to export tracing spans to Jaeger
 - Related to, https://github.com/matrix-org/matrix-public-archive/issues/26
2022-07-14 11:08:50 -05:00
Eric Eastwood 13eb92b067
Make sure we finish sending the HTML payload before we exit the process (#38)
I encountered a page which responded successfully but all of the Hydrogen HTML was missing. It just had the boilerplate around it.

What I am guessing happened is that since `process.send` is async, with a sufficiently large
payload and race condition, `process.exit(0)` was being called before it finished sending.

Related:
 - https://stackoverflow.com/questions/34627546/process-send-is-sync-async-on-nix-windows
 - 56d9584a0e
 - https://github.com/nodejs/node/issues/6767
2022-07-06 19:24:29 -05:00
Eric Eastwood 7eaa103a28
Fix large option payloads throwing E2BIG and ENAMETOOLONG (#37)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/36
2022-07-05 18:00:29 -05:00
Eric Eastwood f738dbc1da
Stop Hydrogen from running in the background after we get our SSR HTML render data (#36)
We now run the Hydrogen render in a `child_process` so we can exit the whole render process. We still use the `vm` to setup the browser-like globals. With a `vm`, everything continues to run even after it returns and there isn't a way to clean up, stop, kill, terminate the vm script or context so we need this extra `child_process` now to clean up. I don't like the complexity necessary for this though. I wish the `vm` API allowed for this use case. The only way to stop a `vm` is the `timeout` and we want to stop as soon as we return.

Fix https://github.com/matrix-org/matrix-public-archive/issues/34
2022-07-05 17:30:52 -05:00