Commit Graph

26 Commits

Author SHA1 Message Date
Eric Eastwood 2dff7ecea5
Refactor `fetchEndpointAsText`/`fetchEndpointAsJson` to return `res` alongside `data` (#122)
Split out of https://github.com/matrix-org/matrix-public-archive/pull/121
where we needed to use `res.url`.
2022-11-03 04:12:00 -05:00
Eric Eastwood 08254cbb49
Add a way to jump forwards and backwards to more activity in the room (seamless navigation) (#114)
Fix https://github.com/matrix-org/matrix-public-archive/issues/46
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/71

Summary:

 - Changes the "Jump to next activity in room" to actually continue you to the next 100 messages ahead. Previously, it only jumped you to the single next event in the room which meant a lot of backwards overlap each time.
    - Jumping this direction will also start your scroll position at the top of the timeline to continue reading seamlessly `?continue=top`
 - Adds "Jump to previous activity in room" to the top of the timeline to continue reading the previous part of the conversation.

[1]: There is a caveat with seamless here which is also commented on in the code:

> XXX: This is flawed in the fact that when we go `/messages?dir=b` it could  backfill messages which will fill up the response before we perfectly connect and  continue from the position they were jumping from before. When `/messages?dir=f`  backfills, we won't have this problem anymore because any messages backfilled in  the forwards direction would be picked up the same going backwards.

(need forwards fill MSC)
2022-11-02 04:27:30 -05:00
Eric Eastwood 7a88ea0c19
Add support for room aliases (#107)
Also does friendly redirects if you don't exactly use the right URL pattern.
For example, if you paste the full room ID with the `!` like `/roomid/!foo:bar`,
it will properly redirect you to `/roomid/foo:bar`. It also does this sort of
thing for URL encoded room ID's and aliases.

Fix https://github.com/matrix-org/matrix-public-archive/issues/25
2022-10-27 01:09:13 -05:00
Eric Eastwood b34c1b817d
Add homeserver selector to room directory landing page (#87)
Opting for the simple solution and using `include_all_networks` instead of needing to fetch the information about the third-party networks.

Fix https://github.com/matrix-org/matrix-public-archive/issues/6 (last piece done with this PR)
2022-10-20 02:06:43 -05:00
Eric Eastwood be837515fe
Show surrounding messages for a full screen of content (#71)
1. Add surrounding messages to the given messages so we have a full screen of content to make it feel lively even in quiet rooms
    - As you scroll around the timeline across different days, the date changes in the URL, calendar, etc
 2. Add summary item to the bottom of the timeline that explains if we couldn't find any messages in the specific day requested 
    - Also allows you to the jump to the next activity in the room. Adds `/:roomId/jump?ts=xxx&dir=[f|b]` to facilitate this.
    - Part of https://github.com/matrix-org/matrix-public-archive/issues/46
 3. Add developer options modal which is linked from the bottom of the right-panel
    - Adds an option so you can debug the `IntersectionObserver` and how it's selecting the active day from the top-edge of the scroll viewport.
    - In the future, this will also include a nice little visualization of the backend timing traces
2022-09-20 16:02:09 -05:00
Eric Eastwood 92668996d7
Add search to room directory landing page (#70)
Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-09-15 20:41:55 -05:00
Eric Eastwood 32c77ecffe
Only show `world_readable` or `public` rooms in the archive. Only index `world_readable` (#66)
Only show `world_readable` or `public` rooms in the archive. Only allow `world_readable` rooms to be indexed by search engines.

Related to https://github.com/matrix-org/matrix-public-archive/issues/47
2022-09-08 19:15:07 -05:00
Eric Eastwood 65a371910a
Redirect to last day with message history (#65)
Redirect to last day with message history: `/:roomIdOrAlias` -> `/:roomIdOrAlias/date/:yyyy/:mm/:dd`

Fix https://github.com/matrix-org/matrix-public-archive/issues/60
2022-09-08 02:18:18 -05:00
Eric Eastwood 127d416e6a
Room directory landing page v1 (#61)
Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-09-08 01:30:04 -05:00
Eric Eastwood f6bd581f77
Better `child_process` error handling v2 - timeouts and actually fail process for error in scope (#62)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/51

Better `child_process` error handling for a couple scenarios with the finger pointing at it 👉

Also make sure we handle all of these scenarios:

 1. Child process fork script throws an `uncaughtException` or `unhandledRejection`
    - These are captured and serialized back to the parent and stored in `childErrors` and exposed if we never get a successful rendered HTML response.
 2. Child process fails to startup 
    - Render process is rejected in the `child.on('error', ...` callback
 3. 👉 Child process times out and is aborted
    - Render process is rejected in the `child.on('error', ...` callback and any `childErrors` encountered are logged
 4. 👉 Child process fork script throws an error in scope of in `process.on('message', async (renderOptions) => {`
    - Child exits with code 1 and we reject the render process with the error
 5. Child process exits with code 1 (error)
    - Render process is rejected with any `childError` info
 6. Child process exits with code 0 (success) but never sends back any HTML
    - We have a `returnedData` data check and any child errors encountered are logged
2022-09-02 18:49:45 -05:00
Eric Eastwood b81df10c8e
Use JSON5 for configuration files with comments (#52)
Use JSON5 for configuration files with comments. Now we can leave the available config in `config.default.json` without having to add weird instructions to remove the `xxx`, etc

 - https://www.npmjs.com/package/json5
 - https://www.npmjs.com/package/nconf
 - https://github.com/indexzero/nconf/issues/113#issuecomment-69999413
2022-08-29 20:33:02 -05:00
Eric Eastwood bdaa98e722
Make the `child_process` error catching more robust (`uncaughtException`) (#51)
Split off from https://github.com/matrix-org/matrix-public-archive/pull/43

Listen to `process.on('uncaughtException', ...)` and handle the async errors ourselves so it no longer fails the child process.

And if the process does exit with status code 1 (error), we have those underlying errors serialized and shown.
2022-08-29 19:13:56 -05:00
Eric Eastwood ddfe94beab
OpenTelemetry tracing so we can see spans where the app is taking time (#27)
OpenTelemetry tracing so we can see spans where the app is taking time.
For the user, we specifically show the spans for the external API HTTP requests
that are slow (so we know when the Matrix API is being slow).

Enable tracing:

 - `npm run start -- --tracing`
 - `npm run start-dev -- --tracing`

What does this PR change:

 - Adds OpenTelemetry tracing with some of the automatic instrumentation (includes HTTP and express)
    - We ignore traces for serving static assets (just noise)
 - Adds `X-Trace-Id` to the response headers
 - Adds `window.tracingSpansForRequest` which includes the external HTTP API requests made during the request
 - Adds a fancy 504 timeout page that includes trace details and lists the slow HTTP requests
 - Adds `jaegerTracesEndpoint` configuration to export tracing spans to Jaeger
 - Related to, https://github.com/matrix-org/matrix-public-archive/issues/26
2022-07-14 11:08:50 -05:00
Eric Eastwood 9c8f980e2a Remove random log 2022-07-05 11:21:04 -05:00
Eric Eastwood 17f2c399dd
Expose arguments for `renderHydrogenToString` (pure function) so we can reproduce when error occurs while we render Hydrogen (#35)
`renderHydrogenToString` is a pure function (probably) which means it will give the same output given the same input. This means, that if we give it a certain input and an error occurs, we should be able to reproduce it again if we have the arguments. This PR exposes those arguments in the logged error so we can investigate what's going wrong.

Added so we can investigate https://github.com/matrix-org/matrix-public-archive/issues/34 better and reproduce locally.
2022-07-05 10:30:12 -05:00
Eric Eastwood bd5c14242e
Make sure container is able to start up (#23)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/22
2022-06-15 17:12:44 -05:00
Eric Eastwood 952a3acd4a
Explain how config inherits (#17) 2022-06-09 21:37:07 -05:00
Eric Eastwood 9fc71a3412
Remove `matrix-bot-sdk` usage in tests (#15)
Remove `matrix-bot-sdk` usage in tests because it didn't have timestamp massaging `?ts` and it's not really necessary to rely on since we can just call the API directly 🤷. `matrix-bot-sdk` is also very annoying having to build rust crypto packages.

We're now using direct `fetch` requests against the Matrix API and lightweight `client` object.

All 3 current tests pass 
2022-06-09 20:44:57 -05:00
Eric Eastwood 839e31a35e E2E test but still failing because fetching from start of day before test events happened 2022-02-23 21:25:05 -06:00
Eric Eastwood 0f493a241e Federated homeservers in Docker for e2e tests 2022-02-22 20:25:24 -06:00
Eric Eastwood 6a5d011a45 Link calendar days 2022-02-16 19:58:32 -06:00
Eric Eastwood 166857e0de Fix lints 2022-02-15 21:33:31 -06:00
Eric Eastwood 6c1cf6d46a Stop restarting server when the client bundle updates 2022-02-15 17:30:30 -06:00
Eric Eastwood e0279e080e Add npm run start-dev to build vite and run server at once 2022-02-15 17:17:14 -06:00
Eric Eastwood c0a2a65c2f Add support for the room header (RoomView) 2022-02-10 01:42:02 -06:00
Eric Eastwood 2378ed72c7 Add in date routes 2022-02-09 20:29:18 -06:00