Commit Graph

39 Commits

Author SHA1 Message Date
Eric Eastwood b70439e95b
Add safe search filter for NSFW rooms (#208)
Fix https://github.com/matrix-org/matrix-public-archive/issues/89
2023-05-03 04:45:33 -05:00
Eric Eastwood 9078abf4f1
Timeout requests and stop processing further (#204)
Fix https://github.com/matrix-org/matrix-public-archive/issues/148
Fix https://github.com/matrix-org/matrix-public-archive/issues/40

 - Apply timeout middleware to all room directory and room routes
 - Stop messing with the response after we timeout. Fix https://github.com/matrix-org/matrix-public-archive/issues/148
    - This also involves cancelling any `async/await` things like requests in the routes so we throw an abort error instead of continuing on. Fix https://github.com/matrix-org/matrix-public-archive/issues/40
 - Also abort the route if we see that the user closed the request before we could respond to them
 - Bumps minimum supported Node.js version to v18 because we're now using the built-in native `fetch` in Node.js vs `node-fetch`. This gives us the custom `signal.reason` that we aborted with instead of a generic `AbortError`.
    - This also means we had to add some instrumentation for `fetch` which uses `undici` under the hood. Settled on some unofficial instrumentation: [`opentelemetry-instrumentation-fetch-node`](https://www.npmjs.com/package/opentelemetry-instrumentation-fetch-node)
2023-05-02 00:39:01 -05:00
Eric Eastwood f0a11139cb
Update Hydrogen SDK dependency `0.26.0-scratch` - 2023-04-26 (#189)
Of note:

 - Adds date headers, https://github.com/vector-im/hydrogen-web/pull/938
 - Add "Copy matrix.to permalink" message action, https://github.com/vector-im/hydrogen-web/pull/921
 - Removes `?v=3.13` query parameters from Inter font references in CSS, https://github.com/vector-im/hydrogen-web/pull/961
    - This is what we need to solve https://github.com/matrix-org/matrix-public-archive/pull/187#discussion_r1177638781
2023-04-26 16:18:34 -05:00
Eric Eastwood 0f26dc94d3
Migrate from `eslint-plugin-node` to `eslint-plugin-n` (#179) 2023-04-25 00:39:59 -05:00
Eric Eastwood 6c38f47a0c
Move escape-string-regexp to a normal dep since we use it on the server (#178) 2023-04-25 00:10:01 -05:00
Eric Eastwood 9c0b6fe85e
Production ready build (#175)
- Rename `public` -> `client` so it doesn't get copied automagically as-is (without hashes which we want for cache busting), https://vitejs.dev/guide/assets.html#the-public-directory
     - We still build the version files to `public/` so their copied as-is and Vite handles it for us (so we can use `emptyOutDir`) 
 - Use a multiple entrypoint `.js` Vite build so things can be more intelligently bundled and take less time
     - We aren't using library mode because it doesn't minify or bundle assets
 - Using hash asset tags for cache busting. Hash of the file included in the file name
 - We lookup these hashed assets from `manifest.json` that Vite builds (https://vitejs.dev/guide/backend-integration.html) to serve and preload
 - In terms of optimized bundles, I know the current output isn't great now but will have to opt to fix that up separately in the future. Tracked by https://github.com/matrix-org/matrix-public-archive/issues/176
2023-04-24 23:50:53 -05:00
Eric Eastwood 551b4e72d1
Follow tombstone and predecessor history (#167)
Fix https://github.com/matrix-org/matrix-public-archive/issues/59

Other updates:

 - Update tests to use `/roomid/room1/date/2022/01/03` format instead of trying to retrofit the weird alias stuff on there. Which also makes the fancy to actual URL utilities much more simple.
 - Update to specify `archiveMessageLimit` in the test case because pages have different number of events depending on if we are against a boundary, hidden events, etc.
2023-04-19 01:26:15 -05:00
Eric Eastwood 5aafa01f38
Update linting so we don't need omit the context filling name of some variables (#165)
`"no-unused-vars": ["error", { "destructuredArrayIgnorePattern": "^_" }],` was only [introduced in `eslint@8.11.0`](0fd6bb213a/CHANGELOG.md) so we had to update
2023-04-07 13:01:20 -05:00
Michael[tm] Smith 2999691eea
Enable CORS support (#147)
This change enables CORS support in the archive — to allow web developers to create web applications with frontend JavaScript code that can fetch pages from the archive (for example, for scraping content from chat logs).

Otherwise, without this change, web developers can’t create web apps with frontend JavaScript that can fetch chat logs from the archive and then consume the content of the logs.

It’s imaginable that web developers may find use cases for consuming the chat logs in the archive from frontend JavaScript code — at the simplest level, web apps that fetch and scrape logs to get data out of them or to pull out particular snippets from the logs.

Developers can anyway already scrape the contents of the archive — by using server-side programming languages or by using `curl` or whatever from the command line. They just can’t do the same from frontend JavaScript code, unless CORS support is enabled.
2022-11-28 21:47:57 -06:00
Eric Eastwood b41d15a92c
Add `Copy permalink` option to messages (`matrix.to`) (#136)
This might be a little ambiguous since we'll have permalinks within the Matrix Public Archive with `?at=$xxx` (related https://github.com/matrix-org/matrix-public-archive/issues/137) which is different from `matrix.to` permalinks 🤔

But will just go with it for now 
2022-11-14 17:55:00 -06:00
Eric Eastwood 776347207e
Updates to be compatible with the latest Hydrogen changes (rename `urlCreator` -> `urlRouter`) (#134)
Hydrogen changes:

 - https://github.com/vector-im/hydrogen-web/pull/918
2022-11-10 17:37:08 -06:00
Eric Eastwood 3d604a1d87
Update Hydrogen SDK dependency 0.23.0-scratch (#131) 2022-11-09 19:07:30 -06:00
Eric Eastwood 5bae040d72
Add permalink highlighting - `?at=$xxx` (#123)
ex.
```
http://localhost:3050/r/test-room1:my.synapse.server/date/2022/07/28?at=$O9fDg42JoSh9VdcL820FDUS0Gi_CkPt_hrN1pK1fPEY
````

Fix https://github.com/matrix-org/matrix-public-archive/issues/4

Split out from https://github.com/matrix-org/matrix-public-archive/pull/121
2022-11-03 04:49:07 -05:00
Eric Eastwood 08254cbb49
Add a way to jump forwards and backwards to more activity in the room (seamless navigation) (#114)
Fix https://github.com/matrix-org/matrix-public-archive/issues/46
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/71

Summary:

 - Changes the "Jump to next activity in room" to actually continue you to the next 100 messages ahead. Previously, it only jumped you to the single next event in the room which meant a lot of backwards overlap each time.
    - Jumping this direction will also start your scroll position at the top of the timeline to continue reading seamlessly `?continue=top`
 - Adds "Jump to previous activity in room" to the top of the timeline to continue reading the previous part of the conversation.

[1]: There is a caveat with seamless here which is also commented on in the code:

> XXX: This is flawed in the fact that when we go `/messages?dir=b` it could  backfill messages which will fill up the response before we perfectly connect and  continue from the position they were jumping from before. When `/messages?dir=f`  backfills, we won't have this problem anymore because any messages backfilled in  the forwards direction would be picked up the same going backwards.

(need forwards fill MSC)
2022-11-02 04:27:30 -05:00
Eric Eastwood 7a88ea0c19
Add support for room aliases (#107)
Also does friendly redirects if you don't exactly use the right URL pattern.
For example, if you paste the full room ID with the `!` like `/roomid/!foo:bar`,
it will properly redirect you to `/roomid/foo:bar`. It also does this sort of
thing for URL encoded room ID's and aliases.

Fix https://github.com/matrix-org/matrix-public-archive/issues/25
2022-10-27 01:09:13 -05:00
Eric Eastwood 1e89179f09
Page-load with the correct homeserver selected (#98)
Page-load with the correct homeserver selected (according to `?homeserver`).

Fix https://github.com/matrix-org/matrix-public-archive/issues/92

Also makes sure that the `?homeserver` is always available somewhere in the list; whether that be in the available homeserver list or the added homeserver list depending on it someone cleared it out or never had it because they visited from someone else's link.
2022-10-21 02:09:26 -05:00
Eric Eastwood b34c1b817d
Add homeserver selector to room directory landing page (#87)
Opting for the simple solution and using `include_all_networks` instead of needing to fetch the information about the third-party networks.

Fix https://github.com/matrix-org/matrix-public-archive/issues/6 (last piece done with this PR)
2022-10-20 02:06:43 -05:00
Eric Eastwood be837515fe
Show surrounding messages for a full screen of content (#71)
1. Add surrounding messages to the given messages so we have a full screen of content to make it feel lively even in quiet rooms
    - As you scroll around the timeline across different days, the date changes in the URL, calendar, etc
 2. Add summary item to the bottom of the timeline that explains if we couldn't find any messages in the specific day requested 
    - Also allows you to the jump to the next activity in the room. Adds `/:roomId/jump?ts=xxx&dir=[f|b]` to facilitate this.
    - Part of https://github.com/matrix-org/matrix-public-archive/issues/46
 3. Add developer options modal which is linked from the bottom of the right-panel
    - Adds an option so you can debug the `IntersectionObserver` and how it's selecting the active day from the top-edge of the scroll viewport.
    - In the future, this will also include a nice little visualization of the backend timing traces
2022-09-20 16:02:09 -05:00
Eric Eastwood 127d416e6a
Room directory landing page v1 (#61)
Part of https://github.com/matrix-org/matrix-public-archive/issues/6
2022-09-08 01:30:04 -05:00
Eric Eastwood 33d52e8c07
Fix missing `room.observePowerLevels()` throwing error after render (#57)
Fix missing `room.observePowerLevels()` throwing error after render.

```
Uncaught (in promise) TypeError: this._room.observePowerLevels is not a function
    at RoomViewModel2._recreateComposerOnPowerLevelChange (matrix-public-archive.js:23498:53)
    at new RoomViewModel2 (matrix-public-archive.js:23469:14)
    at mountHydrogen (matrix-public-archive.js:27131:25)
    at matrix-public-archive.js:27174:1
```

Doesn't affect main-line rendering but the error is annoying to see. More Hydrogen boilerplate 😞
2022-08-30 23:40:07 -05:00
Eric Eastwood d9c0f66c9d
Use `DisabledComposerView` instead of no composer to add message (#54)
Use `DisabledComposerView` instead of no composer to add message:

> You're viewing an archive of events from 2022-02-08. Use a [Matrix client](https://matrix.to/#/!HBehERstyQBxyJDLfR:my.synapse.server) to start chatting in this room.

Also refactors some of the `views` and `viewmodels` out to their own files
2022-08-30 19:10:57 -05:00
Eric Eastwood b81df10c8e
Use JSON5 for configuration files with comments (#52)
Use JSON5 for configuration files with comments. Now we can leave the available config in `config.default.json` without having to add weird instructions to remove the `xxx`, etc

 - https://www.npmjs.com/package/json5
 - https://www.npmjs.com/package/nconf
 - https://github.com/indexzero/nconf/issues/113#issuecomment-69999413
2022-08-29 20:33:02 -05:00
Eric Eastwood ddfe94beab
OpenTelemetry tracing so we can see spans where the app is taking time (#27)
OpenTelemetry tracing so we can see spans where the app is taking time.
For the user, we specifically show the spans for the external API HTTP requests
that are slow (so we know when the Matrix API is being slow).

Enable tracing:

 - `npm run start -- --tracing`
 - `npm run start-dev -- --tracing`

What does this PR change:

 - Adds OpenTelemetry tracing with some of the automatic instrumentation (includes HTTP and express)
    - We ignore traces for serving static assets (just noise)
 - Adds `X-Trace-Id` to the response headers
 - Adds `window.tracingSpansForRequest` which includes the external HTTP API requests made during the request
 - Adds a fancy 504 timeout page that includes trace details and lists the slow HTTP requests
 - Adds `jaegerTracesEndpoint` configuration to export tracing spans to Jaeger
 - Related to, https://github.com/matrix-org/matrix-public-archive/issues/26
2022-07-14 11:08:50 -05:00
Eric Eastwood d508521171
Use stable test selectors from Hydrogen (`data-testid`)
Follow-up to https://github.com/matrix-org/matrix-public-archive/pull/29#discussion_r909492946

---

Updated Hydrogen to add the consistent `data-testid` attribute selectors,
https://github.com/vector-im/hydrogen-web/pull/773

```
npm install hydrogen-view-sdk@npm:@mlm/hydrogen-view-sdk@0.0.13-scratch
```
2022-07-05 06:23:47 -05:00
Eric Eastwood cda2d98df0
Add testing to CI (#20)
1. Build test homeserver Docker images which can federate with each other
 2. Run end-to-end (e2e) tests

#### Dev notes

Sharing variables across jobs when the `services` field can't access the `env` context, https://github.community/t/how-to-use-env-with-container-image/17252/24
```yaml
env:
  FOO: bar

jobs:
  set_env:
    outputs:
      var: ${{ steps.save_var.outputs.var }}
    steps:
      - id: save_var
         run: echo "::set-output name=var::${{ env.FOO }}"
  actual_job:
    needs: set_env
    container:
      image: ...whatever_you_need_here...${{ needs.set_env.outputs.var }}
```
2022-06-15 00:59:41 -05:00
Eric Eastwood 9fc71a3412
Remove `matrix-bot-sdk` usage in tests (#15)
Remove `matrix-bot-sdk` usage in tests because it didn't have timestamp massaging `?ts` and it's not really necessary to rely on since we can just call the API directly 🤷. `matrix-bot-sdk` is also very annoying having to build rust crypto packages.

We're now using direct `fetch` requests against the Matrix API and lightweight `client` object.

All 3 current tests pass 
2022-06-09 20:44:57 -05:00
Eric Eastwood dff2209e56
Fix nodemon not being able to watch for changes because vite is interfering (#11)
We use `vite@2.9.6` because `vite>=2.9.7` is interfering with `nodemon`, see https://github.com/vitejs/vite/issues/8492

This means that when using `npm run start-dev` -> [`start-dev.js`](40f9d2ea5a/server/start-dev.js), `nodemon` wasn't restarting when a change was made.
2022-06-07 14:56:46 -05:00
Eric Eastwood 40f9d2ea5a
Update to work with latest `hydrogen-view-sdk@0.0.12` (#10)
Get this project running again after a few months of changes
from `hydrogen-view-sdk` and now finally after
https://github.com/vector-im/hydrogen-web/pull/693
merged to make the SDK friendly to locally link and develop on.
2022-06-06 18:58:45 -05:00
Eric Eastwood db6d3797d7 Working e2e test 2022-02-24 03:27:53 -06:00
Eric Eastwood 839e31a35e E2E test but still failing because fetching from start of day before test events happened 2022-02-23 21:25:05 -06:00
Eric Eastwood 0f493a241e Federated homeservers in Docker for e2e tests 2022-02-22 20:25:24 -06:00
Eric Eastwood aea382e4f8 WIP: start of tests 2022-02-22 16:06:29 -06:00
Eric Eastwood 6a5d011a45 Link calendar days 2022-02-16 19:58:32 -06:00
Eric Eastwood e0279e080e Add npm run start-dev to build vite and run server at once 2022-02-15 17:17:14 -06:00
Eric Eastwood 72a6297ae5 Re-mount on client to get JS interactivity 2022-02-15 01:33:16 -06:00
Eric Eastwood a75b12a608 WIP: Add calendar to right panel 2022-02-14 21:11:55 -06:00
Eric Eastwood 82ba92b806 Add custom RightPanel support 2022-02-10 02:21:49 -06:00
Eric Eastwood a49657f9e3 Fetch and render from homeserver 2022-02-07 19:55:11 -06:00
Eric Eastwood 4ab26ef2d1 Strip out PoC pieces (just leave the SSR stuff) 2022-02-05 04:35:02 -06:00