Set `X-Date-Temporal-Context: [past|present|future]` header for easy cache rules:
- Cache `past` things heavily
- Cache `present`/`future` things for 5 minutes
This accomplishes the goal we set out for:
> - We can cache all responses except for the latest UTC day (and anything in the future). ex. `/!aMzLHLvScQCGKDNqCB:gitter.im/date/2022/10/13`
> - For the latest day, we could set the cache expire after 5 minutes or so
>
> *-- [Matrix Public Archive deployment issue](https://github.com/vector-im/sre-internal/issues/2079)*
And this way we don't have to do any fancy date parsing and comparison from the URL which is probably not even possible Cloudflare cache rules.
Who knows why we can't capture these errors via the more conventional `child.on('error', (err) => { })` listener 🤷
### Before
```
RethrownError: Failed to render Hydrogen to string. In order to reproduce, feed in these arguments into `renderHydrogenToString(...)`:
renderHydrogenToString arguments: { ... }
at renderHydrogenToString (server/hydrogen-render/render-hydrogen-to-string.js:58:11)
--- Original Error ---
RethrownError: Child process exited with code 1
at assembleErrorAfterChildExitsWithErrors (server/child-process-runner/run-in-child-process.js:60:29)
--- Original Error ---
No child errors
```
### After
```
RethrownError: Failed to render Hydrogen to string. In order to reproduce, feed in these arguments into `renderHydrogenToString(...)`:
renderHydrogenToString arguments: { ... }
at renderHydrogenToString (server/hydrogen-render/render-hydrogen-to-string.js:58:11)
--- Original Error ---
RethrownError: Child process exited with code 1
at assembleErrorAfterChildExitsWithErrors (server/child-process-runner/run-in-child-process.js:60:29)
--- Original Error ---
No child errors but there might be something in stderr=node:internal/modules/cjs/loader:936
throw err;
^
Error: Cannot find module '../lib/rethrown-error'
Require stack:
- server/child-process-runner/child-fork-script.js
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
at Function.Module._load (node:internal/modules/cjs/loader:778:27)
at Module.require (node:internal/modules/cjs/loader:1005:19)
at require (node:internal/modules/cjs/helpers:102:18)
at Object.<anonymous> (server/child-process-runner/child-fork-script.js:8:23)
at Module._compile (node:internal/modules/cjs/loader:1103:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'server//child-process-runner//child-fork-script.js'
]
}
```
Don't jump calendar around for events which appear in the future compared to the timestamp in the URL.
This sort of situation occurs often for Gitter imported historical rooms where we have a bunch of history that is timestamp massaged intermixed with a bunch of join events which we couldn't timestamp massage at the time.
Fix https://github.com/matrix-org/matrix-public-archive/issues/182
- Like getting all of the dependencies for a given entry point
- And the favicons
Also fix the problem where `server/hydrogen-render/render-page-html.js` was calling `getFaviconAssetUrls()` right away before the client build had a chance to generate `dist/manifest.json` and result in `Error: Cannot find module '../../dist/manifest.json'`
- Rename `public` -> `client` so it doesn't get copied automagically as-is (without hashes which we want for cache busting), https://vitejs.dev/guide/assets.html#the-public-directory
- We still build the version files to `public/` so their copied as-is and Vite handles it for us (so we can use `emptyOutDir`)
- Use a multiple entrypoint `.js` Vite build so things can be more intelligently bundled and take less time
- We aren't using library mode because it doesn't minify or bundle assets
- Using hash asset tags for cache busting. Hash of the file included in the file name
- We lookup these hashed assets from `manifest.json` that Vite builds (https://vitejs.dev/guide/backend-integration.html) to serve and preload
- In terms of optimized bundles, I know the current output isn't great now but will have to opt to fix that up separately in the future. Tracked by https://github.com/matrix-org/matrix-public-archive/issues/176
We already read it once for the `/health-check` endpoint and cached the response but this way we can use `getVersionTags()` everywhere without worrying about it.
Also, it's no longer `async` so we can use it in things like Express route paths and CDN asset tags more easily.
Fix https://github.com/matrix-org/matrix-public-archive/issues/59
Other updates:
- Update tests to use `/roomid/room1/date/2022/01/03` format instead of trying to retrofit the weird alias stuff on there. Which also makes the fancy to actual URL utilities much more simple.
- Update to specify `archiveMessageLimit` in the test case because pages have different number of events depending on if we are against a boundary, hidden events, etc.
It seems like the `build/` directory is ignored in the GitHub file
finder as a sane default for people who put compiled assets there.
`build-scripts/` probably makes more sense anyway
`"no-unused-vars": ["error", { "destructuredArrayIgnorePattern": "^_" }],` was only [introduced in `eslint@8.11.0`](0fd6bb213a/CHANGELOG.md) so we had to update
- Less test bulk
- Single source of truth: there is no mismatch between the comment and the expectations (we already caught a few mistakes in the conversion thanks to this benefit)
- Easier to maintain and update
- Fix https://github.com/matrix-org/matrix-public-archive/issues/7
- A URL with time looks like
- `/r/too-many-messages-on-day:my.synapse.server/date/2022/11/16T23:59`
- Or when more precision is required (seconds): `/r/too-many-messages-on-day:my.synapse.server/date/2022/11/16T23:59:59`
- Add new custom time picker/scrubber (pictured below) with momentum scrubbing
- Native built-in `<input type="time">` for easier picking if you prefer that and accessibility.
- Uses localized time strings
- Design inspired by Thiago Sanchez's *Time Zone Translate* concept, https://dribbble.com/shots/14590546-Time-Zone-Translate
This change enables CORS support in the archive — to allow web developers to create web applications with frontend JavaScript code that can fetch pages from the archive (for example, for scraping content from chat logs).
Otherwise, without this change, web developers can’t create web apps with frontend JavaScript that can fetch chat logs from the archive and then consume the content of the logs.
It’s imaginable that web developers may find use cases for consuming the chat logs in the archive from frontend JavaScript code — at the simplest level, web apps that fetch and scrape logs to get data out of them or to pull out particular snippets from the logs.
Developers can anyway already scrape the contents of the archive — by using server-side programming languages or by using `curl` or whatever from the command line. They just can’t do the same from frontend JavaScript code, unless CORS support is enabled.
Fixes https://github.com/matrix-org/matrix-public-archive/issues/141
Node.js v19 has `crypto` set on the global already, so this change causes `vmContext.global.crypto` to be assigned only if `vmContext.global.crypto` isn’t already defined.
Otherwise, without this change, the room directory fails to render in Node.js v19+, and instead _"TypeError: Cannot set property crypto of `#<Object>` which has only a getter"_ gets thrown.
This might be a little ambiguous since we'll have permalinks within the Matrix Public Archive with `?at=$xxx` (related https://github.com/matrix-org/matrix-public-archive/issues/137) which is different from `matrix.to` permalinks 🤔
But will just go with it for now ⏩
Eric Eastwood