Previously was seeing some failures like this locally
```
1 failing
1) matrix-viewer
Matrix Viewer
Room directory
pagination is seamless:
AssertionError [ERR_ASSERTION]: Make sure we saw all visible rooms paginating through the directory
+ expected - actual
"planet-1689366398300-room-29"
"planet-1689366398300-room-31"
"planet-1689366398300-room-32"
"planet-1689366398300-room-34"
- "planet-1689366398300-room-34"
"planet-1689366398300-room-35"
"planet-1689366398300-room-37"
"planet-1689366398300-room-38"
"planet-1689366398300-room-4"
at Context.<anonymous> (test/e2e-tests.js:2835:16)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
Happens to address part of https://github.com/matrix-org/matrix-public-archive/issues/271
but made primarily as a follow-up to https://github.com/matrix-org/matrix-public-archive/pull/239
---
Only 42% rooms on the `matrix.org` room directory are `world_readable` which means we will get pages of rooms that are half-empty most of the time if we just naively fetch 9 rooms at a time.
Ideally, we would be able to just add a filter directly to `/publicRooms` in order to only grab the `world_readable` rooms and still get full pages but the filter option doesn't allow us to slice by `world_readable` history visibility.
Instead, we have to paginate until we get a full grid of 9 rooms, then make a final `/publicRooms` request to backtrack to the exact continuation point so next page won't skip any rooms in between.
---
We had empty spaces in the grid before because some rooms in the room directory are private which we filtered out before. But that was a much more rare experience since only 2% of rooms were private .
I'm not sure what exactly is causing the behavior change now besides that I am running on Linux.
The quote fix around the path is from https://stackoverflow.com/questions/51021751/express-js-lint-gives-mistake
`node_modules/` is already part of our `.eslintignore`.
Previously:
```sh
$ npm run lint
> matrix-public-archive@0.1.0 lint
> eslint **/*.js
Oops! Something went wrong! :(
ESLint: 8.37.0
You are linting "node_modules/ipaddr.js", but all of the files matching the glob pattern "node_modules/ipaddr.js" are ignored.
If you don't want to lint these files, remove the pattern "node_modules/ipaddr.js" from the list of arguments passed to ESLint.
If you do want to lint these files, try the following solutions:
* Check your .eslintignore file, or the eslintIgnore property in package.json, to ensure that the files are not configured to be ignored.
* Explicitly list the files from this glob that you'd like to lint on the command-line, rather than providing a glob as an argument.
```
Only `world_readable` can be considered as opting into having history publicly on the web. Anything else must not be archived until there's a dedicated state event for opting into archiving.
The history visibility in Libera rooms is set to `join` which means it's not
accessible in the archive at all. Instead of leading a bunch of people to
`403 Forbidden`, we can just remove it from the default list.
The default list was mostly just copied from the Element list of defaults.
Room cards will now sort by room members descending (highest to lowest) as expected.
Fix https://github.com/matrix-org/matrix-public-archive/issues/218
The `/publicRooms` (room directory) endpoint already returns rooms in the correct order which is why we didn't care about the order before but the different `[].sort(...)` implementations in browsers necessitates we be explicit about it. Ideally, we wouldn't have to use the `ObservableMap.sortValues()` method at all but it seems like one of the only ways to get the values out. In any case, maybe it's more clear what order things are in now.
This bug stems from the fact that `[1, 2, 3, 4, 5].sort((a, b) => 1)` returns different results in Chrome vs Firefox (found from https://stackoverflow.com/questions/55039157/array-sort-behaves-differently-in-firefox-and-chrome-edge)
- Chrome: `[1, 2, 3, 4, 5].sort((a, b) => 1)` -> `[1, 2, 3, 4, 5]` ✅
- Firefox: `[1, 2, 3, 4, 5].sort((a, b) => 1)` -> `[5, 4, 3, 2, 1]` ❌
Useful in moderation scenarios where you want to quarantine media and can quickly/easily look at the data attribute on the media (image/videos). Or simply write a little script to extract all of the `data-mxc-url` and `data-thumbnail-mxc-url` attributes.
ex.
```
<img src="http://localhost:8008/_matrix/media/r0/thumbnail/my.synapse.server/TEyTVUNgvUQZcXlrLwgGfbcp?width=400&height=266&method=scale" alt="Stormclouds.jpg" title="Stormclouds.jpg" data-mxc-url="mxc://my.synapse.server/kxibKhxRfTvFWyuWwWvFuBtE" data-thumbnail-mxc-url="mxc://my.synapse.server/TEyTVUNgvUQZcXlrLwgGfbcp" style="max-width: 400px; max-height: 266px;">
```
- Default to a nice `[matrix]` banner
- There is room for improvement here when the Matrix Public Archive gets it's own logo (https://github.com/matrix-org/matrix-public-archive/issues/94) and maybe says "Matrix Public Archive" somewhere in the banner.
- This is good enough for now (and certainly better than downstream previews using the first image on the page).
- For rooms, it will use the room avatar
Part of https://github.com/matrix-org/matrix-public-archive/issues/202
Image is sized to 1200x630 to match conventions of `og:image`.
Crafted the banner image by modifying the header on the room directory homepage and taking a node screenshot. Page zoom @ 175%
Fix `debugActiveDateIntersectionObserver` checkbox being checked by default when the value was actually `null`.
Now we properly only care about explicit `'true'`, `'false'` from local storage.
Before:
```
<input id="debugActiveDateIntersectionObserver" type="checkbox" checked="null">
```
After:
```
<input id="debugActiveDateIntersectionObserver" type="checkbox">
```
Set `X-Date-Temporal-Context: [past|present|future]` header for easy cache rules:
- Cache `past` things heavily
- Cache `present`/`future` things for 5 minutes
This accomplishes the goal we set out for:
> - We can cache all responses except for the latest UTC day (and anything in the future). ex. `/!aMzLHLvScQCGKDNqCB:gitter.im/date/2022/10/13`
> - For the latest day, we could set the cache expire after 5 minutes or so
>
> *-- [Matrix Public Archive deployment issue](https://github.com/vector-im/sre-internal/issues/2079)*
And this way we don't have to do any fancy date parsing and comparison from the URL which is probably not even possible Cloudflare cache rules.
Davide Girardi