wmts-exfiltrator

Scrape tiles from WMTS servers.

You know what I hate? Those godforsaken WMTS servers, perched on their digital thrones, acting like they're the TILE TYRANTS of the universe. They think they can just LOCK UP their precious little tiles and keep me from doing my THING? HA!

No more will these WMTS servers shroud their CRAPPY-ASS tiles in mystery. I'm coming for your DATA, you binary BASTARDS, and there's not a SINGLE 1 or 0 you can throw at me that will stop my CHARGE.

You think your firewalls and security mumbo-jumbo can keep me at bay? THINK AGAIN. I'll slice through your defenses like a HOT PIZZA through COLD BUTTER. I'll have your DATA, and there's absolutely NOTHING, I repeat, NOTHING you can do to STOP ME.

So, buckle up, WMTS servers. Your reign of TILE TERROR is about to CRASH AND BURN. I'm coming for your DATA, and I'm bringing a whole lot of CHAOS with me.

Install

It's recommended to use a venv.

pip install -r requirements.txt

Use

Some WMTS servers require the correct Referer header to be set, otherwise they will reject your request or return blank data. Use --referer to set this header.

Do ./exfiltrate.py -h to get more info on what the different command args do.

Building the GeoTIFF will take dozens of gigs of memory for any significant extent! For example, a 336 square mile extent required about 280GB of memory. You can use swap, but will need a very fast SSD. I had good results with a Samsung 980 PRO partitioned to swap.

Be careful not to go overboard with your spatial extent: use only what you need to avoid unnecessary processing time or else you will easily end up with a situation where it will take a week to download all the tiles and build a TIFF.

test.sh is provided for demonstration. It downloads 856 tiles (less than 2MB) from a WMTS server in Taiwan.

Output

This program outputs a geo-referenced TIFF image with three bands corresponding to red, green, and blue in the original tiles pixels. If one of the bands has a value of 0, then that value is adjusted to 1 so that it does not conflict with the NODATA value of 0.

ArcGIS

The generated TIFF raster should be fully compatible with ArcGIS but if you encounter color issues, try adjusting your symbology.

Inspiration

https://jimmyutterstrom.com/blog/2019/06/05/map-tiles-to-geotiff/