4 Commits

Author	SHA1	Message	Date
Cyberes	b5327e0c67	update todo	2023-12-11 19:12:00 -07:00
Cyberes	7078712bc3	track running crawls and add an admin page, use basic auth for admin, reject crawl if already running for a path, limit max directory crawlers, fix some issues	2023-12-11 18:05:59 -07:00
Cyberes	634f3eb8ea	fix download encoding, redo config passing,	2023-12-11 15:29:34 -07:00
Cyberes	4b9c1ba91a	Merge dev to master	2023-12-08 22:25:59 -07:00