Commit Graph

66 Commits

Author SHA1 Message Date
Cyberes 17c96e45c3 remove profiling 2023-12-12 00:54:50 -07:00
Cyberes 82636792ea dealing with memory usage 2023-12-11 23:45:09 -07:00
Cyberes 72e6355869 reorganize HTTP routes, improve JSON response 2023-12-11 22:36:41 -07:00
Cyberes 112ab0e08f clarify log item 2023-12-11 21:37:16 -07:00
Cyberes 2579c76f04 fix memory usage related to the worker queue size, reorganize things 2023-12-11 21:35:44 -07:00
Cyberes b5327e0c67 update todo 2023-12-11 19:12:00 -07:00
Cyberes 157f80a463 make workers global, fix worker setup, clean up 2023-12-11 18:50:30 -07:00
Cyberes 7078712bc3 track running crawls and add an admin page, use basic auth for admin, reject crawl if already running for a path, limit max directory crawlers, fix some issues 2023-12-11 18:05:59 -07:00
Cyberes a96708f6cf fix error when a file in the cache is not found on the disk 2023-12-11 16:18:12 -07:00
Cyberes 634f3eb8ea fix download encoding, redo config passing, 2023-12-11 15:29:34 -07:00
Cyberes 4b9c1ba91a Merge dev to master 2023-12-08 22:25:59 -07:00
Cyberes 627f4d2069 limit max workers 2023-07-20 13:06:19 -06:00
Cyberes 4e9d3265fd refactor and performance improvements 2023-07-20 13:06:07 -06:00
Cyberes f40907dd8a add sorting arg to /list and /search with option to sort folders first
fix crawler on /list
fix json encoding empty children array to null on /list
fix recache
2023-07-18 10:58:29 -06:00
Cyberes fabe432ac4 should be pretty good! 2023-07-17 23:20:21 -06:00
Cyberes 2bead0284c Initial commit 2023-07-13 13:54:42 -06:00