 Cyberes
|
e9db83f09b
|
move elastic sync to workers instead of threads, parallel elastic delete sync, reimplement partial elastic sync
|
2023-12-13 14:21:47 -07:00 |
|
Cyberes
|
d16eaf614e
|
fix mior tihgs
|
2023-12-12 18:00:43 -07:00 |
|
Cyberes
|
8d08f04a4f
|
add crawl type indicator to admin status page
|
2023-12-12 17:52:03 -07:00 |
|
Cyberes
|
a39b3ea010
|
reorganize
|
2023-12-12 17:30:47 -07:00 |
|
Cyberes
|
6377b8b6bc
|
move elastic crawlers to workers
|
2023-12-12 17:26:39 -07:00 |
|
Cyberes
|
17c96e45c3
|
remove profiling
|
2023-12-12 00:54:50 -07:00 |
|
Cyberes
|
82636792ea
|
dealing with memory usage
|
2023-12-11 23:45:09 -07:00 |
|
Cyberes
|
72e6355869
|
reorganize HTTP routes, improve JSON response
|
2023-12-11 22:36:41 -07:00 |
|
Cyberes
|
112ab0e08f
|
clarify log item
|
2023-12-11 21:37:16 -07:00 |
|
Cyberes
|
2579c76f04
|
fix memory usage related to the worker queue size, reorganize things
|
2023-12-11 21:35:44 -07:00 |
|
Cyberes
|
b5327e0c67
|
update todo
|
2023-12-11 19:12:00 -07:00 |
|
Cyberes
|
157f80a463
|
make workers global, fix worker setup, clean up
|
2023-12-11 18:50:30 -07:00 |
|
Cyberes
|
7078712bc3
|
track running crawls and add an admin page, use basic auth for admin, reject crawl if already running for a path, limit max directory crawlers, fix some issues
|
2023-12-11 18:05:59 -07:00 |
|
Cyberes
|
a96708f6cf
|
fix error when a file in the cache is not found on the disk
|
2023-12-11 16:18:12 -07:00 |
|
Cyberes
|
634f3eb8ea
|
fix download encoding, redo config passing,
|
2023-12-11 15:29:34 -07:00 |
|
Cyberes
|
4b9c1ba91a
|
Merge dev to master
|
2023-12-08 22:25:59 -07:00 |
|
Cyberes
|
627f4d2069
|
limit max workers
|
2023-07-20 13:06:19 -06:00 |
|
Cyberes
|
4e9d3265fd
|
refactor and performance improvements
|
2023-07-20 13:06:07 -06:00 |
|
Cyberes
|
f40907dd8a
|
add sorting arg to /list and /search with option to sort folders first
fix crawler on /list
fix json encoding empty children array to null on /list
fix recache
|
2023-07-18 10:58:29 -06:00 |
|
Cyberes
|
fabe432ac4
|
should be pretty good!
|
2023-07-17 23:20:21 -06:00 |
|
Cyberes
|
2bead0284c
|
Initial commit
|
2023-07-13 13:54:42 -06:00 |