59fc9891d4
shuffle named batches while respecting and accounting for grad_accum
2023-06-07 18:07:37 +02:00
a8455b9427
wip first fit decreasing for batch/grad accum shuffling
2023-06-07 13:49:06 +02:00
2e8056174d
Always convert YAML tags to strings
2023-04-23 00:36:32 +02:00
2bb35eaa0a
Use filenames for caption if no main prompt in yaml
2023-04-14 00:59:26 +02:00
161e0a563c
Prioritize tags, `image > local > global`, but respect weights
2023-03-21 00:15:53 +01:00
fae0b3c535
Retain original tag order when parsing captions
2023-03-19 23:30:42 +01:00
48f132554c
Assign default value for MAX_CAPTION_LENGTH
2023-03-15 21:52:32 +01:00
6413ada2e5
Add a test for default rating
2023-03-13 01:01:29 +01:00
dea8c6e862
Refactor dataset to support multiple main_prompts
...
...rather than multiple whole captions
2023-03-12 19:48:15 +01:00
0716c40ab6
Add support for enhanced dataset configuration
...
Add support for:
* flip_p.txt
* cond_dropout.txt
* local.yaml consolidated config (including default captions)
* global.yaml consolidated config which applies recursively to subfolders
* flip_p, and cond_dropout config per image
* manifest.json with full image-level configuration
2023-03-08 15:02:14 +01:00
79bcd44bed
Add docker build action
...
And also a VS Code dev container
2023-02-23 02:02:56 +01:00
09d95fac58
Add test for list data resolver
2023-01-29 17:21:12 -08:00
326d861a86
Push DLMA into main, pass config to resolve
...
This patch
* passes the configuration (`argparse.Namespace`) to the resolver,
* pushes the DLMA code into the main function,
* makes DLMA take a `list[ImageTrainItem]` instead of `data_root`,
* makes `EveryDreamBatch` take `DLMA` instead of `data_root`, etc.
* allows `data_root` to be a list.
By doing these things, both `EveryDreamBatch` and DLMA can be free from
data resolution logic. It also reduces the number of arguments which
need to be passed down to EDB and DLMA.
2023-01-29 17:08:54 -08:00
620b157e6a
Fix some name errors
2023-01-23 16:57:02 -08:00
685bada570
Update tests
2023-01-23 12:15:35 -08:00
4e6c5f4d00
Get rid of on_event callback
2023-01-22 23:58:25 -08:00
914a51b057
Add data.resolver module for training data resolution
2023-01-22 22:46:04 -08:00
f4f684a915
Add unit tests for data.image_train_item
2023-01-22 22:43:59 -08:00
damian