86aaf1c4d7
fix big when loss_scale.txt contains 0
2023-11-01 19:00:08 +01:00
da731268b2
put a file loss_scale.txt containing a float in a training folder to apply loss scale (eg -1 for negative examples)
2023-10-31 10:06:21 +01:00
60e10867bc
Fix undersize warning
2023-10-05 01:48:09 -03:00
99a0431d0f
Ignore negative keep_tags values
2023-09-20 19:53:31 -03:00
43984f2ad3
Add --keep_tags to keep first N tags fixed on shuffle
2023-09-20 19:53:30 -03:00
7e5f8244b1
Refactored the _percent_random_crop function to make it simpler and cleaner.
...
- Removed the aspect ratio adjustment after cropping - which caused unnecessary stretch.
- Made sure that the crop pixels values are integers.
2023-07-03 18:39:24 +03:00
53d0686086
add a batch_id.txt file to subfolders or a batch_id key to local yaml to force images for that folder to be processed in the same batch
2023-06-05 01:02:27 +02:00
97f1160496
correct image cropping
2023-06-01 21:06:55 -04:00
0e0b546ef7
help debug crop error someone got
2023-06-01 19:11:22 -04:00
56deb26a59
early work on shuffle_tags.txt and add try around trimming
2023-06-01 16:18:21 -04:00
7dcfa7acbf
Speed up preloading by not loading pixel data
...
PIL lazy loads pixel data, so image size can be accessed without loading the full image.
This commit makes it so only image size and EXIF metadata are fetched from disk, speeding up the preload stage while still supporting transposing images.
2023-05-31 20:48:48 -04:00
5c98cdee70
update to torch2, xformers 20, bnb 381
2023-05-30 22:15:02 -04:00
e3e30a5599
fix exif issues
2023-04-16 14:53:34 -04:00
3697b16344
put exif transpose back in preloading to fix bug
2023-04-16 02:19:27 -04:00
b8f637873f
removing the convert_rgb flag for a simpler design
2023-04-14 20:21:48 -05:00
a839180199
set # of data loaders by the min of batch size or cpu count, do not do an rg b conversion when only loading image metadata
2023-04-14 14:59:28 -05:00
dd98ebe080
Add error handling for bad exif, and rotate before sizing
2023-03-28 19:19:08 +02:00
56256ab9ef
attempt to catch some errors reported on github
2023-03-26 11:52:49 -04:00
48f132554c
Assign default value for MAX_CAPTION_LENGTH
2023-03-15 21:52:32 +01:00
605716a646
conf
2023-03-15 11:22:40 -04:00
da3c183cc5
autofix exif orientation
2023-03-15 11:12:59 -04:00
0716c40ab6
Add support for enhanced dataset configuration
...
Add support for:
* flip_p.txt
* cond_dropout.txt
* local.yaml consolidated config (including default captions)
* global.yaml consolidated config which applies recursively to subfolders
* flip_p, and cond_dropout config per image
* manifest.json with full image-level configuration
2023-03-08 15:02:14 +01:00
8abef6bc74
revert multiline txt for now due to bug
2023-02-28 21:14:19 -05:00
6c87af7711
doc, fix lr defaulting bug, notdreambooth.md, read multicaption from .txt by line
2023-02-26 19:11:42 -05:00
4e37200dda
fix multiplier issues with validation and refactor validation logic
2023-02-08 11:28:45 +01:00
85f19b9a2f
doc and bug with undersized
2023-02-06 13:11:24 -05:00
a7505df0a6
Initialize target_wh to None
2023-01-24 08:33:40 -08:00
44fcbb6aa7
Bugfix: Adding same Tag twice to Caption
2023-01-24 16:02:12 +01:00
620b157e6a
Fix some name errors
2023-01-23 16:57:02 -08:00
e99a948d3c
Fix property name bug
2023-01-23 12:52:53 -08:00
9491ae430c
Initialize is_undersized to False, fix bug in directory resolver
2023-01-23 12:44:48 -08:00
1a0b7994f4
Move target_wh calculation to ImageTrainItem
2023-01-23 12:00:42 -08:00
646f383188
If there are no tags just return the main prompt
2023-01-23 11:27:12 -08:00
b6c7299baf
Merge branch 'main' into refactor-data-resolution
2023-01-23 08:43:23 -08:00
993eabf99a
Add static methods on ImageCaption for deriving captions from various sources
2023-01-23 08:22:44 -08:00
0cf2cd71de
Fix mistake in ImageCaption.parse
2023-01-23 00:14:50 -08:00
85b6aad6a9
If tags are empty, returne __main_prompt
2023-01-22 22:44:44 -08:00
a6cabe8d7d
Add static methods to ImageCaption for deriving captions from various sources
2023-01-22 16:13:50 -08:00
24b00ab35b
add fractional support for multiply.txt
2023-01-22 01:15:50 -05:00
711e263e24
Implemented system to train on a subset of the dataset, favouring higher rated images
2023-01-14 16:18:22 +01:00
f47ceadcc7
Implemented an optimization for the shuffling if all tags have the same weight and added documentation.
2023-01-09 21:53:46 +01:00
a3618409bc
Support more control regarding caption tag shuffeling using yaml files
2023-01-09 21:53:07 +01:00
b316684bdb
bunch of updates, grad ckpting, no drop bucket, shuffle every epoch
2023-01-01 10:45:18 -05:00
4c53f2d55c
various tweaks and bugfixes over holidays
2022-12-27 14:25:32 -05:00
8904724135
fix some quality issues
2022-12-20 03:30:42 -05:00
179fd5395b
hey look ed2
2022-12-17 22:32:48 -05:00
Damian Stewart