bd68e35de3
Gradient accumulation, autocast fix, new latent sampling method, etc
2022-11-20 12:35:26 +09:00
cdc8020d13
change StableDiffusionProcessing to internally use sampler name instead of sampler index
2022-11-19 12:01:51 +03:00
cabd4e3b3b
Merge branch 'master' into gradient-clipping
2022-11-07 22:43:38 +07:00
62e3d71aa7
rework the code to not use the walrus operator because colab's 3.7 does not support it
2022-11-05 17:09:42 +03:00
cb84a304f0
Merge pull request #4273 from Omegastick/ordered_hypernetworks
...
Sort hypernetworks list
2022-11-05 16:16:18 +03:00
bb832d7725
Simplify grad clip
2022-11-05 11:48:38 +07:00
08feb4c364
Sort straight out of the glob
2022-11-04 20:53:11 +07:00
3277f90e93
Merge branch 'master' into gradient-clipping
2022-11-04 18:47:28 +07:00
fd62727893
Sort hypernetworks
2022-11-04 18:34:35 +07:00
39541d7725
Fixes race condition in training when VAE is unloaded
...
set_current_image can attempt to use the VAE when it is unloaded to
the CPU while training
2022-11-04 04:50:22 -04:00
1ca0bcd3a7
only save if option is enabled
2022-11-04 16:09:19 +09:00
f5d394214d
split before declaring file name
2022-11-04 16:04:03 +09:00
283249d239
apply
2022-11-04 15:57:17 +09:00
179702adc4
Merge branch 'AUTOMATIC1111:master' into force-push-patch-13
2022-11-04 15:51:09 +09:00
0d07cbfa15
I blame code autocomplete
2022-11-04 15:50:54 +09:00
0abb39f461
resolve conflict - first revert
2022-11-04 15:47:19 +09:00
4918eb6ce4
Merge branch 'master' into hn-activation
2022-11-04 09:02:15 +03:00
1764ac3c8b
use hash to check valid optim
2022-11-03 14:49:26 +09:00
0b143c1163
Separate .optim file from model
2022-11-03 14:30:53 +09:00
d5ea878b2a
Fix merge conflicts
2022-10-31 13:54:40 +07:00
4123be632a
Fix merge conflicts
2022-10-31 13:53:22 +07:00
cd4d59c0de
Merge master
2022-10-30 18:57:51 +07:00
9d96d7d0a0
resolve conflicts
2022-10-30 20:40:59 +09:00
20194fd975
We have duplicate linear now
2022-10-30 20:40:59 +09:00
17a2076f72
Merge pull request #3928 from R-N/validate-before-load
...
Optimize training a little
2022-10-30 09:51:36 +03:00
3d58510f21
Fix dataset still being loaded even when training will be skipped
2022-10-30 00:54:59 +07:00
a07f054c86
Add missing info on hypernetwork/embedding model log
...
Mentioned here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/1528#discussioncomment-3991513
Also group the saving into one
2022-10-30 00:49:29 +07:00
ab05a74ead
Revert "Add cleanup after training"
...
This reverts commit 3ce2bfdf95
2022-10-30 00:32:02 +07:00
3ce2bfdf95
Add cleanup after training
2022-10-29 19:43:21 +07:00
ab27c111d0
Add input validations before loading dataset for training
2022-10-29 18:09:17 +07:00
05e2e40537
Merge branch 'master' into gradient-clipping
2022-10-29 15:04:21 +07:00
e98f72be33
Merge branch 'AUTOMATIC1111:master' into 3825-save-hypernet-strength-to-info
2022-10-29 00:31:23 -07:00
810e6a407d
Merge pull request #3858 from R-N/log-csv
...
Fix log off by 1 #3847
2022-10-29 07:55:20 +03:00
d3b4b9d7ec
Merge pull request #3717 from benkyoujouzu/master
...
Add missing support for linear activation in hypernetwork
2022-10-29 07:30:14 +03:00
f361e804eb
Re enable linear
2022-10-29 08:36:50 +09:00
9ceef81f77
Fix log off by 1
2022-10-28 20:48:08 +07:00
16451ca573
Learning rate sched syntax support for grad clipping
2022-10-28 17:16:23 +07:00
db5a354c48
Always ignore "None.pt" in the hypernet directory.
2022-10-28 01:41:57 -07:00
b2a8b263b2
Add missing support for linear activation in hypernetwork
2022-10-28 12:54:59 +08:00
2a25729623
Gradient clipping in train tab
2022-10-28 09:44:56 +07:00
462e6ba667
Disable unavailable or duplicate options
2022-10-27 15:40:24 +09:00
029d7c7543
Revert unresolved changes in Bias initialization
...
it should be zeros_ or parameterized in future properly.
2022-10-27 14:44:53 +09:00
cc56df996e
Fix dropout logic
2022-10-27 14:38:21 +09:00
85fcccc105
Squashed commit of fixing dropout silently
...
fix dropouts for future hypernetworks
add kwargs for Hypernetwork class
hypernet UI for gradio input
add recommended options
remove as options
revert adding options in ui
2022-10-27 14:38:21 +09:00
b6a8bb123b
Fix merge
2022-10-26 15:15:19 +08:00
a524d137d0
patch bug (SeverianVoid's comment on 5245c7a)
2022-10-26 10:12:46 +03:00
91bb35b1e6
Merge fix
2022-10-26 15:00:03 +08:00
649d79a8ec
Merge branch 'master' into hn-activation
2022-10-26 14:58:04 +08:00
877d94f97c
Back compatibility
2022-10-26 14:50:58 +08:00
7207e3bf49
remove duplicate keys and lowercase
2022-10-26 09:17:01 +03:00
de096d0ce7
Weight initialization and More activation func
...
add weight init
add weight init option in create_hypernetwork
fstringify hypernet info
save weight initialization info for further debugging
fill bias with zero for He/Xavier
initialize LayerNorm with Normal
fix loading weight_init
2022-10-26 09:17:01 +03:00
c702d4d0df
Fix off-by-one
2022-10-26 13:43:04 +08:00
2f4c91894d
Remove activation from final layer of HNs
2022-10-26 12:10:30 +08:00
18f86e41f6
Removed two unused imports
2022-10-24 17:21:18 +02:00
e9a410b535
check length for variance
2022-10-24 09:07:39 +03:00
0d2e1dac40
convert deque -> list
...
I don't feel this being efficient
2022-10-24 09:07:39 +03:00
348f89c8d4
statistics for pbar
2022-10-24 09:07:39 +03:00
40b56c9289
cleanup some code
2022-10-24 09:07:39 +03:00
b297cc3324
Hypernetworks - fix KeyError in statistics caching
...
Statistics logging has changed to {filename : list[losses]}, so it has to use loss_info[key].pop()
2022-10-24 09:07:39 +03:00
1fbfc052eb
Update hypernetwork.py
2022-10-23 08:34:33 +03:00
48dbf99e84
Allow tracking real-time loss
...
Someone had 6000 images in their dataset, and it was shown as 0, which was confusing.
This will allow tracking real time dataset-average loss for registered objects.
2022-10-22 22:24:19 +03:00
24694e5983
Update hypernetwork.py
2022-10-22 20:25:32 +03:00
6a4fa73a38
small fix
2022-10-22 13:44:39 +00:00
97749b7c7d
Merge branch 'AUTOMATIC1111:master' into master
2022-10-22 22:00:59 +09:00
7912acef72
small fix
2022-10-22 13:00:44 +00:00
fccba4729d
add an option to avoid dying relu
2022-10-22 12:02:41 +00:00
7fd90128eb
added a guard for hypernet training that will stop early if weights are getting no gradients
2022-10-22 14:48:43 +03:00
dcb45dfecf
Merge branch 'master' of upstream
2022-10-22 11:14:46 +00:00
0e8ca8e7af
add dropout
2022-10-22 11:07:00 +00:00
272fa527bb
Remove unused variable.
2022-10-21 16:52:24 +03:00
19818f023c
Match hypernet name with filename in all cases.
2022-10-21 16:52:24 +03:00
51e3dc9cca
Sanitize hypernet name input.
2022-10-21 16:52:24 +03:00
03a1e288c4
turns out LayerNorm also has weight and bias and needs to be pre-multiplied and trained for hypernets
2022-10-21 10:13:24 +03:00
0c5522ea21
Merge branch 'master' into training-help-text
2022-10-21 09:57:55 +03:00
4ff274e1e3
Revise comments.
2022-10-21 09:55:00 +03:00
5245c7a493
Issue #2921-Give PNG info to Hypernet previews.
2022-10-21 09:55:00 +03:00
c23f666dba
a more strict check for activation type and a more reasonable check for type of layer in hypernets
2022-10-21 09:47:43 +03:00
7543cf5e3b
Fixed some typos in the code
2022-10-20 22:43:08 +02:00
8f59129847
Some changes to the tensorboard code and hypernetwork support
2022-10-20 22:37:16 +02:00
f89829ec3a
Revert "fix bugs and optimizations"
...
This reverts commit 108be15500
2022-10-21 01:37:11 +09:00
108be15500
fix bugs and optimizations
2022-10-21 01:00:41 +09:00
a71e021236
only linear
2022-10-20 23:48:52 +09:00
d8acd34f66
generalized some functions and option for ignoring first layer
2022-10-20 23:43:03 +09:00
6b38c2c19c
Merge branch 'AUTOMATIC1111:master' into master
2022-10-20 18:51:12 +09:00
930b4c64f7
allow float sizes for hypernet's layer_structure
2022-10-20 08:18:02 +03:00
6f98e89486
update
2022-10-20 00:10:45 +00:00
166be3919b
allow overwrite old hn
2022-10-20 00:09:40 +01:00
d6ea584137
change html output
2022-10-20 00:07:57 +01:00
2ce52d32e4
fix for #3086 failing to load any previous hypernet
2022-10-19 16:31:12 +00:00
c6e9fed500
fix for #3086 failing to load any previous hypernet
2022-10-19 19:21:16 +03:00
3770b8d2fa
enable to write layer structure of hn himself
2022-10-19 15:28:42 +00:00
42fbda83bb
layer options moves into create hnet ui
2022-10-19 14:30:33 +00:00
7f8670c4ef
Merge branch 'master' into master
2022-10-19 15:18:45 +09:00
da72becb13
Use training width/height when training hypernetworks.
2022-10-19 09:13:28 +03:00
e40ba281f1
update
2022-10-19 01:03:58 +09:00
a5611ea502
update
2022-10-19 01:00:01 +09:00
6021f7a75f
add options to custom hypernetwork layer structure
2022-10-19 00:51:36 +09:00
703e6d9e4e
check NaN for hypernetwork tuning
2022-10-15 17:15:26 +03:00
c7a86f7fe9
add option to use batch size for training
2022-10-15 09:24:59 +03:00
03d62538ae
remove duplicate code for log loss, add step, make it read from options rather than gradio input
2022-10-14 22:43:55 +03:00
flamelaw