Mirrors
/
EveryDream2trainer
mirror of https://github.com/victorchall/EveryDream2trainer.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

doc for embedding perturbation

This commit is contained in:
Victor Hall 2024-06-09 02:03:31 -04:00
parent beec38726a
commit 3b06e9f651
1 changed files with 16 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
doc/ADVANCED_TWEAKING.md
View File

@ -44,6 +44,22 @@ This may also be useful to really "force" a style into the model with a high set
--cond_dropout 0.1 ^
## Conditional Embedding Perturbation
Paper: https://arxiv.org/pdf/2405.20494
--embedding_perturbation 1.0
This is the gamma value in the paper. This can be set to 0.0 to disable. It adds gaussian noise to the embedding vector created by the text encoder.
The noise zero centered with a std_dev of (embedding_perturbation divided by the square root of the embedding dimension) of the text encoder (i.e. 768 for CLIP-L used in SD1.x).
$
\xi \backsim \mathcal{N} (0, \frac{\gamma}{\sqrt{\mathcal{d}}})
$
You can join the Discord server to see [experimental results](https://discord.com/channels/1026983422431862825/1247917538952740955).
## Timestep clamping
Stable Diffusion uses 1000 possible timesteps for denoising steps. Timesteps are always chosen randomly per training example, per step, within the possible or allowed timesteps.