31 lines
1.5 KiB
Markdown
31 lines
1.5 KiB
Markdown
|
Everydream 2 trainer is built using various open source technologies and packages.
|
||
|
|
## Stable Diffusion:
|
||
|
|
|
||
|
|
### Predecessors
|
||
|
|
|
||
|
|
Universiteit van Amsterdam - AutoencoderKL from [paper](https://arxiv.org/abs/1312.6114v11)
|
||
|
|
|
||
|
|
Berkeley - DDPM [DDPM](https://arxiv.org/abs/2006.11239) - [code](https://github.com/hojonathanho/diffusion)
|
||
|
|
|
||
|
|
OpenAI - CLIP (used in SD1.x and SDXL) [Paper](https://arxiv.org/pdf/2103.00020.pdf) - [github](https://github.com/OpenAI/CLIP)
|
||
|
|
|
||
|
|
LAION OpenClip (used in SD2.x and SDXL) [Announcement](https://laion.ai/blog/large-openclip/) - [github](https://github.com/mlfoundations/open_clip)
|
||
|
|
|
||
|
|
### Original Compvis Stable Diffusion (CLIP + LDM + AutoencoderKL)
|
||
|
|
[paper](https://arxiv.org/abs/2112.10752) - [github](https://github.com/CompVis/stable-diffusion)
|
||
|
|
|
||
|
|
## Captioning models:
|
||
|
|
|
||
|
|
Open Flamingo [paper](https://arxiv.org/abs/2308.01390) - [github](https://github.com/mlfoundations/open_flamingo)
|
||
|
|
|
||
|
|
Salesforce BLIP/BLIP2 [blip paper](https://arxiv.org/abs/2201.12086) - [blip2 github (LAVIS)](https://github.com/salesforce/LAVIS) - [blip1 github](https://github.com/salesforce/BLIP)
|
||
|
|
|
||
|
|
Kosmos-2 [paper](https://arxiv.org/abs/2306.14824) - [Github](https://github.com/microsoft/unilm/tree/master/kosmos-2) - [Huggingface](https://huggingface.co/microsoft/kosmos-2-patch14-224)
|
||
|
|
|
||
|
|
## Optimizers
|
||
|
|
|
||
|
|
Tim Dettmers - 8-bit quantization (adamw8bit, etc) [paper](https://arxiv.org/abs/2110.02861) - [github](https://github.com/TimDettmers/bitsandbytes)
|
||
|
|
|
||
|
|
Facebook - D-Adaptation [paper](https://arxiv.org/abs/2301.07733) - [github](https://github.com/facebookresearch/dadaptation)
|
||
|
|
|