37a44bb283
Add ModelEditing pipeline ( #2721 )
...
* TIME first commit
* styling.
* styling 2.
* fixes; tests
* apply styling and doc fix.
* remove sups.
* fixes
* remove temp file
* move augmentations to const
* added doc entry
* code quality
* customize augmentations
* quality
* quality
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2023-03-24 13:01:39 +05:30
b94880e536
Add AudioLDM ( #2232 )
...
* Add AudioLDM
* up
* add vocoder
* start unet
* unconditional unet
* clap, vocoder and vae
* clean-up: conversion scripts
* fix: conversion script token_type_ids
* clean-up: pipeline docstring
* tests: from SD
* clean-up: cpu offload vocoder instead of safety checker
* feat: adapt tests to audioldm
* feat: add docs
* clean-up: amend pipeline docstrings
* clean-up: make style
* clean-up: make fix-copies
* fix: add doc path to toctree
* clean-up: args for conversion script
* clean-up: paths to checkpoints
* fix: use conditional unet
* clean-up: make style
* fix: type hints for UNet
* clean-up: docstring for UNet
* clean-up: make style
* clean-up: remove duplicate in docstring
* clean-up: make style
* clean-up: make fix-copies
* clean-up: move imports to start in code snippet
* fix: pass cross_attention_dim as a list/tuple to unet
* clean-up: make fix-copies
* fix: update checkpoint path
* fix: unet cross_attention_dim in tests
* film embeddings -> class embeddings
* Apply suggestions from code review
Co-authored-by: Will Berman <wlbberman@gmail.com>
* fix: unet film embed to use existing args
* fix: unet tests to use existing args
* fix: make style
* fix: transformers import and version in init
* clean-up: make style
* Revert "clean-up: make style"
This reverts commit 5d6d1f8b324f5583e7805dc01e2c86e493660d66.
* clean-up: make style
* clean-up: use pipeline tester mixin tests where poss
* clean-up: skip attn slicing test
* fix: add torch dtype to docs
* fix: remove conversion script out of src
* fix: remove .detach from 1d waveform
* fix: reduce default num inf steps
* fix: swap height/width -> audio_length_in_s
* clean-up: make style
* fix: remove nightly tests
* fix: imports in conversion script
* clean-up: slim-down to two slow tests
* clean-up: slim-down fast tests
* fix: batch consistent tests
* clean-up: make style
* clean-up: remove vae slicing fast test
* clean-up: propagate changes to doc
* fix: increase test tol to 1e-2
* clean-up: finish docs
* clean-up: make style
* feat: vocoder / VAE compatibility check
* feat: possibly expand / cut audio waveform
* fix: pipeline call signature test
* fix: slow tests output len
* clean-up: make style
* make style
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: William Berman <WLBberman@gmail.com>
2023-03-23 19:00:21 +01:00
1870fb05a9
[docs] Add Colab notebooks and Spaces ( #2713 )
...
* add colab notebook and spaces
* fix image link
2023-03-23 09:48:58 -07:00
df91c44712
Flax controlnet ( #2727 )
...
* add contronet flax
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-23 05:46:23 -10:00
0d7aac3e8d
[Docs] small fixes to the text to video doc. ( #2787 )
...
* small fixes to the text to video doc.
* add: Spaces link.
* add: warning on research-only model.
2023-03-23 18:57:02 +05:30
2ef9bdd76f
Music Spectrogram diffusion pipeline ( #1044 )
...
* initial TokenEncoder and ContinuousEncoder
* initial modules
* added ContinuousContextTransformer
* fix copy paste error
* use numpy for get_sequence_length
* initial terminal relative positional encodings
* fix weights keys
* fix assert
* cross attend style: concat encodings
* make style
* concat once
* fix formatting
* Initial SpectrogramPipeline
* fix input_tokens
* make style
* added mel output
* ignore weights for config
* move mel to numpy
* import pipeline
* fix class names and import
* moved models to models folder
* import ContinuousContextTransformer and SpectrogramDiffusionPipeline
* initial spec diffusion converstion script
* renamed config to t5config
* added weight loading
* use arguments instead of t5config
* broadcast noise time to batch dim
* fix call
* added scale_to_features
* fix weights
* transpose laynorm weight
* scale is a vector
* scale the query outputs
* added comment
* undo scaling
* undo depth_scaling
* inital get_extended_attention_mask
* attention_mask is none in self-attention
* cleanup
* manually invert attention
* nn.linear need bias=False
* added T5LayerFFCond
* remove to fix conflict
* make style and dummy
* remove unsed variables
* remove predict_epsilon
* Move accelerate to a soft-dependency (#1134 )
* finish
* finish
* Update src/diffusers/modeling_utils.py
* Update src/diffusers/pipeline_utils.py
Co-authored-by: Anton Lozhkov <anton@huggingface.co>
* more fixes
* fix
Co-authored-by: Anton Lozhkov <anton@huggingface.co>
* fix order
* added initial midi to note token data pipeline
* added int to int tokenizer
* remove duplicate
* added logic for segments
* add melgan to pipeline
* move autoregressive gen into pipeline
* added note_representation_processor_chain
* fix dtypes
* remove immutabledict req
* initial doc
* use np.where
* require note_seq
* fix typo
* update dependency
* added note-seq to test
* added is_note_seq_available
* fix import
* added toc
* added example usage
* undo for now
* moved docs
* fix merge
* fix imports
* predict first segment
* avoid un-needed copy to and from cpu
* make style
* Copyright
* fix style
* add test and fix inference steps
* remove bogus files
* reorder models
* up
* remove transformers dependency
* make work with diffusers cross attention
* clean more
* remove @
* improve further
* up
* uP
* Apply suggestions from code review
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
* loop over all tokens
* make style
* Added a section on the model
* fix formatting
* grammer
* formatting
* make fix-copies
* Update src/diffusers/pipelines/__init__.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update src/diffusers/pipelines/spectrogram_diffusion/pipeline_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* added callback ad optional ionnx
* do not squeeze batch dim
* clean up more
* upload
* convert jax to nnumpy
* make style
* fix warning
* make fix-copies
* fix warning
* add initial fast tests
* add initial pipeline_params
* eval mode due to dropout
* skip batch tests as pipeline runs on a single file
* make style
* fix relative path
* fix doc tests
* Update src/diffusers/models/t5_film_transformer.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update src/diffusers/models/t5_film_transformer.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update docs/source/en/api/pipelines/spectrogram_diffusion.mdx
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add MidiProcessor
* format
* fix org
* Apply suggestions from code review
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
* make style
* pin protobuf to <4
* fix formatting
* white space
* tensorboard needs protobuf
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Anton Lozhkov <anton@huggingface.co>
2023-03-23 14:06:17 +01:00
14e3a28c12
Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' ( #2732 )
...
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
2023-03-23 13:49:22 +01:00
8e35ef0142
[doc wip] literalinclude ( #2718 )
2023-03-23 13:42:54 +01:00
0d633a42f4
deduplicate training section in the docs. ( #2788 )
2023-03-23 11:21:53 +05:30
9dc84448ac
[Examples] InstructPix2Pix instruct training script ( #2478 )
...
* add: initial implementation of the pix2pix instruct training script.
* shorten cli arg.
* fix: main process check.
* fix: dataset column names.
* simplify tokenization.
* proper placement of null conditions.
* apply styling.
* remove debugging message for conditioning do.
* complete license.
* add: requirements.tzt
* wandb column name order.
* fix: augmentation.
* change: dataset_id.
* fix: convert_to_np() call.
* fix: reshaping.
* fix: final ema copy.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* address PR comments.
* add: readme details.
* config fix.
* downgrade version.
* reduce image width in the readme.
* note on hyperparameters during generation.
* add: output images.
* update readme.
* minor edits to readme.
* debugging statement.
* explicitly placement of the pipeline.
* bump minimum diffusers version.
* fix: device attribute error.
* weight dtype.
* debugging.
* add dtype inform.
* add seoarate te and vae.
* add: explicit casting/
* remove casting.
* up.
* up 2.
* up 3.
* autocast.
* disable mixed-precision in the final inference.
* debugging information.
* autocasting.
* add: instructpix2pix training section to the docs.
* Empty-Commit
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-03-23 10:15:01 +05:30
c681ad1af2
add: section on multiple controlnets. ( #2762 )
...
* add: section on multiple controlnets.
Co-authored-by: William Berman <WLBberman@gmail.com>
* fix: docs.
* fix: docs.
---------
Co-authored-by: William Berman <WLBberman@gmail.com>
2023-03-23 09:55:25 +05:30
92e1164e2e
`mps`: remove warmup passes ( #2771 )
...
* Remove warmup passes in mps tests.
* Update mps docs: no warmup pass in PyTorch 2
* Update imports.
2023-03-22 19:29:27 +01:00
ca1a22296d
[MS Text To Video] Add first text to video ( #2738 )
...
* [MS Text To Video} Add first text to video
* upload
* make first model example
* match unet3d params
* make sure weights are correcctly converted
* improve
* forward pass works, but diff result
* make forward work
* fix more
* finish
* refactor video output class.
* feat: add support for a video export utility.
* fix: opencv availability check.
* run make fix-copies.
* add: docs for the model components.
* add: standalone pipeline doc.
* edit docstring of the pipeline.
* add: right path to TransformerTempModel
* add: first set of tests.
* complete fast tests for text to video.
* fix bug
* up
* three fast tests failing.
* add: note on slow tests
* make work with all schedulers
* apply styling.
* add slow tests
* change file name
* update
* more correction
* more fixes
* finish
* up
* Apply suggestions from code review
* up
* finish
* make copies
* fix pipeline tests
* fix more tests
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* apply suggestions
* up
* revert
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2023-03-22 18:39:33 +01:00
7fe88613fa
[docs] Clarify purpose of reproducibility docs ( #2756 )
...
* clarify purpose of repro docs
* apply feedback
2023-03-21 17:35:21 -07:00
a39d42b91d
[docs] update torch 2 benchmark ( #2764 )
...
* Update benchmark for A100, 3090, 3090 Ti, 4090.
* Link to PyTorch blog.
* Update install instructions.
2023-03-21 17:41:13 +00:00
2120b4eee3
Improve Contribution Doc ( #2043 )
...
* first refactor
* more text
* improve
* finish
* up
* up
* up
* up
* finish
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* up
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* finished
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* finished
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2023-03-21 13:41:29 +01:00
c10d6854c0
Update numbers for Habana Gaudi in documentation ( #2734 )
...
Update numbers for Habana Gaudi in doc
2023-03-21 11:59:28 +01:00
73bdad08a1
add: controlnet entry to training section in the docs. ( #2677 )
...
* add: controlnet entry to training section in the docs.
* formatting.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* wrap in a tip block.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-03-21 07:23:24 +05:30
ba87c1607c
Update text_inversion.mdx ( #2751 )
...
Fix typos
2023-03-20 13:20:50 -07:00
afe59a920e
Update philosophy.mdx ( #2752 )
...
Fix typos
2023-03-20 13:19:43 -07:00
25ed7cb08b
Update dreambooth.mdx ( #2742 )
...
Fix typos
2023-03-20 17:40:56 +00:00
af86b0ccac
Update fp16.mdx ( #2746 )
...
Fix typos
2023-03-20 17:39:55 +00:00
a9f28b687c
Update torch2.0.mdx ( #2748 )
...
Fix typos
2023-03-20 17:39:04 +00:00
d91dc57d8a
Update mps.mdx ( #2749 )
...
Fix typos
2023-03-20 17:33:23 +00:00
a16957159e
[docs] Update ONNX doc to use `optimum` ( #2702 )
...
* minor edits to onnx and openvino docs.
* Apply suggestions from code review
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
---------
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
2023-03-17 18:17:42 +01:00
f4bbcb29c0
fix image link in inpaint doc ( #2693 )
...
fix link
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-16 19:35:27 -10:00
77e0ea8048
[docs] Add safety checker to ethical guidelines ( #2699 )
...
add safety checker
2023-03-16 09:39:39 -07:00
e828232780
Rename attention ( #2691 )
...
* rename file
* rename attention
* fix more
* rename more
* up
* more deprecation imports
* fixes
2023-03-16 00:35:54 +01:00
588e50bc57
[docs] Reorganize table of contents ( #2671 )
...
* reorg toc
* reorg toc some more
* remove duplicate config
2023-03-15 16:28:18 -07:00
a72d14fc8d
[docs] Create better navigation on index ( #2658 )
...
* create updated nav for index
* fix header
* apply feedback
2023-03-15 11:58:04 -07:00
1c2c594e3d
[docs] Add overviews to each section ( #2657 )
...
* add overviews to each section
* fix typo in toctree
* apply feedbacks
2023-03-15 11:57:32 -07:00
c0b4d72095
Update unconditional_image_generation.mdx ( #2686 )
...
Fix typos
2023-03-15 18:19:57 +01:00
78afb84436
Update controlling_generation.mdx ( #2690 )
...
Fix typos
2023-03-15 18:18:41 +01:00
91570b2fda
Update conditional_image_generation.mdx ( #2687 )
...
Fix typos
2023-03-15 18:16:32 +01:00
3584f6b345
Update img2img.mdx ( #2688 )
...
Fix typos
2023-03-15 18:15:59 +01:00
b4bb5345cd
Update kerascv.mdx ( #2685 )
...
Fix typos
2023-03-15 18:15:51 +01:00
e71f73d8df
Update custom_pipeline_overview.mdx ( #2684 )
...
Fix typos
2023-03-15 18:14:37 +01:00
c9477bf8a8
[Docs] Adds a documentation page for evaluating diffusion models ( #2516 )
...
* add a documentation page for evaluating diffuion models.
* fix: checkpoint link.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
* formatting fixes.
* formatting fixes.
* link to partiprompts dataset on hub.
* reflect on Pedro's comments.
Co-authored-by: Pedro <pedro@huggingface.co>
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* reflect on Pedro's comments.
Co-authored-by: Pedro <pedro@huggingface.co>
* update mention of FID.
* Apply suggestions from code review
Co-authored-by: Will Berman <wlbberman@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* minor nit.
* finish edges and add colab notebook.
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* run formatting.
* additional feedback.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: Pedro <pedro@huggingface.co>
Co-authored-by: Will Berman <wlbberman@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
2023-03-15 17:05:01 +05:30
fa7a576191
Update schedulers.mdx ( #2647 )
...
Fix typos
2023-03-13 16:41:28 +01:00
bbab855322
Update loading.mdx ( #2642 )
...
Fix typos
2023-03-11 16:49:05 +01:00
d5ce55293c
[docs] Build Jax notebooks for real ( #2641 )
...
build jax notebooks for real
2023-03-11 01:21:14 +01:00
f1ab955f64
Update basic_training.mdx ( #2639 )
...
Add 'import os'
2023-03-10 14:19:12 +01:00
9360bb94c3
Update quicktour.mdx ( #2637 )
...
Fix typo
2023-03-10 14:17:10 +01:00
4aa68291a9
add translated docs ( #2587 )
...
* add translated docs
* improve translated content
* improve translated content
* Modify the translation content
2023-03-10 13:55:12 +01:00
0650d641a3
Revert "[docs] Build notebooks from Markdown" ( #2625 )
...
Revert "[docs] Build notebooks from Markdown (#2570 )"
This reverts commit 78507bda24
2023-03-09 15:45:24 +01:00
251a34add8
Migrate blog content to docs ( #2477 )
...
* first draft
* ✨ minor edits
* 💄 make style
* oops add to toc
* 🖍 reframe around understanding components
* 🖍 apply feedback
* 🖍 apply feedback
2023-03-09 13:20:49 +01:00
a062e47ec3
add flax pipelines to api doc + doc string examples ( #2600 )
...
* add api doc for flax pipeline + doc string examples
* make style
---------
Co-authored-by: yiyixuxu <yixu@yis-macbook-pro.lan>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-03-09 13:00:29 +01:00
a5d2ee9d47
Add OpenVINO documentation ( #2569 )
...
* Add OpenVINO documentation
* Update docs/source/en/optimization/open_vino.mdx
Co-authored-by: YiYi Xu <yixu310@gmail.com>
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
2023-03-08 20:07:44 +01:00
68545a15d9
[docs] Update unconditional image generation docs ( #2592 )
...
* 📝 update and minor refactor
* ✨ minor edits
2023-03-08 09:47:49 -08:00
445a176bde
[Docs] Fix link to colab ( #2604 )
2023-03-08 12:59:58 +01:00
Bahjat Kawar