Commit Graph

65 Commits

Author SHA1 Message Date
Patrick von Platen c39a511b5f
[Loading] Ignore unneeded files (#1107)
* [Loading] Ignore unneeded files

* up
2022-11-02 19:20:42 +01:00
Grigory Sizov 5cd29d623a
Fix tests for equivalence of DDIM and DDPM pipelines (#1069)
* Fix equality test for ddim and ddpm

* add docs for use_clipped_model_output in DDIM

* fix inline comment

* reorder imports in test_pipelines.py

* Ignore use_clipped_model_output if scheduler doesn't take it
2022-11-02 14:50:32 +01:00
Patrick von Platen 8d6487f3cb
Fix some failing tests (#1041)
* up

* up

* up

* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py

* Apply suggestions from code review
2022-10-28 17:05:00 +02:00
Patrick von Platen d2d9764f35
[Tests] Speed up slow tests (#1040)
* [Tests] Speed up slow tests

* Up

* up
2022-10-28 14:46:39 +02:00
Patrick von Platen 3be9fa97d6
[Accelerate model loading] Fix meta device and super low memory usage (#1016)
* [Accelerate model loading] Fix meta device and super low memory usage

* better naming
2022-10-27 12:11:42 +02:00
Pi Esposito b2e2d1411c
minimal stable diffusion GPU memory usage with accelerate hooks (#850)
* add method to enable cuda with minimal gpu usage to stable diffusion

* add test to minimal cuda memory usage

* ensure all models but unet are onn torch.float32

* move to cpu_offload along with minor internal changes to make it work

* make it test against accelerate master branch

* coming back, its official: I don't know how to make it test againt the master branch from accelerate

* make it install accelerate from master on tests

* go back to accelerate>=0.11

* undo prettier formatting on yml files

* undo prettier formatting on yml files againn
2022-10-26 15:52:57 +02:00
Anton Lozhkov 2c82e0c4eb
Reorganize pipeline tests (#963)
* Reorganize pipeline tests

* fix vq
2022-10-24 16:34:01 +02:00
Kashif Rasul 9bca40296e
[MPS] fix mps failing tests (#934)
fix mps failing tests
2022-10-22 09:33:40 +02:00
Patrick von Platen 25dfd0f8dc
[Tests] Move stable diffusion into their own files (#936)
* [Tests] Move stable diffusion into their own files

* up
2022-10-21 12:49:52 +02:00
Suraj Patil 8be48507a0
fix test_components (#928) 2022-10-20 16:25:12 +02:00
Patrick von Platen db19a9d9d7
[DiffusionPipeline.from_pretrained] add warning when passing unused k… (#870)
[DiffusionPipeline.from_pretrained] add warning when passing unused kwargs
2022-10-20 13:30:01 +02:00
Patrick von Platen 83f8a5ff70
[Stable Diffusion] Add components function (#889)
* [Stable Diffusion] Add components function

* uP
2022-10-20 13:28:11 +02:00
Anton Lozhkov 89d124945a
ONNX supervised inpainting (#906)
* ONNX supervised inpainting

* sync with the torch pipeline

* fix concat

* update ref values

* back to 8 steps

* type fix

* make fix-copies
2022-10-19 17:03:31 +02:00
Patrick von Platen 46557121e6
finish tests (#909) 2022-10-19 16:36:51 +02:00
Suraj Patil b35d88c536
Stable diffusion inpainting. (#904)
* begin pipe

* add new pipeline

* add tests

* correct fast test

* up

* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py

* Update tests/test_pipelines.py

* up

* up

* make style

* add fp16 test

* doc, comments

* up

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Anton Lozhkov <anton@huggingface.co>
2022-10-19 16:11:50 +02:00
Anton Lozhkov 8eb9d9703d
Improve ONNX img2img numpy handling, temporarily fix the tests (#899)
* [WIP] Onnx img2img determinism

* more numpy + seed

* numpy inpainting, tolerance

* revert test workflow
2022-10-19 11:26:32 +02:00
Žilvinas Ledas a9908ecfc1
Stable Diffusion image-to-image and inpaint using onnx. (#552)
* * Stabe Diffusion img2img using onnx.

* * Stabe Diffusion inpaint using onnx.

* Export vae_encoder, upgrade img2img, add test

* updated inpainting pipeline + test

* style

Co-authored-by: anton-l <anton@huggingface.co>
2022-10-18 17:44:01 +02:00
Anton Lozhkov 728a3f3ec1
Rename StableDiffusionOnnxPipeline -> OnnxStableDiffusionPipeline (#887)
Rename and deprecate
2022-10-18 09:14:30 +02:00
Anton Lozhkov cca59ce3a2
Add Apple M1 tests (#796)
* [CI] Add Apple M1 tests

* setup-python

* python build

* conda install

* remove branch

* only 3.8 is built for osx-arm

* try fetching prebuilt tokenizers

* use user cache

* update shells

* Reports and cleanup

* -> MPS

* Disable parallel tests

* Better naming

* investigate worker crash

* return xdist

* restart

* num_workers=2

* still crashing?

* faulthandler for segfaults

* faulthandler for segfaults

* remove restarts, stop on segfault

* torch version

* change installation order

* Use pre-RC version of PyTorch.

To be updated when it is released.

* Skip crashing test on MPS, add new one that works.

* Skip cuda tests in mps device.

* Actually use generator in test.

I think this was a typo.

* make style

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2022-10-17 20:27:30 +02:00
Anton Lozhkov 1d3234cbca
Remove the last of ["sample"] (#842) 2022-10-14 14:45:43 +02:00
Anton Lozhkov 52394b53e2
Bump to 0.6.0.dev0 (#831)
* Bump to 0.6.0.dev0

* Deprecate tensor_format and .samples

* style

* upd

* upd

* style

* sample -> images

* Update src/diffusers/schedulers/scheduling_ddpm.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_ddim.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_karras_ve.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_lms_discrete.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_pndm.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_sde_ve.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/schedulers/scheduling_sde_vp.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-10-14 13:43:52 +02:00
Patrick von Platen e713346ad1
Give more customizable options for safety checker (#815)
* Give more customizable options for safety checker

* Apply suggestions from code review

* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py

* Finish

* make style

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* up

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2022-10-13 15:52:26 +02:00
Anton Lozhkov 26c7df5d82
Fix type mismatch error, add tests for negative prompts (#823) 2022-10-13 15:45:42 +02:00
Patrick von Platen db47b1e4d9
[Dummy imports] Better error message (#795)
* [Dummy imports] Better error message

* Test: load pipeline with LMS scheduler.

Fails with a cryptic message if scipy is not installed.

* Correct

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2022-10-12 14:41:16 +02:00
Patrick von Platen 6bc11782b7
[Img2Img] Fix batch size mismatch prompts vs. init images (#793)
* [Img2Img] Fix batch size mismatch prompts vs. init images

* Remove bogus folder

* fix

* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2022-10-12 13:00:36 +02:00
Patrick von Platen fab17528da
[Low CPU memory] + device map (#772)
* add accelerate to load models with smaller memory footprint

* remove low_cpu_mem_usage as it is reduntant

* move accelerate init weights context to modelling utils

* add test to ensure results are the same when loading with accelerate

* add tests to ensure ram usage gets lower when using accelerate

* move accelerate logic to single snippet under modelling utils and remove it from configuration utils

* format code using to pass quality check

* fix imports with isor

* add accelerate to test extra deps

* only import accelerate if device_map is set to auto

* move accelerate availability check to diffusers import utils

* format code

* add device map to pipeline abstraction

* lint it to pass PR quality check

* fix class check to use accelerate when using diffusers ModelMixin subclasses

* use low_cpu_mem_usage in transformers if device_map is not available

* NoModuleLayer

* comment out tests

* up

* uP

* finish

* Update src/diffusers/pipelines/stable_diffusion/safety_checker.py

* finish

* uP

* make style

Co-authored-by: Pi Esposito <piero.skywalker@gmail.com>
2022-10-10 18:05:49 +02:00
Patrick von Platen f3983d16ee
[Tests] Fix tests (#774)
* Fix tests

* remove bogus file
2022-10-07 19:38:40 +02:00
Suraj Patil 92d7086366
[img2img, inpainting] fix fp16 inference (#769)
* handle dtype in vae and image2image pipeline

* fix inpaint in fp16

* dtype should be handled in add_noise

* style

* address review comments

* add simple fast tests to check fp16

* fix test name

* put mask in fp16
2022-10-07 17:01:51 +02:00
James R T e0fece2b26
Add final latent slice checks to SD pipeline intermediate state tests (#731)
This is to ensure that the final latent slices stay somewhat consistent as more changes are introduced into the library.

Signed-off-by: James R T <jamestiotio@gmail.com>

Signed-off-by: James R T <jamestiotio@gmail.com>
2022-10-07 15:50:20 +02:00
apolinario fdfa7c8f15
Change fp16 error to warning (#764)
* Swap fp16 error to warning

Also remove the associated test

* Formatting

* warn -> warning

* Update src/diffusers/pipeline_utils.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* make style

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-10-07 10:31:52 +02:00
Patrick von Platen ae672d58ef
[Tests] Lower required memory for clip guided and fix super edge-case git pipeline module bug (#754)
* [Tests] Lower required memory

* fix

* up

* uP
2022-10-06 19:15:26 +02:00
Patrick von Platen d9c449ea30
Custome Pipelines (#744)
* [Custom Pipelines]

* uP

* make style

* finish

* finish

* remove ipdb

* upload

* fix

* finish docs

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: apolinario <joaopaulo.passos@gmail.com>

* finish

* final uploads

* remove unnecessary test

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: apolinario <joaopaulo.passos@gmail.com>
2022-10-06 16:54:02 +02:00
Anton Lozhkov 6c64741933
Raise an error when moving an fp16 pipeline to CPU (#749)
* Raise an error when moving an fp16 pipeline to CPU

* Raise an error when moving an fp16 pipeline to CPU

* style

* Update src/diffusers/pipeline_utils.py

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* Update src/diffusers/pipeline_utils.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Improve the message

* cuda

* Update tests/test_pipelines.py

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
2022-10-06 15:51:03 +02:00
Suraj Patil c119dc4c04
allow multiple generations per prompt (#741)
* compute text embeds per prompt

* don't repeat uncond prompts

* repeat separatly

* update image2image

* fix repeat uncond embeds

* adapt inpaint pipeline

* ifx uncond tokens in img2img

* add tests and fix ucond embeds in im2img and inpaint pipe
2022-10-06 14:01:45 +02:00
Patrick von Platen 78744b6a8f
No more use_auth_token=True (#733)
* up

* uP

* uP

* make style

* Apply suggestions from code review

* up

* finish
2022-10-05 17:16:15 +02:00
Nicolas Patry 3dcc75cbd4
Removing `autocast` for `35-25% speedup`. (`autocast` considered harmful). (#511)
* Removing `autocast` for `35-25% speedup`.

* iQuality

* Adding a slow test.

* Fixing mps noise generation.

* Raising error on wrong device, instead of just casting on behalf of user.

* Quality.

* fix merge

Co-authored-by: Nouamane Tazi <nouamane98@gmail.com>
2022-10-05 15:33:13 +02:00
Patrick von Platen f1484b81b0
[Utils] Add deprecate function and move testing_utils under utils (#659)
* [Utils] Add deprecate function

* up

* up

* uP

* up

* up

* up

* up

* uP

* up

* fix

* up

* move to deprecation utils file

* fix

* fix

* fix more
2022-10-03 23:44:24 +02:00
Anton Lozhkov 1070e1a38a
[CI] Speed up slow tests (#708)
* [CI] Localize the HF cache

* pip cache

* de-env

* refactor matrix

* fix fast cache

* less onnx steps

* revert

* revert pip cache

* revert pip cache

* remove debugging trigger
2022-10-03 22:16:23 +02:00
James R T 2558977bc7
Add callback parameters for Stable Diffusion pipelines (#521)
* Add callback parameters for Stable Diffusion pipelines

Signed-off-by: James R T <jamestiotio@gmail.com>

* Lint code with `black --preview`

Signed-off-by: James R T <jamestiotio@gmail.com>

* Refactor callback implementation for Stable Diffusion pipelines

* Fix missing imports

Signed-off-by: James R T <jamestiotio@gmail.com>

* Fix documentation format

Signed-off-by: James R T <jamestiotio@gmail.com>

* Add kwargs parameter to standardize with other pipelines

Signed-off-by: James R T <jamestiotio@gmail.com>

* Modify Stable Diffusion pipeline callback parameters

Signed-off-by: James R T <jamestiotio@gmail.com>

* Remove useless imports

Signed-off-by: James R T <jamestiotio@gmail.com>

* Change types for timestep and onnx latents

* Fix docstring style

* Return decode_latents and run_safety_checker back into __call__

* Remove unused imports

* Add intermediate state tests for Stable Diffusion pipelines

Signed-off-by: James R T <jamestiotio@gmail.com>

* Fix intermediate state tests for Stable Diffusion pipelines

Signed-off-by: James R T <jamestiotio@gmail.com>

Signed-off-by: James R T <jamestiotio@gmail.com>
2022-10-02 19:56:36 +02:00
Josh Achiam a784be2ebe
Allow resolutions that are not multiples of 64 (#505)
* Allow resolutions that are not multiples of 64

* ran black

* fix bug

* add test

* more explanation

* more comments

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-09-30 09:54:40 +02:00
Kashif Rasul bd8df2da89
[Pytorch] Pytorch only schedulers (#534)
* pytorch only schedulers

* fix style

* remove match_shape

* pytorch only ddpm

* remove SchedulerMixin

* remove numpy from karras_ve

* fix types

* remove numpy from lms_discrete

* remove numpy from pndm

* fix typo

* remove mixin and numpy from sde_vp and ve

* remove remaining tensor_format

* fix style

* sigmas has to be torch tensor

* removed set_format in readme

* remove set format from docs

* remove set_format from pipelines

* update tests

* fix typo

* continue to use mixin

* fix imports

* removed unsed imports

* match shape instead of assuming image shapes

* remove import typo

* update call to add_noise

* use math instead of numpy

* fix t_index

* removed commented out numpy tests

* timesteps needs to be discrete

* cast timesteps to int in flax scheduler too

* fix device mismatch issue

* small fix

* Update src/diffusers/schedulers/scheduling_pndm.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-09-27 15:27:34 +02:00
Grigory Sizov 35e9209601
Fix formula for noise levels in Karras scheduler and tests (#627)
fix formula for noise levels in karras scheduler and tests
2022-09-24 18:24:08 +02:00
Anton Lozhkov 6bd005ebbe
[ONNX] Collate the external weights, speed up loading from the hub (#610) 2022-09-21 22:26:30 +02:00
Mishig Davaadorj 8685699392
Mv weights name consts to diffusers.utils (#605) 2022-09-21 11:30:14 +02:00
Anton Lozhkov 8a6833b85c
Add the K-LMS scheduler to the inpainting pipeline + tests (#587)
* Add the K-LMS scheduler to the inpainting pipeline + tests

* Remove redundant casts
2022-09-20 19:10:44 +02:00
Patrick von Platen e5902ed11a
[Download] Smart downloading (#512)
* [Download] Smart downloading

* add test

* finish test

* update

* make style
2022-09-16 19:32:40 +02:00
Anton Lozhkov c1796efd5f
Quick fix for the img2img tests (#530)
* Quick fix for the img2img tests

* Remove debug lines
2022-09-16 15:52:26 +02:00
Anton Lozhkov c727a6a5fb
Finally fix the image-based SD tests (#509)
* Finally fix the image-based SD tests

* Remove autocast

* Remove autocast in image tests
2022-09-16 14:37:12 +02:00
Kashif Rasul f4781a0b27
update expected results of slow tests (#268)
* update expected results of slow tests

* relax sum and mean tests

* Print shapes when reporting exception

* formatting

* fix sentence

* relax test_stable_diffusion_fast_ddim for gpu fp16

* relax flakey tests on GPU

* added comment on large tolerences

* black

* format

* set scheduler seed

* added generator

* use np.isclose

* set num_inference_steps to 50

* fix dep. warning

* update expected_slice

* preprocess if image

* updated expected results

* updated expected from CI

* pass generator to VAE

* undo change back to orig

* use orignal

* revert back the expected on cpu

* revert back values for CPU

* more undo

* update result after using gen

* update mean

* set generator for mps

* update expected on CI server

* undo

* use new seed every time

* cpu manual seed

* reduce num_inference_steps

* style

* use generator for randn

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-09-12 15:49:39 +02:00
Patrick von Platen f55190b275
[Tests] Correct image folder tests (#427)
* [Tests] Correct image folder tests

* up
2022-09-08 16:45:29 +02:00