Mirrors
/
diffusers
mirror of https://github.com/ShivamShrirao/diffusers.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Reproducible images by supplying latents to pipeline (#247) 

* Accept latents as input for StableDiffusionPipeline.

* Notebook to demonstrate reusable seeds (latents).

* More accurate type annotation

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Review comments: move to device, raise instead of assert.

* Actually commit the test notebook.

I had mistakenly pushed an empty file instead.

* Adapt notebook to Colab.

* Update examples readme.

* Move notebook to personal repo.

Co-authored-by: Suraj Patil <surajp815@gmail.com>
This commit is contained in:
Pedro Cuenca 2022-08-25 15:47:05 +02:00 committed by GitHub
parent 89793a97e2
commit bfe37f3159
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 18 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
examples/inference/readme.md
View File

@ -47,4 +47,8 @@ with autocast("cuda"):
images[0].save("fantasy_landscape.png")
```
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/Notebooks/blob/master/image_2_image_using_diffusers.ipynb)
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/Notebooks/blob/master/image_2_image_using_diffusers.ipynb)
## Tweak prompts reusing seeds and latents
You can generate your own latents to reproduce results, or tweak your prompt on a specific result you liked. [This notebook](stable-diffusion-seeds.ipynb) shows how to do it step by step. You can also run it in Google Colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pcuenca/diffusers-examples/blob/main/notebooks/stable-diffusion-seeds.ipynb).

19
src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
View File

@ -46,6 +46,7 @@ class StableDiffusionPipeline(DiffusionPipeline):
guidance_scale: Optional[float] = 7.5,
eta: Optional[float] = 0.0,
generator: Optional[torch.Generator] = None,
latents: Optional[torch.FloatTensor] = None,
output_type: Optional[str] = "pil",
**kwargs,
):
@ -98,12 +99,18 @@ class StableDiffusionPipeline(DiffusionPipeline):
# to avoid doing two forward passes
text_embeddings = torch.cat([uncond_embeddings, text_embeddings])
# get the intial random noise
latents = torch.randn(
(batch_size, self.unet.in_channels, height // 8, width // 8),
generator=generator,
device=self.device,
)
# get the initial random noise unless the user supplied it
latents_shape = (batch_size, self.unet.in_channels, height // 8, width // 8)
if latents is None:
latents = torch.randn(
latents_shape,
generator=generator,
device=self.device,
)
else:
if latents.shape != latents_shape:
raise ValueError(f"Unexpected latents shape, got {latents.shape}, expected {latents_shape}")
latents = latents.to(self.device)
# set timesteps
accepts_offset = "offset" in set(inspect.signature(self.scheduler.set_timesteps).parameters.keys())