From d35688bf191b01477ad303664fef8ac1396a262b Mon Sep 17 00:00:00 2001 From: frostydad <64224601+Cyberes@users.noreply.github.com> Date: Sat, 1 Oct 2022 15:32:18 -0600 Subject: [PATCH] organize Former-commit-id: af6cfdb3fff1d5050d897aa99573c121eb03f037 [formerly 4a5e84afffe2a473b56c3cff210227daedc6f6b4] Former-commit-id: d17fbb4d238eeebc9482a04adfd12c738f6ff91b --- ...> Cyberes_Textual_Inversion_Training.ipynb | 0 README.md | 22 ++- docs/Paperspace Guide for Idiots.md | 77 ++++++++++ other/README.md | 31 ++++ other/generate_tx_models_html.py | 137 ------------------ .../voldy's_colab_paperspace_09-07-2022.ipynb | 0 ...u-diffusion_quick_n_dirty_09-07-2022.ipynb | 0 ...ion_webui_hlky_paperspace_09-07-2022.ipynb | 0 8 files changed, 122 insertions(+), 145 deletions(-) rename Textual Inversion Training.ipynb => Cyberes_Textual_Inversion_Training.ipynb (100%) create mode 100644 docs/Paperspace Guide for Idiots.md create mode 100644 other/README.md delete mode 100644 other/generate_tx_models_html.py rename {old => other/old}/voldy's_colab_paperspace_09-07-2022.ipynb (100%) rename {old => other/old}/waifu-diffusion_quick_n_dirty_09-07-2022.ipynb (100%) rename stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb => other/stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb (100%) diff --git a/Textual Inversion Training.ipynb b/Cyberes_Textual_Inversion_Training.ipynb similarity index 100% rename from Textual Inversion Training.ipynb rename to Cyberes_Textual_Inversion_Training.ipynb diff --git a/README.md b/README.md index 8e16bcd..855094b 100644 --- a/README.md +++ b/README.md @@ -4,25 +4,31 @@ _Google Colab notebooks ported to standard Jupiter for Paperspace._ Don't post bug reports on the Kazakhstan rock collection forum. Create an issue here. - ### Getting Started [Docs/Paperspace Guide for Idiots.md](https://github.com/Engineer-of-Stuff/stable-diffusion-paperspace/blob/main/Docs/Paperspace%20Guide%20for%20Retards.md) ### Notebook Descriptions -**StableDiffusionUI_Voldemort_paperspace.ipynb** +#### StableDiffusionUI_Voldemort_paperspace.ipynb -Voldemort's webUI. His repository is updated frequently so the notebook will update the local copy if it's already installed. Can run the normal model or waifu-diffusion. +AUTOMATIC1111's webUI. Designed to get a newbie set up, features lots of error checking and automation. +#### Cyberes_Textual_Inversion_Training.ipynb +All the commands you need to do create a textual inversion embedding. Does not hold your hand or guide you through the process. Any issues related to this notebook will be closed. -**CodeFormer_Inference_Simplified.ipynb** +#### lfs/latent-diffusion -CodeFormer face fixer standalone. Can process images in bulk. +Holds stuff for latent-diffusion. Probably not needed but latent-diffusion had them stored on some random file hosting service. +#### /other Directory +Other notebooks and code. -**stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb** - -hlky's webui +- CodeFormer_Inference_Simplified.ipynb +- stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb +- sd-concepts-library_Exporter.ipynb +- Huggingface Textual Inversion Training.ipynb +- download_sd-concepts-library.py +- Misc. old stuff diff --git a/docs/Paperspace Guide for Idiots.md b/docs/Paperspace Guide for Idiots.md new file mode 100644 index 0000000..be200f2 --- /dev/null +++ b/docs/Paperspace Guide for Idiots.md @@ -0,0 +1,77 @@ +# Getting Started on Paperspace for Retards <3 + +![paperspace](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/paperspace.png) + +**What is Paperspace?** + +Basically, a cloud compute service for AI development. You run your code on their hardware. + +**Why Paperspace?** + +Paperspace is convenient, free, and you probably don't have a computer good enough to run the AI. They're aimed at professionals whereas Colab is literally for children. + +You'll run your code on their powerful GPUs which is much faster and easier than locally on your computer. + +**Do I have to pay?** + +Only if you feel limited by their free tier. You should have everything you need to generate ~~porn~~ really neat images. I'm pretty Paperspace is cheaper than Google Colab Pro. + +**Why not Google Colab?** + +Google Colab is an alternative, but their free tier is more restrictive than Paperspace's and has less powerful hardware. And, knowing Google, they probably record what you generate. + +**But I want to run it on Google Colab!** + +Ok fine. [Here's the official colab notebook by Voldy himself.](https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxueGij8phCyluOh) + +**But I'd rather run it on my own computer!** + +Running it in the cloud is much easier and you're less likely to mess up your OS, but since you insist here's a Docker container to make is simpler: [AbdBarho/stable-diffusion-webui-docker](https://github.com/AbdBarho/stable-diffusion-webui-docker) + +## Let's do it! + +1. [Create an Account](https://console.paperspace.com/signup) You will be asked for your phone number. **You can use a VOIP number such as Google Voice, they don't block it!** + +2. Click this button and fill out the form that pops up + +![create_project](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/create_project.png) + +3. Click this button to create a notebook + +![create_notebook](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/create_notebook.png) + +4. Under "Runtime" select "Start from Stratch" + +![start_scratch.png](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/start_scratch.png) + +5. Download [StableDiffusionUI_Voldemort_paperspace.ipynb](https://github.com/Engineer-of-Stuff/stable-diffusion-paperspace/blob/main/StableDiffusionUI_Voldemort_paperspace.ipynb) + +6. Upload that file to your notebook + +![upload](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/upload.png) + +7. Follow directions + +![final](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/final.png) + +## Launch the Jupyter Notebook + +Click this symbol in the left vertical menubar. + +![open-jupiter](https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/docs/assets/open-jupiter.png) + +You can access the terminal in the Jupyter Notebook. + +## I need more help :( + +Here's some other guides. + +- [FINAL GUI RETARD GUIDE](https://rentry.org/voldy) +- [Using the WebUI](https://rentry.org/voldy) +- [Using the Inpainter](https://rentry.org/drfar) +- [Textual Inversion](https://rentry.org/aikgx) +- [Crowd-Sourced Prompts](https://lexica.art/) +- [Artist Name Prompts](https://sgreens.notion.site/sgreens/4ca6f4e229e24da6845b6d49e6b08ae7?v=fdf861d1c65d456e98904fe3f3670bd3) + +[Images hosted on Github](https://github.com/Engineer-of-Stuff/stable-diffusion-paperspace) +[Github Mirror](https://github.com/Engineer-of-Stuff/stable-diffusion-paperspace/blob/main/docs/archives/Getting%20Started%20on%20Paperspace.pdf) diff --git a/other/README.md b/other/README.md new file mode 100644 index 0000000..b89deb8 --- /dev/null +++ b/other/README.md @@ -0,0 +1,31 @@ +# Other + +#### CodeFormer_Inference_Simplified.ipynb + +CodeFormer face fixer standalone. Can process images in bulk. + +#### stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb + +An alternative to AUTOMATIC1111's WebUI from a guy named hlky. Ported to Paperspace. + +#### sd-concepts-library_Exporter.ipynb + +Download all textual inversion embeddings in huggingface.co/sd-concepts-library. May or may not work. + +#### Huggingface Textual Inversion Training.ipynb + +Train textual inversion embeddings using the HuggingFace API. Doesn't work as well as my `Textual Inversion Training.ipynb` notebook. + +#### download_sd-concepts-library.py + +Basically `sd-concepts-library_Exporter.ipynb` but a Python script. Can download restricted models. + +# Old + +#### voldy's_colab_paperspace_09-07-2022.ipynb + +First iteration of the WebUI for Paperspace. + +#### waifu-diffusion_quick_n_dirty_09-07-2022.ipynb + +Quick 'n dirty notebook made to quickly test Waifu Diffusion the day it was released. diff --git a/other/generate_tx_models_html.py b/other/generate_tx_models_html.py deleted file mode 100644 index 703d326..0000000 --- a/other/generate_tx_models_html.py +++ /dev/null @@ -1,137 +0,0 @@ -import argparse -import datetime -import os -import shutil -import sys -from urllib import request as ulreq - -import requests -from huggingface_hub import HfApi -from PIL import ImageFile - - -def getsizes(uri): - # https://stackoverflow.com/a/37709319 - # get file size *and* image size (None if not known) - file = ulreq.urlopen(uri) - size = file.headers.get("content-length") - if size: - size = int(size) - p = ImageFile.Parser() - while True: - data = file.read(1024) - if not data: - break - p.feed(data) - if p.image: - return size, p.image.size - break - file.close() - return (size, None) - - -parser = argparse.ArgumentParser() -parser.add_argument('out_file', nargs='?', help='file to save to') -args = parser.parse_args() - -print('Will save to file:', args.out_file) - -# Get list of models under the sd-concepts-library organization -print('Getting list of models...') -api = HfApi() -models_list = [] -for model in api.list_models(author="sd-concepts-library"): - models_list.append(model.modelId.replace('sd-concepts-library/', '')) -models_list.sort() - -html_struct = """ - - - Stable Diffusion Texual Inversion Models - - - - - - -
-

Stable Diffusion Texual Inversion Models

-

Page updates daily. Last updated {datetime.datetime.now().strftime("%A, %B %d %Y")}.

- -

- Generated from huggingface.co/sd-concepts-library -

- -

- Downloaded models are straight from the HuggingFace repositories and are named learned_embeds.bin. Rename to model_name.pt -

- - -
-""" - -i = 1 -for model_name in models_list: - - if i == 3: - break - - print(f'{i}/{len(models_list)} -> {model_name}') - # if os.path.exists(f'{model_name}/learned_embeds.bin'): # double check the file exists since sometimes it hasn't been uploaded yet - # shutil.move(f'{model_name}/learned_embeds.bin', f'{model_name}/{model_name}.pt') - # pass - # else: - # continue - - # Images can be in a few different formats, figure out which one it's in - img_type = None - img_width = None - for type in ['jpeg', 'png', 'jpg']: - r = requests.head(f'https://huggingface.co/sd-concepts-library/{model_name}/resolve/main/concept_images/0.{type}', allow_redirects=True) - if r.status_code == 200: - img_type = type - img_width = getsizes(f'https://huggingface.co/sd-concepts-library/{model_name}/resolve/main/concept_images/0.{type}')[1][0] - break - - html_struct = html_struct + f"""

{model_name}

-

Download {model_name}

-

View Repository

-
-
- -
-
- -
-
- -
-
""" - i = i + 1 - -html_struct = html_struct + '
' - -f = open(args.out_file, 'w') -f.write(html_struct) -f.close() diff --git a/old/voldy's_colab_paperspace_09-07-2022.ipynb b/other/old/voldy's_colab_paperspace_09-07-2022.ipynb similarity index 100% rename from old/voldy's_colab_paperspace_09-07-2022.ipynb rename to other/old/voldy's_colab_paperspace_09-07-2022.ipynb diff --git a/old/waifu-diffusion_quick_n_dirty_09-07-2022.ipynb b/other/old/waifu-diffusion_quick_n_dirty_09-07-2022.ipynb similarity index 100% rename from old/waifu-diffusion_quick_n_dirty_09-07-2022.ipynb rename to other/old/waifu-diffusion_quick_n_dirty_09-07-2022.ipynb diff --git a/stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb b/other/stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb similarity index 100% rename from stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb rename to other/stable_diffusion_webui_hlky_paperspace_09-07-2022.ipynb