clean up, add more doc

StableDiffusionUI_Voldemort_paperspace.ipynb

@@ -21,7 +21,10 @@
     "- [Using the Inpainter](https://rentry.org/drfar)\n",
     "- [Textual Inversion](https://rentry.org/aikgx)\n",
     "- [Crowd-Sourced Prompts](https://lexica.art/)\n",
-    "- [Artist Name Prompts](https://sgreens.notion.site/sgreens/4ca6f4e229e24da6845b6d49e6b08ae7?v=fdf861d1c65d456e98904fe3f3670bd3)"
+    "- [Artist Name Prompts](https://sgreens.notion.site/sgreens/4ca6f4e229e24da6845b6d49e6b08ae7?v=fdf861d1c65d456e98904fe3f3670bd3)\n",
+    "- [Stable Diffusion Models](https://cyberes.github.io/stable-diffusion-models)\n",
+    "- [Textual Inversion Models](https://cyberes.github.io/stable-diffusion-textual-inversion-models/)\n",
+    "- [Have I Been Trained?](https://haveibeentrained.com/)"
    ]
   },
   {
@@ -32,7 +35,7 @@
    "source": [
     "## Installation and Setup\n",
     "\n",
-    "You must reinstall everything each time you restart the machine. If already downloaded dependencies will be auto-updated."
+    "You must reinstall everything each time you restart the machine. If already downloaded, dependencies will be auto-updated."
    ]
   },
   {
@@ -127,7 +130,11 @@
     "    print('GFPGANv1.3.pth already downloaded')\n",
     "    \n",
     "# Make sure your models storage directory exists\n",
-    "!mkdir -p $model_storage_dir"
+    "!mkdir -p $model_storage_dir\n",
+    "\n",
+    "# Link the output folder to /notebooks/outputs\n",
+    "!mkdir -p /notebooks/stable-diffusion-webui/outputs\n",
+    "!ln -s /notebooks/stable-diffusion-webui/outputs /notebooks/outputs"
    ]
   },
   {
@@ -145,15 +152,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "I've provided a few different ways of aquiring the models. Try the torrent option first.\n",
+    "I've provided a few different ways of aquiring the models. Try the torrent option first. You don't need to repeat this step if you've already downloaded the models.\n",
     "\n",
-    "You don't need to repeat this step if you've already downloaded the models.\n",
+    "There are a few additional models available here: https://cyberes.github.io/stable-diffusion-models\n",
+    "\n",
+    "If you're interested in textual inversion, here's the database: https://cyberes.github.io/stable-diffusion-textual-inversion-models\n",
     "\n",
     "**Filesize and Storage Disclaimer**\n",
     "\n",
-    "Paperspace free tier has only 5GB of storage space. The standard model is 4GB and waifu is 3.5GB/7.2GB. To get around this limited storage capacity we will download the models to `/storage/`, a non-persistent directory. Each time you restart the machine you will have to redownload the models.\n",
+    "Paperspace free tier has only 5GB of storage space. The standard model is 4GB and waifu is 3.5GB/7.2GB. There's really not a way to get around this limitation, but here's a few suggestions.\n",
-    "\n",
-    "If you don't want to have to redownload the models every time you restart your machine you have three options:\n",
     "1. Add a payment method to your account. Storage overages are billed at \\$0.29/GB and billing occurs monthly and runs at midnight on the first of each month. With a payment method on file, Paperspace will let you use more storage and if you time it right you shouldn't actually be charged for it.\n",
     "2. Upgrade to a Pro account. They'll give you 15GB and you'll get longer runtimes and more powerful free GPUs.\n",
     "3. Use my referral code `KQLRH37` You'll get \\$10 credit that you should be able to put towards the storage overage charges. Redeem the code at the bottom of the Billing page.\n",
@@ -336,7 +343,7 @@
     "# Get some storage back\n",
     "!pip cache purge\n",
     "!rm $model_storage_dir/*.aria2\n",
-    "!apt remove --purge -y aria2\n",
+    "!apt remove --purge -y aria2 p7zip-full\n",
     "!apt autoremove --purge -y\n",
     "!apt clean\n",
     "\n",
@@ -395,9 +402,13 @@
    "source": [
     "Run this block to launch the WebUI. You will get a link to nnn.gradio.app, that's your WebUI. Follow it.\n",
     "\n",
-    "If you have any issues, restart the kernel.\n",
+    "If you have a lot of VRAM and aren't generating large images you can add the flag `--disable-opt-split-attention` to disable VRAM optimizations for a speed boost.\n",
     "\n",
-    "If you have a lot of VRAM and aren't generating large images you can add the flag `--disable-opt-split-attention` to disable VRAM optimizations for a speed boost."
+    "**Troubleshooting**\n",
+    "\n",
+    "- If you have any issues, try restarting the kernel.\n",
+    "- `EOFError: Ran out of input` probably means you ran out of storage space and the model `.ckpt` file wasn't downloaded completely. Try cleaning up your files or you can reset your storage with the _Reset Storage_ block in the Tools section below.\n",
+    "- If you're having issues with your results not loading, try running an ngrok proxy. More details in the Tools section below."
    ]
   },
   {
@@ -428,7 +439,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This block will rename and compress the outputs with 7zip max compression. It expects you to have `log/` and `outputs/` in `/notebooks/`."
+    "This block will rename and compress the outputs with 7zip max compression. It expects you to have `log/` and `outputs/` in `/notebooks/stable-diffusion-webui/`."
    ]
   },
   {
@@ -443,14 +454,12 @@
     "!apt update\n",
     "!apt install -y p7zip-full\n",
     "from datetime import datetime\n",
-    "import os\n",
     "datetime_str = datetime.now().strftime('%m-%d-%Y_%H:%M:%S')\n",
     "%cd /notebooks/\n",
     "!mkdir $datetime_str\n",
-    "!mv log $datetime_str\n",
+    "!mv /notebooks/stable-diffusion-webui/log $datetime_str\n",
-    "!mv outputs $datetime_str\n",
+    "!mv /notebooks/stable-diffusion-webui/outputs/* $datetime_str\n",
-    "print('Compressing, please wait...')\n",
+    "!7z a -t7z -m0=lzma2 -mx=9 -mfb=64 -md=32m -ms=on {datetime_str}.7z {datetime_str}/"
-    "os.system(f'7z a -t7z -m0=lzma2 -mx=9 -mfb=64 -md=32m -ms=on {datetime_str}.7z {datetime_str}/')"
    ]
   },
   {
@@ -488,7 +497,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "**Show graphics card info**"
+    "### Show graphics card info"
    ]
   },
   {
@@ -504,7 +513,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "**Download the latest version of this notebook from Github**\n",
+    "### Download the latest version of this notebook from Github\n",
     "\n",
     "Run this and refresh the page (press F5). Don't save anything or you will overwrite the downloaded file."
    ]
@@ -515,9 +524,72 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!mv /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb.backup\n",
+    "!mv /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb.backup # save your old notebook to a backup\n",
     "!wget https://raw.githubusercontent.com/Engineer-of-Stuff/stable-diffusion-paperspace/main/StableDiffusionUI_Voldemort_paperspace.ipynb -O /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Reset Repository\n",
+    "\n",
+    "Sometimes AUTOMATIC1111 breaks something. Go to https://github.com/AUTOMATIC1111/stable-diffusion-webui/commits/master and choose a commit to revert to."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!git reset --hard <commit>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Delete all files and reset storage\n",
+    "\n",
+    "This will delete ALL your files in `/notebooks/` and `/storage/`. Use if you're having issues with zero storage space and you don't want to delete your notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment the lines below to run this block. You can highlight the lines and do ctrl + /\n",
+    "# !rm -rf /storage/*\n",
+    "# !mv /notebooks/StableDiffusionUI_Voldemort_paperspace.ipynb / # move the notebook out of the directory before we nuke it\n",
+    "# !rm -rf /notebooks/*\n",
+    "# !mv /StableDiffusionUI_Voldemort_paperspace.ipynb /notebooks/ # move it back"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### ngrok Proxy\n",
+    "\n",
+    "There's an issue with Gradio's infrastructure where you can't load an image larger than 2MB. We can get around this by running our own proxy with ngrok. Here's how to do it:\n",
+    "\n",
+    "1. Add `--port 7860` to the `!python ... webui.py` line in the Launch WebUI section above.\n",
+    "2. Launch the WebUI.\n",
+    "3. Go to [ngrok.com](https://ngrok.com/) in a new tab and sign up. Don't close that tab yet.\n",
+    "4. Open the terminal by clicking on the terminal button in the left sidebar menu. If you're not sure which one that is, hover over the items and a little tooltip will appear.\n",
+    "5. Run this in the terminal. Make sure to replace `<your authtoken>` with your auth token.\n",
+    "\n",
+    "```bash\n",
+    "pip install pyngrok\n",
+    "ngrok authtoken <your authtoken>\n",
+    "ngrok http 7860\n",
+    "```\n",
+    "\n",
+    "ngrok will assign you a subdomain and display a link to it. That's the link to your WebUI instance. If you want, create a new file named `start-ngrok.sh` and copy those lines into it so you can quickly connect."
+   ]
   }
  ],
  "metadata": {