Mirrors
/
EveryDream2trainer
mirror of https://github.com/victorchall/EveryDream2trainer.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

caption cog notebook2

This commit is contained in:
Victor Hall 2024-03-24 10:00:00 -04:00
parent 93659f3eb4
commit 87bfe652ce
1 changed files with 8 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
CaptionCog.ipynb
View File

@ -6,14 +6,16 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"# Cog Captioning\n", "# Cog Captioning\n",
"<a href=\"https://colab.research.google.com/github/nawnie/EveryDream2trainer/blob/main/CaptionCog.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"This notebook is an implementation of [CogVLM](https://github.com/THUDM/CogVLM) for image captioning. \n", "This notebook is an implementation of [CogVLM](https://github.com/THUDM/CogVLM) for image captioning. \n",
"\n", "\n",
"This may require HIGH RAM shape on Google Colab, but T4 16gb is enough (even if slow).\n", "Read [Docs](doc/CAPTION_COG.md) for basic usage guide. \n",
"\n", "\n",
"1. Read [Docs](doc/CAPTION_COG.md) for basic usage guide. \n", "Open in [Google Colab](https://colab.research.google.com/github/victorchall/EveryDream2trainer/blob/main/CaptionCog.ipynb) **OR** use Runpod/Vast/Whatever using the EveryDream2trainer docker container/template and open this notebook.\n",
"2. Open in [Google Colab](https://colab.research.google.com/github/victorchall/EveryDream2trainer/blob/main/CaptionCog.ipynb) **OR** Runpod/Vast using the EveryDream2trainer docker container/template and open this notebook.\n", "\n",
"3. Run the cells below to install dependencies.\n", "### Requirements\n",
"4. Place your images in \"input\" folder or change the data_root to point to a Gdrive folder." "Ampere or newer GPU with 16GB+ VRAM (ex. A100, 3090, 4060 Ti 16GB, etc). \n",
"The 4bit quantization loading requires bfloat16 datatype support, which is not supported on older Turing GPUs like the T4 16GB, which rules out using free tier Google Colab.\n"
] ]
}, },
{ {
@ -95,7 +97,7 @@
"## Run captions.\n", "## Run captions.\n",
"\n", "\n",
"Place your images in \"input\" folder, or you can change the image_dir to point to a Gdrive folder.\n", "Place your images in \"input\" folder, or you can change the image_dir to point to a Gdrive folder.\n",
"\n" "Note: Colab may complain that you are running out of disk space, but it should still work.\n"
] ]
}, },
{ {