Add stable diffusion example #1149

sumanthgenz · 2022-09-02T04:02:04Z

Add a Dockerfile needed to create docker environment to run a stable diffusion web tool on GPU.
Add a yaml to launch an instance with a GPU to run said Dockerfile and run the stable diffusion tool (using ssh with port forwarding).

concretevitamin

Thanks @sumanthgenz! I tried it out and it works great. Some comments.

concretevitamin · 2022-09-02T17:22:12Z

examples/stable_diffusion_docker.yaml

+
+
+resources:
+  cloud: azure


Can we leave this out? So people have flexibility.

concretevitamin · 2022-09-02T17:31:17Z

examples/stable_diffusion_docker.yaml

+  accelerators: V100:1
+
+file_mounts:
+  /stable_diffusion: ./docker/stable_diffusion


Change the source (RHS) to .? See the other comment for dir structure.

concretevitamin · 2022-09-02T17:32:16Z

examples/stable_diffusion_docker.yaml

@@ -0,0 +1,35 @@
+
+# Steps to run this example:


This is great! Can we also put them in a README.md? Can even add Step 0: pip install skypilot and point to our Installation docs page. Maybe some title like "Run your own Stable Diffusion UI in the cloud with one click"?

examples/stable_diffusion/ README.md this.yaml Dockerfile

Goal is for people to come to this dir, look at README and immediately get started.

+1, we should move this to examples/stable_diffusion

concretevitamin · 2022-09-02T17:45:18Z

examples/docker/stable_diffusion/Dockerfile

@@ -0,0 +1,58 @@
+# Note: this file is from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile 


Seems like this file has changes on top of the referred to file. Could we describe those changes here? Wondering if we can take that existing Dockerfile and run it with SkyPilot.

The docker in the instance cannot compile the original dockerfile from the repo. I removed <<EOF heredoc to fix that, so that was the only change. I will specify that we made that change.

concretevitamin · 2022-09-02T17:51:50Z

examples/docker/stable_diffusion/Dockerfile

+RUN  python /info.py /stable-diffusion/frontend/frontend.py
+
+WORKDIR /stable-diffusion
+ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch CLI_ARGS="" \


Are we using the Stable Diffusion ckpt sd-v1-4.ckpt, or the models downloaded on L44-46?

We are using sd-v1-4.ckpt

concretevitamin · 2022-09-02T17:55:12Z

In the README, it'd also be great to include some (setting, prompt) combos that generate great-looking pics. (I tried my own prompts and they didn't work out so great.)

romilbhardwaj

Just ran on GCP V100, looks great!

romilbhardwaj · 2022-09-02T19:25:20Z

examples/stable_diffusion_docker.yaml

@@ -0,0 +1,35 @@
+
+# Steps to run this example:


+1, we should move this to examples/stable_diffusion

romilbhardwaj · 2022-09-02T19:56:57Z

examples/stable_diffusion_docker.yaml

+
+run: |
+  cd stable-diffusion-webui-docker
+  docker-compose up --build


is the --build required? We already run docker-compose build in setup

good point, i will remove it

romilbhardwaj · 2022-09-02T19:57:43Z

examples/docker/stable_diffusion/Dockerfile

+# download dev UI version, update the sha below in case you want some other version
+# RUN <<EOF
+# git clone https://github.com/hlky/stable-diffusion-webui.git
+# cd stable-diffusion-webui
+# # map to this file: https://github.com/hlky/stable-diffusion-webui/blob/master/.github/sync.yml
+# git reset --hard 49e6178fd82ca736f9bbc621c6b12487c300e493
+# cp -t /stable-diffusion/scripts/ webui.py relauncher.py txt2img.yaml
+# cp -t /stable-diffusion/configs/webui webui.yaml
+# cp -t /stable-diffusion/frontend/ frontend/*
+# cd / && rm -rf stable-diffusion-webui
+# EOF


If not used, should we just remove this? Or maybe make the comments more descriptive - right now its not clear what this code block is for.

romilbhardwaj

The readme looks great! Will be very useful for new users.

romilbhardwaj · 2022-09-06T22:59:29Z

examples/stable_diffusion/README.md

+
+2. Run sky launch -c stable-diffusion stable_diffusion_docker.yaml
+
+3. Run ssh -L 7860:localhost:7860 stable-diffusion


Add codeblock formatting:

Run `ssh -L 7860:localhost:7860 stable-diffusion`

romilbhardwaj · 2022-09-06T23:00:32Z

examples/stable_diffusion/README.md

+
+1. Install skypilot by following these [instructions](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)
+
+2. Run sky launch -c stable-diffusion stable_diffusion_docker.yaml


perhaps add a block git clone and cd to this directory:

git clone https://github.com/skypilot-org/skypilot.git && cd examples/stable_diffusion sky launch -c stable-diffusion stable_diffusion_docker.yaml

The idea is to have a copy-paste-able readme :)

romilbhardwaj · 2022-09-06T23:16:39Z

examples/stable_diffusion/README.md

@@ -0,0 +1,33 @@
+## Setup
+
+1. Install skypilot by following these [instructions](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)


Nit: Add a period at the end of bullet points that are complete sentences

romilbhardwaj · 2022-09-06T23:17:09Z

examples/stable_diffusion/README.md

+
+3. Run ssh -L 7860:localhost:7860 stable-diffusion
+
+4. Open http://localhost:7860/ in browser.


Add codeblock formatting:

Open [`http://localhost:7860/`](http://localhost:7860/) in browser.

romilbhardwaj · 2022-09-06T23:19:48Z

examples/stable_diffusion/README.md

+
+4. Open http://localhost:7860/ in browser.
+
+5. Type in text prompt and click "Generate" 


Nit: Add a period at the end of bullet points that are complete sentences

examples/stable_diffusion/README.md

romilbhardwaj · 2022-09-06T23:42:08Z

examples/stable_diffusion/README.md

+4. Open http://localhost:7860/ in browser.
+
+5. Type in text prompt and click "Generate" 
+


Add:

6. Once you are done, run `sky down stable-diffusion` to terminate the VM.

romilbhardwaj · 2022-09-06T23:42:50Z

examples/stable_diffusion/Dockerfile

@@ -1,4 +1,6 @@
-# Note: this file is from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile 
+# Note: this file is modified from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile 


*adapted from

romilbhardwaj · 2022-09-06T23:44:03Z

examples/stable_diffusion/stable_diffusion_docker.yaml

@@ -0,0 +1,24 @@
+# To launch a 1 GPU instance to run stable diffusion web tool.


Nit: SkyPilot YAML run stable diffusion web tool on 1 V100 GPU.

romilbhardwaj · 2022-09-07T00:20:25Z

examples/stable_diffusion/README.md

+   - "donkey playing poker"
+   - "UC Berkeley student writing code on a laptop"
+   - "Marvel vs. DC"
+   - "toilet human"


Maybe let's remove this one :)

romilbhardwaj

Looks good! The example works well - we can iterate on the README as required later.

add stable diffusion basic launcher and runner example

df84ee4

concretevitamin reviewed Sep 2, 2022

View reviewed changes

romilbhardwaj reviewed Sep 2, 2022

View reviewed changes

fix linting

5c61d1b

sumanthgenz requested review from concretevitamin and romilbhardwaj and removed request for concretevitamin and romilbhardwaj September 3, 2022 21:43

fix readme

45a349b

romilbhardwaj reviewed Sep 6, 2022

View reviewed changes

romilbhardwaj reviewed Sep 7, 2022

View reviewed changes

Sumanth added 2 commits September 6, 2022 17:33

address PR comments

4e1f531

fix numbering and path to image in readme

4fb6a2b

romilbhardwaj approved these changes Sep 8, 2022

View reviewed changes

concretevitamin merged commit 3f8f6d4 into skypilot-org:master Sep 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add stable diffusion example #1149

Add stable diffusion example #1149

sumanthgenz commented Sep 2, 2022

concretevitamin left a comment

concretevitamin Sep 2, 2022

concretevitamin Sep 2, 2022

concretevitamin Sep 2, 2022

romilbhardwaj Sep 2, 2022

concretevitamin Sep 2, 2022

sumanthgenz Sep 3, 2022

concretevitamin Sep 2, 2022

sumanthgenz Sep 3, 2022

concretevitamin commented Sep 2, 2022

romilbhardwaj left a comment

romilbhardwaj Sep 2, 2022

romilbhardwaj Sep 2, 2022

sumanthgenz Sep 3, 2022

romilbhardwaj Sep 2, 2022

romilbhardwaj left a comment

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 6, 2022

romilbhardwaj Sep 7, 2022

romilbhardwaj left a comment

		@@ -0,0 +1,58 @@
		# Note: this file is from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile


		2. Run sky launch -c stable-diffusion stable_diffusion_docker.yaml

		3. Run ssh -L 7860:localhost:7860 stable-diffusion


		1. Install skypilot by following these [instructions](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)

		2. Run sky launch -c stable-diffusion stable_diffusion_docker.yaml

		@@ -0,0 +1,33 @@
		## Setup

		1. Install skypilot by following these [instructions](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html)


		3. Run ssh -L 7860:localhost:7860 stable-diffusion

		4. Open http://localhost:7860/ in browser.


		4. Open http://localhost:7860/ in browser.

		5. Type in text prompt and click "Generate"

		@@ -1,4 +1,6 @@
		# Note: this file is from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile
		# Note: this file is modified from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile

		@@ -0,0 +1,24 @@
		# To launch a 1 GPU instance to run stable diffusion web tool.

Add stable diffusion example #1149

Add stable diffusion example #1149

Conversation

sumanthgenz commented Sep 2, 2022

concretevitamin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

concretevitamin commented Sep 2, 2022

romilbhardwaj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romilbhardwaj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romilbhardwaj left a comment

Choose a reason for hiding this comment