What if we could generate composite images in Stable Diffusion while working in Photoshop? And what if we could keep the subject intact? And in real time as well?
In this tutorial, we'll see how Segment Anything can help us blend the original subject of an active image in Photoshop into a generated image in Stable Diffusion, all in real time, while we're editing and comping in Photoshop.
If you like my contributions, you can buy me a coffee here: ko-fi.com/risunobushi
Ever wondered how to use Stable Diffusion in your traditional workflows in professional environments as a creative?
Stable Diffusion for Professional Creatives is a series of videos about how we can apply generative AI in production environments, with real life examples and guides.
Useful links (by category)
IMPORTANT: some models may have different filenames as the ones showed in the video. That's because I rename them after downloading in order to remember which model does its thing. Refer to the instruction in this description to properly set everything up.
Workflow:
- json file, import in comfyUI and install any missing nodes: pastebin.com/tzPWd8s2
Nodes (if you haven't installed them via importing the provided workflow using ComfyUI Manager):
- Photoshop to ComfyUI: github.com/NimaNzrii/comfyui-...
IMPORTANT: AFTER INSTALLING, SINCE IT'S BEEN UPDATED FROM THE VERSION I'M RUNNING, YOU NEED TO DOWNGRADE IT. OPEN THE FOLDER comfyui-Photoshop, right click, open in terminal, and run: git checkout --force 403d4a9af1f947c95367cd40ff8ad6ae65e5df41
THIS WILL DOWNGRADE THE REPO TO THE VERSION I'M USING
- ComfyUI Manager GitHub: github.com/ltdrdata/ComfyUI-M...
- IPAdapter Plus v2 GitHub: github.com/cubiq/ComfyUI_IPAd...
- Segment Anything / Grounding Dino GitHub: github.com/storyicon/comfyui_...
Models:
- RealVis XL (SDXL Lightning used in this video): civitai.com/models/139562/rea...
- SDXL LoRA (I'm using the 4 steps LoRA): huggingface.co/ByteDance/SDXL...
- ControlNet SDXL: huggingface.co/lllyasviel/sd_...
- Segment Anything / Grounding Dino models (same GitHub, go down the page to find them):
github.com/storyicon/comfyui_...
IPAdapter (Just in case your v2 download doesn't come with IPAdapter models preinstalled or you'd like to install your own):
- IPAdapter Plus XL model. Place it into your "\models\ipadapter" folder and use it in your Load IPAdapter Model node: huggingface.co/h94/IP-Adapter...
- IPAdapter Plus 1.5 model. Place it into your "\models\ipadapter" folder and use it in your Load IPAdapter Model node: huggingface.co/h94/IP-Adapter...
- CLIPVision ViT-H model (works with IPAdapter *PLUS*, both 1.5 and XL, only. For IPAdapter *STANDARD*, not used in this video and kind of deprecated, you need a ViT-G model). Place it into "\models\clip_vision" and use it in your Load CLIPVision Model node: huggingface.co/h94/IP-Adapter...
Troubleshooting:
If your PopUp node is acting up, try the following:
- close comfyui
- go inside your comfyui folder, in comfyui - custom_nodes - comfyui-popup_preview - window
- open the folder inside the terminal
- type "pip install -r requirements.txt" without the ""
- press Enter
- wait for the requirements to be installed
- try launching comfyui again
This worked for me on the one time I could replicate one of the issues you commented about.
Timestamps:
00:00 - Intro
00:54 - Workflow Overview
01:55 - Downloading the SDXL Lightning models
03:06 - Building the workflow (Core)
04:55 - Building the workflow (IPAdapter)
07:21 - Building the workflow (ControlNet)
10:02 - Building the workflow (Segment Anything)
13:09 - Building the workflow (Blending the images)
15:04 - Setting up Photoshop
15:54 - Explaining what's happening in the background
17:37 - Editing in Photoshop x Stable Diffusion
18:56 - Complex Backgrounds
22:19 - Troubleshooting
23:16 - Outro
#stablediffusion #stablediffusiontutorial #ai #generativeai #generativeart #comfyui #comfyuitutorial #risunobushi_ai #moodboards #reference #sdxl #sd #risunobushi #andreabaioni
Негізгі бет Ғылым және технология Photoshop x Stable Diffusion x Segment Anything: Edit in real time, keep the subject
Пікірлер: 55