ComfyUI for Everything (other than stable diffusion)

Hey there!

Recently I have realised that most of the time I do more than Stable Diffusion inside ComfyUI. So I wanted to collect them all together and share them with you!

Here are more than 30 workflows I have used in ComfyUI for the last couple of months other than Stable Diffusion.

I see many capabilities in ComfyUI just like Grasshopper. These use cases can be used to make our workflows more comprehensive.  

Resources

• Single-Click ComfyUI Installation for WindowsOS
• Single-Click ComfyUI Installation for RunPod
• Custom Workflows
• Scrintal

You can find details about them here:

1- Image to Text with LLaVA
LLaVA is a sophisticated tool that can understand the contents of an image and provide detailed descriptions. The location of the building or the materials used on the building in the image.

2- Background Removal for Images
You can remove the background of an Image with Comfy UI, here I prepared 3 workflows that you can do this. In the first two, you don’t have much control over which is the focused elements so it removes the background randomly but in the third one, you get to write what you want to keep and adjust the creativity level.

3- Background Removal for Videos
With this Video-to-Mask component, we can cleanly segment the subject from the background in just a few clicks. We can customize the frame rate and duration depending on the video input.

4- Text Generation – ChatBot inside ComfyUI
LLMLoader component can create text generation locally on your computer, we just need to explain the model that we want and it will create prompts or generate any text for our project.

5- Upscale Enhancer: Enhancing Image Quality
We don’t need to go to Photoshop or any Image filter application anymore, we have the same components that can do it inside the Comfy UI, with these components we can enhance images from sharpening details to adding texture.

6- Image Filters
We can add a personal touch to photos with image filter components inside Comfy UI. From channel shakes to watercolor effects, these filters allow us to transform our images with just a few clicks, we even have control over the directions of the shake or color.

7- Image Enhancements
With this workflow, we can adjust the brightness, contrast, and saturation of our images using filter adjustment components. Add film grain for a vintage feel or apply custom looks to give our photos a unique style.

8- Text to Audio and Image to Audio
By only prompting a location and a scene we can transform this text into audio, with the Audio LDM component creating a sound effect. It is not only from text, with this second workflow you can create sound effects from images. Upload your image and Lava will translate your image to text and LLM Model generates it to a sound effect text then it will turn out to be a sound effect.

9- Image Effects
We can experiment with depth and motion blur effects using Comfy UI’s image filters. Blur backgrounds for a professional touch or apply color adaptation to give your images a cohesive look.

10- Color Palette Generator
We can create a perfect color palette for your project with Comfy UI’s color palette generator components. It is possible to create a mood board with the colors located exactly like in the image or pick dominant colors from an image and create a 9-color mood board.

Color Palette by Location

Grid Color Palette

11- Image to 3D
We can transform images into 3D views. These components create 6 different photos from different parts of the building in this case. It will provide you photos from the back, sides, top, and down, by looking at the image you have provided. It is almost having the model as a 3D model. It might be not really handy for architectural purposes but I am sure it is super useful for furniture design or game development.

Image to 3D Images

12- Remove Objects and Inpainting
We can remove objects from the images and inpaint missing areas with Comfy UI’s object removal and inpainting components.

Remove Object

Inpaint any Component

13- Add Text on an Image
Add text overlays to your images with Comfy UI’s text-on-image feature. We can write any text on them in this case I prefer to write the materials of the building and make a comparison table.

Creating Caption and Frame

With Comfy UI, the possibilities are endless. I see many capabilities in Comfy UI just like Grasshopper that’s why I wanted to share some use cases that we can use to make our workflows more comprehensive. From image processing to text generation, and audio creation…. So why juggle multiple software when you can do it all with Comfy UI?