Ever thought about creating alterations to your designs and seeing them transform instantly? I’ve been exploring this very topic lately, and I can assure you that it’s quite significant. Integrating Stable Diffusion with a cloud-based platform called RunPod has opened up new possibilities for design visualization, especially in parametric modeling.
Start with a 3D Model (Possibly Parametric)
The process starts with the PolyPlaza parametric model created in Grasshopper and Rhino. (Scripted by decoded_facades).
The beauty of parametric models lies in their flexibility; you can play around with the parameters, and the model adapts instantly. This fluidity is what I wanted to capture in real-time renders.
With the workflow I’ve pieced together, any view on your computer screen can be rendered in real-time, not just parametric models but any model from software like SketchUp, Archicad, or Revit.
Stable Diffusion on Cloud Server: RunPod
To make this process even smoother, I used RunPod. It’s a cloud-based service for accessing high-end GPUs and use Stable Diffusion on the cloud.
But of course, you can use it locally on your computer as well. This step is additional but recommended if you don’t have a decent GPU.
RunPod provides access to powerful GPUs, ensuring smooth and efficient rendering. Although the platform is paid for, the prices are rather fair given the power it puts in your hands.
After signing up and funding my wallet, I selected a GPU that balanced performance and cost, specifically the RTX 3090, which offers a sweet spot for both.
You can find the guide I prepared on how to set up Runpod for Comfy UI with the necessary installing models on my Patreon page.
Comfy UI Integration
With the infrastructure set up, it’s time to focus on the main part of the workflow, Stable Diffusion.
I used ComfyUI for a more user-friendly experience, configuring it to work seamlessly with RunPod. The setup involved installing various models and extensions, ensuring everything I needed was right in my hands.
The Real Magic: Real-Time Rendering
Now, onto the most exciting part: real-time rendering.
This process allowed me to see immediate visual feedback as I played with the model’s parameters. This real-time visualization is a game-changer, offering an insight into how even minor adjustments can dramatically alter the design.
Enhancing the Workflow
The workflow contains techniques like depth maps of the model and even to make it more accurate we use edge detection, adding layers of detail to the renders.
Later on, I add a reference image workflow to get the aspects of the real image of the same building. These methods made my design more realistic, capturing the details and details that differentiate each model.
This exploration into real-time rendering with Stable Diffusion has opened up new horizons in design visualization. The ability to see your designs evolve in real-time, with every tweak and adjustment, is truly transformative.
It’s not just about the technology; it’s about how it redefines the creative process, making design more interactive and responsive.
I’ve shared all the material, scripts, and tools for those who are interested in following along on my Patreon page.
This approach provides insight into the direction that design visualization is taking, regardless of your level of experience with digital design.
So why not give it a go and see where your imagination leads you?
I hope you liked the results and are excited to try it yourself. See you in the next one 🙂
Just occasional emails with great value!)