Type a text description into Midjourney, DALL-E, or Stable Diffusion, and within seconds you get back a fully formed image that never existed before. The technology making that possible is called a diffusion model — and despite how seamless the result looks, the underlying process is a genuinely strange and clever trick: the model learns to create images by first learning how to destroy them.
What Is a Diffusion Model?
A diffusion model is a type of generative AI model that creates images (and increasingly, video and audio) by starting with random visual noise and gradually refining it, step by step, into a coherent output. The name comes from the physics concept of diffusion — the way particles spread out and become disordered over time, like a drop of ink dispersing in water. Diffusion models are trained to run that process in reverse: turning disorder into a coherent, structured image.
How Diffusion Models Are Trained
Training a diffusion model involves an unusual two-part setup:
Step one: learning to add noise. The model is shown real images, and random visual “noise” (static, essentially) is gradually added to them across many steps, until the original image is completely obscured into pure noise. This part doesn’t require any cleverness — it’s a fixed, mechanical process.
Step two: learning to remove noise. This is where the actual learning happens. The model is trained to predict, at each step, exactly what noise was added — effectively learning to reverse the noising process and recover a clean image. Repeated across millions of images, the model gets progressively better at “denoising” any noisy input into something that looks like a plausible real image.
How This Produces a New Image from a Text Prompt
Once trained, generating a brand new image works by running the denoising process in reverse, starting from scratch: the model begins with a canvas of pure random noise, and then repeatedly applies its learned denoising step, each time nudging the noise slightly closer to a coherent image — guided by a text prompt (processed through a text embedding, which translates your words into a numerical representation the model can use to steer the process).
At each step, the model essentially asks: “given this current noisy image and this text description, what would make this look slightly more like the described thing, and slightly less like noise?” Repeated over dozens of steps, pure static gradually resolves into a detailed, coherent image that reflects the prompt. No single step creates the whole image — it emerges gradually, the way a photograph develops in a darkroom.
Why This Approach Works So Well
Diffusion models produce noticeably more coherent, detailed, and varied images than earlier generative approaches, like GANs (generative adversarial networks), which dominated AI image generation before diffusion models took over. Part of the reason is stability: the step-by-step denoising process gives the model many small opportunities to course-correct, rather than needing to produce an entire finished image in one shot. This incremental refinement tends to produce fewer glaring artifacts and more coherent overall composition.
Why Image Generation Takes Longer Than Text
Because diffusion models build an image through dozens of sequential refinement steps rather than generating it in a single pass, image generation is typically more computationally intensive than generating a comparable amount of text with a language model. This is why AI image generation tools sometimes have a visible progress bar or take several seconds — you’re watching an iterative refinement process play out, not a single instantaneous lookup.
Beyond Still Images
The same core diffusion principles have been extended to AI video generation tools, with an added layer of complexity: the model has to maintain coherence not just within a single frame, but across a whole sequence of frames over time, so that objects, lighting, and motion stay consistent from one frame to the next. Diffusion techniques have also been adapted for audio generation, where the “image” being denoised is a visual representation of sound (a spectrogram) rather than a picture.
The Limitations of Diffusion Models
Diffusion models inherit the general limitations of generative AI: they reflect patterns and biases present in their training images, they can struggle with tasks requiring precise logical consistency (text rendered within images, or getting the correct number of fingers on a hand, were notoriously difficult problems for early diffusion models), and questions remain unresolved about the copyright status of training images and generated outputs.
The Bottom Line
A diffusion model creates images by learning to reverse a process of gradually adding random noise to a picture — starting from pure static and refining it, step by step, into a coherent image guided by a text prompt. It’s a counterintuitive but remarkably effective technique, and it’s the engine behind nearly every major AI image and video generator in use today.