How AI image generation works

A diffusion model is trained on millions or billions of image-text pairs. In doing so, it doesn't store individual images. It learns statistical relationships between terms and visual features.

Imagine the model should later generate "a cat wearing a hat." To do that, it must have learned during training what cats look like — fur texture, eye shape, posture — and, independently, what hats look like — shape, brim, material. Both concepts are stored as separate patterns that can later be combined in any way.

This knowledge is stored in what's called a latent space — an abstract mathematical space in which similar visual features sit close together. The latent space is a kind of invisible collection of image building blocks, sorted by similarity. The model no longer knows any original images — only the learned statistical patterns.

When a prompt is entered, the model activates the corresponding patterns in the latent space. The prompt acts like a weighting of certain features. The model combines the activated patterns so that an image emerges that statistically matches the description as closely as possible.

It does not select an existing image — it computes a new one that combines the desired properties with high probability.

The control is indirect: what's influenced isn't individual pixels, but the probabilities with which certain structures appear together.

The term comes from the technical process of how the image is formed. The model starts with random noise — comparable to image static — and removes that noise step by step. Across many small computational steps, an increasingly clear image structure emerges.

The more precisely and consistently a prompt is formulated, the more stable and reproducible the result becomes.

This is one of the most important points for understanding prompts — and one of the most frequently misunderstood.

Before the diffusion process begins, the entire prompt is translated by a text encoder into a single, comprehensive vector. This vector contains the meaning of the complete prompt — not word by word in sequence, but as a whole, all at once.

A helpful way to picture it: the prompt isn't read like a sentence you work through word by word. It's more like a single photograph taken of the entire text at once — with all the relationships between the words already captured.

This vector then accompanies every single step of the diffusion process. At each step — from the noisy starting image to the finished result — the model "looks" at the full prompt vector again and decides which areas of the emerging image should be influenced by which parts of the prompt.

The mechanism behind this is called attention. It lets the model "weight" how strongly a given word in the prompt influences a given area of the image. In the example "a black-and-white cat wearing a top hat in a retro café," attention ensures that "black-and-white" and "top hat" act on the cat, while "retro café" shapes the surroundings — even though these words sit far apart in the sentence.

What this doesn't mean: the model has no memory in the human sense, and it doesn't read the prompt sequentially the way a person reads a sentence. There's no note-taking, no remembering, no understanding. There's a mathematical representation of the entire prompt, which gets consulted again at every computational step.

In practice, this means: the entire prompt acts on the entire image, at every step. Word order does play a role — but differently than in a sentence read left to right. It influences how much weight attention assigns to particular terms.

The calculations in each diffusion step are carried out by a neural network — a computational model made up of many interconnected units, organized in layers. An input layer, several processing layers, an output layer. Each unit processes numerical values, weights them, sums them, and passes the result on.

A helpful way to picture it: a production line. At the start, raw material arrives — the noisy image state plus the prompt vector. The material passes through several stations where it's processed and altered. Each station makes decisions based on its trained settings — the weights. At the end of the line stands an image that's slightly less noisy than at the start.

What attention mechanisms add to this production line: each station doesn't just have the material in front of it, but also has access at any time to the complete prompt vector — like a board hanging above the entire production line that every station can read at any moment. There's no handing off from station to station, no note that travels along — instead, a shared reference that everyone accesses simultaneously.