Can LLMs Think in Parallel? Explaining NVIDIA's Nemotron and Diffusion Models for Text Generation

Traditional LLMs work by predicting one token after another. It is always sequential, and the next prediction is always based on the tokens generated so far. Which means that once you generate something, there is no going back.

This is the basis of an .

To contrast this, we have diffusion models, the backbone of how images are generated.

These models start with pure noise and refine it step by step, making the output better each time. Picture dropping ink into a glass of water and watching it spread until it's a cloudy mess. That spreading is diffusion.

Diffusion models learn to play that film backwards: start from the mess, and un-mix it back into something clear.

Side-by-side diagram: an autoregressive model writing one token at a time versus a diffusion model refining a whole block from noise — Autoregressive models versus diffusion models

The fundamentals: autoregressive LLMs versus diffusion models

Image-style diffusion is hard for natural language because text is made of discrete symbols. Each token is a fixed value from a finite vocabulary.

You can make a pixel slightly brighter, but you cannot make a token slightly more "strawberry".

Text diffusion models so far have worked around this with masking: hide tokens, then fill in the blanks over several steps. But every step is still a hard swap from one token to another, not a smooth refinement.

That brings us to their paper on Large Language Continuous Diffusion Models, and the model it introduces, .

What do continuous diffusion models do, and why do they matter?

Diffusion models for text have mostly been pushed for speed: generating many tokens in parallel. NVIDIA proposes using them in a continuous form.

Here's the watered-down version. Every piece of output is a token. Autoregressive models predict each token from the ones before it, one after another. In a continuous diffusion model, a whole chunk of tokens starts out as noise, and the model cleans all of them up together, step by step.

Note

How Sigma works, in five steps

Tokens become vectors. Every token maps to a small vector of 16 numbers, like a lookup table. Sigma adds noise to these vectors and trains the model to remove it.
Each pass blends a best guess. The model scores every possible token and blends them into one best guess. It also feeds its last guess back in, so each step builds on the previous one.
Chunks of 32. Text is generated in chunks of 32 tokens. Tokens inside a chunk see each other, while earlier text is .
It doesn't start from scratch. Sigma starts from an existing autoregressive model instead of training a new one.
Speed vs. quality. Fewer cleanup steps make it faster, so the team spread the vectors out to keep quality stable.

16numbers per token vector

32tokens denoised together per chunk

Diagram of continuous diffusion for text: a chunk of noisy token vectors denoised together over several steps into clean tokens — A simple depiction of how continuous diffusion models generate text

How will this matter in software engineering?

If you want to see where this is heading in practice, NVIDIA's Nemotron-Labs Diffusion blog is the one to read. Nemotron is Sigma's older sibling: it uses masked diffusion instead of continuous, but it shows exactly how diffusion could end up in the tools we use every day.

The use cases are any task that is latency sensitive. Sequential token generation takes time, and every extra word is another wait. NVIDIA points to interactive chat, agentic loops and on-device assistants as the places where this parallelism pays off most.

Towards Speed-of-Light Text Generation with Nemotron-Labs Diffusion Language ModelsA blog post by NVIDIA on Hugging Face.huggingface.co

Nemotron-Labs Diffusion is built around a simple idea: autoregressive and diffusion generation shouldn't be separate model families. They should be capabilities of the same model.

The model supports three generation modes:

Autoregressive mode generates like a standard left-to-right LLM.
Diffusion mode generates block by block, cleaning up noise each time.
Self-speculation mode uses diffusion to draft several candidate tokens, then uses autoregressive decoding to verify them. It combines the speed of diffusion-style drafting with the reliability of autoregressive verification.

That last one is the interesting part for engineers. At the output is identical to plain autoregressive decoding, yet NVIDIA reports a big jump in speed.

~865tokens per second

~4×the autoregressive baseline

0change in output at temperature 0

Same answer, just faster.

Diagram of self-speculative decoding: a diffusion pass drafts several tokens, and an autoregressive pass verifies and accepts them — How self-speculation combines diffusion drafting with autoregressive verification

And speed is only half of it. Because diffusion models can go back and revise tokens they've already generated, they're a better fit for editing existing text and tasks.