Traditional autoregressive language models generate text sequentially, one token at a time, leading to slower outputs with limited coherence and quality.

Diffusion models are an alternative approach. Instead of direct prediction, they iteratively refine noise, enabling faster generation, dynamic error correction, and greater control. This makes them particularly effective for editing tasks, including in math and code.

https://github.com/ML-GSAI/LLaDA