Researchers from Stanford University and Together AI have proposed a new text generation paradigm called Next-Concept Modeling (NCM), which differs from autoregressive and diffusion models by drawing inspiration from next-scale prediction methods in the image domain. NCM first allows the model to generate concept outlines in parallel, then fills in details layer by layer in parallel to form complete text. NCM is achieved through two-stage training: first, training a multi-scale residual VQ-VAE to obtain multi-layer discrete text encodings that can be supplemented layer by layer; then, freezing this system and training a Transformer to predict the next layer's encoding. Experiments show that NCM achieves generation speeds 10-31 times faster than diffusion language models while maintaining similar or superior generation quality, and significantly reduces the amount of tokens required for training. On reasoning tasks such as Sudoku, NCM outperforms autoregressive and diffusion baselines and can be further optimized by selecting from multiple generated candidates. Currently, NCM has limited model parameter scale, and its performance after scaling up, as well as its ability to generate precise text and long-form content, remains to be verified. However, NCM simulates the human writing pattern of conceive (conceiving ideas) before to begin writing (putting pen to paper), and is expected to change the development path of future text generation.
