Research map / Image generation / Autoregressive and latent generation
Any-Order Any-Subset AutoRegressive Model
Tianqi Du, Lizhe Fang, Weijie Yang, Chenheng Zhang, Zeming Wei, Yifei Wang, Yisen Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add progressive stage 1 and 2 (AR init, group expansion) before order permutation (ablation) | A3 curriculum · TriviaQA | accuracy: 11.3 -> 15.6 | Improved |
| add progressive stage 1 and 2 before order permutation (ablation) | A3 curriculum · PIQA | accuracy: 64.2 -> 69.4 | Improved |
| add progressive stage 1 and 2 before order permutation (ablation) | A3 curriculum · ROCStories | ROUGE-1/2/L: 13.1/2.2/12.4 -> 13.2/2.3/12.6 | No clear effect |
| increase training tokens from 6B to 10B (ablation) | A3-8B · TriviaQA | accuracy: 16.2 -> 22.5 | Improved |
| increase training tokens from 6B to 10B (ablation) | A3-8B · log(perplexity) | log(perplexity) by Llama-3.1-8B: 2.9 -> 2.3 | Improved |
| scale model from 1B to 8B | A3 · TriviaQA | accuracy: 10.2 -> 19.4 | Improved |
| scale model from 1B to 8B | A3 · PIQA | accuracy: 64.7 -> 78.1 | Improved |
| use A3-8B instead of diffusion DiffuLlama-7B | A3 · PIQA | accuracy: 63.3 -> 78.1 | Improved |
| use A3-8B instead of AR Llama-3.1-8B | A3 · TriviaQA | accuracy: 52.1 -> 19.4 | Hurt |
| use entropy-based dynamic sampling instead of random sampling (step=1024) (ablation) | A3-8B · conditional generation | perplexity: 49.3 -> 11.2 | Improved |
| use dynamic resampling instead of groupwise AR sampling (ablation) | A3-8B · unconditional generation | perplexity | Improved |
| increase decoding group size (ablation) | A3-8B · unconditional generation | perplexity | Hurt |
Closest papers by meaning
- MARS: Enabling Autoregressive Models Multi-Token Generation (NeurIPS 2026 · similarity 0.86)
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.85)
- Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass (NeurIPS 2024 · similarity 0.85)
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time (ICLR 2025 · similarity 0.84)
- Multi-Branch Self-Drafting for LLM Inference Acceleration (AAAI 2025 · similarity 0.84)
- When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
