Research map / Image generation / Autoregressive and latent generation
Spot the Error: Non-autoregressive Graphic Layout Generation with Wireframe Locator
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a wireframe-image-based learned locator that detects erroneous layout tokens for iterative non-autoregressive layout generation, improving over AR and NAR (BLT) baselines on RICO and PubLayNet.
Problem
Autoregressive layout generation suffers from limited global context and error propagation, while existing iterative non-autoregressive methods use simple confidence-based heuristics to choose tokens to re-mask, which are inaccurate.
Approach
Render the generated layout sequence as a wireframe image and use it both as conditioning for the decoder and as input to a learned locator (Faster R-CNN based) that classifies which attribute tokens are erroneous; locator training data is built from decoder outputs by matching them to retrieved real layouts.
Key results
- On PubLayNet C→SP, the locator model reaches PixelFID 2.33 vs 4.60 for BLT and Overlap 7.57 vs 8.73.
- On PubLayNet CS→P, the locator model achieves the best SeqFID (2.04) and PixelFID (1.44) among compared models.
- Pixel-space locator (Faster R-CNN) reaches F1 59.82 vs 29.52 for object-space locator in detecting erroneous attributes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the wireframe-conditioned decoder and learning-based locator | BLT-style NAR layout generation · PubLaynet C→SP | PixelFID: 4.60 -> 2.33 | Helped |
| Add the wireframe-conditioned decoder and learning-based locator | BLT-style NAR layout generation · PubLaynet CS→P | Overlap: 8.96 -> 9.69 | Hurt |
| Use pixel-space wireframe input instead of object-space sequence input (ablation) | Locator · PubLaynet | F1: 29.52 -> 59.82 | Helped |
| Replace FasterRCNN with DETR as the pixel-space locator backbone (ablation) | Locator · PubLaynet | F1: 59.82 -> 52.84 | Hurt |
| Train the locator on decoder-generated outputs rather than random-noise layouts (ablation) | Locator · PubLaynet | PixelFID: 19.23 -> 2.33 | Helped |
| Train the locator on decoder-generated outputs rather than random-noise layouts (ablation) | Locator · PubLaynet | Overlap: 19.69 -> 7.57 | Helped |
| Iteratively refine the generated layouts (ablation) | BLT · PubLaynet | Overlap: 26.89 -> 10.23 | Helped |
| Use pixel representation instead of object representation for toy error detection at noise range 0.1 (ablation) | Toy noisy-layout detector · Toy noisy-layout detection, noise range 0.1 | F1: 82.17 -> 90.45 | Helped |
| Remove HSP from BLT (ablation) | BLT · PubLaynet C→SP | PixelFID: 4.60 -> 3.82 | Helped |
| Remove HSP from BLT (ablation) | BLT · Rico C→SP | Overlap: 72.64 -> 68.66 | Helped |
Surprising findings
- NAR (BLT) overlap is higher than AR (26.89 vs 23.80 all elements) at the first iteration, showing a repetitive-token issue in NAR.
- Object-space locator performs worse than the decoder-only model on end-to-end PixelFID (14.11 vs 113.36 dec-only is worse, but locator still lowers overlap only modestly), and the random-noise training data gives much worse locator results than decoder-output data (F1 29.53 vs 59.82).
Limitations
- Locator performance on Rico is weaker than the decoder-only variant on some metrics (e.g., Overlap and Alignment on CS→P), and the locator model has higher Overlap on PubLayNet CS→P (9.69 vs 8.44).
- Only layout generation without content awareness is studied; content-aware generation is left for future work.
- Relies on heuristic pipeline choices such as overlap thresholds and matching cost weights that are not deeply analyzed.
Experimental setup
Datasets: RICO PubLayNet
Benchmarks: RICO PubLayNet
Metrics: SeqFID PixelFID Max IoU Overlap Alignment F1 score
Baselines: LayoutVAE VTN LayoutGAN++ LayoutTransformer BLT BLT w/o HSP
Builds on: BLT LayoutTransformer Faster R-CNN BERT DETR
Closest papers by meaning
- Detect Anything in Graphic Design: Element-Level Rewards for Autoregressive Detection (NeurIPS 2026 · similarity 0.85)
- Desigen: A Pipeline for Controllable Design Template Generation (CVPR 2024 · similarity 0.84)
- OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps (NeurIPS 2025 · similarity 0.84)
- Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation (NeurIPS 2025 · similarity 0.84)
- Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.84)
- SpotActor: Training-Free Layout-Controlled Consistent Image Generation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.