Research map / Detection and segmentation / CNN features

RATS! Patches Talk Through Registers: Emergent Parts in Register Attention Transformers

NeurIPS 2026 · Detection and segmentation › CNN features · approach: Convolution-transformer backbones · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RATS, a ViT whose self-attention is replaced by a register-token L→N→N→L bottleneck (compress–communicate–broadcast) with registers partitioned across heads, trained with DINO self-distillation so registers emerge as part-level segmenters and a part dictionary.

Problem

Standard ViTs aggregate all patches into a single [CLS] token and have no explicit mechanism for grouping patches into part-level representations; the question is whether self-supervised training can make such part structure emerge.

Approach

Replace per-block self-attention with a register bottleneck: patches compress into N registers partitioned across heads, registers communicate with each other, and broadcast back to patches; trained with unmodified DINO so each register specializes to a part-like region, with no auxiliary part loss.

Key results

• RATS-B/16 with 192 registers reaches 39.54 average M2O mIoU over five part-segmentation benchmarks versus 27.45 for AdaSlot (ViT-B/16, 33 registers/slots) per the reported table.
• On ADE20K with Mask2Former, RATS pretraining gives 46.52 mIoU versus 45.41 for DINO; on COCO, AP^m 38.1 versus 37.9.
• RATS-S/16 with 6 heads-partitioned registers reports lower latency (81.7 ms vs 172.8 ms for ViT-S/16) and memory (0.49 GB vs 1.08 GB) at identical parameter count (~21.7M).

Surprising findings

• Applying a separate DINO loss per head degrades mIoU compared with a single shared DINO loss over all registers.
• Segmentation quality peaks around epoch 160-180 and then declines through epoch 300, which the authors attribute to diminishing register diversity.

Limitations

• Low-rank bottleneck attention (rank at most n per head) limits capacity; dense tasks require an added parallel full-rank L→L branch during fine-tuning.
• Discovered part boundaries follow visual co-occurrence and may not align with human-defined part taxonomies.
• Benchmarks rely on SAM 2.1 generated proxy ground-truth masks for ImageNet and COCO rather than human annotations.

Experimental setup

Datasets: ImageNet-1k COCO 2017 ADE20K ImageNet-S919 PartImageNet SAM 2.1 proxy masks

Benchmarks: COCO 2017 val (SAM masks) ADE20K ImageNet val (SAM masks) ImageNet-S919 PartImageNet ADE20K semantic segmentation COCO 2017 detection and instance segmentation

Metrics: many-to-one mIoU one-to-one mIoU ARI mAP R@1 valPw util% aAcc AP box AP mask

Baselines: DINOv1 DINOv3 Mamba with register SPFormer SPiT AdaSlot Perceiver-IO style

Builds on: DINO Vision Transformer Perceiver IO Mask2Former DINOv2

Closest papers by meaning

• Register and [CLS] tokens induce a decoupling of local and global features in large ViTs (NeurIPS 2025 · similarity 0.88)
• CleverBirds: A Multiple-Choice Benchmark for Fine-grained Human Knowledge Tracing (NeurIPS 2025 · similarity 0.86)
• Vision Transformers Don't Need Trained Registers (NeurIPS 2025 · similarity 0.86)
• Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts (NeurIPS 2025 · similarity 0.85)
• Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers? (NeurIPS 2025 · similarity 0.85)
• Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.85)