Research map / Detection and segmentation / CNN features

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

NeurIPS 2026 · Detection and segmentation › CNN features · approach: Plain and high-resolution ViTs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TEAL, a two-stage pixel-level active learning method that performs MaxHerding over frozen DINOv3 token features and then margin-based uncertainty refinement on a finer Mini-DPT decoder grid, for low-budget semantic segmentation.

Problem

Pixel-level active learning for semantic segmentation under extreme budgets (1 or 10 labeled pixels per image) with ViT backbones, where pixel-space selection on interpolated token features produces artifacts and sparse signals make decoder training hard.

Approach

Perform diversity-based candidate selection directly on the native ViT token lattice using two-level MaxHerding over multi-layer token descriptors, then refine candidates by margin uncertainty on a 4x finer lightweight decoder grid to choose final pixel queries.

Key results

• TEAL improves over the strongest baseline by up to +24.36 mIoU with one pixel per image (Pascal Context, 72.59 vs 48.23 for 2-Stage eDALD).
• With matched DINOv3 backbones, TEAL gains 0.50 to 2.47 mIoU at one pixel per image over the matched baseline, and reaches 97.1% of fully supervised performance on Pascal Context at 10 pixels per image.
• Pixel-space herding places 60.1% of queries on token edges versus 0% for token-space herding, and TEAL's token-space plus refinement design closes the gap to pixel herding (49.54 vs 47.34 mIoU on CamVid at 1 px).

Surprising findings

• ViT features with a token-space acquisition outperform diffusion features in the low-budget pixel regime, contrary to prior findings.
• Pixel-space herding hurts at ten pixels per image on some datasets, and eDALD falls behind PixelPick at 10 px on Cityscapes and Pascal Context.

Limitations

• Runtime and memory grow with image resolution due to the DINO forward pass and self-attention over more tokens.
• The lightweight Mini-DPT decoder can struggle on complex scenes with many boundaries.
• Evaluation is at 256x256 resolution for main results, with only limited higher-resolution experiments.

Experimental setup

Datasets: CamVid Cityscapes ADE20K Pascal Context

Benchmarks: CamVid Cityscapes ADE20K Pascal Context

Metrics: mIoU

Baselines: PixelPick 2-Stage eDALD eDALD DINO pixel herding BALD eBALD

Builds on: MaxHerding PixelPick 2-Stage eDALD Mini-DPT DINOv3

Closest papers by meaning

• Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation (NeurIPS 2025 · similarity 0.90)
• Subobject-level Image Tokenization (ICML 2025 · similarity 0.88)
• Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.87)
• ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers (CVPR 2024 · similarity 0.87)
• Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs (ICLR 2026 · similarity 0.87)
• Active Label Correction for Semantic Segmentation with Foundation Models (ICML 2024 · similarity 0.87)