Research map / Large language models / Alignment and preferences

POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation

CVPR 2025 · Large language models › Alignment and preferences · approach: Robust and calibrated DPO · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes POPEN, which fine-tunes an LVLM (PixelLM architecture) with segmentation-aware preference optimization (text and segmentation-embedding DPO-style losses with curriculum preference collection) and a preference-score-weighted ensemble at inference, reducing hallucination and improving reasoning segmentation.

Problem

LVLM-based reasoning segmentation methods produce hallucinated text responses and imprecise segmentation masks, and standard text-only DPO does not optimize segmentation quality.

Approach

Extends DPO to segmentation by collecting segmentation-embedding preferences from noise-perturbed images ranked by SAM-based localization and boundary IoU in a curriculum (localization first, then boundary), trains a similarity-based preference loss on segmentation embeddings, and ensembles multiple LVLM responses with preference-score-biased attention.

Key results

• On MUSE test (13B), POPEN reaches 48.3 gIoU / 67.9 cIoU versus PixelLM's 45.0 / 61.9 and LISA's 27.3 / 38.2.
• On MUSE test (7B), CHAIR CS drops from 22.2 (PixelLM) to 9.5 with POPEN.
• On refCOCOg val(U) and test(U) with fine-tuning, POPEN (ft) reaches 75.4 and 75.6, above GSVA (ft) at 72.7 and 73.3.

Measured effects reported in the paper

Limitations

• Text-based DPO and segmentation-embedding DPO require heavy preference data construction, including ChatGPT corrections and 30 perturbed images per sample.
• Ensemble requires generating K=3 responses at inference, increasing inference cost.
• Reported LISA and PixelLM numbers were reproduced by the authors and differ from originally published numbers, possibly due to differing ChatGPT versions.

Experimental setup

Datasets: MUSE ADE20K COCO-Stuff LVIS-PACO refCOCO LLaVA-150k

Benchmarks: MUSE refCOCO refCOCO+ refCOCOg

Metrics: gIoU cIoU CHAIR CS CHAIR CI GPT score

Baselines: LISA GSVA GLaMM PixelLM MCN VLT CRIS LAVT ReLA

Builds on: DPO PixelLM LISA RLHF

Closest papers by meaning

• CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation (ICLR 2026 · similarity 0.84)
• LISA: Reasoning Segmentation via Large Language Model (CVPR 2024 · similarity 0.84)
• DR$^2$Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models (ICML 2026 · similarity 0.83)
• Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration (ICML 2025 · similarity 0.83)
• SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning (ICML 2026 · similarity 0.83)
• Bias-Variance Optimized Preference Optimization for Large Reasoning Models (NeurIPS 2026 · similarity 0.83)