Research map / Detection and segmentation / Open-set and few-shot detection

DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation

CVPR 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Few-shot segmentation · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DSV-LFS, an end-to-end few-shot segmentation framework that fuses an LLaVA-generated query-specific semantic prompt (via a new <SEMprompt> token) with a SAM-based dense-matching visual prompt to guide a SAM-style mask decoder.

Problem

Few-shot semantic segmentation struggles to generalize to novel classes because limited support images give incomplete, appearance-biased class representations; prior LLM-based FSS needs multi-stage training and produces text that must be post-processed into masks.

Approach

Introduce a <SEMprompt> token into a multimodal LLM so it adapts ChatGPT-generated class descriptions to the query image, and combine the resulting semantic prompt with a 4D dense-matching visual prompt from support-query correlations to drive a prompt-based mask decoder in a single-stage, end-to-end model.

Key results

• On COCO-20i 1-shot, DSV-LFS reaches 71.33 mIoU mean (vs LLaFS 53.9 reported in table), a gain of roughly 17 points.
• On PASCAL-5i 1-shot, DSV-LFS achieves 75.21? mean-level comparable to LLaFS (75.6 5-shot-level table entries); the paper says it is comparable to LLaFS and better than non-LLM methods.
• Semantic prompt alone reaches 68.99 mIoU on COCO-20i 1-shot; adding visual prompt gives 71.33 (about +3 points).

Measured effects reported in the paper

Surprising findings

• On PASCAL-5i the method only matches the LLM-based LLaFS rather than surpassing it, which the authors attribute to dataset saturation.

Limitations

• Gains on PASCAL-5i are comparable to LLaFS rather than clearly superior.
• Relies on ChatGPT-generated class descriptions and a multimodal LLM, adding inference cost not quantified.

Experimental setup

Datasets: PASCAL-5i COCO-20i

Benchmarks: PASCAL-5i COCO-20i COCO-20i to PASCAL-5i

Metrics: mIoU

Baselines: LLaFS HDMNet MIANet BAM SCCAN MSI IPMT AAFormer SSP NTRENet ABCNet IFA

Builds on: LISA LLaVA Segment Anything HSNet LLaFS

Closest papers by meaning

• LLaFS: When Large Language Models Meet Few-Shot Segmentation (CVPR 2024 · similarity 0.89)
• SAM-Aware Graph Prompt Reasoning Network for Cross-Domain Few-Shot Segmentation (AAAI 2025 · similarity 0.88)
• VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning (NeurIPS 2025 · similarity 0.88)
• A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation (NeurIPS 2024 · similarity 0.88)
• Unlocking the Potential of Pre-trained Vision Transformers for Few-Shot Semantic Segmentation through Relationship Descriptors (CVPR 2024 · similarity 0.88)
• Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.88)