Research map / Detection and segmentation / Open-set and few-shot detection
Advancing Saliency Ranking with Human Fixations: Dataset Models and Benchmarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SIFR, a large-scale saliency ranking dataset built from eye-tracking fixations on MS-COCO images, and QAGNet, a Mask2Former-based model with a tri-tiered nested graph for ranking salient instances.
Problem
Existing saliency ranking detection (SRD) datasets rely on mouse-trajectory proxies for gaze, and existing methods often rank only a few salient objects or cap the number of outputs.
Approach
Build SRD ground truth from genuine eye-tracking fixations with re-annotated instance masks and no cap on salient instances, and pair it with QAGNet, which aggregates multi-scale query representatives from a transformer detector through a tri-tiered nested graph (SSG, MSG, GRG) with representative aggregation and feedback stages.
Key results
- QAGNet outperforms prior SRD methods on SA-SOR and MAE across ASSR, IRSR and SIFR, surpassing the second-best model by 11.0%, 14.5% and 11.1% respectively on SA-SOR.
- The SIFR dataset comprises 8389 images with 52173 instances, with no cap on the number of salient instances per image.
- Ablation shows the RA/RF tri-tiered graph modules raise SA-SOR from 0.5807 (average-query baseline) to 0.6086 on SIFR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QAGNet with Swin-L | SRD models · ASSR | SA-SOR: 0.7090 -> 0.7873 | Helped |
| Use QAGNet with Swin-L | SRD models · IRSR | SA-SOR: 0.5648 -> 0.6468 | Helped |
| Use QAGNet with Swin-L | SRD models · SIFR | SA-SOR: 0.5585 -> 0.6206 | Helped |
| Use QAGNet with Swin-L instead of the strongest baseline | SRD models · ASSR | SOR: 0.8937 -> 0.8591 | Hurt |
| Average all 9 query representations instead of using only the last query (ablation) | Mask2Former rank prediction on SIFR · SIFR | SA-SOR: 0.5623 -> 0.5807 | Helped |
| Add the RA-stage graph modules and RF-stage feedback (ablation) | QAGNet on SIFR · SIFR | SA-SOR: 0.5623 -> 0.6086 | Helped |
| Use 2 hidden layers instead of 1 (ablation) | QAGNet on SIFR · SIFR | SA-SOR: 0.6086 -> 0.6119 | Helped |
| Add a short connection (ablation) | QAGNet with 2 hidden layers on SIFR · SIFR | SA-SOR: 0.6084 -> 0.6119 | Helped |
| Use 3 hidden layers instead of 2 (ablation) | QAGNet on SIFR · SIFR | SA-SOR: 0.6119 -> 0.6089 | Hurt |
| Increase query count from 50 to 100 (ablation) | QAGNet on SIFR · SIFR | SA-SOR: 0.6061 -> 0.6119 | Helped |
| Increase query count from 100 to 200 (ablation) | QAGNet on SIFR · SIFR | SA-SOR: 0.6119 -> 0.6128 | Helped |
| Increase query count from 100 to 200 (ablation) | QAGNet on SIFR · SIFR | SOR: 0.7899 -> 0.7886 | No clear effect |
Surprising findings
- IRSR and OCOR achieve high SOR scores yet drop markedly on SA-SOR, since SOR does not penalize missing salient objects.
Limitations
- Eye-tracking data collected from only eight participants with 3-second free viewing of images.
- Confidence-threshold inference can produce more instances than the limited ASSR/IRSR protocols allow, requiring top-k workarounds for comparison.
Experimental setup
Datasets: SIFR ASSR IRSR MS-COCO
Benchmarks: SIFR ASSR IRSR
Metrics: SA-SOR SOR MAE
Baselines: RSDNet ASSR IRSR SOR OCOR
Builds on: Mask2Former Salient Object Ranking (RSDNet) IRSR pairwise ranking loss Graph Attention Network
Closest papers by meaning
- SeqRank: Sequential Ranking of Salient Objects (AAAI 2024 · similarity 0.86)
- Language-Guided Salient Object Ranking (CVPR 2025 · similarity 0.86)
- A Motion-aware Spatio-temporal Graph for Video Salient Object Ranking (NeurIPS 2024 · similarity 0.86)
- CaRDiff: Video Salient Object Ranking Chain of Thought Reasoning for Saliency Prediction with Diffusion (AAAI 2025 · similarity 0.86)
- Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement (CVPR 2024 · similarity 0.85)
- Alignment-Free RGB-T Salient Object Detection: A Large-Scale Dataset and Progressive Correlation Network (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.