Research map / Point clouds / Point cloud classification
CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CLIP-PCQA, a language-driven no-reference point cloud quality assessment method that retrieves among learnable five-level quality descriptions via CLIP similarity and predicts an opinion score distribution.
Problem
No-reference point cloud quality assessment (NR-PCQA) typically regresses a direct mapping to MOS, which does not match how humans assess quality via discrete descriptions and score distributions.
Approach
Project point clouds to color and depth maps, fine-tune two CLIP ViT-B/16 visual encoders, match visual features to learnable prompts containing five ITU-R quality adjectives via cosine similarity, convert similarities to softmax probabilities as an opinion score distribution, and supervise with EMD and quantile losses plus a contrastive loss.
Key results
- On LS-PCQA, CLIP-PCQA reaches PLCC 0.755 and SRCC 0.736, exceeding the second-best method by more than 0.089 PLCC and 0.091 SRCC.
- On BASICS, CLIP-PCQA achieves PLCC 0.932 and SRCC 0.872, beating 3DTA (0.907/0.825).
- Cross-database training on SJTU-PCQA and testing on BASICS gives PLCC 0.685; training on LS-PCQA and testing on SJTU-PCQA gives 0.873.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine color, depth, and text modalities rather than use color and depth only (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.688 -> 0.755 | Helped |
| use color and text modalities rather than depth and text only (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.536 -> 0.738 | Helped |
| use learnable prompts rather than fixed prompts (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.729 -> 0.755 | Helped |
| place the quality token in the middle of the learnable prompt rather than at the beginning (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.728 -> 0.755 | Helped |
| place the quality token in the middle of the learnable prompt rather than at the end (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.742 -> 0.755 | Helped |
| add quantile loss to EMD loss (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.734 -> 0.743 | Helped |
| add contrastive loss to EMD loss (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.734 -> 0.745 | Helped |
| add contrastive loss to EMD and quantile losses (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.743 -> 0.755 | Helped |
| replace RN50 with ViT-B/16 visual encoder (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.710 -> 0.755 | Helped |
| replace RN50x4 with ViT-B/16 visual encoder (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.724 -> 0.755 | Helped |
| replace RN101 with ViT-B/16 visual encoder (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.721 -> 0.755 | Helped |
| use 6 projection viewpoints rather than 2, 4, 8, or 10 (ablation) | CLIP-PCQA · LS-PCQA | PLCC: 0.745, 0.743, 0.755, 0.752, 0.751 | Mixed |
Surprising findings
- Using the full patch feature map instead of the CLIP class token improves results (PLCC and SRCC drop by 0.014 and 0.017 with the cls token on LS-PCQA).
- Fixed text prompts performed worse than learnable prompts (PLCC 0.729 vs 0.755 on LS-PCQA).
Limitations
- Evaluated only on three PCQA databases with 6-view projections, so generalization beyond these is not established.
- Predicted OSD is not directly constrained by the loss, so distribution quality is only verified qualitatively.
Experimental setup
Datasets: SJTU-PCQA LS-PCQA BASICS
Benchmarks: SJTU-PCQA LS-PCQA BASICS
Metrics: PLCC SRCC RMSE
Baselines: MSE-p2po MSE-p2pl PSNR_yuv PCQM PointSSIM MPED GraphSIM TCDM PQA-Net ResSCNN MM-PCQA GMS-3DQA
Builds on: CLIP CoOp ViT
Closest papers by meaning
- Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment (CVPR 2024 · similarity 0.89)
- Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimization (NeurIPS 2024 · similarity 0.87)
- Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision (ICLR 2024 · similarity 0.84)
- Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.84)
- Revisiting MLLM Based Image Quality Assessment: Errors and Remedy (AAAI 2026 · similarity 0.83)
- FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.