Research map / 3D vision / 3D generation
SeqMvRL: A Sequential Fusion Framework for Multi-view Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeqMvRL, which casts multi-view fusion as sequential decision-making, using a pairwise integrator as RL environment and a DQN-based next-view selector trained with a cluster-compactness/separation reward.
Problem
Parallel fusion of all views at once in multi-view representation learning cannot resolve inter-view conflicts, and identifying a good sample-specific view fusion order without supervision is hard.
Approach
Models multi-view fusion as a sequential decision process: a pairwise integrator (environment) progressively fuses views while a DQN-based next-view selector (agent) chooses which view to fuse next, using a reward that favors compact, well-separated clusters and consistency with pseudo-labels.
Key results
- On Flower17 clustering, SeqMvRL reaches 46.23% ACC, 2.11 points above the best prior method.
- On COIL-20 clustering, SeqMvRL achieves 79.68% ACC, 2.61 points above the best prior method (CoMVC/DSMVC-level baselines).
- Classification averages roughly 1.5% accuracy improvement over the state of the art across four datasets, e.g., 90.33% ACC on COIL-20.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sequential fusion with learned view selection instead of the strongest reported parallel-fusion baseline | SeqMvRL · Flower17 | ACCclu: ↑2.11 | Helped |
| use sequential fusion with learned view selection instead of the strongest reported baseline | SeqMvRL · COIL-20 | ACCclu: ↑2.61 | Helped |
| use sequential fusion with learned view selection instead of the strongest reported baseline | SeqMvRL · Caltech-7 | ACCclu: ↑6.73 | Helped |
| use sequential fusion with learned view selection instead of the strongest reported baseline | SeqMvRL · Caltech-20 | ACCclu: ↓0.39 | Hurt |
| use SeqMvRL instead of the strongest reported classification baseline | SeqMvRL · Flower17 | ACCcls: ↑1.13 | Helped |
| use SeqMvRL instead of the strongest reported classification baseline | SeqMvRL · COIL-20 | ACCcls: ↑1.00 | Helped |
| use SeqMvRL instead of the strongest reported classification baseline | SeqMvRL · Caltech-7 | ACCcls: ↑1.98 | Helped |
| use SeqMvRL instead of the strongest reported classification baseline | SeqMvRL · Caltech-20 | ACCcls: ↑1.72 | Helped |
| add PI to the pre-trained LFE representation (ablation) | SeqMvRL · COIL-20 | ACCclu: 54.33 -> 75.67 | Helped |
| add NVS to LFE and PI (ablation) | SeqMvRL · COIL-20 | ACCclu: 75.67 -> 79.68 | Helped |
| add PI to the pre-trained LFE representation (ablation) | SeqMvRL · Caltech-7 | ACCclu: 41.08 -> 55.56 | Helped |
| add NVS to LFE and PI (ablation) | SeqMvRL · Caltech-7 | ACCclu: 55.56 -> 58.25 | Helped |
Surprising findings
- Permuted sequential fusion orders can perform worse than parallel fusion, showing that view order is crucial and a poor order degrades performance.
Limitations
- Relies on pseudo-labels and K-means cluster centers from the PI for the reward, since no supervision is available.
- Benefit of allowing view reuse depends on view quality and must be set as a hyperparameter.
- Evaluated mainly on small image datasets with hand-crafted features and clustering/classification only.
Experimental setup
Datasets: Flower17 COIL-20 Caltech-7 Caltech-20 Oxford-IIIT Pet CAT
Benchmarks: Flower17 COIL-20 Caltech-7 Caltech-20 Oxford-IIIT Pet CAT
Metrics: ACC NMI ARI Precision F-score
Baselines: EAMC SiMVC CoMVC DCP DSMVC MFLVC PDMF RCML CLOVEN
Builds on: Deep Q-Network Deep Divergence-based Clustering loss
Closest papers by meaning
- Learning Fused State Representations for Control from Multi-View Observations (ICML 2025 · similarity 0.87)
- Deep Incomplete Multi-view Learning via Cyclic Permutation of VAEs (ICLR 2025 · similarity 0.85)
- Rethinking Multi-view Representation Learning via Distilled Disentangling (CVPR 2024 · similarity 0.85)
- Enhanced then Progressive Fusion with View Graph for Multi-View Clustering (CVPR 2025 · similarity 0.84)
- Multi-View Causal Representation Learning with Partial Observability (ICLR 2024 · similarity 0.84)
- DMR: Decomposed Multi-Modality Representations for Frames and Events Fusion in Visual Reinforcement Learning (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.