Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-Supervised Interpretable End-to-End Learning via Latent Functional Modularity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoNet, a functionally modular Vision-Transformer end-to-end navigation network trained with imitation loss plus a self-supervised latent-guided contrastive loss, with post-hoc SVM-based decoding of latent decisions for interpretability.
Problem
End-to-end visual autonomous navigation via imitation learning lacks transparent high-level decision-making and typically needs task-level labels (e.g., go-straight, turn-left) for conditional learning.
Approach
A modular perception-planning-control network whose planning module is trained with a self-supervised latent-guided contrastive loss (pairs judged by perceptual feature similarity, no task labels), plus a post-hoc linear multiclass SVM with calibrated probabilities that decodes latent decisions into interpretable task posteriors and entropy.
Key results
- MoNet latent decision reaches terminal similarity score 1.47 vs 1.37 for ViTNet perception features and 1.15 for control-level features, reported as 7%-28% improvement.
- In real-robot tests MoNet succeeded on all 8 right-turn and 32 straight-intersection trials, while ViTNet achieved 5/8 (63%) on right turns; overall CA success 18/19 vs 17/19.
- Removing the LGC loss (MoNet-NoLGC) causes the planning representations to collapse, with near-1.0 similarity score.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MoNet instead of ViTNet | end-to-end navigation policy · RT | success rate: 0.63 -> 1.00 | Helped |
| use MoNet instead of ViTNet | end-to-end navigation policy · CA | success rate: 0.89 -> 0.95 | Helped |
| add latent-guided contrastive loss (ablation) | MoNet · driving-task specificity | similarity score | Helped |
| add the planning module (ablation) | ViTNet-style perception-control network · validation demonstrations | L1 loss | No clear effect |
| add latent-guided contrastive learning (ablation) | MoNet · validation demonstrations | L1 loss | No clear effect |
| replace addition with element-wise multiplication in control modulation (ablation) | MoNet · planning performance | similarity score: 1.47 -> 1.58 | Mixed |
| replace the planning module with an identity function (ablation) | MoNet · planning performance | similarity score: 1.15 -> 1.29 | Helped |
| use latent decision h^d instead of ViTNet perception feature z^p | planning representation · planning performance | similarity score: 1.37 -> 1.47 | Helped |
| use latent decision h^d instead of ViTNet control feature z^c | planning representation · planning performance | similarity score: 1.15 -> 1.47 | Helped |
Surprising findings
- MoNet-MUL achieved the highest terminal similarity score (1.58) but was not chosen as the main model because the RSM analysis showed it struggled to separate LT and RT.
- The straight-intersection (SI) task was merged into straight (ST) because its latent decisions were highly similar to ST, rather than treated as a distinct class.
Limitations
- Evaluated only in controlled indoor corridor environments with a small RC car; not tested outdoors or in dynamic scenes.
- Single-frame model with no temporal modeling; authors propose LSTM-based extension as future work.
- Sensorimotor evaluation is small (143 driving tasks, 16 episodes per model) and the SI/ST merge was a manual design decision.
Experimental setup
Datasets: self-collected real-robot indoor navigation demonstrations (88,326 pairs)
Benchmarks: real-world indoor corridor navigation (ST, SI, LT, RT, CA tasks)
Metrics: L1 loss representational similarity matrix diagonal score task success rate t-SNE cluster separation
Baselines: ViTNet MoNet-MUL MoNet-Iden MoNet-NoLGC
Builds on: ViT conditional imitation learning F1TENTH
Closest papers by meaning
- Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformers (ICLR 2025 · similarity 0.85)
- Monet: Reasoning in Latent Visual Space Beyond Image and Language (CVPR 2026 · similarity 0.85)
- Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.85)
- Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning (ICML 2025 · similarity 0.84)
- Think in Latent, Explain in Language: Self-Explainable Latent Reasoning (ICML 2026 · similarity 0.84)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.