Research map / Image generation / Autoregressive and latent generation
Interactive Mars Image Content-Based Search with Interpretable Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a ProtoPNet-style prototype classifier (Proto-MSLNet) on Mars Science Laboratory surface images with an added intra-class diversity loss, to provide interpretable evidence for the NASA PDS Image Atlas.
Problem
Content-based classification of planetary images in the NASA PDS Image Atlas needs to be interpretable so users and developers can validate the evidence behind classifications; existing classifiers are opaque.
Approach
Applies the ProtoPNet prototypical-part architecture to Curiosity rover imagery and adds a diversity loss term that discourages prototypes within a class from being visually similar or from the same training image, and evaluates prototype correctness and diversity.
Key results
- Proto-MSLNet (ResNet18) reaches 86.39% Acc(0.9) at 15.5% abstention for the non-interpretable baseline, versus about 2.5 points lower accuracy with 24.5% abstention for the prototype version (reported as a trade-off).
- The diversity loss raised test accuracy by about 2 percentage points without significant train accuracy drop.
- For Night Sky, the diversity loss substantially increased prototype diversity, from at most about 4 unique source images among 10 prototypes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the diversity loss term (ablation) | Proto-MSLNet · MSL surface test | accuracy: 2 percentage point rise | Helped |
| Add a prototypical layer | VGG19 (MSLNet) · MSL surface test | accuracy: 81.3% -> 75.1% | Hurt |
| Add a prototypical layer | ResNet18 (MSLNet) · MSL surface test | accuracy: 79.5% -> 74.8% | Hurt |
| Add a prototypical layer | ResNet18 (MSLNet) · MSL surface test | abstention rate: 15.5% -> 24.5% | Hurt |
| Remove TempCal (ablation) | VGG19 + P-MSLNet · MSL surface test | Acc (0.9): 82.5% -> 94.5% | Helped |
| Remove VectorCal (ablation) | VGG19 + P-MSLNet · MSL surface test | Acc (0.9): 82.5% -> 94.9% | Helped |
| Remove TempCal (ablation) | ResNet18 + P-MSLNet · MSL surface test | Acc (0.9): 83% -> 91.63% | Helped |
| Remove VectorCal (ablation) | ResNet18 + P-MSLNet · MSL surface test | Acc (0.9): 83% -> 92.2% | Helped |
Surprising findings
- Prototype classifiers trade a drop in accuracy and a large rise in abstention rate for interpretability, which the authors deploy anyway.
Limitations
- Small validation and test sets and only a few hyperparameter trials; a more rigorous search is planned.
- Interpretable model has lower accuracy and higher abstention than the deployed black-box classifier.
- No user study yet to evaluate explanation usefulness.
Experimental setup
Datasets: MSL surface dataset ImageNet
Benchmarks: MSL surface dataset
Metrics: accuracy accuracy at 0.9 confidence threshold abstention rate prototype diversity prototype correctness (in-class prototype fraction)
Baselines: MSLNet (VGG19) MSLNet (ResNet18) most common class baseline
Builds on: ProtoPNet ImageNet pretraining Adam
Closest papers by meaning
- Mars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks (NeurIPS 2025 · similarity 0.84)
- Position: Data-driven Discovery with Large Generative Models (ICML 2024 · similarity 0.78)
- AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing (NeurIPS 2025 · similarity 0.78)
- DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery (CVPR 2025 · similarity 0.78)
- VESTA: Visual Exploration with Statistical Tool Agents (NeurIPS 2026 · similarity 0.77)
- T-MARS: Improving Visual Representations by Circumventing Text Feature Learning (ICLR 2024 · similarity 0.77)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.