Research map / Vision-language and multimodal / Human and embodied modeling
Towards Automated Petrography
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LITHOS, a large public dataset of paired PPL/XPL thin-section image patches with expert grain annotations across 25 mineral classes, benchmarks standard vision models, and proposes a frozen dual-ViT cross-attention baseline.
Problem
Automating petrography (mineral identification in thin-section microscopy images) is hampered by the lack of large, diverse, publicly available annotated datasets and reproducible polarization-aware models.
Approach
Combines paired plane-polarized and cross-polarized light patches with expert grain annotations (class, coordinates, major/minor axes) and a dual-encoder transformer that fuses frozen single-polarization ViT encoders via cross-attention decoders and an alpha-weighted sum.
Key results
- Binary task: LITHOS Baseline reaches F1 0.861 ± 0.005 and accuracy 0.851 ± 0.002, above the best single-polarization Swin Transformer (XPL) F1 0.855.
- Multi-class task: LITHOS Baseline reaches accuracy 0.623 ± 0.019 and macro F1 0.432 ± 0.002, versus 0.424 for the best single-polarization Swin Transformer (XPL).
- The dataset comprises 211,604 256×256 patches and 105,802 expert-annotated grains across 25 mineral classes, far larger than prior public petrographic datasets (Song et al. with 875 grains and 10 classes).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add paired PPL and XPL inputs with dual-encoder fusion | LITHOS Baseline · LITHOS binary classification (Quartz vs. Non-Quartz) | F1-score: 0.855 \pm 0.006 -> 0.861 \pm 0.005 | Helped |
| Add paired PPL and XPL inputs with dual-encoder fusion | LITHOS Baseline · LITHOS binary classification (Quartz vs. Non-Quartz) | recall: 0.894 \pm 0.021 -> 0.888 \pm 0.023 | Hurt |
| Add paired PPL and XPL inputs with dual-encoder fusion | LITHOS Baseline · LITHOS multi-class classification (25 categories) | F1-score: 0.424 \pm 0.011 -> 0.432 \pm 0.002 | Helped |
| Add paired PPL and XPL inputs with dual-encoder fusion | LITHOS Baseline · LITHOS multi-class classification (25 categories) | recall: 0.421 \pm 0.001 -> 0.415 \pm 0.006 | Hurt |
| Use alpha-weighted summation instead of concatenation followed by linear projection (ablation) | LITHOS Baseline fusion module · LITHOS mineral classification | classification metrics | No clear effect |
Surprising findings
- Low-frequency classes such as Opaque, Heavy mineral and Other have F1-scores below 0.2 despite the overall gains.
- Quartz accounts for about 49.8% of annotated points, so the long-tail imbalance strongly limits macro metrics.
Limitations
- Geographically limited dataset, all samples from Colombian soils and formations.
- Strong class imbalance with quartz dominating; rare classes perform poorly.
- The baseline is large (~674M parameters) because it combines two ViT encoders, and the stated supervision uses only the grain-defining intersection rather than the full axis paths.
Experimental setup
Datasets: LITHOS Dataset
Benchmarks: LITHOS Benchmark (binary Quartz vs Non-Quartz) LITHOS Benchmark (25-class multi-class)
Metrics: accuracy precision recall F1-score
Baselines: ResNet GoogLeNet ViT Swin Transformer
Builds on: Vision Transformer cross modality transformers
Closest papers by meaning
- LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation (NeurIPS 2026 · similarity 0.83)
- Infusing Synthetic Data with Real-World Patterns for Zero-Shot Material State Segmentation (NeurIPS 2024 · similarity 0.82)
- LUWA Dataset: Learning Lithic Use-Wear Analysis on Microscopic Images (CVPR 2024 · similarity 0.81)
- RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025 · similarity 0.81)
- Mars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks (NeurIPS 2025 · similarity 0.80)
- Auto-Linear Phenomenon in Subsurface Imaging (ICML 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.