Research map / Architectures and efficiency / Attention and transformers
Analytical-Chemistry-Informed Transformer for Infrared Spectra Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ACT, a Transformer for IR spectra with learnable chemometric-inspired spectral processing (reversible baseline correction, patch tokenization) and spectral-attention (inter-spectral reference attention plus intra-spectral attention guided by a band correlation matrix), evaluated on 9 calibration-transfer and regular spectral tasks.
Problem
Calibration transfer in IR spectroscopy: models degrade when training and test spectra come from different spectrometers or collection conditions, and existing deep networks ignore IR-specific prior knowledge from analytical chemistry.
Approach
Embed chemometrics knowledge into a Transformer: reversible baseline correction with learnable baseline restoration and label projection, patch-based spectral tokens, and spectral-attention that attends to a set of reference calibration spectra (inter-spectral) and uses a learnable band-correlation matrix (intra-spectral) to learn domain-invariant representations without target-domain data.
Key results
- On calibration transfer tasks ACT reduces RMSEP by about 27% on Tablet tasks versus the best deep baseline, and more than 20% overall as claimed in the abstract (e.g., Tablet(1,2) RMSEP 6.941 vs 8.810 for Spectraformer).
- On regular analysis ACT achieves an average rank of 1.2 across regression and classification tasks, versus second-best 3.1 (AggMapNet).
- Ablation on Tablet(1,2) shows base model RMSEP 9.977 improves to 6.941 with tokenization, spectral-attention and learnable processing combined.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ACT instead of Spectraformer | IR spectral modeling · Tablet(1, 2) | RMSEP: 8.810 ± 2.048 -> 6.941 ± 3.255 | Helped |
| use ACT instead of Spectraformer | IR spectral modeling · Tablet(2, 1) | RMSEP: 9.279 ± 2.239 -> 6.155 ± 0.985 | Helped |
| use ACT instead of AggMapNet | IR spectral modeling · MF(R562,R568) | RMSEP: 2.631 ± 0.227 -> 2.293 ± 0.624 | Helped |
| use ACT instead of AggMapNet | IR spectral modeling · MF(R568,R562) | RMSEP: 3.775 ± 0.124 -> 2.223 ± 0.202 | Helped |
| add the proposed tokenization method (ablation) | Base model · Tablet(1, 2) | RMSEP: 9.977 -> 8.491 | Helped |
| add learnable spectral processing without baseline reconstruction (ablation) | Base + Token · Tablet(1, 2) | RMSEP: 8.491 -> 8.054 | Helped |
| add baseline reconstruction to learnable spectral processing (ablation) | Base + Token + LearnProc’ · Tablet(1, 2) | RMSEP: 8.054 -> 7.833 | Helped |
| add intra-spectral attention (ablation) | Base + Token · Tablet(1, 2) | RMSEP: 8.491 -> 7.562 | Helped |
| add inter-spectral attention (ablation) | Base + Token · Tablet(1, 2) | RMSEP: 8.491 -> 7.265 | Helped |
| combine intra- and inter-spectral attention (ablation) | Base + Token · Tablet(1, 2) | RMSEP: 8.491 -> 7.152 | Helped |
| replace learnable spectral processing with SNV preprocessing (ablation) | Base + Token + SpectrAttn · Tablet(1, 2) | RMSEP: 6.941 -> 9.077 | Hurt |
| replace learnable spectral processing with derivative preprocessing (ablation) | Base + Token + SpectrAttn · Tablet(1, 2) | RMSEP: 6.941 -> 8.032 | Hurt |
Surprising findings
- Using target-domain data for training improves ACT on Tablet(1,2) substantially (RMSEP 6.94 to 5.10), and random reference-set seed changes RMSEP from 5.59 to 7.78.
Limitations
- Calibration transfer gains are not uniform: on MF(R568,R562) ACT's MAE is worse than AggMapNet (-6.0%), and standard deviations are large on Tablet tasks.
- Hyper-parameters are tuned per dataset, and the reference-spectra selection strongly affects performance.
Experimental setup
Datasets: Tablet Melamine Mango DMC Strawberry Apple leaf
Benchmarks: Tablet calibration transfer Melamine calibration transfer Mango DMC Strawberry puree Apple leaf
Metrics: RMSEP MAE ACC F1
Baselines: DeepSpectra AggMapNet TeaNet Spectraformer PLS di-PLS JDOT DIPALS
Builds on: Vanilla Transformer chemometric iterative polynomial baseline correction DeepSpectra
Closest papers by meaning
- IR-Agent: Expert-Inspired LLM Agents for Structure Elucidation from Infrared Spectra (ICLR 2026 · similarity 0.84)
- Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistry (NeurIPS 2024 · similarity 0.81)
- From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction (ICLR 2024 · similarity 0.81)
- RamanBench: A Large-Scale Benchmark for Machine Learning on Raman Spectroscopy (NeurIPS 2026 · similarity 0.81)
- Molecule Design by Latent Prompt Transformer (NeurIPS 2024 · similarity 0.80)
- ChemPile: A 250 GB Diverse and Curated Dataset for Chemical Foundation Models (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.