Research map / Architectures and efficiency / RNNs and normalization
BERTground: A Transformer-Based Model of Background Spectra on the ISS-Based NICER Space Telescope
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BERTground, a BERT-based transformer with physics-grouped feature tokenizers and a tabular deep residual predictor that models NICER background X-ray spectra from 46 MKF spacecraft/detector variables, outperforming SCORPEON.
Problem
Estimating the highly variable background spectra of the NICER X-ray telescope on the ISS, which is needed to subtract background photons from source observations; existing models (3C50, Space Weather, SCORPEON) are rule-based or simple and fail for high-background observations.
Approach
Group physically related MKF features into single tokens via sigmoid-activated group tokenizers, process them with BERT encoders, and replace the MLP predictor with a tabular deep residual network with skip connections from the input, mapping 46 features to 1180 energy-channel counts.
Key results
- BERTground achieves overall RMSE 2.0061 vs SCORPEON 2.9337 and ResNet 2.0868 on the test set.
- For extreme-background observations (>=1000 photons), BERTground RMSE is 25.0204 vs SCORPEON 36.6201.
- Intensity R2 of 0.99 for BERTground versus 0.03 for SCORPEON.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BERTground instead of SCORPEON | NICER background estimation · Overall (all cases) | RMSE: 2.9337 -> 2.0061 | Helped |
| Use BERTground instead of SCORPEON | NICER background estimation · Extreme background (≥1000 photons) | RMSE: 36.6201 -> 25.0204 | Helped |
| Use BERTground instead of SCORPEON | NICER background estimation · High background (100−1000 photons) | RMSE: 0.6313 -> 0.5528 | Helped |
| Use BERTground instead of SCORPEON | NICER background estimation · Medium background (10−100 photons) | RMSE: 0.1585 -> 0.1572 | Helped |
| Use BERTground instead of SCORPEON | NICER background estimation · Low background (< 10 photons) | RMSE: 0.0586 -> 0.0582 | Helped |
| Use BERTground instead of ResNet | NICER background estimation · Overall (all cases) | RMSE: 2.0868 -> 2.0061 | Helped |
| Use BERTground instead of MLP | NICER background estimation · Low background (< 10 photons) | RMSE: 0.0548 * -> 0.0582 | Hurt |
Surprising findings
- SCORPEON's intensity R2 is only 0.03 on the test set, far below the proposed model.
- For low-background observations (<10 photons), the MLP (0.0548) slightly beats BERTground (0.0582).
Limitations
- Single-second observations are modeled independently, without temporal context.
- SCORPEON is not formally published, making comparison harder to verify.
- Not the best model for low-background observations.
Experimental setup
Datasets: NICER background observations from blank sky regions BKGD RXTE 1-6 and 8 (MKF variables)
Benchmarks: NICER background test set (sky region RXTE BKGD 5)
Metrics: RMSE R2 normalized spectra comparison light curve comparison
Baselines: SCORPEON Linear Regression MLP ResNet 3C50 Space Weather
Builds on: BERT FT-Transformer ResNet SCORPEON
Closest papers by meaning
- AstroCompress: A benchmark dataset for multi-purpose compression of astronomical data (ICLR 2025 · similarity 0.81)
- VisRec: A Semi-Supervised Approach to Visibility Data Reconstruction in Radio Astronomy (AAAI 2025 · similarity 0.81)
- VESTA: Visual Exploration with Statistical Tool Agents (NeurIPS 2026 · similarity 0.80)
- Probabilistic Data-Driven Modelling of Astrophysical Transients: The Neural Process Family for Ultrafast and Class-Agnostic Light Curve Reconstruction (NeurIPS 2026 · similarity 0.80)
- AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy (NeurIPS 2025 · similarity 0.80)
- A systematic evaluation of vision-language models for observational astronomical reasoning tasks (NeurIPS 2026 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.