Research map / Label-efficient and robust learning / Recognition and long-tail
FAIR Universe HiggsML Uncertainty Dataset and Competition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Describes the FAIR Universe HiggsML Uncertainty Challenge and its 280M-event tabular dataset of simulated LHC collisions, where participants produce 68% confidence intervals on the Higgs signal strength under six systematic biases.
Problem
Measuring the Higgs boson signal strength with valid confidence intervals when the simulator is imperfect and systematic (epistemic) nuisance biases shift the data distribution away from the training data.
Approach
A public competition and benchmark dataset with parameterised systematic biases (six nuisance parameters supplied via a biasing script), a quantile score balancing interval width and coverage, and a permanent release enabling long-term benchmarking of uncertainty-aware methods; top entries use calibrated multiclassifiers with unbinned likelihood surrogates and density-ratio/contrastive normalising
Key results
- Top public leaderboard quantile scores: HEPHY 0.878, Ibrahime 0.823, Hzume 0.179.
- On re-evaluation over 1000 trials of 100 pseudo-experiments, merged scores were HEPHY -0.582, Ibrahim -0.576, Hzume -2.16; bootstrap showed the top two cannot be reliably ranked.
- HEPHY's unbinned surrogate improved the expected 1-sigma CI on the signal strength by 20% over a traditional binned classifier-template analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use an unbinned surrogate instead of a binned classifier-template analysis | HEPHY likelihood model · HiggsML Uncertainty Challenge inference | expected 1σ confidence interval: 20% | Helped |
| use the unbinned model instead of the binned case | HEPHY likelihood model · HiggsML Uncertainty Challenge inference | impact of calibration-related systematics on μ: up to 65% | Helped |
| rank HEPHY ahead of Ibrahim based on final quantile score | HEPHY and Ibrahim · HiggsML Uncertainty Challenge final evaluation | quantile score: -0.582 vs -0.576 | No clear effect |
Surprising findings
- Clipping confidence intervals to the maximum mu of 3 caused CI width to fall at large mu, a competition-specific artefact the authors flag as a 'hack'.
- The two leading approaches had very similar performance but weakly correlated results, suggesting the optimum has not been reached.
Limitations
- Biases are exactly parameterised ('known unknowns'); unexpected biases are not covered.
- Simulation uses leading-order Pythia and simplified Delphes detector simulation rather than full ATLAS Geant4 simulation.
- Only one lepton, one hadronic tau and up to two jets are provided, omitting additional particle-identification information.
Experimental setup
Datasets: FAIR Universe HiggsML dataset (280M events, 28 features)
Benchmarks: FAIR Universe HiggsML Uncertainty Challenge leaderboard
Metrics: quantile score coverage average interval width penalty function
Baselines: XGBoost-based baseline with bin calibration
Builds on: HiggsML challenge XGBoost MINUIT
Closest papers by meaning
- The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning (NeurIPS 2024 · similarity 0.77)
- Scaling Physical Reasoning with the PHYSICS Dataset (NeurIPS 2025 · similarity 0.76)
- CosmoBench: A Multiscale, Multiview, Multitask Cosmology Benchmark for Geometric Deep Learning (NeurIPS 2025 · similarity 0.76)
- Patch Hierarchical Attention Transformers for Efficient Particle Jet Tagging (NeurIPS 2026 · similarity 0.75)
- On Fine-Grained Distinct Element Estimation (ICML 2025 · similarity 0.74)
- The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning (NeurIPS 2024 · similarity 0.74)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.