Research map / Large language models / AI and society
Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Algebraic Combinatorics Dataset Repository (ACD Repo), nine ML datasets from algebraic combinatorics framed as conjecture-generation tasks, with narrow-model and LLM baselines.
Problem
Existing ML-for-math resources mostly cover textbook or competition-level math, not the open-ended, research-level raw mathematical data (examples) that mathematicians use to form conjectures.
Approach
A collection of nine datasets, each pairing an open or foundational algebraic combinatorics question with a large set of examples and an ML-friendly prediction task, intended for interpretability-driven or program-synthesis-driven conjecture generation rather than standard benchmarking.
Key results
- Narrow models (MLPs, transformers) struggle on symmetric group character regression and RSK, with RSK models no better than guessing the mean.
- o1-mini and GPT-4o produced program-synthesis solutions reaching 100% test accuracy on Schubert structure constants, but this exploited a spurious sampling artifact (permutation length parity).
- On mHeight, LLMs and narrow models achieve fairly high accuracy, though for n=10 naive majority-class guessing already reaches 94.2%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use program synthesis with mathematical context instead of direct in-context prediction | LLMs on Schubert structure constants · Schubert structure constants (n = 4, 5) | accuracy: 50-60% -> 100% | Helped |
| Exploit the original zero-coefficient sampling strategy | LLMs on Schubert structure constants · Schubert structure constants (n = 4, 5) | accuracy: 100% | Helped |
| Use a specialized graph neural network architecture | Quiver mutation equivalence classification · Quiver mutation equivalence | accuracy | Helped |
Surprising findings
- LLM program synthesis reached 100% accuracy on Schubert structure constants by reverse-engineering a spurious correlation from the dataset sampling process.
- Transformers struggled with permutation parity in one-line notation but did much better with inversion-vector representations.
Limitations
- Datasets are not designed as traditional benchmarks; high accuracy may not yield mathematical insight.
- Sampling strategies can introduce spurious correlations that models exploit (e.g., Schubert constants, Grassmannian negatives).
- Tables with baseline results were largely empty in the provided text extraction, so specific numbers could not be confirmed.
Experimental setup
Datasets: Algebraic Combinatorics Dataset Repository (ACD Repo) Grassmannian cluster variable SSYT data from Chang et al. (positive examples)
Benchmarks: ACD Repo nine tasks (character regression, mHeight, Grassmannian cluster classification, KL coefficients, RSK, Schubert
Metrics: accuracy macro F1 Matthews correlation coefficient regression accuracy
Baselines: MLP transformer Claude 3.5 Sonnet GPT-4o GPT-4o Mini o1-mini
Builds on: Sage PGExplainer Murnaghan-Nakayama rule Robinson-Schensted-Knuth correspondence
Closest papers by meaning
- On a Combinatorial Problem Arising in Machine Teaching (ICML 2024 · similarity 0.87)
- Neural Discovery in Mathematics: Do Machines Dream of Colored Planes? (ICML 2025 · similarity 0.87)
- HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematics (ICLR 2025 · similarity 0.86)
- Extending Test-Time Augmentation with Metamorphic Relations for Combinatorial Problems (ICML 2024 · similarity 0.85)
- Position: $C^*$-Algebraic Machine Learning $-$ Moving in a New Direction (ICML 2024 · similarity 0.85)
- Machines and Mathematical Mutations: Using GNNs to Characterize Quiver Mutation Classes (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.