Research map / Label-efficient and robust learning / Recognition and long-tail
Applying language models to algebraic topology: generating simplicial cycles using multi-labeling in Wu's formula
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames generation of simplicial cycles (elements of the intersection of normal closures in Wu's formula for pi_{n+1}S^2) as language modelling over Dyck-like words, using multi-label information via ignore, mask, and prompt strategies with GPT-2 models trained from scratch.
Problem
Generating elements of the intersection of normal closures R_0∩...∩R_n in a free group, which are the numerator of Wu's formula for homotopy groups of S^2, i.e. sampling simplicial cycles without knowing the denominator.
Approach
Reformulate simplicial cycle generation as sampling from intersections of Dyck-n-like word sets, and train GPT-2 language models that use multi-labels (membership in each normal closure) via ignoring, attention-head masking, or prompt tokens, compared against random search, evolutionary, and greedy non-neural baselines.
Key results
- Prompt-based model achieved completion ratio 0.684 for n=4 and 0.550 for n=5, versus 0.032 (random) and 0.016 (greedy) for n=4 baselines.
- For n=3 the Ignore model (0.36) outperformed Prompt (0.30) and Mask (0.16), while Greedy reached 0.479, the best n=3 result.
- Non-trivial words (elements of the full intersection outside the symmetric commutator subgroup) are substantially harder for the models to generate than trivial ones.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-2 language models instead of non-neural baselines | Prompt · n=3 | completion ratio: 0.479 -> 0.30 | Hurt |
| Use GPT-2 language models instead of non-neural baselines | Prompt · n=4 | completion ratio: 0.016 -> 0.684 | Helped |
| Use GPT-2 language models instead of non-neural baselines | Prompt · n=5 | completion ratio: 0.025 -> 0.550 | Helped |
| Use multi-label prompts instead of ignoring multi-label information | GPT-2 · n=3 | completion ratio: 0.36 -> 0.30 | Hurt |
| Use multi-label prompts instead of ignoring multi-label information | GPT-2 · n=4 | completion ratio: 0.356 -> 0.684 | Helped |
| Use multi-label prompts instead of ignoring multi-label information | GPT-2 · n=5 | completion ratio: 0.263 -> 0.550 | Helped |
| Mask attention heads using multi-labels instead of ignoring multi-label information | GPT-2 · n=3 | completion ratio: 0.36 -> 0.16 | Hurt |
| Mask attention heads using multi-labels instead of ignoring multi-label information | GPT-2 · n=4 | completion ratio: 0.356 -> 0.584 | Helped |
| Mask attention heads using multi-labels instead of ignoring multi-label information | GPT-2 · n=5 | completion ratio: 0.263 -> 0.487 | Helped |
Surprising findings
- The greedy non-neural algorithm achieved the highest completion ratio for n=3 (0.479), exceeding all deep models.
- Some generated words were not in the symmetric commutator subgroup, i.e. models produce non-trivial candidates only partially reliably.
Limitations
- Only n up to 5 generators evaluated due to computational resources.
- Generated non-trivial words cannot be verified by human experts; evaluation relies on automatic metrics.
- Deep models struggle with long words in the full intersection, requiring a more compact tokenization.
Experimental setup
Datasets: online-generated free group words from normal closures R_i and partial symmetric commutator subgroups
Benchmarks: online-generated validation set of free group words for n=3,4,5
Metrics: completion ratio reduction ratio cross-entropy loss
Baselines: random search evolutionary (1+1) algorithm greedy algorithm ignore mask prompt
Builds on: GPT-2 Wu's formula SeqGAN Dyck language recognition
Closest papers by meaning
- Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematics (ICML 2025 · similarity 0.84)
- Graphcode: Learning from multiparameter persistent homology using graph neural networks (NeurIPS 2024 · similarity 0.83)
- Incremental Topological Ordering and Cycle Detection with Predictions (ICML 2024 · similarity 0.83)
- Cover learning for large-scale topology representation (ICML 2025 · similarity 0.82)
- Geometry Informed Tokenization of Molecules for Language Model Generation (ICML 2025 · similarity 0.82)
- $\texttt{STRCMP}$: Integrating Graph Structural Priors with Language Models for Combinatorial Optimization (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.