Research map / Reinforcement learning / Visual and representation RL
The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning
Lucas Fagan, Michele Tarquini, Ali Shehper, Maksymilian Manko, Angus Gruen, Coco Huang, Giorgi Butbaia, Davide Passaro, Sergei Gukov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use substitution supermoves with the Dual-Ring Transformer instead of elementary AC-moves with ResNet | PPO agent · Miller-Schupp benchmark (1190 presentations) | presentations solved: 457 -> 591 | Improved |
| add AC-19 and AC-1M training data (ablation) | PPO-Sub-DRT · Miller-Schupp benchmark (1190 presentations) | presentations solved: 591 -> 610 | Improved |
| use cyclic relative positional encodings instead of canonical form with absolute positional encodings (ablation) | PPO with substitutions · Miller-Schupp benchmark (1190 presentations) | presentations solved: 562.6 -> 588.2 | Improved |
| add AC-19 data to canonical-form PPO (ablation) | PPO-Sub-Canon · Miller-Schupp benchmark (1190 presentations) | presentations solved: 562.6 -> 575.5 | Improved |
| use substitution moves and canonical form in greedy search | greedy search · Miller-Schupp benchmark (1190 presentations) | nodes needed: 1M -> 614 nodes | Improved |
| increase greedy search node budget from 1M to 10M (ablation) | GS-Sub · Miller-Schupp benchmark (1190 presentations) | presentations solved: 640 -> 640 | No clear effect |
| increase greedy search node budget from 10K to 1M (ablation) | GS-Sub · Miller-Schupp benchmark (1190 presentations) | presentations solved: 604 -> 640 | Improved |
| use PPO policy as heuristic in beam search | PPO · Miller-Schupp benchmark (1190 presentations) | mean substitution path length: 13.77 -> 12.39 | Improved |
| use automorphic numbers as reward signal for greedy search | greedy search · Miller-Schupp benchmark (1190 presentations) | presentations solved | No clear effect |
| train generator with automorphism game rather than random automorphisms (ablation) | AC-19 data generation · 200 AC-trivial AC-19 presentations (10K node budget) | difficulty of generated presentations | Improved |
| train on all 1190 benchmark presentations instead of only the 640 AC-trivial ones (ablation) | PPO · Miller-Schupp benchmark (1190 presentations) | presentations solved | Hurt |
Closest papers by meaning
- WHAT MAKES MATH PROBLEMS HARD FOR REINFORCEMENT LEARNING: A CASE STUDY (NeurIPS 2025 · similarity 0.91)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- DiMa: Understanding the Hardness of Online Matching Problems via Diffusion Models (ICML 2025 · similarity 0.85)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.85)
- Computational Hardness of Reinforcement Learning with Partial $q^{\pi}$-Realizability (NeurIPS 2025 · similarity 0.85)
- Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
