Research map / Label-efficient and robust learning / Domain adaptation and generalization
Progressive Self-Learning for Domain Adaptation on Symbolic Regression of Integer Sequences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes progressive self-learning (PSL) for domain adaptation in symbolic regression of integer sequences, combining ITTD-based source data generation with a verified self-training loop using a transformer formula generator and beam search.
Problem
Neural machine translation-based symbolic regression of integer sequences trained only on randomly generated data generalizes poorly to target OEIS sequences due to distribution gaps between generated and target data.
Approach
Builds source data using initial terms from the target domain (ITTD) to narrow the sequence distribution gap, then runs a self-learning loop in which a transformer generates beam-search candidate formulas for target sequences, verifies them against the observed terms, and retrains on the verified formulas (preferring fewest operators or lowest recurrence degree).
Key results
- PSL reaches 53.1% acc(np=1) on OEIS Easy25 vs 33.4% for DSR, and 54.9% vs 34.5% on OEIS Easy35.
- PSL reaches 27.4% acc(np=10) on OEIS Easy25 and 29.5% on OEIS Easy35, improving over DSR's 19.2% and 21.3%.
- The self-learning loop grows discovered formulas from over 9500 to over 12k on OEIS target sequences.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use progressive self-learning instead of DSR's random-data-only training | SRIS formula generation · OEIS Easy25 (ni = 15, np = 1) | accuracy (%): 33.4 -> 53.1 | Helped |
| Use progressive self-learning instead of DSR's random-data-only training | SRIS formula generation · OEIS Easy25 (ni = 15, np = 10) | accuracy (%): 19.2 -> 27.4 | Helped |
| Use progressive self-learning instead of DSR's random-data-only training | SRIS formula generation · OEIS Easy35 (ni = 25, np = 1) | accuracy (%): 34.5 -> 54.9 | Helped |
| Use progressive self-learning instead of DSR's random-data-only training | SRIS formula generation · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 21.3 -> 29.5 | Helped |
| Replace random initial terms with ITTD (ablation) | PG1 source data · OEIS Easy35 (ni = 25, np = 1) | accuracy (%): 17.2 -> 22.7 | Helped |
| Replace random initial terms with ITTD (ablation) | PG1 source data · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 10.8 -> 14.7 | Helped |
| Increase recurrence degree from d = 6 to d = 12 (ablation) | PG2 source data · OEIS Easy35 (ni = 25, np = 1) | accuracy (%): 22.7 -> 25.8 | Helped |
| Increase recurrence degree from d = 6 to d = 12 (ablation) | PG2 source data · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 14.7 -> 18.7 | Helped |
| Add 2 million linear recurrence pairs (ablation) | PG3 source data · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 18.7 -> 19.5 | Helped |
| Increase ordinary recurrence pairs from 3 million to 5 million while retaining 2 million linear recurrence pairs (ablation) | PG4 source data · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 19.0 -> 19.5 | Helped |
| Replace 2 million linear recurrence pairs with ordinary recurrence pairs (ablation) | PG4 source data · OEIS Easy35 (ni = 25, np = 1) | accuracy (%): 25.0 -> 25.8 | Mixed |
| Replace 2 million linear recurrence pairs with ordinary recurrence pairs (ablation) | PG4 source data · OEIS Easy35 (ni = 25, np = 10) | accuracy (%): 19.0 -> 18.7 | Hurt |
Surprising findings
- Formulas for several OEIS sequences previously without known formulas were discovered, e.g., A134342, A213685, A157795 flagged as having no known formula in OEIS.
Limitations
- Uses a single model for all sequences, which makes it hard to capture diverse formula patterns.
- Accuracy drops substantially when predicting 10 terms (np=10) compared with 1 term.
- Newly discovered formulas are found in later iterations with diminishing returns, favoring simple formulas.
Experimental setup
Datasets: OEIS randomly generated recurrence sequence-formula pairs
Benchmarks: OEIS Easy25 OEIS Easy35
Metrics: accuracy acc(np=1) accuracy acc(np=10) number of discovered formulas
Baselines: FindSequenceFunction (FSF) FindLinearRecurrence (FLR) DSR
Builds on: Deep symbolic regression (d'Ascoli et al. 2022) Lample and Charton 2020 transformer for symbolic math hindsight experience replay
Closest papers by meaning
- Deep symbolic regression for recurrence prediction (ICML 2022 · similarity 0.86)
- Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search (ICML 2023 · similarity 0.84)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.83)
- SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training (ICLR 2024 · similarity 0.83)
- A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data (ICML 2024 · similarity 0.82)
- Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.