Research map / Architectures and efficiency / RNNs and normalization
Symbolic Learning to Optimize: Towards Interpretability and Scalability
Wenqing Zheng, Tianlong Chen, Ting-Kuei Hu, Zhangyang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill and meta-tune the learned optimizer as a symbolic rule | ResNet-50 on CIFAR-10 · ResNet-50 on CIFAR-10 | test accuracy: 95.31 -> 95.40 | Improved |
| Use the distilled symbolic rule instead of AdamW | BERT base cased · GLUE: SST-2, QQP, QNLI, MRPC, MNLI, CoLA | task metrics: SST-2 accuracy 89.15 -> 91.53; QQP F1/Accuracy 86.3/89.2 -> 87.8/90.8; QNLI accuracy 89.2 -> 90.1; MRPC F1/Accuracy 83.5 | Improved |
| Distill RP(small)(extra) into a symbolic equation | RP(small)(extra) · P2-trained, P3 evaluation | evaluation loss: 0.08 ± 0.05 -> 0.09 ± 0.04 | No clear effect |
| Distill DM into a symbolic equation | DM · P2-trained, P3 evaluation | evaluation loss: 0.23 ± 0.13 -> 0.33 ± 0.26 | Hurt |
| Use equation complexity 100 rather than 10 (ablation) | Distilled RP(small)(extra) · ResNet-50 on CIFAR-10 | test accuracy: 93.5 -> 95.4 | Improved |
| Increase equation complexity from 100 to 200 (ablation) | Distilled RP(small)(extra) · ResNet-50 on CIFAR-10 | test accuracy: 95.4 -> 12.0 | Hurt |
| Remove hyperbolic functions from the symbolic expression (ablation) | DM distilled rule · P2 | R² score: 0.54 -> 0.53 | No clear effect |
| Meta-fine-tune the symbolic rule distilled from RP(small)(extra) (ablation) | ResNet-50 on CIFAR-10, pretraining from P1/P2/P4 · ResNet-50 on CIFAR-10 | test accuracy: 91.3 -> 95.4; 93.8 -> 95.3; 94.4 -> 95.5 | Improved |
| Meta-fine-tune the symbolic rule distilled from DM (ablation) | ResNet-50 on CIFAR-10, pretraining from P1/P2/P4 · ResNet-50 on CIFAR-10 | test accuracy: 13.4 -> 10.0; 70.0 -> 70.7; 83.2 -> 84.5 | Mixed |
| Add more diverse gradient-related input features (ablation) | RP(small)(extra) versus simpler DM/RP variants · P2-trained, P3 evaluation | evaluation loss: DM 0.23 ± 0.13; RP 0.12 ± 0.07; RP(small) 0.09 ± 0.05; RP(small)(extra) 0.08 ± 0.05 | Improved |
| Change the meta-pretraining problem while retaining the RP(small)(extra) feature design (ablation) | RP(small)(extra) symbolic rule · ResNet-50 on CIFAR-10 | test accuracy after meta-fine-tuning: 95.4; 95.3; 95.5 | No clear effect |
Closest papers by meaning
- SYMBOL: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning (ICLR 2024 · similarity 0.90)
- SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery (NeurIPS 2026 · similarity 0.87)
- Learning Provably Improves the Convergence of Gradient Descent (NeurIPS 2025 · similarity 0.87)
- Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · similarity 0.86)
- Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length (ICLR 2025 · similarity 0.86)
- MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
