Research map / Learning theory and algorithms / Combinatorial and search optimization
End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives LQR optimal control for fractional-order linear time-invariant systems, proposes a least-squares system identification with sample-complexity bounds, and builds FOLOC, an end-to-end RNN/Transformer/FNO framework that jointly identifies system parameters and predicts optimal control.
Problem
Optimal control and system identification for fractional-order linear time-invariant (FOLTI) systems, which are non-Markovian with long-range memory and lack standard control methodologies.
Approach
Extends LQR to Grünwald–Letnikov fractional-order discrete dynamics via closed-form least-squares and Lagrange-multiplier solutions, and couples a learned system-identification module (RNN+MLP, attention encoder) with a Fourier Neural Operator control module under a composite loss.
Key results
- Mean MSE decreases by 81.36% compared with the LTI system learning framework on synthetic FOLTI data.
- Increasing time horizon from T=8 to T=128 reduces MSE by 97.37% and MAE by 92.23%.
- Cart-pole overall mean MSE below 3.99e-5 and quadrotor mean MSE 0.13 with noise up to scale 1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use fractional-order system learning rather than LTI system learning | Theoretical system identification · Synthetic FOLTI system | mean MSE: decreases by 81.36% | Helped |
| Increase the time horizon from T = 8 to T = 128 | FOLOC · Synthetic FOLTI system | MSE and MAE: decrease by 97.37% and 92.23% respectively | Helped |
| Vary fractional order across alpha = 0.1, 0.3, 0.5, 0.7, and 0.9 | FOLOC · Synthetic FOLTI system | MSE and MAE | No clear effect |
| Increase state dimension from n = 2 to n = 6 or n = 8 | FOLOC · Synthetic FOLTI system | MSE and MAE: 13.01% (18.79%) at n = 6 and 24.06% (25.45%) at n=8 | Helped |
| Vary training sample count from N = 1000 to N = 8000 | FOLOC · Synthetic FOLTI system | MSE: MSE only decreases by 1.65% at N = 1000 compared with N = 8000 | No clear effect |
| Use non-Gaussian process noise instead of Gaussian noise | FOLOC · Synthetic FOLTI system | MSE/MAE: maximal increases of MSE/MAE by 24.42%/6.67% (Cauchy/Sinc-squared) | Hurt |
| Vary Gaussian noise scale | FOLOC · Synthetic FOLTI system | MSE and MAE | No clear effect |
| Vary Gaussian noise scale | FOLOC · Cart-pole | MSE and MAE | No clear effect |
| Vary Gaussian noise scale | FOLOC · Quadrotor | MSE and MAE | No clear effect |
Surprising findings
- Noise scale sigma=1 Gaussian noise gives a noticeably higher error (MSE 10.2e-3) than smaller noise scales, while smaller scales perform similarly.
- Training with fewer samples (N=1000) yields MSE only 1.65% worse than with N=8000, suggesting limited sensitivity to sample count.
Limitations
- Framework is restricted to LTI-based fractional-order systems; non-LTI settings not validated.
- System identification assumes diagonal elements of A are known and the theory relies on strict assumptions and Gaussian process noise.
- Learned intermediate parameters do not map one-to-one to theoretical quantities.
Experimental setup
Datasets: synthetic FOLTI trajectories with Gaussian and other noise cart-pole simulated dynamics quadrotor simulated dynamics
Benchmarks: synthetic FOLTI systems cart-pole dynamics quadrotor dynamics
Metrics: MSE MAE
Baselines: end-to-end LTI learning framework
Builds on: linear quadratic regulator Grünwald–Letnikov fractional derivative Fourier Neural Operator neural operator-assisted Krylov iterations
Closest papers by meaning
- LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning (NeurIPS 2026 · similarity 0.83)
- Variational Inference for SDEs Driven by Fractional Noise (ICLR 2024 · similarity 0.83)
- Model-free Low-Rank Reinforcement Learning via Leveraged Entry-wise Matrix Estimation (NeurIPS 2024 · similarity 0.83)
- Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation (AAAI 2025 · similarity 0.83)
- Neural Variable-Order Fractional Differential Equation Networks (AAAI 2025 · similarity 0.83)
- Neural Time-Reversed Generalized Riccati Equation (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.