Research map / Reinforcement learning / Policy gradient and actor-critic
Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a fractional RL/DRL framework (Dinkelbach-based, with a provably linearly convergent fractional Q-learning) to jointly optimize task updating (waiting) and offloading to minimize expected time-average AoI in MEC systems.
Problem
Minimizing the time-average Age-of-Information of computational tasks in mobile edge computing with unknown edge load dynamics, where the AoI objective is a ratio (fractional) and the action space is hybrid (discrete offloading plus continuous waiting time).
Approach
Reformulate the fractional AoI objective via Dinkelbach's method and a discounted MDP, then iterate a quotient coefficient outer loop with an inner Q-learning loop; combine this fractional cost with D3QN (discrete offloading) and DDPG (continuous updating) in a distributed per-device algorithm.
Key results
- Frac. DRL (OFL+U) reduces average AoI by up to 57.6% compared with the non-fractional DRL benchmark when the drop coefficient is 1.0.
- Joint updating and offloading (Frac. DRL (OFL+U)) reduces AoI by up to 31.3% over offloading-only Frac. DRL (OFL).
- At 75 GHz edge capacity, Frac. DRL (OFL+U) achieves 54.8% AoI reduction vs PGOA and 73.6% vs Non-Frac. DRL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fractional DRL rather than non-fractional DRL | MEC task scheduling · drop coefficient 1.0 | average AoI: 57.6% | Helped |
| jointly optimize task updating and offloading rather than offloading only (ablation) | Frac. DRL · convergence evaluation | average AoI: 31.3% | Helped |
| add task updating policy to offloading policy (ablation) | Frac. DRL · edge capacity 55 GHz | average AoI: up to 19.9% | Helped |
| add task updating policy to offloading policy (ablation) | Frac. DRL · task density 0.65 | average AoI: up to 24.2% | Helped |
| use learning rate 3×10^-4 instead of an excessively large or small D3QN learning rate (ablation) | Frac. DRL (OFL) · hyperparameter evaluation | average AoI: null | Helped |
| use D3QN learning rate 3×10^-1 (ablation) | Frac. DRL (OFL) · hyperparameter evaluation | convergence: null | Hurt |
| use D3QN learning rate 3×10^-5 (ablation) | Frac. DRL (OFL) · hyperparameter evaluation | convergence speed: null | Hurt |
| reduce D3QN batch size to 8 rather than 32 (ablation) | Frac. DRL (OFL) · hyperparameter evaluation | average AoI: null | Hurt |
| use RMSProp or Adam rather than gradient descent (ablation) | Frac. DRL (OFL) · hyperparameter evaluation | converged average AoI: null | Helped |
| increase DDPG actor and critic learning rates above 3×10^-4 and 3×10^-3 (ablation) | Frac. DRL (OFL+U) · hyperparameter evaluation | convergence and average AoI: null | Hurt |
| use DDPG batch size 64 rather than larger or smaller batch sizes (ablation) | Frac. DRL (OFL+U) · hyperparameter evaluation | converged average AoI: null | Helped |
| switch between Adam, gradient descent, and RMSProp (ablation) | Frac. DRL (OFL+U) · hyperparameter evaluation | average AoI: null | No clear effect |
Surprising findings
- Optimal waiting before generating the next task can beat zero-wait, a counter-intuitive effect for AoI versus delay.
Limitations
- Experiments only in a simulated MEC environment with exponential/lognormal assumptions.
- Non-stationarity from multiple independently learning devices is not handled (noted as future work).
Experimental setup
Benchmarks: simulated MEC environment
Metrics: average AoI AoI reduction percentage
Baselines: Random scheduling PGOA Non-Frac. DRL Frac. DRL (OFL)
Builds on: Dinkelbach's method D3QN DDPG Speedy Q-Learning
Closest papers by meaning
- Horizon Reduction Makes RL Scalable (NeurIPS 2025 · similarity 0.81)
- End-to-End Learning Framework for Solving Non-Markovian Optimal Control (ICML 2025 · similarity 0.80)
- Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control (NeurIPS 2025 · similarity 0.80)
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (AAAI 2025 · similarity 0.80)
- DIME: Diffusion-Based Maximum Entropy Reinforcement Learning (ICML 2025 · similarity 0.80)
- DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agent (ICLR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.