Research map / Multi-agent systems and games / Cooperative multi-agent RL
Gradient-Guided Credit Assignment and Joint Optimization for Dependency-Aware Spatial Crowdsourcing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RMO, a two-stage recommend-and-match framework for dependency-aware spatial crowdsourcing that uses MAPPO-based subtask recommendation plus KM matching with a multi-dimensional utility, linked by meta-gradients for credit assignment and joint optimization.
Problem
Assigning workers to spatial tasks composed of interdependent subtasks with skill requirements and deadlines, where sparse task-completion rewards make credit assignment hard and the recommendation and matching stages are optimized separately.
Approach
Uses a meta-gradient that links a sparse-reward policy to an auxiliary reward-shaped policy, so the recommendation network and the matching utility weights (via a meta-parameter) are updated in one unified gradient step.
Key results
- RMO achieves 2.2x to 6.6x higher total revenue than baselines across two datasets.
- Meta-gradient-guided training accelerates policy learning; on Haikou, sparse-reward policy underperforms even after 1.0e6 steps.
- Utility hyperparameters selected as {0.95, 0.10} for Chengdu and {0.96, 0.06} for Haikou.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use meta-gradient-guided training instead of sparse-reward-only training (ablation) | RMO · Chengdu | average revenue | No clear effect |
| Use meta-gradient-guided training instead of sparse-reward-only training (ablation) | RMO · Haikou | average revenue | Helped |
| Use reward shaping instead of meta-gradient-guided training (ablation) | RMO · Chengdu and Haikou | average revenue | Mixed |
| Jointly optimize task recommendation and matching utility (ablation) | RMO · Chengdu and Haikou | total revenue | Helped |
| Initialize utility parameters η to different values: {0.2, 0.2}, {0.5, 0.5}, or {0.8, 0.8} (ablation) | RMO · Chengdu and Haikou | utility parameter values | No clear effect |
| Use RMO instead of the compared baselines | RMO · Two real-world datasets | total revenue: 2.2× to 6.6× | Helped |
Surprising findings
- Reward-shaped policy yields higher early revenue but is overtaken by sparse-reward and meta-gradient policies as training progresses.
Limitations
- Meta-gradients involve second-order derivatives and show heightened sensitivity, with sudden hyperparameter jumps during training.
- Evaluation is entirely simulation-based with synthetic workers and skills.
- Gradient derivation approximates dF/du by dF/dr, assuming a positive correlation.
Experimental setup
Datasets: OpenStreetMap road networks DiDi orders
Benchmarks: Chengdu DiDi-derived simulation Haikou DiDi-derived simulation
Metrics: total platform revenue average revenue
Baselines: GREEDY KM REC GAME ODTA
Builds on: MAPPO Kuhn-Munkres algorithm Graph Attention Network Generalized Advantage Estimation
Closest papers by meaning
- MAGNOLIA: Matching Algorithms via GNNs for Online Value-to-go Approximation (ICML 2024 · similarity 0.84)
- Two-sided Competing Matching Recommendation Markets With Quota and Complementary Preferences Constraints (ICML 2024 · similarity 0.83)
- UNEX-RL: Reinforcing Long-Term Rewards in Multi-Stage Recommender Systems with UNidirectional EXecution (AAAI 2024 · similarity 0.83)
- Fair Resource Allocation in Multi-Task Learning (ICML 2024 · similarity 0.82)
- SP-CACW: Convergence-Aware Client Weighting for Selfish Personalized Learning (NeurIPS 2026 · similarity 0.82)
- Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.