Research map / Label-efficient and robust learning / Recognition and long-tail

DFF: Decision-Focused Fine-Tuning for Smarter Predict-Then-Optimize with Limited Data

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Dense features and foundation priors · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Decision-Focused Fine-tuning (DFF), which wraps any backbone predictor with a trust-region bias correction layer (input-dependent multiplicative weights bounded by a sigmoid, residual bias zero) trained with SPO+ decision loss, to improve decision quality while bounding prediction shift under limited data.

Problem

Decision-focused learning under limited data can distort predictions away from their physical meaning (multiplicative shifts), and DFL is hard to apply to non-differentiable backbone models such as tree ensembles and simulation models in predict-then-optimize pipelines.

Approach

Fine-tune a fixed backbone prediction with an input-dependent multiplicative-plus-bias correction whose output is constrained to a percentage trust region around the original prediction (offset-scaled sigmoid), so the decision loss is optimized via SPO+ without touching the backbone's weights, and the prediction shift is theoretically bounded.

Key results

• On the synthetic benchmarks, DFF (fine-tuned 2-fold Boost) achieves the lowest NDR in most settings, e.g., 4.39% vs 4.49% for 2-fold Boost on network flow Dataset 1, and 4.81% on Dataset 2.
• On DiDi XGBoost fine-tuning, DFF reduces NDR to 2.39% (vs 2.54% XGBoost and 2.40% NN-SPO+) while keeping MSE at 0.49 (XGBoost 0.45; NN-SPO+ 2.04).
• Fine-tuning the DiDi simulation model with DFF reduces normalized decision regret by 24.37% relative to the original simulation model (2.79% to 2.11%) with MSE essentially unchanged.

Measured effects reported in the paper

Surprising findings

• Direct DL training (NN-SPO+) induced multiplicative shifts, producing unimodal predictions where the ground truth is bimodal, while DFF's predictions matched the ground truth distribution.
• On the portfolio Dataset 1, NN-MSE outperformed DFF because the trust region limits improvement over the weaker 2-fold Boost base model.

Limitations

• Constrained fine-tuning cannot exceed the trust region, so gains are limited when the backbone is weak (e.g., portfolio Dataset 1).
• Experiments use only SPO+ for gradients and a fixed epsilon of 0.5 for synthetic data; generality to other decision-loss gradient methods is only claimed.
• Extension to constraints with uncertain parameters and multi-task or multi-objective settings is left to future work.

Experimental setup

Datasets: synthetic network flow data synthetic portfolio optimization data DiDi Chuxing ride-hailing subsidy market data

Benchmarks: network flow problem portfolio optimization problem DiDi resource allocation (subsidy allocation) shortest path (predictions distribution)

Metrics: normalized decision regret (NDR) MSE RMSE increment bound cosine similarity

Baselines: Random Forest SPO Forest Boost Dboost NN-MSE NN-SPO+ 2-fold Boost XGBoost Average allocation

Builds on: SPO+ PyEPO cross-fitting hyper-network

Closest papers by meaning

• Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints (NeurIPS 2025 · similarity 0.88)
• Solver-Free Decision-Focused Learning for Linear Optimization Problems (NeurIPS 2025 · similarity 0.88)
• Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · similarity 0.85)
• Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.85)
• Contextual Optimization Under Model Misspecification: A Tractable and Generalizable Approach (ICML 2025 · similarity 0.85)
• Locally Convex Global Loss Network for Decision-Focused Learning (AAAI 2025 · similarity 0.84)