Distributional Learning and Optimization for Decision-Making and Generative Modeling

QC 20261030

Abstract

Probability distributions characterize possible outcomes and their likelihoods, providing a foundation for evaluating and improving decision-making under uncertainty. Distributional learning and optimization seek to characterize, optimize, and steer these distributions to achieve task-specific objectives. A central challenge is to learn and exploit relevant distributional information from finite observations while maintaining statistical reliability and computational tractability. This thesis addresses this challenge in several decision-making and generative modeling problems, and makes four specific contributions. 

The first contribution focuses on steering diffusion and flow models towards desired outcomes. We propose a new steering method that modifies the source distribution while keeping the pretrained vector field fixed. This method is demonstrated with theoretical guarantees and state-of-the-art performance across several image inverse problems. We also develop a tail-aware flow fine-tuning method, which uses conditional value-at-risk (CVaR) to improve either the upper- or lower-tail reward performance. Experiments on high-dimensional text-to-image generation and molecular design demonstrate its superior tail performance with a lower computational cost compared to existing methods.

The second contribution concerns optimizing worst-case performance over a set of plausible distributions to improve robustness. We propose a novel distributionally robust optimization framework that accounts for outliers through a new discrepancy inspired by unbalanced optimal transport.  We establish strong duality and derive tractable reformulations that enable efficient optimization in both centralized and federated settings. Experiments on regression and classification demonstrate the effectiveness of our methods.

The third contribution focuses on risk-averse learning in online games, where agents minimize the CVaR of their cost distributions using only their own sampled costs. We develop zeroth-order learning algorithms that incorporate sample reuse and residual feedback to make effective use of the limited observations. We establish high-probability sub-linear regret for these algorithms and demonstrate their effectiveness in Cournot games.

The last contribution concerns distributional reinforcement learning in stochastic linear quadratic control. For a fixed feedback controller, we derive a closed-form expression of the cumulative random return, which is applicable to any type of independent and identically distributed disturbances. Moreover, we analyze the return distribution, including its variance bound, sensitivity to model errors, and finite-sample approximation errors. Finally, we demonstrate the utility of this distributional information by applying it to risk-averse control.

Link to DiVA