Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics

DynaFront@NeurIPS, December 6, 2025

Transcript

1. Decision-Dependent Stochastic Optimization: Zhiyu He The Role of Distribution Dynamics DynaFront@NeurIPS, December 6, 2025 Joint work with Saverio Bolognani, Florian Dörfler, Michael Muehlebach 1
2. (re)optimize decision-making involving 𝑢𝑢 𝜇𝜇0 𝜇𝜇𝑘𝑘 Randomness is common (re)sample & random variable 𝑝𝑝 follows 𝜇𝜇 𝑘𝑘 decision variable random variable performance objective preferences position demands price optimal decision endogenous uncertainty & so is feedback 2
3. Comparison Performative prediction Our formulation covariate shift [Perdomo et al., 2020; Mendler-Dünner et al., 2020] prediction of ML models feature decision 𝑢𝑢 steady state distribution dynamics random var. 𝑝𝑝 ~ 𝜇𝜇ss (𝑢𝑢) steering convergence of distributions & optimality 3 with params. random var. 𝑝𝑝 ~ 𝜇𝜇𝑘𝑘 𝑝𝑝 ~ 𝜇𝜇ss (𝑢𝑢) performative stability structure of dynamics
4. Prior works under changing distributions Mean-field formulation Control in probability spaces Performative prediction Distributionally robust optimization [Duchi and Namkoong, 2021, Kuhn et al., 2024] conservative & non-adaptive [Perdomo et al., 2020; Hardt and Mendler-Dünner, 2023] (static) distribution maps & performative stability [Caines, 2021, Laurière et al., 2022, Cui and Koeppl, 2021] cumulative cost & effort of learning mean-field dynamics [Chen et al., 2021, Lanzetti et al., 2023] aim for a desired final distribution instead of opt. 4 others: feedback opt., online stochastic opt., … less attention on dynamics & optimality
5. Take-home messages distribution dynamics finite-sample generalization optimality decision take into account & control leverage fixed sample distributions exploit adaptation & steering 𝜇𝜇ss (𝑢𝑢𝑘𝑘 ) 𝜇𝜇 𝑘𝑘 5
6. Preview example: Affinity maximization in a polarized population opportunistic political party promoting an ideology large-scale population of individuals 6
7. Polarized dynamics steady state given a fixed ideological position 𝑢𝑢 perpendicular angle → not influenced acute angle → influenced obtuse angle → anti-influenced Model [Hazła et al., 2024; Gaitonde et al., 2021] 𝑢𝑢 𝑝𝑝𝑘𝑘 𝑝𝑝𝑘𝑘 = 𝑢𝑢 = 1 𝑝𝑝𝑘𝑘 T𝑢𝑢 utility individual position ideology initial position 7
8. Polarized dynamics Model [Hazła et al., 2024; Gaitonde et al., 2021] Goal population-wide affinity optimization utility nonlinear map Distributional steady state 𝜇𝜇ss utility individual position ideology initial position 8
9. Problem formulation distribution dynamics state decision exogenous input / param. steady-state distribution steady-state map steady-state distribution utility decision-dependent problem pushforward contracting in 𝑝𝑝 & Lipschitz in 𝑢𝑢, 𝑑𝑑 finite absolute moments differentiable differentiable decision distribution shifts 9
10. Challenges from feedback loops random variable 𝑝𝑝 decision-making decision dependence dynamics non-stationary distributions unknown nonlinear dynamics convergence optimality 10
11. Ideally evaluate exact gradients of Key idea steady state induced by 𝑑𝑑 ∼ 𝜇𝜇 𝑑𝑑 & 𝑝𝑝 ∼ ℎ(𝑢𝑢,⋅) # 𝜇𝜇 𝑑𝑑 chain rule & law of total derivative conditions on Φ allow swapping ∇ & 𝔼𝔼 Challenges hard to evaluate 𝔼𝔼 (integral) no access to the steady state online decision-making! use current samples from 𝜇𝜇𝑘𝑘 11
12. Online stochastic algorithm adaption anticipation & steering current steady-state a finite-sample approximation of composite stochastic gradient … approximate the true gradient mini batch 12 𝜇𝜇ss (𝑢𝑢𝑘𝑘 ) 𝜇𝜇𝑘𝑘
13. Optimality & convergence of distributions 13 𝜇𝜇ss (𝑢𝑢𝑘𝑘 ) 𝜇𝜇 𝑘𝑘
14. Ingredient: Wasserstein distance 𝜇𝜇 Metric measuring the discrepancy between distributions cost of moving mass 𝜈𝜈 transport plan minimum cost of transporting 𝜇𝜇 onto 𝜈𝜈 price our case: composite norm joint distribution with marginals 𝜇𝜇 & 𝜈𝜈 Why Wasserstein? general distributions with disjoint supports + tractable theory 14
15. Perturbed contraction of distribution dynamics With the assumption of Lipschitz dynamics: closeness between 𝜇𝜇𝑘𝑘 & 𝜇𝜇ss (𝑢𝑢𝑘𝑘 ) when 𝑢𝑢 𝑘𝑘 is close to 𝑢𝑢 𝑘𝑘+1 𝜇𝜇 𝑘𝑘 weakly converges to 𝜇𝜇ss(𝑢𝑢 𝑘𝑘 ) type-1 Wasserstein distance ∈ (0, 1) drift of decisions implication close to 𝜇𝜇𝑘𝑘 𝜇𝜇ss (𝑢𝑢 𝑘𝑘 ) involves 15
16. Optimality in expectation For any window of length 𝑇𝑇, with the step size dynamics & composite structure same rate as SGD for static, nonconvex problems nonconvex naïve vanilla gradient (no anticipation) non-vanishing bias 16
17. Convergence of distributions For any window of length 𝑇𝑇, with the step size 𝜇𝜇𝑘𝑘 weakly converges to 𝜇𝜇ss (𝑢𝑢𝑘𝑘 ) in the long run why this rate? Wasserstein distance coupling of distribution dynamics & algorithm rate of second moments of gradients 17 take the square root
18. High-probability guarantees For any window of length 𝑇𝑇, with , for fixed 𝜏𝜏 ∈ (0,1) holds with probability at least 1 − 𝜏𝜏 same dependence on 𝑇𝑇 & benign scaling with 𝜏𝜏 second moment of gradient Wasserstein distance failure probability 18
19. Φ𝑁𝑁 (with 𝜇𝜇𝑘𝑘 𝑁𝑁) generalize to Φ (with 𝜇𝜇𝑘𝑘 )? How will solutions found based on Generalization true distribution empirical distribution Why? Seeding trials Restricted sampling (privacy, resource constraints) decision decision Question V.S. 𝜇𝜇 𝑘𝑘 fixed initial samples 19
20. Generalization can only access samples 𝑁𝑁 & optimize the empirical objective with a high probability solutions generalize well to the true objective For any window of length 𝑇𝑇, any fixed 𝜏𝜏 ∈ (0,1) when & holds with probability at least 1 − 𝜏𝜏 size of exogeneous inputs from 𝜇𝜇𝑘𝑘 𝑁𝑁 20
21. Affinity Maximization in a Polarized Population 21
22. Optimality vanilla stochastic gradient descent without anticipation our composite algorithm with anticipation & steering anticipate & shape distribution dynamics improvement in optimality 22
23. Histograms of position-decision angles Optimal solution (via IPOPT) Vanilla Composite higher population-wide affinities more acute angles close to the histogram of the optimal solution more samples fall into [95°, 110°] lower affinity 23
24. Performance Optimization with Discrete Choice Distributions 24
25. Distribution Dynamics 25 choice distribution price / loss Model initial distribution Goal maximize gains & preserve diversity gain entropy regular. nonconvex problem
26. Optimality & Distributional Convergence 26 Anticipate & shape distribution dynamics improvement in optimality Relative optimality gap Wasserstein distance current & steady-state distributions
27. Summary Distribution shift is amenable to control go with the flow! Dynamics & feedback Distribution perspective convergence, optimality, generalization adapt to & shape distributions 27
28. Z. He S. Bolognani M. Muehlebach F. Dörfler arXiv GitHub 28 Extensions: game setup, coupling constraints, interacting dynamics …