Game-theoretic control Receding Horizon Games Model Predictive Control RHG Stability analysis Social choice theory Dynamic resource allocation (energy, water, thermal load…) Economic MPC Optimal job scheduling in computing centers Optimal economic battery operation
network https://bit.ly/3ocqlxp A world of finite resources which need to be shared amongst selfish agents. Ever more allocation decisions are automated. Controllers decide who gets how much when. Game theory models cooperation between self- interested decision makers. → Everyone's decision is optimal given what others do. J. F. Nash, “Equilibrium points in n-person games”, 1950 Solution concept: Nash Equilibrium Predictive Control in Competitive & Cooperative Environments
problems can be cast as Dynamic Resource Allocation Bioresources Fish Forests Water A world of finite resources which need to be shared amongst selfish agents. Ever more allocation decisions are automated. Controllers decide who gets how much when. Traffic control Smart grid Thermal building control Infrastructure ▪ Everyone wants the resource, heterogeneous needs → M conflicting objectives ▪ Limited resource is available → Coupling constraint ▪ System follows dynamics ▪ Need to predict into future (to avoid depletion, collision) Model predictive control Game Theory Receding Horizon Games
et al., “ALGAMES: A Fast Augmented Lagrangian Solver for Constrained Dynamic Games, 2021 Autonomous driving A. Liniger & J. Lygeros, “A Noncooperative Game Approach to Autonomous Racing, 2020 Car racing R. Spica et al., “A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing,” 2020 Drone racing E. R. Stephens et al., “Game Theoretic Model Predictive Control for Distributed Energy Demand-Side Management,” 2015 Demand-Side Management A. D. Paola et al., “Distributed Coordination of Price-Responsive Electric Loads: A Receding Horizon Approach,” 2018 Electric load scheduling
MPC Agent 1 MPC 2 MPC 3 MPC 4 Coupled Coupled Coupled 1 Global MPC Split into M separate controllers ▪ M MPC problems inherently coupled ▪ Can not be combined into one
self-interested agents aim to control a dynamical system. Three sources of coupling arise: 1) Coupled global (local) dynamics: → shared battery system, aquifer water level 2) Coupled stage cost: → energy price increases with consumption 3) Local and coupling constraints: → joint battery state constraints Control input of agent v: Control input of other agents:
IEEE Control System Magazine, 2022. | Facchinei et al., 40R, 2009. We need a “fixed point” concept, equivalent to a minimizer in optimization problems → In competitive games, this is an equilibrium point, specifically a Nash equilibrium
Why select v-GNE from the entire set? 1) Existence & uniqueness results under broad set of assumptions 2) Fast algorithms solving for v-GNEs based on different communication structures 3) Explicit bounds & guarantees on efficiency in terms of utilitarian sum of costs 4) Fairness in terms of equal penalty, or common “shadow price” with access to resource Set of all GNEs is large and potentially infinite BUT for real-time implementation MUST select one Generalized Normalized Variational 20
panel if the power line cannot feed both? ▪ How to decide the charging rate of multiple electric vehicles charging in parallel? ▪ Who gets water in a water distribution network? ▪ Which drivers deserve to use the short path? ▪ How much bandwidth do I allocate to each server? The world of comparability Comparisons in optimization and control
A B “Person B values time more than person A” “Person A values renewable energy 2 times more than person B” “It would be better to allocate water to person A than to person B” Comparability Intrapersonal & interpersonal comparisons
multi-agent engineering systems Comparability Types of comparability (ONC) Ordinal non-comparability (OLC) Ordinal-level Comparability (CNC) Cardinal Non-comparability (CUC) Cardinal unit comparability (CFC) Cardinal Full Comparability I. Shilov, E. Elokda, S. Hall, H. H. Nax, and S. Bolognani, “Welfare and Cost Aggregation for Multi-Agent Control: When to Choose Which Social Cost Function, and Why?
H. Nax, and S. Bolognani, “The Limits of “Fairness” of the Variational Generalized Nash Equilibrium,” For uniform “shadow price” to be meaningful require exact information on gradients of cost functions! Selecting the v-GNE imposes stronger comparability requirements than required for the set of GNEs.
select a “fair” GNE Step 1: Pick appropriate comparability notion Step 2: Select desired fairness notion Step 3: Select GNE maximizing fairness BUT very difficult to solve!! Only in 1D can characterize set of GNEs S. Hall, F. Dörfler, H. H. Nax, and S. Bolognani, “The Limits of “Fairness” of the Variational Generalized Nash Equilibrium,”
Potential games in receding-horizon Pro: ▪ Potential game can be cast as 1 OCP → Apply economic MPC results ▪ State and input constraints Con: ▪ Requires potential structure of game (smaller class) → coupling in cost to be symmetric → restricts “selfishness” of agents Pro: ▪ Treats the M interdependent OCPs, the” full game” ▪ Certificates numerically verifiable, even in a distributed manner Con: ▪ Results currently only hold for input constraints & stable systems ▪ Insights for cost function design are limited Result 2: General games in receding-horizon S. Hall, G. Belgioioso, D. Liao-McPherson, and F. Dorfler, “Receding Horizon Games with Coupling Constraints for Demand-Side Management” S. Hall, D. Liao-McPherson, G. Belgioioso, and F. Dörfler, “Stability Certificates for Receding Horizon Games,”
exists a globally asymptotically stable equilibrium point , (ii) the OCP is recursively feasible; (iii) constraints are satisfied for all times. We get an LMI condition that can be verified numerically!
be verified locally Distributed LMI check Decoupled Dynamics Particularly relevant in large-scale multi-agent settings: (1) Allows for plug-and-play: Agents can join & leave the game any time (e.g. disconnect from grid, leave allocation mechanism …) (2) Verification can be performed locally, reduces communication requirements & computational burden
Side Management Electric vehicle charging Groundwater allocation Supply chains S. Hall, G. Belgioioso, D. Liao-McPherson, and F. Dorfler, “Receding Horizon Games with Coupling Constraints for Demand-Side Management,” S. Hall, L. Guerrini, F. Dörfler, and D. Liao- McPherson, “Receding Horizon Games for Modeling Competitive Supply Chains,”
▪ Reduce cost (energy prices vary over day), buy from grid when energy cheap, discharge battery or shift consumption when energy is expensive ▪ Store energy for times when solar supply is low or blackouts happen (safety backup) Kolter et al., 2011. A single household (smart home) wants to manage their energy consumption optimally. The individual control problem encompasses several components: ▪ Thermal control, temperature & humidity in rooms ▪ EV charging ▪ Electricity consumption of large appliances ▪ Solar panel control ▪ Battery charging Receding Horizon Gams for Demand-Side Management
Cost function: My energy price depends on aggregate demand, i.e., on how much everyone else is buying from the grid 47 Energy Management For a collection of households 2. Coupling constraint: All players together can only buy a max amount of energy Problem: Aggregate demand peaks Production peaks but coupled through same grid! Hours of the day Aggregate energy load Shift peaks!
of New York, collected in 2019 Peak shaving RHG for Demand-Side Management Simulation results Disturbance rejection Advantages of Receding Horizon Games 1. Better approximation of infinite horizon cost; 2. Avoids undesired “end-of-day” effects; 3. Able to reject disturbances, fulfil joint constraints. S. Hall, G. Belgioioso, D. Liao-McPherson, and F. Dorfler, “Receding Horizon Games with Coupling Constraints for Demand-Side Management,”
for non-potential case ▪ Stability for time-varying RHGs ▪ Real-time iterations in RHGs ▪ Turnpike and dissipativity in general RHGs Application ▪ RHG for Demand-side Management ▪ RHG for competitive supply chains ▪ RHG for ground water management ▪ Full-scale DSM ◦ Large Scale Dataset, Distributed algorithm, IEEE 37-bus network, Physical line constraints, other price function, uncertain forecasts etc… 50 Summary & Future Work Receding Horizon Games
advisors & collaborators Giuseppe Belgioioso KTH Stockholm Dominic Liao-McPherson UBC Vancouver Heinrich Nax University of Zürich Saverio Bolognani ETH Zürich Florian Dörfler ETH Zürich And my industry collaborators at Timm Faulwasser TU Hamburg
floriandoerfler
mongodb
lynnandtonic
tammyeverts
shlominoach
mraible
dougneiner
lauravandoore
chrisshort
lara
chriscoyier
jakevdp
rmw
