Online Feedback Optimization with Applications to Power Systems Florian Dörfler Automatic Control Laboratory, ETH Zürich Optimization and Control of Infrastructure Networks School 2021

Acknowledgements Adrian Hauswirth Saverio Bolognani Miguel Picallo Dominic Liao McPherson Giuseppe Belgioioso Lukas Ortmann Verena Häberle URRICULUM VITAE me: Irina Subotić th: 22.03.1993, Belgrade, Serbia mail: [email protected] UCATION p 2016- sent Swiss Federal Institute of Technology in Zurich • Master of Science in Robotics, Systems and Control Irina Subotić 1 / 46

feedforward optimization Optimization System d estimate u w y complex specifications & decision optimal, constrained, & multivariable strong requirements precise model, full state, disturbance estimate, & computationally intensive vs. feedback control Controller System r + u y w − simple feedback policies suboptimal, unconstrained, & SISO forgiving nature of feedback measurement driven, robust to uncertainty, fast & agile response → typically complementary methods are combined via time-scale separation Optimization Controller System r + u y − offline & feedforward real-time & feedback 2 / 46

Example: power system balancing offline optimization: dispatch based on forecasts of loads & renewables 0 50 100 150 200 0 10 20 30 40 50 60 70 80 90 100 marginal costs in €/MWh Capacity in GW Renewables Nuclear energy Lignite Hard coal Natural gas Fuel oil online control based on frequency Frequency Control Power System 50Hz + u y frequency measurement − re-schedule set-point to mitigate severe forecasting errors (redispatch, reserve, etc.) more uncertainty & fluctuations → infeasible & inefficient to separate optimization & control 50 Hz 51 49 generation load control [Milano, 2018] Re-scheduling costs Germany [mio. €] !!" #$% !&' !&( %%(# %%"& %!() %!*! !"## !"#! !"#$ !"#% !"#& !"#' !"#( !"#) [Bundesnetzagentur, Monitoringbericht 2011-2019] 3 / 46

Synopsis & proposal for control architecture power grid: separate decision layers hit limits under increasing uncertainty similar observations in other large-scale & uncertain control systems: process control systems & queuing / routing / infrastructure networks proposal: open with inputs & outputs and online iterative & non-batch optimization algorithm as feedback real-time interconnected control optimization algorithm e.g., ˙ u = −∇φ(y, u) dynamical system ˙ x = f(x, u, w) y = h(x, u, w) actuation u measurement y operational constraints u ∈ U disturbance w 4 / 46

Overview resources, context, & warm-up algorithms & closed-loop stability analysis projected gradient flows on manifolds robust implementation aspects power system case studies throughout 5 / 46

RESOURCES, CONTEXT, & WARM-UP

Resources for today 2020 EECI School https://sites.google.com/view/eeci-igsc-m11 Online Feedback Optimization with Applications to Power Systems Florian Dörfler ETH Zürich European Control Conference 2020 2020 ECC Plenary https://ecc20.eu Publications about 'Online Optimization' Articles in journal, book chapters 1. V. Häberle, A. Hauswirth, L. Ortmann, S. Bolognani, and F. Dörfler. Non-convex Feedback Optimization with Input and Output Constraints. IEEE Control Systems Letters, 5(1):343-348, 2021. Keyword(s): Online Optimization, Nonlinear Optimization, Nonlinear Control Design. [bibtex-entry] 2. A. Hauswirth, S. Bolognani, and F. Dörfler. Projected Dynamical Systems on Irregular Non-Euclidean Domains for Nonlinear Optimization. SIAM Journal on Control and Optimization, 59(1):635-668, 2021. Keyword(s): Online Optimization, Nonlinear Optimization. [bibtex-entry] 3. A. Hauswirth, S. Bolognani, G Hug, and F. Dörfler. Optimization Algorithms as Robust Feedback Controllers. January 2021. Note: Submitted. Available at http://arxiv.org/abs/2103.11329. Keyword(s): Power Networks, Power Flow Optimization, Online Optimization, Nonlinear Optimization. [bibtex-entry] Publications http://people.ee.ethz.ch/~floriand/ Optimization Algorithms as Robust Feedback Controllers Adrian Hauswirth, Saverio Bolognani, Gabriela Hug, and Florian Dörfler Department of Information Technology and Electrical Engineering, ETH Zürich, Switzerland Abstract Mathematical optimization is one of the cornerstones of modern engineering research and practice. Yet, throughout all application domains, mathematical optimization is, for the most part, considered to be a numerical discipline. Opti- mization problems are formulated to be solved numerically with specific algorithms running on microprocessors. An emerging alternative is to view optimization algorithms as dynamical systems. While this new perspective is insightful in itself, liberating optimization methods from specific numerical and algorithmic aspects opens up new possibilities to endow complex real-world systems with sophisticated self-optimizing behavior. Towards this goal, it is necessary to un- derstand how numerical optimization algorithms can be converted into feedback controllers to enable robust “closed-loop optimization”. In this article, we review several research streams that have been pursued in this direction, including extremum seeking and pertinent methods from model predictive and process control. However, our primary focus lies on recent methods under the name of “feedback-based optimization”. This research stream studies control designs that directly implement optimization algorithms in closed loop with physical systems. Such ideas are finding widespread application in the design and retrofit of control protocols for communication networks and electricity grids. In addition to an overview over continuous-time dynamical systems for optimization, our particular emphasis in this survey lies on closed-loop stability as well as the enforcement of physical and operational constraints in closed-loop implementations. We further illustrate these methods in the context of classical problems, namely congestion control in communication networks and optimal frequency control in electricity grids, and we highlight one potential future application in the form of autonomous reserve dispatch in power systems. arXiv:2103.11329v1 [math.OC] 21 Mar 2021 2021 Survey paper https://arxiv.org/abs/2103.11329 6 / 46

Recall: online feedback optimization proposal: combine feedback control & feedforward optimization via open with inputs & outputs and online iterative & non-batch optimization algorithm as feedback real-time interconnected control optimization algorithm e.g., ˙ u = −∇φ(y, u) dynamical system ˙ x = f(x, u, w) y = h(x, u, w) actuation u measurement y operational constraints u ∈ U disturbance w 7 / 46

Academic warm-up example ˙ x = f(x, u) −∇φ(y)∇h(u) u + y = x + d + d controller: optimization of φ(y) plant with steady state x = h(u) 8 / 46

−2 −1 0 1 2 0 2 4 6 8 objective Φ(y) y 0 2 4 6 8 10 12 14 −2 −1 0 1 2 3 = 0.03 t 0 2 4 6 8 10 12 14 −2 −1 0 1 2 3 = 0.15 t 0 0.2 0.4 0.6 0.8 1 1.2 1.4 −2 −1 0 1 2 3 = 0.3 t u (algebraic gradient flow) u (dynamic interconnection) ζ (plant state) 9 / 46

Historical roots & conceptually related work process control: reducing the effect of uncertainty in sucessive optimization Optimizing Control [Garcia & Morari, 1981/84], Self-Optimizing Control [Skogestad, 2000], Modifier Adaptation [Marchetti et. al, 2009], Real-Time Optimization [Bonvin, ed., 2017], ... extremum-seeking: derivative-free but hard for high dimensions & constraints [Leblanc, 1922], ...[Wittenmark & Urquhart, 1995], ...[Krstić & Wang, 2000], ..., [Feiling et al., 2018] MPC with anytime guarantees (though for dynamic optimization): real-time MPC [Zeilinger et al. 2009], real-time iteration [Diel et al. 2005], [Feller & Ebenbauer 2017], etc. optimal routing, queuing, & congestion control in communication networks: e.g., TCP/IP [Kelly et al., 1998/2001], [Low, Paganini, & Doyle 2002], [Srikant 2012], [Low 2017], ... optimization algorithms as dynamic systems: much early work [Arrow et al., 1958], [Brockett, 1991], [Bloch et al., 1992], [Helmke & Moore, 1994], ... & recent revival [Holding & Lestas, 2014], [Cherukuri et al., 2017], [Lessard et al., 2016], [Wilson et al., 2016], [Wibisono et al, 2016], ... recent system theory approaches inspired by output regulation [Lawrence et al. 2018] & robust control methods [Nelson et al. 2017], [Colombino et al. 2018] 10 / 46

Theory literature inspired by power systems lots of recent theory development stimulated by power systems problems [Simpson-Porco et al., 2013], [Bolognani et al, 2015], [Dall’Anese & Simmonetto, 2016], [Hauswirth et al., 2016], [Gan & Low, 2016], [Tang & Low, 2017], ... 1 A Survey of Distributed Optimization and Control Algorithms for Electric Power Systems Daniel K. Molzahn,⇤ Member, IEEE, Florian D¨ orfler,† Member, IEEE, Henrik Sandberg,‡ Member, IEEE, Steven H. Low,§ Fellow, IEEE, Sambuddha Chakrabarti,¶ Student Member, IEEE, Ross Baldick,¶ Fellow, IEEE, and Javad Lavaei,⇤⇤ Member, IEEE Abstract—Historically, centrally computed algorithms have been the primary means of power system optimization and con- trol. With increasing penetrations of distributed energy resources requiring optimization and control of power systems with many controllable devices, distributed algorithms have been the subject of significant research interest. This paper surveys the literature of distributed algorithms with applications to optimization and control of power systems. In particular, this paper reviews distributed algorithms for offline solution of optimal power flow (OPF) problems as well as online algorithms for real-time solution of OPF, optimal frequency control, optimal voltage control, and optimal wide-area control problems. Index Terms—Distributed optimization, online optimization, electric power systems I. INTRODUCTION CENTRALIZED computation has been the primary way that optimization and control algorithms have been ap- plied to electric power systems. Notably, independent system operators (ISOs) seek a minimum cost generation dispatch for large-scale transmission systems by solving an optimal power flow (OPF) problem. (See [1]–[8] for related litera- ture reviews.) Other control objectives, such as maintaining scheduled power interchanges, are achieved via an Automatic Generation Control (AGC) signal that is sent to the generators that provide regulation services. These optimization and control problems are formulated using network parameters, such as line impedances, system topology, and flow limits; generator parameters, such as cost functions and output limits; and load parameters, such as an estimate of the expected load demands. The ISO collects all the necessary parameters and performs a central computation to solve the corresponding optimization and control problems. With increasing penetrations of distributed energy resources (e.g., rooftop PV generation, battery energy storage, plug-in vehicles with vehicle-to-grid capabilities, controllable loads ⇤: Argonne National Laboratory, Energy Systems Division, Lemont, IL, USA, [email protected]. Support from the U.S. Department of En- ergy, Office of Electricity Delivery and Energy Reliability under contract DE-AC02-06CH11357. †: Swiss Federal Institute of Technology (ETH), Automatic Control Labora- tory, Z¨ urich, Switzerland, [email protected] ‡: KTH Royal Institute of Technology, Department of Automatic Control, providing demand response resources, etc.), the centralized paradigm most prevalent in current power systems will poten- tially be augmented with distributed optimization algorithms. Rather than collecting all problem parameters and performing a central calculation, distributed algorithms are computed by many agents that obtain certain problem parameters via communication with a limited set of neighbors. Depending on the specifics of the distributed algorithm and the application of interest, these agents may represent individual buses or large portions of a power system. Distributed algorithms have several potential advantages over centralized approaches. The computing agents only have to share limited amounts of information with a subset of the other agents. This can improve cybersecurity and reduce the expense of the necessary communication infrastructure. Distributed algorithms also have advantages in robustness with respect to failure of individual agents. Further, with the ability to perform parallel computations, distributed algorithms have the potential to be computationally superior to centralized algorithms, both in terms of solution speed and the maxi- mum problem size that can be addressed. Finally, distributed algorithms also have the potential to respect privacy of data, measurements, cost functions, and constraints, which becomes increasingly important in a distributed generation scenario. This paper surveys the literature of distributed algorithms with applications to power system optimization and control. This paper first considers distributed optimization algorithms for solving OPF problems in offline applications. Many dis- tributed optimization techniques have been developed con- currently with new representations of the physical models describing power flow physics (i.e., the relationship between the complex voltage phasors and the power injections). The characteristics of a power flow model can have a large impact on the theoretical and practical aspects of an optimization formulation. Accordingly, the offline OPF section of this survey is segmented into sections based on the power flow model considered by each distributed optimization algorithm. This paper then focuses on online algorithms applied to OPF, optimal voltage control, and optimal frequency control problems for real-time purposes. Note that algorithms related to those reviewed here have Steven Low Enrique Mallada John Simpson-Porco Changhong Zhao Claudio De Persis Nima Monshizadeh Arjan Van der Schaft Marcello Colombino Emiliano Dall’Anese Sairaj Dhople Andrey Bernstein Krishnamurthy Dvijotham Andrea Simonetto Na Li Sergio Grammatico Yue Chen Florian Dörfler Saverio Bolognani Sandro Zampieri Jorge Cortez Henrik Sandberg Karl Johansson Ioannis Lestas Andre Jokic early adoption: KKT control [Jokic et al, 2009] literature kick-started ∼ 2013 by groups from Caltech, UCSB, UMN, Padova, KTH, & Groningen changing focus: distributed & simple → centralized & complex models/methods implemented in microgrids (NREL, DTU, EPFL, ...) & conceptually also in transactive control pilots (PNNL) 11 / 46

ALGORITHMS & CLOSED-LOOP STABILITY ANALYSIS

Notes on optimization & variational analysis

No content

Stylized optimization problem & algorithm simple optimization problem minimize y,u φ(y, u) subject to y = h(u) u ∈ U cont.-time projected gradient flow ˙ u = Πg U −∇φ h(u), u = Πg U − ∂h ∂u I ∇φ(y, u) y=h(u) Fact: a regular† solution u:[0, ∞]→X converges to critical points if φ has Lip- schitz gradient & compact sublevel sets. projected dynamical system ˙ x ∈ Πg X [f](x) arg min v∈TxX v − f(x) g(x) domain X vector field f metric g tangent cone TX all sufficiently regular† † regularity conditions made precise later 12 / 46

Notes on gradient flows & projections

No content

No content

Metrics: gradient vs. Newton flows −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 Euclidean & variable metric in a non-convex potential gradient & Newton flows in a convex potential 13 / 46

Algorithm in closed-loop with LTI dynamics optimization problem minimize y,u φ(y, u) subject to y = Hio u + Rdo w u ∈ U → open & scaled projected gradient flow ˙ u = ΠU − HT io I ∇φ(y, u) LTI dynamics ˙ x = Ax + Bu + Ew y = Cx + Du + Fw const. disturbance w & steady-state maps x = −A−1B His u −A−1E Rds w y = D − CA−1B Hio u + F − CA−1E Rdo w U u B w E A ∇u φ D F HT io ∇y φ y C + x + + + + + − + + − 14 / 46

Stability, feasibility, & asymptotic optimality Theorem: Assume that regularity of cost function φ: compact sublevel sets & -Lipschitz gradient LTI system asymptotically stable: ∃ τ > 0 , ∃ P 0 : PA + AT P −2τP sufficient time-scale separation (small gain): 0 < < 2τ cond(P ) · 1 Hio Then the closed-loop system is stable and globally converges to the critical points of the optimization problem while remaining feasible at all times. Proof: LaSalle/Lyapunov analysis via singular perturbation [Saberi & Khalil ’84] Ψδ (u, e) = δ · eT P e LTI Lyapunov function + (1 − δ) · φ h(u), u objective function with parameter δ ∈(0, 1) & steady-state error coordinate e=x − His u − Rds w → derivative ˙ Ψδ (u, e) is non-increasing if ≤ and for optimal choice of δ 15 / 46

Derivation of stability result ˙ x = f(x, u) −∇φ(y)∇h(u) u y = x controller: optimization of φ(y) plant with steady state x = h(u) h(u) ˙ x = f(x, u) −∇φ(y)∇h(u) h(u) u + − + x y slow reduced dynamics (slow) fast boundary layer dynamics + x − h(u) 16 / 46

h(u) ˙ x = f(x, u) −∇φ(y)∇h(u) h(u) u + − + x y slow reduced dynamics (slow) fast boundary layer dynamics + x − h(u) 17 / 46

No content

No content

Example: optimal frequency control dynamic LTI power system model power balancing objective control generation set-points unmeasured load disturbances measurements: frequency + constraint variables (injections & flows) linearized swing dynamics 1st-order turbine-governor primary frequency droop DC power flow approximation optimization problem → objective: φ(y, u) = cost(u) economic generation + 1 2 max{0, y − y} 2 Ξ + 1 2 max{0, y − y} 2 Ξ operational limits (line flows, frequency, ...) → constraints: actuation u ∈ U & steady-state map y = Hio u + Rdo w → control ˙ u = ΠU (. . . ∇φ) ≡ super-charged Automatic Generation Control 18 / 46

Test case: contingencies in IEEE 118 system events: generator outage at 100 s & double line tripping at 200 s 0 50 100 150 200 250 300 0 2 4 6 Time [s] Power Generation (Gen 37) [p.u.] Setpoint Output 19 / 46

How conservative is < ? still stable for = 2 −5 0 5 ·10−2 Frequency Deviation from f0 [Hz] System Frequency 0 5 10 15 20 0 1 2 3 Time [s] Line Power Flow Magnitudes [p.u.] 23→26 90→26 flow limit other lines unstable for = 10 −2 0 2 4 Frequency Deviation from f0 [Hz] System Frequency 0 5 10 15 20 0 2 4 Time [s] Line Power Flow Magnitudes [p.u.] 23→26 90→26 flow limit other lines Note: conservativeness problem dependent & depends, e.g., on penalty scalings 20 / 46

Highlights & comparison of approach Weak assumptions on plant internal stability → no observability / controllability → no passivity or primal-dual structure measurements & steady-state I / O map → no knowledge of disturbances → no full state measurement → no dynamic model Weak assumptions on cost Lipschitz gradient + properness → no (strict/strong) convexity required Parsimonious but powerful setup potentially conservative bound, but → minimal assumptions on optimization problem & plant robust & extendable proof → nonlinear dynamics → time-varying disturbances → general algorithms take-away: open online optimization algorithms can be applied in feedback → Hauswirth, Bolognani, Hug & Dörfler (2020) “Timescale Separation in Autonomous Optimization” → Menta, Hauswirth, Bolognani, Hug & Dörfler (2018) “Stability of Dynamic Feedback Optimization with Applications to Power Systems” 21 / 46

Further continuous-time optimization flows

No content

Constraint enforcement −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (a) Penalty Function −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (b) Barrier Function 3 3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (c) Projected Saddle-Flow −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (e) Projected Gradient Flow −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (f) Mixed Saddle-Flow −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (d) Augmented Saddle-Flow 22 / 46

Nonlinear systems & general algorithms general system dynamics ˙ x = f(x, u) with steady-state map x = h(u) incremental Lyapunov function W(x, u) w.r.t error coordinate x − h(u) ˙ W(x, u) ≤ −γ x − h(u) 2 ∇u W(x, u) ≤ ζ x − h(u) variable-metric Q(u) ∈ Sn + gradient flow ˙ u = − Q(u)−1 ∇φ(u) examples: Newton method Q(u)=∇2φ(u) or mirror descent Q(u)=∇2ψ(∇ψ(u)−1) stability condition: ζ γ ·supu Q(u)−1 < 1 Similar results for algorithms with memory: momentum methods (e.g., heavy-ball) (exp. stable) primal-dual saddle flows non-examples: bounded-metric or Lipschitz assumption violated 0 10 20 30 40 50 0 5 10 15 20 Cost Value Dynamic IC Algebraic IC 0 20 40 60 80 100 10-10 10-5 100 105 1010 Cost Value Dynamic IC Algebraic IC cost value algebraic plant dynamic plant algebraic plant dynamic plant discontinuous subgradient Nesterov acceleration 23 / 46

Performance on random problem instances 0 100 200 300 400 10−3 10−2 10−1 100 101 102 Φ(x(t), u(t)) − Φ(x⋆, u⋆) 0 100 200 300 400 10−1 100 101 x(t) − x⋆ 0 5 10 15 20 10−6 10−3 100 103 Φ(x(t), u(t)) − Φ(x⋆, u⋆) 0 5 10 15 20 10−3 10−2 10−1 100 101 x(t) − x⋆ feedback gradient flow with ǫ = 200ǫ⋆ feedback gradient flow with ǫ = ǫ⋆ 0 20 40 60 80 100 100 101 102 Φ(x(t), u(t)) − Φ(x⋆, u⋆) 0 20 40 60 80 100 100 100.5 101 x(t) − x⋆ feedback Newton flow with ǫ = ǫ⋆ in feedback with dynamics dynamics in steady state 24 / 46

Performance on random instances cont’d |x| η ∈ ∂φ(x) 0 5 10 15 20 25 0 5 10 15 20 Φ(x(t), u(t)) − Φ(x⋆, u⋆) 0 5 10 15 20 25 0 2 4 x(t) − x⋆ φ(x) = ˙ x ∈ −∂φ(x) =    −1 if x > 0 +1 if x < 0 −1, +1 if x = 0 in feedback with dynamics dynamics in steady state 0 20 40 60 0 200 400 Φ(x(t), u(t)) − Φ(x⋆, u⋆) 0 20 40 60 0 10 20 30 40 x(t) − x⋆ feedback subgradient flow (no-convergence due non-Lipschitzness) feedback saddle-point flow 25 / 46

Highly nonlinear & dynamic power test case Nordic system: case study known for voltage collapse (South Sweden ’83) (static) voltage collapse: sequence of events → saddle-node bifurcation high-fidelity model of Nordic system RAMSES + Python + MATLAB state: heavily loaded system & large power transfers: north → central load buses with Load Tap Changers generators equipped with Automatic Voltage Regulators, Over Excitation Limiters, & speed governor control g15 g11 g20 g19 g16 g17 g18 g2 g6 g7 g14 g13 g8 g12 g4 g5 g10 g3 g1 g9 4011 4012 1011 1012 1014 1013 1022 1021 2031 cs 4046 4043 4044 4032 4031 4022 4021 4071 4072 4041 1042 1045 1041 4063 4061 1043 1044 4047 4051 4045 4062 400 kV 220 kV 130 kV synchronous condenser CS NORTH CENTRAL EQUIV. SOUTH 4042 2032 41 1 5 3 2 51 47 42 61 62 63 4 43 46 31 32 22 11 13 12 72 71 26 / 46

Voltage collapse event: 250 MW load ramp from t = 500 s to t = 800 s unfortunate control response: non-coordinated + saturation extra demand is balanced by primary frequency control cascade of activation of over-excitation limiters load tap changers increase power demand at load buses bifurcation: voltage collapse very hard to mitigate via conventional controllers → apply feedback optimization to coordinate set-points of Automatic Voltage Controllers 27 / 46

Voltage collapse averted ! distance-to-collapse objective : φ = −log det power flow Jacobian 28 / 46

PROJECTED GRADIENT FLOWS ON MANIFOLDS

Motivation: AC optimal power flow optimization objective: economic dispatch, losses, distance to collapse, ... operational constraints: generation capacity, voltage bands, congestion, ... control: (partial) voltage/current measurements & actuation via set-points of generation, loads, tap-changers, ... AC power flow (steady-state constraint) 2 5 3 4 6 7 8 9 10 11 12 13 nodal voltage current injection power injections line impedance line current power flow Ohm’s Law Current Law AC power AC power flow equations (all variables and parameters are -valued) 29 / 46

Why is power flow optimization challenging? graphical illustration of AC power flow [Hiskens, 2001] imagine constraints slicing this set ⇒ nonlinear, non-convex, disconnected additionally the parameters are ±20% uncertain ...this is only steady state! Ohm’s Law Current Law AC power AC power flow equations (all variables and parameters are -valued) [Molzahn, 2016] 30 / 46

Optimal power flow approaches Properties OPF methods with strict AC network model OPF methods based on convex relaxation OPF methods with linearized network models Computational difficulty NP hard problem Terminate in a polynomial time (for most cases, the rank of computational difficulty is SOCP < QP < SDP) Terminate in a polynomial time Convergence Not guaranteed Guaranteed Guaranteed Solution quality Obtained solution strictly subject to power flow equations and operational limits; obtained solution may be a local optimum When relaxations are inexact, the physical meaning of the obtained solutions is unclear Solutions are not strictly subject to power flow equations, but are close to be AC feasible Industry preference Currently used for the optimization of reactive power, such as in the tertiary voltage control No evidence of industrial applications Currently used in the clearing engine of power markets and power system planning, among others Daniel K. Molzahn and Ian A. Hiskens A Survey of Relaxations and Approximations of the Power Flow Equations Foundations and Trends in Electric Energy Systems, 4(1-2), 2019. 31 / 46

Notes on AC power flow

No content

Key insights on physical equality constraint 1.5 1 0.5 q 2 0 -0.5 -1 1.5 1 0.5 p 2 0 -0.5 -1 1.2 1 1.4 0.8 0.6 v 2 vdc idc m iI v LI CI GI RI τm θ, ω vf v if τe is Lθ M rf rs rs v iT LT C G Gq C v RT iI AC power flow is complex but takes the form of a smooth manifold → local tangent plane approximations, local invertibility, & generic LICQ → regularity (algorithmic flexibility) → Hauswirth, Bolognani, Hug, & Dörfler (2015) “Fast power system analysis via implicit linearization of the power flow manifold” → Bolognani & Dörfler (2018) “Generic Existence of Unique Lagrange Multipliers in AC Optimal Power Flow” AC power flow is attractive steady state for ambient physical dynamics → physics enforce feasibility even for non-exact (e.g., discrete) updates → robustness (algorithm & model) → Gross, Arghir, & Dörfler (2018) “On the steady-state behavior of a nonlinear power system model” 32 / 46

Feedback optimization on the manifold challenging specifications on closed-loop trajectories: 1. stay on manifold at all times 2. satisfy constraints at all times 3. converge to optimal solution feedback optimization algorithm ˙ x = Πg X (−gradφ(x)) physical steady-state power system (AC power flow) Sk + wk = 1 zk ∗ Vk (V∗ k − V∗) renewables loads w generation setpoints measurements prototypical optimal power flow minimize φ(x) subject to x ∈ X = M ∩ K φ : Rn → R objective function M ⊂ Rn AC power flow manifold K ⊂ Rn operational constraints v Tx X X projection of trajectory on feasible cone 33 / 46

Simple low-dimensional case studies ... ...can have simple feasible sets ...or can have really complex sets v0 = 1 slack bus generator qG ∈ [q, q] vref = 1 load pL(t) pG 1j θ0 = 0 0 -2 0.5 3 v 1 2 pG-pL 0 qG 1 0 2 -1 application demands sophisticated level of generality ! 34 / 46

Projected dynamical systems on irregular domains Theorem: Consider a Carathéodory solution x : [0, ∞) → X of the initial value problem ˙ x = Πg X (−gradφ(x)) , x(0) = x0 ∈ X . If φ has compact sublevel sets on X, then x(t) converges to the set of critical points of φ on X. Hidden assumption: existence, uniqueness, & completeness of Carathéodory solution x(t) ∈ X in absence of convexity, Euclidean space, ...? X = x : x 2 2 = 1 , x 1 ≤ √ 2 regularity conditions constraint set vector field metric manifold existence of Krasovski loc. compact loc. bounded bounded C1 existence of Carathéodory Clarke regular C0 C0 C1 uniqueness of solutions prox regular C0,1 C0,1 C1,1 → Hauswirth, Bolognani, & Dörfler (2018) “Projected Dynamical Systems on Irregular Non-Euclidean Domains for Nonlinear Optimization” → Hauswirth, Bolognani, Hug, & Dörfler (2016) “Projected gradient descent on Riemanniann manifolds with applications to online power system optimization” 35 / 46

Notes on regularity conditions

No content

ROBUST IMPLEMENTATION ASPECTS

Robust implementation of projections projection & integrator → windup → robust anti-windup approximation → saturation often “for free” by physics ˙ u = ΠU [k(x, ·)](u) K → ∞ K PU k(·, ·) ˙ x = f(x, ·) + − u PU (u) − + disturbance → time-varying domain f(x) Πt X f(x) X(t) X(t + δ) temporal tangent cone & vector field ensure suff. regularity & tracking certificates → Hauswirth, Dörfler, & Teel (2020) “Anti-Windup Approximations of Oblique Projected Dynamical Systems for Feedback-based Optimization” handling uncertainty when enforcing non-input constraints : x ∈ X or y ∈ Y cannot measure state x directly → Kalman filtering: estimation & separation cannot enforce constraints on y =h(u) by projection (not actuated & h(·) unknown) → soft penalty or dualization + grad flows (inaccurate, violations, & strong assumptions) → project on 1st order prediction of y =h(u) y+ ≈ h(u) measured + ∂h ∂u steady-state I/O sensitivity w feasible descent direction ⇒ global convergence to critical points → Häberle, Hauswirth, Ortmann, Bolognani, & Dörfler (2020) “Enforcing Output Constraints in Feedback-based Optimization” → Hauswirth, Subotić, Bolognani, Hug, & Dörfler (2018) “Time-varying Projected Dynamical Systems with Applications...” 36 / 46

Example: transmission grid management 5 km SOMME Amiens Aumale Abbeville Saint-Valéry- sur-Somme Ailly- le-Haut- Clocher Ville-le-Marclet Airaines Bourbel Envermeu Amargue Villers- Bocage Brailly-Cornehotte Frévent Rue Saucourt Mers-Le Tréport Penly Centrale nucléaire de Penly Croixrault Fouilloy Picquigny Neufchâtel SEINE- MARITIME OISE Poste d’Argœuves Poste de Beauchamps Poste de Limeux Vers Gravelines Poste de Blocaux Source : RTE, Chi res fin février 2017 Manche 400 000 volts RTE Grid 225 000 volts 90 000 volts Production mini. 0 MW Conso mini. 142 MW Conso max. 505 MW Production max. 693 MW SURPLUS DÉFICIT Production 782 MW 2017 1 010 MW 2019 rapid changes in generation/demand → violation of line & voltage limits → resolve most economically & under severe uncertainty & disturbances 37 / 46

Real-time operation approaches 1. traditional hierarchical & time-scale separated feedback / feedforward optimization stage SC-OPF, market real-time operation automated/manual services/re-dispatch low-level automatic controllers droop, AGC power system disturbance u x generation setpoints state estimation prediction (load, generation, downtimes) schedule 2. repeated feedforward optimal power flow based on model & prediction optimization stage SC-OPF, market real-time operation automated/manual services/re-dispatch low-level automatic controllers droop, AGC power system disturbance u x generation setpoints state estimation prediction (load, generation, downtimes) schedule 38 / 46

Real-time operation approaches cont’d 3. online feedback optimization using all of today’s concepts Feedback optimization Physical plant h(u; w) u ∈ U y ∈ Y w φ(u, y) minimize nonlinear steady-state map exogenous input Power grid power demand uncontrollable generators power flow equations generator set-points line and bus measurements operation cost 39 / 46

Tracking performance under disturbances G 1 G 2 C 1 C 3 C 2 W S Generator Synchronous Condensor Solar Wind G C S W primary sources. This results in a time-varying dditional operational constraints that need to be branches. The total generation cost we aim to ator in [$/h], given as aip2 i +bipi , where ai, bi > 0 5.3. The marginal operating cost of the solar and Algorithm 1, where the controller receives field ute. The demand profile is shown in Figure 5.4, approximately 20% between 20:30 and 21:30 at 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 70 80 90 100 110 120 30 bus power flow test case. net demand: load, wind, & solar (discontinuous) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 20 40 60 80 100 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 -50 0 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0.95 1 1.05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 -50 0 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0.95 1 1.05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0.1 0.2 0.3 40 / 46

Optimality despite disturbances & uncertainty transient trajectory feasibility practically exact tracking of ground-truth optimizer (omniscient & no computation delay) robustness to model mismatch (asymptotic optimality under wrong model) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 100 200 300 (a) Simulation results of controlled 30 bus power system with exact Jacobian matrix ru,y F(u, y, w). Figure 5.5: Simulation results of controlled ru,yF(u, y, w) and a constant approximatio and the colors are the same as in Table 5.3. offline optimization feedback optimization model uncertainty feasible ? φ − φ∗ v − v∗ feasible ? φ − φ∗ v − v∗ loads ±40% no 94.6 0.03 yes 0.0 0.0 line params ±20% yes 0.19 0.01 yes 0.01 0.003 2 line failures no -0.12 0.06 yes 0.19 0.007 conclusion: simple algorithm performs extremely well & robust 41 / 46

EXPERIMENTS

Experimental case study @ DTU EVSE 1 EVSE 2 EVSE 3 EVSE 4 EVSE 5 EVSE 6 EVSE 7 EVSE 8 Busbar A Busbar B Busbar A Busbar B 630 kVA 100 kVA Battery Ext. 117-5 Cable C2 Cable C1 CEE Ext. 117-2 Chg. post Cable D1 PV NEVIC Busbar A Busbar B Busbar C 150 kVA 100 kVA Gaia Flexhouse PV Cable B1 Cable B2 Busbar B Busbar B Busbar A 200 kVA Static load Diesel CEE Aircon Cable A1 Cable A2 PV Busbar B Busbar A Busbar B Cable F1 Flexhouse 2 Flexhouse 3 Cable E1 Cable E2 CEE CHP Heatpump 1 Booster Heater Cable F1 Crossbar switch Load conv. SYSLAB breaker overview Building 716 Building 715 Building 319 Building 117 Ship Shore Mach. set Container 1 Container 2 Container 3 I I I PCC v1 v2 v3 R1 , L1 R2 , L2 R3 , L3 p1 , q1 p2 , q2 p3 , q3 PV1 PV2 Battery ±8 kVAr Static load ±6 kVAr ±6 kVAr 0 kVAr 10 kW 0 kVAr 0 kW −15 kW Voltage [p.u.] 1 0.99 1.06 1.05 0.95 21 min experiment with events t = 3 min: control turned ON t ∈ [11, 14] min: Pbatt = 0 kW base-line controllers decentralized nonlinear proportional droop control (IEEE 1547.2018) vi qi qmax i qmin i vmin vmax qi (t + 1) = fi (vi (t)) 1 comparison of three controllers decentralized control feedforward optimization feedback optimization → Ortmann, Hauswirth, Caduff, Dörfler, & Bolognani (2020) “Experimental Validation of Feedback Optimization in Power Distribution Grids” 42 / 46

Decentralized feedback control decentralized nonlinear proportional droop control 0.97 1 1.03 1.05 1.07 −5 0 5 0 5 10 15 20 0.97 0.98 0.99 1 Time [min] 0 5 10 15 20 0 1 2 0.97 0.98 0.99 1 Voltage [p.u.] 0 1 2 Reactive Power [kVAr] Battery PV2 PV1 constraint violations due to local control saturation & lack of coordination 43 / 46

Successive feedforward optimization centralized, omniscient, & successively updated at high sampling rate 0.97 1 1.03 1.05 1.07 −5 0 5 0 5 10 15 20 0.97 0.98 0.99 1 Time [min] 0 5 10 15 20 −2 −1 0 0.97 0.98 0.99 1 Voltage [p.u.] −2 −1 0 Reactive Power [kVAr] Battery PV2 PV1 performs well but persistent constraint violation due to model uncertainty 44 / 46

Feedback optimization primal-dual flow with 10 s sampling time requiring only model I/O sensitivity ∇h (or an estimate) 0.97 1 1.03 1.05 1.07 −5 0 5 0 5 10 15 20 0.97 0.98 0.99 1 Time [min] 0 5 10 15 20 −6 −4 −2 0 0.97 0.98 0.99 1 Voltage [p.u.] −4 −2 0 Reactive Power [kVAr] Battery PV2 PV1 excellent performance & model-free(!) since ∇h(u) approximated by 1 1 1 1 1 1 1 1 1 45 / 46

CONCLUSIONS

Conclusions Summary open & online feedback optimization algorithms as controllers approach: projected dynamical systems & time-scale separation unified framework: broad class of systems, algorithms, & programs illustrated throughout with non-trivial power systems case studies Ongoing work & open directions analysis: robustness, performance, stochasticity, sampled-data algorithms: 0th-order, sensitivity estimation, distributed, game theory power systems: more experiments, virtual power plant extensions further app’s: seeking equilibria in uncertain & constrained systems It works in theory and in practice ! 46 / 46

Thanks ! Florian Dörfler http://control.ee.ethz.ch/~floriand [link] to related publications