Online Feedback Optimization Florian Dörfler, ETH Zürich DGIST 2024

feedforward planning optimization system system model ˆ w estimate u w y complex specifications & decision optimal, constrained, & multivariable strong requirements precise model, full state, disturbance estimate, & computationally intensive vs. feedback control controller system r u y w − simple feedback policies suboptimal, unconstrained, & SISO forgiving nature of feedback measurement driven, robust to model uncertainty, fast & agile response → typically complementary methods are combined via time-scale separation optimization controller system system model r u y − ˆ w estimate w offline & feedforward real-time & feedback 1 / 27

Price of time-scale separation in power balancing offline optimization: dispatch based on forecasts of loads & renewables 0 50 100 150 200 0 10 20 30 40 50 60 70 80 90 100 marginal costs in €/MWh Capacity in GW Renewables Nuclear energy Lignite Hard coal Natural gas Fuel oil online control based on frequency Frequency Control Power System 50Hz + u y frequency measurement − re-schedule set-point to mitigate severe forecasting errors (redispatch, reserve, etc.) more uncertainty & fluctuations → infeasible & inefficient to separate optimization & control 50 Hz 51 49 generation load control [Milano, 2018] Re-scheduling costs Germany [mio. €] !!" #$% !&' !&( %%(# %%"& %!() %!*! !"## !"#! !"#$ !"#% !"#& !"#' !"#( !"#) [Bundesnetzagentur, Monitoringbericht 2012-2019] 2 / 27

Synopsis & proposal for control architecture power grid: separate decision layers hit limits under increasing uncertainty similar observations in other large-scale & uncertain control systems : process control systems & queuing / routing / infrastructure networks proposal: open with inputs & outputs and online running & non-batch optimization algorithm as feedback real-time interconnected control optimization algorithm e.g., u+ = u−∇φ(y, u) dynamical system ˙ x = f(x, u, w) y = h(x, u, w) actuation u measurement y operational constraints u ∈ U disturbance w 3 / 27

Context: historical roots & related work process control: reducing the effect of uncertainty in sucessive optimization Optimizing Control [Garcia & Morari, ’81 & ’84], Self-Optimizing Control [Skogestad, ’00], Modifier Adaptation [Marchetti et. al, ’09], Real-Time Optimization [Bonvin ed. ’17, Krishnamoorthy et al. ’22], ... optimal routing, queuing, & congestion control in communication networks (e.g. TCP/IP) [Kelly et al., ’98], [Low, Paganini, & Doyle ’02], [Srikant ’12], ... & in power systems [Jokic et al ’09], [Bolognani & Zampieri ’13], [Dall’Anese & Simonetto ’16], [Hauswirth et al, ’16], ... extremum-seeking: derivative-free & suited for unconstrained low-dim. problems [Leblanc, 1922], ...[Wittenmark & Urquhart, 1995], ...[Krstić & Wang, 2000], ..., [Feiling et al., 2018] real-time MPC with anytime guarantees for dynamic (optimal control) problems: [Diel et al. 2005], [Zeilinger et al. 2009], [Feller & Ebenbauer 2017], ...[Liao-McPherson et al. ’20] policy gradient RL: optimal control solved by model-free gradient interactions with plant: [Kadake ’01], [Peters & Schaal, ’07], [Duan et al. ’16], [Fazel et al. ’19], ...[Hu et al. ’23] recent system theory involving regulation, robust, hybrid control, etc.: [Lawrence et al. 2018], [Colombino et al. 2018], [Simpson-Porco ’20], [Hauswirth et al, ’20], [Bianchin, Poveda, ’22], ... 4 / 27

Outline of today theory: optimization algorithms in closed loop tutorial: stylized warm-up example & academic analysis practical, robust, & performant extensions new: model-free & data-driven implementations power systems case studies: sims → industry deployment 5 / 27

Acknowledgements Adrian Hauswirth Saverio Bolognani Lukas Ortmann Zhiyu He Dominic Liao McPherson Giuseppe Belgioioso Miguel Picallo Verena Häberle Giulia de Pasquale URRICULUM VITAE me: Irina Subotić th: 22.03.1993, Belgrade, Serbia mail: [email protected] UCATION p 2016- sent Swiss Federal Institute of Technology in Zurich • Master of Science in Robotics, Systems and Control Irina Subotić 6 / 27

STYLIZED WARM-UP EXAMPLE & ACADEMIC ANALYSIS

Stylized optimization problem & algorithm simple optimization problem minimize y,u φ(y, u) subject to y = h(u) u ∈ U cont.-time projected gradient flow ˙ u = ΠU −∇φ h(u), u = ΠU − ∂h ∂u I · ∇φ(y, u) y=h(u) Fact: a regular† solution u:[0, ∞]→U converges to critical points if φ has Lip- schitz gradient & compact sublevel sets. projected dynamical system ˙ u ∈ ΠU −∇φ · arg min v∈TuU v + ∇φ · U Tu U domain U gradient field ∇φ(·) metric · tangent cone TuU all sufficiently regular† † for details → Hauswirth et al. (2021) “Projected Dynamical Systems on Irregular Non-Euclidean Domains for Nonlinear Optimization” 7 / 27

Algorithm in closed loop with LTI dynamics optimization problem minimize y,u φ(y, u) subject to y = Hiou + Hdow u ∈ U → scaled & open projected gradient flow ˙ u = ΠU − HT io I · ∇φ(y, u) requiring only steady-state sensitivity Hio LTI dynamics ˙ x = Ax + Bu + Ew y = Cx + Du + Fw const. disturbance w & steady-state maps x = −A−1B His u −A−1E Hds w y = D − CA−1B Hio u + F − CA−1E Hdo w U u B w E A ∇u φ D F HT io ∇y φ y C + x + + + + + − + + − 8 / 27

Stability, feasibility, & asymptotic optimality Theorem [Hauswirth et al. ’20]: Assume that regularity of cost function φ: compact sublevel sets & -Lipschitz gradient LTI system exp. stable with rate τ > 0: ∃ P 0 s.t. PA + AT P −2τP sufficient time-scale separation (small gain): 0 < < 2τ cond(P ) · 1 Hio ⇐⇒ system gain · algorithm gain < 1 Then the closed-loop system is stable & globally converges to the critical points of the optimization problem while remaining feasible at all times. Proof: LaSalle/Lyapunov analysis via singular perturbation [Saberi & Khalil ’84] Vδ(u, e) = δ · eT P e LTI Lyapunov function + (1 − δ) · φ h(u), u algorithm merit function with parameter δ ∈(0, 1) & steady-state error coordinate e=x−Hisu−Hdsw → derivative ˙ Vδ(u, e) is non-increasing if ≤ & for judicious choice of δ 9 / 27

Example: optimal power system balancing dynamic power system model control generation set-points unmeasured load disturbances measurements: frequency + injections + line flows linearized swing dynamics 1st-order turbine-governor DC power flow approximation optimization problem → objective: φ(y, u) = cost(u) economic generation + 1 2 max{0, y − y} 2 Ξ + 1 2 max{0, y − y} 2 Ξ soft penalty on operational limits (e.g., flows) → hard constraints: actuation u ∈ U enforced by projection & steady-state map y = Hiou + Hdow enforced by physics → control ˙ u = ΠU − HT io I · ∇φ(y, u) via open projected gradient flow 10 / 27

Test case: contingencies in IEEE 118 system events: generator outage at 100 s & double line tripping at 200 s 0 50 100 150 200 250 300 0 2 4 6 Time [s] Power Generation (Gen 37) [p.u.] Setpoint Output 11 / 27

How conservative is < ? still stable for = 2 −5 0 5 ·10−2 Frequency Deviation from f0 [Hz] System Frequency 0 5 10 15 20 0 1 2 3 Time [s] Line Power Flow Magnitudes [p.u.] 23→26 90→26 flow limit other lines unstable for = 10 −2 0 2 4 Frequency Deviation from f0 [Hz] System Frequency 0 5 10 15 20 0 2 4 Time [s] Line Power Flow Magnitudes [p.u.] 23→26 90→26 flow limit other lines 12 / 27

Highlights of online feedback optimization Weak assumptions on plant internal stability → no further structure required measurements & steady-state sensitivity → no need for model, state, or disturbances Weak assumptions on optimization Lipschitz gradient + properness → no (strict / strong) convexity required Parsimonious but powerful setup minimal assumptions + potentially conservative time-scale separation strong conclusions: stability, feasibility (safety), & convergence to optimality robust & extendable methodology → nonlinear & sampled-data dynamics → general equilibrium seeking algorithms → time-varying disturbances, noise, ... gradient flow ˙ u= − ∇φ(y, u) LTI system ˙ x = Ax + Bu + Ew y = Cx + Du + Fw actuation u measurement y projection u ∈ U disturbance w take-aways: open online optimization algorithms can be applied in feedback & methodology is robust + extendable 13 / 27

GENERAL NONLINEAR DYNAMICS & DISTURBANCES

General nonlinear systems & disturbances nonlinear system ˙ x = f(x, u) with steady-state map x = h(u) so that f(h(u), u) = 0 open-loop incrementally stable w.r.t steady state: ∃ Lyapunov function V (x, u) s.t. ˙ V (x, u) ≤ −γ x − h(u) 2 dissipation rate γ w.r.t. steady state & ∇uV (x, u) ≤ ζ x − h(u) ζ-Lipschitz in steady-state error ⇒ local / global closed-loop stability, convergence to critical points, & feasibility if system gain · algorithm gain < 1 where the system gain is ζ/γ = Lipschitz constant / dissipation rate time-varying disturbances: ˙ x = f(x, u, w(t)) assume input-to-state stable (ISS) dynamics w.r.t. bounded disturbance variations ˙ w(t) ⇒ tracking certificate: closed-loop ISS w.r.t. ˙ w(t) t x(t) ∼ ˙ w(t) ∞ 14 / 27

Tracking performance under disturbances ! " ! # $ " $ % $ # & ' !"#"$%&'$ ()#*+$'#',-./'#0"#-'$ ('1%$ 23#0 ! $ ' & primary sources. This results in a time-varying dditional operational constraints that need to be branches. The total generation cost we aim to ator in [$/h], given as aip2 i +bipi , where ai, bi > 0 5.3. The marginal operating cost of the solar and Algorithm 1, where the controller receives field ute. The demand profile is shown in Figure 5.4, approximately 20% between 20:30 and 21:30 at 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 70 80 90 100 110 120 30 bus power flow test case. disturbance = net demand = load - wind + solar 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 20 40 60 80 100 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 -50 0 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0.95 1 1.05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 -50 0 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0.95 1 1.05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0.1 0.2 0.3 15 / 27

Optimality despite disturbances & uncertainty transient trajectory feasibility practically exact tracking of ideal optimal power flow (OPF) (omniscient & no computation delay) robustness to model mismatch (asymptotic optimality under wrong model) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 100 200 300 (a) Simulation results of controlled 30 bus power system with exact Jacobian matrix ru,y F(u, y, w). Figure 5.5: Simulation results of controlled ru,yF(u, y, w) and a constant approximatio and the colors are the same as in Table 5.3. offline optimization feedback optimization model uncertainty feasible ? φ − φ∗ v − v∗ feasible ? φ − φ∗ v − v∗ loads ±40% no 94.6 0.03 yes 0.0 0.0 line params ±20% yes 0.19 0.01 yes 0.01 0.003 2 line failures no -0.12 0.06 yes 0.19 0.007 conclusion: simple algorithm performs extremely well in challenging environment 16 / 27

GENERAL OPTIMIZATION FLOWS & DISCRETE-TIME ALGORITHMS

More general optimization flows variable metrics different ways of enforcing constraints gradient: ˙ u = − ∇φ(u) Newton: ˙ u = − ∇2φ(u)−1 · ∇φ(u) −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (e) Projected Gradient Flow −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (f) Mixed Saddle-Flow −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (a) Penalty Function −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 (b) Barrier Function 3 3 penalty function barrier function projected gradient flow primal-dual saddle flow 17 / 27

Certificates for general optimization flows variable-metric gradient flow with Q(u) 0 ˙ u = − Q(u) · ∇φ(u) example: Newton method Q(u)=(∇2φ(u))−1 or mirror descent Q(u)=(∇2ψ(∇ψ(u)−1))−1 stability, convergence, & feasibility if system gain · algorithm gain < 1 with algorithm gain · ∇h(u) · supu Q(u) Similar results for algorithms with memory: momentum methods (e.g., heavy-ball) ¨ u + D(u) · ˙ u = −Q(u) · ∇φ(u) (exp. stable) primal-dual saddle flows as long as the algorithm gain is bounded a few non-examples for unbounded gain: 0 10 20 30 40 50 0 5 10 15 20 Cost Value Dynamic IC Algebraic IC 0 20 40 60 80 100 10-10 10-5 100 105 1010 Cost Value Dynamic IC Algebraic IC cost value algebraic plant dynamic plant algebraic plant dynamic plant discontinuous subgradient Nesterov acceleration 18 / 27

Practical & robust implementation aspects projection & integrator → windup → robust anti-windup approximation → saturation often “for free” by physics K PU k(·, u) ˙ x = f(x, ·) + − u PU (u) − + ˙ u = ΠU [k(x, ·)](u) K → ∞ disturbance → time-varying domain f(x) Πt X f(x) X(t) X(t + δ) temporal tangent cone & vector field ensure suff. regularity & tracking certificates → Hauswirth, Dörfler, & Teel (2020) “Anti-Windup Approximations of Oblique Projected Dynamical Systems for Feedback-based Optimization” handling uncertainty when enforcing non-input constraints : x ∈ X or y ∈ Y cannot measure state x directly → Kalman filtering: estimation & separation cannot enforce constraints on y =h(u) by projection (not actuated & h(·) unknown) → soft penalty or dualization + grad flows (inaccurate, violations, & strong assumptions) → project on 1st order prediction of y =h(u) y+ ≈ h(u) measured + ∂h ∂u steady-state I/O sensitivity w feasible descent direction ⇒ global convergence to critical points → Häberle, Hauswirth, Ortmann, Bolognani, & Dörfler (2020) “Enforcing Output Constraints in Feedback-based Optimization” → Hauswirth, Subotić, Bolognani, Hug, & Dörfler (2018) “Time-varying Projected Dynamical Systems with Applications...” 19 / 27

Sampled-data setting continuous-time plant: same assumptions as before sampling rate τ & 0th order hold discrete-time algorithm with strictly decreasing merit function & bounded gain examples: strongly quasi-non- expansive operator (ADMM, DR, prox, alternating projection, ...) ⇒ local / global closed-loop ISS if system gain · algorithm gain < 1 ⇒ system gain decreases in τ i.e. sufficiently slow sampling sample hold discrete-time algorithm continuous-time plant z+ = T(z, y) u = q(z) ˙ x = f(x, u, w) y = g(x, w) τ sampling period 0th order hold w . ILLUSTRATIVE EXAMPLES , we demonstrate that the algorithmic pre- mption 3) in Theorem 5 are sharp in the y is not satisfied, in general, one cannot thm-plant interconnection (21) to be robust sturbances or even stable7. gle-input single-output dynamic plant gov- ond-order differential equation 20 / 27

Example: building temperature control Fig. 7. Example building generated via the BRCM toolbox [45]. VII. APPLICATION EXAMPLES A. Temperature Regulation in Smart Buildings In this section, we illustrate how FES can be applied to smart building automation. Consider the 5-room single-story office building in Figure 7. Its dynamics are of the form ˙ x = Ax + Bu u + Bw w + nu X i=1 (Bwu,i w + Bxu,i x) ui , (44) and are generated using the BRCM toolbox [45]. The state x 2 L113 contains the temperatures of the rooms and wall layers, floor layers, etc. The control inputs u 2 L8 are an air handling unit (AHU) consisting of air flow (0–1 kg/s), Fig. 8. Simulations of the SQP controller (46) on the building dynam The comfort temperature (output) constraints are (approximately) throughout the simulations, while heating and cooling effort is minim is to penalize the comfort constraint violations of the temperatures. A 1-norm penalty on the violation is us two reasons. First, the electrical cost of heating is lin the control input. Second, ' is an exact penalty functio so for a well-tuned parameter ⌘, the (disturbance-free) building model from BRCM toolbox with 118 states (bilinear dynamics), 10 disturbances, 8 inputs, & 7 outputs objective: minimize energy cost & keep temperatures in comfort range online SQP (sequential quadratic programming) for feedback optimization note: algorithm is not predictive & doesn’t use any forecast or reference 7. Example building generated via the BRCM toolbox [45]. VII. APPLICATION EXAMPLES Temperature Regulation in Smart Buildings this section, we illustrate how FES can be applied to rt building automation. Consider the 5-room single-story e building in Figure 7. Its dynamics are of the form = Ax + Bu u + Bw w + nu X i=1 (Bwu,i w + Bxu,i x) ui , (44) are generated using the BRCM toolbox [45]. The state 113 Fig. 8. Simulations of the SQP controller (46) on the building dynamics (44 The comfort temperature (output) constraints are (approximately) satisfi throughout the simulations, while heating and cooling effort is minimized. is to penalize the comfort constraint violations of the room temperatures. A 1-norm penalty on the violation is used fo two reasons. First, the electrical cost of heating is linear i comparison to hysteresis (threshold- based) control: 32% cost reduction & 28% reduction in constraint violations 21 / 27

Feedback equilibrium seeking for games motivation: multi-area power system different system operators whose cost functions are not aligned physical & operational coupling game theory as lingua franca: minuiUi φi(yi, ui) subject to constraints coupling (ui, yi) opt. solution = Nash equilibrium equilibrium-seeking algorithms using local gradients ∂ ∂ui φi(yi, ui) similar assumptions as before & system gain · algorithm gain < 1 ⇒ all results extend analogously ! local feedback optimization local feedback optimization local feedback optimization local feedback optimization local feedback optimization centralized optimality seeking multi-area equilibrium seeking price of anarchy → Belgioioso et al. (2022) “Online Feedback Equilibrium Seeking” 22 / 27

ALL ALGORITHMS REQUIRE THE GRADIENT ∂ ∂u φ h(u), u = ∂h ∂u I · ∇φ(y, u) y=h(u) & THUS THE MODEL SENSITIVITY ∂h ∂u ! MODEL-FREE IMPLEMENTATIONS WITHOUT SENSITIVITY ?

Example: power grid operation !""#$% &&'#$% ##("#$% Amiens Aumale Abbeville SOMME RTE Grid wind power [%] 90% 5 minutes UNICORN project with RTE automation of Blocaux zone rapid change in generation → line / voltage limits violations → resolve most economically & under severe uncertainty & time-varying disturbances Technical problem setup simulation of entire French grid → power flow + tap changer actuation & sensing in Blocaux → tap, reactive & active power → voltage & current magnitudes realistic constraints & cost → curtailment + losses 23 / 27

Current mode of operation 0 200 400 600 800 1000 1200 1400 1600 Time [sec] 0 10 20 30 40 50 60 70 80 90 100 Wind Power [%] Wind Power [%] Available Used offline optimization & curtailment at 70% to not violate line / voltage limits 24 / 27

Feedback optimization using constant (wrong) sensitivity Power Curtailment [MW] Line Usage [p.u.] 25 / 27

Model-free feedback optimization feedback optimization is robust to inaccurate sensitivity, though the performance might be inferior online sensitivity estimation of ∂h ∂u ≈ yt+1−yt ut+1−ut via Kalman filter 0th-order optimization building one-point gradient estimates ∂ ∂u φ(u) = lim δ 0 E η δ φ(u + δ · η) where η is random probing direction & δ is (small) smoothing parameter → constructed via single actuation stochastic extremum seeking [Liu & Krstic ’16] performs less favorable ground truth: exact sensitivity online sensitivity estimation stochastic extremum seeking 0th-order optimization method robust to inaccurate sensitivity → Colombino et al. (2020) “Towards robustness guarantees for feedback-based optimization” → Picallo et al. (2022) “Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation” → He et al. (2022) “Model-Free Nonlinear Feedback Optimization” 26 / 27

IN VIVO

Commercial deployment at Swiss utility → Ortmann et al. (2022) “Deployment of an Online ...” virtual grid reinforcement through reactive power/voltage support & power flow control strong economic incentives (rewards & penalties) from higher-level system operator feedback optimization on legacy hardware runs robustly, 24 / 7, & makes money in presence of time-varying incentives 11:56 11:57 11:58 11:59 12:00 12:01 12:02 12:03 12:04 12:05 12:06 12:07 12:08 12:09 12:10 12:11 12:13 12:14 12:15 0.8 cap. 0.85 cap. 0.9 cap. 0.95 cap. 1 0.95 ind. 0.9 ind. 0.85 ind. 0.8 ind. Time Setpoint 11:56 11:57 11:58 11:59 12:00 12:01 12:02 12:03 12:04 12:05 12:06 12:07 12:08 12:09 12:10 12:11 12:13 12:14 12:15 200 150 100 50 0 50 100 150 200 kW or kVAr Setpoint Reactive Power Active Power [7] S d I 0 [8] L p C [9] E T [10] A o o [11] J I e h C [12] H 27 / 27

CONCLUSIONS

Conclusions Summary open & online feedback optimization algorithms as controllers unified framework for broad class of systems, algorithms, decision- making problems, interconnection scenarios, & implementation aspects illustrated throughout with non-trivial power systems case studies complete TRL scale covered: theory −→ industrial deployment Ongoing work & open directions theory: time-scale separation, model-free, & performative prediction new application domains: supply chains & recommender systems It works in theory and in practice ! 28 / 27

Main resources for today https://sites.google.com/view/eeci-autonomous-power-systems 2021 SmartGridComm Tutorial here Publications about 'Online Optimization' Articles in journal, book chapters 1. V. Häberle, A. Hauswirth, L. Ortmann, S. Bolognani, and F. Dörfler. Non-convex Feedback Optimization with Input and Output Constraints. IEEE Control Systems Letters, 5(1):343-348, 2021. Keyword(s): Online Optimization, Nonlinear Optimization, Nonlinear Control Design. [bibtex-entry] 2. A. Hauswirth, S. Bolognani, and F. Dörfler. Projected Dynamical Systems on Irregular Non-Euclidean Domains for Nonlinear Optimization. SIAM Journal on Control and Optimization, 59(1):635-668, 2021. Keyword(s): Online Optimization, Nonlinear Optimization. [bibtex-entry] 3. A. Hauswirth, S. Bolognani, G Hug, and F. Dörfler. Optimization Algorithms as Robust Feedback Controllers. January 2021. Note: Submitted. Available at http://arxiv.org/abs/2103.11329. Keyword(s): Power Networks, Power Flow Optimization, Online Optimization, Nonlinear Optimization. [bibtex-entry] Publications http://people.ee.ethz.ch/~floriand/ Optimization Algorithms as Robust Feedback Controllers Adrian Hauswirth, Saverio Bolognani, Gabriela Hug, and Florian Dörfler Department of Information Technology and Electrical Engineering, ETH Zürich, Switzerland Abstract Mathematical optimization is one of the cornerstones of modern engineering research and practice. Yet, throughout all application domains, mathematical optimization is, for the most part, considered to be a numerical discipline. Opti- mization problems are formulated to be solved numerically with specific algorithms running on microprocessors. An emerging alternative is to view optimization algorithms as dynamical systems. While this new perspective is insightful in itself, liberating optimization methods from specific numerical and algorithmic aspects opens up new possibilities to endow complex real-world systems with sophisticated self-optimizing behavior. Towards this goal, it is necessary to un- derstand how numerical optimization algorithms can be converted into feedback controllers to enable robust “closed-loop optimization”. In this article, we review several research streams that have been pursued in this direction, including extremum seeking and pertinent methods from model predictive and process control. However, our primary focus lies on recent methods under the name of “feedback-based optimization”. This research stream studies control designs that directly implement optimization algorithms in closed loop with physical systems. Such ideas are finding widespread application in the design and retrofit of control protocols for communication networks and electricity grids. In addition to an overview over continuous-time dynamical systems for optimization, our particular emphasis in this survey lies on closed-loop stability as well as the enforcement of physical and operational constraints in closed-loop implementations. We further illustrate these methods in the context of classical problems, namely congestion control in communication networks and optimal frequency control in electricity grids, and we highlight one potential future application in the form of autonomous reserve dispatch in power systems. arXiv:2103.11329v1 [math.OC] 21 Mar 2021 2021 Survey paper https://arxiv.org/abs/2103.11329