A new approach to optimal control with constraints Wuchen Li Georgia Institute of Technology AIMS 2016 Joint work with Shui-Nee Chow (Math), Magnus Egerstedt (ECE), Jun Lu (Wells Fargo), and Haomin Zhou (Math) Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 1 / 25 

Outline 1 Problems Optimal control with constraints Current Numerical methods 2 Method of Evolving Junctions 3 Numerical experiments Path-planning problem Differential games 4 Summary Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 2 / 25 

Introduction: Optimal control with constraints Optimal Control : to find controls (inputs) to a dynamical system with constraints in order to “optimize” a given performance functional. Goals: In this talk, we present a new fast algorithm for this problem. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 3 / 25 

An example: Shortest path problem To “control” a drone from a starting point to an ending point without touching obstacle(s) in R3. (a) Drone (b) Shortest path Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 4 / 25 

Introduction: Problem formulation min x,u T 0 L(x(t), u(t), t)dt + ψ(x(T), T), subject to ˙ x = f (x(t), u(t)), 0 ≤ t ≤ T, x(0) = x0, x(T) = xT , with constraints φ(x(t), u(t), t) ≥ 0, 0 ≤ t ≤ T. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 5 / 25 

Introduction: Difficulties This problem is an infinite dimensional optimization problem, the variables x(t), u(t) are in different Banach spaces. Moreover, the constraints (depending on time) induce more difficulties. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 6 / 25 

Introduction: Current Numerical methods State-space method P: The solution of Hamilton-Jacobi-Bellman equation is a global minimizer; C: Solving a nonlinear PDE. Indirect method P: Solving boundary value problems for ODEs (shooting method) ; C: Only local minimizers; Constraints are hard to be handled. Direct method P: Powerful nonlinear programming techniques; C: Large dimensional optimization; Sacrifice of accuracy. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 7 / 25 

Motivation New method: Method of Evolving junctions (MEJ). All local and global minimizers have the same structure−“Separable”; By leveraging such structure, we reduce the optimal control to a set of finite, but different, dimensional optimization problems; Such reduction helps us to find the global minimizer(s) by using stochastic differential equations: Monte-Carlo simulation; Finite dimensional optimization in different spaces. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 8 / 25 

Motivation: Separable structure Definition A path x(t) is said to be separable if there exists a finite partition: t0 < t1 < t2 < · · · < tN < tN+1 = tf such that x(t)|[ti ,ti+1] alternates between segments where constraints are either active or inactive. We call the partition points ˜ xi = (ti , x(ti )) as junctions. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 9 / 25 

New method: Functional If a trajectory is separable, then it is determined by junction points {˜ x0, · · · , ˜ xN+1} and the cost functional can be expressed as a function of junctions: J(˜ x0, ˜ x1, · · · , ˜ xN+1) := i Ji (˜ xi−1, ˜ xi ), where J(˜ xi , ˜ xi+1) represents the optimal cost connecting two junctions: it is either in free space or on constraints. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 10 / 25 

New method: Constraints Need to handle constraints 1 φ(x(t), t) ≥ 0. The optimal trajectory connecting ˜ xi and ˜ xi+1 must not violate the constraints: V (˜ xi , ˜ xi+1) := min ti ≤t≤ti+1,1≤k≤p φk(γi (t), t) = 0, where γi (t) is the optimal path connecting junction pairs ˜ xi , ˜ xi+1. 1In order to explain idea clearly, we only consider state constraints. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 11 / 25 

New method: Finite optimization As a result, in order to find the optimal trajectory, only the optimal junctions need to be computed. We gain a tremendous dimension reduction since the number of junctions is finite. The optimal control problem can be written as min ˜ x0,··· ,˜ xN+1 J(˜ x0, ˜ x1, · · · , ˜ xN+1) such that V (˜ xi , ˜ xi+1) = 0. The method is called Method of evolving junctions (MEJ). Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 12 / 25 

MEJ: Intermittent diffusion Consider d ˜ x = P˜ x [−∇J(˜ x)dθ + σ(θ)dW (θ)], where ˜ x = (˜ x0, ˜ x1, · · · , ˜ xN, ˜ xN+1), θ is an artificial time variable that is different from t, W (θ) the standard Brownian motion, P˜ x the orthogonal projection onto the tangent plane at ˜ x, and σ(θ) is a piecewise constant function. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 13 / 25 

MEJ: Intermittent diffusion Consider d ˜ x = P˜ x [−∇J(˜ x)dθ + σ(θ)dW (θ)], where ˜ x = (˜ x0, ˜ x1, · · · , ˜ xN, ˜ xN+1), θ is an artificial time variable that is different from t, W (θ) the standard Brownian motion, P˜ x the orthogonal projection onto the tangent plane at ˜ x, and σ(θ) is a piecewise constant function. We create stochastic processes in different finite dimensional spaces . Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 13 / 25 

MEJ: Dimension changes ˜ xk ˜ y ˜ xk+1 ˜ xk−1 ˜ xk = ˜ xk+1 ˜ xk+2 ˜ y ˜ z Insert junction Remove junction Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 14 / 25 

Example 1: Shortest path problem (c) (d) (e) (f) Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 15 / 25 

Example 1: Shortest path problem Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 16 / 25 

Example 2: Path planning problem The problem is to control a robot that moves from a starting point to a destination; avoids any collision with static or moving obstacles; attains a global minimum for energy consumptions. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 17 / 25 

Problem formulation Mathematically, the problem can be written as min γ,v T 0 v(t)2dt + cT, subject to ˙ γ(t) = v(t), γ(0) = X, γ(T) = Y , γ(t) ∈ R2 c (t), ||v(t)|| ≤ vm, t ∈ [0, T]. Here v and t represent the velocity and time respectively, constant vm is the maximal speed that the robot can move, · is the 2-norm, and R2 c (t) is the time dependent obstacle-free space. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 18 / 25 

Example 2: Moving disks environment In this case, obstacles are 6 disks with constant velocities. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 19 / 25 

Example 2: Shrinking/Expanding disks environment In this case, obstacles are shrinking/ expanding disks with constant velocities. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 20 / 25 

Example 3: Differential Games The game is to control two robots that move from start points to their destinations; avoid any collisions with each other; attain their own global minimum for energy consumptions. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 21 / 25 

Example 3: Differential games Robot X solves one optimal control min x T 0 ˙ x(t)2dt s.t. x(0) = A, x(T) = A1, dist(x(t), y(t)) ≥ , while Robot Y faces the other optimal control min y T 0 ˙ y(t)2dt s.t. y(0) = B, y(T) = B1, dist(x(t), y(t)) ≥ . Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 22 / 25 

Example 3: Nash equilibria Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 23 / 25 

Summary We conclude that MEJ has the following properties: Fundamentally dimension reduction; Fast and accurate; Ability to find a globally optimal path as well as a series of locally optimal paths by the adoption of intermittent diffusion. Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 24 / 25 

Thanks! Wuchen Li (Georgia Institute of Technology)A new approach to optimal control with constraints AIMS 2016 25 / 25