Slide 1

Slide 1 text

Mean Field Game MURI Report Stanley Osher (UCLA) August, 2021 1 AFOSR MURI FA9550-18-1-0502

Slide 2

Slide 2 text

MURI’s Product 2

Slide 3

Slide 3 text

5G communication AI UAV COVID 19 pandemic control 3

Slide 4

Slide 4 text

Mean field game 4

Slide 5

Slide 5 text

Mean field system A typical time-dependent MFG system (derived from (9)) has the form 8 > > > > > < > > > > > : t 2 + H (x, Dx (x, t), ⇢(x, t), t) = 0 ⇢t 2 ⇢ divx (⇢DpH (x, Dx (x, t), ⇢(x, t), t)) = 0 ⇢ 0, R Td ⇢(x, t)dx = 1, t 2 [0, T] ⇢(x, T) = ⇢T (x), (x, 0) = 0(x, ⇢(x, 0)), x 2 Td . (10) Given: H : Td ⇥ Rd ⇥ X ⇥ R ! R is a periodic in space and convex in momentum Hamiltonian, where X = R+ or X = L 1(Td ; R+) or X = R+ ⇥ L 1(Td ; R+). H = L ⇤ is a di↵usion parameter 0 : Td ⇥ X ! R, ⇢T : Td ! R given initial-terminal conditions Unknowns: , ⇢ : Td ⇥ [0, T] ! R has the form 5

Slide 6

Slide 6 text

Mean Field Control Problems For Vaccine Distribution Wonjun Lee, Siting Liu, Wuchen Li, Stanley Osher 2021 1 6

Slide 7

Slide 7 text

2 7

Slide 8

Slide 8 text

Vaccine: Manufacturing, Delivery, and Consumption When and where to deliver? I Amount of vaccine is limited. I Vaccine production is limited I Delivery is expensive 3 8

Slide 9

Slide 9 text

Spatial SIRV dynamic model We formulate the optimization problem that satisfies the following equations and constraints. @t⇢S + r · mS = ⇢S K ⇤ ⇢I + ⌘2 S 2 ⇢S ✓1⇢V ⇢S @t⇢I + r · mI = ⇢S K ⇤ ⇢I ⇢I + ⌘2 I 2 ⇢I @t⇢R + r · mR = ⇢I + ⌘2 R 2 ⇢S + ✓1⇢V ⇢S PDE of the vaccine distribution @t⇢V = f(t, x) ✓1⇢V ⇢S , 0  t < T 0 @t⇢V + r · mV = ✓2⇢V ⇢S , T 0  t  T and constraints on the vaccine density function ⇢V and the factory function f(t, x). 0  f(t, x)  fmax (t, x) 2 [0, T 0] ⇥ ⌦factory f(t, x) = 0 (t, x) 2 [0, T 0] ⇥ ⌦\⌦factory ⇢V (t, x)  Cfactory (t, x) 2 [0, T 0] ⇥ ⌦factory 4 Vaccine model 9

Slide 10

Slide 10 text

Methods Consider X i=S,I,R,V Ei(⇢i(T, ·)) + Z T 0 Z ⌦ X i=S,I,R Fi(⇢i, mi)dx dt + Z T T 0 Z ⌦ FV (⇢V , mV )dx dt + Z T 0 GP ((⇢S + ⇢I + ⇢R)(t, ·)) + GV (⇢V (t, ·))dt + Z T 0 0 G0(f(t, ·))dt Here, the cost functional satisfies the following conditions. I minimize the transportation cost for moving each population; I minimize the total number of infected people and susceptible people by maximizing the usage of the vaccines at time T ; I maximize the total number of recovered people at time T ; I avoid high concentration of population and vaccines at each time t 2 (0, T ); I minimize the amount of vaccines produced during t 2 (0, T 0); I minimize the transportation cost for delivering vaccines during t 2 (T 0, T ). 5 10

Slide 11

Slide 11 text

Mean field systems for Vaccine distributions We highlight the dynamical systems of vaccine distributions in the model. 8 > > > > > > > > > < > > > > > > > > > : @t V + GV ⇢ (⇢V ) + ⇢S ✓1( R S) ✓2 V ) = 0 @t V ↵V 2 |r V |2 + GV ⇢ (⇢V ) + ⇢S ✓1( R S) ✓2 V ) = 0 @t⇢V f + ✓2⇢S⇢V = 0 @t⇢V 1 ↵V r · (⇢V r V ) + ✓2⇢S⇢V = 0 6 11

Slide 12

Slide 12 text

Example I 7 12

Slide 13

Slide 13 text

Example II 8 13

Slide 14

Slide 14 text

Future directions The proposed mean field control formulations can be adjusted to model other natural disasters. I Rescue/Control problems in California wildfires; I Evaluation plans for Florida hurricanes, e.t.c. References: I Lee, W., Liu, S., Li, W., Osher, S. (2021). Mean field control problems for vaccine distribution. arXiv:2104.11887. 9 Evacuation 14

Slide 15

Slide 15 text

A Fast Proximal Gradient Method and Convergence Analysis for Dynamic Mean Field Planning Jiajia Yu1 Rongjie Lai1 Wuchen Li2 Stanley Osher3 June 7, 2021 1Department of Mathematics, Rensselaer Polytechnic Institute, United States. 2Department of Mathematics, University of South Carolina, United States. 3Department of Mathematics, University of California, Los Angeles, United States. Funding: J. Yu and R. Lai’s work are supported in part by an NSF career award DMS–1752934. W. Li and S. Osher’s work are supported in part by AFOSR MURI FP 9550-18-1-502. 15

Slide 16

Slide 16 text

Introduction Density transporting/fitting (a) crowd simulation (b) source distribution (c) generative models Applications of density transporting/fitting 2 16

Slide 17

Slide 17 text

Formulations Let C(⇢0, ⇢1) := {(⇢, m) : @t⇢ + rx · m = 0, ⇢(0, ·) = ⇢0, ⇢(1, ·) = ⇢1} . I Optimal Transport min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) dxdt. I Mean Field Planning min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + F(x, ⇢(t, x))dxdt. Let C(⇢0) := {(⇢, m) : @t⇢ + rx · m = 0, ⇢(0, ·) = ⇢0}. I Mean Field Game min ⇢,m2C(⇢0) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + F(x, ⇢(t, x))dxdt + Z ⌦ G(x, ⇢(1, x))dx. 3 17

Slide 18

Slide 18 text

Algorithm: Proximal Gradient Descent MFP: min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + F(x, ⇢(t, x))dxdt | {z } Y(⇢,m) , min ⇢,m Y(⇢, m) + XC(⇢0,⇢1) (⇢, m) where XC(⇢0,⇢1) (⇢, m) = ( +1, (⇢, m) 62 C(⇢0, ⇢1), 0, (⇢, m) 2 C(⇢0, ⇢1). Proximal Gradient Descent method for MFP: 8 > > > > > > > > > > > > > > < > > > > > > > > > > > > > > : ⇣ ⇢k+ 1 2 , mk+ 1 2 ⌘ = ⇢k, mk ⌘k ⇢,mY ⇢k, mk , | {z } gradient (forward) descent easy and fast to compute for di↵erentiable F ⇢k+1, mk+1 = ProjC(⇢0,⇢1) ⇣ ⇢k+ 1 2 , mk+ 1 2 ⌘ , | {z } proximal (backward) descent = ⇣ ⇢k+ 1 2 , mk+ 1 2 ⌘ rt,m 1 t,mrt,m · ⇣ ⇢k+ 1 2 , mk+ 1 2 ⌘ . | {z } independent to the form of F fast to compute with Fast Cosine Transformation 4 18

Slide 19

Slide 19 text

Summary: E ciency and Flexibility PGD is a flexible algorithm for MFP: I The gradient descent step is easy to compute as long as regularizer F is di↵erentiable. I The proximal descent step is independent to the form of F. PGD is also e cient: I The gradient descent step can be implemented elementwise. I The proximal descent step is e cient to compute with fast cosine transform. I If ⌦ = [0, 1]d 1 and the domain [0, 1] ⇥ ⌦ is uniformly discretized to nd grids, then the computation cost for each step is O(dnd log n). I By applying accelerated PGD and multigrid strategies, we can further reduce the number of iterations and total time for convergence. 5 19

Slide 20

Slide 20 text

Convergence: Discretized solution ! continuous optimizer Our algorithm produces (Pk, Mk) as an approximation of (⇢k, mk) on grid points. With mild assumptions, We, for the first time, establish the following convergence based on an optimization algorithm: Theorem (Yu-Lai-Li-Osher’21) I (Pk, Mk) ! (⇢k, mk) on grid points for any fixed k as mesh size h ! 0. I (Pk, Mk) ! (⇢⇤, m⇤) on grid points as h ! 0, k ! 1. I (P⇤, M⇤) ! (⇢⇤, m⇤) on grid points as h ! 0 6 20

Slide 21

Slide 21 text

E ciency and Accuracy (Optimal Transport) min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) dxdt Table: E ciency and accuracy comparison of OT with best performance highlighted in red. (ALG=Augmented Lagrangian, G-prox=G-prox PDHG. Left: 1D. Right: 2D. nt =64, nx = ny =256.) 7 21

Slide 22

Slide 22 text

Flexibility I: regions with obstacles min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + Q⇢(t, x)Q(x)dxdt where Q(x) = ( 1, x 2 white region, 0, otherwise , . Figure: Evolution of densities for various ⇢0, ⇢1, Q(x). 8 22

Slide 23

Slide 23 text

Flexibility II: di↵erent density regularizers min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + Q⇢(t, x)Q(x) + EFE(⇢(t, x))dxdt where Q(x) = ( 1, ⇢0(x) = 0 and ⇢1(x) = 0, 0, otherwise , . (a) Q = 0, FE(a) = 0 (b) Q 6= 0, FE(a) = a2 2 (c) Q 6= 0, FE(a) = 1 a Figure: Evolution of densities for MFP with di↵erent density regularizers. 9 23

Slide 24

Slide 24 text

Generalization to MFG (MFP) min ⇢,m2C(⇢0,⇢1) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + F(x, ⇢(t, x))dxdt (MFG) min ⇢,m2C(⇢0) Z 1 0 Z ⌦ km(t, x)k2 2 2⇢(t, x) + F(x, ⇢(t, x))dxdt Z ⌦ ⇢1(x)⇢(1, x)dx (a) MFP (b) MFG Figure: Evolution of densities for OT, MFP and MFG. 10 Mean field planning Mean field Control/Game 24

Slide 25

Slide 25 text

APAC-Net: Alternating the Population and Control Neural Networks, for Mean-Field Games Problems Alex Tong Lin Joint with: Samy Wu Fung, Levon Nurbekyan, Wuchen Li, and Stanley Osher June 5, 2021 1 25

Slide 26

Slide 26 text

Variational Mean-Field Games I Namely for some mean-field games, the equilibrium solution can be found by minimizing an overall “energy” (e.g. multiply the value function for a single agent by ⇢): A(⇢, v) = min ⇢,v Z T 0 Z ⌦ ⇢(x, t)L(v) + F(x, ⇢) dx dt + Z ⌦ g(x)⇢(x, T) dx where x = x(t), v = v(t) above, and where the optimization has the constraint, @t⇢ ⌫ ⇢ + div(⇢v) = 0. 2 26

Slide 27

Slide 27 text

Variational Mean-Field Games / APAC-Net Preview After some elevating the constraints into the objective, integrating by parts, and calculating the Legendre transform, we get = max ' min ⇢ Z T 0 Z ⌦ F(x, ⇢(x, t)) dx dt + Z ⌦ ⇣ g(x) '(x, T) ⌘ ⇢(x, T) dx + Z ⌦ '(x, 0)⇢(x, 0) dx + Z T 0 Z ⌦ ✓ @t'(x, t) + '(x, t) L⇤( r'(x, t)) ◆ ⇢(x, t) dx dt which means we end up with a sampling problem – this is a preview of APAC-Net. This is in the spirit of Feynman-Kac. Then the idea is to turn ⇢ and ' into neural networks and train as in GANs (Generative Adversarial Networks). 3 27

Slide 28

Slide 28 text

GAN Training I Training has a discriminator and a generator. The generator produces samples (analogous to our ⇢), and the discriminator evaluates the quality of those samples (analogous to our '). 4 28

Slide 29

Slide 29 text

Why Neural Networks? Because the usual way to do things – using a grid to discretize space – becomes impossible in high dimensions. 5 29

Slide 30

Slide 30 text

Numerical Results - Obstacles & Congestion I H(p) = kpk2 I A 20 dimensional obstacle problem where we have an obstacle in (x1, x2) and in (x3, x4) and in (x5, x6), etc. Congestion penalty is active in the bottlenecks. Figure: A screencapture of a video that will be played 6 30

Slide 31

Slide 31 text

Numerical Results: A realistic example, the Quadcopter The dynamics of a quadcopter are: 8 > > > > > > < > > > > > > : ¨ x = u m (sin( ) sin( ) + cos( ) cos( ) sin(✓)) ¨ y = u m ( cos( ) sin( ) + cos( ) sin(✓) sin( )) ¨ z = u m cos(✓) cos( ) g ¨ = ˜ ⌧ ¨ ✓ = ˜ ⌧✓ ¨ = ˜ ⌧ where u is the thrust, g is the gravitational acceleration (9.81m/s2), and x, y, z are the spatial coordinates, , ✓, are the angular coordinates, and ˜ ⌧ , ˜ ⌧✓ , ˜ ⌧ . Turns 12-dimensional when you transfer to first-order system. 7 31

Slide 32

Slide 32 text

Numerical Results: A realistic example, the Quadcopter 8 32

Slide 33

Slide 33 text

Numerical Results: A realistic example, the Quadcopter Movie: Figure: A screencapture of a video that will be played 9 33