in the mathematics community and M. Huang, P. Caines, R. Malham´ e in the engineering one to model huge populations of identical, rational agents that play non-cooperative di↵erential games. In a PDE form, MFG resemble macroscopic equations from statistical physics in game-theoretic/optimal control framework. Yield near optimal solutions for huge populations of optimizing agents. Thus, tremendous simpliﬁcation in practical problems. engineer ﬁeld !4
with a distribution density ⇢(x, t) play a non-cooperative di↵erential game. An individual agent faces an optimization problem (x, t) = inf E t Z 0 L(Xs, vs, ⇢(·, s), s)ds + 0(X0, ⇢(·, 0)), (9) where dXs = vsds + p 2 dWs, XT = x, and L is the Lagrangian. In Nash equilibrium, each agent cannot do better unilaterally. Optimal strategies are given by the value function, , that solves the corresponding Hamilton-Jacobi equation. The density, ⇢(x, t), evolves according to optimal actions of agents. Thus, the Fokker-Planck equation. !5
(9)) has the form 8 > > > > > < > > > > > : t 2 + H (x, Dx (x, t), ⇢(x, t), t) = 0 ⇢t 2 ⇢ divx (⇢DpH (x, Dx (x, t), ⇢(x, t), t)) = 0 ⇢ 0, R Td ⇢(x, t)dx = 1, t 2 [0, T] ⇢(x, T) = ⇢T (x), (x, 0) = 0(x, ⇢(x, 0)), x 2 Td . (10) Given: H : Td ⇥ Rd ⇥ X ⇥ R ! R is a periodic in space and convex in momentum Hamiltonian, where X = R+ or X = L 1(Td ; R+) or X = R+ ⇥ L 1(Td ; R+). H = L ⇤ is a di↵usion parameter 0 : Td ⇥ X ! R, ⇢T : Td ! R given initial-terminal conditions Unknowns: , ⇢ : Td ⇥ [0, T] ! R above !6
of H, L on the density, ⇢. Local interaction if H, L depend on ⇢ locally. For instance, H(x, p, ⇢, t) = ˜ H(x, p, t) ln ⇢ H(x, p, ⇢, t) = ˜ H(x, p, t) ⇢ Non-local interaction if H, L depend on ⇢ non-locally. For instance, H(x, p, ⇢, t) = ˜ H(x, p, t) F 0 @ x, Z Td K(x, y)⇢(y)dy 1 A Congestion interaction if we penalize moving in dense area. For instance (soft congestion) H(x, p, ⇢, t) = ⇢↵ ˜ H ✓ x, p ⇢↵ , ⇢, t ◆ , ↵ > 0 !7
2t Hopf-Lax on density space E.g. (Burgers’) Hamilton-Jacobi on density space Characteristics on density space (Nash equilibrium in mean ﬁeld games) @tµs + r · (µs r s) = 0 @t s + 1 2 (r s)2 = 0 Math Review: mean ﬁeld games @ @t U(t, µ) + 1 2 Z Td (r µ U(t, µ))2µ(x)dx = 0, U(0, µ) = F(µ) !8
x)µ(t, x)dt, with the fact Z 1 0 f(t)dt = c = 1 |⌦| ⇣ Z ⌦ µ1(x)dx Z ⌦ µ0(x)dx ⌘ . then by Jensen’s inequality and integrating the time variable t, we obtain UW1(µ0, µ1) = inf m n Z ⌦ km(x)kdx + 1 ↵ Z ⌦ µ0(x)dx Z ⌦ µ1(x)dx : µ1(x) µ0(x) + r · m(x) = 1 |⌦| ⇣Z ⌦ µ1(x)dx Z ⌦ µ0(x)dx ⌘o . 16 From this equation, we see the momentum is independent of alpha. 16
⌦ = [0, 1], the L1 unnormalized Wasserstein metric has the following explicit solution: UW1(µ0, µ1) = Z ⌦ Z x 0 µ1(y)dy Z x 0 µ0(y)dy x Z ⌦ (µ1(z) µ0(z))dz dx + 1 ↵ ⇣ Z ⌦ µ1(z)dz Z ⌦ µ0(z)dz ⌘ . 17 If the total masses are equal, the additional term disappears. 17
n Z ⌦ (1, x)µ(1, x)dx Z ⌦ (0, x)µ(0, x)dx ↵ 2 Z 1 0 ⇣ Z ⌦ (t, x)dx ⌘2 dt o where the supremum is taken among all : [0, 1] ! ⌦ satisfying @t (t, x) + 1 2 kr (t, x)k2 0. 25 This is connected to one of UCLA’s fast algorithms for OT by M. Jacobs and F. Leger, 2019. 21
of MFG from torus strategy set to a manifold M, point clouds, and graphs (discrete strategy set) U(t, µ) = inf v,µ,f Z 1 0 Z M kv(t, x)kpµ(t, x) F(µ(t, ·))dxdt + G(µ(0, ·)) such that the dynamical constraint is satisﬁed @tµ + r · (µv) = 0, µ(1, x) = µ1(x) Consider !32
Quadratic interaction energy with linear periodic potential energy We test our algorithms using examples in Achdou, Capuzzo-Dolcetta, Mean Field Games: Numerical Methods, 2009. !35
interactions 8 > < > : t + D + H(x, r ) = R ⌦ K(x, y)⇢(y, t)dy, ⇢t + D⇤⇢ div(⇢rpH(x, r )) = 0, (x, 0) = 0(x), ⇢(x, T) = ⇢T (x). (13) Above, K is a monotone symmetric interaction kernel, D is a monotone linear operator (e.g. D = ). Strategy: Model the interaction in a Fourier space. Speciﬁcally, given a basis {x 7! fi (x)}i expand the kernel K(x, y) = X i,j kij fi (x)fj (y). !37
(t)fi (x), where ai (t) = X j kij Z ⌦ ⇢(y, t)fi (y)dy. Hence, (13) can be written as 8 > > > > < > > > > : t + D + H(x, r ) = P i ai (t)fi (x), ⇢t + D⇤⇢ div(⇢rpH(x, r )) = 0, ai (t) = P j kij R ⌦ ⇢(y, t)fi (y)dy, 8i, (x, 0) = 0(x), ⇢(x, T) = ⇢T (x). (14) The problem !38
⇢ a · K 1 a 2 Z ⌦ a(x, T)⇢T (x)dx s.t. (16) holds , (15) where K = (kij ), and a is the viscosity solution of ( t + D + H(x, r ) = P i ai (t)fi (x), (x, 0) = 0(x). (16) Advantages: (15) is a convex optimization problem! Flexible and sparse representations of mean-ﬁeld interactions Grid-free approximation methods for ﬁrst-order problems Compatibility with powerful convex optimization techniques such as ADMM, PDHG the system It is !39
the variational problem @t⇢(✓(t), x) + r · (⇢(✓(t), x)v(t, x)) = 0, ✓(T) ﬁxed such that the constrained continuity equation holds Typical parameterized density are given by probability models, such as mixed Gaussian, deep neural networks, graphical models etc. inf ⇢(✓,·) Z T 0 n Z L(x, v)⇢(✓(t), x)dx F(⇢(✓(t), ·)) o dt + G(⇢(✓(0), ·)) !48