Share
Embed
Transcript
53 slides
-
Mean-field games (MFG) framework Introduced by P.-L. Lions, J.-M. Lasry
in the mathematics community and M. Huang, P. Caines, R. Malham´ e in the engineering one to model huge populations of identical, rational agents that play non-cooperative di↵erential games. In a PDE form, MFG resemble macroscopic equations from statistical physics in game-theoretic/optimal control framework. Yield near optimal solutions for huge populations of optimizing agents. Thus, tremendous simplification in practical problems. engineer field !4 -
Nash equilibrium in a differential game A continuum of players
with a distribution density ⇢(x, t) play a non-cooperative di↵erential game. An individual agent faces an optimization problem (x, t) = inf E t Z 0 L(Xs, vs, ⇢(·, s), s)ds + 0(X0, ⇢(·, 0)), (9) where dXs = vsds + p 2 dWs, XT = x, and L is the Lagrangian. In Nash equilibrium, each agent cannot do better unilaterally. Optimal strategies are given by the value function, , that solves the corresponding Hamilton-Jacobi equation. The density, ⇢(x, t), evolves according to optimal actions of agents. Thus, the Fokker-Planck equation. !5 -
The PDE formulation A typical time-dependent MFG system (derived from
(9)) has the form 8 > > > > > < > > > > > : t 2 + H (x, Dx (x, t), ⇢(x, t), t) = 0 ⇢t 2 ⇢ divx (⇢DpH (x, Dx (x, t), ⇢(x, t), t)) = 0 ⇢ 0, R Td ⇢(x, t)dx = 1, t 2 [0, T] ⇢(x, T) = ⇢T (x), (x, 0) = 0(x, ⇢(x, 0)), x 2 Td . (10) Given: H : Td ⇥ Rd ⇥ X ⇥ R ! R is a periodic in space and convex in momentum Hamiltonian, where X = R+ or X = L 1(Td ; R+) or X = R+ ⇥ L 1(Td ; R+). H = L ⇤ is a di↵usion parameter 0 : Td ⇥ X ! R, ⇢T : Td ! R given initial-terminal conditions Unknowns: , ⇢ : Td ⇥ [0, T] ! R above !6 -
Classification of MFG Mean-field interaction or non-linearity is the dependence
of H, L on the density, ⇢. Local interaction if H, L depend on ⇢ locally. For instance, H(x, p, ⇢, t) = ˜ H(x, p, t) ln ⇢ H(x, p, ⇢, t) = ˜ H(x, p, t) ⇢ Non-local interaction if H, L depend on ⇢ non-locally. For instance, H(x, p, ⇢, t) = ˜ H(x, p, t) F 0 @ x, Z Td K(x, y)⇢(y)dy 1 A Congestion interaction if we penalize moving in dense area. For instance (soft congestion) H(x, p, ⇢, t) = ⇢↵ ˜ H ✓ x, p ⇢↵ , ⇢, t ◆ , ↵ > 0 !7 -
U(t, µ) = inf ⇢2P(Td) F(⇢) + dW (⇢, µ)2
2t Hopf-Lax on density space E.g. (Burgers’) Hamilton-Jacobi on density space Characteristics on density space (Nash equilibrium in mean field games) @tµs + r · (µs r s) = 0 @t s + 1 2 (r s)2 = 0 Math Review: mean field games @ @t U(t, µ) + 1 2 Z Td (r µ U(t, µ))2µ(x)dx = 0, U(0, µ) = F(µ) !8 -
Density Control: Optimal Transport dW (⇢, µ)2 = inf ⇢s,us
n Z 1 0 Z R u2 s ⇢sdxdt: @s⇢s + r · (⇢sus) = 0, ⇢0 = ⇢, ⇢1 = µ o 8 What happens if the total population density is not normalized/balanced? 9 -
0 1 0 Density (7(x; t)) Time (t) 0.5 0.5
0.5 Space (x) 1 0 1 14 -
Unnormalized L1 Wasserstein metric Let p = 1: UW1(µ0, µ1)
= inf v,f(t) n Z 1 0 Z ⌦ kvkµdxdt + 1 ↵ Z 1 0 |f(t)|dt · |⌦|: @tµ + r · (µv) = f(t) o . 15 15 -
Time independent solution Denote m(x) = Z 1 0 v(t,
x)µ(t, x)dt, with the fact Z 1 0 f(t)dt = c = 1 |⌦| ⇣ Z ⌦ µ1(x)dx Z ⌦ µ0(x)dx ⌘ . then by Jensen’s inequality and integrating the time variable t, we obtain UW1(µ0, µ1) = inf m n Z ⌦ km(x)kdx + 1 ↵ Z ⌦ µ0(x)dx Z ⌦ µ1(x)dx : µ1(x) µ0(x) + r · m(x) = 1 |⌦| ⇣Z ⌦ µ1(x)dx Z ⌦ µ0(x)dx ⌘o . 16 From this equation, we see the momentum is independent of alpha. 16 -
Closed form solution In one space dimension on the interval
⌦ = [0, 1], the L1 unnormalized Wasserstein metric has the following explicit solution: UW1(µ0, µ1) = Z ⌦ Z x 0 µ1(y)dy Z x 0 µ0(y)dy x Z ⌦ (µ1(z) µ0(z))dz dx + 1 ↵ ⇣ Z ⌦ µ1(z)dz Z ⌦ µ0(z)dz ⌘ . 17 If the total masses are equal, the additional term disappears. 17 -
Unnormalized L2 Wasserstein metric Let p = 2: UW2(µ0, µ1)
= inf v,f(t) n Z 1 0 Z ⌦ kvk2µdxdt + 1 ↵ Z 1 0 f(t) 2dt · |⌦|: @tµ + r · (µv) = f(t) o . 22 18 -
Minimizer system The minimizer (v(t, x), µ(t, x), f(t)) for
UOT problem satisfies v(t, x) = r (t, x), f(t) = ↵ 1 |⌦| Z ⌦ (t, x)dx, and 8 > > > > < > > > > : @tµ(t, x) + r · (µ(t, x)r (t, x)) = ↵ 1 |⌦| Z ⌦ (t, x)dx @t (t, x) + 1 2 kr (t, x)k2 = 0 µ(0, x) = µ0(x), µ(1, x) = µ1(x). 23 19 -
Unnormalized Monge-Ampere equation Denote (x) = 1 2 kxk2 +
(0, x), Following the Hopf-Lax formula, the minimizer of unnormalized OT satisfies µ(1, r (x))Det(r2 (x)) µ(0, x) =↵ Z 1 0 Det ⇣ tr2 (x) + (1 t)I ⌘ · Z ⌦ ⇣ (y) kyk2 2 + tkr (y) yk2 2 ⌘ Det ⇣ tr2 (y) + (1 t)I ⌘ dydt. 24 20 -
Unnormalized Kantorovich problem 1 2 UW2(µ0, µ1) 2 = sup
n Z ⌦ (1, x)µ(1, x)dx Z ⌦ (0, x)µ(0, x)dx ↵ 2 Z 1 0 ⇣ Z ⌦ (t, x)dx ⌘2 dt o where the supremum is taken among all : [0, 1] ! ⌦ satisfying @t (t, x) + 1 2 kr (t, x)k2 0. 25 This is connected to one of UCLA’s fast algorithms for OT by M. Jacobs and F. Leger, 2019. 21 -
Generalized unnormalized OT In fact, when p=2, the minimization has
the unconstrained optimization formulation: Models suggested by Chen, Georgiou, Ling, Tannenbaum, 2017 for p=1,2. where Lµ = r · (µr) !27 -
Mean field game on manifolds Here we replace the domain
of MFG from torus strategy set to a manifold M, point clouds, and graphs (discrete strategy set) U(t, µ) = inf v,µ,f Z 1 0 Z M kv(t, x)kpµ(t, x) F(µ(t, ·))dxdt + G(µ(0, ·)) such that the dynamical constraint is satisfied @tµ + r · (µv) = 0, µ(1, x) = µ1(x) Consider !32 -
MFG in 1D via unconstrained optimization Evolution of density with
Phi(0,x) fixed !34 -
Classical MFG examples via unconstrained optimization The evolution of density
Quadratic interaction energy with linear periodic potential energy We test our algorithms using examples in Achdou, Capuzzo-Dolcetta, Mean Field Games: Numerical Methods, 2009. !35 -
Classical MFG examples via unconstrained optimization The evolution of density:
quadratic interaction energy with linear periodic potential energy !36 -
Spectral methods in nonlocal MFG Goal: Solve MFG with nonlocal
interactions 8 > < > : t + D + H(x, r ) = R ⌦ K(x, y)⇢(y, t)dy, ⇢t + D⇤⇢ div(⇢rpH(x, r )) = 0, (x, 0) = 0(x), ⇢(x, T) = ⇢T (x). (13) Above, K is a monotone symmetric interaction kernel, D is a monotone linear operator (e.g. D = ). Strategy: Model the interaction in a Fourier space. Specifically, given a basis {x 7! fi (x)}i expand the kernel K(x, y) = X i,j kij fi (x)fj (y). !37 -
Furthermore, Z ⌦ K(x, y)⇢(y, t)dy = X i ai
(t)fi (x), where ai (t) = X j kij Z ⌦ ⇢(y, t)fi (y)dy. Hence, (13) can be written as 8 > > > > < > > > > : t + D + H(x, r ) = P i ai (t)fi (x), ⇢t + D⇤⇢ div(⇢rpH(x, r )) = 0, ai (t) = P j kij R ⌦ ⇢(y, t)fi (y)dy, 8i, (x, 0) = 0(x), ⇢(x, T) = ⇢T (x). (14) The problem !38 -
New variational principle: System (14) is equivalent to inf a
⇢ a · K 1 a 2 Z ⌦ a(x, T)⇢T (x)dx s.t. (16) holds , (15) where K = (kij ), and a is the viscosity solution of ( t + D + H(x, r ) = P i ai (t)fi (x), (x, 0) = 0(x). (16) Advantages: (15) is a convex optimization problem! Flexible and sparse representations of mean-field interactions Grid-free approximation methods for first-order problems Compatibility with powerful convex optimization techniques such as ADMM, PDHG the system It is !39 -
Examples: Take a periodic domain ⌦ = T2, and periodic
Gaussian interaction kernels K ,µ (x, y) = 2µ2 ⇡ 2 X ↵2Z2 e 2|x y ↵|2 2 , x, y 2 T2. (17) Furthermore, suppose that H(x, p) = |p|2 2 , (x, p) 2 T2 ⇥ R2, D = 0. (18) Observation: Trigonometric system yields diagonal (sparse) representations for translation invariant kernels! !40 -
Figure: Gaussian kernels for ( , µ) = (0.1, 0.75),
(0.1, 0.5), (1, 0.5). Figure: The terminal distribution ⇢T (x1, x2 ) and the initial cost 0 (x1, x2 ). !41 -
Constrained MFG Given a parameterized density ⇢(✓, x) , consider
the variational problem @t⇢(✓(t), x) + r · (⇢(✓(t), x)v(t, x)) = 0, ✓(T) fixed such that the constrained continuity equation holds Typical parameterized density are given by probability models, such as mixed Gaussian, deep neural networks, graphical models etc. inf ⇢(✓,·) Z T 0 n Z L(x, v)⇢(✓(t), x)dx F(⇢(✓(t), ·)) o dt + G(⇢(✓(0), ·)) !48 -
Evolutionary dynamics for Robotics min ⇢ KL(⇢k⇢ref ) + 1
2 Z Z W(x, y)⇢(x)⇢(y)dxdy Wasserstein Gradient flow @t⇢ = r · (⇢rW ⇤ ⇢) + r · (⇢r log ⇢ ⇢ref ) Lagrangian coordinates ˙ X = rW ⇤ ⇢ + r log ⇢ ⇢ref !50