Istituto Italiano di Tecnologia & Massachusetts Institute of Technology Genoa, Italie Journées du GdR MOA 2 Décembre, 2015 Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 1/19
: H → Rm locally Lipschitz, H Hilbert. Solve MIN (f1(x), ..., fm(x)) : x ∈ C ⊂ H convex. We consider the usual order(s) on Rm: a ĺ b ⇔ ai ≤ bi for all i = 1, ..., m, a ă b ⇔ ai < bi for all i = 1, ..., m. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 3/19
: H → Rm locally Lipschitz, H Hilbert. Solve MIN (f1(x), ..., fm(x)) : x ∈ C ⊂ H convex. We consider the usual order(s) on Rm: a ĺ b ⇔ ai ≤ bi for all i = 1, ..., m, a ă b ⇔ ai < bi for all i = 1, ..., m. x is a Pareto point if y ∈ C such that F(y) ň F(x) x is a weak Pareto point if y ∈ C such that F(y) ă F(x) Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 3/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. How to solve it? Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. How to solve it? genetic algorithm −→ no theoretical guarantees. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. We are going to present a method which: Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the steepest descent dynamic ˙ x(t) + ∇f (x(t)) = 0, Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the steepest descent dynamic ˙ x(t) + ∇f (x(t)) = 0, is cooperative, i.e. all objective functions decrease simultaneously, Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
: H → Rm locally Lipschitz. Solve MIN f1(x), ..., fm(x) : x ∈ C ⊂ H convex. We are going to present a method which: generalizes the steepest descent dynamic ˙ x(t) + ∇f (x(t)) = 0, is cooperative, i.e. all objective functions decrease simultaneously, is independent of any choice of parameters. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 4/19
H −→ Rm locally Lipschitz, C = H Hilbert. Definition For all x ∈ H, s(x) := − (co {∂Cfi (x)}i=1,...,m)0 is the (common) steepest descent direction at x. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 6/19
H −→ Rm locally Lipschitz, C = H Hilbert. Definition For all x ∈ H, s(x) := − (co {∂Cfi (x)}i=1,...,m)0 is the (common) steepest descent direction at x. Remarks in the smooth case If m = 1 then s(x) = −∇f1(x). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 6/19
H −→ Rm locally Lipschitz, C = H Hilbert. Definition For all x ∈ H, s(x) := − (co {∂Cfi (x)}i=1,...,m)0 is the (common) steepest descent direction at x. Remarks in the smooth case If m = 1 then s(x) = −∇f1(x). At each x, s(x) selects a convex combination: s(x) = − m i=1 θi (x)∇fi (x) = −∇fθ(x) (x) where fθ(x) = m i=1 θi (x)fi . Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 6/19
H −→ Rm locally Lipschitz, C = H Hilbert. Definition For all x ∈ H, s(x) := − (co {∂Cfi (x)}i=1,...,m)0 is the (common) steepest descent direction at x. Remarks in the smooth case If m = 1 then s(x) = −∇f1(x). At each x, s(x) selects a convex combination: s(x) = − m i=1 θi (x)∇fi (x) = −∇fθ(x) (x) where fθ(x) = m i=1 θi (x)fi . s(x) is the steepest descent: s(x) s(x) = argmin d∈BH max i=1,...,m ∇fi (x), d . Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 6/19
descent dynamic: (SD) ˙ x(t) = s(x(t)), i.e. (SD) ˙ x(t) + (co {∂Cfi (x(t))}i )0 = 0. A solution is a trajectory x : [0, +∞[−→ H being absolutely continuous (on bounded intervals), satisfying (SD) for a.e. t ≥ 0. It is the continuous version of the steepest descent algorithm studied by Svaiter, Fliege, Iusem, ... Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 7/19
2014) A cooperative dynamic Let x : R+ −→ H be a solution of (SD) ˙ x(t) = s(x(t)). For all i = 1, ..., m, the function t → fi (x(·)) is decreasing. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 10/19
2014) A cooperative dynamic Let x : R+ −→ H be a solution of (SD) ˙ x(t) = s(x(t)). For all i = 1, ..., m, the function t → fi (x(·)) is decreasing. Convergence in the convex case Assume that the objective functions are convex. Then any bounded trajectory weakly converges to a weak Pareto point. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 10/19
2014) A cooperative dynamic Let x : R+ −→ H be a solution of (SD) ˙ x(t) = s(x(t)). For all i = 1, ..., m, the function t → fi (x(·)) is decreasing. Convergence in the convex case Assume that the objective functions are convex. Then any bounded trajectory weakly converges to a weak Pareto point. Existence in the convex case Suppose that H is finite dimensional. Then, for any initial data, there exists a global solution to (SD). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 10/19
convex constraint C ⊂ H: (SD) ˙ x(t) + (NC (x(t)) + co {∂Cfi (x(t))}i )0 = 0. Uniqueness? Yes, if {∇fi (x(·))}i=1,...,m are affinely independants. Convergence to Pareto points? Guaranteed by endowing Rm with a different order (recall F = (f1, ..., fm) : H −→ Rm). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 11/19
among the weak Paretos (= the zeros of x → s(x)) the closest to a desired state? → Tikhonov penalization Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 13/19
among the weak Paretos (= the zeros of x → s(x)) the closest to a desired state? → Tikhonov penalization ˙ x(t) − s(x(t)) + ε(t)(x(t) − xd ) = 0, ε(t) ↓ 0, ∞ 0 ε(t) dt = +∞. See the works of Attouch, Cabot, Czarnecki, Peypouquet (...) in the monotone case. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 13/19
f1, ..., fm be smooth, with L-Lipschitz gradient. (ISD) m¨ x(t) = −γ ˙ x(t) + s(x(t)). Assume that γ ≥ L. Existence Suppose that H is finite dimensional. Then, for any initial data, there exists a global solution to (ISD). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 17/19
f1, ..., fm be smooth, with L-Lipschitz gradient. (ISD) m¨ x(t) = −γ ˙ x(t) + s(x(t)). Assume that γ ≥ L. Existence Suppose that H is finite dimensional. Then, for any initial data, there exists a global solution to (ISD). Convergence in the convex case Let f1, ..., fm be convex. Then, any bounded trajectory weakly converges to a weak Pareto point. Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 17/19
to multi-objective optimization problems. Open questions: Uniqueness of the trajectories for ˙ x(t) = s(x(t))? Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 18/19
to multi-objective optimization problems. Open questions: Uniqueness of the trajectories for ˙ x(t) = s(x(t))? Understand the asymptotic behaviour of ˙ x(t) − s(x(t)) + ε(t)x(t) = 0 (the set of weak Paretos is non-convex). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 18/19
to multi-objective optimization problems. Open questions: Uniqueness of the trajectories for ˙ x(t) = s(x(t))? Understand the asymptotic behaviour of ˙ x(t) − s(x(t)) + ε(t)x(t) = 0 (the set of weak Paretos is non-convex). Having convergence rates for first and second-order dynamics (the critical values are not unique). Journées du GdR MOA 2015 - Dijon - Guillaume Garrigos 18/19
moa2015
takyamamoto
kamakiri1225
ailaboocu
sansandsoc
settenqb
shiftpn
shimacos
cookiebox26
fukunaga
ks91
kazk1018
scrummasudar
vanstee
holman
nonsquared
jponch
brianwarren
dougneiner
jakevdp
marcelosomers
danielanewman
maltzj
thoeni
destraynor
