Newton's method for solving generalized equation

by V. V. Nguyen

Transcript

1. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Josephy-Newton method for generalized equations: Kantorovich’s and Smale’s approaches Van Vu NGUYEN 1 Laboratoire Xlim, Universit´ e de Limoges Journ´ ees annuelles du GDR-MOA Dijon, 4th December, 2015 1A joint work with Samir ADLY and HUYNH Van Ngai
2. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Outline 1 Introduction 2 Metric regularity property of set-valued maps 3 Kantorovich’s theorem and its generalization for GE 4 The Smale’s α-theory and its extension for inclusion
3. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Outline 1 Introduction 2 Metric regularity property of set-valued maps 3 Kantorovich’s theorem and its generalization for GE 4 The Smale’s α-theory and its extension for inclusion
4. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Generalized equation Model: GE : find x ∈ X such that 0 ∈ f (x) + F(x), (1) where f : X −→ Y is at least continuous, F : X ⇒ Y is set-valued map; X and Y are Banach spaces. Problem (1) covers a lot of applications in applied mathematics, engineering, and sciences. When F ≡ 0, (1) recovers an equation f (x) = 0.
5. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Application: feasibility problem Given g : Rm −→ Rn (at least continu- ous), and ∅ = S ⊂ Rn. FP(g, S) Solve: g(x) ∈ S. Here: f (x) = −g(x) and F(x) = S. g(x) S Fig. 1: A feasibility problem Example: constraint system in mathematical programming. gi (x) 0, i = 1, . . . , p, gj (x) = 0, j = p + 1, . . . , p + q, S = R p − × {0} .
6. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Application: complementarity problem Given g : Rm −→ Rm continuous, and ∅ = K ⊂ Rm is a cone. CP(g, K) Solve: x ∈ K, g(x) ∈ K∗, x ⊥ g(x), K∗ := v ∈ Rm : v, u 0, ∀u ∈ K is the dual cone. K K∗ Fig. 2: A cone and its dual Here, f (x) = −   x g(x) x, g(x)  , F(x) =   K K∗ 0  . K K∗ x g(x) Fig. 3: Illustration for a solution Reference: F. Facchinei, J.-S. Pang. Finite-Dimensional Variational Inequalities and Complementarity Problems, Volume I & II. Springer New York, 2003.
7. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Application: variational inequality Given f : Rn −→ Rn continuous, and a closed set ∅ = K ⊂ Rn. VI(f , K) Solve y − x, f (x) 0, ∀y ∈ K. Here, we can take F(x) = NK (x) (Fig.4). x NK (x) −f(x) K Fig. 4: Illustration of normal cone and a solution of VI(f , K) Reference: F. Facchinei, J.-S. Pang. Finite-Dimensional Variational Inequalities and Complementarity Problems, Volume I & II. Springer New York, 2003. Example: first-order optimality condition min g(x) s.t. x ∈ S optimality condition 0 ∈ ∇g(x) + NS (x).
8. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Solving GE: Josephy-Newton framework Standing assumption: f is C1, F has closed graph. starts at a guess point should be near a solution generates a sequence (xk ) (shortly, JN-seq) by solving subproblem 0 ∈ f (xk ) + Df (xk )(xk+1 − xk ) + F(xk+1 ), k = 0, 1, . . . (2) Remark Not all JN-seqs in (2) are convergent. But if such a sequence converges, its limit is a solution of initial GE.
9. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in An example −f y F x0 x x1 x2 Fig. 5: A JN-seq which is convergent −f F x2 x0 x1 x3 y x Fig. 6: A divergent JN-seq Question For which condition and at which guess point we can generate a convergent JN-seq (xk )? And how fast does it converge?
10. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Local convergence of Josephy-Newton algorithm Theorem Let x∗ be such that 0 ∈ f (x∗) + F(x∗) and let Df be Lipschitz continuous around x∗ with modulus L. Assume that the mapping f + F is metrically regular at (x∗, 0) with modulus κ. If γ > 1 2 κL then there exists a neighborhood U of x∗ such that, for any x0 ∈ U, there is a sequence (xk ) generated by the Josephy-Newton scheme and converges quadratically to x∗ xk+1 − x∗ γ xk − x∗ 2, k = 0, 1, . . . (3) Details: A. L. Dontchev, R. T. Rockafellar. Implicit Functions and Solution Mappings: A View from Variational Analysis. Springer New York, 2009.
11. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Outline 1 Introduction 2 Metric regularity property of set-valued maps 3 Kantorovich’s theorem and its generalization for GE 4 The Smale’s α-theory and its extension for inclusion
12. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Metric regularity property Definitions 1 T : X ⇒ Y is metrically regular on a set V ⊂ X × Y with a modulus τ > 0 if d x, T−1(y) τd y, T(x) , ∀(x, y) ∈ V . (4) 2 T : X ⇒ Y is called metrically regular around (¯ x, ¯ y) ∈ gphT if (4) holds for some τ > 0 and V is a neighborhood of (¯ x, ¯ y). Notations: Reg V (T) := infimum of all moduli τ in (4). reg T, (¯ x, ¯ y) := infimum of all τ > 0 such that T is metrically regular around (¯ x, ¯ y) with modulus τ.
13. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Examples in finite dimension 1 A ∈ Rm×n and T : x −→ Ax then reg T, (¯ x, A¯ x) < ∞ ⇔ A is surjective (5) 2 T : Rn −→ Rm is of class C1, then (Graves’s theorem) reg T, (¯ x, T(¯ x)) < ∞ ⇔ ∇T(¯ x) is surjective. (6) 3 m = n = 1, T(x) = αx + ∂(| · |)(x) (see Fig.7 and Fig.8 next slide). reg(T, (0, 1)) = α−1, if α > 0, +∞, if α 0. (7)
14. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in 1 −1 x y Fig. 7: T is metrically regular around (0, 1) 1 −1 x y Fig. 8: T is not metrically regular around (0, 1)
15. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Outline 1 Introduction 2 Metric regularity property of set-valued maps 3 Kantorovich’s theorem and its generalization for GE 4 The Smale’s α-theory and its extension for inclusion
16. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in A short survey In 1948, Kantorovich published a paper studying the Newton’s method for solving equation f (x) = 0. In the book later [KA81], it was stated by the technique of majorizing function. Kantorovich’s theorem gave sufficient conditions to ensure the existence and convergence of Newton’s sequence whose assumptions focus on input data around starting point. Later, in [GT74], the authors established the optimal error bounds for Kantorovich’s theorem. Extending Kantorovich-type theorem have studied by many works for nonlinear equations f (x) = 0 (see e.g. [Wan99, FS09]) as well as inclusion 0 ∈ f (x) + F(x) (cf. [Don96, RYLW13]).
17. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Classical Kantorovich’s theorem Theorem Let f : X −→ Y be of class C2 in a ball Ω0 = B[x0 , r]. Assume that 1 there exists the linearly continuous operator Γ0 = Df (x0 )−1, 2 Γ0 (f (x0 )) β, 3 Γ0 D2f (x) K, x ∈ Ω0 . If α = 2Kβ 1, and r r0 = 21− √ 1−α α β, then there exists a solution x∗ of such that x∗ − x0 r0 r. Moreover, the Newton’s sequence satisfies the estimation xk − x∗ 1 2k−1 α2k β α , n = 0, 1, . . . Details: L. V. Kantorovich, G. P. Akilov. Analyse Fonctionnelle. ´ Editions de Moscou. Mir, 1981 (translate from Russian).
18. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Our result Let Ω be an open set in Banach space X, and f : X −→ Y is C2 on Ω. We define β(τ, x) := τd 0, f (x) + F(x) , K(τ, x, r) := τ sup z−x r D2f (z) . In addition, we use the notation (see [ACN15]) V T, ¯ x, r, s := (x, y) ∈ X × Y : x − ¯ z r, d y, T(x) < s .
19. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Theorem Let x ∈ Ω, α ∈ (0, 1] and τ > 0, r > 0, s > 0 such that: 1 B[x, r] ⊂ Ω and for Φ(·) = Df (x)(·) + F(·), V = V (Φ, x, 4r, s) one has τ > Reg V (Φ); 2 d 0, f (x) + F(x) < s; 3 2β(τ, x)K(τ, x, r) α; 4 2ηβ(τ, x) r, with η = 1− √ 1−α α . Then it has x∗ ∈ Ω with 0 ∈ f (x∗) + F(x∗) and x − x∗ 2ηβ(τ, x); there exists a sequence xk → x∗ generated by Josephy-Newton scheme satisfying x0 = x and xk − x∗ 4 √ 1−α α θ2k 1−θ2k β(τ, x), θ = 1− √ 1−α 1+ √ 1−α , if α < 1, xk − x∗ 2−k+1β(τ, x), if α = 1.
20. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Outline 1 Introduction 2 Metric regularity property of set-valued maps 3 Kantorovich’s theorem and its generalization for GE 4 The Smale’s α-theory and its extension for inclusion
21. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Short survey about α-type theory After the classical Kantorovich’s theorem, there is another important result for studying Newton method, known as the α-Smale theory, see [Ded06, Sma86]. Smale’s theorem based on the assumption of the analyticity for the function f , and needs the informations only at the reference point. There are some extensions of α-Smale’s theory for nonlinear equation on Banach spaces, and even on Riemannian manifolds, e.g., [Wan99, WHL09, DPM03].
22. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in The α-Smale theorem Assumption: f is analytic. definition β(f , x) = Df (x)−1f (x) , if Df (x) is an isomorphism, +∞, otherwise, γ(f , x) =    supk 2 Df (x)−1 Dk f (x) k! 1 k−1 , if Df (x) is an isomorphism, +∞, otherwise, α(f , x) = β(f , x)γ(f , x).
23. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in The α-Smale theorem, continue We denote ψ(t) = 2t2 − 4t + 1 and σ = ∞ k=0 1 2 2k −1 ≈ 1.63281 . . . Theorem There exists a constant α0 > 0 having following the property. For x ∈ U satisfying α(f , x) α0 , there is a zero x∗ of f such that x − x∗ σβ(f , x). In addition, the Newton’s sequence (xk ) with x0 = x is well-defined and xk − x∗ σ 1 2 2k −1 x1 − x0 . A chosen value is α0 ≈ 0.1307169 . . . , given by the equation t [ψ(t)]2 = 1 2 .
24. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Our approach Let f : X −→ Y be analytic on Ω, and F : X ⇒ Y has closed graph. We define for x ∈ Ω and τ > 0 corresponding quantities β(τ, x) = τd 0, f (x) + F(x) , γ(τ, f , x) = sup k 2 τ Dk f (x) k! 1 k−1 , α(τ, f , x) = β(τ, x)γ(τ, f , x). Remark We also fix ψ(t) = 2t2 − 4t + 1, σ = ∞ k=0 1 2 2k −1 ≈ 1.63281 . . . and α0 ≈ 0.1307169 . . .
25. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Our result Theorem Let x ∈ U and τ > 0, r, s > 0 such that 1 B[x, r] ⊂ U and τ > Reg V (Φ) for Φ(·) = Df (x)(·) + F(·), V = V (Φ, x, 4r, s); 2 d 0, f (z) + F(z) < s; 3 ηβ(τ, x) r, where η = α0+1− √ α2 0 −6α0+1 4α0 ; 4 α(τ, f , x) α0 . Then, problem 0 ∈ f (x) + F(x) admits a solution x∗ with x − x∗ ηβ(τ, x); there is a JN-seq (xk ) such that x0 = x, and xk − x∗ σ[ψ(α0 )]k 1 2 2k −1 β(τ, x).
26. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in References S. Adly, R. Cibulka, and H. V. Ngai. Newton’s method for solving inclusions using set-valued approximations. SIAM J. Optim., 25:159–184, 2015. J.-P. Dedieu. Points Fixes, Z´ eros et la M´ ethode de Newton. Springer Berlin Heidelberg, 2006. A. L. Dontchev. Local analysis of Newton–type method based on partial linearization. In Letters in Applied Mathematics, volume 32, pages 295–306. Amer. Math. Soc., 1996. J.-P. Dedieu, P. Priouret, G. Malajovich. Newton’s method on Riemannian manifolds: covariant alpha theory. IMA Journal of Numerical Analysis, 23:395–419, 2003. O. P. Ferreira, B. F. Svaiter. Kantorovich’s majorants principle for Newton’s method. Comput Optim Appl, 42:213–229, 2009.
27. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in W. B. Gragg, R. A. Tapia. Optimal error bounds for the Newton–Kantorovich theorem. SIAM J. Numer. Anal., 11:10–13, 1974. L. V. Kantorovich, G. P. Akilov. Analyse Fonctionnelle. ´ Editions de Moscou. Mir, 1981 (translate from Russian). M. H. Rashid, S. H. Yu, C. Li, S. Y. Wu. Convergence analysis of the Gauss–Newton-type method for Lipschitz-like mappings. J Optim Theory Appl, 158:216–233, 2013. S. Smale. Newton’s method estimates from data at one point. In The Merging of Disciplines: New Directions in Pure, Applied, and Computational Mathematics, pages 185–196. Springer New York, 1986. X. Wang. Convergence of Newton’s method and inverse function theorem in Banach space. Mathematics of Computation, 68:169–186, 1999. J.-H. Wang, S. Huang, C. Li. Extended Newton’s method for mappings on Riemannian manifolds with values in a cone. Taiwanese Journal of Mathematics, 13:633–656, 2009.
28. Introduction Metric regularity property of set-valued maps Kantorovich’s theorem and its generalization for GE The Smale’s α-theory and its extension for in Thanks for your attention!