ඇઢܗ࠷దԽͷجૅ – KKT condition – miruca Graduate School of Informatics, Kyoto University March 19, 2019

͜ͷεϥΠυͷ໨త ʰඇઢܗ࠷దԽͷجૅʱ(෱ౡ, 2001) ͷୈ 3 ষʹؔͯ͠ • ๏ઢਲ਼Λ༻͍ͨ࠷దੑ৚݅ʹ͍ͭͯཧղ͢Δ • ෆ౳੍ࣜ໿ΛؚΉ࠷దԽ໰୊ʹର͢Δ KKT ৚݅Λཧղ͢Δ • KKT ৚݅ͷԾఆͰ͋Δ੍໿૝ఆʹ͍ͭͯཧղ͢Δ ˞஫ҙ • ຊεϥΠυͷఆཧ౳ͷ൪߸͸ʰඇઢܗ࠷దԽͷجૅʱʹ४ͣΔ • ਤ͸ͳ͍ͷͰదٓखΛಈ͔͠ͳ͕Βཧղ͢Δ͜ͱΛਪ঑ • (ԋश) ͱॻ͍ͨ΋ͷ͸෇࿥ʹղ౴Λ෇ͨ͠

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ Today’s Topic 1. ઀ਲ਼ͱ࠷దੑ৚݅ 2. KKT ৚݅ 3. ੍໿૝ఆ 3 / 21

1. ઀ਲ਼ͱ࠷దੑ৚݅ 2. KKT ৚݅ 3. ੍໿૝ఆ

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ࠷దԽ໰୊ ࣍ͷ࠷దԽ໰୊Λߟ͑Δɿ minimize x∈Rn f(x) subject to x ∈ S. (1) ͜͜ʹɼؔ਺ f : Rn → R ͱू߹ S ⫅ Rn ͸ॴ༩Ͱ͋Δɽ • ੍໿৚݅ x ∈ S Λຬͨ͢ϕΫτϧ x Λ࣮ޮՄೳղͱ͍͍ɼ࣮ ޮՄೳղશମͷू߹Λ࣮ޮՄೳྖҬͱ͍͏ɽ • S = Rn ͷ৔߹ɼ໰୊ (1) ͸੍໿ͳ͠࠷దԽ໰୊ͱݺ͹ΕΔɽ • ؔ਺ f ͕ತؔ਺Ͱɼू߹ S ͕ತू߹Ͱ͋Δͱ͖ɼ໰୊ (1) ͸ತ ܭը໰୊ (convex programming problem) ͱݺ͹ΕΔɽ 5 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ࠷దղͷछྨ • ࣮ޮՄೳղ x ∈ S ʹରͯ͠ɼ͋Δ ε > 0 ͕ଘࡏͯ͠ f(x) ≧ f(x) (∀x ∈ S ∩ B(x, ε)) (2) ͕੒ཱ͢Δͱ͖ɼx Λ໰୊ (1) ͷہॴత࠷దղͱ͍͏ *1)ɽ • ೚ҙͷ ε > 0 ʹରͯࣜ͠ (2) ͕੒ཱ͢Δɼ͢ͳΘͪ f(x) ≧ f(x⋆) (∀x ∈ ε) (3) Ͱ͋Δͱ͖ɼx⋆ ΛେҬత࠷దղͱ͍͏ɽ ˞ େҬత࠷దղ ⇒ ہॴత࠷దղ (ٯ͸ඞͣ͠΋੒Γཱͨͳ͍) *1)த৺͕ x ∈ Rn Ͱ൒ܘ͕ r > 0 ͷٿΛ B(x, r) = {y ∈ Rn | ∥y − x∥ < r} ͱॻ͖ɼ ։ٿͱݺͿɽ 6 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ತܭը໰୊ʹ͓͚Δ࠷దղ ʮہॴత࠷దղ ⇒ େҬత࠷దղʯΛอূͰ͖Δ৔߹͕͋Δɽ ఆཧ 3.1 ࠷దԽ໰୊ (1) ʹ͓͍ͯɼf ͸ತؔ਺ɼS ͸ತू߹ͱ͢Δɽͦͷͱ ͖ɼ໰୊ (1) ͷ೚ҙͷہॴత࠷దղ͸େҬత࠷దղͰ͋Δɽ • ূ໌͸ɼہॴత࠷దղͰ͋Δ͕େҬత࠷దղͰͳ͍Α͏ͳ఺ x ∈ S ͷଘࡏੑΛԾఆͯ͠ໃ६Λಋ͚͹Α͍ɽ(ԋश) • ࠷దղશମͷू߹͕ತू߹Ͱ͋Δ͜ͱ΋ࣔ͢͜ͱ͕Ͱ͖Δɽ • ತܭը໰୊Ͱͳ͍৔߹ɼҰൠʹ͍ͭ͘΋ͷہॴత࠷దղ͕ଘࡏ ͢ΔͷͰɽେҬత࠷దղΛٻղ͢Δ͜ͱ͸ࠔ೉Ͱ͋Δɽ → ತੑΛԾఆ͠ͳ͍໰୊ʹ͓͍ͯ͸ɼہॴత࠷దղ͕ղੳͷର৅ͱ ͳΔ৔߹͕΄ͱΜͲͰ͋Δɽ 7 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ઀ਲ਼ ໰୊ (1) ʹର͢Δ࠷దੑ৚݅Λಋͨ͘ΊʹඞཁͱͳΔ֓೦Λड़΂Δɽ ఆٛ: ઀ਲ਼ (tangent cone) ఺ x ∈ S ʹऩଋ͢Δ఺ྻ {xk} ⫅ S Λߟ͑Δɽ͜ͷͱ͖ɼ͋Δඇෛ ਺ྻ {αk} Λ༻͍ͯఆٛ͞ΕΔ఺ྻ {αk(xk − x)} ͕ऩଋ͠ɼͦͷ ۃݶ͕ y ∈ Rn ͱͳΔͱ͖ɼy Λू߹ S ͷ఺ x ʹ͓͚Δ઀ϕΫτϧ (tangent vector) ͱݺͿɽ·ͨɼS ͷ఺ x ʹ͓͚Δ઀ϕΫτϧશମ ͷू߹Λ Ts(x) ͱද͠ɼू߹ S ͷ఺ x ʹ͓͚Δ઀ਲ਼ (tangent cone) ͱݺͿɽ • ઀ਲ਼ Ts(x) ͸఺ྻΛ༻͍ͯ࣍ͷΑ͏ʹදݱ͞ΕΔ: Ts(x) := { y ∈ Rn | lim k→∞ αk(xk − x) = y, lim k→∞ xk = x, xk ∈ S, αk ≧ 0 (k = 1, 2, . . .) } . 8 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ๏ઢਲ਼ ઀ਲ਼ͷۃਲ਼ʹ͍ͭͯߟ͑Δɽ ఆٛ: ๏ઢਲ਼ (normal cone) ઀ਲ਼ Ts(x) ͷۃਲ਼ Ts(x)⋆ Λ S ͷ x ʹ͓͚Δ๏ઢਲ਼ (normal cone) ͱݺͼɼNs(x) ͱද͢ɽNs(x) ʹଐ͢ΔϕΫτϧΛ x ʹ͓͚Δ S ͷ๏ઢϕΫτϧ (normal vector) ͱ͍͏ɽ • ๏ઢਲ਼͸࣍ͷΑ͏ʹදݱ͞ΕΔ: Ns(x) = {z ∈ Rn | ⟨z, y⟩ ≦ 0 (∀y ∈ Ts(x))} (4) • ಛʹɼू߹ S ͕ತू߹Ͱ͋Δͱ͖͸࣍ͷΑ͏ʹදݱ͞ΕΔ: Ns(x) = {z ∈ Rn | ⟨z, x − x⟩ ≦ 0 (∀x ∈ S)} (5) • ๏ઢਲ਼ Ns(x) ͸ۭͰͳ͍ดತਲ਼Ͱ͋Δ *2)ɽ *2)೚ҙͷਲ਼ C ʹର͢Δۃਲ਼ C⋆ ͸ดತਲ਼Ͱ͋ΔͨΊ (ఆཧ 2.12)ɽ 9 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ࠷దੑ৚݅ ๏ઢਲ਼Λ༻͍Δ͜ͱʹΑΓɼ໰୊ (1) ʹର͢Δ࠷΋جຊతͳ࠷దੑ ৚݅Λ༩͑Δ͜ͱ͕Ͱ͖Δɽ ఆཧ 3.3 ؔ਺ f : Rn → R ͸఺ x ∈ S ʹ͓͍ͯඍ෼Մೳͱ͢Δɽͦͷͱ͖ɼ ఺ x ͕໰୊ (1) ͷہॴత࠷దղͳΒ͹࣍ͷؔ܎͕੒Γཱͭɿ − ∇f(x) ∈ Ns(x). (6) • ࣜ (6) Λຬͨ͢఺͸໰୊ (1) ͷఀཹ఺ (stationary point) ͱݺ ͹ΕΔɽ • ࣜ (6) ͸఺ x ͕໰୊ (1) ͷہॴత࠷దղͰ͋ΔͨΊͷඞཁ৚݅ Ͱ͋Δ͕े෼৚݅Ͱ͸ͳ͍ɽ • ತܭը໰୊ͷ৔߹ɼࣜ (6) ͕࠷దੑͷඞཁे෼৚݅ͱͳΔɽ 10 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ತܭը໰୊ʹ͓͚Δ࠷దੑ৚݅ ఆཧ 3.4 S ⫅ Rn ͸ۭͰͳ͍ತू߹ɼf : Rn → R ͸఺ x ∈ S ʹ͓͍ͯඍ෼ Մೳͳತؔ਺ͱ͢Δɽ͜ͷͱ͖ɼࣜ (6) ͸ x ͕໰୊ (1) ͷେҬత࠷ దղͰ͋ΔͨΊͷඞཁे෼৚݅Ͱ͋Δɽ ূ໌ ඞཁੑ͸໌Β͔ͳͷͰे෼ੑ͚ͩࣔͤ͹Α͍ɽ(ԋश) 11 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ತܭը໰୊ʹ͓͚Δ࠷దੑ৚݅ ఆཧ 3.4 ΑΓ࣍ͷܥ͕੒Γཱͭɽ ܥ 3.1 ू߹ S ⫅ Rn ͷ಺෦͸ۭͰͳ͘ɼؔ਺ f : Rn → R ͸఺ x ∈ int S*3) ʹ͓͍ͯඍ෼Մೳͱ͢Δɽͦͷͱ͖ɼx ͕໰୊ (1) ͷہ ॴత࠷దղͳΒ͹ ∇f(x) = 0 ͕੒ཱ͢Δɽ͞Βʹɼf ͕ತؔ਺ɼS ͕ತू߹ͳΒ͹ɼ∇f(x) = 0 ͸ x ͕໰୊ (1) ͷେҬత࠷దղͰ͋ ΔͨΊͷඞཁे෼৚݅Ͱ͋Δɽ ূ໌ ɹ x ∈ int S Ͱ͋Δͱ͖ɼTs(x) = Rn Ͱ͋Δ͔Βɼ Ns(x) = {0} ͱͳΔɽΑͬͯɼࣜ (6) ͸ ∇f(x) = 0 ʹؼண͞ΕΔɽ *3)ू߹ S ⫅ Rn ͱ఺ x ∈ Rn ʹରͯ͠ɼB(x, r) ⫅ S ͱͳΔΑ͏ͳ r > 0 ͕ଘࡏ͢Δͱ ͖ɼx Λ S ͷ಺఺ͱ͍͍ɼS ͷ಺఺શମͷू߹Λ S ͷ಺෦ͱ͍͍ɼint S Ͱද͢ɽ 12 / 21

1. ઀ਲ਼ͱ࠷దੑ৚݅ 2. KKT ৚݅ 3. ੍໿૝ఆ

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ෆ౳ࣜΛؚΉ࠷దԽ໰୊ ࣍ͷ࠷దԽ໰୊Λߟ͑Δɿ minimize x∈Rn f(x) subject to gi(x) ≦ 0 (i = 1, . . . , m). (7) ͜͜Ͱɼؔ਺ f ͓Αͼ gi (i = 1, . . . , m) ͸ඍ෼ՄೳͰ͋Δͱ͢Δɽ • ໰୊ (7) ͷ੍໿৚݅͸ɼ໰୊ (1) ͷ࣮ޮՄೳྖҬ S ͕ S = {x ∈ Rn | gi(x) ≦ 0 (i = 1, . . . , m)} (8) ͱද͞ΕΔ৔߹ʹଞͳΒͳ͍ɽ • ໰୊ (7) ͷ࣮ޮՄೳղ x ʹ͓͍ͯɼgi(x) = 0 ͕੒Γ੍ཱͭ໿ ৚݅Λ༗ޮ੍໿৚݅ͱݺͼɼͦͷఴࣈू߹ΛҎԼͰද͢ɿ I(x) = {i ∈ N | gi(x) = 0} ⫅ {1, 2, . . . , m}. 14 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ઢܗԽਲ਼ ࣮ޮՄೳྖҬ S ͕ࣜ (8) Ͱ༩͑ΒΕΔͱ͖ɼ઀ਲ਼ʹมΘΔ֓೦ͱ͠ ͯઢܗԽਲ਼ͱݺ͹ΕΔਲ਼Λߟ͑Δ͜ͱ͕Ͱ͖Δɽ ఆٛ: ઢܗԽਲ਼ (linearizing cone) ू߹ S ͕ࣜ (8) Ͱ༩͑ΒΕ͍ͯΔͱ͖ɼ఺ x ∈ S ʹ͓͚Δ༗ޮ੍໿ ৚݅ʹରԠ͢Δ੍໿ؔ਺ͷޯ഑ ∇gi(x) (i ∈ I(x)) ͱ 90◦ Ҏ্ͷ֯ Λͳ͢ϕΫτϧશମͷू߹ΛઢܗԽਲ਼ͱݺͼɼCs(x) Ͱද͢ɽ • ઢܗԽਲ਼ Cs(x) ͸࣍ͷΑ͏ʹද͞ΕΔ: Cs(x) := {y ∈ Rn | ⟨∇gi(x), y⟩ ≦ 0 (∀i ∈ I(x))} (9) • ઀ਲ਼ Ts(x) ͸ू߹ S ͔Β௚઀ఆٛ͞ΕΔͷʹର͠ɼઢܗԽਲ਼ Cs(x) ͸ؔ਺ gi ʹґଘͯ͠ఆ·Δ͜ͱʹ஫ҙ͢Δɽ • ઀ਲ਼ͱઢܗԽਲ਼͸ඞͣ͠΋Ұக͢Δͱ͸ݶΒͳ͍͕ɼแؚؔ܎ Ts(x) ⫅ Cs(x) ͸ৗʹ੒ཱ͢Δɽ(ิ୊ 3.3) 15 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ Lagrange ؔ਺ Lagrange ؔ਺ͱݺ͹ΕΔؔ਺Λఆٛ͢Δɽ ఆٛ: Lagrange ؔ਺ (Lagragian) ໰୊ (7) ʹରͯ͠ɼ࣍ࣜͰఆٛ͞ΕΔؔ਺ L0 : Rn+m → [−∞, ∞] Λ Lagrange ؔ਺ͱ͍͏ɽ L0(x, λ) =      f(x) + m ∑ i=1 λigi(x) (λ ≧ 0) −∞ (λ ≧̸ 0) (10) ͜͜ʹɼλ = (λ1, . . . , λm)⊤ ∈ Rm Λ Lagrange ৐਺ͱݺͿɽ • ࣜ (10) ʹ͓͍ͯɼλ ≧̸ 0 ͷͱ͖ L0(x, λ) = −∞ ͱఆٛͨ͠ ͷ͸ɼ૒ର໰୊Λఆٛ͢Δࡍʹ౎߹͕Α͍ͨΊͰ͋Δɽ • Lagrange ؔ਺ʹΑͬͯ໰୊ (7) ʹର͢Δ࠷దੑͷඞཁ৚݅Λ ༩͑Δ͜ͱ͕Ͱ͖Δɽ 16 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ KKT ৚݅: ࠷దੑͷඞཁ৚݅ ໰୊ (7) ʹର͢Δ࠷దੑͷඞཁ৚݅ʹ͍ͭͯड़΂Δɽ ఆཧ 3.5 ఺ x Λ໰୊ (7) ͷہॴత࠷దղͱ͢Δɽͦͷͱ͖ɼแؚؔ܎ Cs(x) ⫅ co Ts(x) ͕੒ΓཱͭͳΒ͹ɼ࣍ࣜ: ∇xL0(x, λ) = ∇f(x) + m ∑ i=1 λi∇gi(x) = 0 λi ≧ 0, gi(x) ≧ 0, λigi(x) = 0 (i = 1, . . . , m) (11) Λຬ଍͢Δ Lagrange ৐਺ λ ∈ Rm ͕ଘࡏ͢Δɽ • ࣜ (11) ͸Ұൠʹ KKT ৚݅ (KKT condition) ͱݺ͹ΕΔɽ • ఆཧ 3.5 ͸఺ x ͕໰୊ (7) ͷہॴత࠷దղͰ͋ΔͨΊͷे෼৚ ݅Ͱ͋Δ͜ͱ͸อূ͍ͯ͠ͳ͍ɽ 17 / 21

઀ਲ਼ͱ࠷దੑ৚݅ KKT ৚݅ ੍໿૝ఆ ࠷దੑͷे෼৚݅ ತܭը໰୊ʹ͓͍ͯ͸ɼKKT ৚͕݅࠷దੑͷे෼৚݅ʹ΋ͳΔɽ ఆཧ 3.6 ໰୊ (7) ʹ͓͍ͯɼ໨తؔ਺ f ͱ੍໿ؔ਺ gi ͸ඍ෼Մೳͳತؔ਺ͱ ͢Δɽͦͷͱ͖ɼ͋Δ x ∈ Rn ͱ λ ͕ࣜ (11) Λຬ଍͢ΔͳΒ͹ɼx ͸໰୊ (7) ͷେҬత࠷దղͰ͋Δɽ • ఆཧ 3.5 ͱఆཧ 3.6 ΑΓɼತܭը໰୊ͷͱ͖͸ KKT ৚͕݅େ Ҭత࠷దੑͷඞཁे෼৚݅ͱͳΔɽͭ·Γɼ ∃ (x, λ) s.t. ࣜ (11) ⇔ x ͸໰୊ (7) ͷେҬత࠷దղ • େҬత࠷దղͰ͋Δ͜ͱΛอূͰ͖Δͷ͸ɼತܭը໰୊ʹ͓͍ ͯʮہॴత࠷దղͳΒ͹େҬత࠷దղʯ͕੒ΓཱͭͨΊͰ͋ Δɽ(ఆཧ 3.1) • ূ໌͸Ұ౓ܦݧ͓ͯ͘͠ͱΑ͍ɽ(ԋश) 18 / 21

1. ઀ਲ਼ͱ࠷దੑ৚݅ 2. KKT ৚݅ 3. ੍໿૝ఆ

੍໿૝ఆ ໰୊ (7) ʹର͢Δ୅දతͳ੍໿૝ఆͱͯ͠ҎԼͷ΋ͷ͕͋Δɽ ओͳ੍໿૝ఆ • Ұ࣍ಠ੍ཱ໿૝ఆ: ϕΫτϧ ∇gi(x) (∀i ∈ I(x)) ͸Ұ࣍ಠཱͰ ͋Δɽ • Slater ੍໿૝ఆ: ؔ਺ gi (∀i ∈ I(x)) ͸ತؔ਺Ͱ͋Γɼ gi(x) < 0 (i = 1, . . . , m) ͳΔ఺ x0 ͕ଘࡏ͢Δɽ • Cottle ੍໿૝ఆ: ⟨∇g(x), y⟩ < 0 (∀i ∈ I(x)) Λຬͨ͢ϕΫτ ϧ y ∈ Rn ͕ଘࡏ͢Δɽ • Abadie ੍໿૝ఆ: Cs(x) ⫅ Ts(x) • Guignard ੍໿૝ఆ: Cs(x) ⫅ co Ts(x) • Guignard ੍໿૝ఆ͸ఆཧ 3.5 ͰԾఆ੍ͨ͠໿૝ఆͰ͋Δɽ

੍໿૝ఆͷ૬ޓؔ܎ ੍֤໿૝ఆʹ͍ͭͯ࣍ͷؔ܎͕੒Γཱͭɽ ఆཧ • Ұ࣍ಠ੍ཱ໿૝ఆ ⇒ Cottle ੍໿૝ఆ • Slater ੍໿૝ఆ ⇒ Cottle ੍໿૝ఆ • Cottle ੍໿૝ఆ ⇒ Abadie ੍໿૝ఆ • Abadie ੍໿૝ఆ ⇒ Guignard ੍໿૝ఆ • 5 छྨͷ੍໿૝ఆͷ͏ͪ Guignard ੍໿૝ఆ͕࠷΋ऑ͍ԾఆͰ ͋Δ͕ɼ༩͑ΒΕͨ࠷దԽ໰୊ʹରͯ͠ Cs(x) ⫅ co Ts(x) Ͱ ͋Δ͜ͱΛ൑ఆ͢Δ͜ͱ͸ࠔ೉Ͱ͋Γɼ࣮༻తͰ͸ͳ͍ɽ • Ұ੍࣍໿૝ఆɼSlater ੍໿૝ఆɼCottle ੍໿૝ఆ͸ݕূ͕ൺֱ త༰қͰ͋ΔͨΊɼ࣮ࡍʹΑ͘࢖ΘΕΔɽ

4. ෇࿥

ఆཧ 3.1 ࠷దԽ໰୊ (1) ʹ͓͍ͯɼf ͸ತؔ਺ɼS ͸ತू߹ͱ͢Δɽͦͷͱ ͖ɼ໰୊ (1) ͷ೚ҙͷہॴత࠷దղ͸େҬత࠷దղͰ͋Δɽ ূ໌ ہॴత࠷దԽͰ͋Δ͕େҬత࠷దղͰͳ͍Α͏ͳ఺ x ∈ S ͷ ଘࡏΛԾఆ͢Δɽ͢ͳΘͪɼf(y) < f(x) Λຬͨ͢Α͏ͳ఺ y ∈ S ͕ଘࡏ͢Δɽ͍·ɼू߹ S ͸ತؔ਺ΑΓ೚ҙͷ α ∈ (0, 1) ʹର͠ ͯɼ(1 − α)x + αy ∈ S Ͱ͋Δɽ·ͨɼؔ਺ f ͸ತؔ਺ΑΓ f((1 − α)x + αy) ≦ (1 − α)f(x) + αf(y) < (1 − α)f(x) + αf(x) = f(x) ͕੒Γཱͭɽ্ࣜͰ α → 0 ͷۃݶΛߟ͑Δͱɼx ͷ೚ҙͷۙ๣ͷத ʹ x ΑΓ΋ਅʹখ͍͞໨తؔ਺஋Λ΋࣮ͭޮՄೳղ͕ଘࡏ͢Δ͜ͱ ͕ݴ͑Δɽ͜Ε͸ɼx ͕ہॴత࠷దղͰ͋Δ͜ͱʹ൓͢Δɽ(ূ ໌ऴ)

ఆཧ 3.4 S ⫅ Rn ͸ۭͰͳ͍ತू߹ɼf : Rn → R ͸఺ x ∈ S ʹ͓͍ͯඍ෼ Մೳͳತؔ਺ͱ͢Δɽͦͷͱ͖ɼࣜ (6) ͸ x ͕໰୊ (1) ͷେҬత࠷ దղͰ͋ΔͨΊͷඞཁे෼৚݅Ͱ͋Δɽ ূ໌ ඞཁੑ͸ఆཧ 3.3 ΑΓ໌Β͔ͳͷͰे෼ੑͷΈࣔ͢ɽ͍·ɼ −∇f(x) ∈ Ns(x) ΑΓɼ೚ҙͷ x ∈ S ʹରͯ͠ ⟨−∇f(x), x − x⟩ ≦ 0 ⇔ ⟨∇f(x), x − x⟩ ≧ 0 (12) ͕੒Γཱͭ *4)ɽ·ͨɼ೚ҙͷ x ∈ S ʹରͯ͠ f(x) ≧ (f ͷತੑ) f(x) + ⟨∇f(x), x − x⟩ ≧ (ࣜ (12)) f(x) ͕੒ΓཱͭɽΏ͑ʹɼx ͸໰୊ (1) ͷେҬత࠷దղͰ͋Δɽ(ূ໌ऴ) *4)ू߹ S ͕ತू߹Ͱ͋Δͱ͖๏ઢਲ਼͸ࣜ (5) Ͱ༩͑ΒΕΔ͜ͱΛ༻͍ͨɽ

ఆཧ 3.6 ໰୊ (7) ʹ͓͍ͯɼ໨తؔ਺ f ͱ੍໿ؔ਺ gi ͸ඍ෼Մೳͳತؔ਺ͱ ͢Δɽͦͷͱ͖ɼ͋Δ x ∈ Rn ͱ λ ͕ࣜ (11) Λຬ଍͢ΔͳΒ͹ɼx ͸໰୊ (7) ͷେҬత࠷దղͰ͋Δɽ ূ໌ λ Λݻఆͯ͠ɼؔ਺ ℓ : Rn → R Λ࣍ࣜͰఆٛ͢Δɿ ℓ(x) = f(x) + m ∑ i=1 λigi(x). ͍·ɼf, gi ͸ͱ΋ʹತؔ਺Ͱ λ ≧ 0 Ͱ͋Δ͔Β ℓ ΋ತؔ਺Ͱ͋ Δ *5)ɽ৚݅ΑΓɼx ∈ Rn ͱ λ ͸ࣜ (11) Λຬͨ͢ͷͰ ∇f(x) + m ∑ i=1 λi∇gi(x) = 0

͕੒Γཱͭɽఆཧ 3.4 ΑΓ ℓ ͸ x ʹ͓͍ͯେҬతʹ࠷খͱͳΔɽ Αͬͯɼ೚ҙͷ x ∈ Rn ʹରͯ͠ɼℓ(x) ≦ ℓ(x), i.e., f(x) + m ∑ i=1 λigi(x) =0 ≦ f(x) + m ∑ i=1 λigi(x) ͕੒Γཱͭɽ৚݅ΑΓ λigi(x) = 0 (i = 1, . . . , m) ͔ͭ λ ≧ 0 Ͱ͋ Δ͔Βɼgi(x) ≦ 0 (i = 1, . . . , m) Λຬͨ͢೚ҙͷ x ʹରͯ͠ *6) f(x) + 0 ≦ f(x) + m ∑ i=1 λigi(x) ≦0 ≦ f(x) ͕੒Γཱͭɽ͕ͨͬͯ͠ɼx ͸େҬత࠷దղͰ͋Δɽ(ূ໌ऴ) *5)ʰඇઢܗ࠷దԽͷجૅʱఆཧ 2.26 Λࢀরɽ *6)͢ͳΘͪɼ໰୊ (7) ͷ೚ҙͷ࣮ޮՄೳղʹରͯ͠