ॳ౳֬཰࿦ͷجૅ ϕΠζ౷ܭͷཧ࿦ͱํ๏ษڧձ

֬཰෼෍ͱ֬཰ม਺

֬཰෼෍ ϢʔΫϦουۭؒ ͷݩ ͷؔ਺ ͕ ℝN x = (x1 , …, xN ) q(x) ≥ 0 ∫ q(x)dx ≡ ∫ dx1 ∫ dx2 … ∫ dxN q(x1 , x2 , ⋯, xN ) = 1 Λຬͨ͢ͱ͖ Λ֬཰෼෍͋Δ͍͸֬཰ີ౓ؔ਺ͱ͍͏ɻ q(x) ू߹ ʹ͍ͭͯɺ ͷݩͰͷू߹ ͷ֬཰͸ A ⊂ ℝN q(x) A Q(A) = ∫ A q(x)dx ͜ͷͱ͖ɺؔ਺ ΋֬཰෼෍ͱ͍͏ɻ Q( ⋅ )

֬཰ม਺ ϢʔΫϦουۭؒ ͷ্ʹϥϯμϜʹ஋ΛऔΔม਺ Λ ʮ ʹ஋ΛऔΔ֬཰ม਺ʯͱ͍͏ɻ ℝN X ℝN ʮ ͱͳΔ֬཰ʯ͕ Ͱ͋Δͱ͖ ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯ͋Δ͍͸ ʮ֬཰ม਺ ͷ֬཰෼෍͸ ʹै͏ʯ͋Δ͍͸ ʮ֬཰ม਺ ͷ֬཰෼෍͸ Ͱ͋Δʯͱ͍͏ɻ X ∈ A Q(A) X q(x) X q(x) X Q

۩ମྫਅͷ෼෍ αϯϓϧ ͕͋Δ֬཰෼෍ ʹಠཱʹै͏ ֬཰ม਺ͷ࣮ݱ஋ʢ؍ଌ஋ʣͩͱ͢Δɻ A = xn = {x1 , …, xn } ⊂ ℝN q(x) ͢ͳΘͪ Λ ্ͷ෼෍ xn (ℝN)n q(xn) = n ∏ i=1 q(xi ) = q(x1 )q(x2 )⋯q(xn ) Λ࣋ͭ֬཰ม਺ ͷ࣮ݱ஋Ͱ͋Δͱߟ͑Δɻ ͜ͷͱ͖֬཰෼෍ Λਅͷ෼෍ͱݺͿɻ Xn = (X1 , X2 , …, Xn ) q(x)

ฏۉͱ෼ࢄ

ฏۉͱ෼ࢄ ʹ஋ΛͱΔ֬཰ม਺ ͷ֬཰෼෍Λ ͱ͢Δɻ ℝN X q(x) [f(X)] ≡ ∫ f(x)q(x)dx [f(X)] ≡ [(f(X) − [f(X)])(f(X) − [f(X)])T] = [f(X)f(X)T] − [f(X)][f(X)T] ͱఆٛ͢Δɻ ͕༩͑ΒΕͨͱ͖ɺ֬཰ม਺ ͷฏۉΛ f : ℝN → ℝM f(X) ·ͨ෼ࢄڞ෼ࢄΛ ͱఆٛ͢Δɻ֬཰ม਺Λ໌ه͍ͨ͠ͱ͖͸ ͱॻ͘ɻ X [f(X)]

۩ମྫαϯϓϧͷฏۉ஋ αϯϓϧ Λද֬͢཰ม਺Λ ͱ͢Δɻ ͦͷؔ਺ ͕༩͑ΒΕͨͱ͖ɺͦͷฏۉ஋ΛऔΔૢ࡞ Λ xn = {x1 , …, xn } Xn = (X1 , X2 , …, Xn ) f(Xn) [ ⋅ ] ͱදه͢Δɻ ͜ͷฏۉ஋ ΛʮαϯϓϧͷݱΕํʹର͢Δฏۉ஋ʯͱݺͿɻ [ ⋅ ] [f(XN)] = ∫ ∫ ⋯ ∫ f(x1 , …, xn ) n ∏ i=1 q(xi )dxi

۩ମྫਅͷ෼෍ͷฏۉ αϯϓϧͷ֬཰ม਺Λ Λ༻͍ͯɺ ਅͷ෼෍ ͷਪଌΛߦͬͨޙɺਅͷ෼෍ͷ֬཰ม਺ Λൃੜͤͯ͞ ਪଌ݁ՌͷΑ͞ΛධՁ͍ͨ͠ɻ ͜ͷ֬཰ม਺ ͷؔ਺ ʹ͍ͭͯͷฏۉΛ Xn = (X1 , X2 , …, Xn ) q(x) X X f(X) ͱදه͢Δɻ [f(X)]X = ∫ f(x)q(x)dx

X X−1 ֬཰ۭؒ(Ω = ℝM, ℬ, p) w ∈ Ω ٯ૾X−1(A) ֬཰ີ౓ؔ਺ ֬཰෼෍ q(x) = p(X−1(x)) Մଌۭؒ(Ω′ = ℝN, ℬ′ ) A ∈ ℬ′ X(w) = X x ∈ Ω′ ֬཰෼෍Q(A) = ∫ A q(x)dx f(x) ฏۉ[f(X)] ≡ ∫ f(x)q(x)dx = ∫ f(x)p(X−1(x))dx = ∫ p(w)X(w)dw = ∫ pXdw ֬཰ม਺ ֬཰ม਺ͱ֬཰෼෍ɺฏۉͷؔ܎ ֬཰ۭؒ(Ω′ = ℝN, ℬ′ , q)

ಉ࣌෼෍ͱ৚݅෇͖֬཰

ಉ࣌෼෍ͱ৚݅෇͖෼෍ ͭͷ֬཰ม਺ ͱ ͕͋Δͱ͖ɺͦͷ૊ ͷ֬཰෼෍͕ Ͱ͋Δͱ͖ɺ Λಉ࣌֬཰෼෍ͱ͍͏ɻ X Y (X, Y) p(x, y) p(x, y) ·ͨ֬཰ม਺ ͕༩͑ΒΕͨͱ͖ͷ ͷ৚݅෇͖֬཰෼෍Λ࣍ͷΑ͏ ʹఆٛ͢Δɻ X Y p(y|x) = p(x, y) p(x) पล֬཰෼෍͸࣍ͷΑ͏ʹఆٛ͢Δɻ p(x) = ∫ p(x, y)dy p(y) = ∫ p(x, y)dx

ճؼؔ਺ ֬཰ม਺ ͷ֬཰෼෍ ʹ͍ͭͯߟ͑Δɻ ͷͱ͖ͷ ͷฏۉ஋Λ (X, Y) p(X, Y) X = x Y ͱॻ͘ɻ͜ͷؔ਺Λ ͔Β ͷճؼؔ਺ ৚݅෇͖ظ଴஋ ͱ͍͏ɻ x y [Y|x] = ∫ yp(y|x)dy ؔ਺Λ ͕༩͑ΒΕͨͱ͖ͦͷೋ৐ޡࠩΛද͢൚ؔ਺Λ y = f(x) [(Y − f(X))2] = ∫ ∫ (y − f(x))2p(y, x)dxdy ͱॻ͘ͱ͜Ε͸ ͷͱ͖ʹ࠷খʹͳΔɻ f(x) = [Y|x]

ΧϧόοΫɾϥΠϒϥ৘ใྔ

ΧϧόοΫɾϥΠϒϥ৘ใྔ ্ʹೋͭͷ֬཰෼෍ ͕͋Δͱ͖ ℝN q(x), p(x) D(p∥q) = ∫ q(x)log q(x) p(x) dx ͷ͜ͱΛΧϧόοΫɾϥΠϒϥ৘ใྔ͋Δ͍͸૬ରΤϯτϩϐʔͱݺͿ ΧϧόοΫɾϥΠϒϥ৘ใྔ͸͕࣍੒Γཱͭɻ ʹ͍ͭͯ Ͱ͋Δɻ ͱͳΔͷ͸ ͷͱ͖ʹݶΔɻ ∀q(x), p(x) D(q∥p) ≥ 0 D(q∥p) = 0 q(x) = p(x)

ΧϧόοΫɾϥΠϒϥ৘ใྔ ূ໌ ͱ͓͘ͱɺ Ͱ͋Γɺ F(t) = 0 ⇔ t = 0 F(t) = t + et − 1 (−∞ < t < ∞) ΑΓ Ͱ͋Δ͔Β͕ࣔ͞Εͨɻ ∫ q(x)dx = 1 ∫ p(x)dx = 1 ∫ log q(x) p(x) dx = 0 ·ͨɺ ͷͱ͖ɺ Ͱ Ͱ͋Δ͜ͱΛ༻͍ͯ q(x) ≈ p(x) t ≈ 0 F′ ′ (t) ≃ t2/e D(p∥q) ≃ ∫ q(x)(log q(x) − log p(x))2dx ͕੒Γཱͭɻ

ۃݶఆཧ

֬཰ม਺ͷऩଋ

֬཰ऩଋ ֬཰ม਺ ͕ఆ਺ ʹ֬཰ऩଋ͢Δͱ͸ ʹର͠ɺ ʹ͓͍ͯ {Xn }n∈ℕ c ∀ϵ, ∀δ > 0 ∃N ∈ ℕ n > N ⇒ P(∥Xn − c∥ > ϵ) < δ ⇔ P(∥Xn − c∥ < ϵ) = 1 ͱͳΔ͜ͱͰ͋Δɻ ͜Ε͸େ਺ͷऑ๏ଇʹରԠ͍ͯ͠Δɻ Xn c ϵ ඪຊ͕े෼ʹେ͖͍ͱ͖ɺඪຊฏۉ͸฼ฏۉʹऩଋ͢Δ

๏ଇ ෼෍ ऩଋ ֬཰ม਺ͷྻ ͕֬཰ม਺ ʹ๏ଇ ෼෍ ऩଋ͢Δͱ͸ ͷ֬཰෼෍͕ Ͱ ͷ֬཰෼෍͕ Ͱ͋Δͱ͖ɺ ೚ҙͷ༗ք͔ͭ࿈ଓͳؔ਺ ʹରͯ͠ {Xn }n∈ℕ X Xn qn (x) X q(x) F(x) lim n→∞ ∫ F(x)qn (x)dx = ∫ F(x)q(x)dx ⇔ lim n→∞ [F(Xn )] = [F(X)] ͕੒Γཱͭ͜ͱͰ͋Δɻ͜Ε͸த৺ۃݶఆཧʹରԠ͍ͯ͠Δɻ ඪຊ͕े෼ʹେ͖͍ͱ͖ɺ฼ूஂͷ෼෍ʹؔΘΒͣඪຊฏۉͱ฼ฏۉͷࠩ͸ਖ਼ن෼෍ʹै͏

ܦݧաఔ

ϢʔΫϦουۭؒʹ͓͚ΔίϯύΫτੑ ϢʔΫϦουۭؒ ͷ෦෼ू߹ ͕։ू߹ͷ଒ ʹ ͍ͭͯ ͳΒ͹ɺͦͷ༗ݶݸͷ։ू߹ Ͱ ℝN W = {O}λ∈Λ W ⊂ ⋃ λ∈Λ Oλ O1 , …, On ∈ ͱͳΔ΋ͷ͕͋Δͱ͖ɺ ͸ίϯύΫτͰ͋Δͱ͍͏ W ⊂ O1 ∪ … ∪ On W O1 , …, On ∈ W

ؔ਺্ۭؒͷେ਺ͷ๏ଇ ϢʔΫϦουۭؒ ʹ஋ΛऔΔ ͕֬཰ม਺ ͱ ಉ֬͡཰෼෍ʹै͏ͱ͢Δɻ ύϥϝʔλͷू߹ ΛίϯύΫτͱ͢Δɻ ℝN X1 , X2 , …, Xn X w ∈ W ∈ ℝN f(x, w) : ℝN → ℝ1 X [ sup w∈W |f(X, w)|] < ∞, X [ sup w∈W |∇w f(X, w)|] < ∞ ৚݅ ͕੒ΓཱͭͱԾఆ͢Δɻ͜ͷͱ͖ɺ ʹ͍ͭͯ ∀ϵ > 0 P( sup w∈W 1 n n ∑ i=1 f(Xi , w) − X [f(X, w)] < ϵ) = 1 ͜ͷ͜ͱΛؔ਺্ۭؒͷେ਺ͷ๏ଇͱ͍͏

ਖ਼ن֬཰աఔ ू߹ ্ͷؔ਺Ͱ֬཰తʹมಈ͢Δ΋ͷ ͕ɺ ฏۉؔ਺ ͱ૬ؔؔ਺ Λ࣋ͭਖ਼ن֬཰աఔͰ͋Δͱ͸ɺ ֤ ͝ͱʹ ͕ਖ਼ن෼෍ʹै͏֬཰ม਺Ͱ͋Γɺ W ξ(w) m(w) ρ(w, w′ ) w ξ(w) m(w) = ξ [ξ(w)], ρ(w, w′ ) = ξ [ξ(w)ξ(w′ )] ͕੒Γཱͭ͜ͱͰ͋Δɻ͜͜Ͱ ͸ɺ֬཰աఔ ʹ͍ͭͯͷฏۉΛ ද͍ͯ͠ΔɻίϯύΫτू߹্Ͱͷਖ਼ن֬཰աఔ͸ɺ ξ [ ⋅ ] ξ ฏۉؔ਺ͱ૬ؔؔ਺͕ܾ·ΔͱҰҙʹఆ·Δ͜ͱ͕஌ΒΕ͍ͯΔɻ

ܦݧաఔ ͭ͗ʹ X[ sup w∈W |f(X, w) − X [f(X, w)]|α ] < ∞ X[ sup w∈W |∇w (f(X, w) − X [f(X, w)])|α ] < ∞ ͕ Ͱ੒ΓཱͭͱԾఆ͢Δɻ α = 2 Yn (w) = 1 n n ∑ i=1 (f(Xi , w) − X [f(X, w)]) ͜ͷ֬཰աఔ Λܦݧաఔͱ͍͏ɻ Yn (w)

ܦݧաఔ ֬཰աఔ ܦݧաఔ ͸ฏۉ͕ Ͱ૬ؔؔ਺͕ Yn (w) 0 ͷਖ਼ن֬཰աఔ ʹ๏ଇऩଋ͢Δɻ Y(w) ρ(w, w′ ) = X [f(X, w)f(X, w′ )] − X [f(X, w)]X [f(X, w′ )]

֬཰աఔͷ๏ଇऩଋ ֬཰աఔ ܦݧաఔ ͕֬཰աఔ ʹ๏ଇऩଋ͢Δͱ͸ɺ ༗ք࿈ଓͳ൚ؔ਺ ʹ͍ͭͯ Yn (w) Y(w) F( ⋅ ) ͕੒Γཱͭͱ͍͏͜ͱͰ͋Δɻͳ͓ɺ൚ؔ਺ ͕࿈ଓͰ͋Δͱ͸ F( ⋅ ) lim n→∞ [F(Yn )] = Y [F(Y)] lim n→∞ sup w∈W |fn (w) − f(w)| → 0 ⇒ lim n→∞ F(fn ) = F(f ) ͕੒Γཱͭ͜ͱͰ͋Δɻ ͜ͷΑ͏ͳܗͷఆཧΛؔ਺্ۭؒͷத৺ۃݶఆཧͱ͍͏ɻ

ࢀߟࢿྉ w ֬཰࿦ೖ໳ ౉ล੅෉ w ܦݧաఔͱ͸ ౉ล੅෉ w ϕΠζ౷ܭͷཧ࿦ͱํ๏ ౉ล੅෉ w ଌ౓ɾ֬཰ɾϧϕʔάੵ෼ ݪܒհ