Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
初等確率論の基礎
Search
Koga Kobayashi
August 17, 2020
Research
1
170
初等確率論の基礎
「ベイズ統計の理論と方法」勉強会の資料
Koga Kobayashi
August 17, 2020
Tweet
Share
More Decks by Koga Kobayashi
See All by Koga Kobayashi
第13回 Data-Centric AI勉強会, LLMのファインチューニングデータ
kajyuuen
4
1.6k
基礎数学の公式
kajyuuen
1
150
Deep Markov Model を数式で追う (+ Pyroでの追試)
kajyuuen
0
900
Fundamentals of Music Processing (Chapter 5)
kajyuuen
0
85
完全なアノテーションが得られない状況下での固有表現抽出
kajyuuen
3
3.5k
SecHack365 北海道会 LT
kajyuuen
0
510
専門用語抽出手法の研究と 抽出アプリケーションの開発
kajyuuen
1
1.3k
Other Decks in Research
See All in Research
経済学と機械学習:因果推論と密度比推定を中心に
masakat0
0
140
NLP Colloquium
junokim
1
190
20250502_ABEJA_論文読み会_スライド
flatton
0
190
RHO-1: Not All Tokens Are What You Need
sansan_randd
1
170
電通総研の生成AI・エージェントの取り組みエンジニアリング業務向けAI活用事例紹介
isidaitc
1
880
AIグラフィックデザインの進化:断片から統合(One Piece)へ / From Fragment to One Piece: A Survey on AI-Driven Graphic Design
shunk031
0
230
最適決定木を用いた処方的価格最適化
mickey_kubo
4
1.8k
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
720
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
tomoyanonymous
0
140
投資戦略202508
pw
0
370
Delta Airlines® Customer Care in the U.S.: How to Reach Them Now
bookingcomcustomersupportusa
0
110
SSII2025 [TS1] 光学・物理原理に基づく深層画像生成
ssii
PRO
4
4.1k
Featured
See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
The Pragmatic Product Professional
lauravandoore
36
6.8k
How STYLIGHT went responsive
nonsquared
100
5.7k
Designing for humans not robots
tammielis
253
25k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Fireside Chat
paigeccino
39
3.6k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Transcript
ॳ֬ͷجૅ ϕΠζ౷ܭͷཧͱํ๏ษڧձ
֬ͱ֬ม
֬ ϢʔΫϦουۭؒ ͷݩ ͷؔ ͕ ℝN x = (x1 ,
…, xN ) q(x) ≥ 0 ∫ q(x)dx ≡ ∫ dx1 ∫ dx2 … ∫ dxN q(x1 , x2 , ⋯, xN ) = 1 Λຬͨ͢ͱ͖ Λ֬͋Δ͍֬ີؔͱ͍͏ɻ q(x) ू߹ ʹ͍ͭͯɺ ͷݩͰͷू߹ ͷ֬ A ⊂ ℝN q(x) A Q(A) = ∫ A q(x)dx ͜ͷͱ͖ɺؔ ֬ͱ͍͏ɻ Q( ⋅ )
֬ม ϢʔΫϦουۭؒ ͷ্ʹϥϯμϜʹΛऔΔม Λ ʮ ʹΛऔΔ֬มʯͱ͍͏ɻ ℝN X ℝN ʮ
ͱͳΔ֬ʯ͕ Ͱ͋Δͱ͖ ʮ֬ม ͷ֬ Ͱ͋Δʯ͋Δ͍ ʮ֬ม ͷ֬ ʹै͏ʯ͋Δ͍ ʮ֬ม ͷ֬ Ͱ͋Δʯͱ͍͏ɻ X ∈ A Q(A) X q(x) X q(x) X Q
۩ମྫਅͷ αϯϓϧ ͕͋Δ֬ ʹಠཱʹै͏ ֬มͷ࣮ݱʢ؍ଌʣͩͱ͢Δɻ A = xn = {x1
, …, xn } ⊂ ℝN q(x) ͢ͳΘͪ Λ ্ͷ xn (ℝN)n q(xn) = n ∏ i=1 q(xi ) = q(x1 )q(x2 )⋯q(xn ) Λ࣋ͭ֬ม ͷ࣮ݱͰ͋Δͱߟ͑Δɻ ͜ͷͱ͖֬ ΛਅͷͱݺͿɻ Xn = (X1 , X2 , …, Xn ) q(x)
ฏۉͱࢄ
ฏۉͱࢄ ʹΛͱΔ֬ม ͷ֬Λ ͱ͢Δɻ ℝN X q(x) [f(X)] ≡ ∫
f(x)q(x)dx [f(X)] ≡ [(f(X) − [f(X)])(f(X) − [f(X)])T] = [f(X)f(X)T] − [f(X)][f(X)T] ͱఆٛ͢Δɻ ͕༩͑ΒΕͨͱ͖ɺ֬ม ͷฏۉΛ f : ℝN → ℝM f(X) ·ͨࢄڞࢄΛ ͱఆٛ͢Δɻ֬มΛ໌ه͍ͨ͠ͱ͖ ͱॻ͘ɻ X [f(X)]
۩ମྫαϯϓϧͷฏۉ αϯϓϧ Λද֬͢มΛ ͱ͢Δɻ ͦͷؔ ͕༩͑ΒΕͨͱ͖ɺͦͷฏۉΛऔΔૢ࡞ Λ xn = {x1
, …, xn } Xn = (X1 , X2 , …, Xn ) f(Xn) [ ⋅ ] ͱදه͢Δɻ ͜ͷฏۉ ΛʮαϯϓϧͷݱΕํʹର͢ΔฏۉʯͱݺͿɻ [ ⋅ ] [f(XN)] = ∫ ∫ ⋯ ∫ f(x1 , …, xn ) n ∏ i=1 q(xi )dxi
۩ମྫਅͷͷฏۉ αϯϓϧͷ֬มΛ Λ༻͍ͯɺ ਅͷ ͷਪଌΛߦͬͨޙɺਅͷͷ֬ม Λൃੜͤͯ͞ ਪଌ݁ՌͷΑ͞ΛධՁ͍ͨ͠ɻ ͜ͷ֬ม ͷؔ ʹ͍ͭͯͷฏۉΛ
Xn = (X1 , X2 , …, Xn ) q(x) X X f(X) ͱදه͢Δɻ [f(X)]X = ∫ f(x)q(x)dx
X X−1 ۭ֬ؒ(Ω = ℝM, ℬ, p) w ∈ Ω
ٯ૾X−1(A) ֬ີؔ ֬ q(x) = p(X−1(x)) Մଌۭؒ(Ω′ = ℝN, ℬ′ ) A ∈ ℬ′ X(w) = X x ∈ Ω′ ֬Q(A) = ∫ A q(x)dx f(x) ฏۉ[f(X)] ≡ ∫ f(x)q(x)dx = ∫ f(x)p(X−1(x))dx = ∫ p(w)X(w)dw = ∫ pXdw ֬ม ֬มͱ֬ɺฏۉͷؔ ۭ֬ؒ(Ω′ = ℝN, ℬ′ , q)
ಉ࣌ͱ͖݅֬
ಉ࣌ͱ͖݅ ͭͷ֬ม ͱ ͕͋Δͱ͖ɺͦͷ ͷ͕֬ Ͱ͋Δͱ͖ɺ Λಉ࣌֬ͱ͍͏ɻ X Y (X,
Y) p(x, y) p(x, y) ·ͨ֬ม ͕༩͑ΒΕͨͱ͖ͷ ͷ͖݅֬Λ࣍ͷΑ͏ ʹఆٛ͢Δɻ X Y p(y|x) = p(x, y) p(x) पล֬࣍ͷΑ͏ʹఆٛ͢Δɻ p(x) = ∫ p(x, y)dy p(y) = ∫ p(x, y)dx
ճؼؔ ֬ม ͷ֬ ʹ͍ͭͯߟ͑Δɻ ͷͱ͖ͷ ͷฏۉΛ (X, Y) p(X, Y)
X = x Y ͱॻ͘ɻ͜ͷؔΛ ͔Β ͷճؼؔ ͖݅ظ ͱ͍͏ɻ x y [Y|x] = ∫ yp(y|x)dy ؔΛ ͕༩͑ΒΕͨͱ͖ͦͷೋޡࠩΛද͢൚ؔΛ y = f(x) [(Y − f(X))2] = ∫ ∫ (y − f(x))2p(y, x)dxdy ͱॻ͘ͱ͜Ε ͷͱ͖ʹ࠷খʹͳΔɻ f(x) = [Y|x]
ΧϧόοΫɾϥΠϒϥใྔ
ΧϧόοΫɾϥΠϒϥใྔ ্ʹೋͭͷ֬ ͕͋Δͱ͖ ℝN q(x), p(x) D(p∥q) = ∫
q(x)log q(x) p(x) dx ͷ͜ͱΛΧϧόοΫɾϥΠϒϥใྔ͋Δ͍૬ରΤϯτϩϐʔͱݺͿ ΧϧόοΫɾϥΠϒϥใྔ͕࣍Γཱͭɻ ʹ͍ͭͯ Ͱ͋Δɻ ͱͳΔͷ ͷͱ͖ʹݶΔɻ ∀q(x), p(x) D(q∥p) ≥ 0 D(q∥p) = 0 q(x) = p(x)
ΧϧόοΫɾϥΠϒϥใྔ ূ໌ ͱ͓͘ͱɺ Ͱ͋Γɺ F(t) = 0 ⇔ t
= 0 F(t) = t + et − 1 (−∞ < t < ∞) ΑΓ Ͱ͋Δ͔Β͕ࣔ͞Εͨɻ ∫ q(x)dx = 1 ∫ p(x)dx = 1 ∫ log q(x) p(x) dx = 0 ·ͨɺ ͷͱ͖ɺ Ͱ Ͱ͋Δ͜ͱΛ༻͍ͯ q(x) ≈ p(x) t ≈ 0 F′ ′ (t) ≃ t2/e D(p∥q) ≃ ∫ q(x)(log q(x) − log p(x))2dx ͕Γཱͭɻ
ۃݶఆཧ
֬มͷऩଋ
֬ऩଋ ֬ม ͕ఆ ʹ֬ऩଋ͢Δͱ ʹର͠ɺ ʹ͓͍ͯ {Xn }n∈ℕ c ∀ϵ,
∀δ > 0 ∃N ∈ ℕ n > N ⇒ P(∥Xn − c∥ > ϵ) < δ ⇔ P(∥Xn − c∥ < ϵ) = 1 ͱͳΔ͜ͱͰ͋Δɻ ͜Εେͷऑ๏ଇʹରԠ͍ͯ͠Δɻ Xn c ϵ ඪຊ͕ेʹେ͖͍ͱ͖ɺඪຊฏۉฏۉʹऩଋ͢Δ
๏ଇ ऩଋ ֬มͷྻ ͕֬ม ʹ๏ଇ ऩଋ͢Δͱ ͷ͕֬ Ͱ
ͷ͕֬ Ͱ͋Δͱ͖ɺ ҙͷ༗ք͔ͭ࿈ଓͳؔ ʹରͯ͠ {Xn }n∈ℕ X Xn qn (x) X q(x) F(x) lim n→∞ ∫ F(x)qn (x)dx = ∫ F(x)q(x)dx ⇔ lim n→∞ [F(Xn )] = [F(X)] ͕Γཱͭ͜ͱͰ͋Δɻ͜Εத৺ۃݶఆཧʹରԠ͍ͯ͠Δɻ ඪຊ͕ेʹେ͖͍ͱ͖ɺूஂͷʹؔΘΒͣඪຊฏۉͱฏۉͷࠩਖ਼نʹै͏
ܦݧաఔ
ϢʔΫϦουۭؒʹ͓͚ΔίϯύΫτੑ ϢʔΫϦουۭؒ ͷ෦ू߹ ͕։ू߹ͷ ʹ ͍ͭͯ ͳΒɺͦͷ༗ݶݸͷ։ू߹ Ͱ ℝN W
= {O}λ∈Λ W ⊂ ⋃ λ∈Λ Oλ O1 , …, On ∈ ͱͳΔͷ͕͋Δͱ͖ɺ ίϯύΫτͰ͋Δͱ͍͏ W ⊂ O1 ∪ … ∪ On W O1 , …, On ∈ W
্ۭؔؒͷେͷ๏ଇ ϢʔΫϦουۭؒ ʹΛऔΔ ͕֬ม ͱ ಉ֬͡ʹै͏ͱ͢Δɻ ύϥϝʔλͷू߹ ΛίϯύΫτͱ͢Δɻ ℝN X1
, X2 , …, Xn X w ∈ W ∈ ℝN f(x, w) : ℝN → ℝ1 X [ sup w∈W |f(X, w)|] < ∞, X [ sup w∈W |∇w f(X, w)|] < ∞ ݅ ͕ΓཱͭͱԾఆ͢Δɻ͜ͷͱ͖ɺ ʹ͍ͭͯ ∀ϵ > 0 P( sup w∈W 1 n n ∑ i=1 f(Xi , w) − X [f(X, w)] < ϵ) = 1 ͜ͷ͜ͱΛ্ۭؔؒͷେͷ๏ଇͱ͍͏
ਖ਼ن֬աఔ ू߹ ্ͷؔͰ֬తʹมಈ͢Δͷ ͕ɺ ฏۉؔ ͱ૬ؔؔ Λ࣋ͭਖ਼ن֬աఔͰ͋Δͱɺ ֤ ͝ͱʹ ͕ਖ਼نʹै͏֬มͰ͋Γɺ
W ξ(w) m(w) ρ(w, w′ ) w ξ(w) m(w) = ξ [ξ(w)], ρ(w, w′ ) = ξ [ξ(w)ξ(w′ )] ͕Γཱͭ͜ͱͰ͋Δɻ͜͜Ͱ ɺ֬աఔ ʹ͍ͭͯͷฏۉΛ ද͍ͯ͠ΔɻίϯύΫτू߹্Ͱͷਖ਼ن֬աఔɺ ξ [ ⋅ ] ξ ฏۉؔͱ૬͕ܾؔؔ·ΔͱҰҙʹఆ·Δ͜ͱ͕ΒΕ͍ͯΔɻ
ܦݧաఔ ͭ͗ʹ X[ sup w∈W |f(X, w) − X
[f(X, w)]|α ] < ∞ X[ sup w∈W |∇w (f(X, w) − X [f(X, w)])|α ] < ∞ ͕ ͰΓཱͭͱԾఆ͢Δɻ α = 2 Yn (w) = 1 n n ∑ i=1 (f(Xi , w) − X [f(X, w)]) ͜ͷ֬աఔ Λܦݧաఔͱ͍͏ɻ Yn (w)
ܦݧաఔ ֬աఔ ܦݧաఔ ฏۉ͕ Ͱ૬͕ؔؔ Yn (w) 0
ͷਖ਼ن֬աఔ ʹ๏ଇऩଋ͢Δɻ Y(w) ρ(w, w′ ) = X [f(X, w)f(X, w′ )] − X [f(X, w)]X [f(X, w′ )]
֬աఔͷ๏ଇऩଋ ֬աఔ ܦݧաఔ ͕֬աఔ ʹ๏ଇऩଋ͢Δͱɺ ༗ք࿈ଓͳ൚ؔ ʹ͍ͭͯ Yn (w)
Y(w) F( ⋅ ) ͕Γཱͭͱ͍͏͜ͱͰ͋Δɻͳ͓ɺ൚ؔ ͕࿈ଓͰ͋Δͱ F( ⋅ ) lim n→∞ [F(Yn )] = Y [F(Y)] lim n→∞ sup w∈W |fn (w) − f(w)| → 0 ⇒ lim n→∞ F(fn ) = F(f ) ͕Γཱͭ͜ͱͰ͋Δɻ ͜ͷΑ͏ͳܗͷఆཧΛ্ۭؔؒͷத৺ۃݶఆཧͱ͍͏ɻ
ࢀߟࢿྉ w ֬ೖ ล w ܦݧաఔͱ ล w ϕΠζ౷ܭͷཧͱํ๏ ล
w ଌɾ֬ɾϧϕʔάੵ ݪܒհ