Share
Embed
Transcript
34 slides
-
࣍ w $1$ΛಡΜͩཧ༝ w ૬ޓใྔ w ີൺਪఆ w $1$ w
ີൺਪఆΛར༻ͨ͠૬ޓใྔͷ࠷େԽʹΑΔ දݱֶश -
ࣗͷϞνϕʔγϣϯ w "OJNBM"*Ͱղ͚ͳ͔ͬͨΛͲ͏ͱ͔͘Λߟ͍͑ͨ w *OUFSOBMNPEFM4QBUJBMSFBTPOJOH w Ӆ͠͞ΕͯҠಈͰ͖Δ w ၆ᛌࢹਤ w
ڥͷμΠφϛΫε w Ϙʔϧ͕Ͳ͏ಈ͔͘༧ଌͰ͖Δ w $BVTBMSFBTPOJOH -
؍ଌ ੜ Ϟσϧ ΛϞσϦϯά p(x|c) ؍ଌϞσϧΛΘͣʹ ૬ޓใྔ ࠷େԽ I(x;
c) Ξϓϩʔν Y D ίϯςΩετ Γ͍ͨ͜ͱ ࣌ܥྻͷίϯςΩετ$ͷදݱΛ ࣗݾڭࢣֶशͰ ֶश͍ͨ͠ Ծఆ $͔ΒະདྷͷY͕༧ଌͰ͖ΔͳΒ$ྑ͍දݱ $࣌ܥྻͷେҬతͳߏ͕֫ಘͰ͖͍ͯΔ %FDPEFSΛ࡞Δ %FDPEFSΛ࡞Βͳ͍ -
૬ޓใྔ A = a0 A = a1 B = b0
B = b1 A = a0 A = a1 B = b0 B = b1 P(A, B) = P(A)P(B) ૬ޓใྔ I(A, B) = ∫ A ∫ B P(A, B)log P(A, B) P(A)P(B) = P(A|B)P(B)log P(B|A)P(A) P(A)P(B) = − P(B)log P(B) = H(B) = H(A) I(A, B) = ∫ A ∫ B P(A, B)log P(A, B) P(A)P(B) ࠷খͷ࣌ ࠷େͷ࣌ ͖݅Q #c" ͕ %FUFSNJOJTUJDʹͳ͍ͬͯΔ "ͷ݁Ռ͕Θ͔Ε #ͷ͜ͱ͕Θ͔Δ "ͷ݁Ռ͕Θ͔ͬͯ #ͷ͜ͱ͕ԿΘ͔Βͳ͍ = 0 -
؍ଌ ੜ Ϟσϧ ΛϞσϦϯά p(x|c) ؍ଌϞσϧΛΘͣʹ ૬ޓใྔ ࠷େԽ I(x;
c) Ξϓϩʔν Y D ίϯςΩετ Γ͍ͨ͜ͱ ࣌ܥྻͷίϯςΩετ$ͷදݱΛ ࣗݾڭࢣֶशͰ ֶश͍ͨ͠ Ծఆ $͔ΒະདྷͷY͕༧ଌͰ͖ΔͳΒ$ྑ͍දݱ $࣌ܥྻͷେҬతͳߏ͕֫ಘͰ͖͍ͯΔ %FDPEFSΛ࡞Δ %FDPEFSΛ࡞Βͳ͍ -
∑ x ∑ c p(x, c)log p(x ∣ c) p(x)
I(x; c) = ∑ x ∑ c p(x, c)log p(x, c) p(x)p(c) ∑ x ∑ c p(x, c)log p(x|c)p(c) p(x)p(c) Y D ίϯςΩετ p(x) p(x|c) ͜ͷ૬ޓใྔΛ࠷େԽ͢ΔΑ͏ʹ DͷදݱΛֶश͍ͨ͠ ૬ޓใྔʹີൺͷ ܗ͕ग़ͯ͘Δ -
r(x) = pA (x) pB (x) {xA i }nA i=1
{xB j }nB j=1 αϯϓϧ ݸˠϥϕϧ"ׂΓͯ nA αϯϓϧ ݸˠϥϕϧ#ׂΓͯ nB લఏೋͭͷ֬ີͷαϯϓϧಘΒΕ͍ͯΔ͕֬Θ͔Βͳ͍ తೋͭͷͷ֬ີͷൺΛɺαϯϓϧ͔Βֶशͨ͠ྨثΛ༻͍ͯਪఆ͢Δ pA (x) pB (x) ൺ͕r(x) -
pA (x) = p(x|y = A) pB (x) = p(x|y
= B) r(x) = pA (x) pB (x) = p(x|y = A) p(x|y = B) = p(y = A|x)p(x) p(y = A) p(y = B|x)p(x) p(y = B) = p(y = B) p(y = A) p(y = A|x) p(y = B|x) ̂ r(x) = nB nA ̂ p(y = A|x) ̂ p(y = B|x) = nB nA ̂ p(y = A|x) 1 − ̂ p(y = A|x) ີൺͷۙࣅ ྨثΛֶश αϯϓϧ͕"͔Βͷͷ͔#͔Βͷͷ͔Λྨ͢ΔྨثΛ ֶश͢Δͱͦͷग़ྗͷ͕֬ີൺʹ͑Δ -
ͱ ͷ૬ޓใྔΛٻΊΔͷͰͳ͘ɺ ΛΤϯίʔυͨ͠ ͱ ͷ૬ޓใྔͷ࠷େԽΛߟ͑Δ x c x z c
I(x, c) ≥ I(z, c) -
fk (xt+k , ct) ∝ p (xt+k ∣ ct) p
(xt+k) fk (xt+k , ct) = exp (zT t+k Wk ct) ີൺඞͣਖ਼ͷ Y D ίϯςΩετ p(x) p(x|c) ີൺΛ ͱ ͷؔ ͰϞσϧԽ͢Δ z c f ༧ଌઌεςοϓ FYd k -
ℒN = − X log fk (xt+k , ct) ∑
xj ∈X fk (xj , ct) X = {x1 , ⋯xN } w /ݸͷத͔ΒQPTJUJWFTBNQMFΛબͿ$BUFHPSJDBMͷ 4PGUNBYDSPTTFOUSPQZMPTTͷܗ w -PTTͷ࠷దԽʹΑΓ w B ͕ ʹൺྫ͢Δ ʹͳΔ w C ૬ޓใྔ͕࠷େԽ͞ΕΔ fk (xt+k , ct) p(xt+k |ct ) p(xt+k ) ͔Βݸͷ1PTJUJWFTBNQMF ͔Β/ݸͷ/FHBUJWFTBNQMF p(xt+k |ct ) p(xt+k ) I(xt+k , ct ) ≥ log(N) − ℒ ࠷దԽ͢Δ-PTT fk (xt+k , ct) = exp (zT t+k Wk ct) QPTJUJWFTBNQMFͰͷf QPTJUJWFTBNQMF OFHBUJWFTBNQMFͷ ͷ߹ܭ f -
p(x0 |ct )p(x1 ) p(x0 |ct )p(x1 ) + p(x1
|ct )p(x0 ) ྫ/ͷ߹ αϯϓϧ ͕͋Δ࣌ɺ ͕QPTJUJWFTBNQMFͰ͋Δ֬ʁ QPTJUJWF͕ͲΕ͔ͷબࢶ ͱ ͷೋ௨Γߟ͑ΒΕΔ x0 , x1 x0 x0 x1 ͕QPTJUJWF ͕OFHBUJWFͰ͋Δ߹ͷൃੜ֬ x0 x1 ͕QPTJUJWF ͕OFHBUJWFͰ͋Δ߹ͷൃੜ֬ x1 x0 p(x0 |ct ) p(x0 ) p(x0 |ct ) p(x0 ) + p(x1 |ct ) p(x1 ) Ͱ྆ลׂΔͱ p(x0 )p(x1 ) B ͕ ʹͳΔ͜ͱͷઆ໌ fk (xt+k , ct) p(xt+k |ct ) p(xt+k ) -
p(d = i|X, ct ) = p(xi |ct )∏ l≠i
p(xi ) ∑N j=1 {p(xj |ct )∏ l≠j p(xl )} લϖʔδͷઆ໌ΛҰൠԽ αϯϓϧ ͕QPTJUJWFTBNQMFͰ͋Δ֬ xi p(xi |ct ) p(xi ) QPTJUJWFTBNQMFݸ OFHBUJWFTBNQMF/ݸ = p(xi |ct )∏ l≠i p(xi ) ∏ l p(xl ) ∑N j=1 { p(xj |ct )∏ l≠j p(xl ) ∏ l p(xl ) } = p(xi |ct ) p(xi ) ∑N j=1 { p(xj |ct ) p(xj ) } ℒN = − X log fk (xt+k , ct) ∑ xj ∈X fk (xj , ct) B ͕ ʹͳΔ͜ͱͷઆ໌ fk (xt+k , ct) p(xt+k |ct ) p(xt+k ) -PTTͷ࠷దԽʹΑΓαϯϓϧ ͕ QPTJUJWFTBNQMFͰ͋Δ֬ʹऩଋ͢Δ xt+k -
ℒopt N = − X log p(xt+k |ct ) p(xt+k
) p(xt+k |ct ) p(xt+k ) + ∑ xj ∈Xneg p(xj |ct ) p(xj ) = X log p(xt+k |ct ) p(xt+k ) + ∑ xj ∈Xneg p(xj |ct ) p(xj ) p(xt+k |ct ) p(xt+k ) = X log 1 + p(xt+k ) p(xt+k |ct ) ∑ xj ∈Xneg p(xj |ct ) p(xj ) ≈ X log [ 1 + p(xt+k ) p(xt+k |ct ) (N − 1)xj p(xj |ct ) p(xj ) ] ͔Β/ݸαϯϓϦϯά͖ͯͯ͠ ΛٻΊͯΛऔΔ p(x) p(x|c) p(x) αϯϓϦϯάΛظܭࢉʹஔ͖͑ ≈ X log [ 1 + p(xt+k ) p(xt+k |ct ) (N − 1) ] xj p(xj |ct ) p(xj ) = ∫ p(xj ) p(xj |ct ) p(xj ) dxj = ∫ p(xj |ct )dxj = 1 = X log p(xt+k ) p(xt+k |ct ) N + { 1 − p(xt+k ) p(xt+k |ct ) } ≥ X log [ p(xt+k ) p(xt+k |ct ) N ] ظܭࢉऔΔͱΑΓখ͘͞ͳΔ = − I(xt+k , ct ) + log(N) Ͱظ औ͍ͬͯΔ p(xt+k , ct ) I(xt+k , ct ) = ∫ p(xt+k , ct ) p(xt+k |ct ) p(xt ) X = {x1 , ⋯xN } ݸͷQPTJUJWFTBNQMFͱ/ݸͷOFHBUJWFTBNQMF C -PTTͷ࠷খԽ͕૬ޓใྔͷ࠷େԽʹͳΔઆ໌ -
ℒN = − X log fk (xt+k , ct) ∑
xj ∈X fk (xj , ct) X = {x1 , ⋯xN } /ݸͷαϯϓϧͷத͔ΒQPTJUJWFTBNQMFΛબͿ$BUFHPSJDBMͷ TPGUNBYDSPTTFOUSPQZMPTTͷ࠷దԽ ͔ΒݸͷQPTJUJWFTBNQMF ͔Β/ݸͷOFHBUJWFTBNQMF p(xt+k |ct ) p(xt+k ) I(xt+k , ct ) ≥ log(N) − ℒ MPTTΛԼ͛Δ ૬ޓใྔ͕ߴ͘ͳΔ ͕ີൺ ʹൺྫ͢Δ ʹͳΔ ີൺ͕ਖ਼͘͠ٻ·Δ fk -
FOD FOD zpred ztrue znoise W zT pred ztrue ˠେ͖͘
zT pred znoise ˠখ͘͞ c w ୯ʹ ͕ Λ.4&Ͱ༧ଌ͢Δ͚ͩͩͱ %FDPEFS͕ແ͍ͷͰ ͕༧ଌ͍͢͠USJWJBMͳදݱʹFODPEF͞Εͯ͠·͏ શ෦ͱ͔ w ༧ଌର͕QPTJUJWFTBNQMFͱͳΔ࣮ࡍͷ֬ʹ͚ۙͮΔͨΊʹ େɺ খͱ͢ΔΑ͏ͳ੍͕ಇ͘ͷͰɺͦΕ͕͛Δ zpred ztrue z zT pred znoise zT pred znoise ײతͳཧղ QPTJUJWF TBNQMF OFHBUJWF TBNQMFT z pred ztrue znoise -
"VEJP Իૉྨ DMBTT ऀྨ DMBTT $1$ͰTFMGTVQFSWJTFEͰ ֶशͨ͠දݱD͔ΒઢܗྨثΛֶश w 1$.ԻݯΛNTFD୯ҐͰ$POWͰFODPEF ࣍ݩ
w (363//ͰDΛFODPEF ࣍ݩ w $1$ͰTUFQઌͷ༧ଌ w / OFHBUJWFTBNQMFݸʁ -
·ͱΊ w ྨΛར༻ͨ͠ີൺਪఆΛߦ͏͜ͱʹΑΔ ૬ޓใྔͷ࠷େԽ w நతͳදݱΛ؍ଌϞσϧ %FDPEFS ͳ͠Ͱ֫ಘͰ͖Δ w ܭࢉίετ͕͍
w "VEJP 7JTJPO FUDͷ͍Ζ͍ΖͳυϝΠϯʹద༻Մೳ