Tweet
Tweet

Slide 1

Slide 1 text

-FBSOJOHUP&YQMBJO"O*OGPSNBUJPO 5IFPSFUJD1FSTQFDUJWFPO.PEFM *OUFSQSFUBUJPO ಡΈձˏ ༶໌఩

Slide 2

Slide 2 text

 wஶऀ w+JBOCP$IFO 6$#FSLFMFZ "OU'JOBODJBMͷΠϯλʔϯ  w-F4POH (FPSHJB*OTUJUVUFPG5FDIOPMPHZ "OU'JOBODJBM  w.BSUJO+8BJOXSJHIU 6OJWFSTJUZPG$BMJGPSOJB 5IF7PMFPO(SPVQ  w.JDIBFM*+PSEBO 6OJWFSTJUZPG$BMJGPSOJB  wग़య*$.- ࿦จ৘ใ

Slide 3

Slide 3 text

 wϞσϧ͕ʮ൑அ ग़ྗ ͷࠜڌͱͳͬͨಛ௃ʯΛઆ໌͢ΔΑ͏ͳઆ໌ϞσϧΛֶश ͢Δํ๏ΛఏҊ͍ͯ͠Δɽ wैདྷͷઆ໌ϞσϧͱҧͬͯɼҰ౓આ໌ϞσϧͷֶशΛߦ͑͹આ໌ͷੜ੒Λຖճܭ ࢉ͠ͳ͍͍ͯ͘ͷͰඇৗʹߴ଎ɽ wબΜͩཧ༝ wࣗ෼ͷݚڀʹઆ໌Մೳ"*ΛऔΓೖΕ͍ͨͱࢥͬͨͨΊɽ w ࠷ۙϗοτͰ΋͋Δ͔Β ֓ཁ ͲΜͳ࿦จ

Slide 4

Slide 4 text

 wϥϯμϜϑΥϨετ Χʔωϧ๏ɼσΟʔϓχϡʔϥϧωοτϫʔΫͳͲෳࡶͳϞ σϧ͕ఏҊ͞ΕɼͦΕΒʹΑͬͯ༧ଌਫ਼౓͕ߴ·͍ͬͯΔɽ w͔͠͠ɼෳࡶͳϞσϧʹΑΔग़ྗ݁ՌΛਓ͕ؒղऍ͢Δͷ͸ࠔ೉ʹ΋ͳ͍ͬͯ Δɽ wຊ࿦จͰ͸༧ଌʹॏཁͳಛ௃ྔΛݟ͚ͭΔ͜ͱͰɼϞσϧͷղऍੑΛߴΊΑ͏ͱ ͍ͯ͠Δɽ ֓ཁ ݚڀഎܠ

Slide 5

Slide 5 text

  ೖྗͷϕΫτϧʹؔͯ͠ɼਖ਼ղͷग़ྗͷޯ഑ΛٻΊΔɽٻΊͨޯ഑Λ࢖͍ɼೖྗ ʹϚεΫͯ͠આ໌ͱ͢Δɽ  આ໌͍ͨ͠αϯϓϧͷपลͰہॴతʹ؆୯ͳࣝผϞσϧΛ࡞ΓɼͦΕΛ࢖ͬͯઆ ໌Λߦ͏ɽ w -*.&΍%FFQ-*'5 LFSOFM4)"1ͳͲ͕͋Δɽ ֓ཁ طଘݚڀʹ͍ͭͯ

Slide 6

Slide 6 text

 wہॴతͳઆ໌ϞσϧΛશମతʹֶश͠ɼೖྗͷ෼෍΋ߟྀʹೖΕΒΕΔɽ wہॴతͳઆ໌ϞσϧΛ௥Ճ͠ͳͯ͘΋ྑ͍ɽ ֓ཁ طଘݚڀͱͷҧ͍

Slide 7

Slide 7 text

 ߩݙ͸࣍ͷΑ͏ʹͳΔ w ৘ใྔΛ΋ͱʹͯ͠ɼΠϯελϯε͝ͱͷಛ௃બ୒͕Ͱ͖ΔϑϨʔϜϫʔΫΛఏҊ ͨ͠ɽ w Ϟσϧʹґଘͤͣޮ཰ͷΑ͍ɼಛ௃બ୒ͷΞϧΰϦζϜΛఏҊͨ͠ɽ w ఏҊͨ͠ΞϧΰϦζϜ͕ਓ޻σʔλͱϦΞϧσʔλͰޮՌ͕͋Δ͜ͱΛ࣮ݧͨ͠ɽ ֓ཁ ߩݙ

Slide 8

Slide 8 text

 wఏҊख๏Ͱ͸ɼϞσϧʹґଘ͠ͳ͍ͨΊճؼɼ෼ྨϞσϧͷͲͪΒʹ΋దԠͰ͖Δɽ ʢࠓճ͸෼ྨϞσϧͰߟ͑Δʣ wલఏͱͯ͠ɼϞσϧͷग़ྗΛ৚݅෇͖֬཰ ɼઆ໌ม਺ 
 ֬཰ม਺ ͱ͢Δɽ w͜͜Ͱ͸ɼ૬ޓ৘ใྔΛΠϯελϯε͝ͱͷಛ௃બ୒ͷࢦඪͱͯ͠࢖͏ɽ  આ໌ม਺ͱબ୒͞Εͨಛ௃ྔͷ૬ޓ৘ใྔ͕࠷େͱͳΔ͜ͱ͕ࠓճͷ໨ඪɽ ℙm ( ⋅ ∣ x) Y X = x ∈ ℝd I(X; Y) = X,Y [log pXY (X, Y) pX (X)pY (Y) ] ϑϨʔϜϫʔΫ

Slide 9

Slide 9 text

 w  पล෼෍͔Βग़ͯ͘Δɽ w ৚݅෇͖෼෍ ͔Βग़ͯ͘Δɽ w αΠζ͕ Ͱ͋Δ΂͖ू߹ɽಛ௃બ୒ʹ࢖͏ɽ w ͸ϋΠύʔύϥϝʔλ w  ͔Β΂͖ू߹ʹϚοϐϯά͢Δઆ໌Ϟσϧɽ w  Ͱબ͹Εͨ෦෼ू߹ɼ͜ΕʹΑͬͯબ୒͞Εͨಛ௃ͷ֬཰ ม਺ ΛಘΔɽ X X ∼ ℙX ( ⋅ ) Y (Y ∣ x) ∼ ℙm ( ⋅ ∣ x) ℘k = {S ⊂ 2d ||S ∣ = k} k k ℰ ℝd S S = ℰ(x) xs ϑϨʔϜϫʔΫ ໰୊ઃఆ

Slide 10

Slide 10 text

 w৽͍֬͠཰ϕΫτϧ Λ༻͍Δͱࠓճͷ໨త͸࣍ࣜΛ࠷దԽ͢Δ͜ͱʹͳ Δɽ  w૬ޓ৘ใྔΛ࠷େʹ͢ΔΑ͏ͳɼಛ௃ͷ෦෼ۭ͕ؒࣝผϞσϧͰॏཁͳಛ௃Ͱ͋ Δͱઆ໌͢Δ͜ͱ͕Ͱ͖Δɽ Xs ∈ Rk ϑϨʔϜϫʔΫ ໨తؔ਺

Slide 11

Slide 11 text

 w໨తؔ਺Λ௚઀ղ͘͜ͱ͕ࠔ೉Ͱ͋Δ͔Βɼม෼ۙࣅΛ༻͍ͯղ͘ɽ w૬ޓ৘ใྔͷԼքΛ࠷େԽ͢Δ͜ͱΛ໨ඪʹͳΔɽ ఏҊख๏

Slide 12

Slide 12 text

 w ҰൠతͳϞσϧͰ͸ɼ৚݅෇͖෼෍ͷԼքͷظ଴஋Λܭࢉ͢Δ͜ͱ͕ࠔ೉Ͱ͋Δͨ Ίɼ࣍ͷΑ͏ͳม෼଒Λఆٛ͢Δɽ w   w +FOTFOͷෆ౳ࣜʹΑΓɼԼք͕࣍ͷΑ͏ʹมߋ͞ΕΔɽ w := {ℚ ∣ ℚ = {xS → ℚS (Y ∣ xS), S ∈ ℘k}} ఏҊख๏ ม෼Լք

Slide 13

Slide 13 text

 w Λ༻͍Δͱɼࠓճͷ໨ඪ͕࣍ͷΑ͏ʹมΘΔɽ w  wҰൠతͳ ͷ৔߹ɼࣜ  Λղ͘ͷ͸͍·ͩʹࠔ೉Ͱ͋Δɽͦ͜Ͱ࠷దԽՄೳͳ ํ๏ʹม͑Δඞཁ͕͋Δɽ ℰ, ఏҊख๏ ࠷దԽ໰୊

Slide 14

Slide 14 text

 w ΛύϥϝʔλԽ͢ΔͨΊʹɼχϡʔϥϧωοτϫʔΫΛఆٛ͢Δɽ w   wࠓճఆٛͨ͠ Λ༻͍ͯɼ ͱ͢Δɽ gα : ℝd × [c] → [0,1] where[c] = 0,1,…, c − 1 gα ℚx := gα (˜ xs , Y) ఏҊख๏ ΛύϥϝʔλԽ͢Δ

Slide 15

Slide 15 text

 wࣜ  Λ௚઀ਪఆ͢Δʹ͸ɼ ͜ͷ෦෼ۭؒΛ߹ܭ͢Δඞཁ͕͋Γେมɽ w͜ΕΛղܾ͢ΔͨΊʹɼ(VNCFMTPGUNBYUSJDLΛ༻͍Δɽ wΧςΰϦΧϧ෼෍Λ࿈ଓʹͳΔΑ͏ʹ؇࿨͢Δख๏ w(VNCFMTPGUNBYUSJDLΛ࢖ͬͯɼॏΈ͚ͮΒΕͨ෦෼ۭؒͷαϯϓϦϯάͷۙࣅ Λߦͳ͍ͬͯ͘ɽ n Ck ఏҊख๏ αϒηοταϯϓϦϯάͷ࿈ଓ؇࿨ Gumbel-softmax trickͷࢀߟ: http://peluigi.hatenablog.com/entry/2018/06/23/192435

Slide 16

Slide 16 text

 w ͜ͷಛ௃͔Β ͜ͷಛ௃ΛαϯϓϦϯά͢Δํ๏Λ࣍ʹࣔ͢ɽ  ͜ͷಛ௃ϕΫτϧΛಠཱʹ ճαϯϓϦϯάΛߦ͏ɽ  ͔Ϳͬͨಛ௃͸࡟আͯ͠࢒ΓΛ࢒͢ɽ w͜͏͢Δͱ͍͍ͤͥ ͚ͩಛ௃͕࢒Δɽ͜ΕΒΛࣜʹ͢Δͱ࣍ͷΑ͏ʹͳΔɽ d k d k k ఏҊख๏ ಛ௃ΛL͚ͩநग़͢Δ

Slide 17

Slide 17 text

 w࠷ऴతʹɼ࠷దԽ͢Δ΂͖໨తͷ΋ͷ͸࣍ͷ΋ͷͰ͋Δɽ w  w ͸ ʹґଘ͠ͳ͍΋ͷͰ͋Δ͔Βɼ܇࿅࣌Ͱ͸ɼಉ࣌ʹޯ഑߱ԼΛద༻͢Δ ͜ͱ͕Ͱ͖Δɽ X,ζ θ, α ఏҊख๏ ࠷ऴతͳ໨తؔ਺ͱͦͷ࠷దԽ

Slide 18

Slide 18 text

 wֶश͞Εͨઆ໌ϞσϧʹΑͬͯɼαϯϓϧ ͔ΒॏΈϕΫτϧ ࣍ݩͷ ʹϚο ϐϯά͞ΕΔɽ w͔͜͜ΒείΞ ͷେ͖͍ํΛબ୒ͯ͠ɼઆ໌༻ͷಛ௃ͱ͢Δɽ w֤αϯϓϧʹରͯ͠ɼઆ໌ϞσϧΛ௨͚ͩ͢Ͱɼಛ௃Λબ୒͢Δ͜ͱ͕Ͱ͖Δͷ ͰɼطଘݚڀΑΓ΋ޮ཰Α͘આ໌Ͱ͖ɼ͔ͭϞσϧʹґଘ͠ͳ͍Ͱ͓͜ͳ͑Δɽ X d wθ (X) pθ (X) ఏҊख๏ આ໌εςʔδ

Slide 19

Slide 19 text

 wਓ޻σʔλͱ࣮ࡍͷσʔλͷೋͭͰ࣮ݧΛߦ͏ɽ wਓ޻σʔλͰ͸ɼ࣍ݩΨ΢ε෼෍͔Βੜ੒ͯ͠ɼ͔ͦ͜ΒछྨͷσʔληοτΛ࡞੒ɽ w ࣮ݧ σʔληοτ

Slide 20

Slide 20 text

 wຊݚڀͷख๏ -9 ͱطଘख๏ %FFQ-*'5 4)"1 -*.&ʜ Λൺֱ͢Δɽ ࣮ݧ ࣮ݧ݁Ռ

Slide 21

Slide 21 text

 wઆ໌࣌ͷ࣮ߦ࣌ؒͷ݁ՌΛҎԼʹࣔ͢ɽ ࣮ݧ ࣮ݧ݁Ռ

Slide 22

Slide 22 text

 ࣮ݧ݁Ռ ςΩετσʔλͰͷϋΠϥΠτ өըϨϏϡʔͷσʔληοτ

Slide 23

Slide 23 text

 wөըϨϏϡʔͷจʹରͯ͠ɼΫϥ΢υϫʔΧʹݸͷײ৘ʹ߆ΔΩʔϫʔυΛબ ୒ͯ͠΋Β͏ɽ wෳ਺ͷจষΛෳ਺ͷϫʔΧʹධՁͯ͠΋Β͍ɼͦͷฏۉΛ)VNBOBDDVSBDZͱ͢ Δɽ ࣮ݧ݁Ռ Ϋϥ΢υϫʔΧʹධՁͯ͠΋Β͏

Slide 24

Slide 24 text

 w૬ޓ৘ใྔΛجʹͯ͠ɼΠϯελϯε͝ͱͷಛ௃બ୒Λ͢ΔϑϨʔϜϫʔΫΛఏ Ҋͨ͠ɽ w-9ʹΑΓɼ͸͡ΊͯϦΞϧλΠϜͰϒϥοΫϘοΫεͳϞσϧΛઆ໌͢Δ͜ͱ ͕Մೳʹͳͬͨɽ ݁࿦