[読み会]Learning to Explain: An Information-Theoretic Perspective on Model Interpretation

読み会資料.
Learning to Explain: An Information-Theoretic Perspective on Model Interpretation (ICML2018)

Transcript

1. -FBSOJOH
   UP
   &YQMBJO
   "O
   *OGPSNBUJPO 5IFPSFUJD
   1FSTQFDUJWF
   PO
   .PEFM
   *OUFSQSFUBUJPO ಡΈձˏ
   ༶໌఩
2.  wஶऀ
   w+JBOCP
   $IFO
   6$
   #FSLFMFZ
   "OU
   'JOBODJBMͷΠϯλʔϯ
   w-F4POH
   (FPSHJB
   *OTUJUVUF
   PG
   5FDIOPMPHZ
   "OU
   'JOBODJBM
   w.BSUJO
   +
   8BJOXSJHIU
   6OJWFSTJUZ
   PG
   $BMJGPSOJB
   5IF
   7PMFPO
   (SPVQ
   w.JDIBFM
   *
   +PSEBO
   6OJWFSTJUZ
   PG
   $BMJGPSOJB
   wग़య
   *$.- ࿦จ৘ใ
3.  wϞσϧ͕ʮ൑அ ग़ྗ ͷࠜڌͱͳͬͨಛ௃ʯΛઆ໌͢ΔΑ͏ͳઆ໌ϞσϧΛֶश ͢Δํ๏ΛఏҊ͍ͯ͠Δɽ
   wैདྷͷઆ໌ϞσϧͱҧͬͯɼҰ౓આ໌ϞσϧͷֶशΛߦ͑͹આ໌ͷੜ੒Λຖճܭ ࢉ͠ͳ͍͍ͯ͘ͷͰඇৗʹߴ଎ɽ
   wબΜͩཧ༝
   wࣗ෼ͷݚڀʹઆ໌Մೳ"*ΛऔΓೖΕ͍ͨͱࢥͬͨͨΊɽ
   w ࠷ۙϗοτͰ΋͋Δ͔Β ֓ཁ ͲΜͳ࿦จ
4.  wϥϯμϜϑΥϨετ
   Χʔωϧ๏ɼσΟʔϓχϡʔϥϧωοτϫʔΫͳͲෳࡶͳϞ σϧ͕ఏҊ͞ΕɼͦΕΒʹΑͬͯ༧ଌਫ਼౓͕ߴ·͍ͬͯΔɽ
   w͔͠͠ɼෳࡶͳϞσϧʹΑΔग़ྗ݁ՌΛਓ͕ؒղऍ͢Δͷ͸ࠔ೉ʹ΋ͳ͍ͬͯ Δɽ
   wຊ࿦จͰ͸༧ଌʹॏཁͳಛ௃ྔΛݟ͚ͭΔ͜ͱͰɼϞσϧͷղऍੑΛߴΊΑ͏ͱ ͍ͯ͠Δɽ ֓ཁ ݚڀഎܠ
5.   ೖྗͷϕΫτϧʹؔͯ͠ɼਖ਼ղͷग़ྗͷޯ഑ΛٻΊΔɽٻΊͨޯ഑Λ࢖͍ɼೖྗ ʹϚεΫͯ͠આ໌ͱ͢Δɽ
    આ໌͍ͨ͠αϯϓϧͷपลͰہॴతʹ؆୯ͳࣝผϞσϧΛ࡞ΓɼͦΕΛ࢖ͬͯઆ ໌Λߦ͏ɽ
   w -*.&
   ΍
   %FFQ-*'5
   LFSOFM
   4)"1ͳͲ͕͋Δɽ
   ֓ཁ طଘݚڀʹ͍ͭͯ
6.  wہॴతͳઆ໌ϞσϧΛશମతʹֶश͠ɼೖྗͷ෼෍΋ߟྀʹೖΕΒΕΔɽ
   wہॴతͳઆ໌ϞσϧΛ௥Ճ͠ͳͯ͘΋ྑ͍ɽ
   ֓ཁ طଘݚڀͱͷҧ͍
7.  ߩݙ͸࣍ͷΑ͏ʹͳΔ
   w ৘ใྔΛ΋ͱʹͯ͠ɼΠϯελϯε͝ͱͷಛ௃બ୒͕Ͱ͖ΔϑϨʔϜϫʔΫΛఏҊ ͨ͠ɽ
   w Ϟσϧʹґଘͤͣޮ཰ͷΑ͍ɼಛ௃બ୒ͷΞϧΰϦζϜΛఏҊͨ͠ɽ
   w ఏҊͨ͠ΞϧΰϦζϜ͕ਓ޻σʔλͱϦΞϧσʔλͰޮՌ͕͋Δ͜ͱΛ࣮ݧͨ͠ɽ ֓ཁ ߩݙ
8.  wఏҊख๏Ͱ͸ɼϞσϧʹґଘ͠ͳ͍ͨΊճؼɼ෼ྨϞσϧͷͲͪΒʹ΋దԠͰ͖Δɽ ʢࠓճ͸෼ྨϞσϧͰߟ͑Δʣ
   wલఏͱͯ͠ɼϞσϧͷग़ྗΛ৚݅෇͖֬཰ ɼઆ໌ม਺ 

   ֬཰ม਺ ͱ͢Δɽ
   w͜͜Ͱ͸ɼ૬ޓ৘ใྔΛΠϯελϯε͝ͱͷಛ௃બ୒ͷࢦඪͱͯ͠࢖͏ɽ
   
   આ໌ม਺ͱબ୒͞Εͨಛ௃ྔͷ૬ޓ৘ใྔ͕࠷େͱͳΔ͜ͱ͕ࠓճͷ໨ඪɽ ℙm ( ⋅ ∣ x) Y X = x ∈ ℝd I(X; Y) = X,Y [log pXY (X, Y) pX (X)pY (Y) ] ϑϨʔϜϫʔΫ
9.  w 
   पล෼෍͔Βग़ͯ͘Δɽ
   w 
   ৚݅෇͖෼෍ ͔Βग़ͯ͘Δɽ
   w 
   αΠζ͕ Ͱ͋Δ΂͖ू߹ɽಛ௃બ୒ʹ࢖͏ɽ
   w ͸ϋΠύʔύϥϝʔλ
   w 
   ͔Β΂͖ू߹ʹϚοϐϯά͢Δઆ໌Ϟσϧɽ
   w 
   Ͱબ͹Εͨ෦෼ू߹ɼ͜ΕʹΑͬͯબ୒͞Εͨಛ௃ͷ֬཰ ม਺ ΛಘΔɽ
   X X ∼ ℙX ( ⋅ ) Y (Y ∣ x) ∼ ℙm ( ⋅ ∣ x) ℘k = {S ⊂ 2d ||S ∣ = k} k k ℰ ℝd S S = ℰ(x) xs ϑϨʔϜϫʔΫ ໰୊ઃఆ
10.  w৽͍֬͠཰ϕΫτϧ Λ༻͍Δͱࠓճͷ໨త͸࣍ࣜΛ࠷దԽ͢Δ͜ͱʹͳ Δɽ
    
    w૬ޓ৘ใྔΛ࠷େʹ͢ΔΑ͏ͳɼಛ௃ͷ෦෼ۭ͕ؒࣝผϞσϧͰॏཁͳಛ௃Ͱ͋ Δͱઆ໌͢Δ͜ͱ͕Ͱ͖Δɽ Xs ∈ Rk ϑϨʔϜϫʔΫ ໨తؔ਺
11.  w໨తؔ਺Λ௚઀ղ͘͜ͱ͕ࠔ೉Ͱ͋Δ͔Βɼม෼ۙࣅΛ༻͍ͯղ͘ɽ
    w૬ޓ৘ใྔͷԼքΛ࠷େԽ͢Δ͜ͱΛ໨ඪʹͳΔɽ
    ఏҊख๏
12.  w ҰൠతͳϞσϧͰ͸ɼ৚݅෇͖෼෍ͷԼքͷظ଴஋Λܭࢉ͢Δ͜ͱ͕ࠔ೉Ͱ͋Δͨ Ίɼ࣍ͷΑ͏ͳม෼଒Λఆٛ͢Δɽ
    w
    
    w +FOTFOͷෆ౳ࣜʹΑΓɼԼք͕࣍ͷΑ͏ʹมߋ͞ΕΔɽ
    w := {ℚ ∣ ℚ = {xS → ℚS (Y ∣ xS), S ∈ ℘k}} ఏҊख๏ ม෼Լք
13.  w Λ༻͍Δͱɼࠓճͷ໨ඪ͕࣍ͷΑ͏ʹมΘΔɽ
    w
    wҰൠతͳ ͷ৔߹ɼࣜ  Λղ͘ͷ͸͍·ͩʹࠔ೉Ͱ͋Δɽͦ͜Ͱ࠷దԽՄೳͳ ํ๏ʹม͑Δඞཁ͕͋Δɽ ℰ, ఏҊख๏ ࠷దԽ໰୊
14.  w ΛύϥϝʔλԽ͢ΔͨΊʹɼχϡʔϥϧωοτϫʔΫΛఆٛ͢Δɽ
    w
    
    wࠓճఆٛͨ͠ Λ༻͍ͯɼ ͱ͢Δɽ gα : ℝd × [c] → [0,1] where[c] = 0,1,…, c − 1 gα ℚx := gα (˜ xs , Y) ఏҊख๏ ΛύϥϝʔλԽ͢Δ
15.  wࣜ  Λ௚઀ਪఆ͢Δʹ͸ɼ ͜ͷ෦෼ۭؒΛ߹ܭ͢Δඞཁ͕͋Γେมɽ
    w͜ΕΛղܾ͢ΔͨΊʹɼ(VNCFMTPGUNBY
    USJDLΛ༻͍Δɽ
    wΧςΰϦΧϧ෼෍Λ࿈ଓʹͳΔΑ͏ʹ؇࿨͢Δख๏
    w(VNCFMTPGUNBY
    USJDLΛ࢖ͬͯɼॏΈ͚ͮΒΕͨ෦෼ۭؒͷαϯϓϦϯάͷۙࣅ Λߦͳ͍ͬͯ͘ɽ n Ck ఏҊख๏ αϒηοταϯϓϦϯάͷ࿈ଓ؇࿨ Gumbel-softmax trickͷࢀߟ: http://peluigi.hatenablog.com/entry/2018/06/23/192435
16.  w ͜ͷಛ௃͔Β ͜ͷಛ௃ΛαϯϓϦϯά͢Δํ๏Λ࣍ʹࣔ͢ɽ
     ͜ͷಛ௃ϕΫτϧΛಠཱʹ ճαϯϓϦϯάΛߦ͏ɽ
     ͔Ϳͬͨಛ௃͸࡟আͯ͠࢒ΓΛ࢒͢ɽ
    w͜͏͢Δͱ͍͍ͤͥ ͚ͩಛ௃͕࢒Δɽ͜ΕΒΛࣜʹ͢Δͱ࣍ͷΑ͏ʹͳΔɽ
    d k d k k ఏҊख๏ ಛ௃ΛL͚ͩநग़͢Δ
17.  w࠷ऴతʹɼ࠷దԽ͢Δ΂͖໨తͷ΋ͷ͸࣍ͷ΋ͷͰ͋Δɽ
    w
    w ͸ ʹґଘ͠ͳ͍΋ͷͰ͋Δ͔Βɼ܇࿅࣌Ͱ͸ɼಉ࣌ʹޯ഑߱ԼΛద༻͢Δ ͜ͱ͕Ͱ͖Δɽ X,ζ θ, α ఏҊख๏ ࠷ऴతͳ໨తؔ਺ͱͦͷ࠷దԽ
18.  wֶश͞Εͨઆ໌ϞσϧʹΑͬͯɼαϯϓϧ ͔ΒॏΈϕΫτϧ ࣍ݩͷ ʹϚο ϐϯά͞ΕΔɽ
    w͔͜͜ΒείΞ ͷେ͖͍ํΛબ୒ͯ͠ɼઆ໌༻ͷಛ௃ͱ͢Δɽ
    w֤αϯϓϧʹରͯ͠ɼઆ໌ϞσϧΛ௨͚ͩ͢Ͱɼಛ௃Λબ୒͢Δ͜ͱ͕Ͱ͖Δͷ ͰɼطଘݚڀΑΓ΋ޮ཰Α͘આ໌Ͱ͖ɼ͔ͭϞσϧʹґଘ͠ͳ͍Ͱ͓͜ͳ͑Δɽ X d wθ (X) pθ (X) ఏҊख๏ આ໌εςʔδ
19.  wਓ޻σʔλͱ࣮ࡍͷσʔλͷೋͭͰ࣮ݧΛߦ͏ɽ
    wਓ޻σʔλͰ͸ɼ࣍ݩΨ΢ε෼෍͔Βੜ੒ͯ͠ɼ͔ͦ͜ΒछྨͷσʔληοτΛ࡞੒ɽ
    w ࣮ݧ σʔληοτ
20.  wຊݚڀͷख๏ -9 ͱطଘख๏ %FFQ-*'5
    4)"1
    -*.&
    ʜ Λൺֱ͢Δɽ
    ࣮ݧ ࣮ݧ݁Ռ
21.  wઆ໌࣌ͷ࣮ߦ࣌ؒͷ݁ՌΛҎԼʹࣔ͢ɽ
    ࣮ݧ ࣮ݧ݁Ռ
22.  ࣮ݧ݁Ռ ςΩετσʔλͰͷϋΠϥΠτ өըϨϏϡʔͷσʔληοτ
23.  wөըϨϏϡʔͷจʹରͯ͠ɼΫϥ΢υϫʔΧʹݸͷײ৘ʹ߆ΔΩʔϫʔυΛબ ୒ͯ͠΋Β͏ɽ
    wෳ਺ͷจষΛෳ਺ͷϫʔΧʹධՁͯ͠΋Β͍ɼͦͷฏۉΛ)VNBO
    BDDVSBDZͱ͢ Δɽ
    ࣮ݧ݁Ռ Ϋϥ΢υϫʔΧʹධՁͯ͠΋Β͏
24.  w૬ޓ৘ใྔΛجʹͯ͠ɼΠϯελϯε͝ͱͷಛ௃બ୒Λ͢ΔϑϨʔϜϫʔΫΛఏ Ҋͨ͠ɽ
    w-9ʹΑΓɼ͸͡ΊͯϦΞϧλΠϜͰϒϥοΫϘοΫεͳϞσϧΛઆ໌͢Δ͜ͱ ͕Մೳʹͳͬͨɽ ݁࿦