Pathologies of Neural Models Make Interpretations Difficult

by Yasufumi Taniguchi

Slide 1

Slide 1 text

ൃදऀ ୩ޱହ࢙ ҟৗͳڍಈ

Slide 2

Slide 2 text

!2 Pathological behavior ࣭໰จ͕did͚ͩͰ΋ Ϟσϧͷग़ྗ͸ಉ͡ ֬౓΋ߴ͍

Slide 3

Slide 3 text

֓ཁ w NLPʹ͓͚ΔχϡʔϥϧϞσϧͷղੳख๏ΛఏҊ w Ϟσϧ͕λεΫΛղ্͘Ͱॏཁͳ୯ޠΛநग़͢Δख๏ w நग़͞Εͨ୯ޠ͸ਓʹͱͬͯ͸ҙຯෆ໌ w ҰํͰϞσϧ͸நग़୯ޠͰ΋ਖ਼͘͠༧ଌ(Pathology) w ղੳ݁Ռʹجͮ͘ਖ਼ଇԽ߲ΛఏҊ w ਖ਼ଇԽ߲ʹΑͬͯϞσϧͷղऍੑ͸޲্ !3

Slide 4

Slide 4 text

໨࣍ Ϟσϧղੳͷطଘख๏ ఏҊख๏ ࣮ݧ ·ͱΊ !4

Slide 5

Slide 5 text

Ϟσϧղੳͷطଘख๏

Slide 6

Slide 6 text

Ϟσϧղੳͷطଘख๏ !6 Adversarial Example Ϟσϧʹਓͷ௚ײʹ൓͢ΔڍಈΛͤ͞Δαϯϓϧ NLPͷλεΫ ओʹQAλεΫ Ͱ͸ύλʔϯ ਓʹͱͬͯ͸ҙຯͷͳ͍มߋ͕ɺϞσϧͷग़ྗΛܹมͤ͞Δέʔε ਓʹͱͬͯ͸໌Β͔ͳมߋͰ΋ɺϞσϧ͕ग़ྗΛม͑ͳ͍έʔε

Slide 7

Slide 7 text

ग़ྗ͕ܹม͢Δέʔε !7 Jia et al., 2017 ΫΥʔλʔόοΫͷ೥ྸʹ͍ͭͯͷ จॻʹΫΥʔλʔόοΫͷഎ൪߸ʹ ؔ͢ΔจΛ௥Ճ Ϟσϧ͸ޡ౴

Slide 8

Slide 8 text

ग़ྗΛม͑ͳ͍έʔε !8 Mudrakarta et al., 2018 ݐ෺ͷന͍ϨϯΨ͕ରশ͔ʁ spherical (ٿঢ়ͷ) ݐ෺ͷന͍ϨϯΨ͕ٿঢ়͔ʁ ࣭໰จͷҙຯมԽ Ϟσϧͷ༧ଌ͸ෆม

Slide 9

Slide 9 text

2. ఏҊख๏

Slide 10

Slide 10 text

*OQVU3FEVDUJPO • ॏཁͰͳ͍୯ޠΛೖྗ͔Β࡟ΓɺϞσϧͷڍಈΛ෼ੳ • Ϟσϧ͕ਖ਼͍͠ग़ྗΛ͢ΔͨΊʹඞཁͳ࠷௿୯ޠ (ॏཁ ୯ޠ) • Adversarial Example͸Ϟσϧʹͱͬͯͷॏཁ୯ޠʹண໨

Slide 11

Slide 11 text

*OQVU3FEVDUJPO !11 x y Ϟσϧͷ༧ଌ f( ⋅ ) χϡʔϥϧϞσϧ ೖྗܥྻ (จ΍จॻ) xi ೖྗܥྻͷ͋Δཁૉ (୯ޠ) g(xi |x) = f(y|x) − f(y|x−i ) ͋Δ୯ޠ ʹର͢Δ ॏཁ౓Λఆٛ xi g i൪໨ͷ୯ޠΛফͨ͠ೖྗ

Slide 12

Slide 12 text

*OQVU3FEVDUJPO !12 g(xcontest |x) = f(y|x) − f(y|x−contest ) What company won free advertisement due to QuickBooks contest ? What company won free advertisement due to QuickBooks contest ? g͕େ͖͚Ε͹ɺcontest͕ॏཁͳ୯ޠͱͳΔ Ϟσϧͷग़ྗʹେ͖͘د༩͍ͯ͠ΔͨΊ

Slide 13

Slide 13 text

*OQVU3FEVDUJPO !13 g(xi |x) = f(y|x) − f(y|x−i ) ॏཁ౓ͷ௿͍୯ޠΛ࡟আ y͕มԽ͠ͳ͍Α͏ʹɺg͕࠷খͱͳΔ୯ޠiΛ࡟আ ͍ͯ͘͠

Slide 14

Slide 14 text

3. ࣮ݧ

Slide 15

Slide 15 text

ղੳͷର৅λεΫ 1. SQuAD w จॻͱ࣭໰จ͕༩͑ΒΕΔˠ࣭໰จʹରͯ͠Input Reduction w จॻ͔Βղ౴Λநग़͢ΔλεΫ 2. SNLI w จ͕༩͑ΒΕΔˠͭͷจʹରͯ͠Input Reduction w จͷؔ܎Λਪఆ͢ΔλεΫ 3. VQA w ը૾ͱ࣭໰จ͕༩͑ΒΕΔˠ࣭໰จʹରͯ͠Input Reduction w ղ౴Λੜ੒͢ΔλεΫ !15

Slide 16

Slide 16 text

࣮ݧ಺༰ Input Reduction w Ϟσϧ͕ਖ਼͍͠ग़ྗΛ͢ΔαϯϓϧΛର৅ʹ࣮ݧ w Input ReductionΛద༻ͨ͠ೖྗ(Reduced)ʹର͢ΔਓखධՁ w ReducedͱϥϯμϜʹ୯ޠΛམͱͨ͠৔߹(Random)ͷࠩҟͷධՁ Regularization on Reduced Inputs w Input ReductionʹΑΔϞσϧͷPathological behaviorΛܰݮ͢Δਖ਼ଇԽ߲ ޙड़ ͷಋೖ !16

Slide 17

Slide 17 text

Reducedʹର͢ΔਓखධՁ !17 Reducedʹରͯ͠ ਓ͸ਖ਼͍͠༧ଌΛͰ ͖ͳ͍ w Reducedʹର͢Δਓͷਖ਼౴཰ w Ϟσϧͷਖ਼౴཰͕ͷαϯϓϧΛ࢖༻

Slide 18

Slide 18 text

Reducedʹର͢ΔਓखධՁ !18 w ReducedͱRandomͷͲͪΒ͕ࣗવͳจ͔ w vs. Random͸fifty-fiftyͱ౴ׂ͑ͨ߹ Reduced͸ਓʹͱͬ ͯ͸Randomͱಉ͡

Slide 19

Slide 19 text

Reducedͷࣄྫ !19 ʮͲ͜Ͱ࿅शͨ͠ ͔ʯΛฉ͔Ε͍ͯ Δͷ͸Θ͔Δ͕ɺ ʮͲͷνʔϜʯ͔ Θ͔Βͳ͍

Slide 20

Slide 20 text

Reducedͷฏۉ୯ޠ਺ ͭͷλεΫͱ΋ɺ ਖ਼౴͢Δͷʹඞཁͳ୯ޠ਺͸ฏۉd

Slide 21

Slide 21 text

Reducedʹର͢ΔϞσϧͷ֬౓ !21 • Input Reductionͷద༻લޙͰϞσϧͷ ֬౓ʹมԽ͸΄ͱΜͲͳ͍ • Ϟσϧ͸Ӷ͍ϐʔΫΛ࣋ͭΑ͏ͳ෼෍ Λֶश͍ͯ͠Δ͜ͱ͕ݪҼ

Slide 22

Slide 22 text

ਖ਼ଇԽ߲ͷಋೖ !22 ∑ (x,y)∈(X,Y) log(f(y|x)) + λ∑ ¯ x∈ ¯ X H(f(y| ¯ x)) Reducedʹରͯ͠ਖ਼͍͠yΛ ग़ྗ͠ʹ͘͘͢Δ ௨ৗͷ໨తؔ਺ Reducedαϯϓϧ͸௨ৗͷ໨తؔ਺Λ࢖ֶͬͯशͨ͠ ϞσϧΛ༻͍ͯੜ੒

Slide 23

Slide 23 text

ਖ਼ଇԽ߲ͷޮՌ !23 • Ϟσϧͷਫ਼౓͕ඍ૿ • ਖ਼౴ʹඞཁͳ୯ޠ਺ ͕૿Ճ

Slide 24

Slide 24 text

ਖ਼ଇԽ߲ͷޮՌ !24 ਓखධՁͷਫ਼౓΋޲্ Input Reductionͨ͠ೖྗ ͷղऍੑ͕޲্

Slide 25

Slide 25 text

ਖ਼ଇԽͨ͠Ϟσϧͷࣄྫ !25 Input Reductionͨ͠ೖྗ͕ਓͰ΋ ղऍՄೳʹͳͬͨ

Slide 26

Slide 26 text

·ͱΊ ఏҊख๏ w NLPͷχϡʔϥϧϞσϧղੳख๏ͱͯ͠ɺInput ReductionΛఏҊ w ༧ଌʹد༩͠ͳ͍୯ޠΛೖྗ͔Β࡟ΓɺϞσϧΛղੳ ࣮ݧ݁Ռ w ఏҊख๏Λద༻ͨ͠ೖྗ͸ਓʹͱͬͯҙຯෆ໌ w ҰํͰχϡʔϥϧϞσϧ͸ਖ਼͍͠༧ଌΛߦ͏ w ਖ਼ଇԽ߲Λಋೖ͢ΔͱϞσϧͷڍಈ͸վળ !26