宝くじ仮説の話を少し / LT Lottery Ticket Hypothesis

Python機械学習勉強会 in 新潟のLTの資料です。深層学習の「宝くじ仮説」について少しだけ話します。

Transcript

1. ೥຤͍ۙͷͰʮๅ͘͡Ծઆʯͷ࿩Λগ͠  1ZUIPOػցֶशษڧձ JO
   ৽ׁ ৽ׁݝ࢈75VCFS 4,"`T
   8FC CJO 1
2. ࢿྉͷϦϯΫͱࢀߟจݙ 2 IUUQTTQFBLFSEFDLDPNTLBTXFCMUMPUUFSZUJDLFUIZQPUIFTJT ࢀߟจݙ ɾIUUQTEFFQTRVBSFKQMPUUFSZUJDLFU ɾIUUQTMBCPSPBJBDUJWJUZDPMVNOFOHJOFFSๅ͘͡Ծઆ ݩ࿦จ ɾJonathan Frankle, Michael Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, cs.LG : 1803.03635, ICLR 2019 ɾ5JBOMPOH
   $IFO
   +POBUIBO
   'SBOLMF
   4IJZV $IBOH
   4JKJB -JV
   :BOH
   ;IBOH
   ;IBOHZBOH 8BOH
   .JDIBFM
   $BSCJO l5IF
   -PUUFSZ
   5JDLFU
   )ZQPUIFTJT
   GPS
   1SFUSBJOFE
   #&35
   /FUXPSLTz
3. 5XJUUFS࿈ܞ 3 #pyml_niigata ͰͭͿ΍͘ͱʜʜ
4. ࢈ۀ 4 ɾ ڊେ ͳϞσϧʢ#&35ɺ5ɺFUDʣ͕ͳͥਫ਼౓ྑ͍ͷ͔ʁ ɾύϥϝʔλʔ͕ଟ͍΄ͲΨνϟ͕͋ͨΔʂʁ ɾԠ༻͢Δͱ&EHF
   "*Ͱߴਫ਼౓ͳϞσϧΛಈ͔ͤΔʁ
5. ڊେͳϞσϧ 5
6. ڊେͳϞσϧ 6 ɾ͜ΜͳʹେྔͷύϥϝʔλʔΛ͏·͘ߋ৽͢Δ͜ͱͰ͖Δͷʁ ɾաֶश͠ͳ͍ͷʁ ๅ͘͡Ծઆ
7. ๅ͘͡Ծઆ 7 ɾχϡʔϥϧωοτϫʔΫ͸ॳظͷॏΈ͕ཚ਺ ɾ࠷ॳ͔Β͍͍ײ͡ʹֶशՄೳͳ෦෼ωοτϫʔΫ͕͋Δ ɾ͋ͨΓ͘͡ωοτϫʔΫΛத৺ʹֶश͞ΕΔ ɾ͸ͣΕ͘͡ωοτϫʔΫ͸ࢬמΓͯ͠΋ਫ਼౓͸΄΅མͪͳ͍ ɾ#&35Ͱ͸ׂʙׂͷলύϥϝʔλʔ͕Մೳʢʁʣ ɾߴ଎ʹֶशɺਪ࿦͕Ͱ͖Δ ɾ&EHF
   "*Ͱ΋ಈ͔ͤΔʁ
8. Ϟσϧͷ࡟ݮ 8 ʢ·ͩௐ΂͍ͯ·ͤΜ PS[ ʣ ʢ౔ʣʹಈը౤ߘ͠·͢ ʢࠓ͙͢஌Γ͍ͨΜ͚ͩͲͬͯਓ͸ࢀߟจݙ΍ݩ࿦จΛಡΜͰΈ͍ͯͩ͘͞ʣ
9. ·ͱΊ 9 ɾχϡʔϥϧωοτϫʔΫͷॳظͷॏΈ͸ཚ਺ ɾ͋ͨΓωοτϫʔΫͱ͸ͣΕωοτϫʔΫ͕͋Δ ɾ࿈ΨνϟͰ౰ͨΒͳ͍ʁ ԯ࿈͘Β͍΍Ε͹͚ͬ͜͏౰ͨΔΑʂ ɾ͍ͬͯ͏ͷ͕࠷ۙͷ ڊେϞσϧʢ#&35ɺ5ɺFUDʣ ɾ͸ͣΕωοτϫʔΫΛࢬמΓ͢Ε͹গͳ͍ετϨʔδͰߴ଎ʹਪ࿦Ͱ͖Δ ɾ͡Ό͋Ͳ͏΍Δͷʁ ˠ ·ͨࠓ౓ͳʂ
10. ࢿྉͷϦϯΫͱࢀߟจݙ 10 IUUQTTQFBLFSEFDLDPNTLBTXFC΄͛΄͛ ࢀߟจݙ ɾIUUQTEFFQTRVBSFKQMPUUFSZUJDLFU ɾIUUQTMBCPSPBJBDUJWJUZDPMVNOFOHJOFFSๅ͘͡Ծઆ ݩ࿦จ ɾJonathan Frankle, Michael Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, cs.LG : 1803.03635, ICLR 2019 ɾ5JBOMPOH
    $IFO
    +POBUIBO
    'SBOLMF
    4IJZV $IBOH
    4JKJB -JV
    :BOH
    ;IBOH
    ;IBOHZBOH 8BOH
    .JDIBFM
    $BSCJO l5IF
    -PUUFSZ
    5JDLFU
    )ZQPUIFTJT
    GPS
    1SFUSBJOFE
    #&35
    /FUXPSLTz