Upgrade to Pro — share decks privately, control downloads, hide ads and more …

宝くじ仮説の話を少し / LT Lottery Ticket Hypothesis

宝くじ仮説の話を少し / LT Lottery Ticket Hypothesis

Python機械学習勉強会 in 新潟のLTの資料です。深層学習の「宝くじ仮説」について少しだけ話します。

307839dcba61e984dcab7ae753990218?s=128

SKA’s Web

November 14, 2021
Tweet

Transcript

  1. ೥຤͍ۙͷͰʮๅ͘͡Ծઆʯͷ࿩Λগ͠  1ZUIPOػցֶशษڧձ JO৽ׁ ৽ׁݝ࢈75VCFS 4,"`T8FC CJO 1

  2. ࢿྉͷϦϯΫͱࢀߟจݙ 2 IUUQTTQFBLFSEFDLDPNTLBTXFCMUMPUUFSZUJDLFUIZQPUIFTJT ࢀߟจݙ ɾIUUQTEFFQTRVBSFKQMPUUFSZUJDLFU ɾIUUQTMBCPSPBJBDUJWJUZDPMVNOFOHJOFFSๅ͘͡Ծઆ ݩ࿦จ ɾJonathan Frankle, Michael

    Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, cs.LG : 1803.03635, ICLR 2019 ɾ5JBOMPOH$IFO +POBUIBO'SBOLMF 4IJZV $IBOH 4JKJB -JV :BOH;IBOH  ;IBOHZBOH 8BOH .JDIBFM$BSCJO l5IF-PUUFSZ5JDLFU)ZQPUIFTJTGPS1SFUSBJOFE#&35/FUXPSLTz
  3. 5XJUUFS࿈ܞ 3 #pyml_niigata ͰͭͿ΍͘ͱʜʜ

  4. ࢈ۀ 4 ɾ ڊେ ͳϞσϧʢ#&35ɺ5ɺFUDʣ͕ͳͥਫ਼౓ྑ͍ͷ͔ʁ ɾύϥϝʔλʔ͕ଟ͍΄ͲΨνϟ͕͋ͨΔʂʁ ɾԠ༻͢Δͱ&EHF"*Ͱߴਫ਼౓ͳϞσϧΛಈ͔ͤΔʁ

  5. ڊେͳϞσϧ 5

  6. ڊେͳϞσϧ 6 ɾ͜ΜͳʹେྔͷύϥϝʔλʔΛ͏·͘ߋ৽͢Δ͜ͱͰ͖Δͷʁ ɾաֶश͠ͳ͍ͷʁ ๅ͘͡Ծઆ

  7. ๅ͘͡Ծઆ 7 ɾχϡʔϥϧωοτϫʔΫ͸ॳظͷॏΈ͕ཚ਺ ɾ࠷ॳ͔Β͍͍ײ͡ʹֶशՄೳͳ෦෼ωοτϫʔΫ͕͋Δ ɾ͋ͨΓ͘͡ωοτϫʔΫΛத৺ʹֶश͞ΕΔ ɾ͸ͣΕ͘͡ωοτϫʔΫ͸ࢬמΓͯ͠΋ਫ਼౓͸΄΅མͪͳ͍ ɾ#&35Ͱ͸ׂʙׂͷলύϥϝʔλʔ͕Մೳʢʁʣ ɾߴ଎ʹֶशɺਪ࿦͕Ͱ͖Δ ɾ&EHF"*Ͱ΋ಈ͔ͤΔʁ

  8. Ϟσϧͷ࡟ݮ 8 ʢ·ͩௐ΂͍ͯ·ͤΜ PS[ ʣ ʢ౔ʣʹಈը౤ߘ͠·͢ ʢࠓ͙͢஌Γ͍ͨΜ͚ͩͲͬͯਓ͸ࢀߟจݙ΍ݩ࿦จΛಡΜͰΈ͍ͯͩ͘͞ʣ

  9. ·ͱΊ 9 ɾχϡʔϥϧωοτϫʔΫͷॳظͷॏΈ͸ཚ਺ ɾ͋ͨΓωοτϫʔΫͱ͸ͣΕωοτϫʔΫ͕͋Δ ɾ࿈ΨνϟͰ౰ͨΒͳ͍ʁ ԯ࿈͘Β͍΍Ε͹͚ͬ͜͏౰ͨΔΑʂ ɾ͍ͬͯ͏ͷ͕࠷ۙͷ ڊେϞσϧʢ#&35ɺ5ɺFUDʣ ɾ͸ͣΕωοτϫʔΫΛࢬמΓ͢Ε͹গͳ͍ετϨʔδͰߴ଎ʹਪ࿦Ͱ͖Δ ɾ͡Ό͋Ͳ͏΍Δͷʁ

    ˠ ·ͨࠓ౓ͳʂ
  10. ࢿྉͷϦϯΫͱࢀߟจݙ 10 IUUQTTQFBLFSEFDLDPNTLBTXFC΄͛΄͛ ࢀߟจݙ ɾIUUQTEFFQTRVBSFKQMPUUFSZUJDLFU ɾIUUQTMBCPSPBJBDUJWJUZDPMVNOFOHJOFFSๅ͘͡Ծઆ ݩ࿦จ ɾJonathan Frankle, Michael

    Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, cs.LG : 1803.03635, ICLR 2019 ɾ5JBOMPOH$IFO +POBUIBO'SBOLMF 4IJZV $IBOH 4JKJB -JV :BOH;IBOH  ;IBOHZBOH 8BOH .JDIBFM$BSCJO l5IF-PUUFSZ5JDLFU)ZQPUIFTJTGPS1SFUSBJOFE#&35/FUXPSLTz