Slide 52
Slide 52 text
7. ౷ҰޯϑϨʔϜϫʔΫʹΑΔSiamese Self-Supervised LearningͷՁੑͷ୳ٻ
(ݪจ: Exploring the Equivalence of Siamese Self-Supervised Learning via A Uni
fi
ed Gradient
Framework)
ࣗݾڭࢣֶ͖शɺਓؒͷΞϊςʔγϣϯͳ͠Ͱڧྗͳࢹ֮දݱΛநग़͢ΔͨΊͷେ͖ͳՄೳੑΛ͍ࣔͯ͠Δɻࣗ
ݾڭࢣֶ͖शΛ༷ʑͳ؍͔Βѻ͏ͨΊʹɺ༷ʑͳ࡞͕ఏҊ͞Ε͍ͯΔɻ(1) ରൺֶश๏ʢMoCo, SimCLRͳͲʣ
ɺֶशͷํੑΛܾΊΔͨΊʹਖ਼ෛ྆ํͷαϯϓϧΛར༻͢Δɻ(2) ඇରশωοτϫʔΫ๏ʢBYOL, SimSiamͳ
Ͳʣɺ༧ଌωοτϫʔΫͷಋೖͱఀࢭޯૢ࡞ʹΑͬͯෛͷαϯϓϧΛऔΓআ͘ɻ(3) ಛ০๏ʢBarlow Twins,
VICRegͳͲʣɺಛ࣍ݩؒͷੑΛݮΒ͢͜ͱΛతͱ͍ͯ͠Δɻ͜ΕΒͷख๏ɺ༷ʑͳಈػ͔Βɺઃܭ͞
Εͨଛࣦ͕͔ؔͳΓҟͳ͍ͬͯΔΑ͏Ͱ͢ɻ·ͨɺ࠷ऴతͳਫ਼༷ʑͰɺ࡞ʹΑͬͯҟͳΔωοτϫʔΫτ
ϦοΫ͕ར༻͞Ε͍ͯ·͢ɻຊݚڀͰɺ͜ΕΒͷख๏͕ಉ͡ܗࣜʹ౷ҰͰ͖Δ͜ͱΛࣔ͠·͢ɻͦΕͧΕͷଛࣦؔ
Λൺֱ͢ΔͷͰͳ͘ɺޯੳʹΑͬͯ౷Ұ͞ΕͨࣜΛಋ͖ग़͠·͢ɻ͞ΒʹɺެฏͰৄࡉͳ࣮ݧΛߦ͍ɺ྆ऀ
ͷੑೳΛൺֱ͠·ͨ͠ɻͦͷ݁Ռɺ͜ΕΒͷख๏ͷؒʹ΄ͱΜͲΪϟοϓ͕ͳ͘ɺϞϝϯλϜΤϯίʔμͷ༻͕
ੑೳΛ্ͤ͞ΔॏཁͳཁૉͰ͋Δ͜ͱ͕Θ͔ͬͨɻ͜ͷ౷Ұ͞ΕͨϑϨʔϜϫʔΫ͔ΒɺզʑUniGradΛఏҊ͠
·͢ɻUniGradɺࣗݾڭࢣֶ͖शͷͨΊͷγϯϓϧͰޮՌతͳޯܗࣜͰ͢ɻUniGradɺϝϞϦόϯΫ༧ଌ
ωοτϫʔΫΛඞཁͱ͠ͳ͍͕ɺ࠷ઌͷੑೳΛୡ͢Δ͜ͱ͕Ͱ͖ɺଞͷֶशઓུΛ༰қʹ࠾༻͢Δ͜ͱ͕Ͱ͖
Δɻ·ͨɺઢܗධՁଟ͘ͷԼྲྀλεΫͰͷൣͳ࣮ݧʹΑΓɺͦͷ༗ޮੑ͕ࣔ͞Ε͍ͯ·͢ɻίʔυΛެ։͢Δɻ
w తɿࣗݾڭࢣֶ͖ͭशΞϧΰϦζϜͷ౷ҰԽ
w ํ๏ɿ֤ࣗݾڭࢣֶ͖शख๏Λੳ
w Ռɿࣗݾڭࢣֶ͖ͭशͷͨΊͷڞ௨ͷࣜΛಋ͖ग़ͯ͠ɺطଘͱ΄΅ಉͷਫ਼ͩͬͨ
w ݻ༗໊ɿ6OJ(SBE
w ஶऀॴଐɿਗ਼՚େֶ4FOTF5JNF3FTFBSDIᔳߐେֶژਓೳݚڀӃ
http://arxiv.org/abs/2112.05141v1