Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[読み会] Knowledge distillation: A good teacher is...
Search
mei28
October 25, 2022
0
69
[読み会] Knowledge distillation: A good teacher is patient and consistent
読み会資料
Knowledge distillation: A good teacher is patient and consistent(CVPR2022)
mei28
October 25, 2022
Tweet
Share
More Decks by mei28
See All by mei28
[読み会] CHI2025論文紹介
mei28
1
22
[読み会] “Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making
mei28
0
120
[JSAI'24] 人間の判断根拠は文脈によって異なるのか?〜信頼されるXAIに向けた人間の判断根拠理解〜
mei28
2
630
[CHI'24] Fair Machine Guidance to Enhance Fair Decision Making in Biased People
mei28
0
82
[DEIM2024] 卓球の得点予測における重要要素の分析
mei28
0
50
[Human-AI Decision Making勉強会] 意思決定 with AIは個人vsグループで変わるの?
mei28
0
230
[読み会] Words are All You Need? Language as an Approximation for Human Similality Judgements
mei28
0
50
[参加報告] AAAI'23
mei28
0
110
[計算機構論] Learning Models of Individual Behavior in Chess
mei28
0
84
Featured
See All Featured
Measuring & Analyzing Core Web Vitals
bluesmoon
7
530
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
47
9.6k
Practical Orchestrator
shlominoach
189
11k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
109
19k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Documentation Writing (for coders)
carmenintech
72
4.9k
Done Done
chrislema
184
16k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Rails Girls Zürich Keynote
gr2m
95
14k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Transcript
ಡΈձˏ ༶໌ ,OPXMFEHFEJTUJMMBUJPO "HPPEUFBDIFSJTQBUJFOUBOEDPOTJTUFOU
จใͱબཧ༝ • จใ • બཧ༝ • ࣝৠཹͷઃఆͷڭࢣͱੜెͷؔɼ ػցڭࣔͱྨࣅ͍ͯ͠ΔͨΊ
$7ٕज़ൃల͍ͯ͠Δ͚Ͳɼ࣮ࣾձͰ͍ͮΒ͍ • എܠɿେنϞσϧੑೳ͕ߴ͍͕ɼܭࢉίετɼઃඋίε τ͕ߴ͗͢Δ • ˠݱ࣮తʹখ͞ͳϞσϧͷํ͕͍উख͕ྑ͘ॏཁ • େنϞσϧͷೳྗΛ׆༻͢ΔͨΊʹɼੑೳҡ࣋ͨ͠··
ϞσϧΛܰྔԽ͢Δ͜ͱ͕ٻΊΒΕ͍ͯΔ
,%ʮڭࢣͱੜెͷϞσϧϚονϯάʯͱղऍ • ϞσϧͷܰྔԽʹͭͷख๏͕༗໊ • .PEFMQSVOJOH • ϞσϧͷҰ෦ΛΧοτ͢Δ͜ͱͰɼܰྔԽΛ͓͜ͳ͏ɽ • ϞσϧͷߏΛมߋͰ͖ͳ͍FH3FT/FUˠ.PCJMF/FU
• ,OPXMFEHFEJTUJMMBUJPO • ڭࢣʢେ͖Ίʣͱੜెʢখ͞ΊʣΛઃఆ͠ɽੜెڭࢣͱ ಉ͡ೳྗΛࢦ͢
ࣝৠཹɼೖྗσʔλͱֶश͕࣌ؒେࣄ • ͜͜ͰɼϞσϧΛܰྔԽ͢ΔͨΊʹɼࣝৠཹΛར༻ • ࣝৠཹʹ͓͍ͯԿ͕େࣄͳͷ͔Λݟ͚ͭΔ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍
• ֶश࣌ؒଟ͍ํ͕ྑ͍
ઃఆɾ࣮ݧઃఆ • ධՁࢦඪʹը૾ͷΫϥεྨਫ਼Λ༻͍Δ • େنϞσϧΛڭࢣͱઃఆ͠ɼྨਫ਼Λམͱͣ͞ʹɼϞσ ϧͷܰྔԽΛࢦ͢ɽ • ڭࢣɿ#J5ͷࣄલֶशϞσϧ •
ੜెɿ3FT/FU • σʔληοτछྨ খதنͭɼେنͭ • Ϋϥε σʔλαΠζ
ڭࢣͱੜెͷग़ྗΛ͚ۙͮΔ • ࣝৠཹͷֶशʹ༻͍ΔଛࣦͰ,-μΠόʔδΣϯε༻͍Δ • ͦΕͧΕڭࢣͷ༧ଌɼੜెͷ༧ଌ • Ϟσϧͷग़ྗࣗମɼԹ͖ιϑτϚοΫε pt ,
ps KL(pt ||ps ) = ∑ i∈C [−pt,i log ps,i + pt,i log pt,i ]
ϋΠύϥઃఆͱ͔࣮ݧͷখٕͨͪ • ࠷దԽख๏ͱͯ͠"EBNΛ༻͍ΔɽϋΠύϥσϑΥϧτ • εέδϡʔϥ$PTJOFEFDBZΛ༻͍Δ • ֶश҆ఆͷͨΊɼޯͷେ͖͞Λʹ੍ݶ • όοναΠζΛʹઃఆɽͨͩ͠*NBHF/FU
.JYVQʹΑͬͯσʔλ֦ுΛ͢Δ • ͭͷ܇࿅σʔλͷϖΞΛࠞ߹ͯ͠৽͘͠σʔλΛ࡞<> • ܇࿅σʔλ͕ϛοΫε͞ΕΔ͚ͩͰͳ͘ɼϥϕϧϛοΫε • • ͔ΒҰ༷ʹαϯϓϦϯά͢Δ
ຊՈ X = λX1 + (1 − λ)X2 , y = λy1 + (1 − λ)y2 λ ∈ [0,1] λ ∼ Beta(α, α) IUUQTBSYJWPSHBCT
ը૾ͷલॲཧ • σʔλͷલॲཧͰJODFQUJPOTUZMFDSPQΛ༻͍Δ • ݩσʔλͷը૾αΠζͷ ͰϥϯμϜʹΫϦοϓ • ΫϦοϓͨ͠ޙϥϯμϜͳΞεϖΫτൺʹม
• มͨ͋͠ͱݻఆαΠζ Y ʹϦαΠζ IUUQTBSYJWPSHBCT
ओுͷ͓͞Β͍ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍ • ֶश࣌ؒଟ͍ํ͕ྑ͍ • ϩόετੑΛݟΔͨΊʹখதنσʔληοτͭΛར༻
• ަབྷҼࢠΛഉআ͢ΔͨΊʹ͞·͟·ͳϋΠύϥͰ࣮ݧ 1BUJFOU UFBDIFS $POTJTUFOU UFBDIFS
$POTJTUFOUUFBDIFS Ұ؏ͨ͠ڭࢣ ʹ͍ͭͯΈΔ • ڭࢣɼੜెͷઃఆΛͭ༻ҙ͢Δɽ • 'JYFEUFBDIFSɿڭࢣͷग़ྗ͕ݻఆ͞Ε͍ͯΔ • *OEFQFOEFOUOPJTFɿڭࢣɼੜెͰҟͳΔೖྗ
• $POTJTUFOUUFBDIJOHɿڭࢣͱੜెͰಉ͡ೖྗ • 'VODUJPONBUDIJOHɿ$POTJTUFOUUFBDIJOHʴσʔλ֦ு • Լೋ͕ͭҰ؏ͨ͠ڭࢣઃఆΛද͍ͯ͠Δɽ
Ұ؏ͨ͠ڭࢣਫ਼͕ߴ͍ଞաֶशΛى͜͢
1BUJFOUUFBDIFS ڧ͍ڭࢣ ʹ͍ͭͯΈΔ • Ұൠతͳڭࢣ͋Γֶशͷ߹ɼϥϕϧʹର͠ɼը૾͕େ͖͘ ΉՄೳੑ͕͋Δɽ • ͜͜ͰɼࣝৠཹڭࢣͱੜెͷؔϚονϯάͱղऍ •
Ұ؏ͯ͠ಉ͡ೖྗΛ༩͑ΔͳΒɼೖྗࣗମΜͰ͍͍ ˠσʔλ֦ு͕༗ޮͳͷͰ • σʔλ֦ுΛߦ͍ɼେ͖ͳΤϙοΫͰֶश͢ΔͱΑΓੜె Ϟσϧͷੑೳ্͕͢Δ͜ͱΛݕূ͢Δ
աֶशΛճආ͠ͳ͕Βɼੑೳ্͕͢Δ • ͍ઢʢڭࢣʣͱಉͷੑೳʹ౸ୡ • ઢʢసҠֶशʣΑΓ࠷ऴతʹੑೳ͕ྑ͍ • ΦϨϯδʢ͔ΒֶशʣΑΓੑೳ͕͍͍ˠɹࣝৠཹͷޮՌ͕͋Δ
େنσʔληοτʢ*NBHF/FUʣͰࣝৠཹ • Ұ؏ͨ͠ڭࢣաֶश͍ͯ͠ͳ͍ • ·ͨσʔλ֦ு͢Δͱগͳ͍Τϙο ΫͰߴ͍ਫ਼Λୡ • ˠJUFSBUJPOಉ͡
ҟͳΔղ૾Ͱࣝৠཹ͕͏·͍͘͘ • ڭࢣ͕ߴ͍ղ૾ˠੜె͕͍ղ૾ͷํ͕ੑೳ͕ྑ͘ͳΔ
4IBNQPPΛֶͬͯशͨ͠Βߴʹͳͬͨ • σʔλ֦ுʹΑͬͯੑೳ্ͨ͠ ͕ɼͦͷֶश͕࣌ؒ৳ͼͯ͠·͏ • "EBN͔ΒΑΓڧྗͳ࠷దԽख๏Ͱ ͋Δ4IBNQPPΛͬͯΈΔɽ • ˠֶश͕ഒʹͳͬͨʂ
IUUQTBSYJWPSHBCT
సҠֶश͔ͭͬͯࣝৠཹ͍͍Μ͡Όͳ͍ • 1BUJFOUUFBDIFSͷͱ͖ɼసҠֶ श༗ޮ͔ͩͬͨΒࣝৠཹ స Ҡֶश͍͍ͷͰ • ֶशॳظసҠֶशΑ͔͚ͬͨ
Ͳɼ࠷ऴతʹٯస
ࣝৠཹͬͺΓ༗ޮ • ࣝৠཹͬͯແ͍͍ͯͬͯ͘͜ ͱͳ͍ʁ • ࣝৠཹ͕ͳ͍ͱաֶशΛى͜͢
·ͱΊͱײ • ࣝৠཹͰॏཁͳཁૉͱͯ͠ɼ • ɽڭࢣͱੜెͷೖྗ͕ಉ͡ • ɽσʔλ֦ுΛͨ͘͞Μ͢Δ • ɽֶशΤϙοΫΛ૿͢ɹɹɹɹ͜ͱʹ͍ͭͯࣔͨ͠ɽ
• ࣝৠཹͷ৽ख๏ΛఏҊͨ͠Θ͚Ͱͳ͍͕ɼطଘͷϞσϧΛ༻ ͍ͯɼܰྔϞσϧ͕4P5"ΛऔΕΔ͔͠Εͳ͍ϩϚϯΛײ͡Εͨ