Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[読み会] Knowledge distillation: A good teacher is...
Search
mei28
October 25, 2022
0
65
[読み会] Knowledge distillation: A good teacher is patient and consistent
読み会資料
Knowledge distillation: A good teacher is patient and consistent(CVPR2022)
mei28
October 25, 2022
Tweet
Share
More Decks by mei28
See All by mei28
[読み会] CHI2025論文紹介
mei28
1
20
[読み会] “Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making
mei28
0
110
[JSAI'24] 人間の判断根拠は文脈によって異なるのか?〜信頼されるXAIに向けた人間の判断根拠理解〜
mei28
2
610
[CHI'24] Fair Machine Guidance to Enhance Fair Decision Making in Biased People
mei28
0
77
[DEIM2024] 卓球の得点予測における重要要素の分析
mei28
0
49
[Human-AI Decision Making勉強会] 意思決定 with AIは個人vsグループで変わるの?
mei28
0
230
[読み会] Words are All You Need? Language as an Approximation for Human Similality Judgements
mei28
0
48
[参加報告] AAAI'23
mei28
0
100
[計算機構論] Learning Models of Individual Behavior in Chess
mei28
0
81
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
A Modern Web Designer's Workflow
chriscoyier
694
190k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Writing Fast Ruby
sferik
628
62k
Producing Creativity
orderedlist
PRO
346
40k
Fireside Chat
paigeccino
37
3.5k
Documentation Writing (for coders)
carmenintech
72
4.9k
RailsConf 2023
tenderlove
30
1.1k
Gamification - CAS2011
davidbonilla
81
5.3k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Rails Girls Zürich Keynote
gr2m
94
14k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
Transcript
ಡΈձˏ ༶໌ ,OPXMFEHFEJTUJMMBUJPO "HPPEUFBDIFSJTQBUJFOUBOEDPOTJTUFOU
จใͱબཧ༝ • จใ • બཧ༝ • ࣝৠཹͷઃఆͷڭࢣͱੜెͷؔɼ ػցڭࣔͱྨࣅ͍ͯ͠ΔͨΊ
$7ٕज़ൃల͍ͯ͠Δ͚Ͳɼ࣮ࣾձͰ͍ͮΒ͍ • എܠɿେنϞσϧੑೳ͕ߴ͍͕ɼܭࢉίετɼઃඋίε τ͕ߴ͗͢Δ • ˠݱ࣮తʹখ͞ͳϞσϧͷํ͕͍উख͕ྑ͘ॏཁ • େنϞσϧͷೳྗΛ׆༻͢ΔͨΊʹɼੑೳҡ࣋ͨ͠··
ϞσϧΛܰྔԽ͢Δ͜ͱ͕ٻΊΒΕ͍ͯΔ
,%ʮڭࢣͱੜెͷϞσϧϚονϯάʯͱղऍ • ϞσϧͷܰྔԽʹͭͷख๏͕༗໊ • .PEFMQSVOJOH • ϞσϧͷҰ෦ΛΧοτ͢Δ͜ͱͰɼܰྔԽΛ͓͜ͳ͏ɽ • ϞσϧͷߏΛมߋͰ͖ͳ͍FH3FT/FUˠ.PCJMF/FU
• ,OPXMFEHFEJTUJMMBUJPO • ڭࢣʢେ͖Ίʣͱੜెʢখ͞ΊʣΛઃఆ͠ɽੜెڭࢣͱ ಉ͡ೳྗΛࢦ͢
ࣝৠཹɼೖྗσʔλͱֶश͕࣌ؒେࣄ • ͜͜ͰɼϞσϧΛܰྔԽ͢ΔͨΊʹɼࣝৠཹΛར༻ • ࣝৠཹʹ͓͍ͯԿ͕େࣄͳͷ͔Λݟ͚ͭΔ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍
• ֶश࣌ؒଟ͍ํ͕ྑ͍
ઃఆɾ࣮ݧઃఆ • ධՁࢦඪʹը૾ͷΫϥεྨਫ਼Λ༻͍Δ • େنϞσϧΛڭࢣͱઃఆ͠ɼྨਫ਼Λམͱͣ͞ʹɼϞσ ϧͷܰྔԽΛࢦ͢ɽ • ڭࢣɿ#J5ͷࣄલֶशϞσϧ •
ੜెɿ3FT/FU • σʔληοτछྨ খதنͭɼେنͭ • Ϋϥε σʔλαΠζ
ڭࢣͱੜెͷग़ྗΛ͚ۙͮΔ • ࣝৠཹͷֶशʹ༻͍ΔଛࣦͰ,-μΠόʔδΣϯε༻͍Δ • ͦΕͧΕڭࢣͷ༧ଌɼੜెͷ༧ଌ • Ϟσϧͷग़ྗࣗମɼԹ͖ιϑτϚοΫε pt ,
ps KL(pt ||ps ) = ∑ i∈C [−pt,i log ps,i + pt,i log pt,i ]
ϋΠύϥઃఆͱ͔࣮ݧͷখٕͨͪ • ࠷దԽख๏ͱͯ͠"EBNΛ༻͍ΔɽϋΠύϥσϑΥϧτ • εέδϡʔϥ$PTJOFEFDBZΛ༻͍Δ • ֶश҆ఆͷͨΊɼޯͷେ͖͞Λʹ੍ݶ • όοναΠζΛʹઃఆɽͨͩ͠*NBHF/FU
.JYVQʹΑͬͯσʔλ֦ுΛ͢Δ • ͭͷ܇࿅σʔλͷϖΞΛࠞ߹ͯ͠৽͘͠σʔλΛ࡞<> • ܇࿅σʔλ͕ϛοΫε͞ΕΔ͚ͩͰͳ͘ɼϥϕϧϛοΫε • • ͔ΒҰ༷ʹαϯϓϦϯά͢Δ
ຊՈ X = λX1 + (1 − λ)X2 , y = λy1 + (1 − λ)y2 λ ∈ [0,1] λ ∼ Beta(α, α) IUUQTBSYJWPSHBCT
ը૾ͷલॲཧ • σʔλͷલॲཧͰJODFQUJPOTUZMFDSPQΛ༻͍Δ • ݩσʔλͷը૾αΠζͷ ͰϥϯμϜʹΫϦοϓ • ΫϦοϓͨ͠ޙϥϯμϜͳΞεϖΫτൺʹม
• มͨ͋͠ͱݻఆαΠζ Y ʹϦαΠζ IUUQTBSYJWPSHBCT
ओுͷ͓͞Β͍ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍ • ֶश࣌ؒଟ͍ํ͕ྑ͍ • ϩόετੑΛݟΔͨΊʹখதنσʔληοτͭΛར༻
• ަབྷҼࢠΛഉআ͢ΔͨΊʹ͞·͟·ͳϋΠύϥͰ࣮ݧ 1BUJFOU UFBDIFS $POTJTUFOU UFBDIFS
$POTJTUFOUUFBDIFS Ұ؏ͨ͠ڭࢣ ʹ͍ͭͯΈΔ • ڭࢣɼੜెͷઃఆΛͭ༻ҙ͢Δɽ • 'JYFEUFBDIFSɿڭࢣͷग़ྗ͕ݻఆ͞Ε͍ͯΔ • *OEFQFOEFOUOPJTFɿڭࢣɼੜెͰҟͳΔೖྗ
• $POTJTUFOUUFBDIJOHɿڭࢣͱੜెͰಉ͡ೖྗ • 'VODUJPONBUDIJOHɿ$POTJTUFOUUFBDIJOHʴσʔλ֦ு • Լೋ͕ͭҰ؏ͨ͠ڭࢣઃఆΛද͍ͯ͠Δɽ
Ұ؏ͨ͠ڭࢣਫ਼͕ߴ͍ଞաֶशΛى͜͢
1BUJFOUUFBDIFS ڧ͍ڭࢣ ʹ͍ͭͯΈΔ • Ұൠతͳڭࢣ͋Γֶशͷ߹ɼϥϕϧʹର͠ɼը૾͕େ͖͘ ΉՄೳੑ͕͋Δɽ • ͜͜ͰɼࣝৠཹڭࢣͱੜెͷؔϚονϯάͱղऍ •
Ұ؏ͯ͠ಉ͡ೖྗΛ༩͑ΔͳΒɼೖྗࣗମΜͰ͍͍ ˠσʔλ֦ு͕༗ޮͳͷͰ • σʔλ֦ுΛߦ͍ɼେ͖ͳΤϙοΫͰֶश͢ΔͱΑΓੜె Ϟσϧͷੑೳ্͕͢Δ͜ͱΛݕূ͢Δ
աֶशΛճආ͠ͳ͕Βɼੑೳ্͕͢Δ • ͍ઢʢڭࢣʣͱಉͷੑೳʹ౸ୡ • ઢʢసҠֶशʣΑΓ࠷ऴతʹੑೳ͕ྑ͍ • ΦϨϯδʢ͔ΒֶशʣΑΓੑೳ͕͍͍ˠɹࣝৠཹͷޮՌ͕͋Δ
େنσʔληοτʢ*NBHF/FUʣͰࣝৠཹ • Ұ؏ͨ͠ڭࢣաֶश͍ͯ͠ͳ͍ • ·ͨσʔλ֦ு͢Δͱগͳ͍Τϙο ΫͰߴ͍ਫ਼Λୡ • ˠJUFSBUJPOಉ͡
ҟͳΔղ૾Ͱࣝৠཹ͕͏·͍͘͘ • ڭࢣ͕ߴ͍ղ૾ˠੜె͕͍ղ૾ͷํ͕ੑೳ͕ྑ͘ͳΔ
4IBNQPPΛֶͬͯशͨ͠Βߴʹͳͬͨ • σʔλ֦ுʹΑͬͯੑೳ্ͨ͠ ͕ɼͦͷֶश͕࣌ؒ৳ͼͯ͠·͏ • "EBN͔ΒΑΓڧྗͳ࠷దԽख๏Ͱ ͋Δ4IBNQPPΛͬͯΈΔɽ • ˠֶश͕ഒʹͳͬͨʂ
IUUQTBSYJWPSHBCT
సҠֶश͔ͭͬͯࣝৠཹ͍͍Μ͡Όͳ͍ • 1BUJFOUUFBDIFSͷͱ͖ɼసҠֶ श༗ޮ͔ͩͬͨΒࣝৠཹ స Ҡֶश͍͍ͷͰ • ֶशॳظసҠֶशΑ͔͚ͬͨ
Ͳɼ࠷ऴతʹٯస
ࣝৠཹͬͺΓ༗ޮ • ࣝৠཹͬͯແ͍͍ͯͬͯ͘͜ ͱͳ͍ʁ • ࣝৠཹ͕ͳ͍ͱաֶशΛى͜͢
·ͱΊͱײ • ࣝৠཹͰॏཁͳཁૉͱͯ͠ɼ • ɽڭࢣͱੜెͷೖྗ͕ಉ͡ • ɽσʔλ֦ுΛͨ͘͞Μ͢Δ • ɽֶशΤϙοΫΛ૿͢ɹɹɹɹ͜ͱʹ͍ͭͯࣔͨ͠ɽ
• ࣝৠཹͷ৽ख๏ΛఏҊͨ͠Θ͚Ͱͳ͍͕ɼطଘͷϞσϧΛ༻ ͍ͯɼܰྔϞσϧ͕4P5"ΛऔΕΔ͔͠Εͳ͍ϩϚϯΛײ͡Εͨ