Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[読み会] Knowledge distillation: A good teacher is...
Search
mei28
October 25, 2022
0
65
[読み会] Knowledge distillation: A good teacher is patient and consistent
読み会資料
Knowledge distillation: A good teacher is patient and consistent(CVPR2022)
mei28
October 25, 2022
Tweet
Share
More Decks by mei28
See All by mei28
[読み会] CHI2025論文紹介
mei28
1
20
[読み会] “Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making
mei28
0
110
[JSAI'24] 人間の判断根拠は文脈によって異なるのか?〜信頼されるXAIに向けた人間の判断根拠理解〜
mei28
2
610
[CHI'24] Fair Machine Guidance to Enhance Fair Decision Making in Biased People
mei28
0
77
[DEIM2024] 卓球の得点予測における重要要素の分析
mei28
0
49
[Human-AI Decision Making勉強会] 意思決定 with AIは個人vsグループで変わるの?
mei28
0
230
[読み会] Words are All You Need? Language as an Approximation for Human Similality Judgements
mei28
0
48
[参加報告] AAAI'23
mei28
0
100
[計算機構論] Learning Models of Individual Behavior in Chess
mei28
0
81
Featured
See All Featured
GitHub's CSS Performance
jonrohan
1031
460k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Visualization
eitanlees
146
16k
Making the Leap to Tech Lead
cromwellryan
134
9.4k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Making Projects Easy
brettharned
116
6.3k
Building an army of robots
kneath
306
45k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Statistics for Hackers
jakevdp
799
220k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Transcript
ಡΈձˏ ༶໌ ,OPXMFEHFEJTUJMMBUJPO "HPPEUFBDIFSJTQBUJFOUBOEDPOTJTUFOU
จใͱબཧ༝ • จใ • બཧ༝ • ࣝৠཹͷઃఆͷڭࢣͱੜెͷؔɼ ػցڭࣔͱྨࣅ͍ͯ͠ΔͨΊ
$7ٕज़ൃల͍ͯ͠Δ͚Ͳɼ࣮ࣾձͰ͍ͮΒ͍ • എܠɿେنϞσϧੑೳ͕ߴ͍͕ɼܭࢉίετɼઃඋίε τ͕ߴ͗͢Δ • ˠݱ࣮తʹখ͞ͳϞσϧͷํ͕͍উख͕ྑ͘ॏཁ • େنϞσϧͷೳྗΛ׆༻͢ΔͨΊʹɼੑೳҡ࣋ͨ͠··
ϞσϧΛܰྔԽ͢Δ͜ͱ͕ٻΊΒΕ͍ͯΔ
,%ʮڭࢣͱੜెͷϞσϧϚονϯάʯͱղऍ • ϞσϧͷܰྔԽʹͭͷख๏͕༗໊ • .PEFMQSVOJOH • ϞσϧͷҰ෦ΛΧοτ͢Δ͜ͱͰɼܰྔԽΛ͓͜ͳ͏ɽ • ϞσϧͷߏΛมߋͰ͖ͳ͍FH3FT/FUˠ.PCJMF/FU
• ,OPXMFEHFEJTUJMMBUJPO • ڭࢣʢେ͖Ίʣͱੜెʢখ͞ΊʣΛઃఆ͠ɽੜెڭࢣͱ ಉ͡ೳྗΛࢦ͢
ࣝৠཹɼೖྗσʔλͱֶश͕࣌ؒେࣄ • ͜͜ͰɼϞσϧΛܰྔԽ͢ΔͨΊʹɼࣝৠཹΛར༻ • ࣝৠཹʹ͓͍ͯԿ͕େࣄͳͷ͔Λݟ͚ͭΔ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍
• ֶश࣌ؒଟ͍ํ͕ྑ͍
ઃఆɾ࣮ݧઃఆ • ධՁࢦඪʹը૾ͷΫϥεྨਫ਼Λ༻͍Δ • େنϞσϧΛڭࢣͱઃఆ͠ɼྨਫ਼Λམͱͣ͞ʹɼϞσ ϧͷܰྔԽΛࢦ͢ɽ • ڭࢣɿ#J5ͷࣄલֶशϞσϧ •
ੜెɿ3FT/FU • σʔληοτछྨ খதنͭɼେنͭ • Ϋϥε σʔλαΠζ
ڭࢣͱੜెͷग़ྗΛ͚ۙͮΔ • ࣝৠཹͷֶशʹ༻͍ΔଛࣦͰ,-μΠόʔδΣϯε༻͍Δ • ͦΕͧΕڭࢣͷ༧ଌɼੜెͷ༧ଌ • Ϟσϧͷग़ྗࣗମɼԹ͖ιϑτϚοΫε pt ,
ps KL(pt ||ps ) = ∑ i∈C [−pt,i log ps,i + pt,i log pt,i ]
ϋΠύϥઃఆͱ͔࣮ݧͷখٕͨͪ • ࠷దԽख๏ͱͯ͠"EBNΛ༻͍ΔɽϋΠύϥσϑΥϧτ • εέδϡʔϥ$PTJOFEFDBZΛ༻͍Δ • ֶश҆ఆͷͨΊɼޯͷେ͖͞Λʹ੍ݶ • όοναΠζΛʹઃఆɽͨͩ͠*NBHF/FU
.JYVQʹΑͬͯσʔλ֦ுΛ͢Δ • ͭͷ܇࿅σʔλͷϖΞΛࠞ߹ͯ͠৽͘͠σʔλΛ࡞<> • ܇࿅σʔλ͕ϛοΫε͞ΕΔ͚ͩͰͳ͘ɼϥϕϧϛοΫε • • ͔ΒҰ༷ʹαϯϓϦϯά͢Δ
ຊՈ X = λX1 + (1 − λ)X2 , y = λy1 + (1 − λ)y2 λ ∈ [0,1] λ ∼ Beta(α, α) IUUQTBSYJWPSHBCT
ը૾ͷલॲཧ • σʔλͷલॲཧͰJODFQUJPOTUZMFDSPQΛ༻͍Δ • ݩσʔλͷը૾αΠζͷ ͰϥϯμϜʹΫϦοϓ • ΫϦοϓͨ͠ޙϥϯμϜͳΞεϖΫτൺʹม
• มͨ͋͠ͱݻఆαΠζ Y ʹϦαΠζ IUUQTBSYJWPSHBCT
ओுͷ͓͞Β͍ • ੜెͱڭࢣಉ͡ೖྗΛ༻͍Δ͜ͱ͕େࣄ • ೖྗσʔλσʔλ֦ு͢Δ΄Ͳྑ͍ • ֶश࣌ؒଟ͍ํ͕ྑ͍ • ϩόετੑΛݟΔͨΊʹখதنσʔληοτͭΛར༻
• ަབྷҼࢠΛഉআ͢ΔͨΊʹ͞·͟·ͳϋΠύϥͰ࣮ݧ 1BUJFOU UFBDIFS $POTJTUFOU UFBDIFS
$POTJTUFOUUFBDIFS Ұ؏ͨ͠ڭࢣ ʹ͍ͭͯΈΔ • ڭࢣɼੜెͷઃఆΛͭ༻ҙ͢Δɽ • 'JYFEUFBDIFSɿڭࢣͷग़ྗ͕ݻఆ͞Ε͍ͯΔ • *OEFQFOEFOUOPJTFɿڭࢣɼੜెͰҟͳΔೖྗ
• $POTJTUFOUUFBDIJOHɿڭࢣͱੜెͰಉ͡ೖྗ • 'VODUJPONBUDIJOHɿ$POTJTUFOUUFBDIJOHʴσʔλ֦ு • Լೋ͕ͭҰ؏ͨ͠ڭࢣઃఆΛද͍ͯ͠Δɽ
Ұ؏ͨ͠ڭࢣਫ਼͕ߴ͍ଞաֶशΛى͜͢
1BUJFOUUFBDIFS ڧ͍ڭࢣ ʹ͍ͭͯΈΔ • Ұൠతͳڭࢣ͋Γֶशͷ߹ɼϥϕϧʹର͠ɼը૾͕େ͖͘ ΉՄೳੑ͕͋Δɽ • ͜͜ͰɼࣝৠཹڭࢣͱੜెͷؔϚονϯάͱղऍ •
Ұ؏ͯ͠ಉ͡ೖྗΛ༩͑ΔͳΒɼೖྗࣗମΜͰ͍͍ ˠσʔλ֦ு͕༗ޮͳͷͰ • σʔλ֦ுΛߦ͍ɼେ͖ͳΤϙοΫͰֶश͢ΔͱΑΓੜె Ϟσϧͷੑೳ্͕͢Δ͜ͱΛݕূ͢Δ
աֶशΛճආ͠ͳ͕Βɼੑೳ্͕͢Δ • ͍ઢʢڭࢣʣͱಉͷੑೳʹ౸ୡ • ઢʢసҠֶशʣΑΓ࠷ऴతʹੑೳ͕ྑ͍ • ΦϨϯδʢ͔ΒֶशʣΑΓੑೳ͕͍͍ˠɹࣝৠཹͷޮՌ͕͋Δ
େنσʔληοτʢ*NBHF/FUʣͰࣝৠཹ • Ұ؏ͨ͠ڭࢣաֶश͍ͯ͠ͳ͍ • ·ͨσʔλ֦ு͢Δͱগͳ͍Τϙο ΫͰߴ͍ਫ਼Λୡ • ˠJUFSBUJPOಉ͡
ҟͳΔղ૾Ͱࣝৠཹ͕͏·͍͘͘ • ڭࢣ͕ߴ͍ղ૾ˠੜె͕͍ղ૾ͷํ͕ੑೳ͕ྑ͘ͳΔ
4IBNQPPΛֶͬͯशͨ͠Βߴʹͳͬͨ • σʔλ֦ுʹΑͬͯੑೳ্ͨ͠ ͕ɼͦͷֶश͕࣌ؒ৳ͼͯ͠·͏ • "EBN͔ΒΑΓڧྗͳ࠷దԽख๏Ͱ ͋Δ4IBNQPPΛͬͯΈΔɽ • ˠֶश͕ഒʹͳͬͨʂ
IUUQTBSYJWPSHBCT
సҠֶश͔ͭͬͯࣝৠཹ͍͍Μ͡Όͳ͍ • 1BUJFOUUFBDIFSͷͱ͖ɼసҠֶ श༗ޮ͔ͩͬͨΒࣝৠཹ స Ҡֶश͍͍ͷͰ • ֶशॳظసҠֶशΑ͔͚ͬͨ
Ͳɼ࠷ऴతʹٯస
ࣝৠཹͬͺΓ༗ޮ • ࣝৠཹͬͯແ͍͍ͯͬͯ͘͜ ͱͳ͍ʁ • ࣝৠཹ͕ͳ͍ͱաֶशΛى͜͢
·ͱΊͱײ • ࣝৠཹͰॏཁͳཁૉͱͯ͠ɼ • ɽڭࢣͱੜెͷೖྗ͕ಉ͡ • ɽσʔλ֦ுΛͨ͘͞Μ͢Δ • ɽֶशΤϙοΫΛ૿͢ɹɹɹɹ͜ͱʹ͍ͭͯࣔͨ͠ɽ
• ࣝৠཹͷ৽ख๏ΛఏҊͨ͠Θ͚Ͱͳ͍͕ɼطଘͷϞσϧΛ༻ ͍ͯɼܰྔϞσϧ͕4P5"ΛऔΕΔ͔͠Εͳ͍ϩϚϯΛײ͡Εͨ