論文読み：Identifying Mislabeled Data using the Area Under the Margin Ranking (NeurIPS'20) /Area_Under_the_Margin_Ranking

Transcript

1. *EFOUJGZJOH
   .JTMBCFMFE
   %BUB
   VTJOH
   UIF
   "SFB
   6OEFS
   UIF
   .BSHJO
   3BOLJOH ஥फࠜଠ࿕ 
   BMHP൝NFFUJOH
   (FPGG
   1MFJTT
   5JBOZJ ;IBOH
   &UIBO
   &MFOCFSH
   BOE
   ,JMJBO
   2
   8FJOCFSHFS
   /FVS*14`
2. Πϯτϩ nԾઆɿ ޡͬͨϥϕϧ෇͚͞Εͨαϯϓϧ͕Ϟσϧͷ൚ԽੑΛ௿Լͤ͞Δͱ Ծఆ͢ΔͱɼޡͬͨϥϕϧΛݟ෼͚ɼͦͷσʔλΛഉআ͢Ε͹൚Խ ੑ͸্͕Δɽ
3. ͲΜͳ࿦จʁ nഎܠɿ ͞·͟·ͳཁҼʹΑΓɼ࣮ੈքͷσʔληοτʹ͸ޡϥϕϦϯά͞Ε ͨαϯϓϧؚ͕·Ε͍ͯΔ৔߹͕͋Δɽ n໰୊ɿ ޡϥϕϧͷֶशσʔλ͸ɼΦʔόʔϑΟοτ΍ύϑΥʔϚϯε௿ԼΛ ট͘ɽ nఏҊख๏ɿ "SFB
   6OEFS
   UIF
   .BSHJO
   TUBUJTUJD
   "6.౷ܭྔ ʹΑΔޡϥϕϧα ϯϓϧͷࣝผํ๏ΛఏҊɽ n݁Ռɿ $*'"3σʔληοτͷΛ࡟আͨ݁͠Ռɼ3FT/FUϞσϧ ͷςετޡ͕ࠩݮগͨ͠
4. എܠɾ໰୊ nMNIST΍ImageNetͷΑ͏ͳ༗໊ͳσʔληοτͰ͑͞ɼ ༗֐ͳྫΛؚΉɽ
5. ઌߦݚڀ nଟஈύΠϓϥΠϯ[e.g.1,2]΍ϩόετͳଛࣦؔ਺[e.g.3,4] Λ༻͍ͯαϯϓϧΛࣝผ͢Δํ๏͕ݚڀ͞Ε͍ͯΔ [1] P. Chen, B. Liao, G. Chen, and S. Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In ICML, 2019. [2] J. Han, P. Luo, and X. Wang. Deep self-learning from noisy labels. In CVPR, 2019. [3] Y. Xu, P. Cao, Y. Kong, and Y. Wang. LDMI: An information-theoretic noise-robust loss function. In NeurIPS, 2019. [4] Z. Zhang and M. R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurIPS, 2018
6. ఏҊख๏ɿ n"SFB
   6OEFS
   UIF
   .BSHJOʀ"6. l αϯϓϧͷׂΓ౰ͯΫϥεͱϩδοτ஋͕࠷΋ߴ͍ඇׂΓ౰ͯΫϥε ͷϩδοτ஋ؒͷࠩΛଌఆ͢Δ l ֤ΤϙοΫʹ͓͍ͯଌఆ͞ΕͨαϯϓϧͷϚʔδϯΛฏۉԽ͢Δ͜ͱ Ͱଊ͑Δ T：エポックの総数
7. ఏҊख๏ɿ n"SFB
   6OEFS
   UIF
   .BSHJOʀ"6. l ֶशதͷ೚ҙͷΤϙοΫʹ͓͍ͯɺޡϥϕϧαϯϓϧ͸ਖ਼͍͠ϥ ϕϧαϯϓϧΑΓ΋খ͍͞ϚʔδϯΛ࣋ͭ͜ͱ͕༧૝͞ΕΔͱԾ ఆ͢Δ
8. ఏҊख๏ɿ nݕূσʔλͳ͠Ͱͷᮢ஋Λֶश͢Δܭࢉޮ཰ͷྑ͍ઓུΛఏҊɽ l ᮢ஋͸σʔληοτʹґଘ͢Δ nᮢ஋αϯϓϧʹΑΔޡϥϕϧͷഉআ l ֶशதʹᮢ஋αϯϓϧʢِͷσʔλʣΛૠೖ͠ɺϥϕϧ෇͚͞Εͨ σʔλʹΑΔֶशμΠφϛΫεΛ໛฿͢Δɽ l ᮢ஋αϯϓϧͱಉ౳͔ͦΕҎ্ͷ"6.Λ࣋ͭσʔλ͸ɺޡϥϕϦ ϯά͞ΕͨͱԾఆ͢Δ͜ͱ͕Ͱ͖Δɽ
9. ఏҊख๏ɿ nᮢ஋αϯϓϧʹΑΔޡϥϕϧͷഉআ l ֶशσʔλͷҰ෦ΛऔΓग़͠ɼଘࡏ͠ͳ͍Ϋϥεʹ࠶ׂΓ౰ͯΔ FHʣ ֶशσʔληοτ͕DݸͷΫϥεʹଐ͢Δ/ݸͷαϯϓϧΛ࣋ͬͯ ͍Δͱ͢Δͱɼ/ D  ͷαϯϓϧΛબ୒͢Δɽϥϕϧ͸D ͱ͢Δɽ l ˋͷᮢ஋αϯϓϧΑΓ௿͍"6.Λ࣋ͭσʔλΛޡϥϕϧͱࣝผ ͢Δɽʢਤͷփ৭ଠઢʣ
10. ఏҊख๏ɿ nϓϩηεɿ  ᮢ஋αϯϓϧͷαϒηοτD!"# Λ࡞੒͢Δ  ᮢ஋αϯϓϧΛؚΉमਖ਼ֶशηοτD′$%&'( Λ࡞੒͢Δ 3. D′$%&'( ͰωοτϫʔΫΛ࠷ॳʹֶश཰͕Լ͕Δ·Ͱֶश˞ͤ͞ɼશ σʔλͷ"6.Λଌఆ͢Δɽ  ˋᮢ஋αϯϓϧ"6.ʢЋʣΛܭࢉ͢Δɽ  ЋΛᮢ஋ͱͯ͠ɺϥϕϧ෇͚͞ΕͨσʔλΛࣝผ͢Δ ※ 学習率が低下する前に学習を停⽌することで、ネットワークが収束し、その結果、誤ラベルサンプルを記憶してしまうことを防ぐ
11. ݁Ռɿ nൈਮ
12. ॴײ nϥΠϒϥϦެ։͞Ε͍ͯΔͷͰ࢖͍΍͍͢ l IUUQTHJUIVCDPNBTBQQSFTFBSDIBVN nࣗ࡞σʔληοτͷਫ਼౓ఈ্͛Ͱ࢖͑ͦ͏ nϞσϧʹͱͬͯͷ”ֶश͠΍͍͢σʔλ”ͱ”ֶश͠ʹ͍͘ σʔλ”͕Θ͔ΔͷͰɼσʔλαϯϓϦϯάʹ΋࢖͑ͦ͏