Upgrade to Pro — share decks privately, control downloads, hide ads and more …

論文読み:Identifying Mislabeled Data using the Area Under the Margin Ranking (NeurIPS'20) /Area_Under_the_Margin_Ranking

論文読み:Identifying Mislabeled Data using the Area Under the Margin Ranking (NeurIPS'20) /Area_Under_the_Margin_Ranking

Taro Nakasone

October 01, 2022
Tweet

More Decks by Taro Nakasone

Other Decks in Research

Transcript

  1. *EFOUJGZJOH.JTMBCFMFE%BUB VTJOHUIF"SFB6OEFSUIF .BSHJO3BOLJOH ஥फࠜଠ࿕ BMHP൝NFFUJOH (FPGG1MFJTT 5JBOZJ ;IBOH &UIBO&MFOCFSH BOE,JMJBO28FJOCFSHFS

    /FVS*14`
  2. Πϯτϩ nԾઆɿ ޡͬͨϥϕϧ෇͚͞Εͨαϯϓϧ͕Ϟσϧͷ൚ԽੑΛ௿Լͤ͞Δͱ Ծఆ͢ΔͱɼޡͬͨϥϕϧΛݟ෼͚ɼͦͷσʔλΛഉআ͢Ε͹൚Խ ੑ͸্͕Δɽ

  3. ͲΜͳ࿦จʁ nഎܠɿ ͞·͟·ͳཁҼʹΑΓɼ࣮ੈքͷσʔληοτʹ͸ޡϥϕϦϯά͞Ε ͨαϯϓϧؚ͕·Ε͍ͯΔ৔߹͕͋Δɽ n໰୊ɿ ޡϥϕϧͷֶशσʔλ͸ɼΦʔόʔϑΟοτ΍ύϑΥʔϚϯε௿ԼΛ ট͘ɽ nఏҊख๏ɿ "SFB6OEFSUIF.BSHJOTUBUJTUJD "6.౷ܭྔ

    ʹΑΔޡϥϕϧα ϯϓϧͷࣝผํ๏ΛఏҊɽ n݁Ռɿ $*'"3σʔληοτͷΛ࡟আͨ݁͠Ռɼ3FT/FUϞσϧ ͷςετޡ͕ࠩݮগͨ͠
  4. എܠɾ໰୊ nMNIST΍ImageNetͷΑ͏ͳ༗໊ͳσʔληοτͰ͑͞ɼ ༗֐ͳྫΛؚΉɽ

  5. ઌߦݚڀ nଟஈύΠϓϥΠϯ[e.g.1,2]΍ϩόετͳଛࣦؔ਺[e.g.3,4] Λ༻͍ͯαϯϓϧΛࣝผ͢Δํ๏͕ݚڀ͞Ε͍ͯΔ [1] P. Chen, B. Liao, G. Chen,

    and S. Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In ICML, 2019. [2] J. Han, P. Luo, and X. Wang. Deep self-learning from noisy labels. In CVPR, 2019. [3] Y. Xu, P. Cao, Y. Kong, and Y. Wang. LDMI: An information-theoretic noise-robust loss function. In NeurIPS, 2019. [4] Z. Zhang and M. R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurIPS, 2018
  6. ఏҊख๏ɿ n"SFB6OEFSUIF.BSHJOʀ"6. l αϯϓϧͷׂΓ౰ͯΫϥεͱϩδοτ஋͕࠷΋ߴ͍ඇׂΓ౰ͯΫϥε ͷϩδοτ஋ؒͷࠩΛଌఆ͢Δ l ֤ΤϙοΫʹ͓͍ͯଌఆ͞ΕͨαϯϓϧͷϚʔδϯΛฏۉԽ͢Δ͜ͱ Ͱଊ͑Δ T:エポックの総数

  7. ఏҊख๏ɿ n"SFB6OEFSUIF.BSHJOʀ"6. l ֶशதͷ೚ҙͷΤϙοΫʹ͓͍ͯɺޡϥϕϧαϯϓϧ͸ਖ਼͍͠ϥ ϕϧαϯϓϧΑΓ΋খ͍͞ϚʔδϯΛ࣋ͭ͜ͱ͕༧૝͞ΕΔͱԾ ఆ͢Δ

  8. ఏҊख๏ɿ nݕূσʔλͳ͠Ͱͷᮢ஋Λֶश͢Δܭࢉޮ཰ͷྑ͍ઓུΛఏҊɽ l ᮢ஋͸σʔληοτʹґଘ͢Δ nᮢ஋αϯϓϧʹΑΔޡϥϕϧͷഉআ l ֶशதʹᮢ஋αϯϓϧʢِͷσʔλʣΛૠೖ͠ɺϥϕϧ෇͚͞Εͨ σʔλʹΑΔֶशμΠφϛΫεΛ໛฿͢Δɽ l ᮢ஋αϯϓϧͱಉ౳͔ͦΕҎ্ͷ"6.Λ࣋ͭσʔλ͸ɺޡϥϕϦ

    ϯά͞ΕͨͱԾఆ͢Δ͜ͱ͕Ͱ͖Δɽ
  9. ఏҊख๏ɿ nᮢ஋αϯϓϧʹΑΔޡϥϕϧͷഉআ l ֶशσʔλͷҰ෦ΛऔΓग़͠ɼଘࡏ͠ͳ͍Ϋϥεʹ࠶ׂΓ౰ͯΔ FHʣ ֶशσʔληοτ͕DݸͷΫϥεʹଐ͢Δ/ݸͷαϯϓϧΛ࣋ͬͯ ͍Δͱ͢Δͱɼ/ D  ͷαϯϓϧΛબ୒͢Δɽϥϕϧ͸D

    ͱ͢Δɽ l ˋͷᮢ஋αϯϓϧΑΓ௿͍"6.Λ࣋ͭσʔλΛޡϥϕϧͱࣝผ ͢Δɽʢਤͷփ৭ଠઢʣ
  10. ఏҊख๏ɿ nϓϩηεɿ  ᮢ஋αϯϓϧͷαϒηοτD!"# Λ࡞੒͢Δ  ᮢ஋αϯϓϧΛؚΉमਖ਼ֶशηοτD′$%&'( Λ࡞੒͢Δ 3. D′$%&'(

    ͰωοτϫʔΫΛ࠷ॳʹֶश཰͕Լ͕Δ·Ͱֶश˞ͤ͞ɼશ σʔλͷ"6.Λଌఆ͢Δɽ  ˋᮢ஋αϯϓϧ"6.ʢЋʣΛܭࢉ͢Δɽ  ЋΛᮢ஋ͱͯ͠ɺϥϕϧ෇͚͞ΕͨσʔλΛࣝผ͢Δ ※ 学習率が低下する前に学習を停⽌することで、ネットワークが収束し、その結果、誤ラベルサンプルを記憶してしまうことを防ぐ
  11. ݁Ռɿ nൈਮ

  12. ॴײ nϥΠϒϥϦެ։͞Ε͍ͯΔͷͰ࢖͍΍͍͢ l IUUQTHJUIVCDPNBTBQQSFTFBSDIBVN nࣗ࡞σʔληοτͷਫ਼౓ఈ্͛Ͱ࢖͑ͦ͏ nϞσϧʹͱͬͯͷ”ֶश͠΍͍͢σʔλ”ͱ”ֶश͠ʹ͍͘ σʔλ”͕Θ͔ΔͷͰɼσʔλαϯϓϦϯάʹ΋࢖͑ͦ͏