Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CVPR2020読み会 Proxy Anchor Loss for Deep Metric Learning

SatoKeiju
July 18, 2020

CVPR2020読み会 Proxy Anchor Loss for Deep Metric Learning

CVPR2020読み会(後編)の資料です

なるべくMetric Learningに良い感じに入門できるようになってます

SatoKeiju

July 18, 2020
Tweet

Other Decks in Science

Transcript

  1. CVPR2020ಡΈձ (2020/7/18)
 sandglass Proxy Anchor Loss for
 Deep Metric Learning

  2. ·ͣ͸ࣗݾ঺հ !2 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane
  3. ·ͣ͸ࣗݾ঺հ !3 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane K-POPେ޷͖ͳͷͰ Metric LearningͰ
 PyTorchͱTWICEΛ஌Δ
 ABEJAΞυϕϯτΧϨϯμʔ ॻ͖·ͨ͠
  4. ·ͣ͸ࣗݾ঺հ !4 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
  5. ·ͣ͸ࣗݾ঺հ !5 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
  6. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane •

    ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !6 ΤϯδχΞ: @xecus͞Μ ͕։ൃͨ͠ɺإೝূٕज़Λ࢖ͬͨ ΊͪΌͪ͘Ό໘ന͍ϓϩμΫτͷ ஀ੜൿ࿩ ݱࡏΠϯλʔϯͰ͓ख఻͍ͨ͠࿩ Λଓฤͱͯࣥ͠චதͰ͢
  7. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane •

    ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !7 Ϧαʔν: @peisuke͞Μ ʹ༏͘͠എதΛԡ͍͖ͯͨͩ͠ ຊ೔ਓੜॳొஃͰ͢
  8. ΑΖ͓͘͠ئ͍͠·͢ !8

  9. ·ͣ͸͜ͷൃදͷΰʔϧΛ֬ೝʂ !9

  10. ࠓ೔ͷΰʔϧ !10 ͷ͓ؾ࣋ͪΛཧղ͢Δ ঺հ࿦จΑΓҾ༻

  11. ࠓ೔ͷΰʔϧ !11 ͷ͓ؾ࣋ͪΛཧղ͢Δ ԿͷֆͱԿͷࣜʁʁʁ

  12. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !12

    ͷ3ຊͰɺMetric Learningͷੈքͷུ஍ਤΛ͓౉͠Ͱ͖Ε͹ͱࢥ͍ͬͯ·͢ ͍ΖΜͳํͱ͍ΖΜͳ͓࿩͍ͨ͠ͷͰTwitterͳͲ͓ؾܰʹʂ͓ئ͍͠·͢ʂ
  13. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !13

  14. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !14 Black footed Albatross Sooty Albatross

  15. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !15 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ
  16. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !16 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ
  17. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !17 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ ͜ΕΒΛ֫ಘ͢Δͷ͕Metric(ࢦඪ)Learning(ֶश)
  18. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !18

  19. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !19 ม਺ಉ࢜ͷڞ෼ࢄߦྻ • ֤࣍ݩͷ෼ࢄΛߟྀͨ͠Ұൠతͳڑ཭ • ൒ਖ਼ఆ஋ߦྻͰ͋Ε͹͞Βʹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹͱม׵
 ͭ·ΓσʔλؒͷϢʔΫϦουڑ཭͕ద੾ʹͳΔΑ͏ͳม׵LΛֶश
 →ྨࣅσʔλಉ࢜͸ۙ͘ɺඇྨࣅσʔλಉ࢜ͷڑ཭͸ԕ͘͢Δ
 ɹ࠷దԽ໰୊

  20. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !20 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ
  21. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !21 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺
  22. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !22 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN
  23. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !23 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN ࣮ݱ͍ۭͨؒ͠ͷڑ཭తͳ ͓ؾ͕࣋ͪ࿅Γ͜·ΕͨLossؔ਺ ֶश
  24. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !24 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ

  25. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !25 https://twitter.com/sanacottoncandy/status/1131140431775801345 https://twitter.com/WinterFrame/status/1200973640197492736 https://twitter.com/JYPETWICE/status/1272053602790502400 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ

  26. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !26 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ αφͪΌΜ π΢ΟͪΌΜ

  27. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !27 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ

    ը૾ݕࡧ ΫΤϦ Top-1(࠷ۙ๣) https://twitter.com/JYPETWICE_JAPAN/status/1156761954024734721
  28. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !28 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ

    ը૾ݕࡧ ෼ྨʹՃ͑ͯະ஌ͷΫϥε΋ ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ը૾ࣝผ https://twitter.com/JYPETWICE_JAPAN/status/1156761692547629056 New?
  29. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !29 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ

  30. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !30 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ʮֶश࣌ʹແ͍Ϋϥε͕ਪ࿦ʹग़ͯ͘Δʯ՝୊ઃఆʹ˕ • Face Recognition / Verification

    • Person Re-Identification • Image Retrieval ͳͲͳͲ...
  31. !31 ଞʹ΋ Metric Learning for…

  32. Adversarial Attack(ݸਓతʹ໘ന͔ͬͨͷͰ͝঺հ) !32 “Metric Learning for Adversarial Robustness”(NeurIPS’19) Adversarial Example͸

    ಛ௃ྔۭؒʹ͓͍ͯ Ϋϥεͷڥք෇ۙʹҐஔ͍ͯ͠Δʁ(͔Βؒҧ͑Δʁ) Adversarial ExampleΛ ຊ౰ͷΫϥεʹ͍ۙํ޲ʹ ಈ͔͢Loss ը૾ͱAdversarialͳը૾Λಛ௃ྔۭؒʹ౤Ө͢Δͱ…ʁ
  33. !33 “OpenGAN: Open Set Generative Adversarial Networks”(arXiv 2020/5/18) ҟৗݕ஌ “Unsupervised

    Anomaly Detection with Generative Adversarial Networks
 to Guide Marker Discovery”(IPMI’17) GAN Distillation “Relational Knowledge Distillation”(CVPR’19)
  34. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ !34

  35. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !35

  36. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !36

  37. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !37 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
  38. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !38 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
  39. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !39 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ۙͮ͘ ಉ͡Ϋϥεͷը૾
  40. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !40 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
  41. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !41 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
  42. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !42 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ཭ΕΔ ҧ͏Ϋϥεͷը૾
  43. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !43 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾ “Dimensionality Reduction by Learning an Invariant Mapping” (CVPR’06)
  44. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !44 [~]+: ώϯδؔ਺(= max(~, 0))

  45. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !45 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) negative anchor positive
  46. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !46 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dp ʹϚʔδϯαΛ଍ͯ͠΋ ·ͩdn ͷํ͕σΧ͘ͳͬͯ΄͍͠ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍…
  47. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !47 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dpʹϚʔδϯαΛ଍ͯ͠΋ dnΑΓখ͘͢͞Δ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍… “Learning Fine-grained Image Similarity with Deep Ranking” (CVPR’14)
  48. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿

    • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !48 ͳͲͳͲ…
  49. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !49 “FaceNet: A Unified Embedding for

    Face Recognition and Clustering”(CVPR’15) !49 !49 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ Tripletͷ૊Έ߹Θͤʹ͍ͭͯͷʮ೉͠͞ʯΛఆٛ Loss͕େ͖͘ͳΔSemi-hard NegativeϖΞΛޮ཰Α͘࡞੒
  50. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !50 “Improved Deep Metric Learning with

    Multi-class N-pair Loss Objective”(NIPS’16) NΫϥε͔Β2ը૾ͣͭ
 औΓग़ͯ͠ಛ௃ྔʹม׵ N૊ͷηοτΛ࡞Γ
 LossΛܭࢉ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
  51. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿

    • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !51 “L2-constrained Softmax Loss for Discriminative Face Verification”(arXiv 2017/3) ໌ࣔతͳڑ཭࠷దԽ͸͠ͳ͍͕ ಛ௃ྔϕΫτϧΛϊϧϜͰׂͬͯ ಛ௃ྔ࣍ݩۭؒͷٿ໘্ʹ ಺ੵ(cosྨࣅ౓)Ͱදݱྗͷௐ੔ ֶशͷ೉қ౓΋εέʔϦϯάύϥϝʔλαͰௐ੔
  52. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !52 “Deep Metric Learning with Angular

    Loss”(ICCV’17) anchorɺpositiveɺ negativeΛಈ͔͢ ޲͖(ޯ഑)Λ ֯౓͔Βߟ͑Δ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
  53. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !53 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு •

    ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “No Fuss Distance Metric Learning using proxies”(ICCV’17) positiveͱnegativeʹ ݸผͷσʔλ఺Ͱ͸ͳ͘ ֤Ϋϥεͷ୅ද஋ͷ ຒΊࠐΈϕΫτϧ(proxy) Λ༻͍Δ ѹ౗తऩଋͷ଎͞ ຊ೔ͷ࿦จͷϥΠόϧ
  54. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !54 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு •

    ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “ArcFace: Additive Angular Margin Loss for Deep Face Recognition”(CVPR’19) ಛ௃ྔϕΫτϧͷޙஈʹશ݁߹૚Λ઀ଓ Softmaxؔ਺ͱCross Entropy LossͰ Metric LearningΛߦ͏(ਪ࿦࣌͸શ݁߹૚Λ֎͢) ຊ೔ͷ࿦จͷϥΠόϧ
 Ͱ͸ͳ͍͕ඞಡʂ
  55. ͓·͚: CVPR’20ͰMetric Learningͷ࿩ͯ͠Δ࿦จ(ͷҰ෦) !55 •Deep Metric Learning via Adaptive Learnable

    Assessment •End-to-End Illuminant Estimation Based on Deep Metric Learning •Cross-Batch Memory for Embedding Learning •Optimizing Rank-Based Metrics With Blackbox Differentiation •Universal Weighting Metric Learning for Cross-Modal Matching •Fast(er) Reconstruction of Shredded Text Documents via
 Self-Supervised Deep Asymmetric Metric Learning •Moving in the Right Direction: A Regularization for Deep Metric Learning •CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition •Circle Loss: A Unified Perspective of Pair Similarity Optimization
  56. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ (࠶ܝ) !56

  57. ࠓ೔ͷΰʔϧ(࠶ܝ) !57 ͷ͓ؾ࣋ͪΛཧղ͢Δ

  58. ࠓ೔ͷΰʔϧ(࠶ܝ) !58 Triplet Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ

    ଐ͢Δผͷσʔλ఺ • negative: anchorɺpositiveͱ͸ ҟͳΔΫϥεͷσʔλ఺ ↓ • anchorͱpositive͕͖ۙͮɺ anchorͱnegative͕཭ΕΔLoss • σʔλͷ૊Έ߹Θͤ૯਺ଟ͗͢ • ֶशʹޮ͘ϖΞʹ౰ͨΓʹ͍͘
  59. ࠓ೔ͷΰʔϧ(࠶ܝ) !59 N-pair Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ

    ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷσʔλ఺(1ͭͣͭ) • ۙ͞Λߟྀ ↓ • TripletΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (ϛχόονͷதͷ͞ΒʹҰ෦)
  60. ࠓ೔ͷΰʔϧ(࠶ܝ) !60 Lifted Structure Loss • anchor: ͋Δσʔλ఺ • positive:

    anchorͱಉ͡Ϋϥεʹ ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷશσʔλ఺ ↓ • N-pairΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (anchorΫϥεͷҰ෦σʔλ΍ϛ χόον಺ͷΈͰݶఆత)
  61. ࠓ೔ͷΰʔϧ(࠶ܝ) !61 Proxy-NCA Loss • anchor: ͋Δσʔλ఺ • proxy: ֤Ϋϥεʹ͖ͭ1ͭ͋Δ

    ʮ୅ද఺ʯ ↓ • anchorͱશproxyͰ
 N-pair Lossతͳֶश • ऩଋ͕ΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใ͸
 ׆͔͖͠Εͯͳ͍
  62. ࠓ೔ͷΰʔϧ(࠶ܝ) !62 ࢒͢͸ຊ೔ͷ࿦จ Proxy Anchor Loss ͷΈ

  63. ࠓ೔ͷΰʔϧ(࠶ܝ) !63 Proxy Anchor Loss • anchor: ͋ΔΫϥεͷproxy • positive:

    anchorͷproxyʹଐ͢ ΔΫϥεͷϛχόον಺શσʔλ ఺ • negative: anchorͱҟͳΔΫϥε ͷϛχόον಺શσʔλ఺ ↓ • ऩଋ͕͞ΒʹΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใΛ׆͔͠ ·͍ͬͯ͘Δ
  64. ࠓ೔ͷΰʔϧ(࠶ܝ) !64 ͍͟ղઆ΁

  65. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !65

  66. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss !66

  67. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ

    ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ
  68. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ

    ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ શembedding(σʔλಛ௃ྔ) ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ શembedding(σʔλಛ௃ྔ)ϕΫτϧ ʁ
  69. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !69

  70. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !70 “easier-to-interpret form”

  71. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !71 “smooth approximation of ReLU”

    ͭ·Γඍ෼Մೳ࿈ଓͳmax(0, z)
  72. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !72 “smooth approximation of max

    function” LogSumExp
  73. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪ !73 ͋Δproxy pͱhardest positive (p͔ΒҰ൪ԕ͍positive)͕ۙͮ͘

    ͋Δproxy pͱhardest negative (p͔ΒҰ൪͍ۙnegative)͕཭ΕΔ “smooth approximation of max function”
  74. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !74

  75. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !75 proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ

  76. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !76 anchorͱ֤positiveͱͷྨࣅ౓͕ ߴ͍΄Ͳখ͘͞ͳͬͯخ͍߲͠ proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ anchorͱ֤negativeͱͷྨࣅ౓͕

    ௿͍΄Ͳখ͘͞ͳͬͯخ͍߲͠
  77. !77 ͲΜͳͱ͜Ζ͕ྑ͘ͳͬͯΔʁ

  78. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !78

  79. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !79 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  80. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !80 σʔλͷۭؒతߏ଄΍෼෍Λ
 Α͘ଊֶ͑ͨश͕Ͱ͖͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  81. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !81 ୅ද఺Λ࢖͍ͳ͕Β σʔλશମ΋ݟΕ͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  82. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !82 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ Recall@KείΞ • Retrieval(ݕࡧ)λεΫ΍Metric LearningͷੑೳΛଌΔͷʹΑ͘࢖ΘΕΔ

    • ֶशͨ͠ม׵(ωοτϫʔΫ)ʹςετσʔλΛશͯೖྗͯ͠embeddingԽͨ࣌͠ʹ
 ֤σʔλ఺ʹ͍ͭͯɺͦͷۭؒ಺Ͱ͍ۙॱϥϯΩϯάTop-KҐͷதʹ
 ಉ͡Ϋϥεͷσʔλ఺͕͋ΔΑ͏ͳ఺ͷׂ߹ • ྫ͑͹Recall@2ͳΒɺRecall@1(࠷ۙ๣఺͕ಉ͡ΫϥεͰ͋ΔΑ͏ͳ఺ͷׂ߹)ʹ
 ʮ࠷دΓ͸ผΫϥεͷσʔλ఺͕ͩ2൪໨ʹ͍ۙ఺ͳΒಉ͡ΫϥεʯͰ͋ΔϠπΒΛ଍ͨ͠
 είΞͱͳΔ • ʮະ஌ΫϥεΛ͖ͪΜͱ෼཭Ͱ͖Δ͔ʯΛଌΔͨΊɺσʔληοτΛ2෼ׂͯ͠
 ֶशͱςετΛ׬શʹผͷΫϥεͰߦ͏(ͳ͔ͳ͔ϋʔυ)
  83. ࿦จᐌ͘2: ྨࣅ౓ʹԠͯ͡஋͕ͪΌΜͱมΘΔʂ !83 Proxy-NCA Loss Proxy Anchor Loss positive proxyʹ͍ͭͯ

    มԽ͕ݻఆ શσʔλ఺ʹ͍ͭͯ hard͞ʹԠֶͨ͡श
  84. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !84 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U:

    1Ϋϥε͋ͨΓͷproxy਺
  85. Ϋϥε਺C << σʔλ૯਺M ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !85 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺

    B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
  86. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !86 ૊Έ߹Θͤ૯਺͕ଟ͗ͯ͢
 ֶशʹ༗ޮͳσʔλϖΞΛ Ҿ͖౰ͯΔͷʹ͔͔࣌ؒΔ Ϋϥε਺C << σʔλ૯਺M M:

    ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
  87. !87 ࣮ݧͨ͠ΒSOTAͳΜͰ͠ΐ…ʁ

  88. ࣮ݧ݁Ռ: SOTAʂ !88

  89. ࣮ݧ݁Ռ: SOTAʂ !89 ௗ ं

  90. ࣮ݧ݁Ռ: SOTAʂ !90 ಛ௃ྔͷ ࣍ݩ਺Λ ͪΌΜͱ ߹Θͤͯ ൺֱʂ (݁ߏޮ͍ͯ͘Δ)

  91. ࣮ݧ݁Ռ: SOTAʂ !91 σΧ͍ը૾࢖ͬͯΔͷʹ΋߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)

  92. ࣮ݧ݁Ռ: SOTAʂ !92 backbone networkهࡌͯ͠Δ͠߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)

  93. ࣮ݧ݁Ռ: SOTAʂ !93 ஸೡʂ

  94. ࣮ݧ݁Ռ: SOTAʂ !94 Stanford Online Products (ΊͪΌͪ͘Ό͍ΖΜͳ෺) In-shop(ϑΝογϣϯΞΠςϜ)

  95. ࣮ݧ݁Ռ: SOTAʂ !95 ஸೡʂ ஸೡʂ

  96. !96 ͚ͩͰ͸ͳ͘…ʂ

  97. ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁ՌTop4 !97 CUB-200 SOP In-shop Cars-196 ΋͸΍ ؒҧͬͯΔϠπ ਓ͕΍ͬͯ΋ Ϝζ͍͡ΌΜ…

  98. ϋΠύϥߟ࡯ͦͷ1: όοναΠζ !98 ʮ1όον಺ͷσʔλ਺͕ଟ͍ํ͕
 σʔλશମͷߏ଄ΛΑΓଊ͑ΒΕΔʯ ҙ֎ͱ ͔ࠩͭͳ͍ͳ…

  99. ϋΠύϥߟ࡯ͦͷ2: ಛ௃ྔۭؒͷ࣍ݩ਺ !99 ʮେ͖͍ํ͕΋ͪΖΜੑೳྑ͍ ͕ ݕࡧ଎౓ͱ͔͸τϨʔυΦϑʯ σΧ͗͢Δͱ ੑೳԼ͕ΔΜͩ…

  100. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !100 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ…

  101. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !101 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ…

    • ෳ਺ͷೖྗಉ࢜ͷʮࠩΛڧௐ͢Δ౓߹͍ʯ • cosྨࣅ౓Λ࢖͏Metric LearningͰΑ͘ಋೖ͞ΕΔ
 (ͦ΋ͦ΋ͷ஋Ҭ͕-1 ≦ cosx ≦ 1ͳͷͰ͕͖ࠩͭʹ͍ͨ͘Ί) • DistillationΛఏҊͨ͠࿦จͰొ৔ͨ͠softmaxԹ౓ύϥϝʔλΛٯ਺ʹͨ͠΋ͷ ※͍͔ͭ͘ͷ࿦จΛݟ͖ͯͨײ͡20~60͙Β͍͕ଥ౰ͳΠϝʔδͩͬͨͷͰೲಘ εέʔϦϯάύϥϝʔλ “Distilling the Knowledge in a Neural Network”(NIPS’14 Workshop)
  102. ͞ΒʹarXiv൛Ͱ͸… !102

  103. Appendix1: backbone networkͱը૾αΠζ !103 ʮ࣮ݧ͸Inception-BNͰ΍͚ͬͨͲResNetͩͱ΋ͬͱੌ͍ɺ GPUϝϞϦ࣍ୈͰը૾͸σΧ͍ํ͕ྑ͍ʯ ͦΓΌͦ͏ͩ…

  104. Appendix2: ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁Ռ(vs Proxy-NCA) !104 CUB-200 SOP In-shop Cars-196

  105. Appendix3: ςετσʔλͷ෼෍ͲΜͳײ͡ʁ(t-SNEՄࢹԽ) !105 CUB-200 SOP In-shop Cars-196

  106. ͞Βʹ͞Βʹ… !106

  107. ͔ͳΓ៉ྷͳஶऀPyTorch࣮૷ʂ !107 https://github.com/tjddus9597/Proxy-Anchor-CVPR2020 • LossΫϥε͔Βtrain/testίʔυͷ࣮૷͸΋ͪΖΜ • σʔληοτ4छͷDatasetΫϥε࣮૷΋͋Γ • backbone networkͨͪͷ࣮૷΋͋Γ

    • طଘݚڀͷLossͨͪ΋ࢼͤΔΑ͏ʹͳͬͯΔ
 (Pytorch-Metric-Learningύοέʔδ͕ඞཁ) ※͜ͷൃද·Ͱʹಈ͔͓͖͔ͯͨͬͨ͠ͷͰ͕࣌ؒ͢଍Γͣʀʀ
  108. ౳ʑ Deep Metric Learning in CVͷݱঢ়͕ ͱͯ΋ྑ͍ײ͡ʹ·ͱ·͍ͬͯΔҹ৅Ͱ͢ ͜ͷࢿྉยखʹ͝ҰಡΛʂ !108

  109. ·ͱΊ !109 •Metric Learning(ಛʹCV෼໺)ͷ࿩ •ຊ೔ͷ࿦จ: Proxy Anchor Loss ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ

  110. !110