CVPR2020読み会 Proxy Anchor Loss for Deep Metric Learning

8f5c72286a3cac9fc87d2b81689d4b7b?s=47 SatoKeiju
July 18, 2020

CVPR2020読み会 Proxy Anchor Loss for Deep Metric Learning

CVPR2020読み会(後編)の資料です

なるべくMetric Learningに良い感じに入門できるようになってます

8f5c72286a3cac9fc87d2b81689d4b7b?s=128

SatoKeiju

July 18, 2020
Tweet

Transcript

  1. CVPR2020ಡΈձ (2020/7/18)
 sandglass Proxy Anchor Loss for
 Deep Metric Learning

  2. ·ͣ͸ࣗݾ঺հ !2 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane
  3. ·ͣ͸ࣗݾ঺հ !3 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane K-POPେ޷͖ͳͷͰ Metric LearningͰ
 PyTorchͱTWICEΛ஌Δ
 ABEJAΞυϕϯτΧϨϯμʔ ॻ͖·ͨ͠
  4. ·ͣ͸ࣗݾ঺հ !4 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
  5. ·ͣ͸ࣗݾ঺հ !5 • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter:

    @TodayInsane • ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢
  6. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane •

    ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !6 ΤϯδχΞ: @xecus͞Μ ͕։ൃͨ͠ɺإೝূٕज़Λ࢖ͬͨ ΊͪΌͪ͘Ό໘ന͍ϓϩμΫτͷ ஀ੜൿ࿩ ݱࡏΠϯλʔϯͰ͓ख఻͍ͨ͠࿩ Λଓฤͱͯࣥ͠චதͰ͢
  7. • ໊લ: ࠤ౻ ܒथ Sato Keiju • Twitter: @TodayInsane •

    ॴଐ: ౦ژେֶిؾܥ޻ֶઐ߈ͷ৘ใܥ म࢜2೥ • ֶ෦: CMOSΞφϩάిࢠճ࿏ in ਆށେֶిి • גࣜձࣾABEJAͰ௕ظΠϯλʔϯ 2019/4 ʙ • Vue.jsͱ͔Firebaseͱ͔Goͱ͔Kubernetesͱ͔AWSͱ͔Deep Learningͱ͔Λϝϯλʔ͞Μํʹڭ͑ͯ΋Β͍·͍ͬͯ͘·͢ ·ͣ͸ࣗݾ঺հ !7 Ϧαʔν: @peisuke͞Μ ʹ༏͘͠എதΛԡ͍͖ͯͨͩ͠ ຊ೔ਓੜॳొஃͰ͢
  8. ΑΖ͓͘͠ئ͍͠·͢ !8

  9. ·ͣ͸͜ͷൃදͷΰʔϧΛ֬ೝʂ !9

  10. ࠓ೔ͷΰʔϧ !10 ͷ͓ؾ࣋ͪΛཧղ͢Δ ঺հ࿦จΑΓҾ༻

  11. ࠓ೔ͷΰʔϧ !11 ͷ͓ؾ࣋ͪΛཧղ͢Δ ԿͷֆͱԿͷࣜʁʁʁ

  12. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !12

    ͷ3ຊͰɺMetric Learningͷੈքͷུ஍ਤΛ͓౉͠Ͱ͖Ε͹ͱࢥ͍ͬͯ·͢ ͍ΖΜͳํͱ͍ΖΜͳ͓࿩͍ͨ͠ͷͰTwitterͳͲ͓ؾܰʹʂ͓ئ͍͠·͢ʂ
  13. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !13

  14. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !14 Black footed Albatross Sooty Albatross

  15. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !15 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ
  16. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !16 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ
  17. Metric Learning: σʔλͷڑ཭ࢦඪΛ֫ಘ͢Δ !17 Black footed Albatross Sooty Albatross Black

    footed Albatross Sooty Albatross • ಉ͡Ϋϥεͷσʔλಉ࢜͸͍ۙ • ҟͳΔΫϥεؒ͸཭Ε͍ͯΔ σʔλ΍ ͦͷಛ௃ྔ ࣅͯΔͱ ۙ͘ͳΔ ۭؒ ͜ΕΒΛ֫ಘ͢Δͷ͕Metric(ࢦඪ)Learning(ֶश)
  18. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !18

  19. ͸͡·Γ͸ม׵ߦྻ(ϚϋϥϊϏεڑ཭)͔Β… !19 ม਺ಉ࢜ͷڞ෼ࢄߦྻ • ֤࣍ݩͷ෼ࢄΛߟྀͨ͠Ұൠతͳڑ཭ • ൒ਖ਼ఆ஋ߦྻͰ͋Ε͹͞Βʹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹͱม׵
 ͭ·ΓσʔλؒͷϢʔΫϦουڑ཭͕ద੾ʹͳΔΑ͏ͳม׵LΛֶश
 →ྨࣅσʔλಉ࢜͸ۙ͘ɺඇྨࣅσʔλಉ࢜ͷڑ཭͸ԕ͘͢Δ
 ɹ࠷దԽ໰୊

  20. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !20 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ
  21. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !21 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺
  22. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !22 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN
  23. Deep LearningͷൃలͰMetric Learning΋Ξπ͍ !23 Channel Width Height ը૾: H ×

    W × C࣍ݩσʔλ Hand-craftedͳಛ௃ྔઃܭ x1 x2 : : : : : : : :
 : xN N࣍ݩಛ௃ϕΫτϧ = N࣍ݩۭؒ಺ͷ఺ Deep CNN ࣮ݱ͍ۭͨؒ͠ͷڑ཭తͳ ͓ؾ͕࣋ͪ࿅Γ͜·ΕͨLossؔ਺ ֶश
  24. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !24 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ

  25. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !25 https://twitter.com/sanacottoncandy/status/1131140431775801345 https://twitter.com/WinterFrame/status/1200973640197492736 https://twitter.com/JYPETWICE/status/1272053602790502400 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ

  26. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !26 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ αφͪΌΜ π΢ΟͪΌΜ

  27. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !27 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ

    ը૾ݕࡧ ΫΤϦ Top-1(࠷ۙ๣) https://twitter.com/JYPETWICE_JAPAN/status/1156761954024734721
  28. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !28 ಉ͡Ϋϥεϥϕϧͷ͍ͭͨը૾͸ۙ͘ ҧ͏Ϋϥε͸཭ΕͯຒΊࠐ·ΕΔ ʮಉ͡Ϋϥεͱ൑அ͢Δڑ཭ͷᮢ஋ʯΛ ͍͍ײ͡ʹઃఆ͢Δ ը૾෼ྨ (ओʹϢʔΫϦουڑ཭Ͱ)ΫΤϦը૾ͷ
 ຒΊࠐΈ఺ʹ͍ۙॱʹྨࣅը૾ͱ͢Δ

    ը૾ݕࡧ ෼ྨʹՃ͑ͯະ஌ͷΫϥε΋ ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ը૾ࣝผ https://twitter.com/JYPETWICE_JAPAN/status/1156761692547629056 New?
  29. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !29 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ

  30. ը૾ೝࣝ: ֤Ϋϥε͕·ͱ·Δಛ௃ྔۭؒͰࣝผ !30 ະ஌ͷΫϥε΋ʮະ஌ͷΫϥεʯͱͯ͠ରԠͰ͖Δ ʮֶश࣌ʹແ͍Ϋϥε͕ਪ࿦ʹग़ͯ͘Δʯ՝୊ઃఆʹ˕ • Face Recognition / Verification

    • Person Re-Identification • Image Retrieval ͳͲͳͲ...
  31. !31 ଞʹ΋ Metric Learning for…

  32. Adversarial Attack(ݸਓతʹ໘ന͔ͬͨͷͰ͝঺հ) !32 “Metric Learning for Adversarial Robustness”(NeurIPS’19) Adversarial Example͸

    ಛ௃ྔۭؒʹ͓͍ͯ Ϋϥεͷڥք෇ۙʹҐஔ͍ͯ͠Δʁ(͔Βؒҧ͑Δʁ) Adversarial ExampleΛ ຊ౰ͷΫϥεʹ͍ۙํ޲ʹ ಈ͔͢Loss ը૾ͱAdversarialͳը૾Λಛ௃ྔۭؒʹ౤Ө͢Δͱ…ʁ
  33. !33 “OpenGAN: Open Set Generative Adversarial Networks”(arXiv 2020/5/18) ҟৗݕ஌ “Unsupervised

    Anomaly Detection with Generative Adversarial Networks
 to Guide Marker Discovery”(IPMI’17) GAN Distillation “Relational Knowledge Distillation”(CVPR’19)
  34. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ !34

  35. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !35

  36. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !36

  37. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !37 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
  38. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !38 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ಉ͡Ϋϥεͷը૾
  39. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !39 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ۙͮ͘ ಉ͡Ϋϥεͷը૾
  40. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !40 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
  41. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !41 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾
  42. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !42 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ΑΓ ཭ΕΔ ҧ͏Ϋϥεͷը૾
  43. ࢝૆: 2ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Contrastive Loss !43 Deep CNN x1 x2 : :

    : : : : : :
 : xN x1 x2 : : : : : : : :
 : xN ҧ͏Ϋϥεͷը૾ “Dimensionality Reduction by Learning an Invariant Mapping” (CVPR’06)
  44. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !44 [~]+: ώϯδؔ਺(= max(~, 0))

  45. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !45 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) negative anchor positive
  46. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !46 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dp ʹϚʔδϯαΛ଍ͯ͠΋ ·ͩdn ͷํ͕σΧ͘ͳͬͯ΄͍͠ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍…
  47. ࢝૆ͷࢠ(ʁ): 3ͭͷσʔλ఺Λ͚ۙͮΔ/཭͢Triplet Loss !47 Deep CNN anchor: ͋Δσʔλ positive: anchorͱ

    ಉΫϥε negative: ผͷΫϥε [~]+: ώϯδؔ਺(= max(~, 0)) anchor positive negative dn dp dpʹϚʔδϯαΛ଍ͯ͠΋ dnΑΓখ͘͢͞Δ • จ຺(ϥϕϧͱλεΫ)ʹؔ܎ͳ͘
 ૬ରతʹۙ͘/ԕ͘͢Ε͹Α͍ • ૊Έ߹Θͤ਺͕๲େͰ
 ֶश͕ऩଋ͠ͳ͍… “Learning Fine-grained Image Similarity with Deep Ranking” (CVPR’14)
  48. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿

    • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !48 ͳͲͳͲ…
  49. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !49 “FaceNet: A Unified Embedding for

    Face Recognition and Clustering”(CVPR’15) !49 !49 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ Tripletͷ૊Έ߹Θͤʹ͍ͭͯͷʮ೉͠͞ʯΛఆٛ Loss͕େ͖͘ͳΔSemi-hard NegativeϖΞΛޮ཰Α͘࡞੒
  50. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !50 “Improved Deep Metric Learning with

    Multi-class N-pair Loss Objective”(NIPS’16) NΫϥε͔Β2ը૾ͣͭ
 औΓग़ͯ͠ಛ௃ྔʹม׵ N૊ͷηοτΛ࡞Γ
 LossΛܭࢉ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
  51. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿

    • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ !51 “L2-constrained Softmax Loss for Discriminative Face Verification”(arXiv 2017/3) ໌ࣔతͳڑ཭࠷దԽ͸͠ͳ͍͕ ಛ௃ྔϕΫτϧΛϊϧϜͰׂͬͯ ಛ௃ྔ࣍ݩۭؒͷٿ໘্ʹ ಺ੵ(cosྨࣅ౓)Ͱදݱྗͷௐ੔ ֶशͷ೉қ౓΋εέʔϦϯάύϥϝʔλαͰௐ੔
  52. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !52 “Deep Metric Learning with Angular

    Loss”(ICCV’17) anchorɺpositiveɺ negativeΛಈ͔͢ ޲͖(ޯ഑)Λ ֯౓͔Βߟ͑Δ • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு • ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄
  53. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !53 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு •

    ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “No Fuss Distance Metric Learning using proxies”(ICCV’17) positiveͱnegativeʹ ݸผͷσʔλ఺Ͱ͸ͳ͘ ֤Ϋϥεͷ୅ද஋ͷ ຒΊࠐΈϕΫτϧ(proxy) Λ༻͍Δ ѹ౗తऩଋͷ଎͞ ຊ೔ͷ࿦จͷϥΠόϧ
  54. Contrastive / Triplet Loss͔Β೿ੜ→੝Γ্͕͖ͬͯͨ !54 • ֶशʹ༗ޮͳϖΞΛ୳͢/࡞Δ • ͞Βʹߟྀ͢ΔΫϥε਺Λ֦ு •

    ۭؒΛ௒ٿ໘্ʹ੍໿ • ڑ཭ࢦඪΛ֯౓Ͱ • ୅ද఺ͰMetric Learning • Ϋϥε෼ྨܕωοτϫʔΫߏ଄ “ArcFace: Additive Angular Margin Loss for Deep Face Recognition”(CVPR’19) ಛ௃ྔϕΫτϧͷޙஈʹશ݁߹૚Λ઀ଓ Softmaxؔ਺ͱCross Entropy LossͰ Metric LearningΛߦ͏(ਪ࿦࣌͸શ݁߹૚Λ֎͢) ຊ೔ͷ࿦จͷϥΠόϧ
 Ͱ͸ͳ͍͕ඞಡʂ
  55. ͓·͚: CVPR’20ͰMetric Learningͷ࿩ͯ͠Δ࿦จ(ͷҰ෦) !55 •Deep Metric Learning via Adaptive Learnable

    Assessment •End-to-End Illuminant Estimation Based on Deep Metric Learning •Cross-Batch Memory for Embedding Learning •Optimizing Rank-Based Metrics With Blackbox Differentiation •Universal Weighting Metric Learning for Cross-Modal Matching •Fast(er) Reconstruction of Shredded Text Documents via
 Self-Supervised Deep Asymmetric Metric Learning •Moving in the Right Direction: A Regularization for Deep Metric Learning •CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition •Circle Loss: A Unified Perspective of Pair Similarity Optimization
  56. ౳ʑ… ͋͘·Ͱ͘͝Ұ෦ͷ࿩͔͍ͭܰ঺հͰ͢ͷͰ ͍Ζ͍Ζ୳ͨ͠Γࢼͨ͠Γͯ͠Έ͍ͯͩ͘͞ʂ (࠶ܝ) !56

  57. ࠓ೔ͷΰʔϧ(࠶ܝ) !57 ͷ͓ؾ࣋ͪΛཧղ͢Δ

  58. ࠓ೔ͷΰʔϧ(࠶ܝ) !58 Triplet Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ

    ଐ͢Δผͷσʔλ఺ • negative: anchorɺpositiveͱ͸ ҟͳΔΫϥεͷσʔλ఺ ↓ • anchorͱpositive͕͖ۙͮɺ anchorͱnegative͕཭ΕΔLoss • σʔλͷ૊Έ߹Θͤ૯਺ଟ͗͢ • ֶशʹޮ͘ϖΞʹ౰ͨΓʹ͍͘
  59. ࠓ೔ͷΰʔϧ(࠶ܝ) !59 N-pair Loss • anchor: ͋Δσʔλ఺ • positive: anchorͱಉ͡Ϋϥεʹ

    ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷσʔλ఺(1ͭͣͭ) • ۙ͞Λߟྀ ↓ • TripletΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (ϛχόονͷதͷ͞ΒʹҰ෦)
  60. ࠓ೔ͷΰʔϧ(࠶ܝ) !60 Lifted Structure Loss • anchor: ͋Δσʔλ఺ • positive:

    anchorͱಉ͡Ϋϥεʹ ଐ͢Δผͷσʔλ఺ • negatives: anchorɺpositiveͰ ͳ͍Ϋϥεͷશσʔλ఺ ↓ • N-pairΑΓ΋Ұ౓ʹશମΛݟΕΔ • ·ͩσʔλશମͱ͸ݴ͑ͳ͍
 (anchorΫϥεͷҰ෦σʔλ΍ϛ χόον಺ͷΈͰݶఆత)
  61. ࠓ೔ͷΰʔϧ(࠶ܝ) !61 Proxy-NCA Loss • anchor: ͋Δσʔλ఺ • proxy: ֤Ϋϥεʹ͖ͭ1ͭ͋Δ

    ʮ୅ද఺ʯ ↓ • anchorͱશproxyͰ
 N-pair Lossతͳֶश • ऩଋ͕ΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใ͸
 ׆͔͖͠Εͯͳ͍
  62. ࠓ೔ͷΰʔϧ(࠶ܝ) !62 ࢒͢͸ຊ೔ͷ࿦จ Proxy Anchor Loss ͷΈ

  63. ࠓ೔ͷΰʔϧ(࠶ܝ) !63 Proxy Anchor Loss • anchor: ͋ΔΫϥεͷproxy • positive:

    anchorͷproxyʹଐ͢ ΔΫϥεͷϛχόον಺શσʔλ ఺ • negative: anchorͱҟͳΔΫϥε ͷϛχόον಺શσʔλ఺ ↓ • ऩଋ͕͞ΒʹΊͪΌͪ͘Ό଎͍ • σʔλશମͷۭؒత৘ใΛ׆͔͠ ·͍ͬͯ͘Δ
  64. ࠓ೔ͷΰʔϧ(࠶ܝ) !64 ͍͟ղઆ΁

  65. ͓͠ͳ͕͖ • Metric Learningͬͯʁ • CVʹ͓͚ΔMetric LearningͷྲྀΕ • ຊ೔ͷ࿦จ঺հ !65

  66. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss !66

  67. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ

    ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ ϛχόον಺ͷશembedding (σʔλಛ௃ྔ)ϕΫτϧ
  68. ఏҊͯ͠Δ಺༰: Proxy Anchor Loss α: εέʔϦϯάύϥϝʔλ δ: Ϛʔδϯ ϛχόον಺ͷσʔλ શΫϥε෼ͷproxyͨͪͱ

    ͦͷݸ਺ proxy pͷΫϥεʹଐ͢Δ શembedding(σʔλಛ௃ྔ) ϕΫτϧ embeddingϕΫτϧxͱ ରԠ͢ΔΫϥεͷproxyϕΫτϧpͷ ྨࣅ౓(ຊ࿦จͰ͸cosྨࣅ౓) શproxyͱ ͦͷݸ਺(=Ϋϥε਺) proxy pͷΫϥεʹଐ͞ͳ͍ શembedding(σʔλಛ௃ྔ)ϕΫτϧ ʁ
  69. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !69

  70. Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !70 “easier-to-interpret form”

  71. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !71 “smooth approximation of ReLU”

    ͭ·Γඍ෼Մೳ࿈ଓͳmax(0, z)
  72. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪΛཧղ͍ͨ͠ !72 “smooth approximation of max

    function” LogSumExp
  73. “easier-to-interpret form” Proxy Anchor Lossͷ͓ؾ࣋ͪ !73 ͋Δproxy pͱhardest positive (p͔ΒҰ൪ԕ͍positive)͕ۙͮ͘

    ͋Δproxy pͱhardest negative (p͔ΒҰ൪͍ۙnegative)͕཭ΕΔ “smooth approximation of max function”
  74. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !74

  75. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !75 proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ

  76. Proxy Anchor Lossͷਅͷ͓ؾ࣋ͪ !76 anchorͱ֤positiveͱͷྨࣅ౓͕ ߴ͍΄Ͳখ͘͞ͳͬͯخ͍߲͠ proxyΛanchor(ج४)ʹͯ͠ positive΋negative΋ શͯͷྨࣅ౓ݟͯΔɺͷਤ anchorͱ֤negativeͱͷྨࣅ౓͕

    ௿͍΄Ͳখ͘͞ͳͬͯخ͍߲͠
  77. !77 ͲΜͳͱ͜Ζ͕ྑ͘ͳͬͯΔʁ

  78. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !78

  79. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !79 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  80. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !80 σʔλͷۭؒతߏ଄΍෼෍Λ
 Α͘ଊֶ͑ͨश͕Ͱ͖͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  81. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !81 ୅ද఺Λ࢖͍ͳ͕Β σʔλશମ΋ݟΕ͍ͯΔ Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ

  82. ࿦จᐌ͘1: ੑೳྑͯ͘ऩଋ଎͍ʂ !82 Recall@1είΞ͕
 Ұ൪ߴ͍ʂ ऩଋ͕଎͍ʂ Recall@KείΞ • Retrieval(ݕࡧ)λεΫ΍Metric LearningͷੑೳΛଌΔͷʹΑ͘࢖ΘΕΔ

    • ֶशͨ͠ม׵(ωοτϫʔΫ)ʹςετσʔλΛશͯೖྗͯ͠embeddingԽͨ࣌͠ʹ
 ֤σʔλ఺ʹ͍ͭͯɺͦͷۭؒ಺Ͱ͍ۙॱϥϯΩϯάTop-KҐͷதʹ
 ಉ͡Ϋϥεͷσʔλ఺͕͋ΔΑ͏ͳ఺ͷׂ߹ • ྫ͑͹Recall@2ͳΒɺRecall@1(࠷ۙ๣఺͕ಉ͡ΫϥεͰ͋ΔΑ͏ͳ఺ͷׂ߹)ʹ
 ʮ࠷دΓ͸ผΫϥεͷσʔλ఺͕ͩ2൪໨ʹ͍ۙ఺ͳΒಉ͡ΫϥεʯͰ͋ΔϠπΒΛ଍ͨ͠
 είΞͱͳΔ • ʮະ஌ΫϥεΛ͖ͪΜͱ෼཭Ͱ͖Δ͔ʯΛଌΔͨΊɺσʔληοτΛ2෼ׂͯ͠
 ֶशͱςετΛ׬શʹผͷΫϥεͰߦ͏(ͳ͔ͳ͔ϋʔυ)
  83. ࿦จᐌ͘2: ྨࣅ౓ʹԠͯ͡஋͕ͪΌΜͱมΘΔʂ !83 Proxy-NCA Loss Proxy Anchor Loss positive proxyʹ͍ͭͯ

    มԽ͕ݻఆ શσʔλ఺ʹ͍ͭͯ hard͞ʹԠֶͨ͡श
  84. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !84 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U:

    1Ϋϥε͋ͨΓͷproxy਺
  85. Ϋϥε਺C << σʔλ૯਺M ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !85 M: ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺

    B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
  86. ࿦จᐌ͘3: ૊Έ߹Θͤ਺͕(طଘݚڀͷதͰ)࠷খʂ !86 ૊Έ߹Θͤ૯਺͕ଟ͗ͯ͢
 ֶशʹ༗ޮͳσʔλϖΞΛ Ҿ͖౰ͯΔͷʹ͔͔࣌ؒΔ Ϋϥε਺C << σʔλ૯਺M M:

    ֶशσʔλͷ૯਺ C: ֶशσʔλͷΫϥε਺ B: 1epochͷόον਺ U: 1Ϋϥε͋ͨΓͷproxy਺ ϙΠϯτ
  87. !87 ࣮ݧͨ͠ΒSOTAͳΜͰ͠ΐ…ʁ

  88. ࣮ݧ݁Ռ: SOTAʂ !88

  89. ࣮ݧ݁Ռ: SOTAʂ !89 ௗ ं

  90. ࣮ݧ݁Ռ: SOTAʂ !90 ಛ௃ྔͷ ࣍ݩ਺Λ ͪΌΜͱ ߹Θͤͯ ൺֱʂ (݁ߏޮ͍ͯ͘Δ)

  91. ࣮ݧ݁Ռ: SOTAʂ !91 σΧ͍ը૾࢖ͬͯΔͷʹ΋߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)

  92. ࣮ݧ݁Ռ: SOTAʂ !92 backbone networkهࡌͯ͠Δ͠߹ΘͤͯΔʂ(݁ߏޮ͍ͯ͘Δ)

  93. ࣮ݧ݁Ռ: SOTAʂ !93 ஸೡʂ

  94. ࣮ݧ݁Ռ: SOTAʂ !94 Stanford Online Products (ΊͪΌͪ͘Ό͍ΖΜͳ෺) In-shop(ϑΝογϣϯΞΠςϜ)

  95. ࣮ݧ݁Ռ: SOTAʂ !95 ஸೡʂ ஸೡʂ

  96. !96 ͚ͩͰ͸ͳ͘…ʂ

  97. ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁ՌTop4 !97 CUB-200 SOP In-shop Cars-196 ΋͸΍ ؒҧͬͯΔϠπ ਓ͕΍ͬͯ΋ Ϝζ͍͡ΌΜ…

  98. ϋΠύϥߟ࡯ͦͷ1: όοναΠζ !98 ʮ1όον಺ͷσʔλ਺͕ଟ͍ํ͕
 σʔλશମͷߏ଄ΛΑΓଊ͑ΒΕΔʯ ҙ֎ͱ ͔ࠩͭͳ͍ͳ…

  99. ϋΠύϥߟ࡯ͦͷ2: ಛ௃ྔۭؒͷ࣍ݩ਺ !99 ʮେ͖͍ํ͕΋ͪΖΜੑೳྑ͍ ͕ ݕࡧ଎౓ͱ͔͸τϨʔυΦϑʯ σΧ͗͢Δͱ ੑೳԼ͕ΔΜͩ…

  100. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !100 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ…

  101. ϋΠύϥߟ࡯ͦͷ3: εέʔϦϯάύϥϝʔλαͱϚʔδϯδ !101 ʮεέʔϦϯάύϥϝʔλ͸ 16Ҏ্Ͱ҆ఆͯ͠Δɺ ͕ͦͬͪे෼ͳେ͖ͩ͞ͱ ϚʔδϯͰ͸͋Μ·Γ͕ࠩग़ͳ͍ʯ εέʔϦϯά ύϥϝʔλ ࢮ׆໰୊͡ΌΜ…

    • ෳ਺ͷೖྗಉ࢜ͷʮࠩΛڧௐ͢Δ౓߹͍ʯ • cosྨࣅ౓Λ࢖͏Metric LearningͰΑ͘ಋೖ͞ΕΔ
 (ͦ΋ͦ΋ͷ஋Ҭ͕-1 ≦ cosx ≦ 1ͳͷͰ͕͖ࠩͭʹ͍ͨ͘Ί) • DistillationΛఏҊͨ͠࿦จͰొ৔ͨ͠softmaxԹ౓ύϥϝʔλΛٯ਺ʹͨ͠΋ͷ ※͍͔ͭ͘ͷ࿦จΛݟ͖ͯͨײ͡20~60͙Β͍͕ଥ౰ͳΠϝʔδͩͬͨͷͰೲಘ εέʔϦϯάύϥϝʔλ “Distilling the Knowledge in a Neural Network”(NIPS’14 Workshop)
  102. ͞ΒʹarXiv൛Ͱ͸… !102

  103. Appendix1: backbone networkͱը૾αΠζ !103 ʮ࣮ݧ͸Inception-BNͰ΍͚ͬͨͲResNetͩͱ΋ͬͱੌ͍ɺ GPUϝϞϦ࣍ୈͰը૾͸σΧ͍ํ͕ྑ͍ʯ ͦΓΌͦ͏ͩ…

  104. Appendix2: ΫΤϦը૾ͷྨࣅ౓ݕࡧ݁Ռ(vs Proxy-NCA) !104 CUB-200 SOP In-shop Cars-196

  105. Appendix3: ςετσʔλͷ෼෍ͲΜͳײ͡ʁ(t-SNEՄࢹԽ) !105 CUB-200 SOP In-shop Cars-196

  106. ͞Βʹ͞Βʹ… !106

  107. ͔ͳΓ៉ྷͳஶऀPyTorch࣮૷ʂ !107 https://github.com/tjddus9597/Proxy-Anchor-CVPR2020 • LossΫϥε͔Βtrain/testίʔυͷ࣮૷͸΋ͪΖΜ • σʔληοτ4छͷDatasetΫϥε࣮૷΋͋Γ • backbone networkͨͪͷ࣮૷΋͋Γ

    • طଘݚڀͷLossͨͪ΋ࢼͤΔΑ͏ʹͳͬͯΔ
 (Pytorch-Metric-Learningύοέʔδ͕ඞཁ) ※͜ͷൃද·Ͱʹಈ͔͓͖͔ͯͨͬͨ͠ͷͰ͕࣌ؒ͢଍Γͣʀʀ
  108. ౳ʑ Deep Metric Learning in CVͷݱঢ়͕ ͱͯ΋ྑ͍ײ͡ʹ·ͱ·͍ͬͯΔҹ৅Ͱ͢ ͜ͷࢿྉยखʹ͝ҰಡΛʂ !108

  109. ·ͱΊ !109 •Metric Learning(ಛʹCV෼໺)ͷ࿩ •ຊ೔ͷ࿦จ: Proxy Anchor Loss ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ

  110. !110