$30 off During Our Annual Pro Sale. View Details »

メルカリにおける写真検索機能

KosukeArase
October 25, 2019

 メルカリにおける写真検索機能

2019/10/25 に開催された,Bonfire Data & Science #1 にてメルカリの写真検索 (画像検索) 機能についてお話した際の資料です!
モデリングから運用,そして写真検索機能はどのように進化していくかについて,メルカリが写真検索に関連して発表した2本の論文を軸にご紹介しています.

KosukeArase

October 25, 2019
Tweet

More Decks by KosukeArase

Other Decks in Technology

Transcript


  1. ϝϧΧϦʹ͓͚Δࣸਅݕࡧػೳ
    ߥ੉ߊհ !,PTVLF"SBTF

    4PGUXBSF&OHJOFFSBU"*&OHJOFFSJOH5FBN

    View Slide


  2. ࣗݾ঺հ

    ߥ੉ߊհ ,PTVLF"SBTF

    w "*&OHJOFFSJOH5FBNΤϯδχΞ
    w ౦ژେֶେֶӃݪాݚڀࣨଔ
    w ೥݄d೥݄Πϯλʔϯ
    w ೥݄৽ଔೖࣾ
    w ೥݄dࣸਅݕࡧϓϩδΣΫτ 5FDI-FBE

    w ઐ໳ը૾ೝࣝɼ࣍ݩ఺܈ೝࣝ
    w ग़඼࣌ͷը૾ೝࣝػೳࣸਅݕࡧػೳ
    w 5XJUUFS (JU)VC!,PTVLF"SBTF

    View Slide


  3. 8IBUJTࣸਅݕࡧ
    ಈըϦϯΫIUUQTZPVUVCFL5OJ&W0$H*
    w ͍ΘΏΔը૾ݕࡧػೳ J04
    ͷΈ
    w ঎඼໊Λ஌Βͳͯ͘΋ը૾͔Β঎඼ΛݕࡧՄೳ

    View Slide

  4. 4ZTUFN0WFSWJFX

    ը૾ొ࿥ϑϩʔ USBJOJOH

    ը૾ݕࡧͷϑϩʔ TFSWJOH

    *OWFSUFEGJMFTZTUFNXJUIUIF
    BTZNNFUSJDEJTUBODFDPNQVUBUJPO
    *7'"%$*7'12
    <+ÉHPV >
    Object Detection
    (SSD-MobileNet v2)

    View Slide


  5. 8IBUJTࣸਅݕࡧ
    w ࣸਅݕࡧͷϞσϦϯάͱγεςϜʹؔ࿈ͯ͠ܭຊͷ࿦จΛൃද
    w 5BLVNB:BNBHVDIJ ,PTVLF"SBTF 3JLV5PHBTIJ 4IVOZB6FUB 

    $MPTJOHUIF(BQ#FUXFFO2VFSZBOE%BUBCBTF

    UISPVHI2VFSZ'FBUVSF5SBOTGPSNBUJPOJO$$F$PNNFSDF7JTVBM4FBSDI 

    5IF4*(*38PSLTIPQ0OF$PNNFSDF 4*(*3F$PN`

    w ࢁޱ୓ਅɼதՏ޺จɼઍ༿ཽհɼߥ੉ߊհɼ

    $$ϚʔέοτϓϨΠεʹ͓͚ΔϏδϡΞϧݕࡧγεςϜɼ

    ୈճը૾ͷೝࣝɾཧղγϯϙδ΢Ϝ .*36
    ɼ
    w ·্ͣͷຊͷ࿦จΛ঺հ͢ΔܗͰϝϧΧϦͷࣸਅݕࡧٕज़ʹ͍ͭͯ࿩͠ɼ

    ͦͷޙࠓޙͷల๬ʹ͍͓ͭͯ࿩͠͠·͢ɽ

    View Slide

  6. $MPTJOHUIF(BQ#FUXFFO2VFSZBOE%BUBCBTF
    UISPVHI2VFSZ'FBUVSF5SBOTGPSNBUJPO
    JO$$F$PNNFSDF7JTVBM4FBSDI
    5BLVNB:BNBHVDIJ ,PTVLF"SBTF 3JLV5PHBTIJ 4IVOZB6FUB
    4*(*3F$PN`

    View Slide

  7. ໰୊఺

    w $$F$PNNFSDFͰ͸ɼग़඼ऀ΋ߪೖऀ΋Ұൠফඅऀ
    w ଟ͘ͷ৔߹ɼ঎඼͸চ΍ςʔϒϧ্ͰࡱӨ͞ΕΔ
    w ҰํɼΫΤϦʹ͸ண༻ը૾͕ଟ͍͕ɼ

    ͦͷ৔߹ண༻ը૾͕ݕࡧ͞Ε΍͍͢
    w ͜ͷΑ͏ͳݕࡧ݁Ռ͸

    #$F$PNNFSDFΛ࿈૝ͤ͞ɼ

    ग़඼ऀͷग़඼ҙཉʹѱӨڹΛ

    ٴ΅͢Մೳੑ͕͋Δ<)BHJV >
    ")BHJVFUBM l/FUXPSL&GGFDUT"SFO`U&OPVHIz 

    )BSWBSE#VTJOFTT3FWJFX

    View Slide

  8. ͱͳΔΑ͏ͳ
    ಛ௃ม׵ϕΫτϧ
    Λࢉग़͢Δ
    ఏҊख๏lண༻zͷಛ௃ΛऑΊΔಛ௃ม׵

    ʔ
    ʔ
    ʔ
    ʔ
    ಛ௃நग़ث




    View Slide

  9. ఏҊख๏lண༻zͷಛ௃ΛऑΊΔಛ௃ม׵


    ʔ
    ʔ
    ʔ
    ʔ
    ಛ௃நग़ث
    ੒ཱ͠ͳ͍



    View Slide

  10. ಛ௃දݱ

    ண༻ը૾͔Βɼlண༻zͷϕΫτϧ ಛ௃ม׵ϕΫτϧ
    ΛҾ͘
    ʔ
    ʔ
    Λɼ-ϊϧϜ͕



    ͷඇෛϕΫτϧͷಛ௃நग़ثͱߟ͑ͯ

    View Slide

  11. ಛ௃දݱ

    lண༻zը૾Ͱ͸ͳ͍ಛ௃ϕΫτϧ͔Βɼ
    lண༻zͷϕΫτϧ ಛ௃ม׵ϕΫτϧ
    ΛҾ͍ͨ࣌ͷӨڹΛͳͤ͘Δ͔΋͠Εͳ͍
    ʔ

    ʔ
    Λɼ-ϊϧϜ͕



    ͷඇෛϕΫτϧͷಛ௃நग़ثͱߟ͑ͯ

    View Slide

  12. ಛ௃ม׵ϕΫτϧ

    τοϓεண༻ը૾ τοϓεඇண༻ը૾
    ϘτϜεண༻ը૾ ϘτϜεඇண༻ը૾

    View Slide

  13. ಛ௃நग़ث
    w .PCJMF/FUW
    w ΤοδσόΠεͰͷॲཧΛݟਾ͑ͯͷϞσϧબ୒
    w ࣝผ໰୊ͱֶͯ͠श
    w (MPCBM"WFSBHF1PPMJOHͷग़ྗ E
    Λ

    -ਖ਼نԽͨ͠΋ͷΛಛ௃ϕΫτϧͱͯ͠ར༻
    w σʔληοτ
    w ໿ඦສը૾ɼ໿ Ϋϥε
    w ϒϥϯυɼ໛༷ɼΧςΰϦͷ૊Έ߹Θͤ
    FHʮφΠΩετϥΠϓΰϧϑϙϩγϟπʯ
    ʮ༡ٔԦτϨʔσΟϯάΧʔυʯ
    w ֶश݁Ռ
    w "DDVSBDZ5PQ 5PQ

    ඇෛͷཁૉͰߏ੒͞ΕΔϕΫτϧ

    View Slide

  14. ݕূ݁Ռ

    ඇண༻ը૾ ண༻ը૾ ண༻ը૾ $SPQQFE

    ඇண༻ண༻ը૾Λ ຕͣͭɼ߹ܭ ຕ༻ҙ͠ɼຕΛΫΤϦը૾ͱͯ͠ར༻
    ݕࡧ݁Ռ͕ඇண༻ը૾͔ͭΧςΰϦ͕Ұகͨ͠৔߹ʹਖ਼ղͱΈͳ͠ɹ
    N"1!ɹ
    ͰධՁ
    ண༻ը૾ʹରͯ͠େ෯ͳվળ͕ΈΒΕɼඇண༻ը૾ʹରͯ͠΋ޮՌ͕ಘΒΕͨ
    N"1!

    View Slide

  15. ݕূ݁Ռ ԯ঎඼͔Βͷݕࡧ


    View Slide

  16. ݕূ݁Ռ ԯ঎඼͔Βͷݕࡧ


    ಛ௃ม׵ʹ൐͏෰০඼Ҏ֎΁ͷӨڹ΋΄ͱΜͲݟΒΕͳ͍

    View Slide

  17. ·ͱΊ

    w $$F$PNNFSDFಛ༗ͷը૾ݕࡧͰͷ՝୊ʹର͢ΔऔΓ૊Έ
    w ݁Ռʹண༻ը૾͕ଟؚ͘·ΕΔͱग़඼ʹ΋ӨڹΛٴ΅͢Մೳੑ͕͋Δ
    w ΫΤϦը૾ͷಛ௃͔Βlண༻zͷಛ௃ΛऑΊΔಛ௃ม׵ΛఏҊ
    w ண༻ը૾ͷΫΤϦʹରͯ͠ݕࡧਫ਼౓ͷେ෯ͳվળΛ֬ೝ
    w ඇண༻ը૾ʹ͍ͭͯ΋ѱӨڹ͸ݟΒΕͣɼఏҊख๏ͷ༗ޮੑΛ֬ೝ
    w ΫΤϦ΁ͷޙॲཧͷΈͷૢ࡞Ͱ͋ΔͨΊɼܰྔ͔ͭมߋ͕༰қ
    w ࠓޙͷ՝୊ɿఏҊͨ͠ಛ௃ม׵ͷ༗ޮੑʹ͍ͭͯͷཧ࿦తઆ໌

    View Slide


  18. $$ϚʔέοτϓϨΠεʹ͓͚Δ

    ϏδϡΞϧݕࡧγεςϜ
    ࢁޱ୓ਅɼதՏ޺จɼઍ༿ཽհɼߥ੉ߊհ
    גࣜձࣾϝϧΧϦ

    .*36

    4ZTUFN5SBDL0SBM

    View Slide

  19. 4ZTUFN0WFSWJFX

    ը૾ొ࿥ϑϩʔ USBJOJOH

    ը૾ݕࡧͷϑϩʔ TFSWJOH

    *OWFSUFEGJMFTZTUFNXJUIUIF
    BTZNNFUSJDEJTUBODFDPNQVUBUJPO
    *7'"%$*7'12
    <+ÉHPV >
    Object Detection
    (SSD-MobileNet v2)

    View Slide

  20. w %PDLFS
    w ίϯςφ؀ڥ 04ϨϕϧͷԾ૝Խ؀ڥ
    Λఏڙ͢Δϛυϧ΢ΣΞ
    w ҟͳΔ؀ڥͰૉૣ͘ΞϓϦέʔγϣϯΛಈ࡞Մೳ
    w ӡ༻ίετͷ௿ݮ΍γεςϜͷ҆ఆԽ
    w ,VCFSOFUFT LT

    w ෳ਺ͷίϯςφͷσϓϩΠɼεέʔϦϯάɼ؅ཧΛߦ͏ͨΊͷ

    ίϯςφΦʔέετϨʔγϣϯγεςϜ
    w $VTUPN3FTPVSDF%FGJOJUJPO $3%
    ͱݺ͹ΕΔಠࣗͷϦιʔεΛఆٛͰ͖Δػೳ͕͋Γɺ

    ։ൃऀ͸ͦͷػೳΛհͯ͠LTͷػೳΛ֦ுͰ͖Δ
    w "84 ($1ͷϚϧνΫϥ΢υߏ੒Λ࠾༻ USBJOJOH&,4 TFSWJOH(,&


    %PDLFSBOE,VCFSOFUFT

    View Slide

  21. w$VTUPN3FTPVSDF%FpOJUJPO $3%

    ಠࣗʹϦιʔεΛఆٛͰ͖ΔLTͷػೳ
    wΧελϜϦιʔεͱΧελϜίϯτϩʔϥͰߏ੒
    wΧελϜίϯτϩʔϥ͕ΧελϜϦιʔεͷ

    ϥΠϑαΠΫϧঢ়ଶʹԠͯ͡

    Ϋϥελͷঢ়ଶΛίϯτϩʔϧ͢Δ
    8IBUJT$VTUPN3FTPVSDF%FGJOJUJPO $3%

    LVCFDUMDSFBUFGGPP@DSEZNM
    LVCFDUMDSFBUFGGPPZNM

    View Slide


  22. "SDIJUFDUVSFɹ

    View Slide


  23. "SDIJUFDUVSF֓ཁਤ
    "84 ($1

    View Slide


  24. 5SBJOJOHɾϦιʔεͷ࡞੒
    "84 ($1

    View Slide


  25. w όονॲཧͷ֤޻ఔΛݸผͷίϯςφΠϝʔδͰ࣮ߦ
    w ϥΠϒϥϦͷґଘؔ܎ͳͲ؀ڥφΠʔϒͳ.-ύΠϓϥΠϯͷ໰୊Λղܾ
    w ύΠϓϥΠϯ%"(͸:".-Ͱهड़
    w ֤޻ఔͷೖग़ྗ͸1FSTJTUFOU7PMVNFʢҎԼ17ʣΛհ͢
    5SBJOJOHɾίϯςφϕʔεύΠϓϥΠϯ
    $3%$VTUPN3FTPVSDF%FGJOJUJPO "//"QQSPYJNBUF/FBSFTU/FJHICPS

    View Slide


  26. 5SBJOJOHɾϦιʔεͷ࡞੒
    w )PVSMZ %BJMZ .POUIMZόον͕ଘࡏ
    w 5SBJOJOHDVTUPNSFTPVSDFΛ

    $SPO+PC͕࡞੒
    w $VTUPNDPOUSPMMFS͕

    DVTUPNSFTPVSDFͰઃఆ͞Εͨ

    ίϯςφϕʔεύΠϓϥΠϯΛ࣮ߦ

    View Slide


  27. 5SBJOJOHɾϦιʔεͷ࡞੒
    w શͯͷόον࣮ߦ৘ใ͕

    ΧελϜϦιʔεͱͯ͠

    LT্ʹ࢒Δ
    w ಉ͡ॲཧΛ࠶࣮ߦग़དྷΔͨ
    Ίɺόονͷ࠶࣮ߦΛ൐͏
    ো֐෮چ࡞ۀ͕༰қ

    View Slide


  28. ը૾ͷμ΢ϯϩʔυ
    "84 ($1

    View Slide


  29. w 4্ʹଘࡏ͢ΔϝϧΧϦɾΠϝʔδετΞ͔Β঎඼ը૾Λμ΢ϯϩʔυ
    w ύΠϓϥΠϯ্΋ͬͱ΋͕͔͔࣌ؒΔ޻ఔ ը૾਺͕๲େͳͨΊ

    w ͦͷͨΊ17ʹҰఆظؒΩϟογϡ͢ΔࣄʹΑͬͯ࠶ΠϯσοΫε͕

    ඞཁͳ࣌ʹ͸ૉૣ͘ύΠϓϥΠϯΛճͤΔΑ͏ʹ͍ͯ͠Δ
    ը૾ͷμ΢ϯϩʔυ

    View Slide


  30. ΞηοτͷΞοϓϩʔυ
    "84 ($1

    View Slide


  31. w ύΠϓϥΠϯͷ੒Ռ෺Ͱ͋Δಛ௃ϕΫτϧͱ

    ۙࣅ࠷ۙ๣୳ࡧ "QQSPYJNBUF/FBSFTU/FJHICPS "//
    *OEFYΛ

    ϞσϧϨϙδτϦʹόʔδϣϯ؅ཧ͞Εͨঢ়ଶͰอଘ
    w ϞσϧϨϙδτϦ͸($4 (PPHMF$MPVE4UPSBHF
    ্ʹߏங
    ΞηοτͷΞοϓϩʔυ

    View Slide


  32. 4FSWJOHΠϝʔδͷϏϧυ
    "84 ($1

    View Slide


  33. ϞσϧϨϙδτϦ͸*NBHF#VJMEFSͱݺ͹ΕΔEBFNPOʹΑΓ؂ࢹ͞Ε͍ͯΔ
    w ৽͍͠Ξηοτ͕௥Ճ͞ΕΔͱࣗಈͰ4FSWJOHίϯςφΠϝʔδΛϏϧυ
    w ίϯςφΠϝʔδ͸"//*OEFY౳4FSWJOHʹ

    ඞཁͳϦιʔεΛશؚͯΜͰ͍Δ
    w Ϗϧυ͞ΕͨίϯςφΠϝʔδΛίϯςφϨδετϦʹϓογϡ
    4FSWJOHΠϝʔδͷϏϧυ
    (PPHMF
    $MPVE
    4UPSBHF
    (PPHMF
    $POUBJOFS
    3FHJTUSZ
    *NBHF
    #VJMEFS
    %BFNPO
    ؂ࢹ
    "//*OEFY
    %PDLFS
    *NBHF

    View Slide


  34. 4FSWJOHϦιʔεͷ࡞੒
    "84 ($1

    View Slide


  35. 4FSWJOHɾϦιʔεͷ࡞੒
    w *NBHF#VJMEFS͸ɼ

    ίϯςφΠϝʔδΛϏϧυͨ͠ޙ

    4FSWJOHΧελϜϦιʔεΛ࡞੒
    w 4FSWJOHΧελϜίϯτϩʔϥ͸

    $3%ϦιʔεͷઃఆΛݩʹ

    ඞཁͳ%FQMPZNFOUɼ4FSWJDF౳Λ࡞੒
    w ຊγεςϜͰ͸ߏங͞Εͨ

    "//*OEFYΛݸผͷ

    *OEFYαʔϏεͱͯ͠σϓϩΠ

    View Slide


  36. 4FSWJDFσΟεΧόϦ
    "84 ($1

    View Slide


  37. αʔϏεσΟεΧόϦ
    w Ϋϥελ্ʹଘࡏ͢Δ*OEFYαʔϏ
    εΛ,VCFSOFUFTΛհͯࣗ͠ಈతʹ
    औಘ͢Δ
    w ͳΔ΂͘େ͖ͳཻ౓ͷ*OEFYΛ

    ࢖༻͢ΔΑ͏ɺҟͳΔظؒɾཻ౓
    ͷ*OEFYαʔϏε )PVSMZ %BJMZ
    .POUIMZ
    Λࣗಈతʹ૊Έ߹ΘͤΔ
    w 3&45*OEFYαʔϏεؒͷ

    ϓϩτίϧ͸H31$Λ࢖༻

    View Slide


  38. ֓ཁਤৼΓฦΓ
    "84 ($1

    View Slide


  39. .-1MBUGPSN-ZLFJPO
    ࣸਅݕࡧ͸-ZLFJPOͱݺ͹ΕΔ಺੡ͷ.-1MBUGPSN্ʹ
    ߏங͞Ε͓ͯΓɺԼهͷػೳ͸-ZLFJPOͷػೳΛ࢖༻͍ͯ͠Δ
    w 5SBJOJOH4FSWJOH$3%ΧελϜίϯτϩʔϥ
    w ίϯςφϕʔεύΠϓϥΠϯ
    w 5SBJOJOH4FSWJOHίϯςφΠϝʔδϏϧμʔ
    w ϞσϧϨϙδτϦ

    View Slide


  40. ϝϧΧϦʹ͓͚ΔϏδϡΞϧݕࡧγεςϜ·ͱΊ
    w ίϯςφϕʔεͷ࠶ݱੑͷߴ͍γεςϜ
    w ,VCFSOFUFTͷ$3%ΧελϜίϯτϩʔϥ΍αʔϏεσΟεΧόϦ౳Λ׆༻
    w ಺੡ͷ.-1MBUGPSN -ZLFJPO
    Ͱ࣮ݱ͞Ε͍ͯΔػೳΛར༻͠ɺ

    ίϯςφϕʔεύΠϓϥΠϯ౳ϩόετͳγεςϜΛߏங
    w Ϋϥ΢υΠϯϑϥΛ,VCFSOFUFTͰந৅Խ͢ΔࣄʹΑͬͯɺϚϧνΫϥ΢υΛ
    ҙࣝ͢Δ͜ͱͳ͘ӡ༻Մೳ

    View Slide


  41. /FYU'VUVSFɹ

    View Slide


  42. 3FBMUJNF*NBHF4FBSDI
    w &EHFଆͰPCKFDUEFUFDUJPOGFBUVSFFYUSBDUJPO
    w GFBUVSFWFDUPSΛΫΤϦͱ͠

    "//*OEFYTFSWJDF͔Βྨࣅ঎඼Λऔಘ
    w ϦΞϧλΠϜͳΠϯλϥΫγϣϯ
    w 69্େ͖ͳϝϦοτ͕͋Δ

    View Slide

  43. w ༷ʑͳτϨʔυΦϑ໰୊͕ଘࡏ͢Δ
    w "DDVSBDZ
    w -BUFODZ
    w &OFSHZDPOTVNQUJPO
    w .PEFMTJ[F
    w ໨తͷ69Λୡ੒͢ΔͨΊʹɺ

    ΞϧΰϦζϜɺΤϯδχΞϦϯά྆ํͰ
    ͦΕΒͷόϥϯεΛߟྀ͢Δඞཁ͕͋Δ
    8IBUNVTUIBQQFOUPNBLF%//XPSLPOFEHF
    ΦϖϨʔγϣϯʹΑͬͯίετ͕ҧ͏
    NPCJMFEFWJDFͰαϙʔτ͍ͯ͠Δ(16ΠϯλʔϑΣʔεͷγΣΞ

    View Slide

  44. 4FSWFS
    TJEF
    $MJFOU
    TJEF w αʔόଆͷ%//GSBNFXPSLʹ͸
    5FOTPS'MPX4FSWJOHΛ࢖༻
    w ΫϥΠΞϯτଆͰ͸5FOTPS'MPX
    -JUF.FEJB1JQFΛ࢖༻
    w .FEJB1JQFΛ࢖༻͢Δ͜ͱͰલॲ
    ཧ΍ޙॲཧ΋4*.%౳Λ࢖༻ͯ͠
    ޮ཰ԽͰ͖Δ
    5'
    -JUF

    0QUJPOBM


    &EHFGVODUJPOBSDIJUFDUVSF

    View Slide


  45. 5IBOLZPVBMMGPSDPNJOHUPEBZɹ
    4PGUXBSF&OHJOFFS &OHJOFFSJOH.BOBHFS
    8FBSFIJSJOH

    View Slide