類似画像検索の裏側 - 機械学習と近似近傍探索 - / similar_image_search_inside

Cd3d2cb2dadf5488935fe0ddaea7938a?s=47 monochromegane
September 12, 2018

類似画像検索の裏側 - 機械学習と近似近傍探索 - / similar_image_search_inside

ペパボ研究所 in da house #1

Cd3d2cb2dadf5488935fe0ddaea7938a?s=128

monochromegane

September 12, 2018
Tweet

Transcript

  1. - ػցֶशͱۙࣅۙ๣୳ࡧ - ࡾ୐༔հ / Pepabo R&D Institute, GMO Pepabo,

    Inc. 2018.09.11 ϖύϘݚڀॴ in da house #1 ྨࣅը૾ݕࡧͷཪଆ
  2. ϓϦϯγύϧΤϯδχΞ ࡾ୐༔հ!NPOPDISPNFHBOF (.0ϖύϘגࣜձࣾϖύϘݚڀॴ IUUQCMPHNPOPDISPNFHBOFDPN

  3. 1. ͳͥྨࣅը૾ݕࡧ͔ 2. ྨࣅը૾ݕࡧΛࢧ͑Δٕज़ 3. ྨࣅը૾ݕࡧͷಋೖ 4. ྨࣅը૾ݕࡧͷ͜Ε͔Β 3 ໨࣍

  4. 1. ͳͥྨࣅը૾ݕࡧ͔

  5. • ৘ใ୳ࡧϓϩηε͸γεςϜଆͷ ٕज़ৄࡉ͚ͩͰ͸ͳ͘ɺ͜ͷϓϩ ηεΛۦಈ͢ΔओମͰ͋Δར༻ऀ ͕ଘࡏ͢Δ • ར༻ऀαΠυΛத৺ͱͨ͠৘ใ୳ ࡧϓϩηεϞσϧͷઌߦݚڀʹΑ Γɺ৘ใཁٻ͸ɺ༷ʑͳจ຺ʹΑ ΓมԽ͍ͯ͘͜͠ͱ͕Θ͔͍ͬͯ

    Δ 5 ͳΊΒ͔ͳϚονϯάʹ޲͚ͯ ৘ใཁٻΛऔΓרٕ͘ज़ͱϓϩηεϞσϧؔ࿈ਤ จ຺ʹΑΓ࠷దͳԠ౴͕มԽ͢ΔͷͰ͋Ε͹ɺ ٕज़ؒͰͷ༏ҐੑධՁͰ͸ͳ͘ɺ ঢ়گʹԠͨ͡બ୒͕ྑ͍ͷͰ͸ͳ͍͔
  6. • ར༻ऀͷऔΓ͏Δߦಈ͕จ຺ʹΑΓมԽ͢ΔͱԾఆ্ͨ͠Ͱɺਫ਼៛ͳจ຺ͷ೺ Ѳฒͼʹ՝୊ઃఆͷ໌֬ԽࢧԉΛ௨ͯ͠ɺ௚ײతͳ໰͍߹ΘͤʹԠ౴Ͱ͖Δ࢓ ૊Έ 6 ͳΊΒ͔ͳϚονϯάʹ޲͚ͯ 1. จ຺ʹΑΓԠ౴ͷධՁ͕มΘΔ͜ͱͷ֬ೝ 2. ར༻ऀͷจ຺ͷύλʔϯ෼ྨͱࣝผख๏ͷཱ֬

    3. ඞཁʹԠͯ͡௚ײతͳ໰͍߹ΘͤʹԠ౴Ͱ͖Δݕࡧ৘ใٕज़ͷಋೖ 4. ՝୊ઃఆͷ໌֬Խࢧԉͷݕ౼
  7. 2. ྨࣅը૾ݕࡧΛࢧ͑Δٕज़

  8. • ಛ௃ྔม׵ • ݸʑͷը૾σʔλͷྨࣅੑΛఆྔతʹൺֱՄೳͳܗࣜʹม׵͢Δ • ۙ๣୳ࡧ • ม׵ͯ͠ಘΒΕͨಛ௃ྔͲ͏͠Λൺֱ͠ɼྨࣅ͢Δू߹Λಛఆ͢Δ 8 ྨࣅը૾ݕࡧΛࢧ͑Δٕज़

  9. ಛ௃ྔม׵

  10. • m࣍ݩͷϕΫτϧxΛn࣍ݩͷϕΫτϧy΁ࣸ૾͢Δؔ਺fΛਪఆ͢Δֶशख๏ 10 χϡʔϥϧωοτϫʔΫ: Ϟσϧ x1 x2 xm b W

    y1 y2 yn 1 2 o1 = σ( m ∑ i=0 w1i xi + b1 ) h 3 o = σ(Wx + b) w11 w12 . . w1m w21 w22 . . w2m . . wh1 wh2 . . whm x1 x2 . . xm + b1 b2 . . bh x f y h wh1 wh2 x1 x2 ೖྗ͝ͱʹॏΈXΛ৐ࢉ όΠΞε ͨ͠΋ͷΛ׆ੑԽؔ਺ʹ௨͢ શϊʔυʹର͢Δॲཧ͸ߦྻͱͯ͠ૢ࡞Ͱ͖Δ
  11. 11 χϡʔϥϧωοτϫʔΫ: ֶश x1 x2 xm y1 y2 yn x

    f y • m࣍ݩͷϕΫτϧxΛn࣍ݩͷϕΫτϧy΁ࣸ૾͢Δؔ਺fΛਪఆ͢Δֶशख๏ t tn t2 t1 W(1) W(2) b(1) b(2) ✓(i+1) = ✓(i) ↵rEk E = 1 2 X n (yn tn)2 E = X n tn ln yn rEk = ( @Ek @✓1 , .. @Ek @✓V ) ग़ྗͱਖ਼ղͷޡࠩΛද͢ଛࣦؔ਺Λఆٛ͢Δɻ ೋ৐࿨ޡࠩʢࠨʣɺަࠩΤϯτϩϐʔޡࠩʢӈʣ ଛࣦؔ਺ʹର͢Δ֤ύϥϝλͷภඍ෼ͷ஋ͷू·Γʢޯ഑ʣΛ༻ ͍ͯଛࣦؔ਺ͷ஋͕খ͘͞ͳΔΑ͏ʹύϥϝλΛߋ৽͢Δ͜ͱΛ ޯ഑߱Լ๏ͱݺͿɻಛʹҰ෦ͷ݁ՌͷΈΛར༻͢Δ֬཰తޯ഑߱ Լ๏͕Α͘༻͍ΒΕΔɻ Αͬͯޯ഑͕ফ͑ͳ͍Α͏ͳͳΊΒ͔ͳ׆ੑԽؔ਺͕޷·ΕΔɻ ·ͨɺ֤ύϥϝλͷภඍ෼ΛݸผʹٻΊΔͷͰ͸ͳ͘ɺχϡʔϥ ϧωοτϫʔΫΛදؔ͢਺GΛ߹੒ؔ਺ͱݟཱͯΔ͜ͱͰɺ্ྲྀ ͷޡࠩͱࣗ਎ͷೖྗ஋ͷΈΛ࢖ͬͯܭࢉྔΛ཈͑ͳ͕Βޮ཰తʹ ޯ഑ΛٻΊΔख๏Λޡࠩٯ఻೻๏ͱݺͿɻ
  12. χϡʔϥϧωοτϫʔΫ: ֶश 12 • ࢀߟ (PʹΑΔޯ഑߱Լ๏ཧ࿦ͱ࣮ફ IUUQTTQFBLFSEFDLDPNNPOPDISPNFHBOFHSBEJFOUEFTDFOUJOHPMBOH θϩ͔Β࡞Δ%FFQ-FBSOJOH

  13. ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ 13        

                      ੵ࿨ԋࢉ ৞ΈࠐΈ                  ࠷େ஋ ೖྗσʔλʹରͯ͠ੵ࿨ԋࢉΛߦ͏ϑΟϧλΛద༻͠ɺ ಛ௃Ϛοϓͱͯ͠ग़ྗ͢Δɻۭؒ৘ใΛߟྀͰ͖Δɻ ࠷దͳϑΟϧλͷॏΈ͸ֶशʹΑͬͯ֫ಘ͢Δɻ • ۭؒత৘ใΛߟྀͨ͠χϡʔϥϧωοτϫʔΫ ϓʔϦϯά ೖྗσʔλʹରۭͯؒ͠ͷू໿Λߦ͏ϑΟϧλΛద༻ ͢Δɻ࠷େ஋΍ฏۉ஋Λར༻͢ΔͷͰֶश͕ෆཁɻ ࡉ͔ͳҐஔมԽʹର͢Δؤڧੑʹߩݙ͢Δɻ ಛ௃நग़ث ࣝผث
  14. 14

  15. ۙ๣୳ࡧ

  16. ߴ࣍ݩϕΫτϧͷۙ๣୳ࡧ wҰൠʹը૾ͷಛ௃ྔ͸ߴ࣍ݩͷϕΫτϧͱͯ͠දݱ͞ΕΔ wߴ࣍ݩͷϕΫτϧू߹͔ΒΫΤϦͱͳΔϕΫτϧͱڑ཭ͷ͍ۙز͔ͭͷ ϕΫτϧΛऔΓग़͢͜ͱΛۙ๣୳ࡧͱݺͿ wର৅ͷಛ௃ྔϕΫτϧ͕େྔʹ͋Δ৔߹ɺϕΫτϧؒͷڑ཭Λ౎౓શ݅ ʹରͯ͠ܭࢉ͍ͯͯ͠͸͕͔͔࣌ؒΔͨΊɺπϦʔ΍ϋογϡΛࣄલʹ ߏங͢Δ͜ͱͰݕࡧͷߴ଎ԽΛਤΔ

  17. ۙࣅۙ๣୳ࡧ wΫΤϦʹ͍ۙάϧʔϓʹਅͷ࠷ۙ๣఺ؚ͕·ΕΔ޻෉͸ܭࢉ࣌ؒΛ૿Ճ ͤ͞ɺ࣍ݩ਺͕ଟ͍৔߹͸ɺશ୳ࡧͱมΘΒͳ͘ͳΔ৔߹΋͋Δ w্هͷ੍໿Λ؇Ίͨ΋ͷ͕ۙࣅ࠷ۙ๣୳ࡧ wਫ਼౓ͱ଎౓ͷόϥϯεΛݟͯ࠾༻͢Δඞཁ͕͋Δ w໦ߏ଄Λ༻͍Δ"//ɺ3BOEPNJ[FELEUSFFɺ'-"//ͳͲ wϋογϡߏ଄Λ༻͍Δ-4)ͳͲ

  18. 4QPUJGZBOOPZ w "QQSPYJNBUF/FBSFTU /FJHICPST0I:FBI w $ ϥΠϒϥϦͱ1ZUIPO౳ͷό ΠϯσΟϯάΛఏڙ w ϥϯμϜαϯϓϦϯάͨ͠ೋ఺Λ

    ݩʹۭؒΛೋ෼ׂΛ܁Γฦͯ͠໦ ߏ଄Λෳ਺ߏங w ୳ࡧ࣌͸෼ׂϕΫτϧͰৼΓ෼͚
  19. 3. ྨࣅը૾ݕࡧͷಋೖ

  20. • ಛ௃ྔม׵ • ݸʑͷը૾σʔλͷྨࣅੑΛఆྔతʹൺֱՄೳͳܗࣜʹม׵͢Δ
 ‎ ఆੑతͳධՁʹ͍ۙ͠ಛ௃ྔม׵ػߏͱͯ͠ͷDeep CNN
 ➕ ֶशͱ༧ଌΛॲཧ͢ΔͨΊͷ֦ுੑͱαʔϏε͔Βͷಠཱੑ •

    ۙ๣୳ࡧ • ม׵ͯ͠ಘΒΕͨಛ௃ྔͲ͏͠Λൺֱ͠ɼྨࣅ͢Δू߹Λಛఆ͢Δ
 ‎ ਫ਼౓ͱ଎౓Λཱ྆͢ΔANN(ۙࣅۙ๣୳ࡧ)
 ➕ σʔλͷू໿ͱαʔϏε͔Βͷಠཱੑ 20 ྨࣅը૾ݕࡧΛࢧ͑Δٕज़
  21. Similar images search system using ANN 21 Feature Similar items

    ANN [2048]float64 query response Deep CNN index Features [n][2048]float64 Deep CNN register find similar features
  22. • ϩά΍DBͳͲͷαʔϏεࢿ࢈ͱ࿈ܞͰ͖Δ
 ‎ ೖग़ྗ͕Cloud Storageܦ༝ • ൺֱత༰қʹϞσϧͷߏஙͱࢼߦ͕ߦ͑Δ
 ‎ TensorFlowΛ࠾༻ •

    ֶश݁ՌΛར༻͢ΔͨΊͷखஈͱͯ͠APIΛఏڙ͢Δ
 ‎ ΦϯϥΠϯ༧ଌαʔϏεʹΑΓϞσϧͷAPIԽ • ্هͷ࢓૊Έ͕εέʔϥϒϧͰ͋Δ͜ͱ
 ‎ ෼ࢄܕτϨʔχϯάΠϯϑϥͱෛՙ෼ࢄαʔϏεͷ࿈ܞ 22 ػցֶशج൫ͱͯ͠ͷGoogle Cloud ML
  23. ಛ௃ྔม׵ w͋Δ࣌఺·Ͱͷ࡞඼ը૾ҰཡΛಛ௃ྔʹม׵͢Δ wม׵ͨ͠ಛ௃ྔҰཡΛۙࣅۙ๣୳ࡧσʔλϕʔεʹ౤ೖ͢Δ Service Object Storage GCP image to data

    data to feature vectorizer by Deep CNN ANN Workers
  24.   5FOTPS'MPXʹΑΔ ಛ௃ྔม׵ͷ࣮૷ ֶशࡁΈωοτϫʔΫͷಡΈࠐΈ தؒ૚Λग़ྗ૚ͱͯ͠ಛ௃நग़ثͱ͢Δ ը૾Λೖྗͱͯ͠ಛ௃ྔʹม׵

  25. • ۙࣅۙ๣୳ࡧΛαʔϏεʹಋೖ͢Δʹ͋ͨΓɺσʔλͷҰݩԽͱෳ਺ͷΞϓϦ έʔγϣϯαʔό͔ΒͷϦΫΤετΛॲཧͰ͖ΔΑ͏ʹHTTPϕʔεͷAPIαʔ ό͕ඞཁ • mruby-annoy on ngx_mrubyͱͯ͠ఏڙ 25 monochromegane/mruby-annoy

    + ngx_mruby
  26. mruby_annoy on ngx_mruby 26 class NNS def call(env) params =

    env['QUERY_STRING'].split('&') .map {|kv| kv.split('=') }.to_h category_id = params['category_id'].to_i product_id = params['product_id'].to_i limit = (params['limit'] || 10).to_i userdata = Userdata.new "annoy_data_key" annoy = userdata.send("category_#{category_id}") return not_found unless annoy nns = annoy.get_nns_by_item(product_id, limit) [200, content_type, [nns.to_json]] end private def not_found return [404, content_type, [{'error' => 'not_found'}.to_json]] end def content_type {'Content-Type' => 'application/json;charset=utf-8'} end end run NNS.new
  27. • ECαΠτʹ͓͍ͯ঎඼ͷ௥Ճߋ৽͸සൟʹൃੜ͢ΔͨΊஞ࣍·ͨ͸ࠩ෼Ͱͷ ΠϯσοΫεߋ৽͕༗༻ • ΠϯσοΫε಺෦IDͱαʔϏεଆͷIDͱͷϚοϐϯά • ͜ΕΒΛຬͨ͠ɼ͔ͭɼΑΓߴ଎ͳۙࣅۙ๣୳ࡧAPIαʔόʔΛgannoyͱͯ͠ ఏڙ • ݱ࣌఺͸ओʹஞ࣍ߋ৽࣌ͷੑೳ໰୊ͷͨΊόοΫΤϯυΛyahoo/NGTʹࠩ

    ͠ସ͑ͯӡ༻த 27 monochromegane/gannoy(-yahoo/NGT)
  28. 28 monochromegane/gannoy(-yahoo/NGT) # Add item $ curl \ ’http://localhost:1323/databases/hoge/features/100’ \

    -H "Content-type: application/json” \ -X PUT \ -d '{"features": [1.0, 0.5, 0.2,..]}’ # Search $ curl \ ’http://localhost:1323/search?database=hoge&id=100’
  29. ྨࣅը૾ݕࡧ 29 Nyah products#show product_id nearest products products#update Gannoy Object

    Storage GCP data to feature vectorizer by Inception-V3 ˞ۙࣅۙ๣୳ࡧ࣌ʹେ෦෼ͷ ΠϯσοΫε΁ͷΞΫηε͕ ൃੜ͢ΔͨΊ࣮༻తͳ଎౓Λ ಘΔͨΊʹ͸σʔλϕʔεϑΝ Πϧ͕શͯϖʔδΩϟογϡ ʹࡌΔαΠζͷϝϞϦ͕ඞཁ • ݕࡧ࣌͸࡞඼IDΛݩʹొ࿥͞Εͨಛ௃ྔͷۙ๣ू߹Λฦ٫͢Δ • ߋ৽࣌͸ߋ৽ޙͷಛ௃ྔΛҰ࣌తʹอଘɼఆظతʹࠩ෼൓өͯ͠੾Γସ͑
  30. ྨࣅը૾ʹΑΔؔ࿈࡞඼ݕࡧ 30

  31. ྨࣅը૾ʹΑΔؔ࿈࡞඼ݕࡧ 31

  32. ྨࣅը૾ʹΑΔؔ࿈࡞඼ݕࡧ 32

  33. 4. ྨࣅը૾ݕࡧͷ͜Ε͔Β

  34. • ಛ௃ྔม׵ • ݸʑͷը૾σʔλͷྨࣅੑΛఆྔతʹൺֱՄೳͳܗࣜʹม׵͢Δ
 ‎ ఆੑతͳධՁʹ͍ۙ͠ಛ௃ྔม׵ػߏͱͯ͠ͷDeep CNN
 ↪︎ αʔϏεʹಛԽͨ͠”ྨࣅੑ”Λ֫ಘ͢Δ •

    ۙ๣୳ࡧ • ม׵ͯ͠ಘΒΕͨಛ௃ྔͲ͏͠Λൺֱ͠ɼྨࣅ͢Δू߹Λಛఆ͢Δ
 ‎ ਫ਼౓ͱ଎౓Λཱ྆͢ΔANN(ۙࣅۙ๣୳ࡧ)
 ↪︎ ΑΓߴ଎Ͱߴਫ਼౓ͳANN 34 ྨࣅը૾ݕࡧΛࢧ͑Δٕज़
  35. SSD 35 • Single Shot MultiBox Detector • ΦϒδΣΫτҐஔͱΫϥε෼ྨΛҰ ׅͰߦ͏Ϟσϧ

    • αʔϏεಠࣗͷΧςΰϦΛֶशσʔ λʹ༻͍ͨ
  36. Sanny େن໛ECαΠτͷͨΊͷ଎౓ͱਫ਼౓Λཱ྆ͨ͠ ෼ࢄՄೳͳۙࣅۙ๣୳ࡧΤϯδϯ

  37. • ΫΤϦͱߴ࣍ݩϕΫτϧू߹Λ೚ҙͷ࣍ݩ਺Ͱ౳෼ͨ͠෦෼ϕΫτϧ୯ҐͰฒ ߦʹۙ๣୳ࡧͨ݁͠Ռͷ࿨ू߹Ͱ͋Δۙ๣ީิ͔Βɼ࠶౓ۙ๣୳ࡧΛߦ͏ɽ 37 ఏҊख๏ (଎౓໘) R@ R@ R@ 9@

    9@ 9@ /SFDPSET // R@ Y@O Y㱨9@ // R@ Y@O Y㱨9@ // R@ Y@O Y㱨9@ \   ^ \   ^ \   ^ BSHNJOE R Y Y㱨\    ^ 㱮 㱮 ෼ղલ ෼ղޙ ᶃ௿࣍ݩۭؒͷฒߦͨۙ͠๣୳ࡧ ᶄݻఆ਺ͷۙ๣ީิͷू໿ ᶅۙ๣ީิͷઢܗ୳ࡧ ᶃ ᶄ ᶅ ଎౓վળ ᶃ ᶄ ᶅ 2VFSZ
  38. • ঎඼ಛੑΛΑ͘දݱ͓ͯ͠Γߴਫ਼౓ʹྨࣅ౓͕ൺֱՄೳͳߴ࣍ݩ͔ͭີͳϕΫ τϧͷू߹Λର৅ͱͨۙ͠ࣅۙ๣୳ࡧ • ֶशࡁΈCNNΛಛ௃நग़ثͱͯ͠ར༻ͯ͠ը૾͔ΒಘΒΕΔಛ௃ྔू߹ • ςΩετΛ෼ࢄදݱ΁ม׵͢ΔWord2vec͔ΒಘΒΕΔಛ௃ྔू߹ • ͜ΕΒ͕ɼݕࡧ࣭໰σʔλ(ΫΤϦ)ʹର͢Δߴ࣍ݩϕΫτϧू߹ͷۙ๣୳ࡧ݁ Ռͷ্Ґू߹͕ɼΫΤϦͱߴ࣍ݩϕΫτϧू߹Λ೚ҙͷ࣍ݩ਺Ͱ౳෼ͨ͠෦෼

    ϕΫτϧ୯ҐͰۙ๣୳ࡧͨ݁͠Ռͷ্Ґू߹ͱྨࣅ͠΍͍͢͜ͱʹண໨ 38 ఏҊख๏ (ਫ਼౓໘) ෦෼͕ྨࣅ͢Ε͹શମ͕ྨࣅ͢ΔՄೳੑ͕ߴ͍σʔλಛੑ
  39. 39 Sanny: ఏҊख๏ͷ࣮૷ 4BOOZ 4BOOZ 4BOOZ // // 㱮 //

    // ᶃ ᶄ ᶅ 2VFSZ "MHPSJTN "MHPSJTN "MHPSJTN • ΫΤϦฒͼʹ୳ࡧର৅σʔλͷ೚ҙ࣍ݩ΁ͷ౳෼ͱ݁Ռͷू໿Λ୲౰͢Δ • ෦෼ϕΫτϧͷۙ๣୳ࡧΞϧΰϦζϜ͸໰Θͳ͍ • ෦෼ϕΫτϧ͝ͱͷ୳ࡧॲཧ͸ಠཱͷͨΊ෼ࢄߏ੒͕Մೳ
  40. None