Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINE's 3D Recognition Technology and Future Prospects

LINE's 3D Recognition Technology and Future Prospects

LINEの3D認識技術と今後の展望
井尻善久(LINE株式会社)
MLAI-TALK #1 での発表資料です
https://line.connpass.com/event/231314/

LINE Developers

December 01, 2021
Tweet

More Decks by LINE Developers

Other Decks in Technology

Transcript

  1. > ઐ໳ɿίϯϐϡʔλϏδϣϯɾϩϘςΟΫε > 0VUEPPSొࢁɾεΩʔɾୌ८ΓɾࣸਅࡱӨɾόΠΫτϥΠΞϧɾɾɾ > *OEPPSϐΞϊԋ૗ɾྺ࢙ɾᗉ੡ɾίʔώʔᖿઝɾ΢ΟεΩʔɺΫϥ ϑτϏʔϧɾɾɾ > ೥ΦϜϩϯೖࣾ >

    إͷݕग़ೝࣝͷσδΧϝɾܞଳి࿩ɺ؂ࢹΧϝϥԠ༻ > ෺ମݕग़ɾŤŞƄŸƃũŖŢŔƃɾ0$3ͷ'"޲͚঎඼Խ > ͠ͳ΍͔ͳ੍ޚΛ࣮ݱ͢Δࣗ཯ιϑτϩϘοτݚڀਪਐ > Ϧαʔνϕϯνϟʔ্ཱͪ͛ 0.30/4*/*$9 > ೥-*/&ೖࣾ > $PNQVUFS7JTJPO-BCͷ্ཱͪ͛ -*/&גࣜձࣾ "*Χϯύχʔ "*։ൃࣨ ࣨ௕ɺ$PNQVUFS7JTJPO-BC Ϛωʔδϟʔ :PTIJIJTB*KJSJ 1I%
  2. "*ٕज़ͷա೤ͿΓ 0 1000 2000 3000 4000 5000 6000 7000 8000

    1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 $713࿦จ౤ߘ਺ͷਪҠ 出展:IEEE digital library各年度proceedings これらを元に独⾃に集計し作成
  3. ίϯϐϡʔλϏδϣϯٕज़ͷա೤ͿΓ QVCMJDBUJPO I  /BUVSF   5IF/FX&OHMBOE+PVSOBMPG.FEJDJOF  

    4DJFODF   *&&&$7'$POGFSFODFPO$PNQVUFS7JTJPOBOE 1BUUFSO3FDPHOJUJPO   5IF-BODFU   "EWBODFE.BUFSJBMT   $FMM   /BUVSF$PNNVOJDBUJPOT   $IFNJDBM3FWJFXT   *OUFSOBUJPOBM$POGFSFODFPO-FBSOJOH 3FQSFTFOUBUJPOT   +"."   /FVSBM*OGPSNBUJPO1SPDFTTJOH4ZTUFNT   1SPDFFEJOHTPGUIF/BUJPOBM"DBEFNZPG4DJFODFT   +PVSOBMPGUIF"NFSJDBO$IFNJDBM4PDJFUZ   "OHFXBOEUF$IFNJF   $IFNJDBM4PDJFUZ3FWJFXT   /VDMFJD"DJET3FTFBSDI   3FOFXBCMFBOE4VTUBJOBCMF&OFSHZ3FWJFXT   +PVSOBMPG$MJOJDBM0ODPMPHZ   1IZTJDBM3FWJFX-FUUFST  QVCMJDBUJPO I  "EWBODFE&OFSHZ.BUFSJBMT   /BUVSF.FEJDJOF   *OUFSOBUJPOBM$POGFSFODFPO.BDIJOF-FBSOJOH   &OFSHZ&OWJSPONFOUBM4DJFODF   "$4/BOP   4DJFOUJGJD3FQPSUT   &VSPQFBO$POGFSFODFPO$PNQVUFS7JTJPO   5IF-BODFU0ODPMPHZ   "EWBODFE'VODUJPOBM.BUFSJBMT   1-P4 0/&   *&&&$7'*OUFSOBUJPOBM$POGFSFODFPO$PNQVUFS 7JTJPO   /BUVSF(FOFUJDT   +PVSOBMPG$MFBOFS1SPEVDUJPO   /BUVSF.BUFSJBMT   4DJFODFPG5IF5PUBM&OWJSPONFOU   $JSDVMBUJPO   #.+   +PVSOBMPGUIF"NFSJDBO$PMMFHFPG$BSEJPMPHZ   "QQMJFE$BUBMZTJT#&OWJSPONFOUBM   4DJFODF"EWBODFT  出展:Google Scholar
  4. ՝୊Λ৐Γӽ͑ΔͨΊओʹΞϓϩʔνଘࡏ Point-based: Qi et al. [CVPR 2017] Alternative representation: Sinha

    et al. [ECCV 2016] Voxel based: Wu et al. [CVPR 2015] Image-based: Kanezaki et al. [CVPR 2018]  αΠζ  ϝογϡ͕ඞཁ  લॲཧ͕ඞཁ  ࠲ඪ஋ ؔ਺ͱͯ͠දݱ͢Ε͹Α͍ͷͰ͸ʁ ओͳղੳख๏
  5. ը૾ ✔ ࠲ඪܥ ✔ ॱং ✔ εέʔϧ ఺܈ ʁ ࠲ඪܥ

    ✔ ॱং ✔ εέʔϧ ఺܈ͷ೉͠͞
  6. model chair bed ʜ table   ʜ  

     ʜ  ճసෆมͷ࣮ݱ
  7. 𝐴!! 𝐴!" ⋯ 𝐴!# 𝐴"! 𝐴"" ⋯ 𝐴"# ⋮ 𝐴$!

    ⋮ 𝐴$" ⋱ ⋮ ⋯ 𝐴$# 𝐱 = 𝑏! 𝑏" ⋮ 𝑏$ min 𝐱 𝐀𝐱 − 𝐛 " " ఺܈" C͕༩͑ΒΕͨ৔߹ɼม׵YΛٻΊΔ 順序が必要! ઢܗճؼ (Linear Regression)
  8. ఺܈" C͕༩͑ΒΕͨ৔߹ɼஔ׵ߦྻ1ͱม׵YΛٻΊΔ 同じ点数が必要! min 𝐱, 𝐏 𝐀𝐱 − 𝐏𝐛 "

    " 𝐴!! 𝐴!" ⋯ 𝐴!# 𝐴"! 𝐴"" ⋯ 𝐴"# ⋮ 𝐴$! ⋮ 𝐴$" ⋱ ⋮ ⋯ 𝐴$# 𝐱 = ⋮ 𝑏$ 𝑏" 𝑏! 𝐏() ∈ {0, 1} 5 ( 𝐏() = 1 5 ) 𝐏() = 1 Shuffled Linear Regression [Ashwin+, 2017]
  9. ఺܈" C͕༩͑ΒΕͨ৔߹ɼஔ׵ߦྻ1ͱม׵YΛٻΊΔ 外れ値と順序の特定が可能! min 𝐱, 𝐏 𝐀𝐱 − 𝐏𝐛 "

    " 𝐴!! 𝐴!" ⋯ 𝐴!# 𝐴"! 𝐴"" ⋯ 𝐴"# ⋮ 𝐴$! ⋮ 𝐴$" ⋱ ⋮ ⋯ 𝐴$# 𝐱 = ⋮ 𝑏$ 𝑏" 𝑏! 𝐏() ∈ {0, 1} 5 ( 𝐏() ≤ 1 5 ) 𝐏() ≤ 1 5 (,) 𝐏() = 𝑘 Generalized Shuffled Linear Regression [Li and Fujiwara+, ICCV2021]
  10. • ֊਺མͪͷஔ׵ߦྻ΋ѻ͑Δ b4IVGGMFE-JOFBS3FHSFTTJPO` • ఺܈͚ͩͰͳ͘ը૾ͷಛ௃఺ͳͲʹ΋ରԠՄೳ Source Target GSLR (ours) Feature

    matching w/ RANSAC SLR Source FMAP ICP BCICP ZoomOut-100 GSLR (ours) 94.4% 62.0% 83.4% 52.6% 37.1% (FOFSBMJ[FE4IVGGMFE-JOFBS3FHSFTTJPO <-JBOE'VKJXBSB *$$7>
  11. 21 -*/&$7-ͷ஫ྗٕज़  $79ٕज़ ࣗવݴޠॲཧ ೖྗ Ի੠ σδλϧ ςΩετ ը૾ಈը

    3(#%5 ݴޠ ςΩετ ը૾ ਤද Ի੠ೝࣝ $713 ॲཧ ੜ੒ Ի੠߹੒ $( ςΩετग़ ྗ 5F9ͳͲ ϚϧνϝσΟΞೖྗʹରԠ͢ΔϚϧνϞʔμϧॲཧ"*ٕज़ $7Y˓˓ٕज़͕ॏཁʹʂ ʢ$7Λத৺ͱͯ͠Έͨͱ͖ͷϚϧνϞʔμϧ"*ٕज़ͷҙຯͰԬຊࢯ͕$79ٕज़ͱ໋໊ʣ
  12. %PDVNFOU6OEFSTUBOEJOH "*0$3 Semantic Information S-Overtime 50% (count) 1 (unitpric e)

    20,000 (price) 20,000 PBI 1,818 Subtotal 18,181 Total 20,000 Cash 100,000 Change 80,000 Tax Included 10% Image Spatial Dependency Parsing for Semi-Structured Document Information Extraction [Hwang+, ACL2020]
  13. #FZPOEDVSSFOU"*0$3ʜ $IBSBDUFSUZQF 5FSNJOPMPHZ (SBNNBS 'PSNMBZPVU 5PQJDTTUZMF %PDVNFOUUZQF %PNBJO LOPXMFEHF 1VSQPTF

    UBTL $VTUPNFS TQFDJGJD LOPXMFEHF $PNNPO LOPXMFEHF 7JTVBMQBUUFSOT $POUFYU 510 MFWFMPGGPOMZXJUI WJTVBMQBUUFSOT $PNCJOBUJPOXJUI/-1 CFDPNFTDSVDJBM $IBSBDUFS -BOHVBHF 8PSE
  14. 26 -*/&$7-ͷ஫ྗٕज़  $79ٕज़ ࣗવݴޠॲཧ ೖྗ Ի੠ σδλϧ ςΩετ ը૾ಈը

    3(#%5 ݴޠ ςΩετ ը૾ ਤද Ի੠ೝࣝ $713 ॲཧ ੜ੒ Ի੠߹੒ $( ςΩετग़ ྗ 5F9ͳͲ ϚϧνϝσΟΞೖྗʹରԠ͢ΔϚϧνϞʔμϧॲཧ"*ٕज़ $7Y˓˓ٕज़͕ॏཁʹʂ ʢ$7Λத৺ͱͯ͠Έͨͱ͖ͷϚϧνϞʔμϧ"*ٕज़ͷҙຯͰԬຊࢯ͕$79ٕज़ͱ໋໊ʣ
  15. STRICTLY CONFIDENTIAL $-07"$IBUCPU -*/&͔ΜͨΜϔϧϓ΍$-07"Ͱഓͬͨ ࣗવݴޠٕज़Λɺ'"2΍઀٬༻#PUʹ ల։Ͱ͖ΔαʔϏε LINE CLOVA Chatzbot $-07"0$3

    ࠃࡍձٞͰੈք࠷ߴਫ४ͱೝΊΒΕͨ OCRٕज़Λਃࠐॻ΍ྖऩॻͳͲͷಡΈऔΓɺ ࣗಈೖྗʹ׆༻Ͱ͖ΔαʔϏε LINE CLOVA OCR $-07"4QFFDI $-07"ͷԻ੠ೝٕࣝज़Λ׆༻͠ɺ ి࿩΍ಈըϝσΟΞͷԻ੠ॻ͖ى͜͠ɺ ి࿩ԠରͷࣗಈԽαʔϏεͳͲΛఏڙ LINE CLOVA Speech $-07"7PJDF $-07"ͷԻ੠߹੒ٕज़Λ׆༻͠ɺ اۀϒϥϯυ΍༻్ʹ͋ͬͨԻ੠ϞσϧΛ࡞੒ ͢ΔαʔϏεΛఏڙ༧ఆ LINE CLOVA Voice $-07"5FYU"OBMZUJDT ςΩετղੳɺײ৘෼ੳٕज़ɻ Ի੠ೝࣝͰىͨ͜͠ςΩετ͔Βͷݕࡧ΍ ײ৘෼ੳͳͲʹ׆༻ɻ LINE CLOVA Text Analytics $-07"7JTJPO ෺ମೝࣝɺը૾ೝٕࣝज़ɻ LINEγϣοϐϯάͷʮSHOPPING LENSʯͰ׆༻ɻ LINE CLOVA Vision $-07"'BDF ߴਫ਼౓ͷإೝٕࣝज़ɻ eKYCʢΦϯϥΠϯຊਓ֬ೝʣ ΍إೝূʹΑΔड෇ͳͲʹ׆༻ɻ LINE CLOVA Face -*/&$-07"ͷ ϓϩμΫτ 4BB4ఏڙ 4BB4ఏڙ -*/&ͷ࣋ͭଟ༷ͳ"*ཁૉٕज़Λجʹ෯޿͍##޲͚ϓϩμΫτΛల։͍ͯ͠·͢ʢҰ෦4BB4ͱͯ͠ఏڙʣɻ
  16. "*Χϯύχʔ ͷ 3%7JTJPO $POTFSWBUJWF %JTSVQUJWF 5JNF    

    *OUFSBDUJWFWJSUVBM FYQFSJFODF "VUPOPNPVT"* XPSLGMPX %JHJUBM.F .F"7"5"3 %JHJUBM*EFOUJUZ #FUUFS$BSF 5SVTUXPSUIZ"* "*'BJSOFTT &YQMBJOBCMF"* %BSL%BUB 0NOJQPUFOU"* (JHBOUJD-BOHVBHFNPEFM 6OMBCFMFE%BUB %BUB.BSLFUQMBDF (FOFSBUJWF*OUFMMJHFODF /FX&EVDBUJPO %FQFOEBCMF455 1SJWBDZQSFTFSWJOH 4FBN%JTDSJNJOBUPS
  17.  ʣ*$"441 ʣ*/5&341&&$) ʣ8"41"" ʣ #JH%BUB  ʢʣ*$"441  &64*1$0

     */5&341&&$)   %$"4&  "14*1"   $713  51%1  '-*$.-  -%3$  *$"441  *$3"  *6*  *$%&  *$$7  各分野最⾼峰の会議で認められるAI 基礎研究成果 ͜Ε·Ͱͷ੒Ռ
  18. Our challenge Innovation by mixing LINE AI assets, especially NLP,

    voice/speech, and CV .JYFE-*/&"*.J-"* .VMUJNPEBMJOQVUPVUQVU
  19. 44 STRICTLY CONFIDENTIAL $-07"0$3 Point 1 ੈք࠷ߴਫ४ͷ"*0$3 Point 2 ͋ΒΏΔॻྨ΍ը૾Λૉૣ͘ςΩετԽ

    Point 3 खॻ͖ͷจࣈ΋ೝࣝՄೳ ԣॻ͖΍ॎॻ͖ɺؙ͘࿷ۂͨ͠จࣈͳͲѱ৚݅ԼͰͷಡΈऔΓɺଟݴޠͷ ೝࣝɺઐ໳༻ޠͷೝࣝͳͲͰߴ͍ਫ਼౓ͱධՁɻจॻղੳͱೝࣝʹؔ͢Δࠃ ࡍձٞ *$%"3 ͷ෼໺ʹͯੈք/PΛ֫ಘ͍ͯ͠·͢ɻ ϑΥʔϚοτ͕ܾ·͍ͬͯΔॻྨ͸΋ͪΖΜɺ͋ΒΏΔελΠϧͷॻྨΛ ਖ਼͘͠ςΩετԽ͠·͢ɻ$-07"0$3ʢྖऩॻɾ੥ٻॻɾϨγʔτಛ ԽܕʣͰ͸ɺϑΥʔϚοτͷࣄલొ࿥͕ෆཁɻ खॻ͖จࣈ΍ɺࣼΊʹͳͬͨจࣈ΋ߴਫ਼౓ͷೝূ͕Մೳ
  20. 45 STRICTLY CONFIDENTIAL 4"1$PVODVS +BQBO ࢴͷ੥ٻॻͷσδλϧԽͷύʔτφʔͱͯ͠-*/&$-07"Λબఆ גࣜձࣾതใಊ%:ϝσΟΞύʔτφʔζ γϦΞϧφϯόʔΛಡΈऔΔ͜ͱͰɺίϯϏχԁ͘͡ΛΦϯϥΠϯԽ -*/&τʔΫϧʔϜ τʔΫϧʔϜ͔Βը૾ΛࡱΔ͚ͩͰจࣈೝࣝػೳ͕ར༻Մೳ

    Ϋϥ΢υαʔϏεͱͷύʔτφʔγοϓ ৽ͨͳιϦϡʔγϣϯͱͯ͠ͷ׆༻ -*/&αʔϏε΁ͷߩݙ ʘ GPS*/70*$&ʗ -*/&Ϩγʔτ -*/&1-"$& ϨγʔτΛ"*ͰಡΈऔΔ͜ͱͰɺֹۚͱ඼໨͕ࣗಈͰ෼ྨɻ ࢧग़؅ཧ΍ར༻͓ͨ͠ళͷޱίϛαΠτ΁ͷ౤ߘͳͲ͕؆୯ʹɻ $-07"0$3ಋೖࣄྫ
  21. 46 STRICTLY CONFIDENTIAL $POGJEFOUJBM -*/&"J$BMM Point 1 ϢʔβʔΛ଴ͨͤͳ͍ར༻ମݧ Point 2

    ਓؒຯ͋;ΕΔࣗવͳର࿩ Point 3 طଘγεςϜ΍-*/&ͱͷ࿈ܞ ࣌ؒ೔ɺडిମ੍Λ༻ҙͰ͖Δ͜ͱ͸΋ͪΖΜɺ൪߸ೖྗͰରԠ ಺༰ΛৼΓ෼͚Δ*73ʢ*OUFSBDUJWF7PJDF3FTQPOTFʣͱ͸ҟͳΓɺॊೈ ʹରԠ͠·͢ɻ ཈༲ͷ͋Δਓؒʹ͍ۙࣗવͳԻ੠ͰɺϢʔβʔʹετϨεΛֻ͚·ͤΜɻ ·ͨɺ"*ʹΑΔԻ੠ೝࣝͷֶशʹΑΓɺԻ੠ೝࣝͱର࿩ͷਫ਼౓͕޲্͠ɺ ରԠ඼࣭͕޲্͠·͢ɻ ͜Ε·ͰՍిडిޙʹߦ͍ͬͯͨΞφϩάͳσʔλ੔උۀ຿΋ɺγες Ϝ࿈ܞʹΑΓܰݮ͠·͢ɻ·ͨɺ-*/&΍4.4ͱ࿈ܞ͢Δ͜ͱͰɺ௨࿩ޙ ʹϢʔβʔʹࣗಈͰϝοηʔδΛૹ৴͢Δ͜ͱ΋ՄೳͰ͢ɻ
  22. 47 STRICTLY CONFIDENTIAL ϠϚτӡ༌גࣜձࣾ ސ٬͔Βͷి࿩ʹΑΔूՙड෇ ਆಸ઒ݝ ৽ܕίϩφి࿩૬ஊ૭ޱ גࣜձࣾΤϏιϧ ҿ৯ళ޲͚༧໿؅ཧγεςϜ גࣜձࣾΧʔϑϩϯςΟΞ

    Χʔϝϯςφϯε༧໿αʔϏε େखاۀͷۀ຿ʹಋೖ ެڞߦ੓ͷෛ୲ܰݮ ϓϥοτϑΥʔϜͱͷػೳ࿈ܞ -*/&"J$BMM ಋೖ࣮੷