%PD'PSNFS&OEUP&OE5SBOTGPSNFS GPS%PDVNFOU6OEFSTUBOEJOH 2021/12/12ɹICCV'21 ಡΈձ
 Yamato OKAMOTO

ຊ೔ɺ঺հ͢Δ࿦จ w %PDVNFOU6OEFSTUBOEJOHΛλʔήοτͱͨ͠5SBOTGPSNFSϞσϧͷఏҊ ͍ͭͰʹࢲͷ঺հ w Ԭຊେ࿨ʢ͓͔΋ͱ΍·ͱʣ w ژେඒೱݚˠΦϜϩϯˠ-*/&גࣜձࣾ ,ZPUP0GGJDF w Ӊ஦Ұָ͍͠$7-BCΛژ౎ʹ࡞Δ΂͘νʔϜϏϧυத w ࠷ۙͷڵຯ͸υΩϡϝϯτղੳٕज़ → LINE DevDay2021Ͱ΋ొஃͯ͠ɺυΩϡϝϯτղੳʹ͍ͭͯ࿩͠·ͨ͠ʙ̇ ɹɹhttps://www.youtube.com/watch?v=D45wIMN6R_I

-*/&BU*$$7 https://linecorp.com/ja/pr/news/ja/2021/3843 https://iccv2021.thecvf.com/sponsors ฐࣾͷΤʔε͕ຊ௨͠·ͨ͠ εϙϯαʔͱͯ͠ϒʔεΛग़ͤͯ͞௖͖·ͨ͠ ɾ(FOFSBMJ[FE4IV ffl FE-JOFBS3FHSFTTJPOʢ0SBMͰ͢ʣ ɹ'FJSBO-J ,FOU'VKJXBSB 'VNJP0LVSB BOE:BTVZVLJ.BUTVTIJUB ɾ"$MPTFS-PPLBU3PUBUJPOJOWBSJBOU%FFQ1PJOU$MPVE"OBMZTJT ɹ'FJSBO-J ,FOU'VKJXBSB 'VNJP0LVSB BOE:BTVZVLJ.BUTVTIJUB

Document Understandingͬͯɺ ͍͍ͬͨԿΛ͢Δͷʁʁ

4FRVFODF-BCFMJOH )FBEFS PSBOHF 2VFTUJPO CMVF "OTXFS HSFFO ʹ෼ྨ ̍ͭ̍ͭͷςΩετΛʮ2VFTUJPOʯʮ"OTXFSʯͳͲͷΠϯςϯτΛϕʔεʹɺάϧʔϓ෼͚͢ΔλεΫɻ '6/4% GPSNVOEFSTUBOEJOHJOOPJTZTDBOOFEEPDVNFOUT %BUBTFU͕༗໊ɻ https://arxiv.org/abs/1905.13538 จࣈྻɺ࠲ඪΛڭࢣϥϕϧͱͯ͠෇༩ ςΩετϦϯΫͷڭࢣϥϕϧ΋͋ΔͷͰɺ ,FZ7BMVFΛநग़͢ΔΑ͏ͳλεΫ΋Մೳɻ

%PDVNFOU$MBTTJGJDBUJPO υΩϡϝϯτը૾ͷΫϥεΛ༧૝͢ΔλεΫɻ 5IF37-$%*1%BUBTFU͕༗໊ɺΫϥεΛؚΉສͷάϨʔεέʔϧը૾Ͱߏ੒͞ΕΔɻ https://www.cs.cmu.edu/~aharley/rvl-cdip/

&OUJUZ&YUSBDUJPO<> • $03%"$POTPMJEBUFE3FDFJQU%BUBTFUGPS1PTU0$31BSTJOH • ςΩετɺ࠲ඪɺΫϥεʢNFOV UPUBM ͳͲछྨʣͷڭࢣϥϕϧ͕෇༩͞Ε͍ͯΔ υΩϡϝϯτ͔Βಛఆ৘ใʢ&YNFOV QSJDFʣΛநग़͢ΔλεΫɻ Ϩγʔτը૾ͷ$03%%BUBTFU΍ɺܖ໿ॻը૾ͷ,MFJTUFS/%"͕͋Δɻ https://github.com/clovaai/cordɹɹhttps://arxiv.org/pdf/2003.02356.pdf

&OUJUZ&YUSBDUJPO<> • ,MFJTUFS/%"ɿ໿෦ɺ߹ܭ໿ϖʔδ͔ΒͳΔ/%"ܖ໿ॻͷσʔληοτɻ • ܖ໿ॻ͝ͱͷ୯ҐͰɺʮܖ໿ظؒʢFGGFDUJWF@EBUFʣʯͳͲछྨͷFOUJUZ͕ڭࢣϥϕϧͱͯ͠෇༩͞Ε͍ͯΔɻ υΩϡϝϯτ͔Βಛఆ৘ใʢNFOV QSJDFʣΛநग़͢ΔλεΫɻ Ϩγʔτը૾ͷ$03%%BUBTFU΍ɺܖ໿ॻը૾ͷ,MFJTUFS/%"͕͋Δɻ https://github.com/clovaai/cordɹɹhttps://arxiv.org/pdf/2003.02356.pdf

͜͜·Ͱͷ·ͱΊ w %PDVNFOU6OEFSTUBOEJOHͱ͸ԿΛ͢Δͷ͔ʁʁ w ༷ʑͰ͋Δɻྫ͑͹ɺॻྨͷछྨࣝผ΍ɺࣄલఆٛͨ͠ಛఆ৘ใΛݟ ͚ͭग़͢Α͏ͳλεΫ͕͋Δɻ w ୅දతͳσʔληοτʹɺϨγʔτը૾΍ܖ໿ॻը૾ΛूΊͨ΋ͷ͕ ͋Γɺจࣈྻɾ࠲ඪɾJOUFOU΍FOUJUZͷڭࢣϥϕϧ͕෇༩͞Ε͍ͯΔɻ

λεΫ΍σʔληοτ͕ଟछଟ༷ͳͷ͸Θ͔ͬͨʂ ͡Ό͊ɺͲΜͳݚڀ͕ͳ͞Ε͍ͯΔͷʁ

܏޲ɿೖྗΛ.VMUJNPEBMʹ͢Δ • $//Ͱநग़ͨ͠7JTJPOಛ௃ྔʹɺ-BOHVBHFͷಛ௃ྔΛԣ͔Βࠩ͠ࠐΜͰɺߏ଄ղੳʹνϟϨϯδͨ͠ݚڀɻ • -BOHVBHFͷಛ௃ྔநग़ʹ͸ɺςΩετͷྖҬݕग़ͱೝࣝΛ͢Δ0$3෦͕ඞཁͱͳΔɻ • ͜ͷ࿦จͰ͸ɺطଘͷ0$3Τϯδϯʢ5FTTFSBDUʣΛར༻͍ͯ͠Δɻ࢒೦ͳ͕Β͜ͷ෦෼͸&&Ͱֶश͞Εͳ͍ɻ υΩϡϝϯτʹ͸ը૾΋ݴޠ΋ؚ·Ε͍ͯͯɺͦΕͧΕ͕ώϯτʹͳΔ͸ͣʂ ͦ͜Ͱɺ྆ํͷಛ௃ྔΛநग़ͯ͠ར༻͢Δ.VMUJNPEBMͳݚڀ͕ਐలதɻ Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network (CVPR'17) https://arxiv.org/pdf/1706.02337.pdf

܏޲ɿڭࢣͳֶ͠शͰ5SBOTGPSNFSߏ଄ΛQSFUSBJO ෳ਺ͷڭࢣͳ͠λεΫͷఏҊʹΑΓ5SBOTGPSNFSߏ଄ͷֶशʹ੒ޭɻ 1.UFYUUPLFOͷҰ෦ΛӅͯ͠ɺͦΕΛ෮ݩͤ͞Δɻ 2.Ұఆׂ߹ͷςΩετΛࠇ͘ృΓͭͿ͢ɻͦͯ͠ɺృΓ௵͞Ε͍ͯΔ͔ $PWFSFE ɺృΓ௵͞Εͯͳ͍͔ /PU$PWFSFE Λ༧ଌͤ͞Δɻ 3.ͱ͖͓Γɺը૾ͱςΩετͷϖΞΛ͢Γସ͑Δɻ͜ͷ৚݅ԼͰɺ 
 υΩϡϝϯτը૾ͱςΩετͷϖΞ͕ద੾͔ʢ.BUDIʣɺ 
 ͢Γସ͑ΒΕ͍ͯΔ͔ʢ/PU.BUDIʣΛ༧ଌͤ͞Δɻ ໠ҖΛฃ͍ͬͯΔ5SBOTGPSNFSʂʂ େྔσʔλͰֶशͰ͖ΔΑ͏ʹڭࢣͳ͠λεΫΛઃఆ͢Δɻ LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding (ACL'21) https://arxiv.org/pdf/2012.14740.pdf

܏޲ɿ5SBOTGPSNFSͷΞʔΩςΫνϟʔΛվળ B 7JTJPOͱ8PSEͷ5PLFOΛྻʹฒ΂ͯॱ൪ʹೖྗ͢Δɻ 
 ҟͳΔ.PEBMಉ࢜ͷσʔλͰద੾ʹ"UUFOUJPO͕͔͔Δ͔ٙ໰ɻ C 7JTJPOͱ8PSEΛҟͳΔ5SBOTGPSNFSͰॲཧ͔ͯ͠Β'VTJPO͢Δɻ 
 "UUFOUJPO͸͔͔Γͦ͏͕ͩɺ.VMUJNPEBMͷԸܙ͕ݮΓͦ͏ɻ D 5SBOTGPSNFSʹೖྗ͢Δલͷ5PLFOஈ֊Ͱ'VTJPOͤ͞Δɻ 
 DPODBUͳͲͰγϯϓϧͳख๏Ͱ'VTJPOͤͯ͞Α͍ͷ͔ո͍͠ɻ ओு͍ͯ͠Δར఺ͱܽ఺΋ਖ਼͍͠ͷ͔ٙΘ͍͠ɺɺɺɺ ΋͸΍ͩΜͩΜͲΕ͕࠷దͳͷ͔෼͔Βͳ͘ͳ͖ͬͯͨɻɻɻɻ 7JTJPO΍-BOHVBHFͳͲੑ࣭ͷҟͳΔಛ௃ྔΛ͍͔ʹ'VTJPOͤ͞Δ͔ʁ ΞʔΩςΫνϟઃܭͰ΋ݚڀ͕ന೤தʂʂ https://arxiv.org/abs/2106.11539

͜͜·Ͱͷ·ͱΊ %PDVNFOU6OEFSTUBOEJOHʹ5SBOTGPSNFS౸དྷɻ ݚڀͷ܏޲͸େ͖̏ͭ͘ʂ ೖྗΛΠΠײ͡ʹ͢Δʂʢ͍͔ʹෳ਺ͷ.PEBMΛऔΓೖΕΔ͔ʣ ֶशΛΠΠײ͡ʹ͢Δʂʢ͍͔ʹڭࢣͳ͠λεΫΛઃܭ͢Δ͔ʣ ΞʔΩςΫνϟΛΠΠײ͡ʹ͢Δʂʢ͍͔ʹ'VTJPOͤ͞Δ͔ʣ

ຊ೔঺հ͢Δ࿦จ ʮ%PD'PSNFSʯʮ%PDVNFOU6OEFSTUBOEJOHʯʴʮ5SBOTGPSNFSʯ CZ"84"* ߩݙϙΠϯτɿ͜͜·Ͱ঺հ͖ͯͨ̏͠఺ΛΠΠײ͡ʹͨ͠ https://openaccess.thecvf.com/content/ICCV2021/papers/Appalaraju_DocFormer_End-to-End_Transformer_for_Document_Understanding_ICCV_2021_paper.pdf

7JTJPO -BOHVBHF 4QBUJBMͷछྨͷಛ௃ྔΛೖྗ 7JTJPO'FBUVSF • 3FT/FUͰ֫ಘͨ͠'FBUVSF.BQΛɺY@$POWͰ೚ҙͷDI਺ E ʹม׵ޙɺMJOFBSUSBOTGPSNFSMBZFSͰ೚ҙͷαΠζ E/ ʹม׵ -BOHVBHF'FBUVSF • 0$3ΤϯδϯͰςΩετΛݕग़ʴೝࣝʴಛ௃ྔԽͨ͠5PLFOΛɺ&NCFEEJOHMBZFSʹ͔͚Δɻʢ˞0$3෦෼͸ֶशෆՄʣ • நग़ͨ͠ςΩετͷݸ਺͕ɺ/ݸʹຬͨͳ͍৔߹͸<1"%>Ͱ݀ຒΊ͢Δɻ/ݸΛ௒͑Δ৔߹͸5PLFOΛഁغ͢Δɻ 4QBUJBM'FBUVSF • 0$3Τϯδϯ͕ݕग़֤ͨ͠ςΩετͷCCPY࠲ඪΛՃ޻ͯ͠4QBUJBM'FBUVSFͱ͢Δɻ • CCPYͷࠨ্࠲ඪɺӈԼ࠲ඪɺ෯ɺߴ͞ɺʢը૾্Ͱࠨ্͔ΒӈԼʹ͔͚ͯCCPYΛφϯόϦϯάͨ͠ͱԾఆͯ͠ʣྡͷ#PYͱͷڑ཭Λɺ&NCFEEJOHMBZFS USBJOBCMF ʹ͔͚Δɻ • 7JTJPOઐ༻ͷ&NCFEEJOHMBZFSͱɺ-BOHVBHFઐ༻ͷ&NCFEEJOHMBZFSΛઃ͚ͯɺͦΕͧΕผʑͷ4QBUJBM'FBUVSFΛੜ੒͢Δɻ • ͜ͷͱ͖ɺ͋Θͤͯ7JTJPO΍-BOHVBHFͷ%1PTJUJPOʢԿ൪໨ͷಛ௃ྔ͔ʁʣ΋ɺ͋Θͤͯ&NCFEEJOHMBZFSʹೖྗ͢Δɻ ը૾σʔλ͔ΒҟͳΔϞμϦςΟͷಛ௃ྔΛநग़ͯ͠ɺೖྗΛΠΠײ͡ʹͨ͠ ຊ࿦จͷΞʔΩςΫνϟͰ͸ɺ֤ϞμϦςΟͰ 
 ಉ͡ݸ਺ ͜ͷਤͰ͸/ ͷಛ௃ྔΛ४උ͍ͯ͠Δɻ CCPYͰUFYUಛ௃ྔͷҐஔ৘ใΛ࣋ͨͤΔͱͱ΋ʹɺ 
 %1PTJUJPOʹΑͬͯɺWJTJPOಛ௃ྔͷҐஔ৘ใ΋࣋ͨͤΒΕΔͱ༧૝ɻ

3FDPOTUSVDUJPOΛؚΉͭͷڭࢣͳ͠λεΫΛఆٛ .BTLFE-BOHVBHF.PEFMJOH • ैདྷͷ#&35ͱಉ༷ʹɺ&ODPEF͞Εͨಛ௃ྔ͔Β.BTLFE5FYUΛ༧ଌ͢Δɻ • ͨͩ͠ɺ.VMUJNPEBMʹͨ͠ಛ௃ྔ͔Β෮ݩͤ͞Δ఺͕ಛ௃తͰ͋Δɻ 3FDPOTUSVDU • &ODPEF͞Εͨશͯͷಛ௃ྔΛར༻ͯ͠ɺ"VUP&ODPEFSͱಉ༷ʹը૾Λ3FDPOTUSVDUJPO͢Δɺ-PTT͸TNPPUI-Λ࠾༻ɻ • ग़ྗ͸ը૾ͷΈ͕ͩɺจࣈը૾Λਖ਼֬ʹ3FDPOTUSVDUJPO͢΂͘ɺ7JTJPOʹՃ͑ͯ-BOHVBHFͷ׆༻๏΋ֶशͯ͘͠ΕΔͱظ଴ɻ 5FYU%FTDSJCF*NBHF • Ұఆׂ߹Ͱը૾ͱ5FYUͷ૊Έ߹ΘͤΛγϟοϑϧͯ͠ɺϖΞ͕ద੾͔Ͳ͏͔ PS Λ༧ଌ͢Δɻ • ্ड़ͷͭͷλεΫ͕ओʹ-PBDM'FBUVSFʹண໨͢ΔҰํͰɺͪ͜Β͸(MPCBM'FBUVSFʹண໨͢΂͖λεΫͰ͋Δɻ • <$-4>Λ&ODͨ͠ಛ௃ྔ͔Β༧૝͍ͯ͠Δͷ͸ɺ"UUFOUJPOͰΠΠײ͡ʹશମ͔Βಛ௃ྔΛू໿͍ͯ͠Δͱظ଴Ͱ͖Δ͔Βɻ ֤छϞʔμϧɺ-PDBM'FBUVSFɺ(MPCBM'FBUVSFΛ׆༻͢ΔΑ͏ ڭࢣͳ͠λεΫΛόϥϯεΑ͘ఏҊͯ͠ɺֶशΛΠΠײ͡ʹͨ͠

ैདྷͱҟͳΔ5SBOTGPSNFSͷΞʔΩςΫνϟΛఏҊ -BOHVBHF'FBUVSFΛ&ODPEF͢ΔաఔͰɺ7JTJPOͱ4QBUJBMΛTLJQDPOOFDUJPOͰ܁Γฦ͠Ϛʔδɻ ֤-BZFS͕7JTJPOͱ4QBUJBMʹΞΫηεՄೳͳɺΠΠײ͡ͷΞʔΩςΫνϟΛ࣮ݱͨ͠ɻ %PD'PSNFSͷΞʔΩςΫνϟ ैདྷͷΞʔΩςΫνϟ

ఏҊΞʔΩςΫνϟͷৄࡉ w -BOHVBHFͱ7JTJPOͦΕͧΕʹઐ༻ͷMBZFSʢࢵͱ྘ʣΛઃ͚ͯɺಛ௃ྔΛม׵ͯ͠࠷ޙʹՃࢉ͢Δ w -BOHVBHF'FBUVSF͸ɺMBZFSΛॏͶΔ͝ͱʹɺҟͳΔϞʔμϧಛ௃Λࠞͥͳ͕Β܁Γฦ͠5SBOTGPSN͞ΕΔ w ҰํͰɺ7JTJPO'FBUVSFͱ4QBUJBM'FBUVSF͸ɺຖճॳظঢ়ଶͰ֤-BZFSʹԣ͔Βྲྀ͠ࠐ·ΕΔ w 4QBUJBM'FBUVSFΛ,FZ΍2VFSZʹ͢ΔMBZFS͸ڞ௨Խʢ੺ʣ͍ͯ͠Δ w ͨͩ͠ɺͦ΋ͦ΋4QBUJBM'FBUVSF͸-BOHVBHFઐ༻ͱ7JTJPOઐ༻Ͱผʑʹ४උͨ͠఺ʹ஫ҙ w 8SFM͸%SFMBUJWFQPTJUJPOFNCFEEJOHͱݺ͹ΕɺJ൪໨ͱK൪໨ͷؔ܎ੑΛ֫ಘ͢ΔͨΊͷֶशՄೳͳύϥϝλ w ࿦จͰ͸ɺྑ͍࣮ݧ݁Ռ͕ಘΒΕͨ͜ͱ͔Β΋ɺʮ֤Ϟʔμϧ΁ͷΞΫηεΛྑͨ͘͜͠ͱ͕ॏཁʂʯͱओு͍ͯ͠Δ https://openaccess.thecvf.com/content/ICCV2021/supplemental/Appalaraju_DocFormer_End-to-End_Transformer_ICCV_2021_supplemental.pdf ελϯμʔυͳܗ͸͜Εʂ

࣮ݧ݁ՌɿैདྷݚڀΑΓ΋ߴੑೳ͔ʁ ๯಄ʹ঺հͨ͠%PDVNFOU6OEFSTUBOEJOHͷλεΫ΍σʔληοτͰ4P5"Λ͔ͬ͞Βͬͨ %PD'PSNFSCBTFˠUSBOTGPSNFSMBZFST %PD'PSNFSMBSHFˠUSBOTGPSNFSMBZFST *54ˠ5FYU *NBHF 4QBUJBM'FBUVSFΛར༻ 54ˠ5FYU 4QBUJBM'FBUVSFͷΈΛར༻

࣮ݧ݁ՌɿैདྷݚڀΑΓ΋ߴੑೳ͔ʁ ͏·͍ͬͯ͘Δྫ .FOVJUFN ੺ 5PBM ੨ 4VC5PUBM ྘ ʹ෼ྨ )FBEFS ੺ 2VFTUJPO ੨ "OTXFS ྘ ʹ෼ྨ

࣮ݧ݁Ռɿ7JTJPO'FBUVSF͸໾ʹཱ͍ͬͯΔ͔ʁ ֤λεΫͰɺʮ54ʯΑΓ΋7JTJPOΛՃ͑ͨʮ547ʯͷํ͕'4DPSF͕େ͖͘ͳͬͨɻ ͨͩ͠ɺ޲্෯͕QPJOUఔ౓ͷ৔߹΋͋Ε͹ɺQPJOUҎԼͷ৔߹΋͋ͬͨɻ %PD'PSNFSCBTFˠUSBOTGPSNFSMBZFST %PD'PSNFSMBSHFˠUSBOTGPSNFSMBZFST *54ˠ5FYU *NBHF 4QBUJBM'FBUVSFΛར༻ 54ˠ5FYU 4QBUJBM'FBUVSFͷΈΛར༻

࣮ݧ݁Ռɿ7JTJPO'FBUVSF͸໾ʹཱ͍ͬͯΔ͔ʁ 7JTJPO'FBUVSF͕໾ʹཱͬͨΘ͔Γ΍͍͢ྫ ߦ໨ͷʮ5"3(&5ʯ͸ɺ54Ͱ͸ޡࣝผ͕ͨ͠ɺ7JTJPOΛՃ͑Δͱ)FBEFSͩͱਖ਼ղͰ͖ͨ ੺ɿ)FBEFS ੨ɿ2VFTUJPO ྘ɿ"OTXFS ਖ਼ղ 5FYUʴ4QBUJBM 5FYUʴ4QBUJBM7JTJPO

࣮ݧ݁ՌɿܰྔԽ΍ޮ཰Խ͸Ͱ͖͍ͯΔ͔ʁ ͋ΒΏΔ৔໘Ͱ׬શউརʂͱ͸ݴ͑ͳ͍͕ɺ ैདྷख๏ͱൺ΂ͯɺগͳ͍ʢ൒෼ҎԼʂʣύϥϝλ਺Ͱಉ౳Ҏ্ͷੑೳΛୡ੒Ͱ͖͍ͯΔͱΘ͔Δɻ %PD'PSNFSCBTFˠUSBOTGPSNFSMBZFST %PD'PSNFSMBSHFˠUSBOTGPSNFSMBZFST *54ˠ5FYU *NBHF 4QBUJBM'FBUVSFΛར༻ 54ˠ5FYU 4QBUJBM'FBUVSFͷΈΛར༻ 😄👍🎉 😄👍🎉

ຊ೔ͷ·ͱΊ %PDVNFOU6OEFSTUBOEJOHͷͨΊͷ5SBOTGPSNFSϞσϧΛఏҊ ೖྗΛΠΠײ͡ʹͨ͠ʂʢ5FYU4QBUJBM7JTJPOʣ ֶशΛΠΠײ͡ʹͨ͠ʂʢ3FDVOTUSVDUJPOλεΫΛ৽نఏҊʣ ΞʔΩςΫνϟΛΠΠײ͡ʹͨ͠ʂʢTLJQߏ଄Ͱ֤ϞʔμϧʹΞΫηε͠΍ͨ͘͢͠ʣ 4P5"ୡ੒͚ͩͰͳ͘ϞσϧܰྔԽͷ؍఺Ͱ΋ྑ͍࣮ݧ݁ՌΛࣔͨ͠ɻ ݸਓతʹؾʹͳͬͨ఺ O൪໨ͷ7JTJPO'FBUVSF͕ɺO൪໨ͷ5FYU'FBUVSFͷCCPYྖҬͷը૾ಛ௃Ͱ͸ͳ͍఺͕ؾʹͳΔɻ 4QBUJBM'FBUVSFΛࠞͥΔաఔͰରԠؔ܎͸औΕ͍ͯΔͱਪଌ͢Δ͕ɺ 7JTJPO'FBUVSFΛ5SBOTGPSNͤ͞Δํ๏͸ɺ·ͩ·ͩݚڀͷ༨஍͕͋Δͱײͨ͡ɻ