Upgrade to Pro — share decks privately, control downloads, hide ads and more …

【論文紹介】TABULAR TRANSFORMERS FOR MODELING MULTIVARIATE TIME SERIES

suzuki
September 02, 2022

【論文紹介】TABULAR TRANSFORMERS FOR MODELING MULTIVARIATE TIME SERIES

suzuki

September 02, 2022
Tweet

More Decks by suzuki

Other Decks in Science

Transcript

  1. ɹɹ͖ͣ͢ 5"#6-"353"/4'03.&34'03 .0%&-*/(.6-5*7"3*"5&5*.&4&3*&4 *OLJU1BEIJ :BJS4DIJ ff *HPS.FMOZL .BUUJB3JHPUUJ :PVTTFG.SPVFI 

    1JFSSF%PHOJO+FSSFU3PTT 3BWJ/BJS &SJL"MUNBO
  2. ࿦จͷ໨తɾഎܠ ɾݸਓ৘ใ΍ػີ৘ใͷอޢͷͨΊͷσʔλ෼ੳͷ੍ݶ ɾ࣌ܥྻදσʔλʹؔ͢Δ.-%-ݚڀྫͷগͳ͞ ੩తදσʔλͱͷൺֱ ɹ౳ ֤࢈ۀ ҩྍ΍ۚ༥ͳͲ ʹ๲େͳදσʔλ͕ଘࡏ㱺༗ޮ׆༻΁ͷ՝୊͋Γ ࣌ܥྻදσʔλʹద༻Ͱ͖Δ৽͍͠%FFQ-FBSOJOH %-

    ख๏ͷఏҊ ໨త എܠ ྫ͑͹ʜ
  3. ࿦จͷ֓ཁ ɾݱ࣮ʹ͍ۙ࣌ܥྻදσʔλͷੜ੒ϞσϧͰ͋Δ5BC(15ͷ঺հ ɾ࣌ܥྻදσʔλͷֶशϞσϧͰ͋Δ5BC#&35ͷ঺հ ɾ5BC#&35ͷֶश๏ͷ঺հ ྫɿΫϨδοτΧʔυɺେؾԚછσʔλ  ɾֶशࡁ5BC#&35ͷ׆༻๏ͱ݁Ռͷ঺հ ෆਖ਼ݕ஌ɺԚછ༧ଌ ˞ຊεϥΠυͰ͸5BC#&35 ෆਖ਼ݕ஌

    ͷΈΛ঺հ
  4. લఏ஌ࣝ

  5. ࣗવݴޠॲཧͱ͸ ɾܗଶૉղੳɿจষΛ୯ޠʹ෼ׂͯ͠඼ࢺ΍׆༻ܗΛ൑ఆ ɾݴޠϞσϧɿจষͷࣗવ͞Λ֬཰ʹΑͬͯදݱ ɾݻ༗දݱநग़ɿจষ͔Βਓ໊ͳͲͷݻ༗໊ࢺɺ೔෇ɺ਺஋දݱΛநग़ ɾจষͷྨࣅ౓ൺֱɿจষײͷ಺༰ͷྨࣅ౓ΛఆྔతʹධՁ ɾจষ෼ྨɿจষΛ༩͑ΒΕͨΧςΰϦʔʹ෼ྨ ྫɿωΨϙδ൑ఆ  ɾจষੜ੒ɿ༩͑ΒΕͨจষ΍৚݅ʹଓ͘จষΛੜ੒ ɾจষߍਖ਼ɿจதͷදهϛεΛमਖ਼

    ೔ৗੜ׆Ͱ࢖༻͢Δݴޠʹؔ͢Δ໰୊ λεΫ ΛίϯϐϡʔλͰղ͘͜ͱ λεΫྫ
  6. 5SBOTGPSNFSͱ͸ ࣗવݴޠॲཧͰ࢖༻͞ΕΔ%-Ϟσϧ ʹൃද<> 5SBOTGPSNFSͷߏ଄<> <>"UUFOUJPOJT"MM:PV/FFE  ಛ௕ ɾ&ODPEFSͱ%FDPEFS͔Βߏங ɾ"UUFOUJPO૚ɿฒྻೖྗʹΑΔܭࢉߴ଎Խ ֶश࣌ؒ୹ॖ

    ܭࢉ͕ૣ͘จ຺৘ใͷอ࣋ʹΑΓਫ਼౓΋ߴ͍ ߏ଄ 㱺จ຺ґଘͨ͠ϕΫτϧม׵ ʴ ୯ޠؒରԠؔ܎ͷ൓ө ɾ1PUFOUJBM&ODPEJOH૚ɿ୯ޠ΁ͷҐஔ৘ใຒࠐ
  7. 5SBOTGPSNFSͷ஀ੜഎܠ ɾ47.ɺղੳ໦ //Ҏલ ɿՄม௕σʔλೖྗෆՄɺฒͼॱະߟྀ ɾ8PSE7FD // ɿ4LJQ(SBNʹΑΔ௿ίετֶश ɾ4FR4FR 3// ɿՄม௕σʔλೖྗՄɺೖྗཤྺʹجͮ͘จ຺ͷऔѻ͍

    ݴޠཧղͱഎܠ஌ࣝͷܽ೗㱺Ԡ༻"*ߏஙʹ೉͋Γ㱺5SBOTGPSNFSͷ஀ੜ 4FR4FR<> 4LJQ(SBN<> <>#&35ʹΑΔࣗવݴޠॲཧೖ໳ 5SBOTGPSNFSҎલͷख๏
  8. #&35ͱ͸ 5SBOTGPSNFSΛ࠷ॳʹ࠾༻ͨ͠૒ํ޲ܕͷࣄલֶशϞσϧ 㱠༧ଌϞσϧ <>#&35ʹΑΔࣗવݴޠॲཧೖ໳ ࣄલֶश<> ಛ௕ ɾग़ྗ૚ͷ௥ՃͷΈͰλεΫ ճؼ෼ྨ ʹରԠՄೳ ɾେྔͷڭࢣͳ͠σʔλ͔ΒݴޠಛੑΛ૒ํ޲ֶश

    ࣄલֶश 㱺จ຺೺Ѳ ɾগྔͷڭࢣ͋Γσʔλ͔Βύϥϝʔλඍௐ੔ 'JOF5VOJOH 㱺࠷దԽɺਫ਼౓޲্
  9. #&35ೖྗܗࣜ ɾ୯Ұͷจষ΍จষϖΞΛτʔΫϯྻʹม׵ ɾಛघτʔΫϯ <$-4> <4&1>ͳͲ ͷ௥Ճ τʔΫϯԽ ϕΫτϧԽ ɾτʔΫϯɺจষλΠϓɺจষதͷҐஔʹԠͨ͡ϕΫτϧԽˠ࿨ࢉ <>#&35ʹΑΔࣗવݴޠॲཧೖ໳

    <>#&351SFUSBJOJOHPG%FFQ#JEJSFDUJPOBM5SBOTGPSNFST  τʔΫϯྻ<> ೖྗϕΫτϧ<> #&35΁ೖྗ͢ΔͨΊʹจষΛஈ֊Ͱม׵
  10. #&35ࣄલֶश ̋.BTLFE-BOHVBHF.PEFM .-.  ɾೖྗτʔΫϯͷΛಛघτʔΫϯ<."4,>ʹஔ׵ ɾஔ׵จষΛ#&35ʹೖྗͯ͠<."4,>ͷத਎Λ༧ଌ େྔͷڭࢣͳ͠ ϥϕϧͳ͠ σʔλ͔Β൚༻తͳݴޠύλʔϯΛֶश ໨త

    ख๏ ̋/FYU4FOUFODF1SFEJDUJPO ɾೋจ͕༩͑ΒΕΔλεΫͰͦΕͧΕͷؔ࿈ੑΛֶश ɾಛघτʔΫϯ<$-4>ʹରԠ͢Δ#&35ͷग़ྗΛ෼ྨثʹೖྗˠ࿈ଓੑΛ൑ఆ <>#&35ʹΑΔࣗવݴޠॲཧೖ໳ ϚεΫԽ<>
  11. #&35'JOF5VOJOH ɾࣄલֶशࡁ#&35ϞσϧΛλεΫ಺༰ʹԠͨ͡෼ྨثʹ઀ଓ ɾϥϕϧ෇͖σʔλͰ#&35ͱ෼ྨثͷ྆ํͷύϥϝʔλΛֶश ɾ#&35ॳظ஋͸ࣄલֶशͰಘͨύϥϝʔλ㱺গσʔλ͔ΒߴੑೳϞσϧΛऔಘ ໨త ख๏ গྔͷڭࢣ͋Γ ϥϕϧ෇͖ σʔλ͔ΒλεΫʹಛԽֶͨ͠शΛ࣮ࢪ 'JOF5VOJOH<>

    <>#&35ʹΑΔࣗવݴޠॲཧೖ໳
  12. ࿦จͷ಺༰ 5BC#&35

  13. )JFSBSDIJDBM5BCVMBS#&35 5BC#&35 ɾଟมྔදσʔλͷֶशʹ#&35ΛԠ༻ͨ͠Ϟσϧ<> ɾஈ֊Ͱ&ODPEF㱺ϑΟʔϧυؒͱߦؒͷ྆ํͷରԠؔ܎Λ൓ө )JFSBSDIJDBM 5BC#&35<> <>5"#6-"353"/4'03.&34'03.0%&-*/(.6-5*7"3*"5&5*.&4&3*&4 

  14. 5BC#&35σʔλલॲཧ ɾֶशʹඞཁͳΧϥϜ ϑΟʔϧυ Λநग़ ɾ࿈ଓม਺ͷϑΟʔϧυΛΧςΰϦԽFY ˠ ੜσʔλ લॲཧσʔλ

  15. 5BC#&35ࣄલֶश ɾສτϥϯβΫγϣϯ ສϢʔβ ͷσʔλΛ࢖༻ ɾ8JOEPX 4USJEF εϥΠσΟϯά΢Οϯυ΢  ɾϥϕϧͷϑΟʔϧυb*T'SBVE `Λআڈ

    λεΫ΁ͷӨڹΛ๷ࢭ  ɾ.-.Ͱ<."4,>༧ଌ $SPTT&OUSPQZ-PTTͰਫ਼౓ධՁ .-. ."4, ."4, ."4, ."4, ."4, ."4, ."4, ."4,
  16. 5BC#&35ෆਖ਼ݕ஌λεΫ ɾλεΫͷͨΊʹτϥϯβΫγϣϯͣͭϚʔδ ɾෆਖ਼σʔλ਺গ ໿ສສ ͷͨΊΞοϓαϯϓϦϯά ɾສςετσʔλͰ'είΞ ద߹཰ͱ࠶ݱ཰ͷௐ࿨ฏۉ ΛධՁ ɾ.-1-45.୯ମͱ5BC#&35͋Γͷ'είΞൺֱ㱺5BC#&35͋Γͷํ͕ߴ͍ 'είΞൺֱ<>

    <>5"#6-"353"/4'03.&34'03.0%&-*/(.6-5*7"3*"5&5*.&4&3*&4