Share
Embed
[ICASSP2020音響音声読み会] State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations
Z. Zhao, F. Tronarp, R. Hostettler and S. Särkkä, "State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations," Proc. of ICASSP 2020, pp. 5295-5299, 2020.
Akira Tamamori
June 19, 2020
Transcript
37 slides
-
State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations
Z. Zhao, F. Tronarp, R. Hostettler and S. Särkkä ICASSP2020 ԻڹԻಡΈձʢΦϯϥΠϯʣ 2020/06/19 Akira Tamamori @ Aichi Institute of Technology -
จݙใ • Z. Zhao, F. Tronarp, R. Hostettler and S.
Särkkä, "State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations,” Proc. of ICASSP 2020, pp. 5295-5299, 2020. • จ https://ieeexplore.ieee.org/document/9054472 • બΜͩཧ༝ • ֬ඍํఔࣜʢSDEʣͷ৴߸ॲཧ / ػցֶशͷԠ༻ʹڵຯ • ৗඍํఔࣜΛNNͱͯ͠ղऍͨ͠จʢNeural ODEʣ NeurIPS2018ͷϕετϖʔύʔɺޙଓͷNeural PDE/SDE͍ • य़ͷԻڹֶձͰਂΨεաఔʹجͮ͘Ի߹ʹSDEతͳղऍ ΛՃ͑ͯੑೳධՁͨ͠ൃද͋Γʢ౦ژେֶͷ܊ࢁઌੜΒʣ 2 ˞จ1%'ஶऀ)1ʹͯ ஶऀ൛͕ެ։͞Ε͍ͯΔ -
จ·ͱΊ • എܠɿ SDE৴߸ॲཧʹ෯͘Ԡ༻ɻͦͷυϦϑτؔΛ σʔλ͔Βਪఆ͢Δ߹ɺGPճؼΛ༻͍Δͷ1ͭͷखஈ • ɿGPճؼO(N^3)ͷܭࢉྔ͕ϘτϧωοΫɻ SDEͷ Ұ࣍ۙࣅղ๏ִ࣌ؒؒΛີʹ͠ͳ͍ͱղͷਫ਼Լ •
ߩݙɿܭࢉྔΛO(NlogN)ʹ͑ͳ͕ΒɺυϦϑτؔΛ GPճؼ͢Δख๏ΛఏҊɻ2࣍ۙࣅղ๏ʹΑΓղͷਫ਼্ • ϙΠϯτɿ σʔλΛٖࣅతͳ࣌ࠁͱΈͳͯ͠ιʔτɺ GPΛঢ়ଶۭؒϞσϧʹมͯ͠ϕΠζϑΟϧλΛద༻ 3 ҙɿ͜ͷจʹʮԻʯͷग़͖ͯ·ͤΜ -
Ψεաఔʢ(BVTTJBO1SPDFTT(1 • ϊϯύϥϝτϦοΫϕΠζͷ1ख๏ • ճؼλεΫʹΑ͘༻͍ΒΕΔ ⇒ ະͷؔܗঢ়Λσʔλ͔Βਪఆ • ܭࢉྔେ: σʔλͷ3ʹൺྫ
• Α͘ΘΕΔه๏ʹ׳ΕΔͷ͕٢ 4 02 2 /0 1 /- 0 1 1 22 /. 㱺ؔ ͕ฏۉؔ ڞࢄؔ ͷΨεաఔʹै͏ Χʔωϧؔ -
֬ඍํఔࣜ Stochastic Differential Equation; SDE • ֬ఆతͳৗඍํఔࣜʹ֬తมಈ͕ՃΘͬͨͷ • ࣌ܥྻʹؚ·ΕΔෆ࣮֬ੑΛଊ͑ͳ͕Β࣌ؒൃలΛϞσϧԽ •
υϦϑτ߲͕ʮฏۉʯʢ֬ఆతมಈʣΛදݱ ⇒ ࠓճGPͰճؼ • ֦ࢄ߲ʮࢄʯʢ֬తมಈͱෆ࣮֬ੑʣΛදݱ ⇒ ന৭ࡶԻ 5 ৗඍํఔࣜʢʹ֬ఆతʣ Ordinal Differential Equation (ODE) ֬ඍํఔࣜʢʹ֬తʣ Stochastic Differential Equation (SDE) υϦϑτ߲ ֦ࢄ߲ ֬త มಈ -
4%&ͷͭͷղ๏ʢੵʣ • Euler-Maruyama (EM) ๏ • ࣌ؒʹؔͯ͠1࣍ͷۙࣅղ๏ • ดܗࣜͰGPճؼ͕ղ͚Δ •
ִ࣌ؒؒΛີʹऔΔඞཁ • GPճؼͷͨΊܭࢉྔO(N^3) • Ito-Taylor ๏ • ࣌ؒʹؔͯ͠2࣍ͷۙࣅղ๏ • EM๏͔Βੵͷਫ਼্ • ดܗࣜͰGPճؼ͕ղ͚ͳ͍ 6 ɿ࣌ؒͷ2࣍ࠩଟ߲ࣜɺཁ ͷ2֊ඍ ɿΨεཚͷଟ߲ࣜɺཁ ͷ1֊ඍ ˞ ࣜͷৄࡉࢀর υϦϑτؔΛ(1ͰϞσϧԽ -
4%&ͷͭͷղ๏ʢੵʣ • Euler-Maruyama (EM) ๏ • ࣌ؒʹؔͯ͠1࣍ͷۙࣅղ๏ • ดܗࣜͰGPճؼ͕ղ͚Δ •
ִ࣌ؒؒΛີʹऔΔඞཁ • GPճؼͷͨΊܭࢉྔO(N^3) • Ito-Taylor ๏ • ࣌ؒʹؔͯ͠2࣍ͷۙࣅղ๏ • EM๏͔Βੵͷਫ਼্ • ดܗࣜͰGPճؼ͕ղ͚ͳ͍ 7 ˞ ࣜͷৄࡉࢀর (1ճؼͷܭࢉྔΛ͑ͭͭ *UP5BZMPS๏Ͱ4%&ղ͚ͨΒ خ͍͠Μ͚ͩͲͳ͊ʜ -
ॏཁͳઌߦݚڀ Scalar temporal GP [Särkkä et al., 2013] • ۭ࣌ؒGPճؼΛଟ࣍ݩઢܗSDEʹมͯ͠ղ͘
⇒ʢඇઢܗʣΧϧϚϯϑΟϧλ/ RTSεϜʔβ͕ར༻ՄʢO(N) ʣ • GPͷಋؔܭࢉՄೳ ⇒ʮঢ়ଶʯͷʹؚΊͯܭࢉ • ͨͩ͠ʮGPͷೖྗʯ࣌ࠁॱʹฒΜͰ͍Δඞཁ ⇒ ͔͠͠ࠓճͷGPͷೖྗʹʮ࣌ࠁʯؚ͕·Ε͍ͯͳ͍ 8 ͦΕͰͬͺΓ4DBMBSUFNQPSBM(1Λ͍͍ͨʂ (1ճؼͷܭࢉྔΛ͑ͭͭ4%&Λղ͚Δ͔ʂʁ ˞ۭ࣌ؒ(1 -
ఏҊख๏ʢʣ • Scalar temporal GPΛద༻ • ͨͩ͠ঢ়ଶʢೖྗʣΛٖࣅతͳ࣌ࠁͱݟͳͯ͠ঢॱʹιʔτ 9 ྫɿ{(5, y1),
(4, y2), (2, y3), (3, y4), (1, y5)} ͷͱ͖ (1, y5), (2, y3), (3, y4), (4, y2), (5, y1) ⇐ʮ࣌ࠁॱʯʹιʔτ ͱݟͳͯ͠ ٖࣅతͳ࣌ࠁ σʔλͷ ঢ়ଶͱଌఆʢೖྗͱग़ྗʣ Λιʔτ ৽͍͠y1 ৽͍͠y2 ৽͍͠y3 ৽͍͠y4 ৽͍͠y5 -
ఏҊख๏ʢʣ • Scalar temporal GPΛద༻ • ΨεաఔΛઢܗ࣌ෆมͳঢ়ଶۭؒϞσϧʹॻ͖͑Δ 10 ɿঢ়ଶϕΫτϧʢ ͱͦͷಋؔʣ
ɿ8JFOFSաఔ Xύϫʔ ॳظɿ ɿΧʔωϧؔ (Matérn) ɿఆߦྻʢΧʔωϧؔʹґଘ ຊεϥΠυͷࢀরʣ ฏԽࣄޙ -
ఏҊख๏ʢʣ • ϑΟϧλϦϯά / εϜʔδϯά • Euler-Maruyama๏ɿઢܗΧϧϚϯϑΟϧλ/ RTSεϜʔβ • Ito-Taylor๏ɿҰൠԽʢඇઢܗʣΨεϑΟϧλ
/ εϜʔβ ⇒ Unscented Kalman filter (UKF) ͳͲ͕͑ΔʢUKFຊධՁ࣮ݧͰར༻ʣ ⇒ Iterated posterior linearization filter / smoother (IPLF / IPLS) • ܭࢉྔɿτʔλϧͰ Λ͑ͳ͍ͷ͕خ͍͠ ⇒ ϑΟϧλϦϯά/εϜʔδϯά ɺιʔτ 11 [G.- Fernandez et al., 2017] (1ճؼͷܭࢉྔΛ͑ͭͭ4%&Λղ͚Δʂ &VMFS.BSVZBNB๏͚ͩͰͳ͘*UP5BZMPS๏͑Δʂ -
࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτ͕ؔطͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼ΛධՁ 1. Ginzburg–Landau double well SDE
2. Modified Benes SDE 12 -
࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτ͕ؔطͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼ΛධՁ 㱺γϛϡϨʔγϣϯΛ܁Γฦ͠ɺฏۉ3.4&ͱ$16࣌ؒΛଌఆ 13 ߲ ઃఆ༰ ࣌ؒʢඵʣ
T = 50s (Ginzburg–Landau double well SDE), T = 5s (Modified Benes SDE) γϛϡϨʔγϣϯ 1000ճࢼߦ ֤αϯϓϦϯάִؒ͝ͱ αϯϓϦϯάִؒʢඵʣ Χʔωϧؔ Matérn 5/2, (Ginzburg–Landau) (Modified Benes) -
࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτ͕ؔطͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼ΛධՁ 㱺γϛϡϨʔγϣϯΛ܁Γฦ͠ɺฏۉ3.4&ͱ$16࣌ؒΛଌఆ • ൺֱख๏ • full-batch
GP (ۙࣅͳ͠ͷૉͷGP) • fully independent conditional sparse GP (FIC) • deterministic training conditional sparse GP (DTC) • Kalman filter / RTS smoother (KF-RTS) • Unscented Kalman filter (UKF-RTS) • Iterated posterior linearization filter (IPLF-RTS) • Iterated posterior linearization smoother (IPLS) 14 &VMFS .BSVZBNB *UP5BZMPS ఏҊ๏ ैདྷ๏ -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • υϦϑτؔͷਪఆσϞ (IT๏: UKF-RTS, EM๏: KF-RTS) ⇒ SDEͷٻੵΛ1௨ΓΒͤͨͱ͖ͷ݁Ռʢ1000ճͷ͏ͪͷ1ճʣ 15
Ginzburg–Landau double well SDE Modified Benes SDE ˞άϨʔͷྖҬ ৴པ۠ؒ 6,'354 ˞άϨʔͷྖҬ ৴པ۠ؒ 6,'354 ˞354ͱ εϜʔβͷҙ ʹઃఆ ʹઃఆ -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • αϯϓϦϯά෯ Λม͑ͨͱ͖ͷRMSEͷਪҠ 22 Ginzburg–Landau double well SDE Modified
Benes SDE *UP5BZMPS&VMFS.BSVZBNBΑΓ3.4&খ ˞ ͕େ͖͍ ͱ͖ࠩݦஶ ఏҊ๏ ఏҊ๏ ఏҊ๏ ఏҊ๏ -
࣮ݧతධՁ • ࣮ݧͦͷ̎ ҎԼͷ࣮σʔλΛར༻ͯ͠ճؼͷ࣮ݧ • ݄ྫ ଠཅࠇ׆ಈσʔλ WDC-SILSO • ےిਤ
(EMG) from ৺ిਤσʔληοτ 23 by Aalto େֶ & Helsinki େֶதԝපӃ ߲ ઃఆ༰ Χʔωϧؔ Matérn 5/2, ൺֱख๏ʢͱʹఏҊ๏ʣ Euler-Maruyama: KF-RTS Ito-Taylor: UKF-RTS, IPLF, and IPLS (ଠཅࠇσʔλ) (EMGσʔλ) -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔʣ 25 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4*1-' ͱ۠ผ͕ ͔ͭͳ͍
,'354ͷυϦϑτؔ ΄΅Ͱɺਪఆ͕͏·͍ͬͯ͘ͳ͍ ,'354 υϦϑτ͕ؔ΄΅̌ ˠ ࠇͷ؍ଌͱϚονͤͣ -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔʣ 26 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4*1-' ͱ۠ผ͕ ͔ͭͳ͍
ࠇ͕ۙͰݮগ ˠ ෛؚ͕·Ε࢝ΊΔ ˠෛؚ͕·Ε࢝ΊΔ ͭ·ΓࠇͷώετάϥϜ ͕ݮগʹస͡Δ -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔʣ 27 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4*1-' ͱ۠ผ͕ ͔ͭͳ͍
ࠇΛ͑ΔͷΘ͔ͣ ˠ ෛͷ͖͕ͦͷେ͖͍ ࠇͷώετάϥϜ ͕Ұఆʹۙ͘ͳΔ ˠ͖̌ -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔʣ 28 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4*1-' ͱ۠ผ͕ ͔ͭͳ͍
*1-'ͷυϦϑτؔͬͱΒ͍͠ਪఆ݁Ռʹͳ͍ͬͯΔ *1-'354 ࠇͷݮগมಈ͕ ઢܗͳυϦϑτؔ ˠઌߦݚڀͱϚον -
࣮ݧ݁Ռʢ࣮ݧͦͷʣ • EMGσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔʣ 30 ΄ͱΜͲͷσʔλ͕ 0.1ͰંΓฦ͢ ˠ υϦϑτؔͦͷपลͰϐʔΫ ˞*1-'͕ *UP5BZMPS
˞6,' *1-4*1-' ͱ۠ผ͕ ͔ͭͳ͍ *1-'ͷυϦϑτؔͬͱΒ͍͠ਪఆ݁Ռʹͳ͍ͬͯΔ ˞ۙ ͰંΓฦ͢ͷ ਓؒͷಈ࡞༝དྷ ʢΒ͍͠ʣ 0.1ۙͰ ϐʔΫ -
·ͱΊ • ֬ඍํఔࣜʹ͓͚ΔυϦϑτؔΛܭࢉྔΛ͑ͭͭ ΨεաఔճؼͰਪఆ͢Δख๏ΛఏҊ • ܭࢉྔ͕O(NlogN) ͳGPճؼɺSDEͷ2࣍ۙࣅղ๏ͷద༻ • σʔλΛٖࣅతͳ࣌ࠁͱΈͳͯ͠ιʔτ͠ɺঢ়ଶۭؒϞσϧʹ มͨ͠GPʹؔͯ͠ϑΟϧλ
/ εϜʔβʔΛద༻ • ఏҊख๏ͷਪఆੑೳΛγϛϡϨʔγϣϯσʔλͱ࣮σʔλΛର ʹݕূ͠ɺఏҊख๏ޭཪʹಈ࡞͢Δ͜ͱΛ֬ೝ • ͖ٞ͢ɿ࣮ࡍʹʮ͑Δख๏ʯͳͷ͔͍ʁ • ଞͷ࣮σʔλͰಈ͘ͷ͔ʁ Իͱ͔Ͳ͏ͳͷʁ • ଟ࣍ݩσʔλՄೳʁσʔλؒͷڑΛͲ͏ೖΕΔʁͲ͏ྻʁ 31 -
ɿଠཅࠇʹؔ͢Δ4%& <>ΑΓ • ଠཅͷࠇܗ͓Αͼࠇ͕ै͏SDE 37 [2] Edward J. Allen and
Chisum Huff, “Derivation of stochastic differential equations for sunspot activity,” Astronomy & Astrophysics, vol. 516, 2010 ϥϯμϜʹมԽ͢Δࠇܗ ࣌ࠁUʹ͓͚Δࠇ ࠇܗͷฏۉʢʹपظతʣ 8JFOFSաఔ ɿఆ