[ICASSP2020音響音声読み会] State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations

Z. Zhao, F. Tronarp, R. Hostettler and S. Särkkä, "State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations," Proc. of ICASSP 2020,  pp. 5295-5299, 2020.

Transcript

1. State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations Z. Zhao, F. Tronarp, R. Hostettler and S. Särkkä ICASSP2020 ԻڹԻ੠ಡΈձʢΦϯϥΠϯʣ 2020/06/19 Akira Tamamori @ Aichi Institute of Technology
   
2. จݙ৘ใ • Z. Zhao, F. Tronarp, R. Hostettler and S. Särkkä, "State-Space Gaussian Process for Drift Estimation in Stochastic Differential Equations,” Proc. of ICASSP 2020, pp. 5295-5299, 2020. • ࿦จ https://ieeexplore.ieee.org/document/9054472 • બΜͩཧ༝ • ֬཰ඍ෼ํఔࣜʢSDEʣͷ৴߸ॲཧ / ػցֶश΁ͷԠ༻ʹڵຯ • ৗඍ෼ํఔࣜΛNNͱͯ͠ղऍͨ͠࿦จʢNeural ODEʣ͸ NeurIPS2018ͷϕετϖʔύʔɺޙଓͷNeural PDE/SDE΋೤͍ • य़ͷԻڹֶձͰਂ૚Ψ΢εաఔʹجͮ͘Ի੠߹੒ʹSDEతͳղऍ ΛՃ͑ͯੑೳධՁͨ͠ൃද͋Γʢ౦ژେֶͷ܊ࢁઌੜΒʣ 2 ˞࿦จ1%'͸ஶऀ)1ʹͯ ஶऀ൛͕ެ։͞Ε͍ͯΔ
3. ࿦จ·ͱΊ • എܠɿ SDE͸৴߸ॲཧʹ෯޿͘Ԡ༻ɻͦͷυϦϑτؔ਺Λ σʔλ͔Βਪఆ͢Δ৔߹ɺGPճؼΛ༻͍Δͷ͸1ͭͷखஈ • ໰୊ɿGPճؼ͸O(N^3)ͷܭࢉྔ͕ϘτϧωοΫɻ SDEͷ Ұ࣍ۙࣅղ๏͸ִ࣌ؒؒΛີʹ͠ͳ͍ͱղͷਫ਼౓௿Լ • ߩݙɿܭࢉྔΛO(NlogN)ʹ཈͑ͳ͕ΒɺυϦϑτؔ਺Λ GPճؼ͢Δख๏ΛఏҊɻ2࣍ۙࣅղ๏ʹΑΓղͷਫ਼౓޲্ • ϙΠϯτɿ σʔλ఺Λٖࣅతͳ࣌ࠁͱΈͳͯ͠ιʔτɺ GPΛঢ়ଶۭؒϞσϧʹม׵ͯ͠ϕΠζϑΟϧλΛద༻ 3 ஫ҙɿ͜ͷ࿦จʹ͸ʮԻʯͷ࿩͸ग़͖ͯ·ͤΜ
4. Ψ΢εաఔʢ(BVTTJBO
   1SPDFTT
   (1 • ϊϯύϥϝτϦοΫϕΠζͷ1ख๏ • ճؼλεΫʹΑ͘༻͍ΒΕΔ ⇒ ະ஌ͷؔ਺ܗঢ়Λσʔλ͔Βਪఆ • ܭࢉྔେ: σʔλ਺ͷ3৐ʹൺྫ • Α͘࢖ΘΕΔه๏ʹ׳ΕΔͷ͕٢ 4 02 2 /0 1 /- 0 1 1 22 /. 㱺ؔ਺ ͕ฏۉؔ਺
   ڞ෼ࢄؔ਺ ͷΨ΢εաఔʹै͏ Χʔωϧؔ਺
5. ֬཰ඍ෼ํఔࣜ Stochastic Differential Equation; SDE • ֬ఆతͳৗඍ෼ํఔࣜʹ֬཰తมಈ͕ՃΘͬͨ΋ͷ • ࣌ܥྻʹؚ·ΕΔෆ࣮֬ੑΛଊ͑ͳ͕Β࣌ؒൃలΛϞσϧԽ • υϦϑτ߲͕ʮฏۉʯʢ֬ఆతมಈʣΛදݱ ⇒ ࠓճ͸GPͰճؼ • ֦ࢄ߲͸ʮ෼ࢄʯʢ֬཰తมಈͱෆ࣮֬ੑʣΛදݱ ⇒ ന৭ࡶԻ 5 ৗඍ෼ํఔࣜʢʹ֬ఆతʣ Ordinal Differential Equation (ODE) ֬཰ඍ෼ํఔࣜʢʹ֬཰తʣ Stochastic Differential Equation (SDE) υϦϑτ߲ ֦ࢄ߲ ֬཰త มಈ
6. 4%&ͷͭͷղ๏ʢ਺஋ੵ෼ʣ • Euler-Maruyama (EM) ๏ • ࣌ؒʹؔͯ͠1࣍ͷۙࣅղ๏ • ดܗࣜͰGPճؼ͕ղ͚Δ • ִ࣌ؒؒΛີʹऔΔඞཁ • GPճؼͷͨΊܭࢉྔ͸O(N^3) • Ito-Taylor ๏ • ࣌ؒʹؔͯ͠2࣍ͷۙࣅղ๏ • EM๏͔Β਺஋ੵ෼ͷਫ਼౓޲্ • ดܗࣜͰGPճؼ͕ղ͚ͳ͍ 6 ɿ࣌ؒͷ2࣍ࠩ෼ଟ߲ࣜɺཁ ͷ2֊ඍ෼ ɿΨ΢εཚ਺ͷଟ߲ࣜɺཁ ͷ1֊ඍ෼ ˞ ࣜͷৄࡉ͸෇࿥ࢀর υϦϑτؔ਺Λ(1ͰϞσϧԽ
7. 4%&ͷͭͷղ๏ʢ਺஋ੵ෼ʣ • Euler-Maruyama (EM) ๏ • ࣌ؒʹؔͯ͠1࣍ͷۙࣅղ๏ • ดܗࣜͰGPճؼ͕ղ͚Δ • ִ࣌ؒؒΛີʹऔΔඞཁ • GPճؼͷͨΊܭࢉྔ͸O(N^3) • Ito-Taylor ๏ • ࣌ؒʹؔͯ͠2࣍ͷۙࣅղ๏ • EM๏͔Β਺஋ੵ෼ͷਫ਼౓޲্ • ดܗࣜͰGPճؼ͕ղ͚ͳ͍ 7 ˞ ࣜͷৄࡉ͸෇࿥ࢀর (1ճؼͷܭࢉྔΛ཈͑ͭͭ *UP5BZMPS๏Ͱ4%&ղ͚ͨΒ خ͍͠Μ͚ͩͲͳ͊ʜ
8. ॏཁͳઌߦݚڀ Scalar temporal GP [Särkkä et al., 2013] • ۭ࣌ؒGPճؼΛଟ࣍ݩઢܗSDEʹม׵ͯ͠ղ͘ ⇒ʢඇઢܗʣΧϧϚϯϑΟϧλ/ RTSεϜʔβ͕ར༻ՄʢO(N) ʣ • GPͷಋؔ਺΋ܭࢉՄೳ ⇒ʮঢ়ଶʯͷ੒෼ʹؚΊͯܭࢉ • ͨͩ͠ʮGPͷೖྗʯ͸࣌ࠁॱʹฒΜͰ͍Δඞཁ ⇒ ͔͠͠ࠓճͷGPͷೖྗʹ͸ʮ࣌ࠁʯؚ͕·Ε͍ͯͳ͍ 8 ͦΕͰ΋΍ͬͺΓ4DBMBS
   UFNQPSBM
   (1Λ࢖͍͍ͨʂ (1ճؼͷܭࢉྔΛ཈͑ͭͭ4%&Λղ͚Δ͔΋ʂʁ ˞ۭ࣌ؒ(1
9. ఏҊख๏ʢʣ • Scalar temporal GPΛద༻ • ͨͩ͠ঢ়ଶʢೖྗʣΛٖࣅతͳ࣌ࠁͱݟͳͯ͠ঢॱʹιʔτ 9 ྫɿ{(5, y1), (4, y2), (2, y3), (3, y4), (1, y5)} ͷͱ͖ (1, y5), (2, y3), (3, y4), (4, y2), (5, y1) ⇐ʮ࣌ࠁॱʯʹιʔτ ͱݟͳͯ͠ ٖࣅతͳ࣌ࠁ σʔλ఺ͷ૊ ঢ়ଶͱଌఆ஋ʢೖྗͱग़ྗʣ Λιʔτ ৽͍͠y1 ৽͍͠y2 ৽͍͠y3 ৽͍͠y4 ৽͍͠y5
10. ఏҊख๏ʢʣ • Scalar temporal GPΛద༻ • Ψ΢εաఔΛઢܗ࣌ෆมͳঢ়ଶۭؒϞσϧʹॻ͖׵͑Δ 10 ɿঢ়ଶϕΫτϧʢ ͱͦͷಋؔ਺ʣ ɿ8JFOFSաఔ X
    ύϫʔ ॳظ஋ɿ ɿΧʔωϧؔ਺ (Matérn) ɿఆ਺ߦྻʢΧʔωϧؔ਺ʹґଘ ຊεϥΠυͷ෇࿥ࢀরʣ ฏ׈Խࣄޙ෼෍
11. ఏҊख๏ʢʣ • ϑΟϧλϦϯά / εϜʔδϯά • Euler-Maruyama๏ɿઢܗΧϧϚϯϑΟϧλ/ RTSεϜʔβ • Ito-Taylor๏ɿҰൠԽʢඇઢܗʣΨ΢εϑΟϧλ / εϜʔβ ⇒ Unscented Kalman filter (UKF) ͳͲ͕࢖͑ΔʢUKF͸ຊධՁ࣮ݧͰ΋ར༻ʣ ⇒ Iterated posterior linearization filter / smoother (IPLF / IPLS) • ܭࢉྔɿτʔλϧͰ Λ௒͑ͳ͍ͷ͕خ͍͠ ⇒ ϑΟϧλϦϯά/εϜʔδϯά͸ ɺιʔτ͸ 11 [G.- Fernandez et al., 2017] (1ճؼͷܭࢉྔΛ཈͑ͭͭ4%&Λղ͚Δʂ &VMFS.BSVZBNB๏͚ͩͰͳ͘*UP5BZMPS๏΋࢖͑Δʂ
12. ࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτؔ਺͕ط஌ͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼౓ΛධՁ 1. Ginzburg–Landau double well SDE 2. Modified Benes SDE 12
13. ࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτؔ਺͕ط஌ͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼౓ΛධՁ 㱺γϛϡϨʔγϣϯΛ܁Γฦ͠ɺฏۉ3.4&ͱ$16࣌ؒΛଌఆ 13 ߲໨ ઃఆ಺༰ ࣌ؒ௕ʢඵʣ T = 50s (Ginzburg–Landau double well SDE), T = 5s (Modified Benes SDE) γϛϡϨʔγϣϯ 1000ճࢼߦ
    ֤αϯϓϦϯάִؒ͝ͱ αϯϓϦϯάִؒʢඵʣ Χʔωϧؔ਺ Matérn 5/2, (Ginzburg–Landau) (Modified Benes)
14. ࣮ݧతධՁ • ࣮ݧͦͷ1 • υϦϑτؔ਺͕ط஌ͷSDEΛ༻͍ͯɺGPճؼͷਪఆਫ਼౓ΛධՁ 㱺γϛϡϨʔγϣϯΛ܁Γฦ͠ɺฏۉ3.4&ͱ$16࣌ؒΛଌఆ • ൺֱख๏ • full-batch GP (ۙࣅͳ͠ͷૉͷGP) • fully independent conditional sparse GP (FIC) • deterministic training conditional sparse GP (DTC) • Kalman filter / RTS smoother (KF-RTS) • Unscented Kalman filter (UKF-RTS) • Iterated posterior linearization filter (IPLF-RTS) • Iterated posterior linearization smoother (IPLS) 14 &VMFS .BSVZBNB *UP5BZMPS ఏҊ๏ ैདྷ๏
15. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • υϦϑτؔ਺ͷਪఆσϞ (IT๏: UKF-RTS, EM๏: KF-RTS) ⇒ SDEͷٻੵΛ1௨Γ૸Βͤͨͱ͖ͷ݁Ռʢ1000ճͷ͏ͪͷ1ճʣ 15 Ginzburg–Landau double well SDE Modified Benes SDE ˞άϨʔͷྖҬ͸ ৴པ۠ؒ 6,'354 ˞άϨʔͷྖҬ͸ ৴པ۠ؒ 6,'354 ˞354ͱ͸ εϜʔβͷҙ ʹઃఆ ʹઃఆ
16. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 16 ఏҊ๏ ैདྷ๏ ஋͕খ͍͞΄Ͳྑ͍ ஋͕খ͍͞΄Ͳྑ͍ s
17. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 17 ఏҊ๏ ैདྷ๏ ఏҊ๏͸ۙࣅͳ͠ͷ(1ͱಉఔ౓ͷۙࣅਫ਼౓ s
18. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 18 ఏҊ๏ ैདྷ๏ ۙࣅͳ͠ͷ(1͸ Ұ൪ܭࢉྔ͕ଟ͍ s
19. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 19 ఏҊ๏ ैདྷ๏ εύʔε(1ͱ ಉఔ౓ͷ$16࣌ؒ s
20. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 20 ఏҊ๏ ैདྷ๏ εύʔε(1͸ߴ଎͕ͩ3.4&͸ѱԽ 㱺ճؼਫ਼౓ͱ3.4&͸τϨʔυΦϑ s
21. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • RMSEͱCPU࣌ؒʢGinzburg–Landau w/ Euler-Maruyama๏ʣ 21 • ۙࣅͳ͠(1͔Βਪఆਫ਼౓ΛଛͳΘͳ͍ • εύʔε(1ฒͷ$16࣌ؒ  ఏҊ๏ ैདྷ๏ ·ͱΊΔͱɺఏҊ๏͸ s
22. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • αϯϓϦϯά෯ Λม͑ͨͱ͖ͷRMSEͷਪҠ 22 Ginzburg–Landau double well SDE Modified Benes SDE 
     *UP5BZMPS͸&VMFS.BSVZBNBΑΓ΋3.4&খ  
       
      
       ˞ ͕େ͖͍ ͱ͖ࠩ͸ݦஶ ఏҊ๏ ఏҊ๏ ఏҊ๏ ఏҊ๏
23. ࣮ݧతධՁ • ࣮ݧͦͷ̎ ҎԼͷ࣮σʔλΛར༻ͯ͠ճؼͷ࣮ݧ • ݄ྫ ଠཅࠇ఺׆ಈσʔλ WDC-SILSO • ےిਤ (EMG) from ৺ిਤσʔληοτ 23 by Aalto େֶ & Helsinki େֶதԝපӃ ߲໨ ઃఆ಺༰ Χʔωϧؔ਺ Matérn 5/2, ൺֱख๏ʢͱ΋ʹఏҊ๏ʣ Euler-Maruyama: KF-RTS Ito-Taylor: UKF-RTS, IPLF, and IPLS (ଠཅࠇ఺σʔλ) (EMGσʔλ)
24. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇ఺σʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 24 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍
25. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇ఺σʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 25 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍ ,'354ͷυϦϑτؔ਺͸ ΄΅Ͱɺਪఆ͕͏·͍ͬͯ͘ͳ͍ ,'354 υϦϑτؔ਺͕΄΅̌ ˠ ࠇ఺਺ͷ؍ଌͱϚονͤͣ
26. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇ఺σʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 26 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍ ࠇ఺਺͕෇ۙͰݮগ ˠ ෛ஋ؚ͕·Ε࢝ΊΔ ˠෛ஋ؚ͕·Ε࢝ΊΔ ͭ·Γࠇ఺਺ͷώετάϥϜ ஋͕ݮগʹస͡Δ
27. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇ఺σʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 27 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍ ࠇ఺਺Λ௒͑Δͷ͸Θ͔ͣ ˠ ෛͷ܏͖͕ͦͷ෼େ͖͍ ࠇ఺਺ͷώετάϥϜ ͕Ұఆ஋ʹۙ͘ͳΔ ˠ܏͖͸̌
28. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • ଠཅࠇ఺σʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 28 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍ *1-'ͷυϦϑτؔ਺͸΋ͬͱ΋Β͍͠ਪఆ݁Ռʹͳ͍ͬͯΔ *1-'354 ࠇ఺਺ͷݮগมಈ͕ ઢܗͳυϦϑτؔ਺ ˠઌߦݚڀͱϚον
29. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • EMGσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 29 ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍
30. ࣮ݧ݁Ռʢ࣮ݧͦͷʣ • EMGσʔλʢࠨ͕؍ଌσʔλɺӈ͕ਪఆ͞Εͨؔ਺ʣ 30 ΄ͱΜͲͷσʔλ͕œ 0.1ͰંΓฦ͢ ˠ υϦϑτؔ਺͸ͦͷपลͰϐʔΫ஋ ˞*1-'͕ *UP5BZMPS ˞6,' *1-4͸*1-' ͱ۠ผ͕ ͔ͭͳ͍ *1-'ͷυϦϑτؔ਺͸΋ͬͱ΋Β͍͠ਪఆ݁Ռʹͳ͍ͬͯΔ ˞
    œ෇ۙ ͰંΓฦ͢ͷ͸ ਓؒͷಈ࡞༝དྷ ʢΒ͍͠ʣ œ 0.1෇ۙͰ ϐʔΫ஋
31. ·ͱΊ • ֬཰ඍ෼ํఔࣜʹ͓͚ΔυϦϑτؔ਺ΛܭࢉྔΛ཈͑ͭͭ Ψ΢εաఔճؼͰਪఆ͢Δख๏ΛఏҊ • ܭࢉྔ͕O(NlogN) ͳGPճؼɺSDEͷ2࣍ۙࣅղ๏ͷద༻ • σʔλ఺Λٖࣅతͳ࣌ࠁͱΈͳͯ͠ιʔτ͠ɺঢ়ଶۭؒϞσϧʹ ม׵ͨ͠GPʹؔͯ͠ϑΟϧλ / εϜʔβʔΛద༻ • ఏҊख๏ͷਪఆੑೳΛγϛϡϨʔγϣϯσʔλͱ࣮σʔλΛର৅ ʹݕূ͠ɺఏҊख๏͸੒ޭཪʹಈ࡞͢Δ͜ͱΛ֬ೝ • ٞ࿦͢΂͖఺ɿ࣮ࡍʹʮ࢖͑Δख๏ʯͳͷ͔͍ʁ • ଞͷ࣮σʔλͰ͸ಈ͘ͷ͔ʁ Ի੠ͱ͔Ͳ͏ͳͷʁ • ଟ࣍ݩσʔλ͸Մೳʁσʔλؒͷڑ཭ΛͲ͏ೖΕΔʁͲ͏੔ྻʁ 31
32. ෇࿥ આ໌͢Δ࣌ؒͷ଍Γͳ͞Λิ͏ 
    
33. ෇࿥ɿ4%&͔Β(1ճؼ΁ͷॻ͖׵͑ • step0: ҎԼͷSDE͔Βελʔτ • step1:υϦϑτؔ਺ΛΨ΢εաఔͰϞσϧԽ 33 υϦϑτؔ਺ʢະ஌ʣ ֦ࢄ߲ʢ8JFOFSաఔʣ Χʔωϧؔ਺
34. ෇࿥ɿ4%&͔Β(1ճؼ΁ͷॻ͖׵͑ • step2-1: ΦΠϥʔɾؙࢁ๏ ⇒࣌ؒʹؔͯ͠1࣍ࠩ෼ํఔࣜ 34 ͨͩ͠ ඍ෼ʢ࿈ଓʣ͔Β ࠩ෼ʢ཭ࢄʣ΁ʂ
35. ෇࿥ɿ4%&͔Β(1ճؼ΁ͷॻ͖׵͑ • step2-2: ҏ౻ɾςΠϥʔ๏ ⇒࣌ؒʹؔͯ͠2࣍ࠩ෼ํఔࣜ 35 ͨͩ͠ ˡ͜Ε͕ ࣍ͷ߲
36. ෇࿥ɿఆ਺ߦྻͨͪͷ۩ମܗ • ҎԼͷ௨Γ 36 ɿमਖ਼ୈ̎छϕοηϧؔ਺ Matérn Χʔωϧ
37. ෇࿥ɿଠཅࠇ఺਺ʹؔ͢Δ4%&
    <>ΑΓ • ଠཅͷࠇ఺ܗ੒଎౓͓Αͼࠇ఺਺͕ै͏SDE 37 [2] Edward J. Allen and Chisum Huff, “Derivation of stochastic differential equations for sunspot activity,” Astronomy & Astrophysics, vol. 516, 2010 
    ϥϯμϜʹมԽ͢Δࠇ఺ܗ੒଎౓ 
    ࣌ࠁUʹ͓͚Δࠇ఺਺ 
    ࠇ఺ܗ੒଎౓ͷฏۉʢʹपظతʣ 
    8JFOFSաఔ ɿఆ਺