Share
Embed
Transcript
250 slides
-
3 ࣌ܥྻͱ ࣌ܥྻ UJNFTFSJFT ࣌ؒͱͱʹෆنଇʹมಈ͢ΔݱΛɺ࿈ଓతʹ؍ଌͯ͠ಘΒΕͨ ͷܥྻ 𝑡؍ଌͨ࣌͠ࠁ 𝑦! ࣌ࠁ 𝑡
ʹ͓͚Δ؍ଌ 𝑦" , … , 𝑦# ࣌ܥྻ ଟ͘ͷ࣌ؒͱͱʹมԽ͍ͯ͘͠σʔλɺ͜ͷΑ͏ͳදهͰදݱ ͢Δ͜ͱ͕Մೳ 𝑇 𝑌 ྫυϧԁ 2022年7⽉1⽇ 𝑦! = 135 -
5 ࣌ܥྻͱ .PUJPO$BQUVSFͷྫ From: Matsubara, Yasuko, and Yasushi Sakurai. "Regime
shifts in streams: Real-time forecasting of co- evolving time sequences." Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. -
6 ࣌ܥྻͱ l ࣌ܥྻ༧ଌকདྷͷมಈΛਪଌ l ࣌ܥྻྨ༩͑ΒΕͨ࣌ܥྻΛछྨ͝ͱʹྨ l ҟৗݕ มԽݕ࣌ܥྻͷதʹଘࡏ͢ΔҟৗΛൃݟ l
ՄࢹԽ ಛநग़ਓ͕ؒղऍՄೳͳܗʹ࣌ܥྻΛม l ੍ޚૢ࡞ՄೳͳมΛมԽͤ͞ɺతมΛ੍ޚ FUD ࣌ܥྻΛѻͬͨ՝ λεΫ -
7 ࣌ܥྻͱ l ࣌ܥྻ༧ଌকདྷͷมಈΛਪଌ l ࣌ܥྻྨ༩͑ΒΕͨ࣌ܥྻΛछྨ͝ͱʹྨ l ҟৗݕ มԽݕ࣌ܥྻͷதʹଘࡏ͢ΔҟৗΛൃݟ l
ՄࢹԽ ಛநग़ਓ͕ؒղऍՄೳͳܗʹ࣌ܥྻΛม l ੍ޚૢ࡞ՄೳͳมΛมԽͤ͞ɺతมΛ੍ޚ FUD ࣌ܥྻΛѻͬͨ՝ λεΫ -
15 ࠓճͷେ·͔ͳ֓ཁ l ౷ܭతϞσϧ l ࣌ܥྻͷੑ࣭ l ݹయత࣌ܥྻϞσϧ l Ϟσϧͷਪఆɾબ
l ਂֶशϕʔεϞσϧ l ۙͷਂֶशͷൃల l ࣌ܥྻ༧ଌ ਂֶश ౷ܭతϞσϧ ਂֶशϕʔεϞσϧ -
17 ࣌ܥྻͷੑ࣭ l ִؒ࣌ܥྻ l αϯϓϦϯάظ͕ؒҰఆ l ݚڀͳͲͰѻ͏ݚڀͷ΄ͱΜͲ͜ΕʹͨΔ ܽଛิͳͲͰִؒʹ͢Δ߹ l
ෆִؒ࣌ܥྻ l Πϕϯτ࣌ܥྻͳͲ l αϯϓϦϯάظ͕ؒૄΒ ִؒ࣌ܥྻ ෆִؒ࣌ܥྻ ࣌ؒ 猫かわいい 1PTU 35 35 -
18 ࣌ܥྻͷੑ࣭ l Ұมྔ࣌ܥྻ l छྨͷܥྻʹΑͬͯߏ l ଟมྔ࣌ܥྻ l ෳͷܥྻʹΑͬͯߏ
Ұมྔ࣌ܥྻ ଟมྔ࣌ܥྻ -
19 ࣌ܥྻͷੑ࣭ l ૬Ճ ฏۉ l ࢄ l ฏۉ͔ΒͷΒ͖ͭ۩߹Λࣔ͢ࢦඪ l
ฏۉ͔ΒͷࠩͰ͋ΔภࠩΛೋͷฏۉ l ڞࢄ l ڞࢄ͕େ͖͍ͱਖ਼ͷڧ͍ؔΛ࣋ͭͳͲɺ ͷରԠ͢Δσʔλͷؔ Λࣔ͢ ఆৗੑඇఆৗੑ 𝐸(𝑦) = µ = 1 𝑛 ) 𝑦! 𝜎" = 𝑉𝐴𝑅 𝑦! = 1 𝑛 ) 𝑦! − µ " Cov(𝑥, 𝑦) = 𝜎#$ = E[ 𝑥 − µ# (𝑦 − µ$ )] -
20 ࣌ܥྻͷੑ࣭ l ڧఆৗੑ TUSJDUTUBUJPOBSJUZ l ҙͷtͱ𝑘 ҙͷ࣌ؒࠩ ʹରͯ͠ɺ σʔλΛੜ͢Δ֬աఔ
f(𝑦!, … , 𝑦!+,)͕ৗʹಉҰͷಉ࣌Λ࣋ͭ l ͕֬࣌ؒมԽ͠ͳ͍ l ྫαΠίϩͷग़ͷܥྻίΠϯτεͷ݁ՌͳͲ ఆৗੑඇఆৗੑ f 𝑦% , … , 𝑦& = f 𝑦' , … , 𝑦'(& -
21 ࣌ܥྻͷੑ࣭ l ऑఆৗੑ XFBLTUBUJPOBSJUZ l ظͱࣗݾڞࢄ͕࣌ʹґଘ͠ͳ͍ l ҙͷtͱ𝑘 ҙͷ࣌ؒࠩ
ʹରͯ͠ҎԼ͕Γཱͭ ࣌ʹґଘ͠ͳ͍ l ࣍ͷϞʔϝϯτ·Ͱ͕ෆม l ҰൠతʹɺʮఆৗੑʯͱݺΕΔ࣌ɺͪ͜ΒΛࢦ͢ࣄ͕ଟ͍ l ࣗݾ૬͕ؔLͱͱʹࢦతʹݮਰ ఆৗੑඇఆৗੑ E 𝑦* = E 𝑦*(& , VAR 𝑦* = VAR 𝑦*(& Cov 𝑦* , 𝑦*(& = 𝛾& > 0 -
22 ࣌ܥྻͷੑ࣭ l ऑఆৗੑ XFBLTUBUJPOBSJUZ l ظͱࣗݾڞࢄ͕࣌ʹґଘ͠ͳ͍ l ҙͷtͱ𝑘 ҙͷ࣌ؒࠩ
ʹରͯ͠ҎԼ͕Γཱͭ ࣌ʹґଘ͠ͳ͍ l ࣍ͷϞʔϝϯτ·Ͱ͕ෆม l ҰൠతʹɺʮఆৗੑʯͱݺΕΔ࣌ɺͪ͜ΒΛࢦ͢ࣄ͕ଟ͍ l ࣗݾ૬͕ؔLͱͱʹࢦతʹݮਰ ఆৗੑඇఆৗੑ E 𝑦* = E 𝑦*(& , VAR 𝑦* = VAR 𝑦*(& Cov 𝑦* , 𝑦*(& = 𝛾& > 0 ଟ͘ͷ࣌ܥྻ౷ܭతϞσϧ͕ఆৗੑΛલఏͱ͍ͯ͠Δͨ Ίɺੳ͢Δ࣌ܥྻ͕ఆৗͰ͋Δͱخ͍͠ ఆৗੑΛલఏͱ͠ͳ͍ͱɺฏۉࢄ͕ҰఆͰແ͘ͳΔͨ Ίɺੳ͕ؤ݈Ͱͳ͘ͳΔ -
23 ࣌ܥྻͷੑ࣭ l ऑఆৗ ਖ਼ن 㱺 ڧఆৗ l ฏۉɺࢄɺڞࢄ͕ଘࡏ͢Δڧఆৗ 㱺
ऑఆৗ ˎ ڧఆৗΛຬ͔ͨ͢Βͱ͍ͬͯɺඞͣ͠ऑఆৗͷ݅Λຬͨ͢ ͱݶΒͳ͍ FH ฏۉɺࢄ͕ଘࡏ͠ͳ͍ͳͲ ίʔγʔͳͲͷ͕ॏ͘ɾ͍ ఆৗੑඇఆৗੑ -
25 ࣌ܥྻͷੑ࣭ l ରม l σʔλมಈͷܹ͍࣌͠ܥྻΛɺτϨϯυΛอͬͨ··খ͞ͳʹม͢Δ ख๏ l ਖ਼ͷΈͷσʔλΛෛͷΛͱΔΑ͏ʹมՄೳ ఆৗੑͷม
log 𝑦* = 𝑧* from: https://datascienceplus.com /time-series-analysis-in-r- part-2-time-series- transformations/ -
26 ࣌ܥྻͷੑ࣭ l #PY$PYม l σʔλͷΛਖ਼نʹ͚ۙͮΔͨΊͷ͖ܕม l 𝜆ύϥϝʔλͰɺ 𝜆ʹΑͬͯ มͷܗ͕ࣜมΘΔ
ఆৗੑͷม 𝑧* = B $!+% , (𝜆 ≠ 0) ln 𝑦 (𝜆 = 0) from: http://www.kmdatascience.com/2017/07/box-cox-transformations-in-python.html -
27 ࣌ܥྻͷੑ࣭ l ϩδοτม l ͔Β·ͰͷΛͱΔׂ߹֬ͷσʔλΛਖ਼ͱෛͷແݶେͷʹม͢ Δ͜ͱͰ࣌ܥྻϞσϦϯάΛ༰қʹ͢Δม ఆৗੑͷม 𝑧* =
log $" %+$" 0,1 ↔ (−∞, ∞) -
28 ࣌ܥྻͷੑ࣭ l ֊ ֊ࠩม l τϨϯυΛऔΓআ͖ɺฏۉʹؔͯ͠ఆৗੑΛຬͨ͢Α͏ʹม ఆৗੑͷม 𝑧* =
𝑦* − 𝑦*+% from: https://datascienceplus.com/time-series-analysis-in-r-part-2-time-series-transformations/ -
29 ࣌ܥྻͷੑ࣭ l قઅ ֊ࠩม l قઅʹΑΔӨڹपظੑΛऔΓআ͖ɺฏۉʹؔͯ͠ఆৗੑΛຬͨ͢Α ͏ʹม l 𝑠
पظ ఆৗੑͷม 𝑧* = 𝑦* − 𝑦*+- = (1 − 𝐵-)𝑦* from: https://datascienceplus.com/time-series-analysis-in-r-part-2-time-series-transformations/ -
30 ࣌ܥྻͷੑ࣭ l ฏԽ l ϊΠζظతͳมಈΛআ͘ޮՌΛظͯ͠ɺΒ͔ʹͳΔΑ͏ʹม l දྫͱͯ͠ɺҠಈฏۉϑΟϧλ .PWJOH"WFSBHF'JMUFS ఆৗੑͷม
𝑧* = 1 M ) ./0 1+% 𝑦. from: https://machinelearningmastery.com/moving-average-smoothing-for-time-series- forecasting-python/ -
32 ࣌ܥྻͷੑ࣭ l ࣗݾڞࢄؔ l ऑఆৗաఔͷࣗݾڞࢄΛԾఆ l ࣗݾڞࢄؔҎԼͷܗͰఆٛ͞ΕΔ l ࣗݾڞࢄؔۮؔCov
𝑦!, 𝑦!+, = 𝛾, = 𝛾-, = Cov 𝑦!, 𝑦!-, l ࣗݾڞࢄؔͷઈରͷ࠷େ𝛾. 𝛾. ≥ 𝛾, l ࣗݾڞࢄؔͷਤࣔͱͯ͠ίϨϩάϥϜ͕༻͍ΒΕΔ ࣌ܥྻͷͷཧղ Cov 𝑦% , 𝑦& = Cov 𝑦* , 𝑦*(& = 𝛾& 𝑘ϥάΛද͢ -
33 ࣌ܥྻͷੑ࣭ l ࣗݾ૬ؔؔ l ऑఆৗաఔͷࣗݾ૬ؔΛԾఆ l ҎԼͷܗͰఆٛ͞ΕΔ ࣌ܥྻͷͷཧղ Cor
𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘ϥάΛද͢ -
37 ࣌ܥྻͷੑ࣭ l εϖΫτϧղੳ l ࣌ܥྻσʔλΛߏपʹղ͠ɺ֤पͱΤωϧΪʔͷؔΛऔ Γग़ͨ͢Ίͷख๏ l Α͘༻͍ΒΕΔͷɺࢄϑʔϦΤม ''5
l ෳࡶͳपظΛऔΓग़͢͜ͱΛՄೳʹ͢Δɻपظੑͱ𝑝͕पظΛࣔ࣌͢ʹ 𝑦! = 𝑦!-/ ͕Γཱͭ͜ͱ ࣌ܥྻͷͷཧղεϖΫτϧղੳ from: https://qiita.com/AnchorB lues/items/5497ee68c3a3 d64875d4 FFT 振動数 振幅 -
38 ࣌ܥྻͷੑ࣭ l ϑʔϦΤม l प𝒇ɼपظؔy(t ͱͨ͠ͱ͖ҎԼͷܗʹͳΔ l ෳͷͰܥྻΛઆ໌Ͱ͖ΔΑ͏ʹղ l
ٯϑʔϦΤม l 𝟐𝝅𝒇ͷਖ਼ݭʹରͯ͠ॏΈΛֻ͚ͯɺੵ͢Δ͜ͱͰݩͷܥྻʹ͢ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝒀 𝒇 = 2 "# # 𝒚(𝒕)𝒆"𝟐𝝅𝒊𝒕𝒇𝒅𝒕 ∗ ∫ "# # 𝒚 𝒕 𝒅𝒕 < ∞Λຬͨ͢ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 -
39 ࣌ܥྻͷੑ࣭ l ϑʔϦΤม l प𝒇ɼपظؔy(t ͱͨ͠ͱ͖ҎԼͷܗʹͳΔ l ෳͷͰܥྻΛઆ໌Ͱ͖ΔΑ͏ʹղ l
ٯϑʔϦΤม l 𝟐𝝅𝒇ͷਖ਼ݭʹରͯ͠ॏΈΛֻ͚ͯɺੵ͢Δ͜ͱͰݩͷܥྻʹ͢ ࣌ܥྻͷͷཧղεϖΫτϧղੳ ∗ ∫ "# # 𝒚 𝒕 𝒅𝒕 < ∞Λຬͨ͢ ॏΈਖ਼ݭ 𝒀 𝒇 = 2 "# # 𝒚(𝒕)𝒆"𝟐𝝅𝒊𝒕𝒇𝒅𝒕 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 -
40 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ϑʔϦΤ𝒀(𝒇)Ͱ͋Δ𝒆𝒊𝒕𝟐𝝅𝒇ͷৼΕ෯Λද͢ 㱺 𝒀(𝒇) 𝟐ΤωϧΪʔ l
ܥྻͷதʹؚ·ΕΔपͱͦͷύϫʔΛՄࢹԽ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 ॏΈਖ਼ݭ -
41 ࣌ܥྻͷੑ࣭ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝑓(𝑡) = B 1, 0 < 𝑡
< 𝜏 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 ೖྗܥྻ 𝐹 𝜔 = 2 "# # 𝑓(𝑡)𝑒")*!𝑑𝑡 = 2 + , 𝑓(𝑡)𝑒")*!𝑑𝑡 = 1 −𝑗𝑤 𝑒")*! + , = 1 𝑗𝑤 1 − 𝑒")*, = -!"#$/& ). 𝑒)*,/0 − 𝑒")*,/0 = 𝑒")*,/0 0 * sin *, 0 ϑʔϦΤม 𝐹 𝜔 = 𝑒")*,/0 2 𝜔 sin 𝜔𝜏 2 = 2 𝜔 sin 𝜔𝜏 2 ৼΕ෯εϖΫτϧ ( 𝐹 𝜔 = 𝐹 𝜔 𝑒)1(*) ) ύϫʔεϖΫτϧ 𝐹 𝜔 0 = 2 𝜔 sin 𝜔𝜏 2 0 = 4 𝜔0 sin0 𝜔𝜏 2 -
42 ࣌ܥྻͷੑ࣭ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝑓(𝑡) = B 1, 0 < 𝑡
< 𝜏 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 ೖྗܥྻ 𝐹 𝜔 = 2 "# # 𝑓(𝑡)𝑒")*!𝑑𝑡 = 2 + , 𝑓(𝑡)𝑒")*!𝑑𝑡 = 1 −𝑗𝑤 𝑒")*! + , = 1 𝑗𝑤 1 − 𝑒")*, = -!"#$/& ). 𝑒)*,/0 − 𝑒")*,/0 = 𝑒")*,/0 0 * sin *, 0 ϑʔϦΤม 𝐹 𝜔 = 𝑒")*,/0 2 𝜔 sin 𝜔𝜏 2 = 2 𝜔 sin 𝜔𝜏 2 ৼΕ෯εϖΫτϧ ( 𝐹 𝜔 = 𝐹 𝜔 𝑒)1(*) ) ύϫʔεϖΫτϧ 𝐹 𝜔 0 = 2 𝜔 sin 𝜔𝜏 2 0 = 4 𝜔0 sin0 𝜔𝜏 2 -
43 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ϑʔϦΤ𝒀(𝒇)Ͱ͋Δ𝒆𝒊𝒕𝟐𝝅𝒇ͷৼΕ෯Λද͢ 㱺 𝒀(𝒇) 𝟐ΤωϧΪʔ l
ܥྻͷதʹؚ·ΕΔपͱͦͷύϫʔΛՄࢹԽ l Οʔφʔɾώϯνϯ 8JFOFS,IJOUDIOF ͷެࣜࣗݾڞࢄؔ𝛾, ͷϑʔ ϦΤมύϫʔεϖΫτϧʹͳΔ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝒚 𝒕 = 2 "# # 𝒀(𝒇)𝒆𝟐𝝅𝒊𝒕𝒇𝒅𝒇 ॏΈਖ਼ݭ Cov 𝑦", 𝑦, = 𝐶ov 𝑦!, 𝑦!+, = 𝛾, 𝑘ϥάΛද͢ 𝑌 𝑓 = ? !"#$ $ 𝛾!𝑒#%&'!( − 1 2 ≤ 𝑓 ≤ 1 2 -
44 ࣌ܥྻͷੑ࣭ l ύϫʔεϖΫτϧ l ۮؔͱίαΠϯมͷੑ࣭͔Β l ྫ୯ৼಈ ࣌ܥྻͷͷཧղεϖΫτϧղੳ 𝑌
𝑓 = ? !"#$ $ 𝛾!𝑒#%&'!( = 𝛾) + 2 ? !"* $ 𝛾!cos (2𝜋𝑘𝑓) 𝛾! = 𝑎% 2 cos(2𝜋𝑓)𝑘) 𝑌 𝑓 = ? !"#$ $ cos(2𝜋𝑓)𝑘)cos(2𝜋𝑓𝑘) = 1 2 ? !"#$ $ cos 2𝜋𝑘 𝑓) − 𝑓 − cos 2𝜋𝑘 𝑓) + 𝑓 = M * %+, 𝑓 = 𝑓) 0, 𝑓 ≠ 𝑓) -
46 ࣌ܥྻϞσϧ ࣌ܥྻϞσϧͷߏங ⼊⼒特徴 𝑦!-W … , 𝑦!-" ͳͲ ホワイトノイズ
𝜀! 時系列モデル 時系列 𝑦! -
47 ࣌ܥྻϞσϧ l ࣌ؒతʹ૬͕ؔແ͍࣌ܥྻ𝜀! l ࣗݾ૬ؔؔ l ֬తͳΒ͖ͭΛදݱ l FH
ಠཱʹੜͨ͠ਖ਼نཚྻͳͲ ന৭ࡶԻ ϗϫΠτϊΠζ 𝐸 𝜀* = 0, 𝑉𝐴𝑅 𝜀* = 𝜎", 𝑅. = 0 𝑗ϥάΛද͢ from: https://machinelearningmastery.com/white-noise-time-series-python/ -
48 ࣌ܥྻϞσϧ l 𝑝࣍ͷઢܗࠩํఔࣜʹϗϫΠτϊΠζΛՃ͑ͨϞσϧ"3 𝒑 l 𝑝࣍"3ϞσϧͱݺΕΔ l աڈͷܥྻ 𝑦!-"
, … 𝑦!-/ ͷೖྗͱϗϫΠτϊΠζͰݱࡏͷܥྻΛճؼ͢Δ Ϟσϧ ࣗݾճؼաఔ "3Ϟσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* -
49 ࣌ܥྻϞσϧ l લͷ𝒒 ݸͷϗϫΠτϊΠζͷՃॏͰఆٛ͞ΕΔ𝒒 ࣍."Ϟσϧ." 𝒒 l աڈͷܥྻ 𝑦!-",
… 𝑦!-/ ʹ༩͑ΒΕΔϗϫΠτϊΠζ ϥϯμϜγϣοΫ ͕࣌ܥྻͷকདྷͷʹൖ͢Δ͜ͱΛϞσϦϯά l ϗϫΠτϊΠζͱಉ༷ʹࣗݾڞࢄظ࣌ʹґଘͤͣɺऑఆৗੑ Λຬͨ͢ Ҡಈฏۉաఔ ."Ϟσϧ 𝑦* = 𝜇 + ) !/0 Q 𝑏! 𝜀*+! -
50 ࣌ܥྻϞσϧ l લͷ𝒑 ݸͷܥྻͱલͷ𝒒 ݸͷϗϫΠτϊΠζͷՃॏʹΑͬͯߏ͞ ΕΔϞσϧ"3." 𝒑, 𝒒) l
"3Ϟσϧ."Ϟσϧ"3."Ϟσϧͷಛघͳྫ l "3Ϟσϧ𝒒 = 0 l ."Ϟσϧ 𝒑 = 0 l ϗϫΠτϊΠζ𝒑 = 0, 𝒒 = 0 l ϗϫΠτϊΠζ𝜀! աڈͷܥྻͱແ૬ؔ𝔼 𝜀! , 𝑦!-X = 0, j > 0 ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀* -
51 ࣌ܥྻϞσϧ l -BHPQFSBUPSΛ༻͍Δ͜ͱͰ"3." 𝒑, 𝒒)ϞσϧΛҎԼͷܗʹॻ͖͑Δ ͜ͱ͕Ͱ͖Δ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ 𝑦*
= ) !/% P 𝑎! 𝑦*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀* 𝐿-BHPQFSBUPS 𝐿% 𝑦* = 𝑦*+% , 𝐿& 𝑦* = 𝑦*+& , (1 − ) !/% P 𝑎! 𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) "30QFSBUPS ."0QFSBUPS 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀* -
52 ࣌ܥྻϞσϧ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ (1 − ) !/% P 𝑎!
𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀* 𝑦* = 𝑎(𝐿)+%𝑏(𝐿)𝜀* 𝑦* = ) ./0 T 𝑔. 𝐿.𝜀* = ) ./0 T 𝑔. 𝜀*+. -
53 ࣌ܥྻϞσϧ ࣗݾճؼҠಈฏۉաఔ "3."Ϟσϧ (1 − ) !/% P 𝑎!
𝐿!)𝑦* = (1 − ) !/% Q 𝑏! 𝐿!)𝜀* = 𝑎(𝐿) = 𝑏(𝐿) 𝑎(𝐿)𝑦* = 𝑏(𝐿)𝜀* 𝑦* = 𝑎(𝐿)+%𝑏(𝐿)𝜀* 𝑦* = ) ./0 T 𝑔. 𝐿.𝜀* = ) ./0 T 𝑔. 𝜀*+. "3."Ϟσϧແݶ࣍ͷ."Ϟσϧ Ͱදݱ͕Մೳ Πϯύϧε Ԡؔ -
54 ࣌ܥྻϞσϧ ΠϯύϧεԠؔ *3' 𝑦* = ) ./0 T 𝑔.
𝜀*+. l ༩͑ΒΕͨϊΠζ Πϯύϧε ͕𝑗ظޙʹ࣌ܥྻʹͲΕ͘Β͍ӨڹΛ༩͑ Δ͔ʢӨڹྗ͕ͲΕ͚ͩΔ͔ʣΛࣔͨؔ͠ 𝑔0 = 1, 𝑔! = ) ./% P 𝑎. 𝑔!+. − 𝑏! *3'ͷࢉग़ from: https://alexchinco.com/impulse-response-functions-for-vars/ -
55 ࣌ܥྻϞσϧ l ࣗݾ૬ؔؔ BVUPDPSSFMBUJPOGVODUJPO"$' l ภࣗݾ૬ؔؔ BVUPDPSSFMBUJPOGVODUJPO1"$' l ϥά𝑘ͷࣗݾ૬ؔΛߟྀ͢Δࡍʹɺϥά1,
… 𝑘 − 1·Ͱͷܥྻͷࣗݾ૬ؔͷ Өڹଘࡏ͢Δ l ͜Ε·ͰͷϥάͷӨڹΛআڈͨ͠૬ؔؔ ࣗݾ૬ؔؔ "$' ͱภࣗݾ૬ؔؔ 1"$' Cor 𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘ϥάΛද͢ PACF 𝑦* , 𝑦*(& = Cov 𝑦*(& 𝑦*(&+% , … , 𝑦*(% , 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦*(& 𝑦*(&+% , … , 𝑦*(% -
56 ࣌ܥྻϞσϧ l ࣗݾ૬ؔؔ BVUPDPSSFMBUJPOGVODUJPO"$' l ภࣗݾ૬ؔؔ BVUPDPSSFMBUJPOGVODUJPO1"$' l ϥά𝑘ͷࣗݾ૬ؔΛߟྀ͢Δࡍʹɺϥά1,
… 𝑘 − 1·Ͱͷܥྻͷࣗݾ૬ؔͷ Өڹଘࡏ͢Δ l ͜Ε·ͰͷϥάͷӨڹΛআڈͨ͠૬ؔؔ ࣗݾ૬ؔؔ "$' ͱภࣗݾ૬ؔؔ 1"$' Cor 𝑦* , 𝑦*(& = 234 $",$"#$ 678 $" 678($"#$) = ;$ ;% = 𝑅& 𝑘ϥάΛද͢ PACF 𝑦* , 𝑦*(& = Cov 𝑦*(& 𝑦*(&+% , … , 𝑦*(% , 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦* 𝑦*(&+% , … , 𝑦*(% 𝑉𝑎𝑟 𝑦*(& 𝑦*(&+% , … , 𝑦*(% -
60 ࣌ܥྻϞσϧ l "3Ϟσϧͷࣗݾ૬ؔͷ͞ແݶ͕ͩɺภࣗݾ૬ؔͷ͞"3 ͷ l ."Ϟσϧͷภࣗݾ૬ؔͷ͞ແݶ͕ͩɺࣗݾ૬ؔͷ͞." ͷ l "3."Ϟσϧࣗݾ૬ؔɺภࣗݾ૬ؔͱʹݮਰ͍͕ͯ͘͠Ұఆ
ʹଘࡏ͢Δ ࣗݾ૬ؔؔ "$' ͱภࣗݾ૬ؔؔ 1"$' -
62 ࣌ܥྻϞσϧ l ฏۉʹؔͯ͠ඇఆৗͳ࣌ܥྻΛL֊ࠩΛߦ͍ɺ"3."ϞσϧΛϑΟο ςΟϯάͨ͠Ϟσϧ"3*." 𝒑, 𝒌, 𝒒) l 𝒌
= 𝟎ͷ࣌"3."Ϟσϧ ࣗݾճؼҠಈฏۉաఔ "3*."Ϟσϧ 𝑧* = ) !/% P 𝑎! 𝑧*+! + ) !/% Q 𝑏! 𝜀*+! + 𝜀* ∆𝑦* = 𝑦* − 𝑦*+% 𝑧* = ∆&𝑦* L֊ࠩΛऔΔ͜ͱͰ ఆৗͳ࣌ܥྻʹม -
63 ࣌ܥྻϞσϧ l قઅੑ֊ࠩΛߟྀͨ͠"3*."Ϟσϧͱɺقઅੑ֊ࠩΛߟྀ͠ͳ͍௨ৗͷ "3*."ϞσϧͷͭͷΛΈ߹Θͨ͠Ϟσϧ 4"3*." "3*." 𝒑, 𝒌, 𝒒) º
𝑷, 𝑲, 𝑸)𝑺 4FBTPOBM"3*."Ϟσϧ 4"3*." 𝑧* U = ) !/% P 𝑎! U𝑧*+! U + ) !/% Q 𝑏! U𝜀*+! + 𝜀* 𝑧* U = 𝑦* − 𝑦*+U 𝑆Ұपظͷ͞ -
64 ࣌ܥྻϞσϧ l ྫ𝑆 = 12ͷ4"3*."Ϟσϧ 4"3*." "3*." 𝟏, 𝟎, 𝟎)
º 𝟏, 𝟎, 𝟎)𝟏𝟐 4FBTPOBM"3*."Ϟσϧ 4"3*." 1 − 𝑎% 𝐿% 1 − 𝑎% U𝐿%" 𝑦* = 𝜀* 𝑦* = 𝑎% 𝑦*+% + 𝑎% U𝑦*+%" − 𝑎% 𝑎% U𝑦*+%V + 𝜀* 𝐿-BHPQFSBUPS 𝐿% 𝑦* = 𝑦*+% , 𝐿& 𝑦* = 𝑦*+& , قઅੑ ඇقઅੑ -
65 ࣌ܥྻϞσϧ 4FBTPOBM"3*."Ϟσϧ 4"3*." Kumar Barik, Aditya, et al. "Analysis
of GHI Forecasting Using Seasonal ARIMA." Data Management, Analytics and Innovation. Springer, Singapore, 2021. 55-69. -
66 ࣌ܥྻϞσϧ l ࣌ܥྻΛτϨϯυɺقઅੑͳͲͷෳͷߏཁૉʹղ͢Δख๏ l "EEJUJWFEFDPNQPTJUJPO l .VMUJQMJDBUJWFEFDPNQPTJUJPO l දతͳख๏
l X11 decomposition l STL decopostion l SEATS decompositon ࣌ܥྻղ 5JNFTFSJFTEFDPNQPTJUJPO 𝑦! = 𝑆! ×𝑇! ×𝑅! , log 𝑦! = log 𝑆! + log 𝑇! + log 𝑅! 𝑦! = 𝑆! + 𝑇! + 𝑅! -
68 ࣌ܥྻϞσϧ l ࣌𝑡ʹ͓͚Δ𝑛ݸͷมྔ͔ΒͳΔܥྻσʔλΛϕΫτϧͱͯ͠ଊ͑Δ l ϗϫΠτϊΠζಉ༷ʹϕΫτϧͱͯ͠ଊ͑Δ l 7"3 Q ϞσϧҎԼͷࣜͱͳΔ
ϕΫτϧࣗݾճؼϞσϧ 7"3Ϟσϧ 𝑌* = (𝑦%,* , 𝑦",* , … , 𝑦',* )W 𝔼 𝜺𝒕 = [0, … , 0]W, 𝔼 𝜺𝒕 𝜺𝒕+𝒌 = [ 𝚺 (𝑘 = 0) 𝟎, (𝑘 ≠ 0) 𝑌* = ) !/% P 𝑨𝒊 𝑌*+! + 𝜺𝒕 𝑨𝒊 = 𝑎! (1,1) ⋯ 𝑎! (1, 𝑛) ⋮ ⋱ ⋮ 𝑎! (𝑛, 1) ⋯ 𝑎! (𝑛, 𝑛) -
69 ࣌ܥྻϞσϧ άϨϯδϟʔҼՌ (SBOHFS$BVTBMJUZ l ࣌ܥྻؒಉ࢜ͷҼՌؔΛਪఆ͢ΔͨΊͷํ๏Ͱɺܥྻಉ࢜ͷ ͖Λൃݟ͢Δख๏ l 𝑦%,* ͷ༧ଌʹ͓͍ͯ
𝑦%,* ͷաڈͷʹج͍ͮͨ༧ଌΑΓɺ 𝑦%,* , 𝑦" * ͷաڈͷΛ༻͍ͨ༧ଌʹΑͬͯ.4&͕খ͘͞ͳΔ߹ 𝒚𝟐 𝒕 ͔Β 𝒚𝟏,𝒕 ͷάϨϯδϟʔҼՌੑ͕ଘࡏ͢Δͱ͢Δ 𝑦% 𝑦" -
70 ࣌ܥྻϞσϧ άϨϯδϟʔҼՌ (SBOHFS$BVTBMJUZ l άϨϯδϟʔҼՌͷൃݟʹ'౷ܭྔΛ༻͍Δͷ͕Ұൠత 𝑦% ͷϞσϧʹΑΔࠩฏํ𝑆𝑆𝑅0 𝑦% ,
𝑦" ͷϞσϧʹΑΔࠩฏํ𝑆𝑆𝑅% '౷ܭྔͷࢉग़𝐹 ≡ (UU]% +UU]&)/8 UU]&/(W +'P +%) '౷ܭྔͷʹج͍ͮͯάϨϯδϟʔҼՌ ͕ଘࡏ͢Δ͔Ͳ͏͔Λݕఆ -
71 ࣌ܥྻϞσϧ l ඇఆৗ࣌ܥྻΛϑΟοςΟϯά͢ΔͨΊͷϞσϦϯάख๏ͷ̍ͭ ہॴఆৗ"3Ϟσϧ ϨδʔϜׂ Matsubara, Yasuko, Yasushi Sakurai,
and Christos Faloutsos. "Autoplait: Automatic mining of co-evolving time sequences." Proceedings of the 2014 ACM SIGMOD international conference on Management of data. 2014. -
74 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$# 𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* 𝑛࣍ݩ࣌ܥྻ 𝑥* 𝑚࣍ݩঢ়ଶϕΫτϧ 𝑣* 𝑘࣍ݩγεςϜϊΠζ 𝑤* 𝑛࣍ݩ؍ଌϊΠζ 𝑣* ~ 𝑁 0, 𝑄* , 𝑤* ~ 𝑁 0, 𝑅* -
75 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#
𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% γεςϜϞσϧ ؍ଌϞσϧ -
76 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#
𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% 𝐹! 𝐺! 𝐻! 𝑥! -
77 ࣌ܥྻϞσϧ "3ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣!$#
𝑦! = 𝐻! 𝑥! + 𝑤! γεςϜϞσϧ ؍ଌϞσϧ 𝑦* = ) !/% P 𝑎! 𝑦*+! + 𝜀* 𝜀* ~ N(0, 𝜎") "3Ϟσϧ 𝑦! 𝑦!-" ⋮ 𝑦!-/+" = 𝑎4 𝑎0 1 ⋯ 𝑎5 ⋮ ⋱ ⋮ ⋯ 1 𝑦!-" 𝑦!-Z ⋮ 𝑦!-/ + 1 0 ⋮ 0 𝜀! 𝑦* = 1 0 … 0 𝑦* 𝑦*+% ⋮ 𝑦*+P(% 𝐹! 𝐺! 𝐻! 𝑥! "3Ϟσϧ࣌ෆมͰ؍ଌϊΠζ 𝑤! = 0ͷ ঢ়ଶۭؒϞσϧͰදݱՄೳ -
78 ࣌ܥྻϞσϧ "3."ϞσϧΛঢ়ଶۭؒϞσϧͰදݱ 𝜀!~ N(0, 𝜎Z) "3."Ϟσϧ 𝑦! _ 𝑦!+"|!-"
⋮ _ 𝑦!+,-"|!-" = 𝑎" 1 𝑎Z ⋯ ⋮ ⋱ 1 𝑎, ⋯ 𝑦!-" _ 𝑦!|!-Z ⋮ _ 𝑦!+,-Z|!-Z + 1 𝑏" ⋮ 𝑏,-" 𝜀! 𝑦* = 1 0 … 0 𝑦* m 𝑦*(%|*+% ⋮ m 𝑦*(&+%|*+% 𝐹! 𝐺! 𝐻! 𝑥! 𝑦! = b \]" / 𝑎\𝑦!-\ + b \]" ^ 𝑏\𝜀!-\ + 𝜀! _ 𝑦!+X|!-" = b \]X+" / 𝑎\ 𝑦!+X-\ + b \]X ^ 𝑏\ 𝜀!+X-\ 𝑘 = max(𝑝, 𝑞 + 1) 𝑥!-" -
79 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) -
80 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 < 𝑡ͷ߹ ༧ଌ -
81 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 = 𝑡ͷ߹ ϑΟϧλ -
82 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 > 𝑡ͷ߹ ฏԽ -
83 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) ࣌ܥྻϞσϧͷܭࢉظ༧ଌɺܽଛͷิ ͳͲͷλεΫ͕͜ͷϑϨʔϜϫʔΫͰ࣮ݱͰ͖Δ -
84 ࣌ܥྻϞσϧ ઢܗɾΨεܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) -
85 ࣌ܥྻϞσϧ ઢܗɾΨεܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) ΧϧϚϯϑΟϧλΛར༻ -
86 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! -
87 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ͭલͷঢ়ଶ 𝑡 − 1 ͔Βݱࡏͷঢ়ଶ 𝑡 Λਪఆ 𝑥!|!$# = 𝔼 𝑥! 𝑦!$# ] = 𝔼 𝐹𝑥!$# + 𝐺! 𝑣! 𝑦!$# ] = 𝐹𝔼 𝑥!$# 𝑦!$# ] = 𝐹𝑥!$#|!$# -
88 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ༧ଌ 𝑥!|!$# = 𝐹! 𝑥!$#|!$# 𝑉!|!$# = 𝐹! 𝑉!$#|!$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ͭલͷঢ়ଶޡࠩ ͔Βݱࡏͷঢ়ଶޡࠩΛਪఆ 𝑉!|!$# = 𝔼[ 𝑥!$ 𝑥!|!$# ' (𝑥!$ 𝑥!|!$# )] = 𝔼[(𝐹 𝑥!$#$ 𝑥!$#|!$# + 𝐺𝑣! )' (𝐹 𝑥!$#$ 𝑥!$#|!$# + 𝐺𝑣! )] = 𝐹𝔼 𝑥!$#$ 𝑥!$#|!$# ' 𝑥!$#$ 𝑥!$#|!$# 𝐹' + 𝐺𝔼 𝑣! '𝑣! 𝐺 -
89 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) -
90 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) 𝐾* ΧϧϚϯήΠϯͱݺΕΔͷΛࢉग़ 𝑤* 𝑛࣍ݩ؍ଌϊΠζ 𝑤* ~ 𝑁 0, 𝑅* -
91 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ਪఆޡࠩͱ؍ଌϊΠζ 𝑹𝒕 ͷʹ͋ͨΔ෦ ΧϧϚϯήΠϯશମͷޡࠩʹ͓͚Δਪఆޡࠩͷׂ ߹ʹ૬ -
92 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ؍ଌͱࣄલʹਪఆ͞Εͨঢ়ଶͱͷࠩ -
93 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ΧϧϚϯήΠϯॏΈͷׂͰ؍ଌޡࠩͱਪఆޡࠩ ͷόϥϯεΛऔΓঢ়ଶ 𝑥* Λߋ৽ -
94 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ϑΟϧλ 𝑥!$# K! = 𝑉!|!$# 𝐻! ((𝐻! 𝑉!|!$# 𝐻! ( + 𝑅! )$# 𝑉!|! = (𝐼 − 𝐾! 𝐻! )𝑉!|!$# 𝐻! 𝑥!|! = 𝑥!|!$# + 𝐾! (𝑦! − 𝐻! 𝑥!|!$# ) ؍ଌޡ͕ࠩେ͖͍ͱڞࢄ͕େ͖͘ͳΔΑ͏ʹߋ৽ -
96 ࣌ܥྻϞσϧ ઢܗɾΨεܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) ΧϧϚϯϑΟϧλΛར༻ -
97 ࣌ܥྻϞσϧ ظ༧ଌΛ͍ͨ͠߹ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺! 𝑝
𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) l ظ༧ଌ 𝑥!")|!$# = 𝐹! 𝑥!$#")|!$# 𝑉!")|!$# = 𝐹! 𝑉!")|!")$# 𝐹! ( + 𝐺! 𝑄! 𝐺! ( 𝑥!$# 𝐻! ϑΟϧλͷॲཧΛলུ͢Δ͜ͱͰ࣮ݱ -
99 ࣌ܥྻϞσϧ ঢ়ଶۭؒϞσϧͷਪఆ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑦% 𝑥% 𝑣% ʜ σʔλʹج͍ͮͯঢ়ଶ 𝑥* Λਪఆ͍ͨ͠ 𝑝(𝑥! |𝑦& ) 𝑗 > 𝑡ͷ߹ ฏԽ -
100 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏԽ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!
𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏԽ 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $# -
101 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏԽ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!
𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏԽ 3BVDI5VOH4USJFCFM TNPPUIFS 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $# 𝐴* ฏۉԽརಘ -
102 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ ฏԽ 𝑦! 𝑥! 𝑣! 𝑤! 𝐹! 𝐺!
𝑝 𝑥! 𝑦& ~ N(𝑥!|& , 𝑉!|& ) 𝑥!$# 𝐻! l ฏԽ 3BVDI5VOH4USJFCFM TNPPUIFS 𝑥!|% = 𝑥!|! + 𝐴! (𝑥!"#|( − 𝑥!"#|! ) 𝑉!|( = 𝑉!|! + 𝐴! (𝑉!"#|( − 𝑉!"#|! )𝐴! ( 𝐴! = 𝑉!|! 𝐹!"# ( 𝑉!"#|! $# ฏۉԽརಘ͕ঢ়ଶߋ৽ʹ͓͚ΔॏΈͷׂ -
104 ࣌ܥྻϞσϧ ඇઢܗɾඇΨεܕঢ়ଶۭؒϞσϧ l ઢܗɾΨεܕঢ়ଶۭؒϞσϧ ͰରॲͰ͖ͳ͍࣌ܥྻͷ ଟ͘ଘࡏ l ߏมԽ l
ҟৗ l ඇରশ l ඇઢܗੑ -
105 ࣌ܥྻϞσϧ ඇઢܗɾඇΨεܕঢ়ଶۭؒϞσϧ 𝑦! 𝑦!"# 𝑦!$# 𝑥!$# 𝑥! 𝑥!"# 𝑣!"#
𝑣! 𝑣!$# 𝑤!$# 𝑤! 𝑤!"# 𝐹! 𝐺! 𝐻! 𝑥! = 𝐹! 𝑥!$# + 𝐺! 𝑣! 𝑦! = 𝐻! 𝑥! + 𝑤! ઢܗɾΨεܕ ඇઢܗɾඇΨεܕ 𝑥! = 𝑓(𝑥!$# , 𝑣! ) 𝑦! = ℎ(𝑥! ) + 𝑤! 𝑣! ~𝑞 𝑣 , 𝑤! ~𝑟(𝑤) -
106 ࣌ܥྻϞσϧ ΧϧϚϯϑΟϧλ l ༧ଌ l ϑΟϧλ l ฏԽ 𝑝
𝑥! 𝑌!$# ) = ; $* * 𝑝 𝑥! 𝑥!$# )𝑝 𝑥!$# 𝑌!$# )𝑑𝑥!$# 𝑥! = 𝑓(𝑥!$# , 𝑣! ) 𝑦! = ℎ(𝑥! ) + 𝑤! 𝑝 𝑥! 𝑌! ) = 𝑝 𝑦! 𝑥! )𝑝 𝑥! 𝑌!$# ) 𝑝 𝑦! 𝑌!–# ) 𝑝 𝑥! 𝑌( ) = 𝑝 𝑥! 𝑌! ) ∫ $* * , -!"# -!), -!"# /$) , -!"# /!) 𝑑𝑥!"# -
108 ࣌ܥྻϞσϧ ཻࢠϑΟϧλ from: https://www.ieice.org/jpn/books/kaishikiji/2005/200512.pdf l ඇઢܗɾඇΨεঢ়ଶۭؒϞσϧͰੵܭࢉ͕ղੳతʹࠔͰ ͋Δ͜ͱ͔Β༻͍ΒΕΔख๏ l ղੳతʹදݱͰ͖ͳ͍ಉ࣌Λʮཻࢠʯ
ʹج͍ͮͨཚϕΫτϧΛੜɻ ੜͨ͠ཚϕΫτϧʹجͮ͘ܦݧ ؔͰٻΊ͍ͨঢ়ଶۭؒͷΛۙࣅɻ -
109 ࣌ܥྻϞσϧ Ϛϧίϑ࿈ϞϯςΧϧϩ๏ .$.$ from: https://www.ieice.org/jpn/books/kaishikiji/2005/200512.pdf l ཚൃੜΞϧΰϦζϜͷҰछͰɺੜ͞ΕͨཚΛ༻͍ͯۙࣅܭ ࢉΛߦ͏ɻ l
ࣄޙͷܗͦͷͷΛਪఆ l ঢ়ଶਪఆ͚ͩͰͳ͘ɺϞσϧͷύϥϝλʔಉ࣌ʹਪఆ͢Δ -
114 Ϟσϧͷਪఆɾબ ओͳϞσϦϯάϑϨʔϜϫʔΫ モデル σʔλ ܦݧతࣝ ཧ ࣌ܥྻ༧ଌ ࣌ܥྻྨ ҟৗݕ
ͲͷΑ͏ͳϞσϧΛֶश͠࠷ద ͳϞσϧΛબ͢ΕΑ͍͔ʁ -
116 Ϟσϧͷਪఆɾબ l ߏங͞ΕͨؔϞσϧ𝑓(ɾ)ͱ؍ଌͱͷؒͷೋޡࠩ ࠩ𝑒! ͷೋ ͕࠷খͱͳΔΑ͏ͳύϥϝʔλΛਪఆ l ֬ͷ֓೦͕ݱΕͳ͍
l ճؼੳͳͲʹ͓͍ͯɺղੳతʹύϥϝʔλಋग़͕Մೳ Ϟσϧͷਪఆ࠷খೋ๏ 0-4 𝐸 = b \]" _ (𝑦\ − 𝑓(𝑥\))Z -
117 Ϟσϧͷਪఆɾબ l ྫճؼϞσϧ "3 l ͭͷΛ࣋ͭճؼϞσϧ l ೋޡࠩ
Ϟσϧͷਪఆ࠷খೋ๏ 0-4 𝐰𝐓 = (𝑤0 , 𝑤% ), 𝐱𝐢 b = (1, 𝑥)ͱఆٛ͢Δͱ 𝑓 𝑥! = v 𝑦! = 𝑤0 + 𝑤% 𝑥! = 𝐰𝐓𝐱 v 𝑦% v 𝑦c = 1, 𝑥% 1, 𝑥d 𝑤0 𝑤% v 𝐲 = 𝐗𝐰 ܭըߦྻ E = (𝐲 − 𝐗𝐰)b(𝐲 − 𝐗𝐰) -
118 Ϟσϧͷਪఆɾબ l ྫճؼϞσϧ "3 l ೋޡࠩEΛ𝐰Ͱඍ Ϟσϧͷਪఆ࠷খೋ๏ 0-4
E = 𝐲 − 𝐗𝐰 b 𝐲 − 𝐗𝐰 = 𝐲𝐓𝐲 − 𝟐𝐰𝐓 𝐗𝐓𝐲 + 𝐰𝐓𝐗𝐓𝐗𝐰 𝜕𝐸 𝜕𝐰 = −𝟐𝐗b𝐲 + 𝐗b𝐗 + 𝐗b𝐗 b 𝐰 = −2𝐗b𝐲 + 𝟐𝐗b𝐗w = 0 w = (𝐗b𝐗)+%𝐗b𝐲 -
119 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ,VMMCBDL-FJCMFS ,- ใྔ l ͭͷ֬ͷࠩҟΛଌఆ͢Δई l
ਪఆ͞ΕΔ༧ଌ͕ਅͷΛଊ͍͑ͯΔ͔ΛධՁ l g(x)͕ਅͷ f(x)͕Ϟσϧͷͱͨ࣌͠ɺ,-ใྔҎԼͷܗ l D`a (g| 𝑓 = 0ͷ࣌ɺ g x = f(x) l ࡾ֯ෆࣜରশੑͳͲͷڑͷެཧຬͨ͞ͳ͍ Def (g||𝑓) = ~ +T T g x log g x f x 𝑑𝑥 ∗ Def (g||𝑓) ≥ 0 -
120 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ਖ਼نಉ࢜ͷ,-ڑ D`a(g||𝑓) = m -b b
g x log g x 𝑑𝑥 − m -b b g x log 𝑓 x 𝑑𝑥 g x ~ N 𝜇c , 𝜎c Z , f x ~ N 𝜇d , 𝜎d Z m -b b g x log 𝑓 x 𝑑𝑥 = − 1 2 log2π𝜎d Z − 𝜎c Z + 𝜇c − 𝜇d Z 2𝜎d Z m -b b g x log g x 𝑑𝑥 = − 1 2 log2π𝜎c Z − 1 2 正規分布の確率密度関数 -
121 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ Def (g||𝑓) = ~ +T T g
x log g x f x 𝑑𝑥 ∗ Def (g||𝑓) ≥ 0 from: https://qiita.com/ceptree/items/9a473b5163d5655420e8 -
122 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ϞσϦϯάʹ͓͚ΔKLใྔͷਪఆ l g(x)͕ਅͷ f(x)͕Ϟσϧͷ Def (g||f)
= ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 -
123 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ϞσϦϯάʹ͓͚ΔKLใྔͷਪఆ l g(x)͕ਅͷ f(x)͕Ϟσϧͷ Def (g||f)
= ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍ -
124 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ϞσϦϯάʹ͓͚ΔKLใྔͷਪఆ l g(x)͕ਅͷ f(x)͕Ϟσϧͷ Def (g||f)
= ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍ ฏۉର g(x)͕Θ͔Βͳ͍ͨΊະ ͕ͩɺͲͷ f(x) ͰҰఆ -
125 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ϞσϦϯάʹ͓͚ΔKLใྔͷਪఆ l g(x)͕ਅͷ f(x)͕Ϟσϧͷ Def (g||f)
= ~ +T T g x log g x f x 𝑑𝑥 = ~ +T T g x log g x 𝑑𝑥 − ~ +T T g x log f x 𝑑𝑥 Θ͔Βͳ͍ ฏۉର g(x)͕Θ͔Βͳ͍ͨΊະ ͕ͩɺͲͷ f(x) ͰҰఆ ૬ରతʹɺϞσϧ 𝐟(𝐱)ͷ ༗ޮੑʹ͍ͭͯධՁͰ͖Δʁ -
126 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ ~ +T T g x log f
x 𝑑𝑥 l ฏۉର l ର ະ 1 𝑛 b \]" W 𝐼(𝑥\) େͷ๏ଇʹΑΓɺ༩͑ΒΕͨ αϯϓϧσʔλͰۙࣅ ਅͷ αϯϓϧσʔλ ͷ 𝜃+ 𝜃678 ฏۉର ର ℓ = ) !/% ' log f x -
127 Ϟσϧͷਪఆɾબ Ϟσϧͷਪఆ࠷ਪఆ l ࠷ਪఆ .-& l ରΛ༻͍ͨύϥϝʔλਪఆ๏ l ରΛ࠷େԽ͢Δύϥϝʔλ
࠷ਪఆ 𝜃fgh ͕ۙࣅతʹ,-ใྔ Λ࠷࠷খԽ͢ΔύϥϝʔλʹͳΔͱਪఆͰ͖Δ l ରΛ࠷େԽ͢Δύϥϝʔλ ࠷ਪఆ 𝜃fgh ؔℓ(𝜃)͕𝜃Ͱඍ ͯ͠ ij(k) ik = 0ͱͳΔͷΛਪఆ ਅͷ αϯϓϧσʔλ ͷ 𝜃+ 𝜃678 ฏۉର ର ℓ(𝜃) = ) !/% ' log f x|θ 𝜃1gh = max i ℓ(𝜃) -
128 Ϟσϧͷਪఆɾબ ࠷খೋ๏ͱ࠷ਪఆ l ྫɿճؼϞσϧͷ࠷ਪఆɻޡࠩΛਖ਼نͱԾఆ v 𝑦! = 𝑓 𝑥!
= 𝑁(𝑤0 + 𝑤% 𝑥! , 𝜎") ℓ(𝜃) = ) !/% ' log 1 2𝜋𝜎" exp(− 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) = − 𝑛 2 log 2𝜋 − 𝑛 2 log 𝜎" − ) !/% ' ( 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) -
129 Ϟσϧͷਪఆɾબ l ߏங͞ΕͨؔϞσϧ𝑓(ɾ)ͱ؍ଌͱͷؒͷೋޡࠩ ࠩ𝑒! ͷೋ ͕࠷খͱͳΔΑ͏ͳύϥϝʔλΛਪఆ l ֬ͷ֓೦͕ݱΕͳ͍
l ճؼੳͳͲʹ͓͍ͯɺղੳతʹύϥϝʔλಋग़͕Մೳ ࠷খೋ๏ͱ࠷ਪఆ 𝐸 = b \]" _ (𝑦\ − 𝑓(𝑥\))Z -
130 Ϟσϧͷਪఆɾબ ࠷খೋ๏ͱ࠷ਪఆ l ྫɿճؼϞσϧͷ࠷ਪఆɻޡࠩΛਖ਼نͱԾఆ v 𝑦! = 𝑓 𝑥!
= 𝑁(𝑤0 + 𝑤% 𝑥! , 𝜎") ℓ(𝜃) = ) !/% ' log 1 2𝜋𝜎" exp(− 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) = − 𝑛 2 log 2𝜋 − 𝑛 2 log 𝜎" − ) !/% ' ( 𝑦! − (𝑤0 + 𝑤% 𝑥! ) " 2𝜎" ) ରͷ࠷େԽ࠷খೋ๏ͱҰக 㱺 ࠷খೋ๏࠷ਪఆͷಛघྫͱҰக -
131 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ 𝜃+ "3 "3*."
." ͲͷϞσϧΛ͏ͱ ྑ͍ʁʁʁ -
132 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ 𝜃+ "3 "3*."
." ͲͷϞσϧΛ͏ͱ ྑ͍ʁʁʁ ର͕࠷େ͖͍ Ϟσϧ͕ྑ͍ͷͰʁ -
135 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ l ࠷ྑ͍ରΛ࣋ͭϞσϧΛબͨ͠߹ l աద߹ PWFSGJU ͷ l
༩͑ΒΕͨσʔλेઆ໌Ͱ͖ͨͱͯ͠ɺ৽͍͠σʔλΛेʹઆ໌Ͱ ͖ΔͱݶΒͳ͍ from: https://datascience.foundation/sciencewhitepaper/u nderfitting-and-overfitting-in-machine-learning -
136 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ l ࠷ྑ͍ରΛ࣋ͭϞσϧΛબͨ͠߹ l աద߹ PWFSGJU ͷ l
༩͑ΒΕͨσʔλेઆ໌Ͱ͖ͨͱͯ͠ɺ৽͍͠σʔλΛेʹઆ໌Ͱ ͖ΔͱݶΒͳ͍ from: https://datascience.foundation/sciencewhitepaper/u nderfitting-and-overfitting-in-machine-learning ෳࡶ͗͢ͳ͍࠷దͳϞσϧΛ બ͢Δ͜ͱ͕ॏཁ -
137 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ l దͳϞσϧબͷͨΊʹ l ରʹΑͬͯਪఆ͞ΕΔ𝜃fgh ͱ ਅͷύϥϝʔλ 𝜃.
ͷؒʹόΠΞε ͕ଘࡏ l ରͱฏۉରͷόΠΞε ΛධՁͯ͠దʹิਖ਼͢Δ͜ͱͰ ࠷దͳϞσϧΛબ͢Δඞཁ -
138 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ l దͳϞσϧબͷͨΊʹ l 𝜃fgh ʹ͓͚Δରͱฏۉର ͷࠩ 𝐷Λิਖ਼͍ͨ͠
1 𝑛 ? '"* , log f x|𝜃-./ ͜ͷ෦Λ ิਖ਼͍ͨ͠ʂ ظΛͱΔ 𝔼0[𝐷] = t #$ $ g x [D]𝑑𝑥 ฏۉର ର 𝐷 = 1 𝑛 ? '"* , log f x|𝜃-./ − t #$ $ g x log f x|𝜃-./ 𝑑𝑥 -
139 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ l దͳϞσϧબͷͨΊʹ l 𝜃fgh ʹ͓͚Δରͱฏۉର ͷࠩ 𝐷Λิਖ਼͍ͨ͠
1 𝑛 ? '"* , log f x|𝜃-./ ͜ͷ෦Λ ิਖ਼͍ͨ͠ʂ ฏۉର ର ظΛͱΔ 𝔼0[𝐷] = t #$ $ g x [D]𝑑𝑥 ͜ͷิਖ਼෦͕ύϥϝʔλͰ͋Δ 𝑝ʹۙࣅͰ͖Δ ิਖ਼ 𝐷 = 1 𝑛 ? '"* , log f x|𝜃-./ − t #$ $ g x log f x|𝜃-./ 𝑑𝑥 -
140 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ"*$ l దͳϞσϧબͷͨΊʹ l ࠷େର όΠΞε 𝜃fgh ʹ͓͚Δฏۉର
1 𝑛 ? '"* , log f x|𝜃-./ ∑!/% ' log f x|𝜃1gh − 𝔼x [𝐷] = 𝑛 ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 ∑!/% ' log f x|𝜃1gh − 𝑝= n ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 𝔼0[𝐷] -
141 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ"*$ l ใྔج४ "*$ l ฏۉରͷ 𝜃fgh ʹ͓͚ΔਪఆྔͰ͋ΔҎԼͷࣜͷࠨลΛº͢Δ͜ͱͰ
౷ܭϞσϧͷࢦඪͱͳΔ l 𝑝ࣗ༝ύϥϝʔλ l ͜ͷࢦඪ͕খ͍͞౷ܭϞσϧ͕ྑ͍Ϟσϧͱਪఆ͞ΕΔ l ใྔج४ͷछ ∑!/% ' log f x|𝜃1gh − 𝑝= n ∫ +T T g x log f x|𝜃1gh 𝑑𝑥 AIC = −2 ) !/% ' log f x|𝜃1gh + 2𝑝 -
147 Ϟσϧͷਪఆɾબ Ϟσϧͷબఆ"*$ l Ϟσϧͷઆ໌ม͕গͳ͍ͱɺ#JBT͕େ͖͘ͳΔ ֶशෆ l Ϟσϧͷઆ໌ม͕ଟ͍ͱɺ#JBTখ͘͞ͳΔ͕ɺ7BSJBODF͕େ ͖͘ͳΓɺաֶशͱͳΔ l
"*$ͳͲͷใྔج४ɺόΠΞεͱόϦΞϯεͷͭΛόϥϯε Α͘࠷খʹ͢Δʢظ༧ଌޡࠩͷ࠷খʣͷͨΊͷࢦඪ ༧ଌޡࠩ #JBT 7BSJBODF -
149 Ϟσϧͷਪఆɾબ Ϟσϧͷબଞͷબํ๏ l ใྔج४ *$ l Ϟσϧͷͯ·Γͷྑ͞ͱɺϞσϧͷෳࡶ͞ʹର͢ΔϖφϧςΟ͔Βߏ ͞ΕΔࢦඪͷҰൠܥ l
ରʹج͍ͮͨؔͱϖφϧςΟؔ𝐼(𝑛)ʹΑͬͯߏ l ϖφϧςΟؔΛͲͷΑ͏ʹઃܭ͢Δ͔Ͱෳͷࢦඪ͕ଘࡏ l ใྔج४͕খ͍͞Ϟσϧ͕ྑ͍Ϟσϧͷࢦඪ l "*$𝐼 𝑛 = 2 ͷϞσϧ IC = −2 ) !/% ' log f x|𝜃1gh + 𝐼(𝑛)𝑝 -
150 Ϟσϧͷਪఆɾબ Ϟσϧͷબଞͷબํ๏ l ҳใྔج४ %*$ l ใྔج४ͷछ l ରͷࣄޙฏۉΛਪఆྔͱͯ͠ར༻͢ΔϕΠζϕʔεͷࢦඪ
l ϖφϧςΟؔࣄલʹΑΔରͱࣄޙฏۉͷରͷࠩ 𝐷IC = −2 ~ log f x|𝜃y1 𝑑𝜃y1 + 𝐼 ・ 2 𝐼 ・ =2 ∫ log f x|𝜃y1 𝑑𝜃y1 − 2 ∫ log f x|𝜃 ∏(𝜃|𝑥) 𝑑𝜃 𝜃y1 = ∫ 𝜃 ∏(𝜃|𝑥) 𝑑𝜃 -
151 Ϟσϧͷਪఆɾબ Ϟσϧͷબଞͷબํ๏ l ϕΠζใྔج४ #*$ l ใྔج४ͷछ l ,-ใྔʹجͮ͘ͷͰͳ͘पลʹج͍ͮͨใྔ
l 𝑛 ≥ 8ͷ࣌ʹ"*$ΑΓϖφϧςΟ͕ڧ͘ͳΔɻͭ·Γɺ"*$ΑΓύϥ ϝʔλ͕খ͍͞ϞσϧΛબ͢Δ 𝐵IC = −2 ) !/% ' log f x|𝜃1gh + 𝑝 log 𝑛 -
152 Ϟσϧͷਪఆɾબ Ϟσϧͷબଞͷબํ๏ l .JOJNVN%FTDSJQUJPO-FOHUI .%- l σʔλΛϞσϧΛ༻͍ͯѹॖ͢Δࡍͷූ߸ͷ࠷খԽʹج͍ͮͨࢦඪ l ใྔج४ਅͷͷڑΛߟྀ͢Δ͕ɺ.%-σʔλѹॖʹ͍ͭͯ
ߟ͑Δ l σʔλͷద߹ ର ͱϞσϧίετͷͭͰߏ l 𝐵͕ූ߸ʹ͋ͨΔ MDL = − ) !/% ' log f x|𝜃1gh + 𝐵𝑝 -
159 ਂֶशͷൃల ਂֶश l ਂֶशਂ͍ߏΛ࣋ͭχϡʔϥϧωοτϫʔΫ l χϡʔϥϧωοτϫʔΫ /FVSBM/FUXPSL ͷߏɾػೳͷ฿͔ Βൃలͨ͠ϞσϧͰɺਓχϡʔϩϯΛΈ߹ΘͤͨωοτϫʔΫ
l Ϟσϧؔͷू߹ ⼊⼒ 出⼒ 𝑓% 𝑓" 𝑓V ඍՄೳͳؔ -
160 ਂֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X
= A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% ؔ 𝑓% ͷྫ 𝐴[%] = 𝑓% 𝑥 = 𝑊𝑥 + 𝑏 ઢܗϞσϧ -
161 ਂֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X
= A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% ؔ 𝑓% ͷྫ 𝐴[%] = 𝑓% 𝑥 = 𝑊𝑥 + 𝑏 ΞϑΟϯม 0.6 0.2 0.8 −0.3 −0.3 0.4 0.1 0.7 ॏΈΛσʔλʹ ߹͏Α͏ʹֶश -
162 ਂֶशͷൃల ϑΟʔυϑΥϫʔυχϡʔϥϧωοτϫʔΫ Input Layer Hidden Layers Output Layer X
= A[0] a[4] A[1] A[3] X Ŷ a[1] n a[2] 1 a[2] 2 a[2] 3 a[2] n a[3] 1 a[3] 2 a[3] 3 a[3] n A[2] A[4] 𝑓% 𝐴[%] = 𝑚𝑎𝑥{𝑥, 0} ׆ੑԽؔ ɾ3F-6 𝐴[%] = 1/(1 + 𝑒+#) ɾ4JHNPJE -
163 ਂֶशͷൃల ਂϞσϧͷֶश ⼊⼒ 出⼒Ŷ 𝑓% 𝑓" 𝑓V l ଛࣦؔ
L 𝑊 = 4 !/% ' ℓ(𝑌! , 𝐹(𝑥! )) ೋଛࣦ NFBOTRVBSFEFSSPS ℓ 𝑦, 𝑦€ = (𝑦 − 𝑦€)" ަࠩΤϯτϩϐʔޡࠩ $SPTT&OUSPQZMPTT ℓ 𝑦, 𝑦€ = − 4 &/% • 𝑦& log 𝑦& € ͜ΕΛ࠷খԽ͢ΔΑ͏ʹֶश min l L 𝑊 -
164 ਂֶशͷൃల ਂϞσϧͷֶश l ޡࠩٯ๏ l ύϥϝʔλߋ৽ͷख๏ l ଛࣦؔͷඍΛޙΖ ೖྗํ
ʹͲΜͲΜ࿈Λ༻͍ͯൖͤ͞Δ 𝜕𝐿 𝜕𝑤% 𝑥 = 𝜕𝐿 𝜕𝑓V 𝜕𝑓V 𝜕𝑓" 𝜕𝑓" 𝜕𝑓% 𝜕𝑓% 𝜕𝑤% 𝑥 ⼊⼒ 𝑓% 𝑓" 𝑓V 出⼒Ŷ L 𝑊 -
165 ਂֶशͷൃల ਂϞσϧͷֶश l ޯ߱Լ๏ l ଛࣦ͕ؔ࠷খͱͳΔΑ͏ʹൖ͞ΕͨޡࠩʹԠͯ͡ɺύϥϝʔλΛগ͠ ͣͭߋ৽͍ͯ͘͠ख๏ l ֤ύϥϝʔλͷଛࣦؔʹର͢Δภඍ
Λߦ͍ύϥϝʔλΛߋ৽ 𝑊* = 𝑊*+% − 𝜂 A 𝜕𝐿 𝜕𝑊 ‚/‚"'& From: https://axa.biopapyrus.jp/deep-learning/gradient_descent_method.htm -
167 ਂֶशͷൃల ਂֶशͷಛ l ສೳۙࣅೳྗ l χϡʔϥϧωοτϫʔΫेͳͷύϥϝʔλ͕͋Εɺ࿈ଓؔ Λҙͷਫ਼ͰۙࣅՄೳ l ཧతʹσʔλ͕ແݶʹ͋Γɼૉࢠ͕ແݶʹ͋Δχϡʔϥϧωοτ
ϫʔΫΛ༻͍ΕͲΜͳͰֶशՄೳ from: https://www.slideshare.net/trinmu/ss-226711018 -
168 ਂֶशͷൃల ਂֶशͷಛ l ଟԽͷԸܙ l χϡʔϥϧωοτϫʔΫͷදݱྗ ͷʹΑͬͯදݱྗ͕ࢦతʹ্ l ਂ͍χϡʔϥϧωοτϫʔΫͷදݱೳ
ྗΛઙ͍χϡʔϥϧωοτϫʔΫͰද ݱ͠Α͏ͱ͢Δͱେ͖ͳԣ෯ χϡʔ ϩϯ ͕ඞཁ Montufar, Guido F., et al. "On the number of linear regions of deep neural networks." Advances in neural information processing systems 27 (2014). Arora, R., Basu, A., Mianjy, P., & Mukherjee, A. (2016). Understanding deep neural networks with rectified linear units. arXiv preprint arXiv:1611.01491. -
169 ਂֶशͷൃల ਂֶशͷಛ l εέʔϦϯάଇ l 5SBOTGPSNFSͷੑೳύϥϝʔλ ɺσʔληοταΠζɺܭࢉ༧ ࢉΛมͱͨ͠γϯϓϧͳ͖ ଇʹै͏
l ͜ͷ๏ଇ͕ݴޠɾը૾ɾಈըͳͲ ͷ༷ʑͳλεΫʹద༻͞ΕΔ͜ͱ Λࣔ͢ from: Henighan, Tom, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun et al. "Scaling laws for autoregressive generative modeling." arXiv preprint arXiv:2010.14701 (2020). -
170 ਂֶशͷൃల ਂֶशͷಛ l 0WFSQBSBNFUFSJ[BUJPO l ଟ͘ͷେنϞσϧͰͷֶशͰσʔλ ύϥϝʔλ l ύϥϝʔλΛ૿͠ɺ࣮ࡍͷֶशʹ͓͍ͯաֶशੜ͡ͳ͍
l χϡʔϥϧωοτϫʔΫͰ൚Խޡࠩͱ܇࿅ޡࠩͷ͕ࠩখ͘͞ͳ ΔͨΊʁʢཧతੳ͕ߦΘΕ͍ͯΔʣ Neyshabur, Behnam, et al. "The role of over- parametrization in generalization of neural networks." 7th International Conference on Learning Representations, ICLR 2019. 2019. -
172 ࣌ܥྻσʔλͱਂֶश 'FFE'PSXBSE//ͱ࣌ܥྻ l ௨ৗͷ'FFE'PSXBE //ʹ࣌ܥྻσʔλΛ ద༻͢Δ͜ͱՄೳ͕ͩɺ࣌ؒͷྲྀΕΛଊ ͑Δ͜ͱ͕Ͱ͖ͳ͍ l ྫ͑ɺ༧ଌͷλεΫͰɺੲͷ෦ܰ
͘ѻͬͯɺ࠷ۙͷ෦ॏཁͳΑ͏ʹѻ͍ ͍ͨ l ࣌𝑡ͷֶशʹ͓͍ͯকདྷͷ𝑡 + 1ΛΘ ͣɺաڈͷ͚ͩΛ༻ֶ͍ͯश͍ͨ͠ l ೖྗ͕ݻఆͰͳ͘ՄมͰ͋Δ͜ͱͷ΄͏ ͕ɺ࣌ܥྻΛೖྗͷࡍʹخ͍͠ -
173 ࣌ܥྻσʔλͱਂֶश 3FDVSSFOU/FVSBM/FUXPSL 3// ・・・ 𝑥% 𝑥" 𝑥V 𝑥W Feature
Vecter l ӅΕঢ়ଶΛ࣋ͬͨχϡʔϥϧωοτϫʔΫ l ࣌ؒใͷ֓೦Λ࣋ͪɺ࣍ͷ࣌ؒʹӅΕঢ়ଶͷใΛ͓͘Δ RNN Block RNN Block RNN Block RNN Block -
174 ࣌ܥྻσʔλͱਂֶश 3FDVSSFOU/FVSBM/FUXPSL 3// 𝑋 Feature Vecter RNN Block from:
https://stanford.edu/~shervine/teaching/cs-230/cheatsheet- recurrent-neural-networks 𝑎o!p = 𝑔"(𝑊 qq𝑎o!-"p + 𝑊 qr𝑥o!p + 𝑏q) 𝑦o!p = 𝑔Z(𝑊 sq𝑎o!p + 𝑏s) -
175 ࣌ܥྻσʔλͱਂֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l ՄมʹରԠ l ϞσϧαΠζ͕ೖྗʹΑͬͯมԽ͠ͳ͍
l աڈͷใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠ڞ༗ l ऑΈ l ܭࢉͷ͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷใ͕͍ͬͯͳ͍Մೳੑ -
176 ࣌ܥྻσʔλͱਂֶश ଞͷ3//ܥ౷ͷϞσϧ l -POH4IPSU5FSN.FNPSZ -45. l ࣌ؒมԽ͢ΔظهԱͷͨΊͷ ϝϞϦ𝑐o!pΛ࣋ͭ l
ͭͷήʔτΛ࣋ͭ͜ͱͰաڈͷใͷ ٫ɺ৽͍͠ೖྗͷऔΓೖΕΛௐ l Γ1 3FMFWBODF(BUF l Γ( 'PSHFU(BUF l Γ2: 6QEBUF(BUF l Γ3 0VUQVU(BUF 𝑐o!p = Γt ⋆ ̃ 𝑐o!p + Γu ⋆ 𝑐o!-"p ̃ 𝑐o!p = tanh(𝑊 v Γw ⋆ 𝑎o!-"p, 𝑥o!p + 𝑏v) 𝑎o!p = Γx ⋆ 𝑐o!p from: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet- recurrent-neural-networks -
177 ࣌ܥྻσʔλͱਂֶश ଞͷ3//ܥ౷ͷϞσϧ l (BUFE3FDVSSFOU6OJU (36 l -45.ΛҰൠԽͨ͠Ϟσϧ l -45.ΑΓܰྔͱͳ͓ͬͯΓɺͭͷ
ήʔτͷΈͰߏ͠ܭࢉޮԽΛ্ l Γ1 3FMFWBODF(BUF l Γ2: 6QEBUF(BUF l -45.(36৽͍͠ใΛ֮͑ɺ աڈͷใΛదʹΕΔػߏΛ උ͑ͨϞσϧ 𝑐o!p = Γt ⋆ ̃ 𝑐o!p + (1 − Γt) ⋆ 𝑐o!-"p ̃ 𝑐o!p = tanh(𝑊 v Γw ⋆ 𝑎o!-"p, 𝑥o!p + 𝑏v) 𝑎o!p = 𝑐o!p from: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks -
178 ࣌ܥྻσʔλͱਂֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l ՄมʹରԠ l ϞσϧαΠζ͕ೖྗʹΑͬͯมԽ͠ͳ͍
l աڈͷใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠ڞ༗ l ऑΈ l ܭࢉͷ͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷใ͕͍ͬͯͳ͍Մೳੑ -
179 ࣌ܥྻσʔλͱਂֶश ํ3// #JEJSFDUJPOBM3// from: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks l ܥྻͷ࠷ॳ͔ΒͷॱൖʹՃ͑ͯɺܥྻ ͷ࠷ޙ͔Βॱʹٯ͖ͷൖɺ྆ํʹ ߦ͏ωοτϫʔΫ
l ࣌ܥྻσʔλʹద༻͞ΕΔ͜ͱଟ͘ͳ ͍͕ɺݴޠγʔΫΤϯεͳͲʹద༻͞Ε Δ͜ͱ͕ଟ͍ -
180 ࣌ܥྻσʔλͱਂֶश 3FDVSSFOU/FVSBM/FUXPSL 3// l ڧΈ l ՄมʹରԠ l ϞσϧαΠζ͕ೖྗʹΑͬͯมԽ͠ͳ͍
l աڈͷใΛߟྀͨ͠ܭࢉ͕Մೳ l ϞσϧͷॏΈ͕͕࣌ؒมԽͯ͠ڞ༗ l ऑΈ l ܭࢉͷ͞ l ݱࡏͷঢ়ଶʹରͯ͠কདྷͷೖྗΛߟྀͰ͖ͳ͍ l աڈͷใ͕͍ͬͯͳ͍Մೳੑ -
181 ࣌ܥྻσʔλͱਂֶश ޯফࣦ 7BOJTIJOHHSBEJFOU ・・・ 𝑥% 𝑥" 𝑥V 𝑥W Feature
Vecter RNN Block RNN Block RNN Block RNN Block 𝜕𝐿 𝜕𝑤% 𝑥% = 𝜕𝐿 𝜕𝑅𝑁𝑁W … 𝜕𝑅𝑁𝑁% 𝜕𝑤% 𝑥% ޡࠩٯʹඞཁͳޯ͕ඇৗʹ খ͘͞ͳΓɺֶश੍͕ޚͰ͖ͳ͘ ͳΔ -
187 ࣌ܥྻσʔλͱਂֶश %FDPEFSͷೖྗ l ֬ʹԠͯ͡ੜσʔλ͔ڭࢣσʔλͷೖྗΛมԽͤ͞Δख๏ 4DIFEVMFE4BNQMJOH from: Bengio, Samy, et
al. "Scheduled sampling for sequence prediction with recurrent neural networks." Advances in neural information processing systems 28 (2015). -
188 ࣌ܥྻσʔλͱਂֶश %FDPEFSͷೖྗ l ("/Λ༻͍ͯੜσʔλͱڭࢣσʔλͷೖྗʹΑΔੜͷࠩΛখ͞ ͘͢Δ 1SPGFTTPS4BNQMJOH from: Lamb, Alex
M., et al. "Professor forcing: A new algorithm for training recurrent networks." A neural information processing systems 29 (2016). -
192 ࣌ܥྻσʔλͱਂֶश %JMBUFE$BVTBM$POWPMVUJPO from: Oord, A. V. D., Dieleman, S.,
Zen, H., Simonyan, K., Vinyals, O., Graves, A., ... & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499. l $//Λ༻͍ͨ࣌ؒܗ Իੜ ʹର͢Δ֬తੜϞσϧͰ͋Δ 8BWF/FUͰఏҊ͞Εͨߏཁૉͷͭɻ l ೖྗ͔ΒॱʹͦΕͧΕɺɺɺݸͣͭεΩοϓ͠ͳ͕ΒΈࠐ ΈΛܭࢉɻΈࠐΈͷਂ͞ʹԠͯ͡ɺೖྗͰ͖Δܥྻ͕ࢦతʹ ૿Ճ͍ͯ͘͠ɻ l શ݁߹//ͷΑ͏ʹύϥϝʔλ͕ଟ͘ͳ͘ɺ3//ͷΑ͏ʹճؼతͳ ଓ͕ແ͍ͷͰɺֶश͕3//ΑΓ͘ͳ͍ͬͯΔ l ظͷܥྻΛೖྗͱͯ͠ޯফࣦ͕ੜ͡ͳ͍ -
193 ࣌ܥྻσʔλͱਂֶश 5SBOTGPSNFS l l"UUFOUJPOJT"MM:PVOFFEzͰఏҊ l ओʹࣗવݴޠॲཧը૾ॲཧͳͲʹ͓͍ͯߴ͍ਫ਼Λୡ͍ͯ͠ΔϞ σϧͷҰछ BERT ViT
from: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. -
195 ࣌ܥྻσʔλͱਂֶश 5SBOTGPSNFS l 4FMG"UUFOUJPOͱ'FFE'PSXBSEͰओʹߏ l &ODPEFS%FDPEFSϞσϧ l $//ͱൺେҬతͳؔΛଊ͑Δ͜ͱ͕Մೳ l
1PTJUJPOBM&ODPEJOHͱ͍͏֤ೖྗͷ ҐஔใΛϕΫτϧͰදݱ -
198 ࣌ܥྻσʔλͱਂֶश 1PTJUJPOBM&ODPEJOH l 5JNF4UBNQ QPTJUJPOBMFNCFEEJOH l ࣌ܥྻσʔλΛೖྗͷࡍʹ݄࣌ؒͳ Ͳͷใ͕͋Εར༻Ͱ͖Δ&NCFEEJOH ͷҰछ
l *OGPSNFS"VUPGPSNFSͳͲͷ࠷৽ͷ 5SBOTGPSNFSϕʔεͷ࣌ܥྻ༧ଌϞσϧ ʹ༻͍ΒΕ͍ͯΔ -
199 ࣌ܥྻσʔλͱਂֶश 5SBOTGPSNFS Transformer Block l 4FMG"UUFOUJPOͱ'FFE'PSXBSEͰओʹߏ l &ODPEFS%FDPEFSϞσϧ l
$//ͱൺେҬతͳؔΛଊ͑Δ͜ͱ͕Մೳ l 1PTJUJPOBM&ODPEJOHͱ͍͏֤ೖྗͷ ҐஔใΛϕΫτϧͰදݱ l &ODPEFS %FDPEFSͱʹ .VMUJIFBEBUUFOUJPOͱ'FFE'PSXBSEͰ ߏ͞Εͨ5SBOTGPSNFS#MPDLͷੵΈॏͶ -
200 5SBOTGPSNFS#MPDL ࣌ܥྻσʔλͱਂֶश • &ODPEFSɼ%FDPEFSͱͱʹ5SBOTGPSNFS #MPDLͷੵΈॏͶͰߏ • 5SBOTGPSNFS#MPDLͷߏ .VMUJ)FBE4FMGBUUFOUJPO 3FTJEVBM$POOFDUJPO
ࠩଓ -BZFS/PSNBMJ[BUJPO 1PTJUJPOXJTF'FFE'PSXBSE %SPQPVU Transformer Block -
201 4FMGBUUFOUJPO ࣌ܥྻσʔλͱਂֶश ⼊⼒系列の 潜在表現 系列⻑ × 次元数 Key :
K Query: Q 𝑊y 𝑊z Value: V Attention Map : M 系列⻑ × 系列⻑ 𝑊{ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( 𝑄𝐾# 𝑑 ) -
202 4FMGBUUFOUJPO ࣌ܥྻσʔλͱਂֶश Value: V Attention Map : M 系列⻑
× 系列⻑ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( 𝑄𝐾# 𝑑 ) ֤࣌ͷಛ͕ଞͷ࣌Λߟྀ ͯ͠࠶ߏ͞ΕΔ -
204 5SBOTGPSNFS#MPDL ࣌ܥྻσʔλͱਂֶश • &ODPEFSɼ%FDPEFSͱͱʹ5SBOTGPSNFS #MPDLͷੵΈॏͶͰߏ • 5SBOTGPSNFS#MPDLͷߏ .VMUJ)FBE4FMGBUUFOUJPO 3FTJEVBM$POOFDUJPO
ࠩଓ -BZFS/PSNBMJ[BUJPO 1PTJUJPOXJTF'FFE'PSXBSE %SPQPVU Transformer Block -
205 3FTJEVBM$POOFDUJPO ࠩଓ ࣌ܥྻσʔλͱਂֶश Residual𝑂𝑢𝑡𝑝𝑢𝑡 = 𝑥 + 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛(𝑥) l
ೖྗ𝑥Λ.VMUJ)FBE4FMG"UUFOUJPOͷग़ྗʹ Ճ͑Δ͜ͱͰɺֶशͷ҆ఆԽΛਤΔ l 3FT/FUͳͲͷϞσϧͰఏҊ͞Εͨख๏Ͱɺ ޯফࣦɾരൃΛճආͰ͖Δ -
206 5SBOTGPSNFS#MPDL ࣌ܥྻσʔλͱਂֶश • &ODPEFSɼ%FDPEFSͱͱʹ5SBOTGPSNFS #MPDLͷੵΈॏͶͰߏ • 5SBOTGPSNFS#MPDLͷߏ .VMUJ)FBE4FMGBUUFOUJPO 3FTJEVBM$POOFDUJPO
ࠩଓ -BZFS/PSNBMJ[BUJPO 1PTJUJPOXJTF'FFE'PSXBSE %SPQPVU Transformer Block -
207 -BZFS/PSNBMJ[BUJPO ࣌ܥྻσʔλͱਂֶश 𝐿𝑎𝑦𝑒𝑟𝑁𝑜𝑟𝑚 𝑥 = 𝛾 𝑉𝑎𝑟 𝑥 ∗
𝑥 − 𝜇 𝑥 + 𝛽 l ֤࣍ݩͷग़ྗΛฏۉࢄʹਖ਼نԽ͢Δख๏ l ޯফࣦɾޯരൃΛ͑Δׂ -
208 5SBOTGPSNFSº *OGMVFO[BGPSFDBTUJOH ࣌ܥྻσʔλͱਂֶश Wu, N., Green, B., Ben, X.,
& O'Banion, S. (2020). Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317. -
209 ਂֶश WT౷ܭతϞσϧ ࣌ܥྻσʔλͱਂֶश Wu, N., Green, B., Ben, X.,
& O'Banion, S. (2020). Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317. .σʔληοτͷTUFQBIFBE GPSFDBTUJOHͷਫ਼ݕূ -
213 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ /#FBUT l 'VMMZ$POOFDUFEͰߏ͞Εͨϒ ϩοΫͷੵΈ্͛ʹΑΔ࣌ܥྻ༧ଌ Ϟσϧ l ϒϩοΫؒͰࠩػߏʹΑΔଓ ͱɺϒϩοΫ෦Ͱઢܗ݁߹Ͱߏ
͞Ε͓ͯΓɺղऍੑͷߴ͍Ϟσϧ ͱͳ͍ͬͯΔʢͲͷ෦ͰԠͯ͠ ͍Δ͔͕͔Δʣ from: Oreshkin, Boris N., et al. "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting." International Conference on Learning Representations. 2019. -
214 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ঢ়ଶۭؒϞσϧ º ਂֶश4 l εςοϓҎ্ͷඇৗʹ͍γʔέϯεʹରԠͰ͖Δঢ়ଶۭؒϞσ ϧΛ׆༻ͨ͠ख๏ΛఏҊ l ঢ়ଶۭؒϞσϧΛ3FDVSSFOUͱ$POWPMVUJPOBMදݱʹஔ͖͑
l )JQQPߦྻΛ3//ʹΈࠐΉ͜ͱͰظهԱੑΛ֫ಘ from: Gu, Albert, Karan Goel, and Christopher Re. "Efficiently Modeling Long Sequences with Structured State Spaces." International Conference on Learning Representations. 2021. -
215 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ਂֶश º ग़ྗ from: Salinas, David, et al.
"DeepAR: Probabilistic forecasting with autoregressive recurrent networks." International Journal of Forecasting 36.3 (2020): 1181-1191. l %FFQ"3 l ֬తͳ༧ଌΛ࣮ݱ͢ΔͨΊʹෛͷೋ߲ؔͱϞϯςΧϧϩαϯϓ Ϧϯάͷग़ྗܗࣜͷಋೖ Training Prediction -
216 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ਂֶश º ग़ྗ from: Guen, Vincent Le, and
Nicolas Thome. "Probabilistic time series forecasting with structured shape and temporal diversity." arXiv preprint arXiv:2010.07349 (2020). l 453*1& l ΨεͳͲͷҰൠతͳ Ͱͳ͘ɺඇఆৗͳ࣌ ܥྻʹରԠͨ࣌ؒ͠తɺ ܗঢ়త؍Ͱॊೈͳ༧ ଌ͕Մೳͳ&ODPEFS %FDPEFSϞσϧͷఏҊ -
217 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ਂֶश º ग़ྗ from: de Bézenac, E., Rangapuram,
S. S., Benidis, K., Bohlke-Schneider, M., Kurle, R., Stella, L., ... & Januschowski, T. (2020). Normalizing kalman filters for multivariate time series analysis. Advances in Neural Information Processing Systems, 33, 2995-3007. l /PSNBMJ[JOH,BMNBO'JMJUFST l ઢܗΨεঢ়ଶۭؒϞσϧΛ/PSNBMJ[JOHGMPXͰิڧͯ͠ଟมྔ࣌ܥྻ ͷϞσϦϯάΛ࣮ݱ͢Δ l 3FBM/71ͷΞʔΩςΫνϟΛར༻ -
218 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ਂֶश º ग़ྗ from: https://lilianweng.github.io/posts/2018-10-13-flow-models/ l /PSNBMJ[JOH'MPX l
୯७ͳ֬ม𝑧. ʹରͯ͠ඇઢܗม𝑓\ ΛॏͶΔ͜ͱʹΑͬͯɺॊೈͳ 𝑝y(𝑍y)Λ֫ಘ͢ΔͨΊͷख๏ -
219 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ෆִؒ࣌ܥྻ from: Chen, Ricky TQ, et al. "Neural
ordinary differential equations." Advances in neural information processing systems 31 (2018). l 0%&3// l &ODPEFS%FDPEFSϞσϧʹ͓͚Δσίʔμͷજࡏมग़ྗΛ0%&/FUʹ ஔ͖͑Δ͜ͱͰෆִؒ࣌ܥྻʹରԠ -
220 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ෆִؒ࣌ܥྻ from: Chen, Ricky TQ, et al. "Neural
ordinary differential equations." Advances in neural information processing systems 31 (2018). l /FVSBM0SEJOBSZ%JGGFSFOUJBM&RVBUJPO l 3FT/FUͱৗඋํఔࣜʹྨࣅʹண͠ɺৗඍํఔࣜͷղ๏Λχϡʔ ϥϧωοτͷදݱʹ༻͍Δख๏ l ͜ΕΛ༻͍Δ͜ͱͰϝϞϦޮͷߴ͍ ࣌ؒ࿈ଓͳϞσϧΛߏங͢Δ͜ͱ͕Մೳ ͱͳΔ l જࡏදݱ0%&4PMWFSʹΑͬͯܭࢉՄೳ -
221 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l *OGPSNFS l 5SBOTGPSNFS͕ظܥྻ༧ଌʹؤ݈Ͱͳ͍͜ͱɺܭࢉޮ͕ѱ͍͜ ͱΛղܾ͢ΔͨΊʹ4FMGBUUFOUJPOͷܭࢉޮ্ͷςΫχοΫΛఏҊ from:
Zhou, Haoyi, et al. "Informer: Beyond efficient transformer for long sequence time-series forecasting." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 12. 2021. Method Complexity per Layer Convolutional 𝑂 𝐾 ~ 𝐷0 ~ 𝐿 Recurrent 𝑂 𝐿 ~ 𝐷0 Self-attention (Transformer) 𝑂 𝐿0 ~ 𝐷 K: the length of filter D: dimensionality of space L: input length N: Number of layers Computational Complexity 𝑶 𝑵×(𝑳𝟐 ‰ 𝑫) -
222 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l *OGPSNFS ̎ͭͷ$PNQMFYJUZݮख๏ΛఏҊ l 1SPC4QBSTFॏཁͷߴ͍"UUFOUJPO.BQͷΈΛར༻ 𝑂
𝐿Z ‹ 𝐷 ˠ 𝑂 𝐿 log 𝐿 ‹ 𝐷 ʹݮ l 4FMGBUUFOUJPO%JTUJMMJOH ηϧϑΞςϯγϣϯΛग़Δ ܥྻͷ͕͞ʹͳΔΑ͏ৠཹ 𝑂 N ‹ ⋯ ˠ 𝑂 2 − 𝜖 ‹ ⋯ ʹݮ -
223 1SPC4QBSTF ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ ⼊⼒系列の 潜在表現 系列⻑(L) × 次元数(D) Key :
K Query: • 𝑄 𝑊y 𝑊z Value: V Attention Map : M 系列⻑ × 系列⻑ 𝑊{ 𝑊xt! Output 𝑠𝑜𝑓𝑡𝑚𝑎𝑥( • 𝑄𝐾# 𝐷 ) 上位u件のQuery のみを利⽤ -
224 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ 5SBOTGPSNFSº ࣌ܥྻ༧ଌ l "VUPGPSNFS l 5SBOTGPSNFS෦ʹ࣌ܥྻղͷػߏΛඋ͑ͨϞσϧ from: Wu,
Haixu, et al. "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting." Advances in Neural Information Processing Systems 34 (2021): 22419-22430. -
225 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻ༧ଌ Ґஔใ º ࣌ܥྻ༧ଌ l %JGGVTJPO$POWPMVUJPOBM3// l ଟมྔ࣌ܥྻͷؔΛάϥϑͱͯ͠ଊ͑ͯ ಓ࿏ͷಓͳͲ
࣌ܥྻ༧ଌΛ ߦ͏ϞσϧΛఏҊ l (SBQI/FVSBM/FUXPSLͱ3//ͷΈ߹Θͤ from: Li, Yaguang, et al. "Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting." International Conference on Learning Representations. 2018. -
231 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻྨ 4IBQFMFU l Ϋϥεྨʹ༗ޮͳܗύλʔϯɻྨʹ༗ޮͳಛ࣌ܥྻશମͰ ͳ͘গͷ෦࣌ܥྻʹදΕΔͱ͍͏ΞΠσΞʹجͮ͘ɻ l 4IBQFMFUΛൃݟޙɺߴʹਪͰ͖Δ from: Grabocka,
Josif, et al. "Learning time-series shapelets." Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014. -
232 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻྨ *ODFQUJPO5JNF l ը૾ྨʹ༻͍ΒΕΔϑϨʔϜϫʔΫͰ͋Δ*ODFQUJPOΛ࣌ܥྻʹԠ༻ l ࣌ܥྻΛ#PUUMFOFDLͰѹॖ͠ɺ$POWPMVUJPOBMΧʔωϧͰֶश from: Ismail Fawaz,
Hassan, et al. "Inceptiontime: Finding alexnet for time series classification." Data Mining and Knowledge Discovery 34.6 (2020): 1936-1962. -
233 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻྨ 30$,&5 l ཚྻͱ࣌ܥྻσʔλΛ͔͚߹ΘͤͨಛྔΛେྔʹੜ͠ɺੜ͞ Εͨܥྻͷಛʢ࠷େͱੵ͕ਖ਼Ͱ͋ͬͨλΠϜΟϯυͷׂ߹ʣ Λ༻͍Δ͜ͱͰ࣌ܥྻྨΛֶश from: Dempster, Angus,
François Petitjean, and Geoffrey I. Webb. "ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels." Data Mining and Knowledge Discovery 34.5 (2020): 1454-1495. 畳み込み Time Window 1.23 2.34 Time Window分の特徴を⽣成 ・・・ 特徴量を作成 最⼤: 2.34 正の割合: 0.21 学習 -
234 ࣌ܥྻλεΫͱۙͷݚڀ࣌ܥྻྨ 5-PTT l ଛࣦؔΛڑֶशʹ͢Δ͜ͱͰڭࢣͳ͠Ͱ࣌ܥྻͷಛྔΛֶश l Ϟσϧࣗମ%JMBUFE$//Λར༻ Franceschi, Jean-Yves, Aymeric
Dieuleveut, and Martin Jaggi. "Unsupervised scalable representation learning for multivariate time series." Advances in neural information processing systems 32 (2019). -
238 ࣌ܥྻλεΫͱۙͷݚڀҟৗݕ &ODPEFS%FDPEFSʹΑΔѹॖ from: Li, Shu, et al. "Fair Outlier
Detection Based on Adversarial Representation Learning." Symmetry 14.2 (2022): 347. -
243 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά 8FCσʔλͷϞσϦϯά from: Murayama, Taichi, Yasuko Matsubara, and Sakurai
Yasushi. "Mining Reaction and Diffusion Dynamics in Social Activities." arXiv preprint arXiv:2208.04846 (2022). l ֦ࢄԠํఔࣜͱχϡϥʔϧωοτϫʔΫͷΈ߹Θͤ -
244 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά 4/4ؒͷϑϩʔ l ଟมྔ)BXLFTաఔΛ༻͍ͨ4/4ؒͷྲྀΕͷՄࢹԽ from: Zannettou, Savvas, et al.
"The web centipede: understanding how web communities influence each other through the lens of mainstream and alternative news sources." Proceedings of the 2017 internet measurement conference. 2017. -
245 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά 1PJOU1SPDFTT աఔ l ΠϕϯτσʔλͷϞσϦϯάʹ༻͍ΒΕΔ౷ܭతϞσϧ l Ұఆظؒͷۭؒɾ࣌ؒɾΠϕϯτͷੑ࣭ΛϞσϦϯά l 1PJTTPOաఔ
l ۚ༥ͳͲͷ֬తʹൃੜ͢ΔࣄͷϞσϦϯάʹ༻͍ΒΕΔ l )BXLFTաఔ l ࣗͷաڈͷΠϕϯτʹґଘ ͠ TFMGFYDJUJOH ͳੑ࣭Λ࣋ͪɺ 4/4ͷόʔετͳͲΛϞσϦϯά -
246 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά ਓؒͷߦಈʹ͍ͭͯͷϞσϦϯά l ਓؒͷߦಈɺӡಈɺਭɺ৯ࣄͳͲΛه͢Δ.PCJMFIFBMUI BQQMJDBUJPOΛ׆༻ͨ͠ߦಈͷϞσϦϯά l (BVTTͷࠞ߹ʹΑΔ*OUFOTJUZͰਓؒͷߦಈ)BXLFT1SPDFTT Ͱपظతͳ૬ޓґଘͷؔΛଊ͑Δ from:
Kurashima, Takeshi, Tim Althoff, and Jure Leskovec. "Modeling interdependent and periodic real-world action sequences." Proceedings of the 2018 world wide web conference. 2018. -
247 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά ਓؒͷձʹ͍ͭͯͷϞσϦϯά l ʮਓͷౖΕΔஉʯʹ͓͚ΔܶதձͷϞσϦϯά from: Guo, Fangjian, et al.
"The bayesian echo chamber: Modeling social influence via linguistic accommodation." Artificial Intelligence and Statistics. PMLR, 2015. -
248 ࣌ܥྻλεΫͱۙͷݚڀϞσϦϯά ϑΣΠΫχϡʔεʹ͍ͭͯͷϞσϦϯά l ϑΣΠΫχϡʔεͷ֦ࢄͰɺ֦ࢄλΠϛϯάͱमਖ਼λΠϛϯάͷͭͷ ظ͕ؒଘࡏͦ͠ΕͧΕόʔετݱ͕ੜ͡Δ͜ͱΛ)BXLFT1SPDFTTͰ ϞσϦϯά from: Murayama, Taichi,
et al. "Modeling the spread of fake news on Twitter." Plos one 16.4 (2021): e0250419. -
250 ͦͷଞ l খఃଇɼݯ࢛ʮใྔن४ʯ l അޱ ొ தଜ ߊʮ৽͍͠৴߸ॲཧͷڭՊॻʯ l
Ҫख ߶ ʮೖ ػցֶशʹΑΔҟৗݕʕ3ʹΑΔ࣮ફΨΠυʯ l ۙߐ ਸ ଜ ढ़Ұʮաఔͷ࣌ܥྻղੳʯ l "JMFFO/JFMTFOʮ࣮ફ ࣌ܥྻղੳ ʕ౷ܭͱػցֶशʹΑΔ༧ ଌʯ l ཧख๏ᶝ ࣌ܥྻղੳ ౦ژେֶʮཧɾσʔλαΠΤϯεڭ ҭϓϩάϥϜʯ ࢀߟॻ
hkefka385
signer
hiola
kentaroy47
drpoonamkc
dragon2
akiraasano
yatabe
adeputra13
juanmartin2026
caitiem20
keathley
csswizardry
stephenakadiri
addyosmani
soysaucechin
twada
palkan
sachag
tonyrice
badams
katarinadahlin
![166 ਂֶशͷൃల ઢܗϞσϧͱͷҧ͍ a[4] a[4] X ઢܗϞσϧ ֶश ਂֶशϞσϧ ΧʔωϧϞσϧ](https://files.speakerdeck.com/presentations/f5b83be0ef6747aa82000b3a75df7836/slide_165.jpg){kind=link}
