χϡʔϥϧίϯϐϡʔςΟϯά Ͱͷׂ w ඇઢܗ࠷దԽΞϧΰϦζϜʹ͓͚Δޡࠩۂ໘ͷධՁ w ֶशσʔλͷมಈ࣌ʹ͓͚Δ࠶ֶशͷߴԽ w ʮמΓࠐΈʯΞϧΰϦζϜʹ͓͚ΔͳॏΈͷಛఆ w ϕΠζωοτϫʔΫͷϥϓϥεۙࣅʹ͓͚Δ༧ଌ ٴͼύϥϝʔλͷܾఆɺϞσϧΤϏσϯεධՁ ʢઅʣ 1212݄16༵
∂2 E ∂w kl ∂w mn → ∂2 E ∂w ij 2 = H ij i, j ∈{1…W} 8ύϥϝʔλͷ૯ //ʹ͓͚Δϔοη ߦྻͷ֤ཁૉ H = ∂2 E ∂w 1 ∂w 1 … ∂2 E ∂w 1 ∂w W ∂2 E ∂w W ∂w 1 ∂2 E ∂w W ∂w W ⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ॏΈόΠΞεͱ͍ͬͨ ύϥϝʔλʹؔͯ͠Ұ༷ʹฒ͢ 1212݄16༵
jrjͷ֓ཁ w ϔοηߦྻ0 8? ͷܭࢉྔ͕ඞཁͩ͠ɺೋ֊ඍେม w ର֯ۙࣅ w ඇର֯ཁૉ͕ແࢹͰ͖ΔͷͰ͋Ε0 8 ͰܭࢉՄೳ w ֎ੵʹΑΔۙࣅ w ेʹֶशࡁΈͷঢ়ଶͰɺ0 8? ͰܭࢉՄೳ w ٯߦྻͷޮͷྑ͍ஞ࣍ܭࢉ͕Մೳʢˠ४χϡʔτϯ๏ʣ w ༗ݶ෯ͷࠩʹΑΔۙࣅ w ٯൖ๏Λ༻͍ͯܭࢉ͞ΕͨϔοηߦྻͷνΣοΫʹ༗ޮ w Ұ෦ٯൖެࣜΛऔΓೖΕΔ͜ͱʹΑΓ0 8? ͰܭࢉՄೳ w ϔοηߦྻͷݫີͳධՁ w ϔοηߦྻͷੵͷߴͳܭࢉ 1212݄16༵
ର֯ۙࣅ w ϔοηߦྻ͕ର֯ߦྻͩͱ͢Εɺٯߦྻݻ༗ΛٻΊΔͷ ༰қ͘ɺඇର߲֯ͷӨڹ͕গͳ͍Ͱ༗ҙٛɻ࣮ࡍܭࢉ ྔ0 8 ͰࡁΉ a j = w ji z i i ∑ ∂2 E n ∂a j 2 = h'(a j )2 w kj w k' j ∂2 E n ∂a k ∂a k' k' ∑ k ∑ + h''(a j ) w kj ∂E n ∂a k k ∑ ∂2 E n ∂w ij 2 = ∂ ∂w ij ∂E n ∂a j z i ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ = ∂2 E n ∂a j 2 z i 2 ∂2 E n ∂a j 2 = h'(a j )2 w kj ∂2 E n ∂a k 2 k ∑ + h''(a j ) w kj ∂E n ∂a k k ∑ ඇର߲͕֯ແࢹग़དྷΔͱ͢Εର߲֯ͷΈܭࢉ͢Ε͍͍ͷͰɹɹɹͷ ܭࢉྔͰࡁΈɺ֤ͷܭࢉҎԼͷΑ͏ʹ୯७Խ͞ΕΔ ∂E n ∂a j = h'(a j ) w kj ∂E n ∂a k k ∑ ΑΓɺϔοηߦྻͷ֤ ٯൖͷެࣜ ʹର͠ɺੵͷඍΛߦ͑ O(W ) ຊདྷɺ͜Εʹؔͯ͠ϔοηߦ ྻશମΛͷܭࢉͰධՁ O(W 2 ) ͨͩ͠ɺϔοηߦྻΛඞཁͱ͢Δओཁ ͳͰۃʹඇର֯Ͱ͋Δ͜ͱ͕ ଟ͘ɺద༻ʹҙ͕ඞཁɾɾɾ 1212݄16༵
w ޡ͕ࠩؔฏํͱͳΔΑ͏ͳʹ͓͍ͯɺेʹ܇࿅͞Εͨঢ়ଶͰ ɺϔοηߦྻͷҰ෦ͷ߲ແࢹग़དྷΔ͘Β͍খ͘͞ͳΓɺ0 8? ͷ ୯७ͳੵԋࢉͰධՁ͕Մೳ Η = ∇∇E = ∇y n (∇y n )T n=1 N ∑ + (y n − t n )∇∇y n n=1 N ∑ ेʹ܇࿅͞ΕͨωοτϫʔΫͰɺZ͕Uʹेۙͮ͘͜ͱʹΑΓɺୈ߲͕̎θϩʹۙ ͖ͮɺແࢹ͢Δ͜ͱ͕ग़དྷΔͷͰҎԼͷΑ͏ʹۙࣅՄೳ Η ∇y n (∇y n )T n=1 N ∑ ≡ b n b n T n=1 N ∑ -FWFOWFSH.BSRVBSEUۙࣅ ʢ֎ੵʹΑΔۙࣅʣ E = 1 2 (y n − t n )2 n=1 N ∑ ͷΑ͏ͳճؼʹର͠ɺɹɹɹɹɹɹɹɹɹɹΑΓ ∇E = (y n − t n )∇y n n=1 N ∑ 1212݄16༵
ϔοηٯߦྻͷஞ࣍ܭࢉ w ֎ੵʹΑΔۙࣅΛϕʔεʹͯ͠ɺ-ݸͷσʔλʹର͠ஞ࣍ ܭࢉ͠ɺ)Ћ*ͷٯߦྻΛঃʑʹݟ͚ͭΔ͜ͱ͕Մೳ Η L+1 = Η L + b L+1 b L+1 T 8PPECVSZ߃ࣜ Η N = b n b n T n=1 N ∑ Η L+1 −1 = Η L + b L+1 b L+1 T ( )−1 = Η L −1 + Η L −1b L+1 b L+1 T Η L −1 1+ b L+1 T Η L −1b L+1 Η 0 = b 1 b 1 T = αI (α 1) Η +αI L +1= N ʹͳΔ·Ͱ܁Γฦ͢͜ͱͰ ͷٯߦྻ͕ݟ͔ͭΔ ͱͯ͠ 1212݄16༵