統計学実践ワークブック 第16章 重回帰分析 pp.125-127

統計学実践ワークブック 第16章 重回帰分析 pp.125-127について、式を導出したり図示したり行間を埋めたりしました。

Transcript

1. ౷ܭֶ࣮ફϫʔΫϒοΫ
   ୈষॏճؼ෼ੳ
   QQ
   

   View Slide

2. ه๏
   ⃗
   1
   ⃗
   0
   X′
   
   ͸ཁૉ͕͢΂ͯͰ͋ΔॎϕΫτϧ ྻϕΫτϧ
   
   ͸ཁૉ͕͢΂ͯͰ͋ΔॎϕΫτϧ ྻϕΫτϧ
   
   ͸ߦྻ9ͷసஔߦྻ
   X
   n×m
   ͸OߦNྻͷߦྻ
   

   View Slide

3. ճؼ෼ੳͷزԿֶతͳೝࣝ
   Im(X)
   Y
   e = Y− ̂
   Y
   ̂
   Y = PX
   Y = X ̂
   β
   ℝn
   Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn
   X
   n×(1+d)
   ̂
   β
   (1+d)×1
   ̂
   Y
   n×1
   =
   , e ⊥ ̂
   Y
   :Λ*N 9
   ʹࣹӨʹͨ͠Β
   ͱͳΔɻ
   ̂
   Y
   ࢒ࠩϕΫτϧ e
   

   View Slide

4. 9ͷ૾ɺࣹӨɺ࢒ࠩ
   Y − ̂
   Y
   ͸
   Im(X) ʹࣹӨ͢Δߦྻɺ͢ͳΘࣹͪӨߦྻΛ PX ͱ͢Δͱɺ
   ̂
   Y = PX
   Y Y Λ Im(X) ΁ࣹӨͨ͠ϕΫτϧͰ͋Δɻ
   ͳͷͰɺ ̂
   Y
   ͱ ͸௚ަ͢Δɻ
   ̂
   Y
   ͱ͜ΖͰɺ ͸ Im(X) ʹଐ͢ΔϕΫτϧͰ΋͋Δ͔Βɺ
   ̂
   Y = X ̂
   β Λຬͨ͢Α͏ͳ ̂
   β ∈ ℝ1+d ͕ଘࡏ͢Δɻ
   X′
   
   e = 0
   ࢒ࠩϕΫτϧ e ʹ͍ͭͯɺ
   ͕੒Γཱͭɻ F͸*N 9
   ͷ௚ަิۭؒʹؚ·ΕΔͱ͍͏͜ͱ
   
   e = Y − ̂
   Y = Y − PX
   Y = Y − X ̂
   β
   ࢒ࠩϕΫτϧ e Λ
   ͱͨ͠ͱ͖ɺ
   Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn
   9ͷ૾Λ*N 9
   ͱද͢ɻ*N 9
   ͸ू߹Ͱ͋ͬͯҎԼͷΑ͏ʹࣔ͞ΕΔɻ
   Y − PX
   Y ͱ
   ͕௚ަ͢ΔΑ͏ͳม׵ ʁ
   ͕
   PX ͳͷͰ͋Δɻ
   PX
   Y
   ͱ X
   

   View Slide

5. 9F
   ∣ ∣ ∣ ⋯ ∣
   ⃗
   1 x1
   x2
   ⋯ xd
   ∣ ∣ ∣ ⋯ ∣
   X =
   − ⃗
   1 ′
   
   −
   −x′
   
   1−
   −x′
   
   2−
   ⋮
   −x′
   
   d
   −
   X′
   
   =
   X′
   
   e =
   − ⃗
   1 ′
   
   −
   −x′
   
   1−
   −x′
   
   2−
   ⋮
   −x′
   
   d
   −
   e =
   ⃗
   1 ′
   
   e
   x1
   ′
   
   e
   ⋮
   xd
   ′
   
   e
   =
   0
   0
   ⋮
   0
   = ⃗
   0
   n × (1 + d)
   (1 + d) × n
   (1 + d ) × n n × 1 (1 + d) × 1
   9͸Eݸͷجఈ ॎϕΫτϧ
   Λ
   ԣʹฒ΂ͨ΋ͷͩͱߟ͑Δɻ
   సஔ͢Δͱɺ
   ɾ֤ॎϕΫτϧΛసஔ
   ɾԣฒͼˠॎฒͼ
   ͱͳΔɻ
   جఈ͸*N 9
   ʹؚ·Ε͍ͯͯɺ
   *N 9
   ͷ೚ҙͷཁૉ͸࢒ࠩϕΫτϧFͱ௚ަ͢
   Δɻ ͦͷΑ͏ʹߏ੒ͨ͠΋ͷ͕࢒ࠩϕΫτϧ
   Ͱ͋Δɺͱ΋ݴ͑Δɻ
   
   ͷجఈ͸ɺ9ͷୈྻ ୈྻ ʜ ୈEྻ
   Ͱ͋Δɻ
   Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn
   ˠ
   

   View Slide

6. ࣹӨߦྻͷಋग़
   X′
   
   e = X′
   
   (Y − ̂
   Y) = X′
   
   (Y − X ̂
   β) = ⃗
   0 ΑΓɺ
   ⟺ X′
   
   Y = X′
   
   X ̂
   β ⟺ ̂
   β = (X′
   
   X)−1X′
   
   Y
   ̂
   Y = X ̂
   β = X(X′
   
   X)−1X′
   
   Y
   ·ͨɺ
   PX
   = X(X′
   
   X)−1X′
   
   Ͱ͋Δ͔Β ̂
   Y = PX
   Y ͱൺֱͯ͠
   X′
   
   X͸Մٯͱ͢Δ
   X′
   
   e = ⃗
   0 ΑΓɺ
   ͱͳΔɻ
   

   View Slide

7. ࣹӨߦྻ1Yͷੑ࣭
   
   PX
   = X(X′
   
   X)−1X′
   
   ͱͯ͠ɺ
   P2
   X
   = PX
   PX
   = (X(X′
   
   X)−1X′
   
   )(X(X′
   
   X)−1X′
   
   ) = X(X′
   
   X)−1(X′
   
   X)(X′
   
   X)−1X′
   
   = X(X′
   
   X)−1X′
   
   = PX
   P′
   
   X
   = (X(X′
   
   X)−1X′
   
   )′
   
   = X′
   
   ′
   
   ((X′
   
   X)−1)′
   
   X′
   
   = X(X′
   
   X)−1X′
   
   = PX
   Im(X) = {Xv|v ∈ ℝ1+d}
   Im(PX
   ) = {PX
   w|w ∈ ℝn}
   X ͸ n × (1 + d) ߦྻͳͷͰɺ
   ͸
   PX n × n ߦྻ O࣍ਖ਼ํߦྻ
   Ͱ͋Δɻ
   ࣹӨߦྻͷೋ৐ͱసஔ͸࠶ͼࣹӨߦྻʹ໭Δ͜ͱΛ֬ೝ͢Δɻ
   ͯ͞ɺ
   ͱ
   ͸౳͍͠ͱͷ͜ͱͳͷͰɺ͜ΕΛࣔͯ͠ΈΔɻ
   *N 1Y
   ΋*N 9
   ΋ͲͪΒ΋ू߹Ͱ͋Δ͜ͱΑΓɺ
   l1
   ∈ Im(X) ⟹ l1
   ∈ Im(PX
   )
   l2
   ∈ Im(PX
   ) ⟹ l2
   ∈ Im(X)
   Λࣔͤ͹ྑ͍ɻ
   

   View Slide

8. ࣹӨߦྻ1Yͷੑ࣭
   
   [1]l1
   ∈ Im(X) ⟹ l1
   ∈ Im(PX
   )
   [2]l2
   ∈ Im(PX
   ) ⟹ l2
   ∈ Im(X)
   l1
   ∈ Im(X)
   Im(X) = {Xv|v ∈ ℝ1+d}
   Im(PX
   ) = {PX
   w|w ∈ ℝn}
   v1
   ∈ ℝ1+d
   l1
   = Xv1
   ∈ ℝn
   w1
   = Xv1
   ∈ ℝn
   l2
   ∈ Im(PX
   ) l2
   = PX
   w2
   = X(X′
   
   X)−1X′
   
   w2
   ∈ ℝn
   l1
   = PX
   w1
   ∈ Im(PX
   )
   l2
   = X(X′
   
   X)−1X′
   
   w2
   = Xv2
   v2
   = (X′
   
   X)−1X′
   
   w2
   ∈ ℝ1+d l2
   = Xv2
   ∈ Im(X)
   ͱͳΔl1
   ΛऔΔͱɺ ͱͳΔ ͕औΕΔɻ
   l1
   = Xv1
   = X(X′
   
   X)−1(X′
   
   X)v1
   = PX
   Xv1 ͱදͤ͹
   ͱͯ͠
   ͱͳΔl2ΛऔΔͱɺ
   ͱͳΔw2
   ∈ ℝn ͕औΕΔɻ ͱදͤ͹
   ͱͯ͠
   Ҏ্ΑΓɺ
   Im(PX
   ) = Im(X)
   

   View Slide

9. F
   0 = ⃗
   1 ′
   
   e
   Im(X)
   Y
   ̂
   Y = PX
   Y = X ̂
   β
   ℝn
   Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn
   e
   e
   ⃗
   1
   ˠ
   ͱ͸ɺ
   ⃗
   1 ͱe͕௚ަ͢Δɺͱݴ͍ͬͯΔɻ
   

   View Slide

10. 
    0 = ⃗
    1 ′
    
    e = ⃗
    1 ′
    
    (Y − X ̂
    β) = n(¯
    y − (1,¯
    x′
    
    ) ̂
    β)ʹ͍ͭͯ෼ղͯ͠ߟ͑Δɻ
    ⃗
    1 ′
    
    Y =
    (
    1
    ⋮
    1
    )
    ′
    
    y1
    ⋮
    yn
    = ∑ yi
    = n¯
    y
    ⃗
    1 ′
    
    X =
    (
    1
    ⋮
    1
    )
    ′
    
    1 , −x′
    
    1
    −
    ⋮ , ⋮
    1 , −x′
    
    n
    −
    = (n, ∑ x′
    
    i) = (n, n ¯
    x′
    
    )
    0 = ⃗
    1 ′
    
    e = ⃗
    1 ′
    
    (Y − X ̂
    β) = (n¯
    y − (n, n ¯
    x′
    
    ) ̂
    β) = n(¯
    y − (1, ¯
    x′
    
    ) ̂
    β)
    ͜ͷΧϯϚ͸ྻΛ۠෼͚͢ΔͨΊͷه߸ɻ
    ಺ੵͰ͸ͳ͍ɻ
    −x′
    
    i
    − = (xi,1
    ⋯ xi,d
    )
    ͸ԣϕΫτϧ
    Ҏ্ΑΓɺ
    ͱͳͬͯɺ ¯
    y = (1, ¯
    x′
    
    ) ̂
    β ΛಘΔɻ
    ¯
    y
    ԣϕΫτϧͨͪͷ࿨͸
    ରԠ͢Δ੒෼͝ͱʹ࿨ΛऔΔ
    

    View Slide

11. QԼஈ
    Y = X ̂
    β + e Y − ⃗
    1 ¯
    y = X ̂
    β − ⃗
    1 ¯
    y + e
    ΑΓ
    Y − ⃗
    1 ¯
    y = X ̂
    β − ⃗
    1 (1,¯
    x′
    
    ) ̂
    β + e
    ¯
    y = (1, ¯
    x′
    
    ) ̂
    β
    ͞Βʹ ΑΓ
    = (X − ⃗
    1 (1,¯
    x′
    
    ))
    ̂
    β + e
    = (
    1 x1
    ′
    
    ⋮ ⋮
    1 xn
    ′
    
    − ⃗
    1 (1,¯
    x′
    
    ))
    ̂
    β + e
    =
    1 − 1 x1
    ′
    
    − x′
    
    ⋮ ⋮
    1 − 1 xn
    ′
    
    − x′
    
    ̂
    β + e
    =
    0 (x1
    − ¯
    x)′
    
    ⋮ ⋮
    0 (xn
    − ¯
    x)′
    
    ̂
    β + e
    

    View Slide

12. Qதஈ
    Im(X)
    Y− ⃗
    1 ¯
    y
    X ̂
    β − ⃗
    1 ¯
    y
    ℝn
    Im(X) = {Xv ∣ v ∈ ℝ1+d} ⊂ ℝn
    e
    Y
    X ̂
    β
    e
    ⃗
    1 ¯
    y͚ͩͣΒ͢
    ͜ͷɹɹ্ͷࡾ֯ܗͰ
    ࡾฏํͷఆཧΛ͢Ε͹ྑ͍ɻ
    ℝn
    

    View Slide

13. ଓ
    Qதஈ
    Y − ⃗
    1 ¯
    y = X ̂
    β − ⃗
    1 ¯
    y + e
    ⟺ Y − ⃗
    1 ¯
    y = X ̂
    β − ⃗
    1 (1,¯
    x′
    
    ) ̂
    β + e
    ΑΓɺ ∥Y − ⃗
    1 ¯
    y∥2 = ∥X ̂
    β − ⃗
    1 (1,¯
    x′
    
    ) ̂
    β∥2 + ∥e∥2
    ⟺ ∑ (yi
    − ¯
    y)2 = ∑ ((xi
    − ¯
    x)′
    
    ̂
    β1:d)
    2
    + ∑ e2
    i
    ͱͳΔɻ
    ̂
    β0
    ܎਺
    ஫໨ϙΠϯτ
    ͸ɺ
    ͕ফ͑Δ
    ͷฏۉ͔Βͷภࠩ
    xi
    xi
    − ¯
    x ͕ग़ݱ͢Δ
    ⃗
    1 ¯
    y Λ࢖ͬͨ͜ͱͰɺ
    ͓ೃછΈ
    ͷมಈͷ෼ղͷ͕ࣜग़ݱ͢Δ
    ͜ͱͰ͋Δɻ
    

    View Slide