Share
Embed
Principal Component Analysis; PCA
はじめての『はじめてのパターン認識』第9章 This is a general introduction of Principal Component Analysis; PCA for machine learning, which includes vector space and subspace, PCA and singular value decomposition; SVD.
Shoh-kudo
July 11, 2019
Transcript
87 slides
-
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜ΕΛҰͭҰͭઆ໌͍ͯ͘͠ʜ -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} V ϕΫτϧۭؒɹͦΕͧΕߦ͢Δɹݸͷ͔࣠ΒͰ͖ͯΔ d Vd=3 ݸ ݸ ݸ 1 2 3 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧͱɼྫ͑ʜ Vd=3 Vd=3 ্࣠ͷϕΫτϧ ্࣠ͷϕΫτϧͱ ɹɹฏ໘͔Β֎ΕͨϕΫτϧ x1 x3 x2 x1 x3 x2 x1 x2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧͱɼྫ͑ʜ Vd=3 Vd=3 εΧϥʔഒͱϕΫτϧͷ͠ࢉͰ දݱͰ͖ͳ͍Α͏ͳϕΫτϧͷ x1 ⟹ a1 x1 x1 ⟹ x1 + x2 εΧϥʔഒ ͠ࢉ x1 x3 x2 x1 x3 x2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧͱɼྫ͑ʜ Vd=3 Vd=3 εΧϥʔഒͱϕΫτϧͷ͠ࢉͰ දݱͰ͖ͳ͍Α͏ͳϕΫτϧͷ ɹ࣍ݩͷϕΫτϧۭؒͰɹݸͷ Ұ࣍ಠཱͳϕΫτϧ͕ଘࡏ͢Δɽ d d x1 x3 x2 x1 x3 x2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧ εΧϥʔഒͱϕΫτϧͷ͠ࢉͰ දݱͰ͖ͳ͍Α͏ͳϕΫτϧͷ εΧϥʔഒͱϕΫτϧͷ͠ࢉ ͰදݱͰ͖ΔϕΫτϧ Vd=3 Vd=3 ɹɹฏ໘্ͷ ϕΫτϧ ฏ໘্ͷ ϕΫτϧ Ұ࣍ैଐ ઢܗैଐ ͳϕΫτϧ x1 x3 x2 x1 x3 x2 x1 x2 x2 x3 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧ εΧϥʔഒͱϕΫτϧͷ͠ࢉͰ දݱͰ͖ͳ͍Α͏ͳϕΫτϧͷ εΧϥʔഒͱϕΫτϧͷ͠ࢉ ͰදݱͰ͖ΔϕΫτϧ Vd=3 Vd=3 εΧϥʔഒ Ұ࣍ैଐ ઢܗैଐ ͳϕΫτϧ x1 x3 x2 x1 x3 x2 ɹɹฏ໘্ͷ ϕΫτϧ x1 x2 ฏ໘্ͷ ϕΫτϧ x2 x3 -
x2 x1 x3 ෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W
x1 , x2 , . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} Ұ࣍ಠཱͳϕΫτϧ εΧϥʔഒͱϕΫτϧͷ͠ࢉͰ දݱͰ͖ͳ͍Α͏ͳϕΫτϧͷ εΧϥʔഒͱϕΫτϧͷ͠ࢉ ͰදݱͰ͖ΔϕΫτϧ Ұ࣍ैଐ ઢܗैଐ ͳϕΫτϧ Vd=3 Vd=3 ϕΫτϧͷ͠ࢉ x2 x1 x3 ɹɹฏ໘্ͷ ϕΫτϧ x1 x2 ฏ໘্ͷ ϕΫτϧ x2 x3 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} εΧϥʔഒͱϕΫτϧͷ͠ࢉ ͰදݱͰ͖ΔϕΫτϧ Ұ࣍ैଐ ઢܗैଐ ͳϕΫτϧΛఆࣜԽ͢Δͱʜ Vd=3 x2 x1 x3 ɹɹฏ໘্ͷ ϕΫτϧ x1 x2 ۭؒɹɹʹؚ·ΕΔҙͷϕΫτϧɹɼ ۭؒɹɹதʹଘࡏ͢ΔҰ࣍ಠཱͳϕΫτϧ ɹɹɹɹʢجఈʣͷઢܕ݁߹ͰදݱͰ͖Δɽ Vd=3 v x1 , x2 , x3 Vd=3 ҙఆ جఈ v = a1 x1 + a2 x2 + a3 x3 = 3 ∑ i=1 ai xi ai xi xi ai -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} εΧϥʔഒͱϕΫτϧͷ͠ࢉ ͰදݱͰ͖ΔϕΫτϧ Ұ࣍ैଐ ઢܗैଐ ͳϕΫτϧΛఆࣜԽ͢Δͱʜ Vd=3 x2 x1 x3 ɹɹฏ໘্ͷ ϕΫτϧ x1 x2 ҙఆ v = d ∑ i=1 ai xi ai xi xi ai ࣍ݩͷϕΫτϧۭؒɹͰҰൠԽ͢Δͱʜ d V جఈ -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 Vd=3 x1 x3 x2 W Λɹͱɹ͔ΒΔϕΫτϧશମͱͨ࣌͠ɼ ϕΫτϧۭؒɹɹʹแؚ͞ΕΔɽ W x1 x2 Vd=3 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Λߟ͑Δɽ Vd=3 Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 Vd=3 x1 x3 x2 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Ͳ͏͔Λߟ͑Δɽ (1) W ≠ ∅ (2) x1 , x2 ∈ W ⟹ x1 + x2 ∈ W (3) x1 ∈ W, λ ∈ R ⟹ λx1 ∈ W Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 Vd=3 x1 x3 x2 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Ͳ͏͔Λߟ͑Δɽ (1) W ≠ ∅ ɹ্࣠ͱɹ্࣠ͷೋछྨͷϕΫτϧʹΑΓ දݱ͞ΕΔϕΫτϧ͕ଘࡏ͢ΔͷͰɼɹ ۭू߹Ͱͳ͍ɽ x1 x2 W Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Ͳ͏͔Λߟ͑Δɽ (2) x1 , x2 ∈ W ⟹ x1 + x2 ∈ W Vd=3 x1 x3 x2 ɹ্࣠ͱɹ্࣠ͷೋछྨͷϕΫτϧͰදݱ ͞ΕΔɹɹฏ໘্ͷશ͕ͯɹʹؚ·ΕΔͷͰ ਖ਼͍͠ɽ x1 x2 x1 x2 W Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 Vd=3 x1 x3 x2 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Ͳ͏͔Λߟ͑Δɽ (3) x1 ∈ W, λ ∈ R ⟹ λx1 ∈ W ɹ্࣠ͱɹ্࣠ͷೋछྨͷϕΫτϧͦΕͧΕͷ εΧϥʔഒ·ͨɹ্ͷϕΫτϧͳͷͰਖ਼͍͠ɽ x1 x2 W Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ ϕΫτϧۭ͕ؒҙͷ࣍ݩͰ͋Δ߹Λߟ͑Δલʹ·ͣɹɹɹͰʜ Vd=3 Vd=3 x1 x3 x2 ͜ͷ࣌ɼ࣍ͷ݅Λຬ͔ͨ͢Ͳ͏͔Λߟ͑Δɽ (2) x1 , x2 ∈ W ⟹ x1 + x2 ∈ W (3) x1 ∈ W, λ ∈ R ⟹ λx1 ∈ W ্ه݅Λ߹ΘͤΔͱϕΫτϧۭؒɹͷҰ࣍ಠཱͳ ϕΫτϧͷઢܕ݁߹ɹۭؒͷͰ͋Δͱݴ͑Δ W W Wd=2 -
෦ۭؒ ෦ۭؒ ɹ࣍ݩϕΫτϧۭؒɹͷ෦ۭؒɹɼɹʹؚ·ΕΔҰ࣍ಠཱͳϕΫτϧɼ ɹɹɹɹɹɹɹɹɹͱҙͷఆɹɹɹɹɹɹΛ༻͍ͯҎԼͷΑ͏ʹఆٛ͢Δɽ d V W x1 , x2
, . . . , xr (r ≤ d) V a1 , a2 , . . . , ad W = {a1 x1 + a2 x2 + . . . + ar xr |ai ∈ R, i = 1,2,...,r} ͜͜·Ͱཧղ্ͨ͠Ͱ෦ۭؒΛಋೖ͠·͢ʜ Vd=3 x1 x3 x2 ཁૉ͕θϩͰͳ͍ۭؒɹ͕ɹ࣍ݩ ϕΫτϧۭؒɹͷ෦ۭؒͰ͋Δ ͱ͍͏ͷɹΑΓখ͍͞ɹݸͷ جఈͷઢܕ݁߹ͰҙͷϕΫτϧ ΛҰҙʹදݱͰ͖ΔϕΫτϧۭؒ Ͱ͋Δͱ͍͏͜ͱɽ W d V d r Wd=2 -
෦ۭؒ ਖ਼نަجఈ ਖ਼نަجఈͱ͕͞ɹͰ ਖ਼نԽ͞Ε͍ͯΔ ޓ͍ʹަ͍ͯ͠Δ ੵ͕ɹ Α͏ͳجఈ ઢܗ݁߹ͰશͯΛදͤΔΑ͏ͳඞཁ࠷ݶͷϕΫτϧͷू߹ Ͱ͋Δɽ
1 0 1 Vd=3 x1 x3 x2 Ұ࣍ಠཱͰ͋ΔجఈɹɹɹɹΛ͕͞ɹͰ͔ͭ ަ͢ΔΑ͏ʹઃఆͨ͠ͷ͕ਖ਼نަجఈɽ x1 , x2 , x3 1 ∥x1 ∥ = ∥x2 ∥ = ∥x3 ∥ = 1 xT 1 x2 = xT 2 x3 = xT 3 x1 = 0 {v ∈ Vd=3 |v = 3 ∑ i=1 ai xi } -
෦ۭؒ ਖ਼نަجఈ ਖ਼نަجఈͱ͕͞ɹͰ ਖ਼نԽ͞Ε͍ͯΔ ޓ͍ʹަ͍ͯ͠Δ ੵ͕ɹ Α͏ͳجఈ ઢܗ݁߹ͰશͯΛදͤΔΑ͏ͳඞཁ࠷ݶͷϕΫτϧͷू߹ Ͱ͋Δɽ
1 0 1 Vd=3 x1 x3 x2 ࠨਤͷΑ͏ͳϕΫτϧҎ֎ʹࡾ࣍ݩͷ ϕΫτϧۭؒʹ͓͍ͯࡾͭͷϕΫτϧ͕ ༩͑ΒΕͨ࣌ʹͦΕΛݩʹࡾͭͷجఈ Λ࡞Δ͜ͱ͕Ͱ͖Δɽ άϥϜʔγϡϛοτͷਖ਼نަԽ -
෦ۭؒ Vd=3 x1 x3 x2 άϥϜʔγϡϛοτͷਖ਼نަԽ ҰͭͷϕΫτϧΛબΜͰਖ਼نԽ͢Δ ͞Λʹ͢Δ ɽ
n1 = x1 ∥x1 ∥ = x1 12 + 02 + 12 = 1 2 ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) ྫ x1 = ( 1 0 1 ) -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ҰͭͷϕΫτϧΛબΜͰਖ਼نԽ͢Δ ͞Λʹ͢Δ ɽ n1 = x1 ∥x1
∥ = x1 12 + 02 + 12 = 1 2 ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) ྫ n1 = 1 2 ( 1 0 1 ) x1 = ( 1 0 1 ) Vd=3 n1 x3 x2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ɹͷใΛͬͯೋͭͷجఈΛ࡞Δɽ ྫ n1 = 1 2 (
1 0 1 ) n1 x1 = ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) ˜ n2 = x2 − 1 ∑ j=1 (nT j x2) nj = (x2 − (nT 1 x2) n1) = ( 2 1 0 ) − 1 2 2 (1 0 1) ( 2 1 0 ) ( 1 0 1 ) = ( 1 1 −1 ) Vd=3 n1 x3 x2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ɹͷใΛͬͯೋͭͷجఈΛ࡞Δɽ ྫ n1 = 1 2 (
1 0 1 ) n1 x1 = ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) n2 = 1 3 ( 1 1 −1 ) ˜ n2 = x2 − 1 ∑ j=1 (nT j x2) nj = (x2 − (nT 1 x2) n1) = ( 2 1 0 ) − 1 2 2 (1 0 1) ( 2 1 0 ) ( 1 0 1 ) = ( 1 1 −1 ) Vd=3 n1 x3 n2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ɹɹͷใΛͬͯࡾͭͷجఈΛ࡞Δɽ ˜ n3 = x3 − 2
∑ j=1 (nT j x3) nj = (x3 − (nT 1 x3) n1 − (nT 2 x3) n2) ྫ n1 , n2 x1 = ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) n1 = 1 2 ( 1 0 1 ) n2 = 1 3 ( 1 1 −1 ) = ( 0 1 1 ) − 1 2 2 (1 0 1) ( 0 1 1 ) ( 1 0 1 ) − 1 3 2 (1 1 − 1) ( 0 1 1 ) ( 1 1 −1 ) = ( −1 2 1 ) Vd=3 n1 x3 n2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ɹɹͷใΛͬͯࡾͭͷجఈΛ࡞Δɽ ˜ n3 = x3 − 2
∑ j=1 (nT j x3) nj = (x3 − (nT 1 x3) n1 − (nT 2 x3) n2) ྫ n1 , n2 x1 = ( 1 0 1 ) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) n1 = 1 2 ( 1 0 1 ) n2 = 1 3 ( 1 1 −1 ) = ( 0 1 1 ) − 1 2 2 (1 0 1) ( 0 1 1 ) ( 1 0 1 ) − 1 3 2 (1 1 − 1) ( 0 1 1 ) ( 1 1 −1 ) = ( −1 2 1 ) n3 = 1 2 3 ( −1 2 1 ) Vd=3 n1 n3 n2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ަ͍ͯ͠Δ͔֬ೝͯ͠ΈΔʜ ྫ x1 = ( 1 0 1
) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) n1 = 1 2 ( 1 0 1 ) n2 = 1 3 ( 1 1 −1 ) n3 = 1 2 3 ( −1 2 1 ) nT 1 n2 = 1 6 − 1 6 = 0 nT 2 n3 = − 1 6 + 2 6 − 1 6 = 0 nT 3 n1 = − 1 2 6 + 1 2 6 = 0 Vd=3 n1 n3 n2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ͪͳΈʹ͏ҰͭͷϕΫτϧΛ͖࣋ͬͯͯಉ༷ͷܭࢉΛ͢Δͱʜ ྫ x1 = ( 1 0 1
) x2 = ( 2 1 0 ) x3 = ( 0 1 1 ) n1 = 1 2 ( 1 0 1 ) n2 = 1 3 ( 1 1 −1 ) n3 = 1 2 3 ( −1 2 1 ) x4 = ( 1 0 0 ) ⟹ ˜ n4 = 1 12 ( 1 −2 −1 ) ͜Ε໌Β͔ʹɹͷఆഒͰ͋ΔͨΊ৽͍͠جఈʹͳΒͳ͍ n3 Vd=3 n1 n3 n2 -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ҙͷɹ࣍ݩͰಉ༷ʹཱ͢Δ͔Λ֬ೝ͢ΔͨΊʹֶతؼೲ๏ΛؤுΔʜ k (1) i = 1 ͷ࣌ʜ n1
= x1 ∥x1 ∥ (2) i = 2 ͷ࣌ʜ ˜ n2 = x2 − (nT 1 x2) n1 , n2 = ˜ n2 ∥ ˜ n2 ∥ nT 1 n2 = nT 1 (x2 − (nT 1 x2) n1) 1 ∥ ˜ n2 ∥ = (nT 1 x2 − (nT 1 x2) nT 1 n1) 1 ∥ ˜ n2 ∥ = (nT 1 x2 − nT 1 x2) 1 ∥ ˜ n2 ∥ = 0 ΑΓΓཱͭɽ جఈͳͷͰϊϧϜ -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ҙͷɹ࣍ݩͰಉ༷ʹཱ͢Δ͔Λ֬ೝ͢ΔͨΊʹֶతؼೲ๏ΛؤுΔʜ k (3) i = k − 1
ͷ࣌ʜ (4) i = k ͷ࣌ʜ ˜ nk−1 = xk−1 − k−2 ∑ j=1 (nT j xk−1) nj , nk−1 = ˜ nk−1 ∥ ˜ nk−1 ∥ ͕ΓཱͭͱԾఆ͢Δͱ ˜ nk = xk − k−1 ∑ j=1 (nT j xk) nj , nk = ˜ nk ∥ ˜ nk ∥ ͱॻ͚Δɽ͜Εʹ͍ͭͯҙͷجఈɹͱͷੵΛܭࢉ͢Δͱ ni -
෦ۭؒ άϥϜʔγϡϛοτͷਖ਼نަԽ ҙͷɹ࣍ݩͰಉ༷ʹཱ͢Δ͔Λ֬ೝ͢ΔͨΊʹֶతؼೲ๏ΛؤுΔʜ k (4) i = k ͷ࣌ʜ ˜
nk = xk − k−1 ∑ j=1 (nT j xk) nj , nk = ˜ nk ∥ ˜ nk ∥ ͱॻ͚Δɽ͜Εʹ͍ͭͯҙͷجఈɹͱͷੵΛܭࢉ͢Δͱ ni ͳͷͰɹɹɹͷΈ͕Γ nT i nj = 0 (i ≠ j) i = j nT i nk = (nT i xk − (nT i xk) nT i ni) 1 ∥ ˜ nk ∥ = (nT i xk − nT i xk) 1 ∥ ˜ nk ∥ = 0 ͜Ε͕ɹɹɹɹɹɹͷશͯͷɹʹ͍ͭͯΓཱͭͷͰ άϥϜʔγϡϛοτͷਖ਼نަԽҙ࣍ݩͰ͑Δ 1 ≤ i ≤ k − 1 i -
෦ۭؒ ·ͱΊ Vd=3 E࣍ݩϕΫτϧۭؒͷҙͷϕΫτϧ Eݸͷجఈͷઢܗ݁߹ͰҰҙʹදݱͰ͖Δ Vd=3 x1 x3 x2 Wd=2
ਖ਼نަجఈ ͕͞Ͱަ͢ΔΑ͏ʹఆΊͨجఈ Vd=3 x1 x3 x2 Vd=3 n1 n3 n2 άϥϜʔγϡϛοτͷਖ਼نަԽ ༩͑ΒΕͨϕΫτϧ͔Β ਖ਼نަجఈΛ࡞Δ ෦ۭؒ ɹɹɹݸͷجఈͰදݱͰ͖ΔϕΫτϧۭؒ r( < d) -
ओੳ ಋग़ͷྲྀΕ ᶃֶशσʔλϕΫτϧ༝དྷͷߦྻɹΛ࡞Δ ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ ᶈग़͖ͯͨݻ༗Λղ͘ ᶉओΛநग़͢Δ ¯ X ᶄֶशσʔλߦྻɹʹϕΫτϧɹΛֻ͚ͨϕΫτϧɹΛ࡞Δ ¯
X aj sj ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } ᶆϕΫτϧɹͷൣғΛࢦఆͯ͠ࢄɹɹɹͷ࠷େԽΛઃఆ͢Δ Var{sj } aj -
ओੳ ᶃֶशσʔλϕΫτϧ༝དྷͷߦྻɹΛ࡞Δ ¯ X X = xT 1 xT 2
⋮ xT N = x11 x12 ⋯ x1d x21 x22 ⋯ x2d ⋮ ⋮ ⋱ ⋮ xN1 xN2 ⋯ xNd ¯ X = (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T = x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd -
ओੳ ¯ Xaj = (x1 − ¯ x)T (x2 −
¯ x)T ⋮ (xN − ¯ x)T aj1 aj2 ⋮ ajd = x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd aj1 aj2 ⋮ ajd ᶄֶशσʔλߦྻɹʹϕΫτϧɹΛֻ͚ͨϕΫτϧɹΛ࡞Δ ¯ X aj sj = (x11 − ¯ x1 )aj1 + (x12 − ¯ x2 )aj2 + ⋯ + (x1d − ¯ xd )ajd (x21 − ¯ x1 )aj1 + (x22 − ¯ x2 )aj2 + ⋯ + (x2d − ¯ xd )ajd ⋮ (xN1 − ¯ x1 )aj1 + (xN2 − ¯ x2 )aj2 + ⋯ + (xNd − ¯ xd )ajd = s1j s2j ⋮ sNj = sj ݸͷಛྔͷҰ࣍ؔͷܗ d -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } = x11 − ¯ x1 x21 − ¯ x1 ⋯ xN1 − ¯ x1 x12 − ¯ x2 x22 − ¯ x2 ⋯ xN2 − ¯ x2 ⋮ ⋮ ⋱ ⋮ x1d − ¯ xd x2d − ¯ xd ⋯ xNd − ¯ xd x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } = x11 − ¯ x1 x21 − ¯ x1 ⋯ xN1 − ¯ x1 x12 − ¯ x2 x22 − ¯ x2 ⋯ xN2 − ¯ x2 ⋮ ⋮ ⋱ ⋮ x1d − ¯ xd x2d − ¯ xd ⋯ xNd − ¯ xd x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } = x11 − ¯ x1 x21 − ¯ x1 ⋯ xN1 − ¯ x1 x12 − ¯ x2 x22 − ¯ x2 ⋯ xN2 − ¯ x2 ⋮ ⋮ ⋱ ⋮ x1d − ¯ xd x2d − ¯ xd ⋯ xNd − ¯ xd x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } = x11 − ¯ x1 x21 − ¯ x1 ⋯ xN1 − ¯ x1 x12 − ¯ x2 x22 − ¯ x2 ⋯ xN2 − ¯ x2 ⋮ ⋮ ⋱ ⋮ x1d − ¯ xd x2d − ¯ xd ⋯ xNd − ¯ xd x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } = x11 − ¯ x1 x21 − ¯ x1 ⋯ xN1 − ¯ x1 x12 − ¯ x2 x22 − ¯ x2 ⋯ xN2 − ¯ x2 ⋮ ⋮ ⋱ ⋮ x1d − ¯ xd x2d − ¯ xd ⋯ xNd − ¯ xd x11 − ¯ x1 x12 − ¯ x2 ⋯ x1d − ¯ xd x21 − ¯ x1 x22 − ¯ x2 ⋯ x2d − ¯ xd ⋮ ⋮ ⋱ ⋮ xN1 − ¯ x1 xN2 − ¯ x2 ⋯ xNd − ¯ xd = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 = Nσ11 Nσ12 ⋯ Nσ1d Nσ12 Nσ22 ⋯ Nσ2d ⋮ ⋮ ⋱ ⋮ Nσ1d Nσ2d ⋯ Nσdd = NΣ ڞࢄߦྻ -
ओੳ ¯ XT ¯ X = (x1 − ¯ x
x2 − ¯ x ⋯ xN − ¯ x) (x1 − ¯ x)T (x2 − ¯ x)T ⋮ (xN − ¯ x)T ᶅϕΫτϧɹͷࢄɹɹɹΛܭࢉ͢Δ sj Var{sj } Var{sj } = 1 N sT j sj = 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj = aT j Σaj = ∑N i=1 (xi1 − ¯ x1 )2 ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi1 − ¯ x1 )(xi2 − ¯ x2 ) ∑N i=1 (xi2 − ¯ x2 )2 ⋯ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ⋮ ⋮ ⋱ ⋮ ∑N i=1 (xi1 − ¯ x1 )(xid − ¯ xd ) ∑N i=1 (xi2 − ¯ x2 )(xid − ¯ xd ) ⋯ ∑N i=1 (xid − ¯ xd )2 = Nσ11 Nσ12 ⋯ Nσ1d Nσ12 Nσ22 ⋯ Nσ2d ⋮ ⋮ ⋱ ⋮ Nσ1d Nσ2d ⋯ Nσdd = NΣ aT j Σaj ڞࢄߦྻ ڞࢄߦྻͰஔ -
ओੳ ᶆϕΫτϧɹͷൣғΛࢦఆͯ͠ࢄɹɹɹͷ࠷େԽΛઃఆ͢Δ Var{sj } aj ओ maximize f(aj ) =
Var{sj } = aT j Σaj subject to g(aj ) = ∥aj ∥2 − 1 = 0 ධՁؔʢ࠷େԽʣ ੍݅ ্هͷෆ੍ࣜ݅ԼͰධՁؔΛղ͘ʢ࠷େԽ͢Δʣ͜ͱΛओͱ͢Δɽ ͜͜Ͱɼ੍݅ͱධՁؔΛಉ࣌ʹຬͨ͢৽͍ؔ͠Λߟ͑Δͱʜ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj − λ (∥aj ∥2 − 1) = aT j Σaj − λ (aT j aj − 1) -
ओੳ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj
− λ (aT j aj − 1) ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ ݅ᶃ ∂˜ L(aj , λ) ∂aj = (Σ + ΣT) aj − 2λaj = 2Σaj − 2λaj = 0 Σaj − λaj = (Σ − λ) aj = (Σ − λE) aj = 0 ࠷େԽͷͨΊͷඍɽݻ༗ͷܗʹͳΔʜ ରশߦྻͳͷͰɹɹɹ͔Β Σ Σ = ΣT -
ओੳ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj
− λ (aT j aj − 1) ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ ݅ᶄ ∂˜ L(aj , λ) ∂λ = − (aT j aj − 1) = 0 aT j aj − 1 = 0 ੍݅ͦͷͷɽ -
ओੳ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj
− λ (aT j aj − 1) ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ ݅ᶅ λ ≥ 0 ϥάϥϯδϡະఆͷఆٛ -
ओੳ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj
− λ (aT j aj − 1) ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ ݅ᶆ λ (aT j aj − 1) = 0 ૬ิੑ݅ɽ ɹɹɹɹɹͳͷͰৗʹΓཱͭʜ aT j aj − 1 = 0 -
ओੳ ϥάϥϯδϡؔ ˜ L(aj , λ) = aT j Σaj
− λ (aT j aj − 1) ᶇ্هΛϥάϥϯδϡͷະఆ๏Ͱղ͘ Ҏ্Λ·ͱΊͯࢄ࠷େԽʹ͓͚Δ,,5 ,BSVTI,VIO5VDLFS ݅ ᶃ (Σ − λE) aj = 0 ᶄ aT j aj − 1 = 0 ᶅ λ ≥ 0 ᶆ λ (aT j aj − 1) = 0 ͕ಘΒΕͨɽ͜Εɼᶃʹ͓͚Δݻ༗Λղ͘͜ͱͰղΛಘΔ͜ͱ͕Ͱ͖Δɽ (Σ − λE) aj = 0 -
ओੳ ᶈग़͖ͯͨݻ༗Λղ͘ ݻ༗ det (Σ − λE) = 0 a1
, a2 , . . . , ad λ1 ≥ λ2 ≥ . . . ≥ λd ߦྻ͕ࣜθϩʹͳΔ͜ͱΛ ׆༻ͯ͠ ݻ༗Λࢉग़ͯ͠ େ͖͍ॱʹฒΔ ݻ༗ϕΫτϧΛٻΊͯ ਖ਼نԽ͢Δ (Σ − λE) aj = 0 -
ओੳ ᶈग़͖ͯͨݻ༗Λղ͘ ݻ༗ ڞࢄߦྻɹ͕࣮ରশߦྻͰ͋Δ͜ͱ͔Βݻ༗ϕΫτϧɹͦΕͧΕަ͢Δɽ Σ a ࣮ରশߦྻͱݻ༗ϕΫτϧͷަੑ Σai = λi
ai , Σaj = λj aj , (λi ≠ λj ) aT i (Σaj) = aT i Σaj = (ΣT ai) T aj = (Σai) T aj = (λi ai) T aj = λi aT i aj aT i (Σaj) = aT i (λj aj) = λj aT i aj λi aT i aj − λj aT i aj = 0 ⟺ aT i aj = 0 (λi ≠ λj ) ಉ͡ͷ (Σ − λE) aj = 0 -
ओੳ ᶈग़͖ͯͨݻ༗Λղ͘ ݻ༗ aT i aj = δij = {
1 (i = j) 0 (i ≠ j) ڞࢄߦྻɹ͕࣮ରশߦྻͰ͋Δ͜ͱ͔Βݻ༗ϕΫτϧɹͦΕͧΕަ͢Δɽ Σ a (Σ − λE) aj = 0 ΫϩωοΧʔͷσϧλ -
ओੳ ᶈग़͖ͯͨݻ༗Λղ͘ ݻ༗ ڞࢄߦྻɹ͕࣮ରশߦྻͰ͋Δ͜ͱ͔Βݻ༗ϕΫτϧɹͦΕͧΕަ͢Δɽ Σ a ಉ͡Α͏ͳಛྔ͕͋ΓɼͦΕΒͷࢄ ڞࢄ͕ಉ͡Α͏ͳΛͱΔͱɹɹ࣍ݩ ΑΓগͳ͍ใ͔࣋ͨ͠ͳ͘ͳΔɽ Σ
= d × d Σ = d × k ϥϯΫམͪ (k ≤ d) d Σ (Σ − λE) aj = 0 ଞͱඃͬͯΔ -
ओੳ (Σ − λE) aj = 0 ᶈग़͖ͯͨݻ༗Λղ͘ ݻ༗ ڞࢄߦྻɹ͕࣮ରশߦྻͰ͋Δ͜ͱ͔Βݻ༗ϕΫτϧɹͦΕͧΕަ͢Δɽ
Σ a ඇθϩݻ༗࠷େͰɹݸಘΒΕΔ͕ɼ ੑ࣭ͷࣅͨಛྔͷ͚ͩڞࢄߦྻͷ ϥϯΫམͪɼθϩʹͳΔݻ༗͕૿͑Δ Σ = d × k ϥϯΫམͪ (k ≤ d) ϥϯΫམ͚ͪͨͩ͠ θϩʹͳΔݻ༗͕૿͑Δ λi = 0 ai = 0 Σ ≠ O ͳͷͰ d -
ओੳ ᶉओΛநग़͢Δ a1 , a2 , . . . ,
ad λ1 ≥ λ2 ≥ . . . ≥ λd ٻΊͨݻ༗ͱͦΕʹରԠ͢Δݻ༗ϕΫτϧ ֶशσʔλɹʹݻ༗ϕΫτϧɹΛֻ͚ͨϕΫτϧɹͷࢄΛߟ͑Δʜ ¯ X a1 s1 Var{s1 } = Var{ ¯ Xa1 } = aT 1 Σa1 = a1 (Σa1) = aT 1 (λ1 a1) = λ1 aT 1 a1 = λ1 ಉ༷ͷܭࢉ͕ଞͷݻ༗ϕΫτϧʹ͍ͭͯΓཱͭͷͰɼ ֶशσʔλɹͱݻ༗ϕΫτϧɹΛֻ͚ͨϕΫτϧɹ͕࠷େͷ ࢄΛ࣋ͭͷ࠷େͷɹΛ࣋ͭɹΛͱͯ͠બΜͩͱ͖ɽ ͢ͳΘͪɼɹɹͷ࣌ɼ࠷େͷࢄɹΛ࣋ͭઢɹɹ͕Ҿ͚Δɽ ¯ X ai si λi ai i = 1 λ1 f1 (x) = aT 1 x = d ∑ i=1 a1i xi f1 (x) -
ओੳ ᶉओΛநग़͢Δ a1 , a2 , . . . ,
ad λ1 ≥ λ2 ≥ . . . ≥ λd ٻΊͨݻ༗ͱͦΕʹରԠ͢Δݻ༗ϕΫτϧ ֶशσʔλɹʹݻ༗ϕΫτϧɹΛֻ͚ͨϕΫτϧɹͷࢄΛߟ͑Δʜ ¯ X a1 s1 Var{s1 } = Var{ ¯ Xa1 } = aT 1 Σa1 = a1 (Σa1) = aT 1 (λ1 a1) = λ1 aT 1 a1 = λ1 ಉ༷ʹɹ͕େ͖͍ॱʹॏཁͳใΛ࣋ͭઢɹɹ͕Ҿ͚Δ λi fi (x) fj (x) = aT j x = d ∑ i=1 aji xi -
ओੳ ᶉओΛநग़͢Δ a1 , a2 , . . . ,
ad λ1 ≥ λ2 ≥ . . . ≥ λd ٻΊͨݻ༗ͱͦΕʹରԠ͢Δݻ༗ϕΫτϧ ͱ͜ΖͰɼશࢄྔɹɹΛߟ͑Δͱɼ Vtotal = d ∑ i=1 λi ͜ͷதͰݻ༗ɹΛҰͭऔΓग़ͯ͠શࢄྔʹ͓͚Δد༩ɹΛΈΔͱɼ ͦͷ͕ͲΕ͚ͩݩͷσʔλΛ࠶ݱ͍ͯ͠Δ͔ΛΔ͜ͱ͕Ͱ͖Δɽ ck = λk Vtotal Vtotal λi ck ݩͷσʔλͷ࠶ݱ Έ͍ͨͳͷʜ -
ओੳ ᶉओΛநग़͢Δ a1 , a2 , . . . ,
ad λ1 ≥ λ2 ≥ . . . ≥ λd ٻΊͨݻ༗ͱͦΕʹରԠ͢Δݻ༗ϕΫτϧ ͱ͜ΖͰɼશࢄྔɹɹΛߟ͑Δͱɼ Vtotal = d ∑ i=1 λi զʑ͕ཉ͍͠ͷྫ͑ɼݩͷσʔλͷΛ࠶ݱͰ͖Δ·ͰʂͳͲ ͳͷͰɼେ͖͍ݻ༗͔Βॱ൪ʹબΜͰશࢄྔͷྦྷੵد༩Λߟ͑Δ rk = k ∑ i=1 ci = ∑k i=1 λi Vtotal ≥ 0.8 Vtotal ɹதΛ͑Δ·ͰͷɹΛ͏ Vtotal λi -
ओੳ ·ͱΊ Var{sj } = 1 N sT j sj
= 1 N ( ¯ Xaj) T ( ¯ Xaj) = 1 N aT j ¯ XT ¯ Xaj = aT j Σaj ˜ L(aj , λ) = aT j Σaj − λ (∥aj ∥2 − 1) Var{s1 } = Var{ ¯ Xa1 } = aT 1 Σa1 = a1 (Σa1) = aT 1 (λ1 a1) = λ1 aT 1 a1 = λ1 (Σ − λE) aj = 0 a1 , a2 , . . . , ad λ1 ≥ λ2 ≥ . . . ≥ λd ฏۉΛج४ʹͨ͠σʔλͷҰ࣍ؔͷڞࢄߦྻΛݩͷڞࢄߦྻͰදݱ͢Δ ɹͷΛ੍ݶͨ͠ϥάϥϯδϡؔΛ࡞Δ ϥάϥϯδϡ͔ؔΒ༩͑ΒΕͨݻ༗Λղ͘ ٻΊΒΕͨݻ༗ϕΫτϧͰ࡞ΔҰ͔࣍ؔΒࢄΛܭࢉ͢ΔͱରԠ͢Δݻ༗ͱҰக ai ࠷େͷݻ༗ʹରԠ͢Δݻ༗ϕΫτϧ͕σʔλͷࢄΛ࠷େʹ͢ΔઢʹҰக͢Δ -
ಛҟղ ಛҟղ X = UΛVT ·ͣఱԼΓతʹಛҟղΛఆٛ͠·͢ʜ ֤ߦྻͷҰൠతͳதΛ֬ೝ͢Δͱʜ U = (u1
, u2 , . . . , um) V = (v1 , v2 , . . . , vn) Λ = λ1 0 ⋯ 0 0 ⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ λp 0 ⋯ 0 0 0 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋮ ⋮ 0 0 0 ⋯ 0 0 ⋯ 0 ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m
× n X m × m U m × n Λ n × n V (XT X) T = XT X ɹɹɹɹͷ࣮ߦྻͳͷͰɹɹɹɹɹͷ࣮ରশߦྻ XT X X m × n n × n -
ಛҟղ X ҙͷߦྻɹʹରͯ͠ʜ XT Xͷਖ਼نԽͨ͠ݻ༗ϕΫτϧΛɹɹɹɹɹɹɹɹͱ͢Δͱݻ༗Λղ͘͜ͱͰɹ V = (v1 , v2
, . . . , vn) XT Xvi = λi vi Λຬͨ͢ݻ༗ɹΛݻ༗ϕΫτϧɹʹରͯ͠ҰͭఆٛͰ͖Δɽ·ͨɼ λi vi vT i vj = δij = { 1 (i = j) 0 (i ≠ j) ΫϩωοΧʔͷσϧλ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ ݻ༗ɹΛਖ਼ͷͷͱθϩͷͷʹ͚Δ͜ͱ͕Ͱ͖Δ λi vi XT Xvi = vi (λi vi)
= λi vT i vi = λi XT Xvi = λi vi λi = vT i XT Xvi = (Xvi) T Xvi = (Xvi) T Xvi = ∥Xvi ∥2 ≥ 0 ⟺ { λi ≠ 0 (1 ≤ i ≤ p) λi = 0 (p + 1 ≤ i ≤ n) ɹ͕େ͖͍ॱʹฒͯΔͷͰ ɹ͕ޙΖʹฒͿΑ͏ʹ࡞ΕΔ λi 0 ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V X ҙͷߦྻɹʹରͯ͠ʜ XT Xͷਖ਼نԽͨ͠ݻ༗ϕΫτϧΛɹɹɹɹɹɹɹɹͱ͢Δͱݻ༗Λղ͘͜ͱͰɹ V = (v1 , v2 , . . . , vn) -
ಛҟղ ࠓɼɹΛҎԼͷΑ͏ʹఆٛ͢Δͱʜ ui uT i uj = 1 λi λj
(Xvi) T Xvj = 1 λi λj vT i XT Xvj = λj λi λj vT i vj = λj λi λj δij = δij = { 1 (i = j) 0 (i ≠ j) ͳͷͰɼɹɹɹɹɹɹޓ͍ʹަ͢Δਖ਼نަجఈʹͳΔʜ {u1 , u, . . . , um } ui = 1 λi Xvi ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ ͳͷͰɼɹɹɹɹɹɹޓ͍ʹަ͢Δਖ਼نަجఈʹͳΔʜ {u1 , u, . . . , um
} ͳͷͰߦྻɹ U U = [u1 , u2 , . . . , up , up+1 , . . . , um] V = [v1 , v2 , . . . , vn] ·ͨɼߦྻɹ V ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ UTU = I VTV = I ߦྻɹͱɹͱʹਖ਼نަجఈ͔ΒͳΔͷͰ U V
ͳͷͰɼɹɹɹɹɹɹޓ͍ʹަ͢Δਖ਼نަجఈʹͳΔʜ {u1 , u, . . . , um } ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ UT XV = uT 1 uT 2 ⋮ um
(Xv1 Xv2 ⋯ Xvn) = uT 1 Xv1 uT 1 Xv2 ⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn ҙͷߦྻɹʹରͯ͠ߦྻɹɼɹΛ࡞༻ͤ͞Δͱʜ U V X ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ uT i Xvj = ( 1 λi Xvi) T
Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) ٻΊͨߦྻͷҰΛߟ͑Δͱʜ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ UT XV = uT 1 Xv1 uT 1 Xv2
⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn = uT 1 Xv1 uT 1 Xv2 ⋯ uT 1 Xvp uT 1 Xvp+1 ⋯ uT 1 Xvn uT 2 Xv1 uT 2 Xv2 ⋯ uT 2 Xvp uT 2 Xvp+1 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ uT p Xv1 uT p Xv2 ⋯ uT p Xvp uT p Xvp+1 ⋯ uT p Xvn uT p+1 Xv1 uT p+1 Xv2 ⋯ uT p+1 Xvp uT p+1 Xvp+1 ⋯ uT p+1 Xvn ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvp uT m Xvp+1 ⋯ uT m Xvn uT i Xvj = ( 1 λi Xvi) T Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V -
ಛҟղ UT XV = uT 1 Xv1 uT 1 Xv2
⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn = λ1 uT 1 Xv2 ⋯ uT 1 Xvp uT 1 Xvp+1 ⋯ uT 1 Xvn uT 2 Xv1 λ2 ⋯ uT 2 Xvp uT 2 Xvp+1 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ uT p Xv1 uT p Xv2 ⋯ λp uT p Xvp+1 ⋯ uT p Xvn uT p+1 Xv1 uT p+1 Xv2 ⋯ uT p+1 Xvp uT p+1 Xvp+1 ⋯ uT p+1 Xvn ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvp uT m Xvp+1 ⋯ uT m Xvn ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V uT i Xvj = ( 1 λi Xvi) T Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) (i = j, 1 ≤ i ≤ p) λi -
ಛҟղ UT XV = uT 1 Xv1 uT 1 Xv2
⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn = λ1 0 ⋯ 0 0 ⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ λp 0 ⋯ 0 0 0 ⋯ 0 uT p+1 Xvp+1 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ 0 0 ⋯ 0 ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V uT i Xvj = ( 1 λi Xvi) T Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) 0 (i ≠ j) -
ಛҟղ UT XV = uT 1 Xv1 uT 1 Xv2
⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn = λ1 0 ⋯ 0 0 ⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ λp 0 ⋯ 0 0 0 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ 0 0 ⋯ 0 ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V uT i Xvj = ( 1 λi Xvi) T Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) 0 (p + 1 ≤ i ≤ n) -
ಛҟղ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m
× n X m × m U m × n Λ n × n V UT XV = uT 1 Xv1 uT 1 Xv2 ⋯ uT 1 Xvn uT 2 Xv2 uT 2 Xv2 ⋯ uT 2 Xvn ⋮ ⋮ ⋱ ⋮ uT m Xvm uT m Xv2 ⋯ uT m Xvn = λ1 0 ⋯ 0 0 ⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ λp 0 ⋯ 0 0 0 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ 0 0 ⋯ 0 = Λ Λ uT i Xvj = ( 1 λi Xvi) T Xvj = 1 λi vT i XT Xvj = λj λi vT i vj = λi δij = λi (i = j, 1 ≤ i ≤ p) 0 (i ≠ j) 0 (p + 1 ≤ i ≤ n) -
ಛҟղ UT XV = Λ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ
ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V UUT XVVT = IXI = X = UΛVT X = UΛVT ɹͱɹਖ਼نަجఈ͔ΒͳΔߦྻͳͷͰʜ U V -
ಛҟղ ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m
× n X m × m U m × n Λ n × n V X = UΛVT ͱ͜ΖͰɼڵຯ͕͋Δͷݻ༗͕ඇθϩͷ෦ͷΈͰ͋Δ͔Β X = Up Λp VT p = Up λ1 0 ⋯ 0 0 λ2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ λp VT p = ( λ1 u1 vT 1 λ2 u2 vT 2 ⋯ λp up vT p ) ͕ͨͬͯ͠ɼɹɹɹɹߦྻɼɹɹɹɹߦྻɼɹɹɹɹߦྻ ಛʹɹɹɹͷ࣌ɼ͡ύλ Q ಉ༷ʹ͓͘͜ͱ͕Ͱ͖Δɽɹ m × p p × p p × n n = p Up Λp VT p -
ಛҟղ UΛq VT = U λ1 0 ⋯ 0 0
⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ λq 0 ⋯ 0 0 0 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ 0 0 ⋯ 0 VT = Xq ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V ɹͷத͔Βɹɹɹɹɹ·Ͱͷݻ༗Λબ͢Δͱɼ ɹɹҎ߱ͷใΛݮͨ͠σʔλɹ͕ಘΒΕΔ λq (q < p) λi q + 1 Xq -
ಛҟղ UΛq VT = U λ1 0 ⋯ 0 0
⋯ 0 0 λ2 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ λq 0 ⋯ 0 0 0 ⋯ 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 ⋯ 0 0 ⋯ 0 VT = Xq ಛҟղ X = UΛVT ࣮ߦྻɹɹɹߦྻɹΛɹɹɹߦྻɹ ɹɹɹߦྻɹ ɹɹɹߦྻɹͷࡾʹ ղ͢Δ͜ͱɽ m × n X m × m U m × n Λ n × n V ɹͷϥϯΫɹͷޡࠩ࠷খͱ͍͏ҙຯͰͷ࠷ྑۙࣅͱ͍͏ͷ͓ͦΒ͘ɼ ࢄͷখ͍͞ํͷใ͔Β͓ͬͯΓݩͷܗΛ͍ͯ͠Δͱ͍͏ҙຯͰݴ͍ͬͯΔʁ X q -
·ͱΊ ෦ۭؒ Vd=3 x1 x3 x2 Wd=2 ओੳ ˜ L(aj
, λ) = aT j Σaj − λ (∥aj ∥2 − 1) (Σ − λE) aj = 0 a1 , a2 , . . . , ad λ1 ≥ λ2 ≥ . . . ≥ λd X = Up Λp VT p = Up λ1 0 ⋯ 0 0 λ2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ λp VT p = ( λ1 u1 vT 1 λ2 u2 vT 2 ⋯ λp up vT p ) ಛҟղ