mingrammer
August 11, 2017
6.9k

# 머신러닝을 위한 기초 수학 살펴보기

머신러닝을 위한 매우 기초적인 수학과 이를 응용한 선형 회귀 학습 예제

August 11, 2017

## Transcript

1. ݠन۞׬ਸਤೠӝୡࣻ೟࢓ಝࠁӝ
(FUUJOHTUBSUFEUPMFBSOUIFMJOFBSBMHFCSBXJUIQZUIPO

ӂ޹੤

2. ݠन۞׬ਸ ਤೠ ӝୡ ࣻ೟ ࢓ಝࠁӝ
MinJae Kwon (@mingrammer)
2017.08.12
(Getting started to learn the linear algebra with python)

3. Name
ӂ޹੤ (MinJae Kwon)
Nickname
@mingrammer
Email
[email protected]
Who
Game Server Engineer @ SundayToz
Blog
https://mingrammer.com
Github
https://github.com/mingrammer
Eng Blog
https://medium.com/@mingrammer

4. 2. ࢶഋ؀ࣻ೟੉ۆ?
4. ୶о ӝୡ ࣻ೟ ࢓ಝࠁӝ
Contents
5. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
1. ࣻ೟੄ ೙ਃࢿ
3. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
6. Next (more LA and Mathematics)

5. ࣻ೟੄ ೙ਃࢿ
੉ߣࣁ࣌਷઱۽ӝୡ ࢶഋ؀
ࣻ೟ਸ׮ܖ޲۽
׮਺ࣻधٜ੉੊ࣼೞ૑ঋ਷ٜ࠙ਸ؀࢚ਵ۽೤פ׮
W ←W −α
∂L
∂W
E = − 1
N
t
nk
log(y
nk
)
k

n

d(u
!
u
!
T
) = 2u
!

6. ࣻ೟੄ ೙ਃࢿ
ਢ ѐߊ
জ ѐߊ
API ѐߊ
ࣻ೟

T(x)

∂θ f (x,θ)dx

−log(t)y(t)

x∇f (x)
1
σ 2π
e
−(x−µ)2
2σ 2
−∞

7. ࣻ೟੄ ೙ਃࢿ
ਢ ѐߊ
জ ѐߊ
API ѐߊ
ࣻ೟

*UEFQFOETPOjCVU
T(x)

∂θ f (x,θ)dx

−log(t)y(t)

x∇f (x)
1
σ 2π
e
−(x−µ)2
2σ 2
−∞

8. ࣻ೟੄ ೙ਃࢿ
ߑޙ੗ࣻ҅ஏ
୶ୌঌҊ્ܻ
ঐഐࢸ҅
঑୷ঌҊ્ܻ
%ݽ؛݂
ѱ੐ূ૓
୭ࣗ࠺ਊঌҊ્ܻ
೐۽ࣁझझாે݂
Ӓې೗୊ܻ
ഛܫݽ؛
ҊബਯҊࢿמ҅࢑ࢸ҅
ݠन۞׬
ؘ੉ఠ߬੉झ
नഐ୊ܻ

9. ࣻ೟੄ ೙ਃࢿ
ݠन۞׬
ߑޙ੗ࣻ҅ஏ
୶ୌঌҊ્ܻ
ঐഐࢸ҅
঑୷ঌҊ્ܻ
%ݽ؛݂
ѱ੐ূ૓
୭ࣗ࠺ਊঌҊ્ܻ
೐۽ࣁझझாે݂
Ӓې೗୊ܻ
ഛܫݽ؛
ҊബਯҊࢿמ҅࢑ࢸ҅
ؘ੉ఠ߬੉झ
नഐ୊ܻ

10. ࣻ೟੄ ೙ਃࢿ
.BDIJOF-FBSOJOH
"MHPSJUINT
5SBJO%BUB .PEFMT
7BMJEBUJPO%BUB
0VUQVUT
/FX%BUB

11. ࣻ೟੄ ೙ਃࢿ
6 7 8 ... 5
7 5 6 ... 3
8 4 1 ... 4
... ... ... ... 2
0 1 5 4 3

W = W −α
∂L
∂W
0 0 1 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 0 0 1

0 1 0 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 1 0 0

4 1 5 ... 8

1
2
(y
k
− t
k
)2

y
k
= eak
eai
∑ 0.5 0.4 ... ... 0.8
0.1 −0.3 ... ... 0.2
... ... ... ... 0.1
... ... ... ... 0.43
0.03 0.23 0.1 0.3 0.13

0 1 0 ... 0

Y = X ⋅W + B

12. ࣻ೟੄ ೙ਃࢿ
6 7 8 ... 5
7 5 6 ... 3
8 4 1 ... 4
... ... ... ... 2
0 1 5 4 3

W = W −α
∂L
∂W
0 0 1 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 0 0 1

0 1 0 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 1 0 0

4 1 5 ... 8

1
2
(y
k
− t
k
)2

y
k
= eak
eai
∑ 0.5 0.4 ... ... 0.8
0.1 −0.3 ... ... 0.2
... ... ... ... 0.1
... ... ... ... 0.43
0.03 0.23 0.1 0.3 0.13

Y = X ⋅W + B
ࢶഋ؀ࣻ೟ ࢶഋ؀ࣻ೟ ࢶഋ؀ࣻ೟
ഛܫҗా҅ ഛܫҗా҅
޷੸࠙೟
0 1 0 ... 0

13. ࣻ೟੄ ೙ਃࢿ
6 7 8 ... 5
7 5 6 ... 3
8 4 1 ... 4
... ... ... ... 2
0 1 5 4 3

W = W −α
∂L
∂W
0 0 1 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 0 0 1

0 1 0 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 1 0 0

4 1 5 ... 8

1
2
(y
k
− t
k
)2

y
k
= eak
eai
∑ 0.5 0.4 ... ... 0.8
0.1 −0.3 ... ... 0.2
... ... ... ... 0.1
... ... ... ... 0.43
0.03 0.23 0.1 0.3 0.13

Y = X ⋅W + B
ഛܫҗా҅ ޷੸࠙೟
ࢶഋ؀ࣻ೟
ؘ੉ఠ಴അ
҅࢑੄ബਯࢿ
୭੸ച
ౠࢿ୶୹
ഛܫ࠙ನ
୶ۿ߂৘ஏ
оࢸѨૐ
೧ࢳ೟੸੽Ӕ
ӝ਎ӝ҅࢑
ಞ޷࠙
੿ӏച
0 1 0 ... 0

14. ࢶഋ؀ࣻ೟੉ۆ?
߭ఠҕр
x
11
x
12
x
13
x
14
x
15
x
21
x
22
x
23
x
24
x
25
x
31
x
32
x
33
x
34
x
35
x
41
x
42
x
43
x
44
x
45
x
51
x
52
x
53
x
54
x
55

T
y
1
y
2
y
3
y
4
y
5

−3 −2 6

2 2 8

߭ఠো࢑
೯۳ো࢑
ରਗ
ࢶഋߑ੿ध
ݠन۞׬
ঐഐച
੿ࠁѨ࢝
੉޷૑೐۽ࣁय
؀ӏݽ୊ܻ

15. ਋ܻח ࢶഋ؀ࣻܳ ׮ܖӝ ਤ೧ Numpyܳ ࢎਊ೤פ׮
ৈӝח౵੉௑੉૑݅
ࣽࣻ1ZUIPO਷ખוܿ
/VNQZח௏যо\$'PSUSBOӝ߈੉ۄࡅܴ
ࡅܳࡺ݅ইפۄߓৌਸബਯ੸ਵ۽୊ܻ೧ݫݽܻب؏ࢎਊೣ
пઙಞܻೠҊࣻળੋఠಕ੉झ৬بҳٜਸઁҕ
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

16. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ (vector)
↟ ߭ఠ
ҕрীࢲ੄ਗࣗܳ಴അ
↟߭ఠחпਗࣗ੄ؘ੉ఠఋੑ੉زੌ
↟ࠁాBSSBZ۽಴അ OVNQZBSSBZ

17. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ (vector)
x = x
1
x
2
... x
n

⎦ x =
x
1
x
2
...
x
n

೯߭ఠ SPXWFDUPS
ৌ߭ఠ DPMVNOWFDUPS

18. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ (vector)

19. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ ো࢑
↟ؔࣅ BEEJUJPO

↟ࡓࣅ TVCUSBDUJPO

↟झணۄғ 4DBMBS1SPEVDU

↟ࢿ࠙ғ &MFNFOUXJTF.VMUJQMJDBUJPO

↟ղ੸ *OOFS1SPEVDU

߭ఠח׮਺੄ো࢑ٜਸࣻ೯ೡࣻ੓਺

20. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ ো࢑ 1
↟ؔࣅ BEEJUJPO

↟ࡓࣅ TVCUSBDUJPO

v + u = (v
1
+ u
1
,...,v
n
+ u
n
)
v − u = (v
1
− u
1
,...,v
n
− u
n
)

21. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ ো࢑ 1

22. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ ো࢑ 2
↟झணۄғ 4DBMBS1SPEVDU

↟ࢿ࠙ғ &MFNFOUXJTF.VMUJQMJDBUJPO

↟ղ੸ *OOFS1SPEVDU

v⊗u = (v
1
u
1
,...,v
n
u
n
)
av = (av
1
,...,av
n
)
v⋅u = v
i
u
i
i=1
n

23. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
߭ఠ ো࢑ 2

24. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
೯۳ (Matrix)
↟NYO੄ഋకܳо૗
↟п೯ৌਸ߭ఠ۽಴അоמ
↟ࠁాରਗBSSBZ۽಴അ

25. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
೯۳ ো࢑
↟ؔࣅ BEEJUJPO

↟ࡓࣅ TVCUSBDUJPO

↟झணۄғ 4DBMBS1SPEVDU

↟ࢿ࠙ғ &MFNFOUXJTF.VMUJQMJDBUJPO

↟೯۳ғ .BUSJY.VMUJQMJDBUJPO

೯۳਷׮਺੄ো࢑ٜਸࣻ೯ೡࣻ੓਺

26. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
೯۳ ো࢑ 1
↟ؔࣅ BEEJUJPO

↟ࡓࣅ TVCUSBDUJPO

X +Y =
x
11
+ y
11
... x
1n
+ y
1n
... ... ...
x
m1
+ y
m1
... x
mn
+ y
mn

X −Y =
x
11
− y
11
... x
1n
− y
1n
... ... ...
x
m1
− y
m1
... x
mn
− y
mn

27. ೯۳ ো࢑ 1
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

28. ೯۳ ো࢑ 2
↟झணۄғ 4DBMBS1SPEVDU

↟ࢿ࠙ғ &MFNFOUXJTF.VMUJQMJDBUJPO

aX =
ax
11
... ax
1n
... ... ...
ax
m1
... ax
mn

X ⊗Y =
x
11
y
11
... x
1n
y
1n
... ... ...
x
m1
y
m1
... x
mn
y
mn

NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

29. ೯۳ ো࢑ 2
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

30. ೯۳ ো࢑ 3
x
11
x
12
x
13
x
14
x
21
x
22
x
23
x
24
x
31
x
32
x
33
x
34

೯۳ғ .BUSJY.VMUJQMJDBUJPO

=
y
11
y
12
y
13
y
21
y
22
y
23
y
31
y
32
y
33
y
41
y
42
y
43

x
11
y
11
+ x
12
y
21
+ x
13
y
31
+ x
14
y
41
... ...
... ... ...
... ... ...

Y Y Y
=
Y
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

31. ೯۳ ো࢑ 3
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

32. ੹஖ (Transpose)
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
XT =
x
11
x
12
x
13
x
14
x
21
x
22
x
23
x
24
x
31
x
32
x
33
x
34

T
=
x
11
x
12
x
13
x
21
x
22
x
23
x
31
x
32
x
33
x
41
x
42
x
43

xT = x
1
x
2
... x
n

T
=
x
1
x
2
...
x
n

33. ੹஖ (Transpose)
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

34. ױਤ ೯۳ (Identity Matrix)
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
I
n
=
1 0 0 ... 0
0 1 0 ... 0
0 0 1 ... 0
... ... ... ... 0
0 0 0 0 1

35. NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
ױਤ ೯۳ (Identity Matrix)

36. ৉೯۳ (Inverse Matrix)
XX−1 = I
n
= X−1X
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ
৉೯۳਷ ೦࢚ ઓ੤ೞ૑ח ঋ਺

37. ৉೯۳ (Inverse Matrix)
NumPy۽ ࢶഋ؀ࣻ೟ ׮ܞࠁӝ

38. ୶о ӝୡ ࣻ೟ ࢓ಝࠁӝ
੿ӏച 1
y
i
= x
i
x
i
∑ ੹୓ sumਵ۽ ա׃ਵ۽ॄ [0, 1] ҳрਵ۽ ੿ӏച

39. ୶о ӝୡ ࣻ೟ ࢓ಝࠁӝ
੿ӏച 1

40. ӝఋ ӝୡ ࣻ೟ ࢓ಝࠁӝ
੿ӏച 2
Xi
= X
i
− E(X)
σ
E(X) = 1
N
X
i
i=1
N

σ = 1
N
(X
i
− E(X))2
i=1
N

಴ળ ੿ӏ ࠙ನܳ ഝਊ೧ ಣӐ = 0, ࠙࢑ = 1۽ ੿ӏച

41. ӝఋ ӝୡ ࣻ೟ ࢓ಝࠁӝ
੿ӏച 2

42. ӝఋ ӝୡ ࣻ೟ ࢓ಝࠁӝ
޷࠙ ߂ Ӓۄ٣঱౟
1
2
3
0 0 1 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 0 0 1

0 1 0 ... 0
1 0 0 ... 0
0 1 0 ... 0
... ... ... ... 0
0 0 1 0 0

৘ஏ ݽ؛ীࢲ੄ Ѿҗчҗ पઁ Ѿҗчҗ੄

рӓ੄ ૑಴ੋ ର੉ “ࣚप"ਸ ઴੉ӝ ਤೣ
ࣚप ೣࣻ Ӓې೐

43. ӝఋ ӝୡ ࣻ೟ ࢓ಝࠁӝ
ಞ޷࠙
f (x) = g(x
1
)+...+ g(x
n
)
∂ f (x)
∂x
1
=
∂g(x
1
)
∂x
1 ׮߸ࣻ ҕр ഑਷ ೣࣻীࢲ ౠ੿ ߸ࣻܳ ؀࢚ਵ۽ ޷࠙

ݠन ۞׬ীࢶ ࠁా ೯۳ਸ ؀࢚ਵ۽ೠ ೯۳ ಞ޷࠙ ࢎਊ
∂L
∂X
=
∂L
∂x
11
...
∂L
∂x
1n
... ... ...
∂L
∂x
m1
...
∂L
∂x
mn

44. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
ਤ੄ѐ֛ٜਸഝਊ೧рױೠ
ࢶഋഥӈݽ؛ਸٜ݅যࠁѷणפ׮

45. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
୭੸੄ ࢶഋ ࢚ҙ ҙ҅ܳ ಴അೞח ૒ࢶਸ ଺ਵ۰Ҋ ೣ → ୭੸੄ ߬ఋܳ ೟ण
y = β
0
x
0
+ β
1
x
1
y
i
= x
i
T β

46. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
y = β
0
x
0
+ β
1
x
1
ױੌؘ੉ఠ

47. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
y = β
0
x
0
+ β
1
x
1
y
i
= x
i0
β
0
+ x
i1
β
1
ױੌؘ੉Tఠ ׮઺ؘ੉ఠ઺Jߣ૩ؘ੉ఠ

48. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
y = β
0
x
0
+ β
1
x
1
y
i
= x
i
T β
y
i
= x
i0
β
0
+ x
i1
β
1
ױੌؘ੉ఠ ׮઺ؘ੉ఠ઺Jߣ૩ؘ੉ఠ Jߣ૩ؘ੉ఠ߭ఠ಴അ

49. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Y = Xβ Y =
y
1
y
2
...
y
n

X =
x
1
T
x
2
T
...
x
n
T

β =
β
0
β
1

ੌ߈ചػ׮઺ؘ੉ఠ੄೯۳಴അ

50. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
1.0
1.1
1.2
...
10.0

51. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
1.0
1.1
1.2
...
10.0

0.1
0.11
0.12
...
1.0

52. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
y = ax + b = b + ax
y
1
= b + ax
1
y
2
= b + ax
2

53. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
1 0.1
1 0.11
1 0.12
1 ...
1 1.0

54. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
r
1
r
2
r
3
...
r
100

1 0.1
1 0.11
1 0.12
1 ...
1 1.0

55. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
r
1
r
2
r
3
...
r
100

1 x
i1
1 x
i2
1 x
i3
... ...
1 x
i20

1 0.1
1 0.11
1 0.12
1 ...
1 1.0

56. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
1 0.1
1 0.11
1 0.12
... ...
1 1.0

w
0
w
1

57. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
1 0.1
1 0.11
1 0.12
... ...
1 1.0

w
0
w
1

y
1
y
2
y
3
...
y
20

w
0
+ 0.1w
1
w
0
+ 0.11w
1
w
0
+ 0.12w
1
...
w
0
+1.0w
1

T

58. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
1 0.1
1 0.11
1 0.12
... ...
1 1.0

w
0
w
1

y
1
y
2
y
3
...
y
20

w
0
+ 0.1w
1
w
0
+ 0.11w
1
w
0
+ 0.12w
1
...
w
0
+1.0w
1

T
1
20
error
i
2
i=1
20
∑ = 1
20
(y
i
− e
i
)2
i=1
20

59. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
1 0.1
1 0.11
1 0.12
... ...
1 1.0

w
0
w
1

y
1
y
2
y
3
...
y
20

w
0
+ 0.1w
1
w
0
+ 0.11w
1
w
0
+ 0.12w
1
...
w
0
+1.0w
1

T
1
20
error
i
2
i=1
20
∑ = 1
20
(y
i
− e
i
)2
i=1
20

∂(mse)
∂W
= [error][x]

60. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
∂(mse)
∂W
೯۳ಞ޷࠙→ ೯۳੄ п ਗࣗী ؀೧ ಞ޷࠙

61. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
∂(mse)
∂w
0
=
∂( (y
i
− (w
0
+ x
i
w
1
))2 )

∂w
0
= (y
i
− (w
0
+ x
i
w
1
))⋅1

∂(mse)
∂w
1
=
∂( (y
i
− (w
0
+ x
i
w
1
))2 )

∂w
1
= (y
i
− (w
0
+ x
i
w
1
))⋅ x
i

62. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
Ӓۄ٣঱౟ ࢸ҅
∂(mse)
∂W
= [ y
i
− (w
0
+ x
i
w
1
)
∑ ]
1 0.1
1 0.11
1 0.12
... ...
1 1.0

= [error][x]

63. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
पઁ ೟ण ױ҅
W ←W −α
∂(mse)
∂W

64. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
೟ण Ӓې೐
<ഥ>&SSPS <ഥ>&SSPS
<ഥ>&SSPS <ഥ>&SSPS

65. ࢶഋ ഥӈ ҳഅ೧ࠁӝ
୭ઙ ೟ण Ѿҗ
&SSPS
β =
−2.152736
11.075026

y
k
= x
k
T β

66. Next (more LA and Mathematics)
↟઱ࢿ࠙࠙ࢳ 1\$"

↟ױੌч࠙೧ 47%

↟-6࠙೧
↟ҊਬчҊਬ߭ఠ
↟಴ળച
↟j
↟߬੉૑উా҅
↟ഛܫӏ஗
↟ࢠ೒݂ߑध
↟୭؀਋بஏ੿
↟j
↟೯۳޷੸࠙೟
↟Ӓۄ٣঱౟
↟j

67. хࢎ೤פ׮
MinJae Kwon (@mingrammer)
Getting started to learn the linear algebra with python
2017.08.12