Deep NLP: 딥러닝을 이용한 자연어처리

Slide 1

Slide 1 text

Deep NLP ٩۞׬ਸ ੉ਊೠ ੗োয୊ܻ ѐߊ੗ܳ ਤೠ ӝࣿ ইਓۄੋҗ ৘ઁ ઺बਵ۽ ӣ࢚਋ @ VCNC [email protected]

Slide 2

Slide 2 text

Why?? • ਋ܻо ೞҊ ੓ח ݽ߄ੌ জ ࠺ૉפझח അ੤о ੿੼ • 5֙, 10֙ ٍীח যڄө? • ݠन۞׬਷ ਋ܻ ࢤഝী ௾ ߸ചܳ оઉৢ ࣻ ੓ח ӝࣿ ઺ ೞա • 100֙ оח ӝস੉ غ۰ݶ ഥࢎب, ࢎۈب ߸ न੉ ೙ਃ • ׮ٜ प۱җ ҃೷ ੓ਵ޲۽ ޤٚ ٮۄ੟ইࢲ ੜ ೡ ࣻ ੓ѷ૑݅, ޷ܻ ળ࠺ೞҊ ҙब о૑ݶ જ ਸ Ѫ

Slide 3

Slide 3 text

ݠन۞׬ • ஹೊఠ җ೟੄ ೠ ࠙ঠ۽, যڃ ۽૒ਸ ೐۽Ӓې߁ ೞ૑ ঋҊب ஹೊ ఠо ੘স੉ա ૑धਸ ߓ਎ ࣻ ੓ѱ ೞח ࠙ঠ • ؀ࠗ࠙ Data-driven prediction ഑਷ decision ਸ ࣻ೯ೞח ঌҊ ્ܻٜ • ؘ੉ఠ ࠙ࢳ ࠙ঠীࢲب օܻ ॳ੉חؘ, ࠂ੟ೠ ݽ؛ਸ ࠙ࢳоо ૒੽ ٜ݅૑ ঋҊ ؘ੉ఠܳ ా೧ ೟णदெ যځೠ ࢎपਸ ৘ஏೞѢա, ऀѹ ૓ ੋࢎ੉౟ܳ ঌইղח ੌ ١ਸ ࣻ೯

Slide 4

Slide 4 text

٩۞׬ (Deep Learning) • ݠन۞׬੄ ೞਤ ࠙ঠ ઺ ೞա, ౠ൤ Neural Network (ੋҕ न҃ ݎ) ҅ৌ ঌҊ્ܻҗ োҙ੓਺ • Deep Neural Network ੉ۄҊ ࠗܰӝب ೣ • ୭Ӕ ಩ߊ੸ਵ۽ ҙबਸ ߉Ҋ ੓ח ࠙ঠ۽, ӝઓ ݠन۞׬ ߑߨۿ੄ ೠ҅ܳ ӓࠂೞח ઺

Slide 5

Slide 5 text

٩۞׬ (Deep Learning) • Rawೠ input (Ӗ੗, ܻࣗ, ױ য ١١) ਸ ੑ۱߉਺ • ઺р઺р ৈ۞ ҅கਵ۽ ؘ੉ఠ ٜਸ ઑ೤ೞৈ ಴അ • ҅கਸ ੼੼ ऺই ৢ۰оݴ ࢚ ਤ ѐ֛ਸ ಴അೡ ࣻ ੓ѱ ؽ

Slide 6

Slide 6 text

ӝઓ ݠन۞׬ ߑߨٜ੄ ೠ҅ • അ੤੄ ݠन۞׬ ߑߨٜ਷ ؀ࠗ࠙ Data-Driven ߑߨٜ • যڃ ੌਸ द೯ೞח ۽૒ਸ ೐۽Ӓې߁ೞחѪ੉ ইפۄ, ೟णೡ ࣻ ੓ ח ҳઑܳ ٜ݅য֬Ҋ ؘ੉ఠܳ ઁҕ, ஹೊఠо ঌইࢲ ೟ण • যڃ ؘ੉ఠܳ, যڃ ҳઑ۽ ೟णदఃוջо ઺ਃ! • ౠ൤ ؘ੉ఠ ࠗ࠙੄ ઺ਃࢿ੉ ௼Ҋ ؘ੉ఠܳ যڌѱ оҕ೧ࢲ ֍וջ о ݽ؛੄ ࢿמਸ ઝ਋ೣ -> Feature Engineering

Slide 7

Slide 7 text

Feature Engineering • ؘ੉ఠܳ оҕ೧ࢲ ஹೊఠܳ ೟णदఆ featureܳ ݅٘۰ݶ, ೧׼ ࠙ ঠী ؀ೠ ੹ޙ੸ੋ ૑ध (domain knowledge)੉ ೙ਃೞݴ दрҗ ࠺ਊ੉ ݆੉ ٘ח ੘স੐ • ਕ௼೒۽਋ - Feature engineering -> Machine training -> Evaluate result -> Idea -> Feature engineering -> …

Slide 8

Slide 8 text

٩۞׬਷ ޤо ׮ܲо? • ٩۞׬ (Deep Neural Network) • ಴അ۱੉ જ਷ (ࠂ੟ೠ ղਊٜਸ ೟णೡ ࣻ ੓ח) ֎౟ਕ௼ܳ ৈ۞க ऺইࢲ, ֎౟ਕ௼о featureٜਸ झझ۽ ٜ݅যղҊ ೟णೡ ࣻ ੓ب ۾ ೞ੗!

Slide 9

Slide 9 text

঱যܳ ਤೠ ٩۞׬ • ٩۞׬ ݽ؛ਸ ഝਊೠ ਺ࢿੋध • ӝઓ੄ ਺ࢿੋध ߑߨ ؀࠺ ࢿמ੉ ࠺ড੸ਵ۽ ೱ࢚ؽ

Slide 10

Slide 10 text

੉޷૑ܳ ਤೠ ٩۞׬ • ୭Ӕ ٩۞׬ਸ ഝਊ೧ о੢ ݆੉ োҳо غҊ ੓ח ࠙ঠ • ImageNet ؀ഥীࢲ જ਷ ࢿ੸ਸ ղݶࢲ ݆਷ ҙबਸ ঳਺

Slide 11

Slide 11 text

੗োয ୊ܻীࢲ੄ ٩۞׬ ਽ਊ • ఫझ౟੄ х੿࠙ࢳ (ӛ੿/ࠗ੿) • ੋҕ૑מ ࠺ࢲ • ૕੄਽׹ • ੗ز ߣ৉

Slide 12

Slide 12 text

٩۞׬ - х੿࠙ࢳ • ੉੹੄ ߑߨ - ৈ۞ ױযٜ੉ ӛ੿/ࠗ੿੄ х੿ਸ о૑ח૑ DBҳ୷ - ޙ੢ী ೧׼ ױযٜ੉ ঴݃ա ١੢ೞח૑ ଺ӝ - ੤޷о ੓׮ - ੤޷о হ׮ - “੤޷” ۄח ױয݅ਵ۽ח ౸ױೡ ࣻ হ਺ • ٩۞׬ਸ ഝਊೠ ߑߨ - ޙߨ੉ա ੄޷੸ੋ Ѫө૑ Ҋ۰ೞৈ ౸߹ оמ

Slide 13

Slide 13 text

٩۞׬ - ૕੄਽׹ • ੉੹੄ ߑߨ - ݆ࣻ਷ ૕ޙ-׹߸ DBܳ ҳ୷ೞҊ, ࢜۽ ٜযয়ח ૕ޙਸ ੉੹੄ ૕׹ ࣇ ীࢲ ଺਺ • ٩۞׬ਸ ഝਊೠ ߑߨ - ৈ۞ ૕ޙী ؀ೠ ׹੉ vector ഋधਵ۽ ݽ؛ী ֣ইٜয੓਺

Slide 14

Slide 14 text

٩۞׬ - ੗زߣ৉ • ੉੹੄ ߑߨ - ই઱ ࠂ੟ೠ ৈ۞ ҅க੄ ߣ৉ ݽ؛ • ٩۞׬ਸ ഝਊೠ ߑߨ - ߣ৉ೡ ޙ੢ਸ vector۽ ಴അೞҊ, ੉ܳ ߸ജೞח ݽ؛ਸ ೟णदఇ

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

ݠन۞׬ ѐਃ

Slide 17

Slide 17 text

ݠन۞׬ (Machine Learning) • “The ﬁeld of study that gives computers the ability to learn without being explicitly programmed.” - Arthur Samuel • “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” - Tom Mitchell • ৘: ߄قѱ੐ਸ فח ஹೊఠ - E = ߄قѱ੐ਸ فח ҃೷஖ - T = ߄قѱ੐ਸ فח ೯ਤ - P = ஹೊఠо ׮਺౸੄ ߄قਸ ੉ӡ ഛܫ

Slide 18

Slide 18 text

ݠन۞׬ (Machine Learning) • Supervised Learning (૑ب ೟ण) - Regression - োࣘ੸ੋ чਸ ৘ஏ • ৘: ࠗز࢑ оѺ ৘ஏ - Classiﬁcation - ࠛোࣘ੸ੋ ч ৘ஏ • ৘: ࢎ૓ਸࠁҊ ࢎޛ ݏ୶ӝ - Regressionޙઁب Classiﬁcation ޙઁ۽ ߄Չࣻب ੓׮! • ৘: ࠗز࢑ оѺਸ ࠁҊ ࢓૑ উ࢓૑ Ѿ੿ • Unsupervised Learning (࠺૑ب ೟ण) - Clustering • Reinforcement Learning (ъച ೟ण) - ѥחߨਸ झझ۽ ߓ਋ח ۽ࠈ

Slide 19

Slide 19 text

Linear Regression

Slide 20

Slide 20 text

Linear Regression • ૘੄ և੉ ؘ੉ఠ۽ ࠗز࢑ оѺਸ ৘ஏ೧ ࠁ੗ • 30 ಣ = 3র 5000݅ਗ • 50ಣ = 5র 5000݅ਗ • 10ಣ = 1র 5000݅ਗ • 20ಣ = ?

Slide 21

Slide 21 text

Linear Regression • y = a * x + b - x: և੉(ಣ) - y: оѺ - h(x) ৬ э੉ ಴അೞӝب ೣ (hypothesis, оࢸ) - ৬ э੉ ಴അ • Training dataܳ learning algorithmী ੸ਊ, h ܳ ҳೣ! • h ܳ ੉ਊೞৈ x ী ؀ೠ Ѿҗч (y) ܳ ৘ஏ оמ

Slide 22

Slide 22 text

Cost Function • যڃ о੿ hܳ ࣁਛਸٸ, Ӓ о੿੉ ঴݃ա ౣ۷חо(error) ܳ ࠺ਊ (cost)۽ р઱ೞৈ ࣻधਸ ࣁ਑ • ੉ۧѱ ੿੄ೠ Costܳ ୭ࣗചೞח ߑೱਵ۽ ౵ۄݫఠ a,bٜਸ ઑ੿ ೞݶ ખ؊ ৢ߄ܲ о੿ h ܳ о૕ ࣻ ੓׮!

Slide 23

Slide 23 text

Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • पઁ ؘ੉ఠח h(x) = 1*x ੋؘ, h(x) = 0.5*x ۽ ৘ஏೠ ҃਋ • য়ରч (Error, Cost): 0 1 2 3 3 2 1 0

Slide 24

Slide 24 text

Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • ੄ 2ରߑ੿ध!!

Slide 25

Slide 25 text

Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • , ী ؀೧ࢲח ইې৬ э਷ ੑ୓੸ੋ ݽন੉ ػ׮ • যڌѱ ୭੷੼ਸ ଺ਸѪੋо?

Slide 26

Slide 26 text

Gradient Descent • য়ରчਸ যڌѱ ୭ࣗചೡѪੋо? - Cost functionਸ ୭ࣗച - Cost function ١Ҋࢶ੄ Ҏ૞ӝ ଺ӝ • ੐੄੄ ૑੼ীࢲ द੘ೞৈ, ӝ਎ӝо ծই૑ח ૑੼ଃਵ۽ ҅ࣘ ੉زೞ੗ • ӝ਎ӝ = Cost function੄ ޷࠙ч

Slide 27

Slide 27 text

Gradient Descent • ੉ җ੿ਸ ࣻ۴ೡٸө૑ ߈ࠂ • a: learning rate

Slide 28

Slide 28 text

Cost Function੄ ޷࠙ч

Slide 29

Slide 29 text

Cost Function੄ ޷࠙ч

Slide 30

Slide 30 text

Linear Regression੄ Gradient Descent

Slide 31

Slide 31 text

Linear Regression੄ Gradient Descent • Bowl-shaped convex function

Slide 32

Slide 32 text

Linear Regression੄ Gradient Descent • Iterationਸ ా೧ ୭੸੼ਸ ଺ইх (for fixed , this is a function of x) (function of the parameters )

Slide 33

Slide 33 text

Linear Regression੄ Gradient Descent (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters )

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Multi Variable Linear Regression • ࠗز࢑ оѺਸ ৘ஏೞחؘח ಣࣻ ݈Ҋب ׮নೠ ੗ܐܳ ଵҊೡ ࣻ ੓ ਸѪ - ߑ੄ іࣻ, ળҕ֙ب, ߓۆ׮ іࣻ ١١.. • ݽ؛੉ ׮೦ध੄ ഋకо ؽ • Feature scaling, feature ઑ೤, ઺ࠂ feature ઁѢ ١ ݆਷ ੘স੉ ೙ਃೞѱ ؽ

Slide 36

Slide 36 text

Logistic Regression

Slide 37

Slide 37 text

Classiﬁcation: Logistic Regression • োࣘ੸ੋ чਸ ৘ஏೞחѪ੉ ইצ, ࠛোࣘ੸ੋ чਸ ৘ஏ - झಅੋ૑ ইצ૑ - ঐੋ૑ ইצ૑ - ੉ ࢎ૓੉ Ҋন੉ੋ૑, ъই૑ੋ૑ Tumor Size Tumor Size (Yes) 1 (No) 0

Slide 38

Slide 38 text

Classiﬁcation: Logistic Regression • Sigmoid (Logistic function) ਸ ࢎਊೞৈ և਷ ߧਤ੄ чਸ, 0~1 ࢎ੉੄ чਵ۽ ߸ജ • ݅ড 0.5ܳ ӝળਵ۽ ೠ׮ݶ, 0.5 ੉࢚:1, 0.5 ޷݅: 0

Slide 39

Slide 39 text

Logistic Cost Function • y = 0 ੌٸ৬ y = 1 ੌٸо ׮ܴ • Gradient Descentܳ ഝਊೞח ߑߨ਷ Linear Regressionҗ زੌ!

Slide 40

Slide 40 text

Regularization • Overﬁtting (җ୭੸ച) ޙઁ - അपࣁ҅੄ पઁ ݽ؛ࠁ׮, ೟णदఅ ݽ؛੉ ࠂ੟بо ؊ ֫ইઉࢲ ઱য૓ ೟णࣇী݅ җبೞѱ ୭੸ചػ ࢚ട - training set ੉ ইצ ࢜۽਍ ޙઁܳ ઁ؀۽ ৘ஏೡ ࣻ হѱ ػ׮ • Feature ੄ іࣻܳ ઴੉Ѣա, Regularizationਵ۽ ೧Ѿ • Regularization - Cost functionী ੄ب੸ਵ۽ ࢚ࣻ೦ਸ ୶оदெ ೟णਸ ߑ೧ೣ - җبೞѱ ೟णغחѪਸ ߑ૑ೞח ബҗ

Slide 41

Slide 41 text

Neural Network

Slide 42

Slide 42 text

Neural Network • ੋр੄ ֱ ҳઑܳ ٮۄೠ ҳઑ • Dendrite: ׮ܲ ׏۠ਵ۽ࠗఠ नഐܳ ߉਺ • Cell body: ݽٚ ੑ۱ਸ ઙ೤ೣ • Axon: ੑ۱੄ ೤੉ ੐҅੼ਸ ֈਵݶ, ׮ܲ ׏۠ਵ۽ नഐܳ ࠁն • Synapses: ׮ܲ ׏۠җ োѾؽ. োѾ ъبী ٮۄ नഐ੄ ࣁӝо ߄Պ - नഐо ъ೧૑Ѣա, ড೧૑חѪ = ೟ण

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Neural Network

Slide 45

Slide 45 text

Neural Network

Slide 46

Slide 46 text

Neural Network Playground • http://playground.tensorﬂow.org/ • 0 hidden layer, 2 neurons (Logistic Regression) - ױࣽೠ 1ର ഋక੄ ؘ੉ఠ߆ী ೟णदః૑ ޅೣ • 2 hidden layer, 2 neurons - Ѿҗо যڌѱ ׮ܲо? - ৈ۞க ऺইب Ѿҗо ࠺तೣ • neuronіࣻܳ טܻݶ ખ؊ ࠂ੟ೠ ഋకܳ ಴അೡ ࣻ ੓׮ • Featureܳ טܻݶ? • Learning rate੉ ௼ݶ? • Deep ೠ ֎౟ਖীࢲ activation function? • Regularization

Slide 47

Slide 47 text

Neural Network - Cost Function Layer 1 Layer 2 Layer 3 Layer 4 Logistic regression: Neural network: • Logistic Regression੄ ഛ੢౸!

Slide 48

Slide 48 text

Neural Network - Back Propagation • Forward propagation Layer 1 Layer 2 Layer 3 Layer 4

Slide 49

Slide 49 text

Neural Network - Back Propagation • Back propagation Layer 1 Layer 2 Layer 3 Layer 4 Intuition: “error” of node in layer . For each output unit (layer L = 4)

Slide 50

Slide 50 text

Neural Network - ੿ܻ • ֎౟ਕ௼ ҳઑܳ ੿ೠ׮ - Input feature੄ іࣻ, Output class੄ іࣻ, hidden layer੄ іࣻ ߂ ରਗ • weightܳ ےؒೞѱ ୡӝച • forward propagation, cost function, back propagation ਸ ҳ അ • ݽٚ ؘ੉ఠٜী ؀ೞৈ, back propagationਸ ా೧ ҳೠ gradient ٜਸ ݽ؛ী ੸ਊ • TensorFlowо ׮ ঌইࢲ ೧઻ਃ

Slide 51

Slide 51 text

(੤޷۽) ੋр੄ ֱ vs. ׏ۡ֎౟ਕ௼ • ੋр੄ ֱ - ড ୌর ѐ੄ ׏۠, ࣻߔ~ࣻୌઑ ѐ੄ दշझ۽ ੉ܖয૗ - ੋрҗ زޛ੄ ֱח ׮׮੊ࢶ • ׏ۡ֎౟ਕ௼ - ੋҕ੸ੋ ׏ۡ֎౟ਕ௼੄ ࢎ੉ૉח ࢚؀੸ਵ۽ ੘਺ - ࢎ੉ૉо ௿ࣻ۾ ೟ण੉ য۰ਕ૗ • AlphaGo - ৈ۞ѐ੄ ׏ۡ ֎౟ਕ௼੄ ઑ೤. (13 layer੄ Convolutional neural network ١) - Input layer: 48ѐ੄ 19x19 ߄ق౸ = 17328 ׏۠ - ࣻߔ݅ѐ੄ ׏۠

Slide 52

Slide 52 text

঱য ݽ؛җ RNN (Recurrent Neural Networks)

Slide 53

Slide 53 text

RNN? • !?!?!? • RNN੉ ޤӡې, ੉۠Ѫਸ ೡࣻ੓חо • RNNਸ Ҿӓ੄ ੋҕ न҃ ݎ ҳઑۄҊ ઱੢ೞח ࢎ ۈٜب ੓਺

Slide 54

Slide 54 text

RNN (Recurrent Neural Networks) • ݆਷ NLP ੘সী ই઱ જ਷ ࢿמਸ ղષ • ѐ֛੉ য۵ӝب ೞҊ ઁ؀۽ ࢸݺೞҊ ੓ח ੗ܐо হ਺ - ઁ؀۽ ੉೧ೞח ࢎۈب ੜ হחѪ э਺ • য়טب ઁ؀۽ ࢸݺೡ ࣻ ੓ਸ૑ ഛप൤ ݽܰѷ਺

Slide 55

Slide 55 text

঱য ݽ؛ (Language Model) • োࣘػ ױযٜ੄ ١੢ഛܫਸ ৘ஏೞח ݽ؛ • ߓо Ҋ౵ࢲ աח ߏਸ ____ - ___ী ٜযт ױযח? - ݡ঻׮, ݡח׮, ૑঻׮,… - p(ݡ঻׮) = 0.25, p(ݡח׮) = 0.15, p(૑঻׮) = 0.05, .. - p(੗زର) = 0.000001 • ޙ੢੄ ࣽࢲա ױযࢶఖ ١ਸ ৘ஏೡ ࣻ ੓ӝী, ੗زߣ৉ ١ ݆਷ NLP Taskী ਬਊೞ׮

Slide 56

Slide 56 text

঱য ݽ؛ (Language Model) • ഛܫ੸ ঱য ݽ؛ਸ ݅٘۰ݶ, ੉੹ী աয়ח ױযܳ ݆੉ ଵઑೡࣻ۾ ੿ഛೞ׮! - 2-gram, 3-gram, 4-gram, .. n-gram • ߓо Ҋ౵ࢲ աח ߏਸ ____ - ߏਸ ____ <-> ߓо Ҋ౵ࢲ աח ߏਸ ____ • n-gramਸ טܾࣻ۾ ݫݽܻо ষ୒աѱ ೙ਃೞ׮! - ੉ ޙઁܳ ೧Ѿ೧઱חѪ੉ RNN(Recurrent Neural Networks)

Slide 57

Slide 57 text

RNN (Recurrent Neural Networks) • दрী ٮۄ (഑਷ ؘ੉ఠ੄ ૓೯ী ٮۄ) ҅ࣘ সؘ੉౟غח ࢚కчਸ р૒ೞҊ ੓ח ֎౟ਕ௼ • ױࣽೠ ࢚కо ইפۄ, ೟णਸ ૑ࣘೡࣻ۾ ੑ۱ਸ ୊ܻೞח ҭ੢൤ ࠂ੟ೠ ۽૒੉ ֣ ইٜѱؽ • ࢜۽ աৢ ױযܳ ৘ஏೞחؘী ੉੹੄ ݽٚ ױযٜ੄ ੿ࠁܳ ଵઑೞѱ ؽ

Slide 58

Slide 58 text

RNN (Recurrent Neural Networks) • ߓо Ҋ౵ࢲ աח ߏਸ ____ - 4ѐ੄ ױযܳ ଵઑ - RNNীࢲח 4 layer neural network੉ ࢤӣ

Slide 59

Slide 59 text

RNN (Recurrent Neural Networks) • RNNਸ ೟णदఃחѪ਷ য۵׮! • Vanishing Gradient ޙઁ - ցޖ য়ې੹੄ ੿ࠁө૑ ଵઑೞ۰׮ࠁפ, য٣ࢲ ੜޅ೮঻ח૑ ೟णೞӝ য۵ѱ ؽ • Exploding Gradient ޙઁ • ReLU / Clipping ١ ৈ۞о૑ ప௼ץਵ۽ ӓࠂ!

Slide 60

Slide 60 text

RNN (Recurrent Neural Networks) • ؊ ੗ࣁೠ ೟ण੗ܐ - http://www.wildml.com/2015/09/recurrent-neural-networks- tutorial-part-1-introduction-to-rnns/

Slide 61

Slide 61 text

LSTM (Long Short-Term Memory) • RNNҗ ਬࢎೠ ҳઑ੄ ֎౟ਕ௼ • RNN਷ ߈ࠂ ҳઑ۽ ੋ೧ ӝর۱ਸ(memory) о૑ѱ غ঻਺ • दр୷ਵ۽ ಽযࠁݶ, ৈ۞ѐ੄ ֎౟ਕ௼о ੓Ҋ п੗о ੗न੄ ׮਺ ֎౟ਕ௼ী ݫࣁ૑ܳ ࠁղחѪҗ э਷ ਗܻ • োࣘػ ؘ੉ఠ৬ Ө਷ োҙ - ঱য ݽ؛, ߣ৉, ਺ࢿੋध ١١ RNNҗ ੉ܳ दр୷ਵ۽ ಽ੉ೠ Ӓܿ

Slide 62

Slide 62 text

LSTM (Long Short-Term Memory) • RNN਷ ୭Ӕ ݻѐ ױয ੿ب੄ ӝর۱ ੿ب݅ਸ о૗ • LSTM: ؊਌ ӟ ӝর۱ਸ о૑ӝ ਤೠ ҳઑ

Slide 63

Slide 63 text

LSTM (Long Short-Term Memory) • Cell State: п ױ҅݃׮ ই઱ ੸਷ ࣻ੿ਸ Ѣ஖ݴ ੿ࠁܳ য়ۖزউ ࠁઓೠ׮.

Slide 64

Slide 64 text

LSTM (Long Short-Term Memory) • Forget gate: ੉੹ cell state઺ী ࠛ೙ਃೠ ੿ࠁܳ ઁѢ • Input gate: ࢜۽਍ ੿ࠁܳ cell stateী ӝর

Slide 65

Slide 65 text

LSTM (Long Short-Term Memory) • Forget gate৬ input gate੄ ઑ೤ਵ۽ cell state ܳ সؘ੉౟ • Output: cell state੄ ੌࠗ࠙ਵ۽ чਸ ୹۱

Slide 66

Slide 66 text

GRU (Gated Recurrent Unit) • LSTMҗ Ѣ੄ ਬࢎೠ ҳઑ, ؊ рױೠ ҳഅ - Forget gate ৬ input gate ܳ ೞա۽ ೤ஜ

Slide 67

Slide 67 text

Attention Mechanism • ੋр੄ ੋ૑җ੿ਸ ࢤп೧ࠁݶ, ݽٚ ࣽрী ݽٚࠗ࠙ী न҃ਸ ॳח Ѫ੉ ইפۄ, ౠ੿ ࣽрী ౠ੿ ࢎޛ੉ա ઱ઁী ૘઺ೞח ݽणਸ ࠁ ੐ • RNNэ਷ ݽ؛੄ ӡ੉о ӡয૕ࣻ۾ ೞա੄ Hidden Stateী ݽٚ ղਊਸ ӝরೞӝо য۰ਕ૗ • Hidden stateٜਸ ੷੢ೞҊ ੓׮о, ੉੹੄ stateٜਸ ഝਊ

Slide 68

Slide 68 text

؊ ࠂ੟ೠ ֎౟ਕ௼

Slide 69

Slide 69 text

Dynamic Memory Networks • যڃ Ӗਸ ੍Ҋ, Ӓ Ӗী ؀ೠ ޛ਺ী ׹ਸ ೞחѪ - ࣻמद೷ ঱য৔৉җ ࠺तೣ - औ૑ ঋ਷ җઁ੉׮!

Slide 70

Slide 70 text

Dynamic Memory Networks • ӝמ߹۽ RNNਸ ઑ೤ • End-to-End ೟ण • ֤ޙਸ ੍੗! - https://arxiv.org/abs/1506.07285

Slide 71

Slide 71 text

Dynamic Memory Networks • Input Module - RNN (GRU), п ޙ੢ٜ੄ hidden stateܳ ੷੢ • Question Module - рױೠ RNN (GRU) • Episodic Memory Module - ੉ঠӝী ؀ೠ ੿ࠁܳ ҙ੢ೞח ݽٕ - ৈ۞ க੄ RNN (GRU) ۽ ҳࢿ

Slide 72

Slide 72 text

Dynamic Memory Networks • ׮ܲ ఋੑ੄ Input Moduleਸ ࠢ੉חѪب оמ!

Slide 73

Slide 73 text

Dynamic Memory Networks • ׮ܲ ఋੑ੄ Input Moduleਸ ࠢ੉חѪب оמ!

Slide 74

Slide 74 text

Dynamic Memory Networks

Slide 75

Slide 75 text

Dynamic Memory Networks

Slide 76

Slide 76 text

Dynamic Memory Networks

Slide 77

Slide 77 text

Dynamic Memory Networks • Attention + Memoryܳ ా೧ ૕ޙী ׹ೞח מ۱ਸ ഛ੢ • ӝמ߹۽ RNNਵ۽ ੉ܖয૓ ߹ب੄ Module, ੑ۱ ഋకо ׮ܲ ݽ ٕ۽ ߸҃ೞחѪب оמೣ • ੉޷૑ ੑ۱ ݽٕҗ Ѿ೤غݶ ֥ۄ਍ Ѿҗܳ ࠁৈષ

Slide 78

Slide 78 text

NMT (Neural Machine Translation) • ࠂ੟ೠ ֎౟ਕ௼ҳઑܳ ഝਊೠ ੗زߣ৉ ֎౟ਕ௼

Slide 79

Slide 79 text

Google NMT • https://translate.google.com • https://research.googleblog.com/2016/09/a-neural-network-for- machine.html

Slide 80

Slide 80 text

٩۞׬ ੸ਊী ؀ೠ ࢤп • ٩۞׬ = ই੉٣য + ؘ੉ఠ + ݽ؛ ೟ण - ݽ؛ ೟ण - ੼੼ ੷۴೧૗ - ই੉٣য - ੼੼ ؊ ݆਷ ࢎۈٜ੉ ઙࢎೞݶࢲ ই੉٣যب ݆ই૕ Ѫ - ؘ੉ఠ - ݠन۞׬, ٩۞׬ द؀ীࢲ о੢ ൞ࣗೠ ੗ਗ • ؀ӏݽ ؘ੉ఠܳ ഝਊೠ ݠन۞׬, ؊ ࠂ੟ೠ ઑ೤੄ ݽ؛, ࠙ঠ߹ ௼ ۽झ ݽ؛ ١ ই૒ ߊ੹ оמࢿ੉ ޖҾޖ૓ೣ • ҕࠗೞҊ োҳ೤द׮!

Slide 81

Slide 81 text

More..? • Join VCNC and join #study_ml ੷൞ח ঱ઁա ࠺౟ਦ ࢲ࠺झܳ ೣԋ ٜ݅ݴ ӝࣿ੸ੋ ޙઁܳ ೣԋ ಽযաт מ۱੓ח ѐߊ੗ܳ ݽदҊ ੓णפ׮. ঱ઁٚ ࠗ׸হ੉ [email protected]۽ ੉ݫੌਸ ઱दӝ ߄ۉפ׮!

Slide 82

Slide 82 text

Thank you!