Deep NLP: 딥러닝을 이용한 자연어처리

6a11050c8147e4f5fbf2637907c27964?s=47 VCNC
December 20, 2017

Deep NLP: 딥러닝을 이용한 자연어처리

머신러닝, 딥러닝 2학기 수업분량을 2시간으로 요약
VCNC 개발팀 워크샵 (2017/12/20)

6a11050c8147e4f5fbf2637907c27964?s=128

VCNC

December 20, 2017
Tweet

Transcript

  1. Deep NLP ٩۞׬ਸ ੉ਊೠ ੗োয୊ܻ ѐߊ੗ܳ ਤೠ ӝࣿ ইਓۄੋҗ ৘ઁ

    ઺बਵ۽ ӣ࢚਋ @ VCNC kevin@between.us
  2. Why?? • ਋ܻо ೞҊ ੓ח ݽ߄ੌ জ ࠺ૉפझח അ੤о ੿੼

    • 5֙, 10֙ ٍীח যڄө? • ݠन۞׬਷ ਋ܻ ࢤഝী ௾ ߸ചܳ оઉৢ ࣻ ੓ח ӝࣿ ઺ ೞա • 100֙ оח ӝস੉ غ۰ݶ ഥࢎب, ࢎۈب ߸ न੉ ೙ਃ • ׮ٜ प۱җ ҃೷ ੓ਵ޲۽ ޤٚ ٮۄ੟ইࢲ ੜ ೡ ࣻ ੓ѷ૑݅, ޷ܻ ળ࠺ೞҊ ҙब о૑ݶ જ ਸ Ѫ
  3. ݠन۞׬ • ஹೊఠ җ೟੄ ೠ ࠙ঠ۽, যڃ ۽૒ਸ ೐۽Ӓې߁ ೞ૑

    ঋҊب ஹೊ ఠо ੘স੉ա ૑धਸ ߓ਎ ࣻ ੓ѱ ೞח ࠙ঠ • ؀ࠗ࠙ Data-driven prediction ഑਷ decision ਸ ࣻ೯ೞח ঌҊ ્ܻٜ • ؘ੉ఠ ࠙ࢳ ࠙ঠীࢲب օܻ ॳ੉חؘ, ࠂ੟ೠ ݽ؛ਸ ࠙ࢳоо ૒੽ ٜ݅૑ ঋҊ ؘ੉ఠܳ ా೧ ೟णदெ যځೠ ࢎपਸ ৘ஏೞѢա, ऀѹ ૓ ੋࢎ੉౟ܳ ঌইղח ੌ ١ਸ ࣻ೯
  4. ٩۞׬ (Deep Learning) • ݠन۞׬੄ ೞਤ ࠙ঠ ઺ ೞա, ౠ൤

    Neural Network (ੋҕ न҃ ݎ) ҅ৌ ঌҊ્ܻҗ োҙ੓਺ • Deep Neural Network ੉ۄҊ ࠗܰӝب ೣ • ୭Ӕ ಩ߊ੸ਵ۽ ҙबਸ ߉Ҋ ੓ח ࠙ঠ۽, ӝઓ ݠन۞׬ ߑߨۿ੄ ೠ҅ܳ ӓࠂೞח ઺
  5. ٩۞׬ (Deep Learning) • Rawೠ input (Ӗ੗, ܻࣗ, ױ য

    ١١) ਸ ੑ۱߉਺ • ઺р઺р ৈ۞ ҅கਵ۽ ؘ੉ఠ ٜਸ ઑ೤ೞৈ ಴അ • ҅கਸ ੼੼ ऺই ৢ۰оݴ ࢚ ਤ ѐ֛ਸ ಴അೡ ࣻ ੓ѱ ؽ
  6. ӝઓ ݠन۞׬ ߑߨٜ੄ ೠ҅ • അ੤੄ ݠन۞׬ ߑߨٜ਷ ؀ࠗ࠙ Data-Driven

    ߑߨٜ • যڃ ੌਸ द೯ೞח ۽૒ਸ ೐۽Ӓې߁ೞחѪ੉ ইפۄ, ೟णೡ ࣻ ੓ ח ҳઑܳ ٜ݅য֬Ҋ ؘ੉ఠܳ ઁҕ, ஹೊఠо ঌইࢲ ೟ण • যڃ ؘ੉ఠܳ, যڃ ҳઑ۽ ೟णदఃוջо ઺ਃ! • ౠ൤ ؘ੉ఠ ࠗ࠙੄ ઺ਃࢿ੉ ௼Ҋ ؘ੉ఠܳ যڌѱ оҕ೧ࢲ ֍וջ о ݽ؛੄ ࢿמਸ ઝ਋ೣ -> Feature Engineering
  7. Feature Engineering • ؘ੉ఠܳ оҕ೧ࢲ ஹೊఠܳ ೟णदఆ featureܳ ݅٘۰ݶ, ೧׼

    ࠙ ঠী ؀ೠ ੹ޙ੸ੋ ૑ध (domain knowledge)੉ ೙ਃೞݴ दрҗ ࠺ਊ੉ ݆੉ ٘ח ੘স੐ • ਕ௼೒۽਋ - Feature engineering -> Machine training -> Evaluate result -> Idea -> Feature engineering -> …
  8. ٩۞׬਷ ޤо ׮ܲо? • ٩۞׬ (Deep Neural Network) • ಴അ۱੉

    જ਷ (ࠂ੟ೠ ղਊٜਸ ೟णೡ ࣻ ੓ח) ֎౟ਕ௼ܳ ৈ۞க ऺইࢲ, ֎౟ਕ௼о featureٜਸ झझ۽ ٜ݅যղҊ ೟णೡ ࣻ ੓ب ۾ ೞ੗!
  9. ঱যܳ ਤೠ ٩۞׬ • ٩۞׬ ݽ؛ਸ ഝਊೠ ਺ࢿੋध • ӝઓ੄

    ਺ࢿੋध ߑߨ ؀࠺ ࢿמ੉ ࠺ড੸ਵ۽ ೱ࢚ؽ
  10. ੉޷૑ܳ ਤೠ ٩۞׬ • ୭Ӕ ٩۞׬ਸ ഝਊ೧ о੢ ݆੉ োҳо

    غҊ ੓ח ࠙ঠ • ImageNet ؀ഥীࢲ જ਷ ࢿ੸ਸ ղݶࢲ ݆਷ ҙबਸ ঳਺
  11. ੗োয ୊ܻীࢲ੄ ٩۞׬ ਽ਊ • ఫझ౟੄ х੿࠙ࢳ (ӛ੿/ࠗ੿) • ੋҕ૑מ

    ࠺ࢲ • ૕੄਽׹ • ੗ز ߣ৉
  12. ٩۞׬ - х੿࠙ࢳ • ੉੹੄ ߑߨ - ৈ۞ ױযٜ੉ ӛ੿/ࠗ੿੄

    х੿ਸ о૑ח૑ DBҳ୷ - ޙ੢ী ೧׼ ױযٜ੉ ঴݃ա ١੢ೞח૑ ଺ӝ - ੤޷о ੓׮ - ੤޷о হ׮ - “੤޷” ۄח ױয݅ਵ۽ח ౸ױೡ ࣻ হ਺ • ٩۞׬ਸ ഝਊೠ ߑߨ - ޙߨ੉ա ੄޷੸ੋ Ѫө૑ Ҋ۰ೞৈ ౸߹ оמ
  13. ٩۞׬ - ૕੄਽׹ • ੉੹੄ ߑߨ - ݆ࣻ਷ ૕ޙ-׹߸ DBܳ

    ҳ୷ೞҊ, ࢜۽ ٜযয়ח ૕ޙਸ ੉੹੄ ૕׹ ࣇ ীࢲ ଺਺ • ٩۞׬ਸ ഝਊೠ ߑߨ - ৈ۞ ૕ޙী ؀ೠ ׹੉ vector ഋधਵ۽ ݽ؛ী ֣ইٜয੓਺
  14. ٩۞׬ - ੗زߣ৉ • ੉੹੄ ߑߨ - ই઱ ࠂ੟ೠ ৈ۞

    ҅க੄ ߣ৉ ݽ؛ • ٩۞׬ਸ ഝਊೠ ߑߨ - ߣ৉ೡ ޙ੢ਸ vector۽ ಴അೞҊ, ੉ܳ ߸ജೞח ݽ؛ਸ ೟णदఇ
  15. None
  16. ݠन۞׬ ѐਃ

  17. ݠन۞׬ (Machine Learning) • “The field of study that gives

    computers the ability to learn without being explicitly programmed.” - Arthur Samuel • “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” - Tom Mitchell • ৘: ߄قѱ੐ਸ فח ஹೊఠ - E = ߄قѱ੐ਸ فח ҃೷஖ - T = ߄قѱ੐ਸ فח ೯ਤ - P = ஹೊఠо ׮਺౸੄ ߄قਸ ੉ӡ ഛܫ
  18. ݠन۞׬ (Machine Learning) • Supervised Learning (૑ب ೟ण) - Regression

    - োࣘ੸ੋ чਸ ৘ஏ • ৘: ࠗز࢑ оѺ ৘ஏ - Classification - ࠛোࣘ੸ੋ ч ৘ஏ • ৘: ࢎ૓ਸࠁҊ ࢎޛ ݏ୶ӝ - Regressionޙઁب Classification ޙઁ۽ ߄Չࣻب ੓׮! • ৘: ࠗز࢑ оѺਸ ࠁҊ ࢓૑ উ࢓૑ Ѿ੿ • Unsupervised Learning (࠺૑ب ೟ण) - Clustering • Reinforcement Learning (ъച ೟ण) - ѥחߨਸ झझ۽ ߓ਋ח ۽ࠈ
  19. Linear Regression

  20. Linear Regression • ૘੄ և੉ ؘ੉ఠ۽ ࠗز࢑ оѺਸ ৘ஏ೧ ࠁ੗

    • 30 ಣ = 3র 5000݅ਗ • 50ಣ = 5র 5000݅ਗ • 10ಣ = 1র 5000݅ਗ • 20ಣ = ?
  21. Linear Regression • y = a * x + b

    - x: և੉(ಣ) - y: оѺ - h(x) ৬ э੉ ಴അೞӝب ೣ (hypothesis, оࢸ) - ৬ э੉ ಴അ • Training dataܳ learning algorithmী ੸ਊ, h ܳ ҳೣ! • h ܳ ੉ਊೞৈ x ী ؀ೠ Ѿҗч (y) ܳ ৘ஏ оמ
  22. Cost Function • যڃ о੿ hܳ ࣁਛਸٸ, Ӓ о੿੉ ঴݃ա

    ౣ۷חо(error) ܳ ࠺ਊ (cost)۽ р઱ೞৈ ࣻधਸ ࣁ਑ • ੉ۧѱ ੿੄ೠ Costܳ ୭ࣗചೞח ߑೱਵ۽ ౵ۄݫఠ a,bٜਸ ઑ੿ ೞݶ ખ؊ ৢ߄ܲ о੿ h ܳ о૕ ࣻ ੓׮!
  23. Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • पઁ ؘ੉ఠח h(x) =

    1*x ੋؘ, h(x) = 0.5*x ۽ ৘ஏೠ ҃਋ • য়ରч (Error, Cost): 0 1 2 3 3 2 1 0
  24. Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • ੄ 2ରߑ੿ध!!

  25. Linear Regression(ױੌ ߸ࣻ)੄ Cost Function • , ী ؀೧ࢲח ইې৬

    э਷ ੑ୓੸ੋ ݽন੉ ػ׮ • যڌѱ ୭੷੼ਸ ଺ਸѪੋо?
  26. Gradient Descent • য়ରчਸ যڌѱ ୭ࣗചೡѪੋо? - Cost functionਸ ୭ࣗച

    - Cost function ١Ҋࢶ੄ Ҏ૞ӝ ଺ӝ • ੐੄੄ ૑੼ীࢲ द੘ೞৈ, ӝ਎ӝо ծই૑ח ૑੼ଃਵ۽ ҅ࣘ ੉زೞ੗ • ӝ਎ӝ = Cost function੄ ޷࠙ч
  27. Gradient Descent • ੉ җ੿ਸ ࣻ۴ೡٸө૑ ߈ࠂ • a: learning

    rate
  28. Cost Function੄ ޷࠙ч

  29. Cost Function੄ ޷࠙ч

  30. Linear Regression੄ Gradient Descent

  31. Linear Regression੄ Gradient Descent • Bowl-shaped convex function

  32. Linear Regression੄ Gradient Descent • Iterationਸ ా೧ ୭੸੼ਸ ଺ইх (for

    fixed , this is a function of x) (function of the parameters )
  33. Linear Regression੄ Gradient Descent (for fixed , this is a

    function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters )
  34. Linear Regression੄ Gradient Descent (for fixed , this is a

    function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters ) (for fixed , this is a function of x) (function of the parameters )
  35. Multi Variable Linear Regression • ࠗز࢑ оѺਸ ৘ஏೞחؘח ಣࣻ ݈Ҋب

    ׮নೠ ੗ܐܳ ଵҊೡ ࣻ ੓ ਸѪ - ߑ੄ іࣻ, ળҕ֙ب, ߓۆ׮ іࣻ ١١.. • ݽ؛੉ ׮೦ध੄ ഋకо ؽ • Feature scaling, feature ઑ೤, ઺ࠂ feature ઁѢ ١ ݆਷ ੘স੉ ೙ਃೞѱ ؽ
  36. Logistic Regression

  37. Classification: Logistic Regression • োࣘ੸ੋ чਸ ৘ஏೞחѪ੉ ইצ, ࠛোࣘ੸ੋ чਸ

    ৘ஏ - झಅੋ૑ ইצ૑ - ঐੋ૑ ইצ૑ - ੉ ࢎ૓੉ Ҋন੉ੋ૑, ъই૑ੋ૑ Tumor Size Tumor Size (Yes) 1 (No) 0
  38. Classification: Logistic Regression • Sigmoid (Logistic function) ਸ ࢎਊೞৈ և਷

    ߧਤ੄ чਸ, 0~1 ࢎ੉੄ чਵ۽ ߸ജ • ݅ড 0.5ܳ ӝળਵ۽ ೠ׮ݶ, 0.5 ੉࢚:1, 0.5 ޷݅: 0
  39. Logistic Cost Function • y = 0 ੌٸ৬ y =

    1 ੌٸо ׮ܴ • Gradient Descentܳ ഝਊೞח ߑߨ਷ Linear Regressionҗ زੌ!
  40. Regularization • Overfitting (җ୭੸ച) ޙઁ - അपࣁ҅੄ पઁ ݽ؛ࠁ׮, ೟णदఅ

    ݽ؛੉ ࠂ੟بо ؊ ֫ইઉࢲ ઱য૓ ೟णࣇী݅ җبೞѱ ୭੸ചػ ࢚ട - training set ੉ ইצ ࢜۽਍ ޙઁܳ ઁ؀۽ ৘ஏೡ ࣻ হѱ ػ׮ • Feature ੄ іࣻܳ ઴੉Ѣա, Regularizationਵ۽ ೧Ѿ • Regularization - Cost functionী ੄ب੸ਵ۽ ࢚ࣻ೦ਸ ୶оदெ ೟णਸ ߑ೧ೣ - җبೞѱ ೟णغחѪਸ ߑ૑ೞח ബҗ
  41. Neural Network

  42. Neural Network • ੋр੄ ֱ ҳઑܳ ٮۄೠ ҳઑ • Dendrite:

    ׮ܲ ׏۠ਵ۽ࠗఠ नഐܳ ߉਺ • Cell body: ݽٚ ੑ۱ਸ ઙ೤ೣ • Axon: ੑ۱੄ ೤੉ ੐҅੼ਸ ֈਵݶ, ׮ܲ ׏۠ਵ۽ नഐܳ ࠁն • Synapses: ׮ܲ ׏۠җ োѾؽ. োѾ ъبী ٮۄ नഐ੄ ࣁӝо ߄Պ - नഐо ъ೧૑Ѣա, ড೧૑חѪ = ೟ण
  43. Neural Network • ੋр੄ ֱ ҳઑܳ ٮۄೠ ҳઑ • Dendrite:

    ׮ܲ ׏۠ਵ۽ࠗఠ नഐܳ ߉਺ • Cell body: ݽٚ ੑ۱ਸ ઙ೤ೣ • Axon: ੑ۱੄ ೤੉ ੐҅੼ਸ ֈਵݶ, ׮ܲ ׏۠ਵ۽ नഐܳ ࠁն • Synapses: ׮ܲ ׏۠җ োѾؽ. োѾ ъبী ٮۄ नഐ੄ ࣁӝо ߄Պ - नഐо ъ೧૑Ѣա, ড೧૑חѪ = ೟ण
  44. Neural Network

  45. Neural Network

  46. Neural Network Playground • http://playground.tensorflow.org/ • 0 hidden layer, 2

    neurons (Logistic Regression) - ױࣽೠ 1ର ഋక੄ ؘ੉ఠ߆ী ೟णदః૑ ޅೣ • 2 hidden layer, 2 neurons - Ѿҗо যڌѱ ׮ܲо? - ৈ۞க ऺইب Ѿҗо ࠺तೣ • neuronіࣻܳ טܻݶ ખ؊ ࠂ੟ೠ ഋకܳ ಴അೡ ࣻ ੓׮ • Featureܳ טܻݶ? • Learning rate੉ ௼ݶ? • Deep ೠ ֎౟ਖীࢲ activation function? • Regularization
  47. Neural Network - Cost Function Layer 1 Layer 2 Layer

    3 Layer 4 Logistic regression: Neural network: • Logistic Regression੄ ഛ੢౸!
  48. Neural Network - Back Propagation • Forward propagation Layer 1

    Layer 2 Layer 3 Layer 4
  49. Neural Network - Back Propagation • Back propagation Layer 1

    Layer 2 Layer 3 Layer 4 Intuition: “error” of node in layer . For each output unit (layer L = 4)
  50. Neural Network - ੿ܻ • ֎౟ਕ௼ ҳઑܳ ੿ೠ׮ - Input

    feature੄ іࣻ, Output class੄ іࣻ, hidden layer੄ іࣻ ߂ ରਗ • weightܳ ےؒೞѱ ୡӝച • forward propagation, cost function, back propagation ਸ ҳ അ • ݽٚ ؘ੉ఠٜী ؀ೞৈ, back propagationਸ ా೧ ҳೠ gradient ٜਸ ݽ؛ী ੸ਊ • TensorFlowо ׮ ঌইࢲ ೧઻ਃ
  51. (੤޷۽) ੋр੄ ֱ vs. ׏ۡ֎౟ਕ௼ • ੋр੄ ֱ - ড

    ୌর ѐ੄ ׏۠, ࣻߔ~ࣻୌઑ ѐ੄ दշझ۽ ੉ܖয૗ - ੋрҗ زޛ੄ ֱח ׮׮੊ࢶ • ׏ۡ֎౟ਕ௼ - ੋҕ੸ੋ ׏ۡ֎౟ਕ௼੄ ࢎ੉ૉח ࢚؀੸ਵ۽ ੘਺ - ࢎ੉ૉо ௿ࣻ۾ ೟ण੉ য۰ਕ૗ • AlphaGo - ৈ۞ѐ੄ ׏ۡ ֎౟ਕ௼੄ ઑ೤. (13 layer੄ Convolutional neural network ١) - Input layer: 48ѐ੄ 19x19 ߄ق౸ = 17328 ׏۠ - ࣻߔ݅ѐ੄ ׏۠
  52. ঱য ݽ؛җ RNN (Recurrent Neural Networks)

  53. RNN? • !?!?!? • RNN੉ ޤӡې, ੉۠Ѫਸ ೡࣻ੓חо • RNNਸ

    Ҿӓ੄ ੋҕ न҃ ݎ ҳઑۄҊ ઱੢ೞח ࢎ ۈٜب ੓਺
  54. RNN (Recurrent Neural Networks) • ݆਷ NLP ੘সী ই઱ જ਷

    ࢿמਸ ղષ • ѐ֛੉ য۵ӝب ೞҊ ઁ؀۽ ࢸݺೞҊ ੓ח ੗ܐо হ਺ - ઁ؀۽ ੉೧ೞח ࢎۈب ੜ হחѪ э਺ • য়טب ઁ؀۽ ࢸݺೡ ࣻ ੓ਸ૑ ഛप൤ ݽܰѷ਺
  55. ঱য ݽ؛ (Language Model) • োࣘػ ױযٜ੄ ١੢ഛܫਸ ৘ஏೞח ݽ؛

    • ߓо Ҋ౵ࢲ աח ߏਸ ____ - ___ী ٜযт ױযח? - ݡ঻׮, ݡח׮, ૑঻׮,… - p(ݡ঻׮) = 0.25, p(ݡח׮) = 0.15, p(૑঻׮) = 0.05, .. - p(੗زର) = 0.000001 • ޙ੢੄ ࣽࢲա ױযࢶఖ ١ਸ ৘ஏೡ ࣻ ੓ӝী, ੗زߣ৉ ١ ݆਷ NLP Taskী ਬਊೞ׮
  56. ঱য ݽ؛ (Language Model) • ഛܫ੸ ঱য ݽ؛ਸ ݅٘۰ݶ, ੉੹ী

    աয়ח ױযܳ ݆੉ ଵઑೡࣻ۾ ੿ഛೞ׮! - 2-gram, 3-gram, 4-gram, .. n-gram • ߓо Ҋ౵ࢲ աח ߏਸ ____ - ߏਸ ____ <-> ߓо Ҋ౵ࢲ աח ߏਸ ____ • n-gramਸ טܾࣻ۾ ݫݽܻо ষ୒աѱ ೙ਃೞ׮! - ੉ ޙઁܳ ೧Ѿ೧઱חѪ੉ RNN(Recurrent Neural Networks)
  57. RNN (Recurrent Neural Networks) • दрী ٮۄ (഑਷ ؘ੉ఠ੄ ૓೯ী

    ٮۄ) ҅ࣘ সؘ੉౟غח ࢚కчਸ р૒ೞҊ ੓ח ֎౟ਕ௼ • ױࣽೠ ࢚కо ইפۄ, ೟णਸ ૑ࣘೡࣻ۾ ੑ۱ਸ ୊ܻೞח ҭ੢൤ ࠂ੟ೠ ۽૒੉ ֣ ইٜѱؽ • ࢜۽ աৢ ױযܳ ৘ஏೞחؘী ੉੹੄ ݽٚ ױযٜ੄ ੿ࠁܳ ଵઑೞѱ ؽ
  58. RNN (Recurrent Neural Networks) • ߓо Ҋ౵ࢲ աח ߏਸ ____

    - 4ѐ੄ ױযܳ ଵઑ - RNNীࢲח 4 layer neural network੉ ࢤӣ
  59. RNN (Recurrent Neural Networks) • RNNਸ ೟णदఃחѪ਷ য۵׮! • Vanishing

    Gradient ޙઁ - ցޖ য়ې੹੄ ੿ࠁө૑ ଵઑೞ۰׮ࠁפ, য٣ࢲ ੜޅ೮঻ח૑ ೟णೞӝ য۵ѱ ؽ • Exploding Gradient ޙઁ • ReLU / Clipping ١ ৈ۞о૑ ప௼ץਵ۽ ӓࠂ!
  60. RNN (Recurrent Neural Networks) • ؊ ੗ࣁೠ ೟ण੗ܐ - http://www.wildml.com/2015/09/recurrent-neural-networks-

    tutorial-part-1-introduction-to-rnns/
  61. LSTM (Long Short-Term Memory) • RNNҗ ਬࢎೠ ҳઑ੄ ֎౟ਕ௼ •

    RNN਷ ߈ࠂ ҳઑ۽ ੋ೧ ӝর۱ਸ(memory) о૑ѱ غ঻਺ • दр୷ਵ۽ ಽযࠁݶ, ৈ۞ѐ੄ ֎౟ਕ௼о ੓Ҋ п੗о ੗न੄ ׮਺ ֎౟ਕ௼ী ݫࣁ૑ܳ ࠁղחѪҗ э਷ ਗܻ • োࣘػ ؘ੉ఠ৬ Ө਷ োҙ - ঱য ݽ؛, ߣ৉, ਺ࢿੋध ١١ RNNҗ ੉ܳ दр୷ਵ۽ ಽ੉ೠ Ӓܿ
  62. LSTM (Long Short-Term Memory) • RNN਷ ୭Ӕ ݻѐ ױয ੿ب੄

    ӝর۱ ੿ب݅ਸ о૗ • LSTM: ؊਌ ӟ ӝর۱ਸ о૑ӝ ਤೠ ҳઑ
  63. LSTM (Long Short-Term Memory) • Cell State: п ױ҅݃׮ ই઱

    ੸਷ ࣻ੿ਸ Ѣ஖ݴ ੿ࠁܳ য়ۖزউ ࠁઓೠ׮.
  64. LSTM (Long Short-Term Memory) • Forget gate: ੉੹ cell state઺ী

    ࠛ೙ਃೠ ੿ࠁܳ ઁѢ • Input gate: ࢜۽਍ ੿ࠁܳ cell stateী ӝর
  65. LSTM (Long Short-Term Memory) • Forget gate৬ input gate੄ ઑ೤ਵ۽

    cell state ܳ সؘ੉౟ • Output: cell state੄ ੌࠗ࠙ਵ۽ чਸ ୹۱
  66. GRU (Gated Recurrent Unit) • LSTMҗ Ѣ੄ ਬࢎೠ ҳઑ, ؊

    рױೠ ҳഅ - Forget gate ৬ input gate ܳ ೞա۽ ೤ஜ
  67. Attention Mechanism • ੋр੄ ੋ૑җ੿ਸ ࢤп೧ࠁݶ, ݽٚ ࣽрী ݽٚࠗ࠙ী न҃ਸ

    ॳח Ѫ੉ ইפۄ, ౠ੿ ࣽрী ౠ੿ ࢎޛ੉ա ઱ઁী ૘઺ೞח ݽणਸ ࠁ ੐ • RNNэ਷ ݽ؛੄ ӡ੉о ӡয૕ࣻ۾ ೞա੄ Hidden Stateী ݽٚ ղਊਸ ӝরೞӝо য۰ਕ૗ • Hidden stateٜਸ ੷੢ೞҊ ੓׮о, ੉੹੄ stateٜਸ ഝਊ
  68. ؊ ࠂ੟ೠ ֎౟ਕ௼

  69. Dynamic Memory Networks • যڃ Ӗਸ ੍Ҋ, Ӓ Ӗী ؀ೠ

    ޛ਺ী ׹ਸ ೞחѪ - ࣻמद೷ ঱য৔৉җ ࠺तೣ - औ૑ ঋ਷ җઁ੉׮!
  70. Dynamic Memory Networks • ӝמ߹۽ RNNਸ ઑ೤ • End-to-End ೟ण

    • ֤ޙਸ ੍੗! - https://arxiv.org/abs/1506.07285
  71. Dynamic Memory Networks • Input Module - RNN (GRU), п

    ޙ੢ٜ੄ hidden stateܳ ੷੢ • Question Module - рױೠ RNN (GRU) • Episodic Memory Module - ੉ঠӝী ؀ೠ ੿ࠁܳ ҙ੢ೞח ݽٕ - ৈ۞ க੄ RNN (GRU) ۽ ҳࢿ
  72. Dynamic Memory Networks • ׮ܲ ఋੑ੄ Input Moduleਸ ࠢ੉חѪب оמ!

  73. Dynamic Memory Networks • ׮ܲ ఋੑ੄ Input Moduleਸ ࠢ੉חѪب оמ!

  74. Dynamic Memory Networks

  75. Dynamic Memory Networks

  76. Dynamic Memory Networks

  77. Dynamic Memory Networks • Attention + Memoryܳ ా೧ ૕ޙী ׹ೞח

    מ۱ਸ ഛ੢ • ӝמ߹۽ RNNਵ۽ ੉ܖয૓ ߹ب੄ Module, ੑ۱ ഋకо ׮ܲ ݽ ٕ۽ ߸҃ೞחѪب оמೣ • ੉޷૑ ੑ۱ ݽٕҗ Ѿ೤غݶ ֥ۄ਍ Ѿҗܳ ࠁৈષ
  78. NMT (Neural Machine Translation) • ࠂ੟ೠ ֎౟ਕ௼ҳઑܳ ഝਊೠ ੗زߣ৉ ֎౟ਕ௼

  79. Google NMT • https://translate.google.com • https://research.googleblog.com/2016/09/a-neural-network-for- machine.html

  80. ٩۞׬ ੸ਊী ؀ೠ ࢤп • ٩۞׬ = ই੉٣য + ؘ੉ఠ

    + ݽ؛ ೟ण - ݽ؛ ೟ण - ੼੼ ੷۴೧૗ - ই੉٣য - ੼੼ ؊ ݆਷ ࢎۈٜ੉ ઙࢎೞݶࢲ ই੉٣যب ݆ই૕ Ѫ - ؘ੉ఠ - ݠन۞׬, ٩۞׬ द؀ীࢲ о੢ ൞ࣗೠ ੗ਗ • ؀ӏݽ ؘ੉ఠܳ ഝਊೠ ݠन۞׬, ؊ ࠂ੟ೠ ઑ೤੄ ݽ؛, ࠙ঠ߹ ௼ ۽झ ݽ؛ ١ ই૒ ߊ੹ оמࢿ੉ ޖҾޖ૓ೣ • ҕࠗೞҊ োҳ೤द׮!
  81. More..? • Join VCNC and join #study_ml ੷൞ח ঱ઁա ࠺౟ਦ

    ࢲ࠺झܳ ೣԋ ٜ݅ݴ ӝࣿ੸ੋ ޙઁܳ ೣԋ ಽযաт מ۱੓ח ѐߊ੗ܳ ݽदҊ ੓णפ׮. ঱ઁٚ ࠗ׸হ੉ jobs@vcnc.co.kr۽ ੉ݫੌਸ ઱दӝ ߄ۉפ׮!
  82. Thank you!