[네이버 테크톡] 사람들과 자연스러운 대화를 나누는 인공지능 '핑퐁' 만들기

[네이버 테크톡] 사람들과 자연스러운 대화를 나누는 인공지능 '핑퐁' 만들기

100억 건의 카카오톡 데이터로 어떻게 똑똑하고 생동감 넘치는 대화형 인공지능을 만들었는지 공유하는 세션입니다. 인공지능을 위한 일상대화 기술인 '핑퐁'과, 사용자가 직접 새로운 패르소나의 캐릭터를 만들수 있게 해주는 "핑퐁 빌더"를 소개합니다. 기존 FAQ 방식의 챗봇 모델이 아닌, 카카오톡 데이터의 장점을 활용할 수 있는 Reply-centric 기반의 학습 방법을 공유합니다. 또한 세션 마지막에는 핑퐁이 얼마나 똑똑하고, 인간 같은 대화를 할 수 있는지 보여주는 놀라운 데모가 준비되어 있습니다. 지금도 m.me/ai.pingpong 에서 페이스북 메신저로 핑퐁과 대화해보실 수 있습니다!

B389eb128688072da7124603ef406524?s=128

Junseong

May 27, 2019
Tweet

Transcript

  1. 1.
  2. 2.

    ݾର 1. ೝಯ য়ߡ࠭ 2. ؘ੉ఠࣇ ࣗѐ & ಽ۰Ҋ ೞח

    ޙઁ 3. Query Centric Model : ؀ࠗ࠙੄ ୁࠈ੉ ࢎਊೞҊ ੓ח ߑध 4. Reaction Model: ࢎۈٜ੄ ݈ী ୭؀ೠ ߈਽೧ࠁ੗! 5. Reply Retrieval Model: ҳ୓੸ੋ ׹߸ਸ ೡ ࣻ ੓ب۾ ೧ࠁ੗! 6. Query Retrieval Model: ݃ ೝಯ ց QAب ੜೞա? 7. Pingpong Bot & Builder : ਋ܻ੄ ݧ੬੉ٜ ࣗѐ 8. Future Works: ਋ܻח খਵ۽ ޖट ੌਸ ೡө?
  3. 5.
  4. 6.

    য়ט ࢲ਎ ӝৡ਷ ৔ೞ 20ب੉ݴ զॿח ݉णפ׮. ъࣻ ഛܫ਷ 10%ੑפ׮.

    Ӓ۞ѱਃ. ৮੹ ೠъী ೖ௼ץ оҊ र਷ զॿ֎ਃ. ӝמ؀ച = “য়ט ࢲ਎ զॿח?” ੌ࢚؀ച = “য়ט զॿ ૓૞ લੋ׮” #0. Ӕؘ ੌ࢚؀ചо ੿ഛ൤ ޤભ?
  5. 7.

    ӝמ ؀ച (৘: “য়ט ࢲ਎ զॿח?”) ੌ࢚ ؀ച (৘: “ற

    য়ט զॿ ૓૞ લੋ׮”) ؀ച੄ ݾ੸ ੿ࠁܳ ঳ח׮ ਗೞח ೯زਸ ೞب۾ ೠ׮ ؀ച ੗୓о ݾ੸ о஖ ಞܻೣ ஘Ӕೣ ؀ച ઱ઁ ੋҕ૑מ ӝמী ݏח ઱ઁ ݽٚ ઱ઁ ޙ੢ ഋక ݺ۸, ૕ޙ ࢤпೡ ࣻ ੓ח ݽٚ ޙ੢ ഋక ੿׹੄ ਬޖ ݺഛೠ ੿׹ ઓ੤ ੿׹੉ ৈ۞ѐੌ ࣻ ੓਺ #0. Ӕؘ ੌ࢚؀ചо ੿ഛ൤ ޤભ?
  6. 8.

    ী੉੹౟ ࢤޛ ҕ࢑ಿ ޖࢤޛ Button UI NLI NLI GUI #1.

    Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? Natural Language Interface = ઁಿ੄ ੋрച
  7. 9.

    ( ઑࢶੌࠁ, 2017֙ 5ਘ 3ੌ ) ࢎۈٜ੉ SKT ‘־ҳ’ীѱ Ѣח

    ݈ ઺ 45%о хࢿ੸ੋ ؀ച बब೧ ҽݽ׬ ਋਎೧ ੜ ੗ 1) Nass, Clifford, and Li Gong. "Speech interfaces from an evolutionary perspective." Communications of the ACM 43.9 (2000): 36-43. 1. ࠗ੿੸ੋ ؀ച ҃೷੄ хࣗ • ࢎਊ੗ח ؀ചഋ AIܳ ݃஖ ࢎۈ୊ۢ ؀ೞח ҃ೱ੉ ੓ਵݴ1) ੉ ী ٮۄ AIীѱ ׮নೠ ؀ചܳ दب೤פ׮. • Ӓ۞ա അ੤ AIח ୊ܻೡ ࣻ ੓ח ؀ചо ౠ੿ ӝמҗ ઱ઁী ೠ੿ غয ੓ਵݴ, ؀ࠗ࠙੄ ݈ী ؀೧ “ޖट ݈ੋ૑ ੜ ݽܰѷযਃ” э ਷ ݫद૑۽ ؀਽೤פ׮. ੉۞ೠ ੋఠۑ࣌਷ ࢎਊ੗ীѱ ࠗ੿੸ੋ ҃೷ਸ ઴ ࣻ߆ী হणפ׮. • ੌ࢚؀ച מ۱਷ ࠗ੿੸ੋ ؀ച ҃೷ਸ хࣗदఃח زदী, ࢎਊ ੗۽ ೞৈӘ AIܳ ഻ঁ ؊ ੋр੸(human-likeness)੉Ҋ ࢤزх ੓ѱ וՙѱ ٜ݅যસפ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? (Jiang J, et al. (2015)) ࢎۈٜ੉ Microsoft Cortanaীѱ Ѣח ݈ ઺ 30%о ࣗ࣍ ؀ച
  8. 10.

    2. ࢎਊ੗ ੋѱ੉૑ݢ౟ ૐ؀ • ઁೠ হח য়೑ بݫੋ ੌ࢚؀ച

    מ۱਷ ࢎਊ੗о AI৬੄ ؀ചܳ ؊਌ ૌӡ ࣻ ੓ب۾ ٜ݅য઱ݴ, ੉ী ٮۄ ࣁ࣌׼ ؀ച ఢ ࣻ (CPS, Conversation-turns per session)৬ п ఢ੄ ޙ੢ ӡ੉ о טযժפ׮1). • Ѿҗ੸ਵ۽ ੌ࢚؀ച מ۱਷ ࢎਊ੗о AI ઁಿਸ ؊ য়ې ࢎਊೞ ب۾ ਬبೞݴ, زदী ׮নೠ ࢎਊ੗ী ؀ೠ ੿ࠁܳ ঳ਸ ࣻ ੓ण פ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? 1) Chen, Chun-Yen, et al. "Gunrock: Building A Human-Like Social Bot By Leveraging Large Scale Real User Data."
  9. 11.

    3. ࠳ے٬җ ҙ҅ ഋࢿ • AIח ੌ࢚؀ചܳ ా೧ ੗न݅੄ ಕܰࣗաܳ

    ٘۞յ ࣻ ੓ਵݴ, ׮ ܲ AI৬ ର߹ചغח Ҋਬ੄ ࠳ے٬җ நܼఠܳ о૑ѱ ؾפ׮. • ੌ࢚؀ച מ۱਷ AI੄ ѐࢿਸ ࢓ܻҊ ࢎਊ੗ীѱ ੗न݅੄ ו՝ਸ ా೧ ౠ߹ೠ ҙ҅ܳ ഋࢿೡ ࣻ ੓ب۾ ذणפ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ?
  10. 12.

    ೞ૑݅, ੌ࢚؀ചח য۵णפ׮ • ੌ࢚؀ചח ઱ઁա ഋకо ޖೠ൤ ׮ন೤פ׮. AIо

    ੌ࢚؀ചী ੌੌ੉ ؀਽ೡ ࣻ ੓ѱ ٜ݅ӝ ਤ೧ࢲח झ௼݀౟ ੘ࢿী ষ୒դ दрҗ ֢۱੉ ೙ਃ೤פ׮. ૑ܖೠ ੌ੉ӝب ೞભ. • Ӓۧѱ ೠ׮Ҋ ೧ب ੌ࢚؀ച੄ ӓ൤ ੌࠗ࠙݅ ழߡೡ ࣻ ੓ਸ ࡺ ੑפ׮.
 #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ?
  11. 14.

    Vision Make machines social ݈੉ ੜ ాೞѱ ؀ച۽ ੋఠۑ࣌ೡ ࣻ

    ੓ѱ ઁಿ੉ ইפۄ ઓ੤۽ ࢎਊ੄ ؀࢚੉ ইפۄ ҙ҅੄ ؀࢚ਵ۽ “ъই૑݅ఀ ࣗ઺ೠ ੋҕ૑מ”
  12. 17.

    Source: Optional source placeholder. Delete if not used. ӣળࢿ (Junseong

    Kim) fb.com/codertimo ScatterLab Machine Learning Engineer ੌ࢚؀ച ੋҕ૑מ ೝಯ Machine Learning Team (࢑সӝמਃਗ) Naver Clova AI Research Intern Atlas Guide NLP/Machine Learning Engineer General Domain Dialog System/Chatbot (Chitchat) General Sentence Embedding (BERT, ELMo) And Diverse Natural Language Processing Tasks… Neural Machine Translation codertimo/BERT-pytorch 2.4k+ stars Research Domain Experience 2017, 2014 Intel ISEF Computer Science Finalist 2016, 2014 Intel KSEF Computer Science Grand Award Working Experience
  13. 18.

    Dataset 100র+ Utterance(2.2TB)੄ ਬ੷ ؀ച ؘ੉ఠ ఫझ౟৆ & োগ੄ җ೟

    : োগܳ ਤೠ ੋҕ૑מ Utterance Variance Context Aware Utterance Massive Dialog System … NLP/Dialog System ݠन۞ցীѲ
  14. 19.

    Goal Engineer ҙ੼ীࢲ੄ Main Goal 100রѤ੄ ؀ച ؘ੉ఠܳ ୭؀ೠ ೟ण೧

    ࢎۈэ਷ ੋҕ૑מਸ ٜ݅੗! ӒܻҊ о੢ ബਯ੸ੋ ݠन۞׬ ݽ؛җ ೟णҗ੿ਸ ଺ইࠁ੗ ޖ঺ਸ ೧ঠ ೞחо?
  15. 20.

    Query Centric Model ஶࣆ ࢸݺ ੌױ о੢ ए਍ ߑߨࠗఠ ب੹೧ࠁ੗!

    э਷ Queryܳ ଺ইࢲ ੉ী ؀਽غח ׹߸ਸ ղࠁղ੗ ঒ ஠஠য়స ؘ੉ఠо ੓ਵפӬ ׹߸ب ׳۰੓ਗ਼ই?!?!
  16. 21.

    Query Centric Model ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching

    ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
  17. 22.

    Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ Query Space ࢎਊ੗о

    ੉۠ ݈ਸ ೡ Ѫ эই! ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
  18. 23.

    Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ ݅աࢲ ߈оਕਃ ੷ب

    ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ੉۠ ૕ޙ੉ য়ݶ ੉۠ ׹߸ਸ ೞ੗ ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
  19. 24.

    Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ

    ੷ب ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ࢎਊ੗੄ ૕੄ ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
  20. 25.

    Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ

    ੷ب ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
  21. 27.

    Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC ؘ੉ఠ •

    ஠஠য়స ؘ੉ఠ ઺ীࢲ ࢎਊ੗੄ ੿ࠁо ನೣغ૑ ঋ਷ ޙ੢ਸ Rule Based Filtering • ࠺ࣘয, ࢶ੿੸, ࠈ੉ ੉ঠӝ ೞӝী ੸೤ೞ૑ ঋ਷ ղਊਸ Rule Based Filtering • ೙ఠ݂ ػ ؘ੉ఠٜਸ ੹ࣻ Ѩࢎೞৈ ؀׹ೞݶ উغח ղਊਸ Human Filtering • -> ೝಯ੉ ࢎਊೡ ࣻ ੓ח Query-Reply Pairٜਸ ٜ݅য م
  22. 28.

    Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC ؘ੉ఠ Q->

    ݆੉ ݆੉ ࢎی೧ਃ! ஠஠য়స ؘ੉ఠ ࣇ Reply select: աب ݆੉݆੉ ࢎی೧ਃ ਬࢎ Q’:য়טب ݆੉݆੉ ࢎی೧ਃ
  23. 29.

    Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC੄ ޙઁ੼ •

    ׹߸ਵ۽ ࢎਊغח ஠஠য়స ؀׹੉ ޙݓ ੿ࠁܳ ׸Ҋ ੓যࢲ য࢝ೠ ҃਋о ݆਺ • ಕܰࣗաܳ ਬ૑ೡ ࣻ হח ޙઁ → ࠈ੄ ಕܰࣗաܳ ׹߸ী ֣ৈյ ࣻ হ਺ (ݒ ׹߸݃׮ ݈ైо ׮ܰҊ ݽف ଻౴୓) • Blackbox → ࠈਸ ࢎਊೞח ࢎਊ੗(B2B)ҙ੼ীࢲ ׹߸ਸ controlೡ ࣻ হ਺ → customization੉ ࠛоמೣ • Query Search Variance ܳ טܻ۰ݶ Ӓ݅ఀ ஠஠য়స ؘ੉ఠܳ ੹ࣻ Ѩࢎ ೧ঠೣ → উӒ۞ݶ बब੉ э਷ ޙઁ Ex ) ਽ ૑ೞ୍ ఋҊ оח ઺ -> ೡݠפ૘ਵ۽ оחѢঠ? Ex ) ࢎی೧! -> ਔ ><>< աوաو, ࢎی೯ -> ਽ աب Ex ) ই ૓૞ ޷஘ ѱ੐ೞҊ र׮ -> XX աب ѱ੐ XXೞҊ र׮ Ex ) ֎੉ߡ ଻౴ ࠈ: ੉ٮо োۅ೧! -> ਔ ੉ٮ ஠సೡѱ!
  24. 30.

    Query Centric Model ࠁ׮ ࠄ૕੸ਵ۽ ੌ࢚؀ചী ੉ ੽Ӕ੉ ࠛоמೠ ੉ਬ

    ੌ࢚؀ച э਷ ҃਋ীח ૕੄ী ؀ೠ Search Spaceо ցޖ ևӝ ٸޙী Q'ਸ Nѐ۽ ੿੄ೡ ࣻ হ਺ ੌ࢚؀ചо Ӕࠄ੸ਵ۽ য۰਍ ੉ਬח ழߡ ೧ঠ ೞח ؀ച੄ ߧਤо ޖೠೞѱ ௼ӝ ٸޙ੐ ޖೠ൤ ௾ Query Spaceী ݽٚ Query Variance ܳ ࢎۈ੉ ੌੌ੉ ଻਎ ࣻ হ਺ ࢎۈ਷ ؀ചܳ ೡ ٸ Query -> Information -> ੉ী ੸೤ೠ ؀׹ਸ ೣ (1:1 ؀਽੉ ইש)
  25. 31.

    Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ ࢎۈ੉ ٛח Query Spaceח

    ޖೠೞ૑݅ ࢎۈ੄ Reply Spaceח ਬೠೞ૑ ঋਸө? ࢎۈٜ਷ ઱۽ ೞח ݈੉ ੿೧ઉ ੓૑ ঋਸө? ੸੺ೞѱ ੜ Queryী ؀਽दఃݶ ؀ചо ࢿ݀ೡ ࣻ ੓૑ ঋਸө?
  26. 32.

    Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ ஠సਵ۽ ؀ചೡٸ ಣӐ੸ੋ ࢎۈٜ੄

    Reply Space Reaction ই૓૞? ߓҊ೐׮ ցف? ੷۠ƀƀ ݏযݏয żż … Answer to Query ਽ ݡ঻য ইפ ই૒ উоࠌ૑ ծਫ਼੗ӝূ ই૒ ੌ۞ ,…. Context Aware Reply Ӕؘ џо ૓૞ Ӓۗয? ৬ ੉۠ ࢚ട਷ ખ ইפ૑ ইөࠗఠ ইޖѪب ޅݡ঻ਗ਼ই KB & Background Aware Reply য়ט ࢲ਎ 30بۆ׮ ੉ѱ աۄջ ց ৈ஘੉ی ೻য઎׮ݶࢲ ই ഘ؀ ୷ઁ ૑դ઱৓חؘ ޅщ׮Ҋ Ӓۗ૑
  27. 33.

    Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ যו੿ب ਬೠೠ Reply Spaceܳ

    ೞաೞա ੼۸೧ աоݶࢲ ੌ࢚؀ച Coverageܳ ֫ৈࢲ ࢎۈэ਷ ؀ച ੋҕ૑מਸ ٜ݅੗! Context+Query -> Reply ী ؀ೠ Distributionਸ Deep Learningਸ ా೧ࢲ ೟णदఆ ࣻ ੓਺ +ষ୒աѱ ௾ ؘ੉ఠ۽ ୭Ҋ੄ NLU, NLGݽ؛ਸ ٜ݅ ࣻ ੓਺
  28. 34.

    Reaction և਷ Coverage, উ੹ೠ ׹߸ ܻঘ࣌਷ ؀ച઺ীࢲ о੢ ݆਷ Query

    Coverageܳ ыח ప௼ץ पઁ۽ ࢎۈٜ੉ о੢ ݆੉ ࢎਊೞח ؀ച ߑध ৘: ܻঘ࣌݅ ੜ೧ب {թ੗|ৈ੗}஘ҳীѱ ੉ࢄ߉ח ؀ചо оמೣ ੸੺ೠ ߈਽җ ࢚؀ߑ੄ ૕੄ী ؀ೠ ҕх਷ ؀ചܳ ਬبೞҊ গ଱ҙ҅ܳ ഋࢿೣ ୐ߣ૩ ݽ؛: Reaction
  29. 35.

    ݽٚ ؀ച੄ Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ੉ ؀ࠗ࠙ ݆੉

    ࢎਊೞח ୨ 1200ѐ ੿ب੄ Reaction Class ੿੄ Reaction ই ইޖѪب উݡ঻য ƕ ੼बदр੉ঠ!!!! ߏਸ উоઉ৳֎…?? ೵ য়ט әध উաৡ؀ ࠗݽש੉ য়ט ߏਸ উ೧઱࣑য Ӓܶ੉ ղ ߊ ߃ীࢲ ӵ઎য.. ইөࠗఠ ߓо ցޖ ই೐֎. ে ߓҊ೐ѷ׮.. ਋ইই ߓҊ೐ѱٮঠ ೳ ૓૞ ߓҊ೐ѷ׮ ޷஘Ѣ ইջ? ૓૞ ߓҊ೐ѷ׮ ೵ ߓҊ౵ࢲ যڂ೧.. ೵ ҡଳই?!?! ?? ҡଳই?? և਷ Coverage, উ੹ೠ ׹߸
  30. 36.

    Reaction ই ইޖѪب উݡ঻য ƕ ੼बदр੉ঠ!!!! ߏਸ উоઉ৳֎…?? ೵ য়ט

    әध উաৡ؀ ࠗݽש੉ য়ט ߏਸ উ೧઱࣑য Ӓܶ੉ ղ ߊ ߃ীࢲ ӵ઎য.. ইөࠗఠ ߓо ցޖ ই೐֎. ে ߓҊ೐ѷ׮.. ਋ইই ߓҊ೐ѱٮঠ ೳ ૓૞ ߓҊ೐ѷ׮ ޷஘Ѣ ইջ? ૓૞ ߓҊ೐ѷ׮ ೵ ߓҊ౵ࢲ যڂ೧.. ೵ ҡଳই?!?! ?? ҡଳই?? ೵ ߓҊ೐ѷ׮ ƕƕ (ߓҊ೓ Class) ҡଳই?!?! (ѣ੿ Class) և਷ Coverage, উ੹ೠ ׹߸ ࣻୌ݅ѐ੄ Reaction Datasetਸ Labeling হ੉ ҳ୷ ݽٚ ؀ച੄ Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ੉ ؀ࠗ࠙ ݆੉ ࢎਊೞח ୨ 1200ѐ ੿ب੄ Reaction Class ੿੄
  31. 37.

    Reaction և਷ Coverage, উ੹ೠ ׹߸ ੉ۧѱ݅ ೧ب 1000݅+ ੉࢚੄ Reaction

    Training Pairܳ ٜ݅ ࣻ ੓঻਺ ؀ച ؘ੉ఠо ੓ਵפ ׮নೠ Query Varianceܳ ੗زਵ۽ ٜ݅ ࣻ ੓਺
  32. 38.

    Reaction և਷ Coverage, উ੹ೠ ׹߸ Classification Model Retrieval Model Queryinput

    Query Encoder Ensemble XGBoost Reranking Reaction Class Select Among 1K class ૓૞ ൨ٜ঻ѷ׮ ƕƕ લ૑݃ই!! …
  33. 39.

    Reaction և਷ Coverage, উ੹ೠ ׹߸ ؀಴ Response ٘ܿ੉ Response Reaction

    Class ੸਷ন੄ Response݅ ੘ࢿ೧ب ಕܰࣗաܳ ٘۞յ ࣻ ੓਺ ࠈীѱ গ଱ҙ҅ܳ ഋࢿೞѱ ೣ
  34. 41.

    Pros: 1. ੌ࢚؀ച ழߡܻ૑о ݒ਋ ֫׮. ਟ݅ೞݶ ؀ࠗ࠙੄ ׹ী ֤ܻ੸ੋ

    ׹߸ਸ ೡ ࣻ ੓਺. 2. ׹߸੄ ݈ైܳ ࣻ੿ೡ ࣻ ੓যࢲ, ಕܰࣗաܳ ߈৔ೞৈ ੌ࢚؀ച ࠈਸ ٜ݅ ࣻ ੓਺ Reaction և਷ Coverage, উ੹ೠ ׹߸ Cons: 1. উ੹ೠ ׹߸ਸ ୶ҳ ೣਵ۽ࢲ ҳ୓੸ੋ ׹߸ਸ ղࠁղ૑ ޅೣ 2. ܻঘ࣌ਸ ઺बਵ۽ ೞ׮ࠁפ, ؀ചܳ ੉যաоӝ ൨ٝ
  35. 42.

    Reply Retrieval Model ҳ୓੸ੋ ׹߸, ੸੺ೠ ؀׹ ಕܰࣗաо ߈৔ػ ׹߸ٜਸ

    ੘ࢿ೧઱ݶ ঌইࢲ ૕੄(Q)ী ؀೧ о੢ ੸੺ೞݶࢲ ҳ୓੸ੋ ׹߸ਸ ղࠁն ಕܰࣗա, ࢚ध ١ী ؀ೠ ੿ࠁܳ ׸਷ ҳ୓੸ੋ ׹߸ਸ ੘ࢿೞৈ ੌ࢚؀ചܳ ܻ٘ೞѢա ੉যт ࣻ ੓ب۾ ب਑ਸ ઱ח ݽ؛ ܻঘ࣌ ݽ؛੄ ҳ୓ࢿ੉ হח ޙઁ੼ਸ Mainਵ۽ ೧Ѿ
  36. 43.

    Reply Retrieval Model Next Utterance Prediction Model https://arxiv.org/abs/1705.02364 Supervised Learning

    of Universal Sentence Representations from Natural Language Inference Data Query Reply Query৬ Replyܳ ઱Ҋ ੉ѱ Continuousೠ Utteranceੋ૑ Next Utterance Classification Response = argmax(Model(Equery , Ereplyi ))
  37. 44.

    Reply Retrieval Model Next Utterance Prediction Model ࢎਊ੗੄ ஠஠য়స ؘ੉ఠܳ

    ߄ఔਵ۽ Query-Reply 50রѤ੄ Pairहਸ ݅ٞ (single ఢ ؀ച) Query, Replyܳ Sentence Encoderী ֍Ҋ concat೧ࢲ FNNਸ కਕ Binary Classification Pairؘ੉ఠܳ ੉ਊ೧ Next Utterance Prediction (BERT੄ Next Sentence Prediction) N:1 ࠺ਯ۽ Negative Sampling ೧ࢲ Random ಕযب ೟ण (੉ٸ N਷ Negative੄ іࣻ)
  38. 46.

    Reply Retrieval Model Next Utterance Prediction Model: Unbalanced ࢎۈٜ੉ Reactionਸ

    ցޖ ݆੉೧ࢲ উ੹ೠ ׹߸ਸ ೞب۾ ೟ण੉ غযߡܿ ҳ୓੸ੋ ׹߸੉ ਋ࢶद غب۾ Reranking Logic੉ ೙ਃ۽ ೮਺ (੉ ࠗ࠙ী ݆਷ दрਸ ٜ੐) ਋ܻ૘ ъই૑о ૘ਸ աщ૑ ޤঠ ೳ ই૒ ޅ଺ওয? Q 0.79 ೵ ૓૞ਃ? 0.92 Insight: ݈ਸ ੜೞח ୁࠈਸ ݅٘۰ݶ Ӓր ࢎۈٜ੉ ೞח ݈݅ ߓ਋ݶ উغщҳա.. ೟णਸ ೞ׮ ࠁפ…
  39. 47.

    Reply Retrieval Model Next Utterance Prediction Model: Unbalanced RMM Prediction

    Result ੉ઁ ׮নೠ ݈ਸ ೡ ઴ ইח ೝಯ ؊੉࢚ ܻঘ࣌݅ ೞ؍ ೝಯ੉ ইפঠ!
  40. 48.

    Reply Retrieval Model Next Utterance Prediction Model Pros: 1. Reaction

    ࠁ׮ ഻ঁ ҳ୓੸ੋ ׹߸ਸ ղ֬਺( য়ט਷ {૞੢ݶ,૤ࡴ}੉ ݡҊ र਷ Ѧ → աب! ఔࣻਭب ݡҊ र׮) 2. ࠈ ੗न(?)ী ؀ೠ ղਊ ࡺ ইפۄ ࢚ध੉ա ࢎਊ੗ী ؀ೠ ૕ޙ ١ب ֍ਸ ࣻ ੓যࢲ ؀ചܳ ؊ ੉যաт ࣻ ੓਺ 3. ࠈ ӝദ੗о Qী ߧਤܳ ੌੌ੉ ੑ۱ೞ૑ ঋইب ؽ Cons 1. ই૒਷ ӝദ੗о ׹߸(R)ী ؀ೠ Spaceܳ ੌੌ੉ ੘ࢿ೧ঠ ೣ. 1. Ӓېࢲ ৈ۞ߣ పझ౟ܳ ೞݶࢲ ಕܰࣗաী ౠചػ Reply setਸ ࣻ੿೧ঠೣ 2. ੘ࢿೠ Replyо ౠ੿ Qী աৡ׮ח ࠁ੢੉ হӝ ٸޙী, Qܳ ৘ஏೞӝо য۰਑.
  41. 49.

    Qܳ ৘ஏೞӝ য۵׮ח Reply retrieval model੄ Consܳ ӓࠂೞӝ ਤ೧ ѐߊ.

    ౠ൤ ೐۽೙ ׹߸੄ ҃਋ ࢎਊ੗о ૕ޙਸ ೡ ҃਋ ԙ ׹ਸ ೧઻ঠೞחؘ, Reply Retrieval Model਷ Ӓ۞ܻۄח ࠁ੢੉ হ਺. Query Retrieval Model Motivation ࢎਊ੗੄ ૕੄ী ৮߷ೞѱ 1:1۽ ؀਽غח ׹߸ਸ ҳഅೞӝ ਤ೧ ѐߊ
  42. 50.

    Query Retrieval Model Using Next-Utterance Task Pretrained Encoding ߡ౟ীࢲب LM+Next

    Sentence Predictionਵ۽ context ೟ण Ӓۢ Next Utterance Sub-Taskܳ ೟णೠ Encoderਸ ࢎਊೞݶ જ਷ Embeddingਸ ыҊ ੓ਸө? Query Reply
  43. 52.

    ೞ૑݅ ੗ࣁ൤ ଺ইࠁݶ ࠺तೠ ݈੉૑݅ ׮ܲ ੄޷ٜ੉ ੓਺ Query Retrieval

    Model Dataset ਊ঱੉ա ୓঱੉ زੌೞݶ ޹хೞѱ ߈਽ೞח ҃਋о ݆਺
  44. 53.

    Query Retrieval Model Dataset ௼ۄ਋٘ࣗयਵ۽ “э਷ ੄޷ ޙ੢ ଺ӝ” పझ௼

    ૓೯ 50000ѐ ಕযী ؀ೠ ؀ച୓ э਷ ੄޷ ଺ӝ ؘ੉ఠࣇ ѐߊ
  45. 54.

    Query Retrieval Model Using Next-Utterance Task Pretrained Encoding Query Query

    زੌೠ ޙ੢ ଺ӝ పझ௼۽ Transfer Learning Datasetਸ ੉ਊ೧ࢲ Fine-Tuning
  46. 57.

    Query Retrieval Model Pros and Cons Cons: 1. э਷ Qী

    ؀೧ ೠ੿ػ ׹߸݅ ઁҕೞӝ ٸޙী variationਸ ݅٘ח ؘ ೠ҅о ੓਺ Pros: 1. ӝઓ ୁࠈ ࠽؊ীࢲ ࢎਊೞ؍ ߑध੉ۄ ࢎۈٜীѱ ੊ࣼೣ 2. ࢎਊ੗੄ ݈ী ؀೧ যڃ ׹߸ਸ ղࠁյ૑ ࠺Ү੸ ాઁೞӝ ए਑ 3. ֎੉ߡ, ஠஠য়, Dialogflow ١ ݽٚ ࠽؊ী ੓ח ݽ؛੉૑݅, Ӓ઺ীࢲب SOTA
  47. 60.

    Pingping Bot Prebuilt Bot & App ъ۱ೠ ೲࣁ ಕܰࣗա৬ ߊۇೠ

    ಕܰࣗաܳ ղࡼח ೝಯ੄ ف ই੉ٜ ౵੉౴ ܖա Ӓթ੗ ೲࣁ઺
  48. 62.

    оө਍ Future Works NER, Tokenizer, Typo-Correction, Semantic Parsing Question and

    Answering : external KB, user KB Persona Reply Style Transfer Reply Generation Context Aware Reply End-to-End Dialog System Hyper-Personalization SOTA Language Modeling ೧ঠೡ ੌ੉ ցޖ ݆ইਃ