[네이버 테크톡] 사람들과 자연스러운 대화를 나누는 인공지능 '핑퐁' 만들기

100억 건의 카카오톡 데이터로 어떻게 똑똑하고 생동감 넘치는 대화형 인공지능을 만들었는지 공유하는 세션입니다. 인공지능을 위한 일상대화 기술인 '핑퐁'과, 사용자가 직접 새로운 패르소나의 캐릭터를 만들수 있게 해주는 "핑퐁 빌더"를 소개합니다. 기존 FAQ 방식의 챗봇 모델이 아닌, 카카오톡 데이터의 장점을 활용할 수 있는 Reply-centric 기반의 학습 방법을 공유합니다. 또한 세션 마지막에는 핑퐁이 얼마나 똑똑하고, 인간 같은 대화를 할 수 있는지 보여주는 놀라운 데모가 준비되어 있습니다. 지금도 m.me/ai.pingpong 에서 페이스북 메신저로 핑퐁과 대화해보실 수 있습니다!

Transcript

1. 사람들과 자연스러운 대화를 나누는 인공지능 ‘핑퐁’ 만들기 스캐터랩(ScatterLab) Machine Learning Engineer 김준성 ੌ࢚؀ച ੋҕ૑מ 스캐터랩(ScatterLab) CEO 김종윤
2. ݾର 1. ೝಯ য়ߡ࠭ 2. ؘ੉ఠࣇ ࣗѐ & ಽ۰Ҋ ೞח ޙઁ 3. Query Centric Model : ؀ࠗ࠙੄ ୁࠈ੉ ࢎਊೞҊ ੓ח ߑध 4. Reaction Model: ࢎۈٜ੄ ݈ী ୭؀ೠ ߈਽೧ࠁ੗! 5. Reply Retrieval Model: ҳ୓੸ੋ ׹߸ਸ ೡ ࣻ ੓ب۾ ೧ࠁ੗! 6. Query Retrieval Model: ݃ ೝಯ ց QAب ੜೞա? 7. Pingpong Bot & Builder : ਋ܻ੄ ݧ੬੉ٜ ࣗѐ 8. Future Works: ਋ܻח খਵ۽ ޖट ੌਸ ೡө?
3. 핑퐁과 일상대화 기술은 무엇인가? 스캐터랩(ScatterLab) CEO 김종윤
4. ೝಯ਷ ؀ചഋ ੋҕ૑מਸ ਤೠ ੌ࢚؀ച ӝࣿੑפ׮
5. #0. Ӕؘ ੌ࢚؀ചо ੿ഛ൤ ޤભ? ӝמ؀ച = “য়ט ࢲ਎ զॿח?” ੌ࢚؀ച = “য়ט զॿ ૓૞ લੋ׮”
6. য়ט ࢲ਎ ӝৡ਷ ৔ೞ 20ب੉ݴ զॿח ݉णפ׮. ъࣻ ഛܫ਷ 10%ੑפ׮. Ӓ۞ѱਃ. ৮੹ ೠъী ೖ௼ץ оҊ र਷ զॿ֎ਃ. ӝמ؀ച = “য়ט ࢲ਎ զॿח?” ੌ࢚؀ച = “য়ט զॿ ૓૞ લੋ׮” #0. Ӕؘ ੌ࢚؀ചо ੿ഛ൤ ޤભ?
7. ӝמ ؀ച (৘: “য়ט ࢲ਎ զॿח?”) ੌ࢚ ؀ച (৘: “ற য়ט զॿ ૓૞ લੋ׮”) ؀ച੄ ݾ੸ ੿ࠁܳ ঳ח׮ ਗೞח ೯زਸ ೞب۾ ೠ׮ ؀ച ੗୓о ݾ੸ о஖ ಞܻೣ ஘Ӕೣ ؀ച ઱ઁ ੋҕ૑מ ӝמী ݏח ઱ઁ ݽٚ ઱ઁ ޙ੢ ഋక ݺ۸, ૕ޙ ࢤпೡ ࣻ ੓ח ݽٚ ޙ੢ ഋక ੿׹੄ ਬޖ ݺഛೠ ੿׹ ઓ੤ ੿׹੉ ৈ۞ѐੌ ࣻ ੓਺ #0. Ӕؘ ੌ࢚؀ചо ੿ഛ൤ ޤભ?
8. ী੉੹౟ ࢤޛ ҕ࢑ಿ ޖࢤޛ Button UI NLI NLI GUI #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? Natural Language Interface = ઁಿ੄ ੋрച
9. ( ઑࢶੌࠁ, 2017֙ 5ਘ 3ੌ ) ࢎۈٜ੉ SKT ‘־ҳ’ীѱ Ѣח ݈ ઺ 45%о хࢿ੸ੋ ؀ച बब೧ ҽݽ׬ ਋਎೧ ੜ ੗ 1) Nass, Clifford, and Li Gong. "Speech interfaces from an evolutionary perspective." Communications of the ACM 43.9 (2000): 36-43. 1. ࠗ੿੸ੋ ؀ച ҃೷੄ хࣗ • ࢎਊ੗ח ؀ചഋ AIܳ ݃஖ ࢎۈ୊ۢ ؀ೞח ҃ೱ੉ ੓ਵݴ1) ੉ ী ٮۄ AIীѱ ׮নೠ ؀ചܳ दب೤פ׮. • Ӓ۞ա അ੤ AIח ୊ܻೡ ࣻ ੓ח ؀ചо ౠ੿ ӝמҗ ઱ઁী ೠ੿ غয ੓ਵݴ, ؀ࠗ࠙੄ ݈ী ؀೧ “ޖट ݈ੋ૑ ੜ ݽܰѷযਃ” э ਷ ݫद૑۽ ؀਽೤פ׮. ੉۞ೠ ੋఠۑ࣌਷ ࢎਊ੗ীѱ ࠗ੿੸ੋ ҃೷ਸ ઴ ࣻ߆ী হणפ׮. • ੌ࢚؀ച מ۱਷ ࠗ੿੸ੋ ؀ച ҃೷ਸ хࣗदఃח زदী, ࢎਊ ੗۽ ೞৈӘ AIܳ ഻ঁ ؊ ੋр੸(human-likeness)੉Ҋ ࢤزх ੓ѱ וՙѱ ٜ݅যસפ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? (Jiang J, et al. (2015)) ࢎۈٜ੉ Microsoft Cortanaীѱ Ѣח ݈ ઺ 30%о ࣗ࣍ ؀ച
10. 2. ࢎਊ੗ ੋѱ੉૑ݢ౟ ૐ؀ • ઁೠ হח য়೑ بݫੋ ੌ࢚؀ച מ۱਷ ࢎਊ੗о AI৬੄ ؀ചܳ ؊਌ ૌӡ ࣻ ੓ب۾ ٜ݅য઱ݴ, ੉ী ٮۄ ࣁ࣌׼ ؀ച ఢ ࣻ (CPS, Conversation-turns per session)৬ п ఢ੄ ޙ੢ ӡ੉ о טযժפ׮1). • Ѿҗ੸ਵ۽ ੌ࢚؀ച מ۱਷ ࢎਊ੗о AI ઁಿਸ ؊ য়ې ࢎਊೞ ب۾ ਬبೞݴ, زदী ׮নೠ ࢎਊ੗ী ؀ೠ ੿ࠁܳ ঳ਸ ࣻ ੓ण פ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ? 1) Chen, Chun-Yen, et al. "Gunrock: Building A Human-Like Social Bot By Leveraging Large Scale Real User Data."
11. 3. ࠳ے٬җ ҙ҅ ഋࢿ • AIח ੌ࢚؀ചܳ ా೧ ੗न݅੄ ಕܰࣗաܳ ٘۞յ ࣻ ੓ਵݴ, ׮ ܲ AI৬ ର߹ചغח Ҋਬ੄ ࠳ے٬җ நܼఠܳ о૑ѱ ؾפ׮. • ੌ࢚؀ച מ۱਷ AI੄ ѐࢿਸ ࢓ܻҊ ࢎਊ੗ীѱ ੗न݅੄ ו՝ਸ ా೧ ౠ߹ೠ ҙ҅ܳ ഋࢿೡ ࣻ ੓ب۾ ذणפ׮. #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ?
12. ೞ૑݅, ੌ࢚؀ചח য۵णפ׮ • ੌ࢚؀ചח ઱ઁա ഋకо ޖೠ൤ ׮ন೤פ׮. AIо ੌ࢚؀ചী ੌੌ੉ ؀਽ೡ ࣻ ੓ѱ ٜ݅ӝ ਤ೧ࢲח झ௼݀౟ ੘ࢿী ষ୒դ दрҗ ֢۱੉ ೙ਃ೤פ׮. ૑ܖೠ ੌ੉ӝب ೞભ. • Ӓۧѱ ೠ׮Ҋ ೧ب ੌ࢚؀ച੄ ӓ൤ ੌࠗ࠙݅ ழߡೡ ࣻ ੓ਸ ࡺ ੑפ׮.
 #1. Ҷ੉ ੌ࢚؀ചо ৵ ೙ਃೞભ?
13. Vision Make machines social
14. Vision Make machines social ݈੉ ੜ ాೞѱ ؀ച۽ ੋఠۑ࣌ೡ ࣻ ੓ѱ ઁಿ੉ ইפۄ ઓ੤۽ ࢎਊ੄ ؀࢚੉ ইפۄ ҙ҅੄ ؀࢚ਵ۽ “ъই૑݅ఀ ࣗ઺ೠ ੋҕ૑מ”
15. Mission Statement ࣁ҅ ୭Ҋ੄ ੋҕ૑מ ੌ࢚؀ച ӝࣿҗ ઁಿਸ ٜ݅Ҋ ੹౵ೞӝ
16. Open-Domain Conversational AI: Technical Section 스캐터랩(ScatterLab) Machine Learning Engineer 김준성
17. Source: Optional source placeholder. Delete if not used. ӣળࢿ (Junseong Kim) fb.com/codertimo ScatterLab Machine Learning Engineer ੌ࢚؀ച ੋҕ૑מ ೝಯ Machine Learning Team (࢑সӝמਃਗ) Naver Clova AI Research Intern Atlas Guide NLP/Machine Learning Engineer General Domain Dialog System/Chatbot (Chitchat) General Sentence Embedding (BERT, ELMo) And Diverse Natural Language Processing Tasks… Neural Machine Translation codertimo/BERT-pytorch 2.4k+ stars Research Domain Experience 2017, 2014 Intel ISEF Computer Science Finalist 2016, 2014 Intel KSEF Computer Science Grand Award Working Experience
18. Dataset 100র+ Utterance(2.2TB)੄ ਬ੷ ؀ച ؘ੉ఠ ఫझ౟৆ & োগ੄ җ೟ : োগܳ ਤೠ ੋҕ૑מ Utterance Variance Context Aware Utterance Massive Dialog System … NLP/Dialog System ݠन۞ցীѲ
19. Goal Engineer ҙ੼ীࢲ੄ Main Goal 100রѤ੄ ؀ച ؘ੉ఠܳ ୭؀ೠ ೟ण೧ ࢎۈэ਷ ੋҕ૑מਸ ٜ݅੗! ӒܻҊ о੢ ബਯ੸ੋ ݠन۞׬ ݽ؛җ ೟णҗ੿ਸ ଺ইࠁ੗ ޖ঺ਸ ೧ঠ ೞחо?
20. Query Centric Model ஶࣆ ࢸݺ ੌױ о੢ ए਍ ߑߨࠗఠ ب੹೧ࠁ੗! э਷ Queryܳ ଺ইࢲ ੉ী ؀਽غח ׹߸ਸ ղࠁղ੗ ঒ ஠஠য়స ؘ੉ఠо ੓ਵפӬ ׹߸ب ׳۰੓ਗ਼ই?!?!
21. Query Centric Model ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
22. Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ Query Space ࢎਊ੗о ੉۠ ݈ਸ ೡ Ѫ эই! ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
23. Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ ݅աࢲ ߈оਕਃ ੷ب ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ੉۠ ૕ޙ੉ য়ݶ ੉۠ ׹߸ਸ ೞ੗ ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
24. Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ ੷ب ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ࢎਊ੗੄ ૕੄ ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
25. Query Centric Model উ֞? ߓҊ౵ ࠁҊर׮ ਵ੉ҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ ੷ب ߓҊ౵ਃ աف ৵ Ӓې?? Query Space ஶࣆ ࢸݺ о੢ ݆੉ ࢎਊغח Query Matching ߑध੄ ୁࠈ ݽ؛ Sentence Representationਸ ೟ण೧ࢲ Cosine-Similarityо ୭؀ച غח Scriptܳ Retrievalೞѱ ଺ӝ Response = argmax(cos(Queryembed , Scriptembedi ))
26. Query Centric Model https://youtu.be/rhHQMA80TcE ஠஠য়స Query Retrieval Model: Pingpong-POC
27. Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC ؘ੉ఠ • ஠஠য়స ؘ੉ఠ ઺ীࢲ ࢎਊ੗੄ ੿ࠁо ನೣغ૑ ঋ਷ ޙ੢ਸ Rule Based Filtering • ࠺ࣘয, ࢶ੿੸, ࠈ੉ ੉ঠӝ ೞӝী ੸೤ೞ૑ ঋ਷ ղਊਸ Rule Based Filtering • ೙ఠ݂ ػ ؘ੉ఠٜਸ ੹ࣻ Ѩࢎೞৈ ؀׹ೞݶ উغח ղਊਸ Human Filtering • -> ೝಯ੉ ࢎਊೡ ࣻ ੓ח Query-Reply Pairٜਸ ٜ݅য م
28. Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC ؘ੉ఠ Q-> ݆੉ ݆੉ ࢎی೧ਃ! ஠஠য়స ؘ੉ఠ ࣇ Reply select: աب ݆੉݆੉ ࢎی೧ਃ ਬࢎ Q’:য়טب ݆੉݆੉ ࢎی೧ਃ
29. Query Centric Model ஠஠য়స Query Retrieval Model: Pingpong-POC੄ ޙઁ੼ • ׹߸ਵ۽ ࢎਊغח ஠஠য়స ؀׹੉ ޙݓ ੿ࠁܳ ׸Ҋ ੓যࢲ য࢝ೠ ҃਋о ݆਺ • ಕܰࣗաܳ ਬ૑ೡ ࣻ হח ޙઁ → ࠈ੄ ಕܰࣗաܳ ׹߸ী ֣ৈյ ࣻ হ਺ (ݒ ׹߸݃׮ ݈ైо ׮ܰҊ ݽف ଻౴୓) • Blackbox → ࠈਸ ࢎਊೞח ࢎਊ੗(B2B)ҙ੼ীࢲ ׹߸ਸ controlೡ ࣻ হ਺ → customization੉ ࠛоמೣ • Query Search Variance ܳ טܻ۰ݶ Ӓ݅ఀ ஠஠য়స ؘ੉ఠܳ ੹ࣻ Ѩࢎ ೧ঠೣ → উӒ۞ݶ बब੉ э਷ ޙઁ Ex ) ਽ ૑ೞ୍ ఋҊ оח ઺ -> ೡݠפ૘ਵ۽ оחѢঠ? Ex ) ࢎی೧! -> ਔ ><>< աوաو, ࢎی೯ -> ਽ աب Ex ) ই ૓૞ ޷஘ ѱ੐ೞҊ र׮ -> XX աب ѱ੐ XXೞҊ र׮ Ex ) ֎੉ߡ ଻౴ ࠈ: ੉ٮо োۅ೧! -> ਔ ੉ٮ ஠సೡѱ!
30. Query Centric Model ࠁ׮ ࠄ૕੸ਵ۽ ੌ࢚؀ചী ੉ ੽Ӕ੉ ࠛоמೠ ੉ਬ ੌ࢚؀ച э਷ ҃਋ীח ૕੄ী ؀ೠ Search Spaceо ցޖ ևӝ ٸޙী Q'ਸ Nѐ۽ ੿੄ೡ ࣻ হ਺ ੌ࢚؀ചо Ӕࠄ੸ਵ۽ য۰਍ ੉ਬח ழߡ ೧ঠ ೞח ؀ച੄ ߧਤо ޖೠೞѱ ௼ӝ ٸޙ੐ ޖೠ൤ ௾ Query Spaceী ݽٚ Query Variance ܳ ࢎۈ੉ ੌੌ੉ ଻਎ ࣻ হ਺ ࢎۈ਷ ؀ചܳ ೡ ٸ Query -> Information -> ੉ী ੸೤ೠ ؀׹ਸ ೣ (1:1 ؀਽੉ ইש)
31. Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ ࢎۈ੉ ٛח Query Spaceח ޖೠೞ૑݅ ࢎۈ੄ Reply Spaceח ਬೠೞ૑ ঋਸө? ࢎۈٜ਷ ઱۽ ೞח ݈੉ ੿೧ઉ ੓૑ ঋਸө? ੸੺ೞѱ ੜ Queryী ؀਽दఃݶ ؀ചо ࢿ݀ೡ ࣻ ੓૑ ঋਸө?
32. Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ ஠సਵ۽ ؀ചೡٸ ಣӐ੸ੋ ࢎۈٜ੄ Reply Space Reaction ই૓૞? ߓҊ೐׮ ցف? ੷۠ƀƀ ݏযݏয żż … Answer to Query ਽ ݡ঻য ইפ ই૒ উоࠌ૑ ծਫ਼੗ӝূ ই૒ ੌ۞ ,…. Context Aware Reply Ӕؘ џо ૓૞ Ӓۗয? ৬ ੉۠ ࢚ട਷ ખ ইפ૑ ইөࠗఠ ইޖѪب ޅݡ঻ਗ਼ই KB & Background Aware Reply য়ט ࢲ਎ 30بۆ׮ ੉ѱ աۄջ ց ৈ஘੉ی ೻য઎׮ݶࢲ ই ഘ؀ ୷ઁ ૑դ઱৓חؘ ޅщ׮Ҋ Ӓۗ૑
33. Reply Centric Model ೝಯ౱੄ ੽Ӕ ੹ۚ যו੿ب ਬೠೠ Reply Spaceܳ ೞաೞա ੼۸೧ աоݶࢲ ੌ࢚؀ച Coverageܳ ֫ৈࢲ ࢎۈэ਷ ؀ച ੋҕ૑מਸ ٜ݅੗! Context+Query -> Reply ী ؀ೠ Distributionਸ Deep Learningਸ ా೧ࢲ ೟णदఆ ࣻ ੓਺ +ষ୒աѱ ௾ ؘ੉ఠ۽ ୭Ҋ੄ NLU, NLGݽ؛ਸ ٜ݅ ࣻ ੓਺
34. Reaction և਷ Coverage, উ੹ೠ ׹߸ ܻঘ࣌਷ ؀ച઺ীࢲ о੢ ݆਷ Query Coverageܳ ыח ప௼ץ पઁ۽ ࢎۈٜ੉ о੢ ݆੉ ࢎਊೞח ؀ച ߑध ৘: ܻঘ࣌݅ ੜ೧ب {թ੗|ৈ੗}஘ҳীѱ ੉ࢄ߉ח ؀ചо оמೣ ੸੺ೠ ߈਽җ ࢚؀ߑ੄ ૕੄ী ؀ೠ ҕх਷ ؀ചܳ ਬبೞҊ গ଱ҙ҅ܳ ഋࢿೣ ୐ߣ૩ ݽ؛: Reaction
35. ݽٚ ؀ച੄ Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ੉ ؀ࠗ࠙ ݆੉ ࢎਊೞח ୨ 1200ѐ ੿ب੄ Reaction Class ੿੄ Reaction ই ইޖѪب উݡ঻য ƕ ੼बदр੉ঠ!!!! ߏਸ উоઉ৳֎…?? ೵ য়ט әध উաৡ؀ ࠗݽש੉ য়ט ߏਸ উ೧઱࣑য Ӓܶ੉ ղ ߊ ߃ীࢲ ӵ઎য.. ইөࠗఠ ߓо ցޖ ই೐֎. ে ߓҊ೐ѷ׮.. ਋ইই ߓҊ೐ѱٮঠ ೳ ૓૞ ߓҊ೐ѷ׮ ޷஘Ѣ ইջ? ૓૞ ߓҊ೐ѷ׮ ೵ ߓҊ౵ࢲ যڂ೧.. ೵ ҡଳই?!?! ?? ҡଳই?? և਷ Coverage, উ੹ೠ ׹߸
36. Reaction ই ইޖѪب উݡ঻য ƕ ੼बदр੉ঠ!!!! ߏਸ উоઉ৳֎…?? ೵ য়ט әध উաৡ؀ ࠗݽש੉ য়ט ߏਸ উ೧઱࣑য Ӓܶ੉ ղ ߊ ߃ীࢲ ӵ઎য.. ইөࠗఠ ߓо ցޖ ই೐֎. ে ߓҊ೐ѷ׮.. ਋ইই ߓҊ೐ѱٮঠ ೳ ૓૞ ߓҊ೐ѷ׮ ޷஘Ѣ ইջ? ૓૞ ߓҊ೐ѷ׮ ೵ ߓҊ౵ࢲ যڂ೧.. ೵ ҡଳই?!?! ?? ҡଳই?? ೵ ߓҊ೐ѷ׮ ƕƕ (ߓҊ೓ Class) ҡଳই?!?! (ѣ੿ Class) և਷ Coverage, উ੹ೠ ׹߸ ࣻୌ݅ѐ੄ Reaction Datasetਸ Labeling হ੉ ҳ୷ ݽٚ ؀ച੄ Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ੉ ؀ࠗ࠙ ݆੉ ࢎਊೞח ୨ 1200ѐ ੿ب੄ Reaction Class ੿੄
37. Reaction և਷ Coverage, উ੹ೠ ׹߸ ੉ۧѱ݅ ೧ب 1000݅+ ੉࢚੄ Reaction Training Pairܳ ٜ݅ ࣻ ੓঻਺ ؀ച ؘ੉ఠо ੓ਵפ ׮নೠ Query Varianceܳ ੗زਵ۽ ٜ݅ ࣻ ੓਺
38. Reaction և਷ Coverage, উ੹ೠ ׹߸ Classification Model Retrieval Model Queryinput Query Encoder Ensemble XGBoost Reranking Reaction Class Select Among 1K class ૓૞ ൨ٜ঻ѷ׮ ƕƕ લ૑݃ই!! …
39. Reaction և਷ Coverage, উ੹ೠ ׹߸ ؀಴ Response ٘ܿ੉ Response Reaction Class ੸਷ন੄ Response݅ ੘ࢿ೧ب ಕܰࣗաܳ ٘۞յ ࣻ ੓਺ ࠈীѱ গ଱ҙ҅ܳ ഋࢿೞѱ ೣ
40. Reaction https://youtu.be/Cabc5JOPqEk https://goo.gl/yANLGz ٘ܿ੉ی ؀ച ೧ࠁӝ
41. Pros: 1. ੌ࢚؀ച ழߡܻ૑о ݒ਋ ֫׮. ਟ݅ೞݶ ؀ࠗ࠙੄ ׹ী ֤ܻ੸ੋ ׹߸ਸ ೡ ࣻ ੓਺. 2. ׹߸੄ ݈ైܳ ࣻ੿ೡ ࣻ ੓যࢲ, ಕܰࣗաܳ ߈৔ೞৈ ੌ࢚؀ച ࠈਸ ٜ݅ ࣻ ੓਺ Reaction և਷ Coverage, উ੹ೠ ׹߸ Cons: 1. উ੹ೠ ׹߸ਸ ୶ҳ ೣਵ۽ࢲ ҳ୓੸ੋ ׹߸ਸ ղࠁղ૑ ޅೣ 2. ܻঘ࣌ਸ ઺बਵ۽ ೞ׮ࠁפ, ؀ചܳ ੉যաоӝ ൨ٝ
42. Reply Retrieval Model ҳ୓੸ੋ ׹߸, ੸੺ೠ ؀׹ ಕܰࣗաо ߈৔ػ ׹߸ٜਸ ੘ࢿ೧઱ݶ ঌইࢲ ૕੄(Q)ী ؀೧ о੢ ੸੺ೞݶࢲ ҳ୓੸ੋ ׹߸ਸ ղࠁն ಕܰࣗա, ࢚ध ١ী ؀ೠ ੿ࠁܳ ׸਷ ҳ୓੸ੋ ׹߸ਸ ੘ࢿೞৈ ੌ࢚؀ചܳ ܻ٘ೞѢա ੉যт ࣻ ੓ب۾ ب਑ਸ ઱ח ݽ؛ ܻঘ࣌ ݽ؛੄ ҳ୓ࢿ੉ হח ޙઁ੼ਸ Mainਵ۽ ೧Ѿ
43. Reply Retrieval Model Next Utterance Prediction Model https://arxiv.org/abs/1705.02364 Supervised Learning of Universal Sentence Representations from Natural Language Inference Data Query Reply Query৬ Replyܳ ઱Ҋ ੉ѱ Continuousೠ Utteranceੋ૑ Next Utterance Classification Response = argmax(Model(Equery , Ereplyi ))
44. Reply Retrieval Model Next Utterance Prediction Model ࢎਊ੗੄ ஠஠য়స ؘ੉ఠܳ ߄ఔਵ۽ Query-Reply 50রѤ੄ Pairहਸ ݅ٞ (single ఢ ؀ച) Query, Replyܳ Sentence Encoderী ֍Ҋ concat೧ࢲ FNNਸ కਕ Binary Classification Pairؘ੉ఠܳ ੉ਊ೧ Next Utterance Prediction (BERT੄ Next Sentence Prediction) N:1 ࠺ਯ۽ Negative Sampling ೧ࢲ Random ಕযب ೟ण (੉ٸ N਷ Negative੄ іࣻ)
45. Reply Retrieval Model Next Utterance Prediction Model Reply Set
46. Reply Retrieval Model Next Utterance Prediction Model: Unbalanced ࢎۈٜ੉ Reactionਸ ցޖ ݆੉೧ࢲ উ੹ೠ ׹߸ਸ ೞب۾ ೟ण੉ غযߡܿ ҳ୓੸ੋ ׹߸੉ ਋ࢶद غب۾ Reranking Logic੉ ೙ਃ۽ ೮਺ (੉ ࠗ࠙ী ݆਷ दрਸ ٜ੐) ਋ܻ૘ ъই૑о ૘ਸ աщ૑ ޤঠ ೳ ই૒ ޅ଺ওয? Q 0.79 ೵ ૓૞ਃ? 0.92 Insight: ݈ਸ ੜೞח ୁࠈਸ ݅٘۰ݶ Ӓր ࢎۈٜ੉ ೞח ݈݅ ߓ਋ݶ উغщҳա.. ೟णਸ ೞ׮ ࠁפ…
47. Reply Retrieval Model Next Utterance Prediction Model: Unbalanced RMM Prediction Result ੉ઁ ׮নೠ ݈ਸ ೡ ઴ ইח ೝಯ ؊੉࢚ ܻঘ࣌݅ ೞ؍ ೝಯ੉ ইפঠ!
48. Reply Retrieval Model Next Utterance Prediction Model Pros: 1. Reaction ࠁ׮ ഻ঁ ҳ୓੸ੋ ׹߸ਸ ղ֬਺( য়ט਷ {૞੢ݶ,૤ࡴ}੉ ݡҊ र਷ Ѧ → աب! ఔࣻਭب ݡҊ र׮) 2. ࠈ ੗न(?)ী ؀ೠ ղਊ ࡺ ইפۄ ࢚ध੉ա ࢎਊ੗ী ؀ೠ ૕ޙ ١ب ֍ਸ ࣻ ੓যࢲ ؀ചܳ ؊ ੉যաт ࣻ ੓਺ 3. ࠈ ӝദ੗о Qী ߧਤܳ ੌੌ੉ ੑ۱ೞ૑ ঋইب ؽ Cons 1. ই૒਷ ӝദ੗о ׹߸(R)ী ؀ೠ Spaceܳ ੌੌ੉ ੘ࢿ೧ঠ ೣ. 1. Ӓېࢲ ৈ۞ߣ పझ౟ܳ ೞݶࢲ ಕܰࣗաী ౠചػ Reply setਸ ࣻ੿೧ঠೣ 2. ੘ࢿೠ Replyо ౠ੿ Qী աৡ׮ח ࠁ੢੉ হӝ ٸޙী, Qܳ ৘ஏೞӝо য۰਑.
49. Qܳ ৘ஏೞӝ য۵׮ח Reply retrieval model੄ Consܳ ӓࠂೞӝ ਤ೧ ѐߊ. ౠ൤ ೐۽೙ ׹߸੄ ҃਋ ࢎਊ੗о ૕ޙਸ ೡ ҃਋ ԙ ׹ਸ ೧઻ঠೞחؘ, Reply Retrieval Model਷ Ӓ۞ܻۄח ࠁ੢੉ হ਺. Query Retrieval Model Motivation ࢎਊ੗੄ ૕੄ী ৮߷ೞѱ 1:1۽ ؀਽غח ׹߸ਸ ҳഅೞӝ ਤ೧ ѐߊ
50. Query Retrieval Model Using Next-Utterance Task Pretrained Encoding ߡ౟ীࢲب LM+Next Sentence Predictionਵ۽ context ೟ण Ӓۢ Next Utterance Sub-Taskܳ ೟णೠ Encoderਸ ࢎਊೞݶ જ਷ Embeddingਸ ыҊ ੓ਸө? Query Reply
51. ࢤпࠁ׮ ҡଳ਷ؘ???! Query Retrieval Model Dataset
52. ೞ૑݅ ੗ࣁ൤ ଺ইࠁݶ ࠺तೠ ݈੉૑݅ ׮ܲ ੄޷ٜ੉ ੓਺ Query Retrieval Model Dataset ਊ঱੉ա ୓঱੉ زੌೞݶ ޹хೞѱ ߈਽ೞח ҃਋о ݆਺
53. Query Retrieval Model Dataset ௼ۄ਋٘ࣗयਵ۽ “э਷ ੄޷ ޙ੢ ଺ӝ” పझ௼ ૓೯ 50000ѐ ಕযী ؀ೠ ؀ച୓ э਷ ੄޷ ଺ӝ ؘ੉ఠࣇ ѐߊ
54. Query Retrieval Model Using Next-Utterance Task Pretrained Encoding Query Query زੌೠ ޙ੢ ଺ӝ పझ௼۽ Transfer Learning Datasetਸ ੉ਊ೧ࢲ Fine-Tuning
55. https://docs.google.com/spreadsheets/d/1LWsrffq93u5U55-O7Yij8Lhl15Dlbr8mlucabTUB9jo/edit#gid=1973099187 Query Retrieval Model Result: SOTA of Chatbot Builder য়৘!! ੹୓ SOTA!! RAW Ѿҗ &Test Dataset
56. Query Retrieval Model Result: SOTA of Chatbot Builder
57. Query Retrieval Model Pros and Cons Cons: 1. э਷ Qী ؀೧ ೠ੿ػ ׹߸݅ ઁҕೞӝ ٸޙী variationਸ ݅٘ח ؘ ೠ҅о ੓਺ Pros: 1. ӝઓ ୁࠈ ࠽؊ীࢲ ࢎਊೞ؍ ߑध੉ۄ ࢎۈٜীѱ ੊ࣼೣ 2. ࢎਊ੗੄ ݈ী ؀೧ যڃ ׹߸ਸ ղࠁյ૑ ࠺Ү੸ ాઁೞӝ ए਑ 3. ֎੉ߡ, ஠஠য়, Dialogflow ١ ݽٚ ࠽؊ী ੓ח ݽ؛੉૑݅, Ӓ઺ীࢲب SOTA
58. https://landing.pingpong.us/ Pingping Builder Chatbot Builder For Open-Domain Chatbot
59. Pingping Bot Prebuilt Bot & App ֎੉ߡ ݏ୺ഋ ؘݽੑפ׮ ^^ https://m.me/ai.pingpong
60. Pingping Bot Prebuilt Bot & App ъ۱ೠ ೲࣁ ಕܰࣗա৬ ߊۇೠ ಕܰࣗաܳ ղࡼח ೝಯ੄ ف ই੉ٜ ౵੉౴ ܖա Ӓթ੗ ೲࣁ઺
61. Pingping Bot Prebuilt Bot & App ׮ܲ ݒ۱੓ח ࠈٜب COMMING SOOOOON Pingpong Family
62. оө਍ Future Works NER, Tokenizer, Typo-Correction, Semantic Parsing Question and Answering : external KB, user KB Persona Reply Style Transfer Reply Generation Context Aware Reply End-to-End Dialog System Hyper-Personalization SOTA Language Modeling ೧ঠೡ ੌ੉ ցޖ ݆ইਃ
63. ਃ્ ೞҊ ੓ח ੌ ೧ঠೡ ੌ੉ ցޖ ݆ইਃ Context Aware BERT for Open-Domain Dialog System
64. 감사합니다 ੌ࢚؀ച ੋҕ૑מ