Share
Embed
[네이버 테크톡] 사람들과 자연스러운 대화를 나누는 인공지능 '핑퐁' 만들기
100억 건의 카카오톡 데이터로 어떻게 똑똑하고 생동감 넘치는 대화형 인공지능을 만들었는지 공유하는 세션입니다. 인공지능을 위한 일상대화 기술인 '핑퐁'과, 사용자가 직접 새로운 패르소나의 캐릭터를 만들수 있게 해주는 "핑퐁 빌더"를 소개합니다. 기존 FAQ 방식의 챗봇 모델이 아닌, 카카오톡 데이터의 장점을 활용할 수 있는 Reply-centric 기반의 학습 방법을 공유합니다. 또한 세션 마지막에는 핑퐁이 얼마나 똑똑하고, 인간 같은 대화를 할 수 있는지 보여주는 놀라운 데모가 준비되어 있습니다. 지금도 m.me/ai.pingpong 에서 페이스북 메신저로 핑퐁과 대화해보실 수 있습니다!
Junseong
May 27, 2019
Transcript
64 slides
-
사람들과 자연스러운 대화를 나누는 인공지능 ‘핑퐁’ 만들기 스캐터랩(ScatterLab) Machine Learning
Engineer 김준성 ੌ࢚ച ੋҕמ 스캐터랩(ScatterLab) CEO 김종윤 -
ݾର 1. ೝಯ য়ߡ࠭ 2. ؘఠࣇ ࣗѐ & ಽ۰Ҋ ೞח
ޙઁ 3. Query Centric Model : ࠗ࠙ ୁࠈ ࢎਊೞҊ ח ߑध 4. Reaction Model: ࢎۈٜ ݈ী ୭ೠ ߈೧ࠁ! 5. Reply Retrieval Model: ҳੋ ߸ਸ ೡ ࣻ ب۾ ೧ࠁ! 6. Query Retrieval Model: ݃ ೝಯ ց QAب ੜೞա? 7. Pingpong Bot & Builder : ܻ ݧ੬ٜ ࣗѐ 8. Future Works: ܻח খਵ۽ ޖट ੌਸ ೡө? -
#0. Ӕؘ ੌ࢚ചо ഛ ޤભ? ӝמച = “য়ט ࢲ զॿח?”
ੌ࢚ച = “য়ט զॿ લੋ” -
য়ט ࢲ ӝৡ ೞ 20بݴ զॿח ݉णפ. ъࣻ ഛܫ 10%ੑפ.
Ӓ۞ѱਃ. ৮ ೠъী ೖץ оҊ र զॿ֎ਃ. ӝמച = “য়ט ࢲ զॿח?” ੌ࢚ച = “য়ט զॿ લੋ” #0. Ӕؘ ੌ࢚ചо ഛ ޤભ? -
ӝמ ച (: “য়ט ࢲ զॿח?”) ੌ࢚ ച (: “ற
য়ט զॿ લੋ”) ച ݾ ࠁܳ ח ਗೞח ೯زਸ ೞب۾ ೠ ച о ݾ о ಞܻೣ Ӕೣ ച ઁ ੋҕמ ӝמী ݏח ઁ ݽٚ ઁ ޙ ഋక ݺ۸, ޙ ࢤпೡ ࣻ ח ݽٚ ޙ ഋక ਬޖ ݺഛೠ ઓ ৈ۞ѐੌ ࣻ #0. Ӕؘ ੌ࢚ചо ഛ ޤભ? -
ী ࢤޛ ҕಿ ޖࢤޛ Button UI NLI NLI GUI #1.
Ҷ ੌ࢚ചо ৵ ਃೞભ? Natural Language Interface = ઁಿ ੋрച -
( ઑࢶੌࠁ, 2017֙ 5ਘ 3ੌ ) ࢎۈٜ SKT ‘־ҳ’ীѱ Ѣח
݈ 45%о хࢿੋ ച बब೧ ҽݽ ೧ ੜ 1) Nass, Clifford, and Li Gong. "Speech interfaces from an evolutionary perspective." Communications of the ACM 43.9 (2000): 36-43. 1. ࠗੋ ച ҃ хࣗ • ࢎਊח ചഋ AIܳ ݃ ࢎۈۢ ೞח ҃ೱ ਵݴ1) ী ٮۄ AIীѱ নೠ ചܳ दبפ. • Ӓ۞ա അ AIח ܻೡ ࣻ ח ചо ౠ ӝמҗ ઁী ೠ غয ਵݴ, ࠗ࠙ ݈ী ೧ “ޖट ݈ੋ ੜ ݽܰѷযਃ” э ݫद۽ פ. ۞ೠ ੋఠۑ࣌ ࢎਊীѱ ࠗੋ ҃ਸ ࣻ߆ী হणפ. • ੌ࢚ച מ۱ ࠗੋ ച ҃ਸ хࣗदఃח زदী, ࢎਊ ۽ ೞৈӘ AIܳ ഻ঁ ؊ ੋр(human-likeness)Ҋ ࢤزх ѱ וՙѱ ٜ݅যસפ. #1. Ҷ ੌ࢚ചо ৵ ਃೞભ? (Jiang J, et al. (2015)) ࢎۈٜ Microsoft Cortanaীѱ Ѣח ݈ 30%о ࣗ࣍ ച -
2. ࢎਊ ੋѱݢ ૐ • ઁೠ হח য় بݫੋ ੌ࢚ച
מ۱ ࢎਊо AI৬ ചܳ ؊ ૌӡ ࣻ ب۾ ٜ݅যݴ, ী ٮۄ ࣁ࣌ ച ఢ ࣻ (CPS, Conversation-turns per session)৬ п ఢ ޙ ӡ о טযժפ1). • Ѿҗਵ۽ ੌ࢚ച מ۱ ࢎਊо AI ઁಿਸ ؊ য়ې ࢎਊೞ ب۾ ਬبೞݴ, زदী নೠ ࢎਊী ೠ ࠁܳ ਸ ࣻ ण פ. #1. Ҷ ੌ࢚ചо ৵ ਃೞભ? 1) Chen, Chun-Yen, et al. "Gunrock: Building A Human-Like Social Bot By Leveraging Large Scale Real User Data." -
3. ࠳ے٬җ ҙ҅ ഋࢿ • AIח ੌ࢚ചܳ ా೧ न݅ ಕܰࣗաܳ
٘۞յ ࣻ ਵݴ, ܲ AI৬ ର߹ചغח Ҋਬ ࠳ے٬җ நܼఠܳ оѱ ؾפ. • ੌ࢚ച מ۱ AI ѐࢿਸ ܻҊ ࢎਊীѱ न݅ ו՝ਸ ా೧ ౠ߹ೠ ҙ҅ܳ ഋࢿೡ ࣻ ب۾ ذणפ. #1. Ҷ ੌ࢚ചо ৵ ਃೞભ? -
ೞ݅, ੌ࢚ചח য۵णפ • ੌ࢚ചח ઁա ഋకо ޖೠ নפ. AIо
ੌ࢚ചী ੌੌ ೡ ࣻ ѱ ٜ݅ӝ ਤ೧ࢲח झ݀ ࢿী ষդ दрҗ ֢۱ ਃפ. ܖೠ ੌӝب ೞભ. • Ӓۧѱ ೠҊ ೧ب ੌ࢚ച ӓ ੌࠗ࠙݅ ழߡೡ ࣻ ਸ ࡺ ੑפ. #1. Ҷ ੌ࢚ചо ৵ ਃೞભ? -
Vision Make machines social ݈ ੜ ాೞѱ ച۽ ੋఠۑ࣌ೡ ࣻ
ѱ ઁಿ ইפۄ ઓ۽ ࢎਊ ࢚ ইפۄ ҙ҅ ࢚ਵ۽ “ъই݅ఀ ࣗೠ ੋҕמ” -
Source: Optional source placeholder. Delete if not used. ӣળࢿ (Junseong
Kim) fb.com/codertimo ScatterLab Machine Learning Engineer ੌ࢚ച ੋҕמ ೝಯ Machine Learning Team (সӝמਃਗ) Naver Clova AI Research Intern Atlas Guide NLP/Machine Learning Engineer General Domain Dialog System/Chatbot (Chitchat) General Sentence Embedding (BERT, ELMo) And Diverse Natural Language Processing Tasks… Neural Machine Translation codertimo/BERT-pytorch 2.4k+ stars Research Domain Experience 2017, 2014 Intel ISEF Computer Science Finalist 2016, 2014 Intel KSEF Computer Science Grand Award Working Experience -
Dataset 100র+ Utterance(2.2TB) ਬ ച ؘఠ ఫझ & োগ җ
: োগܳ ਤೠ ੋҕמ Utterance Variance Context Aware Utterance Massive Dialog System … NLP/Dialog System ݠन۞ցীѲ -
Goal Engineer ҙীࢲ Main Goal 100রѤ ച ؘఠܳ ୭ೠ ण೧
ࢎۈэ ੋҕמਸ ٜ݅! ӒܻҊ о ബਯੋ ݠन۞ ݽ؛җ णҗਸ ইࠁ ޖਸ ೧ঠ ೞחо? -
Query Centric Model ஶࣆ ࢸݺ ੌױ о ए ߑߨࠗఠ ب೧ࠁ!
э Queryܳ ইࢲ ী غח ߸ਸ ղࠁղ য়స ؘఠо ਵפӬ ߸ب ׳۰ਗ਼ই?!?! -
Query Centric Model ஶࣆ ࢸݺ о ݆ ࢎਊغח Query Matching
ߑध ୁࠈ ݽ؛ Sentence Representationਸ ण೧ࢲ Cosine-Similarityо ୭ച غח Scriptܳ Retrievalೞѱ ӝ Response = argmax(cos(Queryembed , Scriptembedi )) -
Query Centric Model উ֞? ߓҊ ࠁҊर ਵҳ Query Space ࢎਊо
۠ ݈ਸ ೡ Ѫ эই! ஶࣆ ࢸݺ о ݆ ࢎਊغח Query Matching ߑध ୁࠈ ݽ؛ Sentence Representationਸ ण೧ࢲ Cosine-Similarityо ୭ച غח Scriptܳ Retrievalೞѱ ӝ Response = argmax(cos(Queryembed , Scriptembedi )) -
Query Centric Model উ֞? ߓҊ ࠁҊर ਵҳ ݅աࢲ ߈оਕਃ ب
ߓҊਃ աف ৵ Ӓې?? Query Space ۠ ޙ য়ݶ ۠ ߸ਸ ೞ ஶࣆ ࢸݺ о ݆ ࢎਊغח Query Matching ߑध ୁࠈ ݽ؛ Sentence Representationਸ ण೧ࢲ Cosine-Similarityо ୭ച غח Scriptܳ Retrievalೞѱ ӝ Response = argmax(cos(Queryembed , Scriptembedi )) -
Query Centric Model উ֞? ߓҊ ࠁҊर ਵҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ
ب ߓҊਃ աف ৵ Ӓې?? Query Space ࢎਊ ஶࣆ ࢸݺ о ݆ ࢎਊغח Query Matching ߑध ୁࠈ ݽ؛ Sentence Representationਸ ण೧ࢲ Cosine-Similarityо ୭ച غח Scriptܳ Retrievalೞѱ ӝ Response = argmax(cos(Queryembed , Scriptembedi )) -
Query Centric Model উ֞? ߓҊ ࠁҊर ਵҳ উ֞ೞࣁਃ! ݅աࢲ ߈оਕਃ
ب ߓҊਃ աف ৵ Ӓې?? Query Space ஶࣆ ࢸݺ о ݆ ࢎਊغח Query Matching ߑध ୁࠈ ݽ؛ Sentence Representationਸ ण೧ࢲ Cosine-Similarityо ୭ച غח Scriptܳ Retrievalೞѱ ӝ Response = argmax(cos(Queryembed , Scriptembedi )) -
Query Centric Model য়స Query Retrieval Model: Pingpong-POC ؘఠ •
য়స ؘఠ ীࢲ ࢎਊ ࠁо ನೣغ ঋ ޙਸ Rule Based Filtering • ࠺ࣘয, ࢶ, ࠈ ঠӝ ೞӝী ೞ ঋ ղਊਸ Rule Based Filtering • ఠ݂ ػ ؘఠٜਸ ࣻ Ѩࢎೞৈ ೞݶ উغח ղਊਸ Human Filtering • -> ೝಯ ࢎਊೡ ࣻ ח Query-Reply Pairٜਸ ٜ݅য م -
Query Centric Model য়స Query Retrieval Model: Pingpong-POC ؘఠ Q->
݆ ݆ ࢎی೧ਃ! য়స ؘఠ ࣇ Reply select: աب ݆݆ ࢎی೧ਃ ਬࢎ Q’:য়טب ݆݆ ࢎی೧ਃ -
Query Centric Model য়స Query Retrieval Model: Pingpong-POC ޙઁ •
߸ਵ۽ ࢎਊغח য়స ޙݓ ࠁܳ Ҋ যࢲ য࢝ೠ ҃о ݆ • ಕܰࣗաܳ ਬೡ ࣻ হח ޙઁ → ࠈ ಕܰࣗաܳ ߸ী ֣ৈյ ࣻ হ (ݒ ߸݃ ݈ైо ܰҊ ݽف ) • Blackbox → ࠈਸ ࢎਊೞח ࢎਊ(B2B)ҙীࢲ ߸ਸ controlೡ ࣻ হ → customization ࠛоמೣ • Query Search Variance ܳ טܻ۰ݶ Ӓ݅ఀ য়స ؘఠܳ ࣻ Ѩࢎ ೧ঠೣ → উӒ۞ݶ बब э ޙઁ Ex ) ೞ୍ ఋҊ оח -> ೡݠפਵ۽ оחѢঠ? Ex ) ࢎی೧! -> ਔ ><>< աوաو, ࢎی೯ -> աب Ex ) ই ѱೞҊ र -> XX աب ѱ XXೞҊ र Ex ) ֎ߡ ࠈ: ٮо োۅ೧! -> ਔ ٮ సೡѱ! -
Query Centric Model ࠁ ࠄਵ۽ ੌ࢚ചী Ӕ ࠛоמೠ ਬ
ੌ࢚ച э ҃ীח ী ೠ Search Spaceо ցޖ ևӝ ٸޙী Q'ਸ Nѐ۽ ೡ ࣻ হ ੌ࢚ചо Ӕࠄਵ۽ য۰ ਬח ழߡ ೧ঠ ೞח ച ߧਤо ޖೠೞѱ ӝ ٸޙ ޖೠ Query Spaceী ݽٚ Query Variance ܳ ࢎۈ ੌੌ ࣻ হ ࢎۈ ചܳ ೡ ٸ Query -> Information -> ী ೠ ਸ ೣ (1:1 ইש) -
Reply Centric Model ೝಯ Ӕ ۚ ࢎۈ ٛח Query Spaceח
ޖೠೞ݅ ࢎۈ Reply Spaceח ਬೠೞ ঋਸө? ࢎۈٜ ۽ ೞח ݈ ೧ઉ ঋਸө? ೞѱ ੜ Queryী दఃݶ ചо ࢿ݀ೡ ࣻ ঋਸө? -
Reply Centric Model ೝಯ Ӕ ۚ సਵ۽ ചೡٸ ಣӐੋ ࢎۈٜ
Reply Space Reaction ই? ߓҊ ցف? ۠ƀƀ ݏযݏয żż … Answer to Query ݡয ইפ ই উоࠌ ծਫ਼ӝূ ই ੌ۞ ,…. Context Aware Reply Ӕؘ џо Ӓۗয? ৬ ۠ ࢚ട ખ ইפ ইөࠗఠ ইޖѪب ޅݡਗ਼ই KB & Background Aware Reply য়ט ࢲ 30بۆ ѱ աۄջ ց ৈی যݶࢲ ই ഘ ୷ઁ դחؘ ޅщҊ Ӓۗ -
Reply Centric Model ೝಯ Ӕ ۚ যוب ਬೠೠ Reply Spaceܳ
ೞաೞա ۸೧ աоݶࢲ ੌ࢚ച Coverageܳ ֫ৈࢲ ࢎۈэ ച ੋҕמਸ ٜ݅! Context+Query -> Reply ী ೠ Distributionਸ Deep Learningਸ ా೧ࢲ णदఆ ࣻ +ষաѱ ؘఠ۽ ୭Ҋ NLU, NLGݽ؛ਸ ٜ݅ ࣻ -
Reaction և Coverage, উೠ ߸ ܻঘ࣌ ചীࢲ о ݆ Query
Coverageܳ ыח పץ पઁ۽ ࢎۈٜ о ݆ ࢎਊೞח ച ߑध : ܻঘ࣌݅ ੜ೧ب {թ|ৈ}ҳীѱ ࢄ߉ח ചо оמೣ ೠ ߈җ ࢚ߑ ী ೠ ҕх ചܳ ਬبೞҊ গҙ҅ܳ ഋࢿೣ ߣ૩ ݽ؛: Reaction -
ݽٚ ച Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ ࠗ࠙ ݆
ࢎਊೞח ୨ 1200ѐ ب Reaction Class Reaction ই ইޖѪب উݡয ƕ बदрঠ!!!! ߏਸ উоઉ৳֎…?? য়ט әध উաৡ ࠗݽש য়ט ߏਸ উ೧࣑য Ӓܶ ղ ߊ ߃ীࢲ ӵয.. ইөࠗఠ ߓо ցޖ ই֎. ে ߓҊѷ.. ইই ߓҊѱٮঠ ೳ ߓҊѷ Ѣ ইջ? ߓҊѷ ߓҊࢲ যڂ೧.. ҡଳই?!?! ?? ҡଳই?? և Coverage, উೠ ߸ -
Reaction ই ইޖѪب উݡয ƕ बदрঠ!!!! ߏਸ উоઉ৳֎…?? য়ט
әध উաৡ ࠗݽש য়ט ߏਸ উ೧࣑য Ӓܶ ղ ߊ ߃ীࢲ ӵয.. ইөࠗఠ ߓо ցޖ ই֎. ে ߓҊѷ.. ইই ߓҊѱٮঠ ೳ ߓҊѷ Ѣ ইջ? ߓҊѷ ߓҊࢲ যڂ೧.. ҡଳই?!?! ?? ҡଳই?? ߓҊѷ ƕƕ (ߓҊ Class) ҡଳই?!?! (ѣ Class) և Coverage, উೠ ߸ ࣻୌ݅ѐ Reaction Datasetਸ Labeling হ ҳ୷ ݽٚ ച Replyܳ Edit Distanceਸ ӝળਵ۽ Clustering ࢎۈٜ ࠗ࠙ ݆ ࢎਊೞח ୨ 1200ѐ ب Reaction Class -
Reaction և Coverage, উೠ ߸ ۧѱ݅ ೧ب 1000݅+ ࢚ Reaction
Training Pairܳ ٜ݅ ࣻ ച ؘఠо ਵפ নೠ Query Varianceܳ زਵ۽ ٜ݅ ࣻ -
Reaction և Coverage, উೠ ߸ Classification Model Retrieval Model Queryinput
Query Encoder Ensemble XGBoost Reranking Reaction Class Select Among 1K class ൨ٜѷ ƕƕ લ݃ই!! … -
Reaction և Coverage, উೠ ߸ Response ٘ܿ Response Reaction
Class ন Response݅ ࢿ೧ب ಕܰࣗաܳ ٘۞յ ࣻ ࠈীѱ গҙ҅ܳ ഋࢿೞѱ ೣ -
Pros: 1. ੌ࢚ച ழߡܻо ݒ ֫. ਟ݅ೞݶ ࠗ࠙ ী ֤ܻੋ
߸ਸ ೡ ࣻ . 2. ߸ ݈ైܳ ࣻೡ ࣻ যࢲ, ಕܰࣗաܳ ߈ೞৈ ੌ࢚ച ࠈਸ ٜ݅ ࣻ Reaction և Coverage, উೠ ߸ Cons: 1. উೠ ߸ਸ ୶ҳ ೣਵ۽ࢲ ҳੋ ߸ਸ ղࠁղ ޅೣ 2. ܻঘ࣌ਸ बਵ۽ ೞࠁפ, ചܳ যաоӝ ൨ٝ -
Reply Retrieval Model ҳੋ ߸, ೠ ಕܰࣗաо ߈ػ ߸ٜਸ
ࢿ೧ݶ ঌইࢲ (Q)ী ೧ о ೞݶࢲ ҳੋ ߸ਸ ղࠁն ಕܰࣗա, ࢚ध ١ী ೠ ࠁܳ ҳੋ ߸ਸ ࢿೞৈ ੌ࢚ചܳ ܻ٘ೞѢա যт ࣻ ب۾ بਸ ח ݽ؛ ܻঘ࣌ ݽ؛ ҳࢿ হח ޙઁਸ Mainਵ۽ ೧Ѿ -
Reply Retrieval Model Next Utterance Prediction Model https://arxiv.org/abs/1705.02364 Supervised Learning
of Universal Sentence Representations from Natural Language Inference Data Query Reply Query৬ Replyܳ Ҋ ѱ Continuousೠ Utteranceੋ Next Utterance Classification Response = argmax(Model(Equery , Ereplyi )) -
Reply Retrieval Model Next Utterance Prediction Model ࢎਊ য়స ؘఠܳ
߄ఔਵ۽ Query-Reply 50রѤ Pairहਸ ݅ٞ (single ఢ ച) Query, Replyܳ Sentence Encoderী ֍Ҋ concat೧ࢲ FNNਸ కਕ Binary Classification Pairؘఠܳ ਊ೧ Next Utterance Prediction (BERT Next Sentence Prediction) N:1 ࠺ਯ۽ Negative Sampling ೧ࢲ Random ಕযب ण (ٸ N Negative іࣻ) -
Reply Retrieval Model Next Utterance Prediction Model: Unbalanced ࢎۈٜ Reactionਸ
ցޖ ݆೧ࢲ উೠ ߸ਸ ೞب۾ ण غযߡܿ ҳੋ ߸ ࢶद غب۾ Reranking Logic ਃ۽ ೮ ( ࠗ࠙ী ݆ दрਸ ٜ) ܻ ъইо ਸ աщ ޤঠ ೳ ই ޅওয? Q 0.79 ਃ? 0.92 Insight: ݈ਸ ੜೞח ୁࠈਸ ݅٘۰ݶ Ӓր ࢎۈٜ ೞח ݈݅ ߓݶ উغщҳա.. णਸ ೞ ࠁפ… -
Reply Retrieval Model Next Utterance Prediction Model: Unbalanced RMM Prediction
Result ઁ নೠ ݈ਸ ೡ ইח ೝಯ ؊࢚ ܻঘ࣌݅ ೞ؍ ೝಯ ইפঠ! -
Reply Retrieval Model Next Utterance Prediction Model Pros: 1. Reaction
ࠁ ഻ঁ ҳੋ ߸ਸ ղ֬( য়ט {ݶ,ࡴ} ݡҊ र Ѧ → աب! ఔࣻਭب ݡҊ र) 2. ࠈ न(?)ী ೠ ղਊ ࡺ ইפۄ ࢚धա ࢎਊী ೠ ޙ ١ب ֍ਸ ࣻ যࢲ ചܳ ؊ যաт ࣻ 3. ࠈ ӝദо Qী ߧਤܳ ੌੌ ੑ۱ೞ ঋইب ؽ Cons 1. ই ӝദо ߸(R)ী ೠ Spaceܳ ੌੌ ࢿ೧ঠ ೣ. 1. Ӓېࢲ ৈ۞ߣ పझܳ ೞݶࢲ ಕܰࣗաী ౠചػ Reply setਸ ࣻ೧ঠೣ 2. ࢿೠ Replyо ౠ Qী աৡח ࠁ হӝ ٸޙী, Qܳ ஏೞӝо য۰. -
Qܳ ஏೞӝ য۵ח Reply retrieval model Consܳ ӓࠂೞӝ ਤ೧ ѐߊ.
ౠ ۽ ߸ ҃ ࢎਊо ޙਸ ೡ ҃ ԙ ਸ ೧ঠೞחؘ, Reply Retrieval Model Ӓ۞ܻۄח ࠁ হ. Query Retrieval Model Motivation ࢎਊ ী ৮߷ೞѱ 1:1۽ غח ߸ਸ ҳഅೞӝ ਤ೧ ѐߊ -
Query Retrieval Model Using Next-Utterance Task Pretrained Encoding ߡীࢲب LM+Next
Sentence Predictionਵ۽ context ण Ӓۢ Next Utterance Sub-Taskܳ णೠ Encoderਸ ࢎਊೞݶ જ Embeddingਸ ыҊ ਸө? Query Reply -
ೞ݅ ࣁ ইࠁݶ ࠺तೠ ݈݅ ܲ ٜ Query Retrieval
Model Dataset ਊա زੌೞݶ хೞѱ ߈ೞח ҃о ݆ -
Query Retrieval Model Dataset ۄ٘ࣗयਵ۽ “э ޙ ӝ” పझ
೯ 50000ѐ ಕযী ೠ ച э ӝ ؘఠࣇ ѐߊ -
Query Retrieval Model Using Next-Utterance Task Pretrained Encoding Query Query
زੌೠ ޙ ӝ పझ۽ Transfer Learning Datasetਸ ਊ೧ࢲ Fine-Tuning -
Query Retrieval Model Pros and Cons Cons: 1. э Qী
೧ ೠػ ߸݅ ઁҕೞӝ ٸޙী variationਸ ݅٘ח ؘ ೠ҅о Pros: 1. ӝઓ ୁࠈ ࠽؊ীࢲ ࢎਊೞ؍ ߑधۄ ࢎۈٜীѱ ࣼೣ 2. ࢎਊ ݈ী ೧ যڃ ߸ਸ ղࠁյ ࠺Ү ాઁೞӝ ए 3. ֎ߡ, য়, Dialogflow ١ ݽٚ ࠽؊ী ח ݽ؛݅, Ӓীࢲب SOTA -
Pingping Bot Prebuilt Bot & App ֎ߡ ݏഋ ؘݽੑפ ^^
https://m.me/ai.pingpong -
Pingping Bot Prebuilt Bot & App ъ۱ೠ ೲࣁ ಕܰࣗա৬ ߊۇೠ
ಕܰࣗաܳ ղࡼח ೝಯ ف ইٜ ܖա Ӓթ ೲࣁ -
Pingping Bot Prebuilt Bot & App ܲ ݒ۱ח ࠈٜب COMMING
SOOOOON Pingpong Family -
оө Future Works NER, Tokenizer, Typo-Correction, Semantic Parsing Question and
Answering : external KB, user KB Persona Reply Style Transfer Reply Generation Context Aware Reply End-to-End Dialog System Hyper-Personalization SOTA Language Modeling ೧ঠೡ ੌ ցޖ ݆ইਃ -
ਃ્ ೞҊ ח ੌ ೧ঠೡ ੌ ցޖ ݆ইਃ Context Aware
BERT for Open-Domain Dialog System