$30 off During Our Annual Pro Sale. View Details »

Scaling and Potential Uses for the HyperCLOVA Japanese Base Model

Scaling and Potential Uses for the HyperCLOVA Japanese Base Model

Toshinori Sato (LINE / NLP Development Team / Engineering Manager)
Takato Yamazaki (LINE / NLP Development Team / Software Engineer)

https://tech-verse.me/ja/sessions/52
https://tech-verse.me/en/sessions/52
https://tech-verse.me/ko/sessions/52

Tech-Verse2022
PRO

November 17, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. None
  2. Toshinori Sato (@overlast) - Natural Language Processing - Information Retrieval

    Principal Software Engineer / Manager - Japanese Corpus / Evaluation / Application - mecab-ipadic-NEologd HyperCLOVA OSS: Main Contributor of NEologd ࢁ࡚ ఱ Takato Yamazaki Takato Yamazaki (@y_takaten) - Dialog Systems - HyperCLOVA Applications Software Engineer - Aizawa Lab. / University of Tokyo - “Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation”, Findings of ACL 2021 - Young Researcher Award for Excellent Research at 11th Dialogue System Symposium MS Degree in Computer Science
  3. At #linedevday 2021, we presented JP 39B model

  4. 82B

  5. Agenda Realizing Applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World
  6. Agenda Realizing Applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World
  7. Meta-learning to treat target task information as text input and

    predict the next word from that input Pros - Highly economical in terms of time - High accuracy can be achieved without model tuning if the appropriate information and example (few-shot) can be described Cons - Requires a considerably larger number of parameters than when using a tunable mode - Requires a linguistic background and extensive social experience Two Major Ways of Controlling Output with Foundation Model Prompting Fine-tuning Meta-learning to treat target task information as text input and predict the next word from that input Pros - Highly economical in terms of money - High accuracy can be achieved if sufficient time is available for model training and parameter exploration Cons - Need to have the right amount of properly labeled teacher data for each task - Need to create a model for each task
  8. Q: But how do we use HyperCLOVA? A: Prompt Programming

  9. None
  10. https://huggingface.co/spaces/stabilityai/stable-diffusion

  11. ; 5FDI7FSTFJTBOPOMJOFUFDIOJDBMDPOGFSFODF XIJDIXJMMCFIFMEBTBKPJOUQSPKFDUCFUXFFO-*/&BOE :BIPP+"1"/PO/PWFNCFSUIBOEUI"EJWFSTF BSSBZPGNFNCFSTGSPNUIFUXPIPTUDPNQBOJFTBOE PUIFSQBSUJDJQBUJOHPSHBOJ[BUJPOTXJMMTIBSFUIFJS DVUUJOHFEHFDIBMMFOHFTBOEBDDVNVMBUFELOPXMFEHF 1SPNQUBTBRVFSZTUSJOHPG4UBCMF%JGGVTJPO “Prompting” is

    becoming very common w/ Stable Diffusion
  12. “Prompting” is becoming very common w/ Stable Diffusion Generate ;

    5FDI7FSTFJTBOPOMJOFUFDIOJDBMDPOGFSFODF XIJDIXJMMCFIFMEBTBKPJOUQSPKFDUCFUXFFO-*/&BOE :BIPP+"1"/PO/PWFNCFSUIBOEUI"EJWFSTF BSSBZPGNFNCFSTGSPNUIFUXPIPTUDPNQBOJFTBOE PUIFSQBSUJDJQBUJOHPSHBOJ[BUJPOTXJMMTIBSFUIFJS DVUUJOHFEHFDIBMMFOHFTBOEBDDVNVMBUFELOPXMFEHF 1SPNQUBTBRVFSZTUSJOHPG4UBCMF%JGGVTJPO
  13. ; 5FDI7FSTFJTBOPOMJOFUFDIOJDBMDPOGFSFODF XIJDIXJMMCFIFMEBTBKPJOUQSPKFDUCFUXFFO-*/&BOE :BIPP+"1"/PO/PWFNCFSUIBOEUI"EJWFSTF BSSBZPGNFNCFSTGSPNUIFUXPIPTUDPNQBOJFTBOE PUIFSQBSUJDJQBUJOHPSHBOJ[BUJPOTXJMMTIBSFUIFJS DVUUJOHFEHFDIBMMFOHFTBOEBDDVNVMBUFELOPXMFEHF 1SPNQUBTBRVFSZTUSJOHPG4UBCMF%JGGVTJPO Hmm, this

    does look like a diverse array of Japanese members… “Prompting” is becoming very common w/ Stable Diffusion Generate
  14. HyperCLOVA is also used with Prompt

  15. "BOE#BSFDIJUDIBUUJOH HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGBEJBMPH

  16. "BOE#BSFDIJUDIBUUJOH ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH #/JDF8IBU`TZPVSCFTUTDPSF 

    ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGBEJBMPH
  17. === "BOE#BSFDIJUDIBUUJOH ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH #/JDF8IBU`TZPVSCFTUTDPSF

     ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  18. === "BOE#BSFDIJUDIBUUJOH ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH #/JDF8IBU`TZPVSCFTUTDPSF

     ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " 1SPNQU HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  19. === "BOE#BSFDIJUDIBUUJOH 5BTL%FTDSJQUJPO ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH

    #/JDF8IBU`TZPVSCFTUTDPSF  ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF %JBMPH&YBNQMFº/ ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " $VSSFOU%JBMPH HyperCLOVA is also used with Prompt *OUIJTDBTF  JU`TDBMMFETIPU 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  20. === "BOE#BSFDIJUDIBUUJOH 5BTL%FTDSJQUJPO ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH

    #/JDF8IBU`TZPVSCFTUTDPSF  ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF %JBMPH&YBNQMFº/ ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " $VSSFOU%JBMPH HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  21. === "BOE#BSFDIJUDIBUUJOH 5BTL%FTDSJQUJPO ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF  ")NN *MJLFUPEPCPXMJOH

    #/JDF8IBU`TZPVSCFTUTDPSF  ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF %JBMPH&YBNQMFº/ ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " $VSSFOU%JBMPH HyperCLOVA is also used with Prompt Input the prompt and inference 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  22. 8IBULJOEPGNPWJFT EPZPVXBUDI === "BOE#BSFDIJUDIBUUJOH 5BTL%FTDSJQUJPO ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF 

    ")NN *MJLFUPEPCPXMJOH #/JDF8IBU`TZPVSCFTUTDPSF  ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF %JBMPH&YBNQMFº/ ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " $VSSFOU%JBMPH Generate Input the prompt and inference HyperCLOVA is also used with Prompt 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  23. 8IBULJOEPGNPWJFT EPZPVXBUDI === "BOE#BSFDIJUDIBUUJOH 5BTL%FTDSJQUJPO ")J IPXBSFZPV  #*`NHSFBU8IBUEPZPVEPJOZPVSGSFFUJNF 

    ")NN *MJLFUPEPCPXMJOH #/JDF8IBU`TZPVSCFTUTDPSF  ".ZCFTUTDPSFJT*VTVBMMZHFUBSPVOE #5IBU`TBXFTPNF*`MMOFWFSCFBCMFUPHFUUIBUTDPSF %JBMPH&YBNQMFº/ ")J XIBUEPZPVEPJOZPVSGSFFUJNF  #*MJLFUPXBUDINPWJFT " $VSSFOU%JBMPH 8IBULJOEPGNPWJFTEPZPVXBUDI (FOFSBUFEXJUI )ZQFS$-07" HyperCLOVA is also used with Prompt Input the prompt and inference Generate 1SPNQUUPHFOFSBUFBSFTQPOTFPGUIFEJBMPH
  24. That's wonderful! What do you think is the most important

    for your management work? OR Image Text
  25. Image <4IPLP XIPJTBMXBZTDIFFSGVM BTLTRVFTUJPOTBCPVUUIFDMJFOUT PDDVQBUJPO>  "DMJFOUJTBNBOJOIJTMBUFT 4IPLP RVFTUJPO /JDFUPNFFUZPV*BNBUPVSJTUHVJEF8IBUEP

    ZPVEPGPSZPVSMJWJOH  $VTUPNFS*BNBNBOBHFSJOBO*5DPNQBOZ 4IPLP RVFTUJPO  BIJHIMZEFUBJMFEFQJDDJOFNBUJDDPODFQUBSU$(SFOEFS EJHJUBMQBJOUJOHBSUXPSLEJFTFMQVOLTUFBNJOHIBMGNBOIBMG SPCPU#Z)PLVTBJ,BUTVTIJLB )JSPTIJHF6UBHBXB &JTFO ,FJTBJ ,VOJZPTIJ6UBHBXB ,VOJTBEB6UBHBXB 4IVOTIPV ,BUTVLBXB 4IJHFOPCV:BOBHBXB USFOEJOHPO"SU4UBUJPO  TVCUMFNVUFEDJOFNBUJDDPMPST NBEFJO.BZB #MFOEFSBOE 1IPUPTIPQ PDUBOFSFOEFS FYDFMMFOUDPNQPTJUJPO DJOFNBUJD BUNPTQIFSF EZOBNJDESBNBUJDDJOFNBUJDMJHIUJOH QSFDJTF DPSSFDUBOBUPNZ BFTUIFUJD WFSZJOTQJSBUJPOBM BSUIPVTF Output Format Text Output Example Prompt Example Number of people who can agree on an output itself Number of people who receive an output as intended Evaluation Viewpoint Prompt is written differently depending on output modality
  26. Image <4IPLP XIPJTBMXBZTDIFFSGVM BTLTRVFTUJPOTBCPVUUIFDMJFOUT PDDVQBUJPO>  "DMJFOUJTBNBOJOIJTMBUFT 4IPLP RVFTUJPO /JDFUPNFFUZPV*BNBUPVSJTUHVJEF8IBUEP

    ZPVEPGPSZPVSMJWJOH  $VTUPNFS*BNBNBOBHFSJOBO*5DPNQBOZ 4IPLP RVFTUJPO  BIJHIMZEFUBJMFEFQJDDJOFNBUJDDPODFQUBSU$(SFOEFS EJHJUBMQBJOUJOHBSUXPSLEJFTFMQVOLTUFBNJOHIBMGNBOIBMG SPCPU#Z)PLVTBJ,BUTVTIJLB )JSPTIJHF6UBHBXB &JTFO ,FJTBJ ,VOJZPTIJ6UBHBXB ,VOJTBEB6UBHBXB 4IVOTIPV ,BUTVLBXB 4IJHFOPCV:BOBHBXB USFOEJOHPO"SU4UBUJPO  TVCUMFNVUFEDJOFNBUJDDPMPST NBEFJO.BZB #MFOEFSBOE 1IPUPTIPQ PDUBOFSFOEFS FYDFMMFOUDPNQPTJUJPO DJOFNBUJD BUNPTQIFSF EZOBNJDESBNBUJDDJOFNBUJDMJHIUJOH QSFDJTF DPSSFDUBOBUPNZ BFTUIFUJD WFSZJOTQJSBUJPOBM BSUIPVTF Output Format Text Output Example Prompt Example Number of people who can agree on an output itself Number of people who receive an output as intended Evaluation Viewpoint Prompt is written differently depending on output modality
  27. Image <4IPLP XIPJTBMXBZTDIFFSGVM BTLTRVFTUJPOTBCPVUUIFDMJFOUT PDDVQBUJPO>  "DMJFOUJTBNBOJOIJTMBUFT 4IPLP RVFTUJPO /JDFUPNFFUZPV*BNBUPVSJTUHVJEF8IBUEP

    ZPVEPGPSZPVSMJWJOH  $VTUPNFS*BNBNBOBHFSJOBO*5DPNQBOZ 4IPLP RVFTUJPO  BIJHIMZEFUBJMFEFQJDDJOFNBUJDDPODFQUBSU$(SFOEFS EJHJUBMQBJOUJOHBSUXPSLEJFTFMQVOLTUFBNJOHIBMGNBOIBMG SPCPU#Z)PLVTBJ,BUTVTIJLB )JSPTIJHF6UBHBXB &JTFO ,FJTBJ ,VOJZPTIJ6UBHBXB ,VOJTBEB6UBHBXB 4IVOTIPV ,BUTVLBXB 4IJHFOPCV:BOBHBXB USFOEJOHPO"SU4UBUJPO  TVCUMFNVUFEDJOFNBUJDDPMPST NBEFJO.BZB #MFOEFSBOE 1IPUPTIPQ PDUBOFSFOEFS FYDFMMFOUDPNQPTJUJPO DJOFNBUJD BUNPTQIFSF EZOBNJDESBNBUJDDJOFNBUJDMJHIUJOH QSFDJTF DPSSFDUBOBUPNZ BFTUIFUJD WFSZJOTQJSBUJPOBM BSUIPVTF Output Format Text Output Example Prompt Example Number of people who can agree on an output itself Number of people who receive an output as intended Evaluation Viewpoint Prompt is written differently depending on output modality
  28. Prompt is written differently depending on output modality Image <4IPLP

    XIPJTBMXBZTDIFFSGVM BTLTRVFTUJPOTBCPVUUIFDMJFOUT PDDVQBUJPO>  "DMJFOUJTBNBOJOIJTMBUFT 4IPLP RVFTUJPO /JDFUPNFFUZPV*BNBUPVSJTUHVJEF8IBUEP ZPVEPGPSZPVSMJWJOH  $VTUPNFS*BNBNBOBHFSJOBO*5DPNQBOZ 4IPLP RVFTUJPO  BIJHIMZEFUBJMFEFQJDDJOFNBUJDDPODFQUBSU$(SFOEFS EJHJUBMQBJOUJOHBSUXPSLEJFTFMQVOLTUFBNJOHIBMGNBOIBMG SPCPU#Z)PLVTBJ,BUTVTIJLB )JSPTIJHF6UBHBXB &JTFO ,FJTBJ ,VOJZPTIJ6UBHBXB ,VOJTBEB6UBHBXB 4IVOTIPV ,BUTVLBXB 4IJHFOPCV:BOBHBXB USFOEJOHPO"SU4UBUJPO  TVCUMFNVUFEDJOFNBUJDDPMPST NBEFJO.BZB #MFOEFSBOE 1IPUPTIPQ PDUBOFSFOEFS FYDFMMFOUDPNQPTJUJPO DJOFNBUJD BUNPTQIFSF EZOBNJDESBNBUJDDJOFNBUJDMJHIUJOH QSFDJTF DPSSFDUBOBUPNZ BFTUIFUJD WFSZJOTQJSBUJPOBM BSUIPVTF Output Format Text Output Example Prompt Example Number of people who can agree on an output itself Number of people who receive an output as intended Evaluation Viewpoint That's wonderful! What do you think is the most important for your management work?
  29. What is better output in response for user's input? ===

    "BOE#BSFDIJUDIBUUJOH #)FZ XIBU`TVQ ")PX`TJUHPJOH "OZUIJOHOFXXJUIZPVMBUFMZ  #*IFBSEUIBUUIFOFXUFOUIPVTBOEZFOCJMMXJMMIBWF 4IJCVTBXB&JJDIJPOJU CVUEPZPVLOPXBCPVUIJN  " (FOFSBUFEXJUI)ZQFS$-07" 3FTQPOTFGSPNVTFS • The user's response contains a named entity “Shibusawa Eiichi” • And we can detect that “Shibusawa Eiichi” includes in the entries of Wikipedia • For simplicity, we would like to make use of the WIkipedia text after extracting “Shibusawa Eiichi” from the user's response ? "*WFCFFOSFBEJOHBMPUMBUFMZ #5IBUTHSFBU8IBUBSFZPVSFBEJOH  "*WFCFFOSFBEJOHCPPLTCZ4PTFLJ/BUTVNF #)FZ *LOPXIJN5IFQFSTPOJOUIF ZFOCJMM SJHIU
  30. A: I've been reading a lot lately. B: That's great!

    What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: ɹɹɹɹ ? Can the training data be accurately retrieved from a model?
  31. A: I've been reading a lot lately. B: That's great!

    What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: I don’t really know him, who’s that? Can the training data be accurately retrieved from a model? We should assume that an accurate knowledge in the foundation model cannot be extracted. No !! Foundation Model is not a search engine !!
  32. HyperCLOVA Prompt chaining can generate complex output Knowledge: Soseki Natsume

    was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. Dialog: A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === Dialog: B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: "VUPHFOFSBUFE,OPXMFEHF A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: )BOENBEF,OPXMFEHF I don’t really know him, who’s that? (Something better with knowledge…) Copy Copy
  33. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN /BNFE&OUJUZ3FDPHOJUJPO
  34. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN 4IJCVTBXB&JJDIJ /BNFE&OUJUZ3FDPHOJUJPO
  35. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ %#4FBSDI FH8JLJQFEJB
  36. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ %#4FBSDI FH8JLJQFEJB Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double-entry accounting, joint- stock corporations and modern note-issuing banks. He founded the first modern bank based on joint stock ownership in Japan. The bank was aptly named The First National Bank (Dai Ichi Kokuritsu Ginkō, now merged into Mizuho Bank) and had the power to issue its own notes. Through this bank, he founded hundreds of other joint stock corporations in Japan. Many of these companies still survive … Search Result
  37. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN Original: Natsume Sōseki (9 February 1867 – 9 December 1916), born Natsume Kin'nosuke, was a Japanese novelist. He is best known around the world for his novels Kokoro, Botchan, I Am a Cat, Kusamakura and his unfinished work Light and Darkness. He was also a scholar of British literature and writer of haiku, kanshi, and fairy tales. From 1984 until 2004, his portrait appeared on the front of the Japanese 1,000 yen note. Summary: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. === Original: Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double- entry accounting, joint-stock corporations and modern note-issuing banks… Summary: 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ 4VNNBSJ[BUJPO /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ %#4FBSDI FH8JLJQFEJB
  38. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN Original: Natsume Sōseki (9 February 1867 – 9 December 1916), born Natsume Kin'nosuke, was a Japanese novelist. He is best known around the world for his novels Kokoro, Botchan, I Am a Cat, Kusamakura and his unfinished work Light and Darkness. He was also a scholar of British literature and writer of haiku, kanshi, and fairy tales. From 1984 until 2004, his portrait appeared on the front of the Japanese 1,000 yen note. Summary: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. === Original: Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double- entry accounting, joint-stock corporations and modern note-issuing banks… Summary: 4VNNBSJ[BUJPO /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ Eiichi Shibusawa was born in 1840 in Kuroaraijima, Fukaya City. During his lifetime, he was involved in the establishment of approximately 500 companies and is known as the "father of Japanese capitalism”. Summary Result 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ %#4FBSDI FH8JLJQFEJB
  39. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN Original: Natsume Sōseki (9 February 1867 – 9 December 1916), born Natsume Kin'nosuke, was a Japanese novelist. He is best known around the world for his novels Kokoro, Botchan, I Am a Cat, Kusamakura and his unfinished work Light and Darkness. He was also a scholar of British literature and writer of haiku, kanshi, and fairy tales. From 1984 until 2004, his portrait appeared on the front of the Japanese 1,000 yen note. Summary: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. === Original: Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double- entry accounting, joint-stock corporations and modern note-issuing banks… Summary: Knowledge: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. Dialog: A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === Knowledge: Eiichi Shibusawa was born in 1840 in Kuroaraijima, Fukaya City. During his lifetime, he was involved in the establishment of approximately 500 companies and is known as the "father of Japanese capitalism”. Dialog: B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: 4VNNBSZ3FTVMU GSPN)ZQFS$-07" 4VNNBSJ[BUJPO 3FTQPOTF(FOFSBUJPO /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ %#4FBSDI FH8JLJQFEJB
  40. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN Original: Natsume Sōseki (9 February 1867 – 9 December 1916), born Natsume Kin'nosuke, was a Japanese novelist. He is best known around the world for his novels Kokoro, Botchan, I Am a Cat, Kusamakura and his unfinished work Light and Darkness. He was also a scholar of British literature and writer of haiku, kanshi, and fairy tales. From 1984 until 2004, his portrait appeared on the front of the Japanese 1,000 yen note. Summary: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. === Original: Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double- entry accounting, joint-stock corporations and modern note-issuing banks… Summary: Knowledge: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. Dialog: A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === Knowledge: Eiichi Shibusawa was born in 1840 in Kuroaraijima, Fukaya City. During his lifetime, he was involved in the establishment of approximately 500 companies and is known as the "father of Japanese capitalism”. Dialog: B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: 4VNNBSZ3FTVMU GSPN)ZQFS$-07" 4VNNBSJ[BUJPO 3FTQPOTF(FOFSBUJPO /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ )BOENBEF4VNNBSZ %#4FBSDI FH8JLJQFEJB
  41. HyperCLOVA Prompt chaining can generate complex output *IFBSEUIBUUIFOFXUFOUIPVTBOE ZFOCJMMXJMMIBWF4IJCVTBXB&JJDIJPO JU

    CVUEPZPVLOPXBCPVUIJN Original: Natsume Sōseki (9 February 1867 – 9 December 1916), born Natsume Kin'nosuke, was a Japanese novelist. He is best known around the world for his novels Kokoro, Botchan, I Am a Cat, Kusamakura and his unfinished work Light and Darkness. He was also a scholar of British literature and writer of haiku, kanshi, and fairy tales. From 1984 until 2004, his portrait appeared on the front of the Japanese 1,000 yen note. Summary: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. === Original: Shibusawa Eiichi, 1st Viscount Shibusawa (March 16, 1840 – November 11, 1931) was a Japanese industrialist use of double- entry accounting, joint-stock corporations and modern note-issuing banks… Summary: Knowledge: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. Dialog: A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === Knowledge: Eiichi Shibusawa was born in 1840 in Kuroaraijima, Fukaya City. During his lifetime, he was involved in the establishment of approximately 500 companies and is known as the "father of Japanese capitalism”. Dialog: B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: 4VNNBSJ[BUJPO 3FTQPOTF(FOFSBUJPO /BNFE&OUJUZ3FDPHOJUJPO 4IJCVTBXB&JJDIJ 4VNNBSZ3FTVMU GSPN)ZQFS$-07" 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ )BOENBEF4VNNBSZ %#4FBSDI FH8JLJQFEJB
  42. HyperCLOVA Prompt chaining can generate complex output Natsume Sōseki (9

    February 1867 – 9 r 1916), born Natsume Kin'nosuke, was se novelist. He is best known around the his novels Kokoro, Botchan, I Am a makura and his unfinished work Light ness. He was also a scholar of British and writer of haiku, kanshi, and fairy m 1984 until 2004, his portrait appeared nt of the Japanese 1,000 yen note. y: Soseki Natsume was a Japanese His real name is Natsume Kinnosuke. it was used on the 1,000 yen bill. Shibusawa Eiichi, 1st Viscount a (March 16, 1840 – November 11, s a Japanese industrialist use of double- ounting, joint-stock corporations and ote-issuing banks… y: Knowledge: Soseki Natsume was a Japanese novelist. His real name is Natsume Kinnosuke. His portrait was used on the 1,000 yen bill. Dialog: A: I've been reading a lot lately. B: That's great! What are you reading? A: I've been reading books by Soseki Natsume. B: Hey, I know him! The person in the 1,000 yen bill, right? === Knowledge: Eiichi Shibusawa was born in 1840 in Kuroaraijima, Fukaya City. During his lifetime, he was involved in the establishment of approximately 500 companies and is known as the "father of Japanese capitalism”. Dialog: B: Hey, what’s up! A: How’s it going? Anything new with you lately? B: I heard that the new ten-thousand yen bill will have Shibusawa Eiichi on it, but do you know about him? A: 4VNNBSJ[BUJPO 3FTQPOTF(FOFSBUJPO Yeah, I know him! He’s the one who laid the foundation for Japanese capitalism. He definitely deserves to be on the bill! 4VNNBSZ3FTVMU GSPN)ZQFS$-07" 4FBSDI3FTVMUPO 4IJCVTBXB&JJDIJ )BOENBEF4VNNBSZ
  43. Agenda Realizing Applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World
  44. 82B For 82B JP Model, we used… • Samples: 10B

    documents • Data size: 1.8TB • Token Size: 530B We do NOT use any data of our LINE Messenger Service • All messages on LINE • All posts on OpenChat Policy for Data Collection LINE LM Corpus Tokenizer Byte-level BPE tokenizer Library Megatron-LM Infrastructure NVIDIA Superpod 128 clustered DGX servers 1,024 A100 GPUs = Architecture Transformer Encoder-Decoder
  45. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

    HyperCLOVA w/ prompting VS BERT-large TASK: RCQA* (answerable ones only) • Removed unanswerable questions from dataset of the normal RCQA task
  46. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

    HyperCLOVA w/ prompting VS BERT-large - Questionɿதࠃ།Ұͷঁఇɺଇఱ෢޳͕ݐͯͨԦேͷ໊લ͸ԿͰ͠ΐ͏? - Contextɿ౰࣌ɺ౜Ԧே͸෢ଇఱʢଇఱ෢޳ʣʹΑΔცୣͰपԦேʹ୅Θ ͍ͬͯͨ͜ͱΛ೔ຊଆ͕೺Ѳ͍ͯ͠ͳ͔ͬͨͨΊɺ҄ాਅਓΒ͸ݱ஍Ͱ एׯͷࠞཚΛੜͨ͡ɻ·ͨɺ൴Β͕౎ɾ௕҆Ͱݟ࣮ͨࡍͷ౎৓΍཯ྩ੍ ͷӡ༻࣮ଶ͸ɺ೔ຊࠃ಺Ͱͷ૝૾ͱ͸ࣅͯඇͳΔ΋ͷͰ͋ͬͨɻͨͱ͑ ͹౻ݪژͰ͸େۃ఼ΛؚΉٶʢ౻ݪٶʣΛ౎৓ͷதԝʹ഑ஔ͍ͯͨ͠ ͕ɺ௕҆৓Λ͸͡Ίͱ͢Δதࠃͷ౎৓Ͱ͸ଠۃٶΛؚΉߖ৓͸ɺ౎৓ͷ ๺୺தԝʹ͋Δͷ͕௨ྫͰ͋ͬͨɻ཯ྩͷӡ༻ܗଶ΋೔ຊͱ͸ҟͳΓɺ ཯ྩͷෆඋΛߦ͏֨ࣜͳͲ΋੍ఆ͞Ε͍ͯͨɻେ͖ͳিܸΛड͚ͯؼࠃ ͨ҄͠ాਅਓΒ͸ɺ͜ΕΒͷ೔தͷ౎৓΍཯ྩ੍ͷࠩҟΛใࠂ͠ɺͷͪ ͷվֵʹੜ͔͞Ε͍ͯ͘ɻ Example of an entry of RCQA possible only tasks TASK: RCQA* (answerable ones only) • Removed unanswerable questions from dataset of the normal RCQA task
  47. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

    HyperCLOVA w/ prompting VS BERT-large - Questionɿதࠃ།Ұͷঁఇɺଇఱ෢޳͕ݐͯͨԦேͷ໊લ͸ԿͰ͠ΐ͏? - Contextɿ౰࣌ɺ౜Ԧே͸෢ଇఱʢଇఱ෢޳ʣʹΑΔცୣͰपԦேʹ୅Θ ͍ͬͯͨ͜ͱΛ೔ຊଆ͕೺Ѳ͍ͯ͠ͳ͔ͬͨͨΊɺ҄ాਅਓΒ͸ݱ஍Ͱ एׯͷࠞཚΛੜͨ͡ɻ·ͨɺ൴Β͕౎ɾ௕҆Ͱݟ࣮ͨࡍͷ౎৓΍཯ྩ੍ ͷӡ༻࣮ଶ͸ɺ೔ຊࠃ಺Ͱͷ૝૾ͱ͸ࣅͯඇͳΔ΋ͷͰ͋ͬͨɻͨͱ͑ ͹౻ݪژͰ͸େۃ఼ΛؚΉٶʢ౻ݪٶʣΛ౎৓ͷதԝʹ഑ஔ͍ͯͨ͠ ͕ɺ௕҆৓Λ͸͡Ίͱ͢Δதࠃͷ౎৓Ͱ͸ଠۃٶΛؚΉߖ৓͸ɺ౎৓ͷ ๺୺தԝʹ͋Δͷ͕௨ྫͰ͋ͬͨɻ཯ྩͷӡ༻ܗଶ΋೔ຊͱ͸ҟͳΓɺ ཯ྩͷෆඋΛߦ͏֨ࣜͳͲ΋੍ఆ͞Ε͍ͯͨɻେ͖ͳিܸΛड͚ͯؼࠃ ͨ҄͠ాਅਓΒ͸ɺ͜ΕΒͷ೔தͷ౎৓΍཯ྩ੍ͷࠩҟΛใࠂ͠ɺͷͪ ͷվֵʹੜ͔͞Ε͍ͯ͘ɻ - Answerɿप Example of an entry of RCQA possible only tasks TASK: RCQA* (answerable ones only) • Removed unanswerable questions from dataset of the normal RCQA task
  48. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

    HyperCLOVA w/ prompting VS BERT-large - Questionɿதࠃ།Ұͷঁఇɺଇఱ෢޳͕ݐͯͨԦேͷ໊લ͸ԿͰ͠ΐ͏? - Contextɿ౰࣌ɺ౜Ԧே͸෢ଇఱʢଇఱ෢޳ʣʹΑΔცୣͰपԦேʹ୅Θ ͍ͬͯͨ͜ͱΛ೔ຊଆ͕೺Ѳ͍ͯ͠ͳ͔ͬͨͨΊɺ҄ాਅਓΒ͸ݱ஍Ͱ एׯͷࠞཚΛੜͨ͡ɻ·ͨɺ൴Β͕౎ɾ௕҆Ͱݟ࣮ͨࡍͷ౎৓΍཯ྩ੍ ͷӡ༻࣮ଶ͸ɺ೔ຊࠃ಺Ͱͷ૝૾ͱ͸ࣅͯඇͳΔ΋ͷͰ͋ͬͨɻͨͱ͑ ͹౻ݪژͰ͸େۃ఼ΛؚΉٶʢ౻ݪٶʣΛ౎৓ͷதԝʹ഑ஔ͍ͯͨ͠ ͕ɺ௕҆৓Λ͸͡Ίͱ͢Δதࠃͷ౎৓Ͱ͸ଠۃٶΛؚΉߖ৓͸ɺ౎৓ͷ ๺୺தԝʹ͋Δͷ͕௨ྫͰ͋ͬͨɻ཯ྩͷӡ༻ܗଶ΋೔ຊͱ͸ҟͳΓɺ ཯ྩͷෆඋΛߦ͏֨ࣜͳͲ΋੍ఆ͞Ε͍ͯͨɻେ͖ͳিܸΛड͚ͯؼࠃ ͨ҄͠ాਅਓΒ͸ɺ͜ΕΒͷ೔தͷ౎৓΍཯ྩ੍ͷࠩҟΛใࠂ͠ɺͷͪ ͷվֵʹੜ͔͞Ε͍ͯ͘ɻ - Answerɿप Example of an entry of RCQA possible only tasks Create Few-shot with a context, a question text and an answer • If the correct answer is contained and easily extracted from the inference result, we judged it is correct TASK: RCQA* (answerable ones only) • Removed unanswerable questions from dataset of the normal RCQA task
  49. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

    HyperCLOVA w/ prompting VS BERT-large - Questionɿதࠃ།Ұͷঁఇɺଇఱ෢޳͕ݐͯͨԦேͷ໊લ͸ԿͰ͠ΐ͏? - Contextɿ౰࣌ɺ౜Ԧே͸෢ଇఱʢଇఱ෢޳ʣʹΑΔცୣͰपԦேʹ୅Θ ͍ͬͯͨ͜ͱΛ೔ຊଆ͕೺Ѳ͍ͯ͠ͳ͔ͬͨͨΊɺ҄ాਅਓΒ͸ݱ஍Ͱ एׯͷࠞཚΛੜͨ͡ɻ·ͨɺ൴Β͕౎ɾ௕҆Ͱݟ࣮ͨࡍͷ౎৓΍཯ྩ੍ ͷӡ༻࣮ଶ͸ɺ೔ຊࠃ಺Ͱͷ૝૾ͱ͸ࣅͯඇͳΔ΋ͷͰ͋ͬͨɻͨͱ͑ ͹౻ݪژͰ͸େۃ఼ΛؚΉٶʢ౻ݪٶʣΛ౎৓ͷதԝʹ഑ஔ͍ͯͨ͠ ͕ɺ௕҆৓Λ͸͡Ίͱ͢Δதࠃͷ౎৓Ͱ͸ଠۃٶΛؚΉߖ৓͸ɺ౎৓ͷ ๺୺தԝʹ͋Δͷ͕௨ྫͰ͋ͬͨɻ཯ྩͷӡ༻ܗଶ΋೔ຊͱ͸ҟͳΓɺ ཯ྩͷෆඋΛߦ͏֨ࣜͳͲ΋੍ఆ͞Ε͍ͯͨɻେ͖ͳিܸΛड͚ͯؼࠃ ͨ҄͠ాਅਓΒ͸ɺ͜ΕΒͷ೔தͷ౎৓΍཯ྩ੍ͷࠩҟΛใࠂ͠ɺͷͪ ͷվֵʹੜ͔͞Ε͍ͯ͘ɻ - Answerɿप Example of an entry of RCQA possible only tasks Few-shots were created randomly by extracting a context from the RCQA dev-set for each inference Create Few-shot with a context, a question text and an answer • If the correct answer is contained and easily extracted from the inference result, we judged it is correct TASK: RCQA* (answerable ones only) • Removed unanswerable questions from dataset of the normal RCQA task
  50. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

         # # # #  )ZQFS$-07"+1 #&35MBSHF .  HyperCLOVA w/ prompting VS BERT-large Acc. Parameters of HyperCLOVA Pre-training BERT-large by using subset of LINE LM corpus HyperCLOVA 6.9B w/ prompting is far below Fine-tuned BERT
  51. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

         # # # #   )ZQFS$-07"+1 #&35MBSHF .  HyperCLOVA w/ prompting VS BERT-large Acc. Parameters of HyperCLOVA HyperCLOVA 13B JP cannot exceed Fine-tuned BERT-large by only prompting
  52. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

         # # # #    )ZQFS$-07"+1 #&35MBSHF .  HyperCLOVA w/ prompting VS BERT-large Acc. Parameters of HyperCLOVA HyperCLOVA 39B JP is near BERT-large with fine tuning and parameter tuning by prompting
  53. Automated evaluation on RCQA* possible only tasks * ղ౴Մೳੑ෇͖ಡղσʔληοτ: http://www.cl.ecei.tohoku.ac.jp/rcqa/

         # # # #     )ZQFS$-07"+1 #&35MBSHF .  HyperCLOVA w/ prompting VS BERT-large Acc. Parameters of HyperCLOVA HyperCLOVA JP 82B with Prompting over BERT-large with full Fine-tuning and Parameter-tuning
  54. Prompt Chaining is Effective for Most of the Applications Writing

    Business Emails Writing Business Documents Writing Novels and more…
  55. Agenda Realizing Applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World
  56. Dialog System

  57. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU
  58. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU Hey, do you like Brown? USER
  59. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU Sorry, I didn’t understand the question. BOT Hey, do you like Brown? USER
  60. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU Sorry, I didn’t understand the question. BOT Hey, do you like Brown? USER Guess what, I just got a LINE GIFT from my friend! USER
  61. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU Sorry, I didn’t understand the question. BOT Hey, do you like Brown? USER Guess what, I just got a LINE GIFT from my friend! USER According to Wikipedia, LINE is a freeware app for instant communications... BOT
  62. • Many deployed models are rule-based • Looks for keywords

    to determine next response • Pros: Fully controllable • Cons: Lacks flexibility Rule-Based Dialog System )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF )PX`TJUHPJOH 5FDI7FSTFJTBNB[JOH *`NHSFBU Sampl Sampl Sampl Sampl )J IPX`TJUHPJOH"SFZPVSFBEZGPS5FDI7FSTF *`NHSFBU :FT *`NTPSFBEZGPSUIFBXFTPNFQSFTFOUBUJPOT • Trending in dialog research field • Uses deep learning technique to generate response • Pros: very flexible and responses are interesting • Cons: Lacks controllability Generation-Based Dialog System
  63. None
  64. Is a single prompt enough to build an amazing dialog

    system? %JBMPH 1SPNQU 6TFS 6UUFSBODF 4ZTUFN 3FTQPOTF
  65. Probably No… A single few-shot prompt was just not enough

  66. Probably No… &OEDPOWFSTBUJPO WFSZFBSMZ A single few-shot prompt was just

    not enough
  67. Probably No… &OEDPOWFSTBUJPO WFSZFBSMZ #PSJOH SFTQPOTFT A single few-shot prompt

    was just not enough
  68. 4ZTUFN 0VUQVU 6TFS *OQVU HyperCLOVA-Based Dialog System Architecture

  69. 4ZTUFN 0VUQVU 6TFS *OQVU HyperCLOVA-Based Dialog System Architecture 1SFQSPDFTT 8IBUDBOXFEPIFSF

    1SPNQU1SPHSBNNJOHBOE 1BSBNFUFS$POUSPMMJOH 4VCUBTLTPMWJOHXJUI )ZQFS$-07"PS/-15FDI *OTUSVDUUIF EJSFDUJPOPGPVUQVU w 4FBSDIJOHLOPXMFEHFT w %FDJEJOHEJBMPHBDU w FUD
  70. 4ZTUFN 0VUQVU 6TFS *OQVU HyperCLOVA-Based Dialog System Architecture 1SFQSPDFTT 8IBUDBOXFEPIFSF

    1SPNQU1SPHSBNNJOHBOE 1BSBNFUFS$POUSPMMJOH 4VCUBTLTPMWJOHXJUI )ZQFS$-07"PS/-15FDI *OTUSVDUUIF EJSFDUJPOPGPVUQVU w 4FBSDIJOHLOPXMFEHFT w %FDJEJOHEJBMPHBDU w FUD (FOFSBUF $PSF0VUQVU
  71. 4ZTUFN 0VUQVU 3FHFOFSBUF 6TFS *OQVU HyperCLOVA-Based Dialog System Architecture 1SFQSPDFTT

    8IBUDBOXFEPIFSF 1SPNQU1SPHSBNNJOHBOE 1BSBNFUFS$POUSPMMJOH 4VCUBTLTPMWJOHXJUI )ZQFS$-07"PS/-15FDI *OTUSVDUUIF EJSFDUJPOPGPVUQVU w 4FBSDIJOHLOPXMFEHFT w %FDJEJOHEJBMPHBDU w FUD 1PTUQSPDFTT 8IBUDBOXFEPIFSF 'JMUFSJOHX/-15FDI "EEJOHBOE&EJUJOHX )ZQFS$-07"PS/-15FDI w $IFDLJOH)BMMVDJOBUJPO w $IFDLJOH&UIJDT w FUD %FUFDUPSDPSSFDU JNQSPQFSPVUQVUT (FOFSBUF $PSF0VUQVU
  72. HyperCLOVA-Based Dialog System Architecture %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU %PFTJUIBWF

    VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE 4ZTUFN 0VUQVU 6TFS *OQVU
  73. HyperCLOVA-Based Dialog System Architecture %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU %PFTJUIBWF

    VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE &YUFSOBM,OPXMFEHF 4ZTUFN 0VUQVU 6TFS *OQVU
  74. HyperCLOVA-Based Dialog System Architecture %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU %PFTJUIBWF

    VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE &YUFSOBM,OPXMFEHF 4ZTUFN 0VUQVU 6TFS *OQVU *NQSPWJOH$POTJTUFODZ w )ZQFS$-07"#BTFE1FSTPOB&YUSBDUJPO4FMFDUJPO6UJMJ[BUJPO ,BXBNPUP
  75. HyperCLOVA-Based Dialog System Architecture %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU %PFTJUIBWF

    VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE &YUFSOBM,OPXMFEHF 4ZTUFN 0VUQVU 6TFS *OQVU *ODSFBTJOH"NPVOUPG3FTQPOTFT *NQSPWJOH$POTJTUFODZ w )ZQFS$-07"#BTFE1FSTPOB&YUSBDUJPO4FMFDUJPO6UJMJ[BUJPO ,BXBNPUP
  76. HyperCLOVA-Based Dialog System Architecture %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU %PFTJUIBWF

    VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE &YUFSOBM,OPXMFEHF "WPJE&OEJOH $POWFSTBUJPO 4ZTUFN 0VUQVU 6TFS *OQVU *ODSFBTJOH"NPVOUPG3FTQPOTFT *NQSPWJOH$POTJTUFODZ w )ZQFS$-07"#BTFE1FSTPOB&YUSBDUJPO4FMFDUJPO6UJMJ[BUJPO ,BXBNPUP
  77. With Preprocess and Postprocess Natural and engaging responses Only Few-Shot

    Prompt Bland and unexciting responses Comparing w/ and wo/ External Modules BOT USER BOT USER
  78. With Preprocess and Postprocess Natural and engaging responses Only Few-Shot

    Prompt Bland and unexciting responses Comparing w/ and wo/ External Modules BOT USER BOT USER
  79. Our system's achievements in dialog competitions Dialog System Live Competition

    4 (2021) Dialogue system researchers gather and do live-evaluations of submitted chatbots Dialog Robot Competition 2022 A competition of controlling humanoid robot that can communicate and recommend a tourist sight inviting boss to a party Situation Track Situated Chat TU1MBDF X)ZQFS$-07"# Open Track Open-Domain Chitchat w/ random topics TU1MBDF X)ZQFS$-07"# recommending a spot Main Competition Tourist- Guide Robot TU1MBDF X)ZQFS$-07"#
  80. Is that technology only for Dialog Systems?

  81. HyperCLOVA Writer

  82. None
  83. None
  84. Email Generation w/ HyperCLOVA Writer Email written by HyperCLOVA Writer

    Rough Plot by Human
  85. Email Generation w/ HyperCLOVA Writer Email written by HyperCLOVA Writer

    Rough Plot by Human Postprocess: NER Validity Check
  86. HyperCLOVA Writer is Extensible NLP APIs

  87. /api/email /api/minute HyperCLOVA Writer is Extensible NLP APIs /api/novel …

    How can HyperCLOVA help you? Please tell us at Twitter w/ #HyperCLOVA ?????
  88. What’s the goal of HyperCLOVA Writer? Human Writing Plan /

    Search / Write
  89. What’s the goal of HyperCLOVA Writer? Human Writing Plan /

    Search / Write Writing w/ HyperCLOVA Plan Generate Fix Tea Time
  90. What’s Next for HyperCLOVA Writer

  91. Agenda Realizing Applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World
  92. 82B Are foundation models perfectly ready for deployment?

  93. Definitely NOT Yet 5FDI7FSTFJTBOPOMJOFUFDIOJDBMDPOGFSFODF XIJDIXJMMCFIFMEBTBKPJOUQSPKFDUCFUXFFO-*/&BOE :BIPP+"1"/PO/PWFNCFSUIBOEUI"EJWFSTF BSSBZPGNFNCFSTGSPNUIFUXPIPTUDPNQBOJFTBOE PUIFSQBSUJDJQBUJOHPSHBOJ[BUJPOTXJMMTIBSFUIFJS DVUUJOHFEHFDIBMMFOHFTBOEBDDVNVMBUFELOPXMFEHF Fairness

    and Ethics of Generated Text Accuracy and Reliability of Generated Text Dealing with the Cost per Inference by a Model There are still many things to think about…
  94. Our First Step Towards Ethical AI 5FDI7FSTFJTBOPOMJOFUFDIOJDBMDPOGFSFODF XIJDIXJMMCFIFMEBTBKPJOUQSPKFDUCFUXFFO-*/&BOE :BIPP+"1"/PO/PWFNCFSUIBOEUI"EJWFSTF BSSBZPGNFNCFSTGSPNUIFUXPIPTUDPNQBOJFTBOE

    PUIFSQBSUJDJQBUJOHPSHBOJ[BUJPOTXJMMTIBSFUIFJS DVUUJOHFEHFDIBMMFOHFTBOEBDDVNVMBUFELOPXMFEHF
  95. Ethics Check in Dialog System %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU

    %PFTJUIBWF VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 6TFS 6UUFSBODF 4ZTUFN 3FTQPOTF 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE
  96. Ethics Check in Dialog System %PFTUIF LOPXMFEHFCBTF IBWFVTFGVMEBUB ,OPXMFEHF#BTFE 3FTQPOTF1SPNQU

    %PFTJUIBWF VTFGVMQFSTPOBEBUB 1FSTPOB$POTJTUFOU 3FTQPOTF1SPNQU (FOFSBM 3FTQPOTF1SPNQU #PSJOH3FTQPOTF %FUFDUJPO $POWFSTBUJPO&OEJOH 3FTQPOTF%FUFDUJPO .VMUJ5VSO 3FTQPOTF1SPNQU 1SPIJCJUFE8PSE %FUFDUJPO 5P3FHFOFSBUJPO 3FHFOFSBUJPO :FT :FT /P /P 1BTTFE %FUFDUFE 6TFS 6UUFSBODF 4ZTUFN 3FTQPOTF 1BTTFE 5P3FHFOFSBUJPO %FUFDUFE %FUFDUFE "WPJE0⒎FOTJWF-BOHVBHFT
  97. Quick Fact Checking to Avoid Hallucination LINE and Yahoo! JAPAN

    will hold the "Tech- Verse" conference online (live streaming format) on Thursday, November 17 and Friday, November 18, 2022. Up until now, LINE and Yahoo Japan have held annual technology conferences as "LINE DEVELOPER DAY" and "Yahoo! JAPAN Tech Conference" respectively, but this year, for the first time, LINE and Yahoo Japan will hold a joint event, "Tech-Verse JAPAN Tech Conference, but this year, for the first time, LINE and Yahoo! ... Original Sentence Summarizer The overview of the sessions and speakers are available on the official website. LINE and Yahoo will jointly hold a technology conference. The theme of the conference will be "Pioneering the Future Society with Technology.” The official website, which went live today, provides an overview of the 80 sessions and speakers, with more details on the panel discussions to follow. …
  98. Quick Fact Checking to Avoid Hallucination LINE and Yahoo! JAPAN

    will hold the "Tech- Verse" conference online (live streaming format) on Thursday, November 17 and Friday, November 18, 2022. Up until now, LINE and Yahoo Japan have held annual technology conferences as "LINE DEVELOPER DAY" and "Yahoo! JAPAN Tech Conference" respectively, but this year, for the first time, LINE and Yahoo Japan will hold a joint event, "Tech-Verse JAPAN Tech Conference, but this year, for the first time, LINE and Yahoo! ... Original Sentence Summarizer The overview of the sessions and speakers are available on the official website. LINE and Yahoo will jointly hold a technology conference. The theme of the conference will be "Pioneering the Future Society with Technology.” The official website, which went live today, provides an overview of the 80 sessions and speakers, with more details on the panel discussions to follow. …
  99. Quick Fact Checking to Avoid Hallucination LINE and Yahoo! JAPAN

    will hold the "Tech- Verse" conference online (live streaming format) on Thursday, November 17 and Friday, November 18, 2022. Up until now, LINE and Yahoo Japan have held annual technology conferences as "LINE DEVELOPER DAY" and "Yahoo! JAPAN Tech Conference" respectively, but this year, for the first time, LINE and Yahoo Japan will hold a joint event, "Tech-Verse JAPAN Tech Conference, but this year, for the first time, LINE and Yahoo! ... Original Sentence Summarizer The overview of the sessions and speakers are available on the official website. LINE and Yahoo will jointly hold a technology conference. The theme of the conference will be "Pioneering the Future Society with Technology.” The official website, which went live today, provides an overview of the 80 sessions and speakers, with more details on the panel discussions to follow. …
  100. Quick Fact Checking to Avoid Hallucination LINE and Yahoo! JAPAN

    will hold the "Tech- Verse" conference online (live streaming format) on Thursday, November 17 and Friday, November 18, 2022. Up until now, LINE and Yahoo Japan have held annual technology conferences as "LINE DEVELOPER DAY" and "Yahoo! JAPAN Tech Conference" respectively, but this year, for the first time, LINE and Yahoo Japan will hold a joint event, "Tech-Verse JAPAN Tech Conference, but this year, for the first time, LINE and Yahoo! ... Original Sentence Summarizer The overview of the sessions and speakers are available on the official website. LINE and Yahoo will jointly hold a technology conference. The theme of the conference will be "Pioneering the Future Society with Technology.” The official website, which went live today, provides an overview of the 80 sessions and speakers, with more details on the panel discussions to follow. … ?
  101. Quick Fact Checking to Avoid Hallucination $POWFSUTFOUFODFTUPHSBQIT 'JOEHSBQINJTNBUDI Hallucinated

  102. Human Generating Writing w/ Foundation Model Planning & Planning Writing

    / Searching Fixing & Fixing Generating ? Can we actually reduce labor cost by using Foundation Model?
  103. Can we actually reduce labor cost by using Foundation Model?

    A: Depends!
  104. Can we actually reduce labor cost by using Foundation Model?

  105. ! Good Case Plan / Write Plan Fix Gen. Can

    we actually reduce labor cost by using Foundation Model?
  106. ! Good Case Plan / Write Plan Fix Gen. "

    Bad Case 1 Plan / Write Plan Fix Gen. Can we actually reduce labor cost by using Foundation Model?
  107. ! Good Case Plan / Write Plan Fix Gen. "

    Bad Case 1 Plan / Write Plan Fix Gen. " Bad Case 2 Plan / Write Plan Fix Gen. × 3 Can we actually reduce labor cost by using Foundation Model?
  108. Utilization of parameter-saving foundation model * JGLUE: Japanese General Language

    Understanding Evaluation - https://github.com/yahoojapan/JGLUE HyperCLOVA w/ LoRA tuning VS Waseda RoBERTa large {"q_id": 3016, "question": "ձࣾͷ࠷ߴ੹೚ऀΛԿͱ͍͏͔ʁ (What do you call the chief executive officer of a company?)", "choice0": "ࣾ௕ (president)", "choice1": "ڭࢣ (teacher)", "choice2": "෦௕ (manager)", "choice3": "όΠτ (part-time worker)", "choice4": "෦Լ (subordinate)", "label": 0} Example of an entry of JCommonsenseQA tasks Verification of LoRA performance of HyperCLOVA Japanese 6.7B model • Unlike Prompting, a solution is possible in 0-shot due to tuning by supervised data TASK: JCommonsenseQA • Japanese version of CommonsenseQA dataset • Five-choice QA questions to assess common sense reasoning ability This experiment was the result of a summer internship 2022 at LINE Corp. • More details will be provided at NLP2023
  109. Experimental results in JCommonsenseQA HyperCLOVA w/ LoRA tuning VS Waseda

    RoBERTa large Human Tohoku BERT Large Waseda RoBERTa Large (s128) HyperCLOVA 6.7B w/ LoRA Accuracy 0.8 0.85 0.9 0.95 1 93.6% 90.7% 81.6% 98.8% +3.0% +14.7% -5.5% HyperCLOVA JP 6.7B with LoRA tuning exceeds JGLUE baseline accuracy
  110. Challenges in Multimodal Systems

  111. Tasks that had been solved by CLOVA Assistant NLP Speech

    Recognition Text Speech Synthesis Text
  112. None
  113. This is one of our “reference” dialog system Open-domain Dialog

    System (e.g. For Dialog system competition 4 w/ HyperCLOVA JP 39B) Speech Recognition (LINE’s Speech-to-Text system) Speech Synthesis (LINE’s Text-to-Speech system) Text Text Voice Voice Chatting Smooth response w/ beautiful voice A dialogue system beyond the “Uncanny Valley” is imminent Wow
  114. Challenge: Active handling of non-textual data in NLP NLP Speech

    Recognition Text Speech Synthesis Text Computer Vision Features Features Control Control Robot / Avatar
  115. Challenge: Active handling of non-textual data in NLP NLP Speech

    Recognition Text Speech Synthesis Text Computer Vision Features Features Control Control Robot / Avatar In the future, NLP engineers need to acquire features from many modalities for NLP that other fields do not need.
  116. Challenge: Active handling of non-textual data in NLP NLP Speech

    Recognition Text Speech Synthesis Text Computer Vision Features Features Control Control Robot / Avatar We also need to send signals to control everything that other fields are not willing to do. In the future, NLP engineers need to acquire features from many modalities for NLP that other fields do not need.
  117. Foundation Model is necessary for Multi-Modal - The history of

    NLP is strongly linked to the development of AI-related technologies - The widespread use of the Foundation Model has made it easier to leverage the output of speech and image recognition - LINE wants to move in the direction of R&D of Multimodal NLP with our own foundation models Foundation Model DNN Traditional only ML Rule only Small LM only Text only Multi-Modal
  118. Agenda Realizing applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World When will the Japanese version of HyperCLOVA be released?
  119. Agenda Realizing applications with Prompting Performance Evaluation of 82B JP

    Model Developing Applications with HyperCLOVA Deploying Foundation Models in Real World When will the Japanese version of HyperCLOVA be released?
  120. CLOVA Studio is already released for KR model

  121. CLOVA Studio for JP model will come at FY2023

  122. Summary - A foundation model application is already in a

    state to produce results that are useful enough for humans, and we are working hard to make it available to you! - However, it is necessary to prepare many subsystems to compensate for the missing functions in the foundation model - LINE is R&D these systems with the aim of providing them to you as well - LINE has successfully built a Japanese model of HyperCLOVA's 82B. We shared evaluation results and know-how for application
  123. Thank you!