Upgrade to Pro — share decks privately, control downloads, hide ads and more …

서비스에 LLM 부스터 달아주기: 요약부터 AI Bot 까지

kakao
November 01, 2024

서비스에 LLM 부스터 달아주기: 요약부터 AI Bot 까지

#LLM #Generative AI #service

카카오에서 진행했던 일부 LLM 학습 및 서빙 과정 및 결과에 대한 소개
LLM을 요약과 같이 특정 specific task에 적용 하는 것 부터 시작하여, AI Bot 서비스로 발전시키는 과정에서 단계별로 발생하였던 테크니컬한 이슈들을 어떻게 해소하였는지 소개합니다.

발표자 : joel.j
AI Alignment joel.j 입니다. LLM 기반 서비스를 만들기 위해 여러가지 일들을 하고 있습니다.

kakao

November 01, 2024
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. $POTUSBJOU1SPNQUT ഑द੉۠1SPNQUܳदب೧ࠁन੸੓ਵनоਃ 
  ਃডѾҗח+TPOਵ۽ٜ݅য઻ $POTUSBJOFE%FDPEJOHহ੉1VSF--.ਵ۽दبೞח҃਋  ਃডѾҗܳ੗੉ղ۽ٜ݅য઻  ޙױਵ۽ٜ݅য઻

    0VUQVU੄'PSNBUਸt੿ഛೞѱuઁೠೞחѪ਷--.ীѱࢤпࠁ׮য۰਍ੌੑפ׮ ब૑যೠҴয୹۱੄ઁೠਸѢחQSPNQUۄݶ ؊؊਌য۰਍ੌੑפ׮
  2. 7JFX %PDVNFOU #VJMEFS 4VNNBSZ 3FQISBTJOH .PEFM $PNQPOFOU )JHIMJHIU &YUSBDUPS .PEFM

    $POTUSBJOU1SPNQUTदझమࢸ҅ ੽ӔߑߨޙઁܳTUBHF۽ଂѐࢲಽӝ 'JOE)JHIMJHIU.PEFMӟਃডޙױਸࢤࢿ 4VNNBSZ3FQISBTJOH.PEFMޙױঀ*OQVUਵ۽߉ই'PSNBUUJOHਸࣻ೯ೞחݽ؛
  3.    ࢚ਊݽ؛ 'JOF - UVOFE.PEFM 'JOF - UVOFE.PEFM

    3FUSZ4ZTUFN    'PSNBUUJOHಣо
  4.    WT࢚ਊݽ؛    8JO -PTF 5JF

    ಿ૕ಣо 3VCSJDӝ߈(15+VEHF +VEHF.PEFM(15 - 5VSCP
  5. ೠҴযীъѤೠݽ؛ٜ݅ӝ #ӏݽ੄--.੉؊ۄب ই૒ೠҴযࢤࢿ਷ઑӘࠛউ੿ೠ࢚కੑפ׮ 2XFO৘द ")551 )ZQFSUFYU5SBOTGFS1SPUPDPM חਢ࠳ۄ਋੷৬ਢࢲߡр੄ాनਸ ਤೠ೐۽ష௒ੑפ׮)551җ)551חпп׮ܲߡ੹ਵ۽ ࢲ۽׮ܲౠ૚җ ࢿמਸઁҕ೤פ׮ৈӝীࢲח)551߂)551ীࢲо੢઱ਃೠѐ੄

    )551ݫࢲ٘৬Ӓ৉ೡী؀೧ࢸݺೞѷणפ׮  (&5৉ೡࢲߡ۽ࠗఠౠ੿ܻࣗझܳਃ୒೤פ׮  ౠ૚நय੉оמ೤פ׮  ইই٣FNQU JEFNQPUFOU ৈ۞ߣഐ୹೧بزੌೠѾҗܳ߈ജ೤פ ׮ؘ੉ఠח63-ীನೣغ޲۽ӡ੉ীઁೠ੉੓णפ׮ 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয় ")551 )ZQFSUFYU5SBOTGFS1SPUPDPM חਢীࢲؘ੉ఠܳҮജೞӝਤೠ೐ ۽ష௒ੑפ׮)551җ)551ݽف੉೐۽ష௒੄ߡ੹੉ݴ ৈ۞ݫࢲ٘ܳా ೧׮নೠ੘সਸࣻ೯೤פ׮ইې੄о੢ੌ߈੸ਵ۽ࢎਊغחо૑ݫࢲ٘৬Ӓٜ੄ ৉ೡী؀೧ࢸݺೞѷणפ׮ (&5о੢ੌ߈੸ੋݫࢲ٘۽ ࢲߡ۽ࠗఠ੗ਗਸਃ୒೤פ׮63-ীನೣػݽٚ ੿ࠁܳా೧ౠ੿ೠ੗ਗਸоઉ২פ׮(&5ݫࢲ٘חউ੹ೞҊ幂زੌ TBGFBOE JEFNQPUFOU ೞ޲۽ زੌೠਃ୒ਸৈ۞ߣࠁղبزੌೠѾҗܳ঳ѱؾפ׮ 2XFO#*OTUSVDU 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয় 2XFO#*OTUSVDU
  6. ೠҴযীъѤೠݽ؛ٜ݅ӝ #ӏݽ੄--.੉؊ۄب ই૒ೠҴযࢤࢿ਷ઑӘࠛউ੿ೠ࢚కੑפ׮ -MBNB৘द ")551ݫࢲ٘)551ݫࢲ٘ח)551ਃ୒ীࢲࢲߡоࣻ೯೧ঠೡز੘ਸ૑੿ೞחߑߨੑפ׮ )551߂)551ীࢲࢎਊغח઱ਃݫࢲ٘ח׮਺җэणפ׮)551߂੄઱ਃݫ ࢲ٘  (&5 

    ৉ೡਃ୒ػ੗ਗਸࢲߡীࢲоઉ২פ׮  ਃ୒ࠄޙহ਺  ਽׹ࢲߡীࢲਃ୒ػ੗ਗਸ߈ജ೤פ׮(&5ݫࢲ٘חࢲߡীࢲؘ੉ఠܳоઉৢٸ ࢎਊؾפ׮ (&5ਃ୒਷நदؼࣻ੓ਵݴ ઱ࣗ栏ীੑ۱ೞৈਃ୒ೡࣻ੓णפ׮ 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয় ")FSFBSFUIFNBKPSNFUIPETPG)551BOE)551 BMPOHXJUIUIFJS SPMFT )551.FUIPET  (&5 5IF(&5NFUIPEJTVTFEUPSFUSJFWFBSFTPVSDFGSPNBTFSWFS*UJT UIFNPTUDPNNPOMZVTFE)551NFUIPE5IFDMJFOUTFOETBSFRVFTUUPUIF TFSWFS BOEUIFTFSWFSSFTQPOETXJUIUIFSFRVFTUFESFTPVSDF j .FUB - -MBNB# .FUB - -MBNB# 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয়
  7. -BOHVBHF5SBOTGFS1P$4UFQ &YQ1P$4UFQ-BOHVBHF5SBOTGFSоز੘ೡѪੋо -MBNB୹द੹-MBNBӝ߈ਵ۽प೷ਸ૓೯೧ࠁওਸٸ ইې৬э੉ೠҴযࢤࢿ੉উغ؍ݽण੉೧ࣗغחѪਸഛੋ ")FSFBSFUIFNBKPSNFUIPETPG)551BOE)551 BMPOHXJUIUIFJS SPMFT )551.FUIPET  (&5

    5IF(&5NFUIPEJTVTFEUPSFUSJFWFBSFTPVSDFGSPNBTFSWFS*UJT UIFNPTUDPNNPOMZVTFE)551NFUIPE5IFDMJFOUTFOETBSFRVFTUUPUIF TFSWFS BOEUIFTFSWFSSFTQPOETXJUIUIFSFRVFTUFESFTPVSDF j 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয় ")551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡ਷׮਺җэणפ׮ (&5(&5ݫࢲ٘חࢲߡীࢲܻࣗझܳਃ୒ೞחؘࢎਊؾפ׮੉ݫࢲ٘חࢲߡীࢲؘ੉ఠܳ Ѩ࢝ೞחؘࢎਊؾפ׮৘ٜܳয ਢ࠳ۄ਋੷оਢಕ੉૑ܳਃ୒ೡٸ(&5ݫࢲ٘ܳࢎਊ೤פ׮ 1045j #FGPSF "GUFS 2)551߂੄઱ਃݫࢲ٘ѐ৬Ӓٜ੄৉ೡী؀ೞৈࢸݺ೧ࠁदয়
  8. ੿۝ಣо 2XFO# *OTUSVDU HFNNBC - JU -MBNB# *OTUSVDU &YQ &YQ

    &YQ %BUBTFU .JYJOH .JYJOH .JYJOH -PHJD,PS EFGBVMU QSPNQU       .5 - #FODI - ,P *OUFSOBM       .5 - #FODI - &/ *OUFSOBM      
  9. 4FSWJOH4QFDVMBUJWFEFDPEJOH -BSHF.PEFMਸࢲࡂೞѱؼ҃਋ 4QFDVMBUJWF%FDPEJOH੉5ISPVHIQVU੄੉੼ਸ઴ࣻ੓णפ׮ ׮݅੉ܳਤ೧߹ب੄%SBGU.PEFM੉೙ਃೞݴэ਷%BUBTFUਵ۽೟ण೧઴೙ਃо੓णפ׮ .PEFM %SBGU.PEFM 51 %SBGU51 %SBGU 5PLFOT

    "WH*OQVU 5PLFOT "WH0VUQVU 5PLFOT "DDFQUBODF 3BUF 5ISPVHIQVU *NQSPWFNF #    # #        #    # #        #    # #       
  10. 4FSWJOH4QFDVMBUJWFEFDPEJOH 4QFDVMBUJWF%FPDEJOH਷JOQVU PVUQVUUPLFO੄ӡ੉ী৔ೱਸ݆੉߉ѱؾפ׮ ޛۿ"DDFQUBODF3BUFب઺ਃೞ૑݅JOQVU PVUQVUUPLFOӡ੉بҭ੢൤݆਷৔ೱਸષਸഛੋ೮णפ׮ .PEFM %SBGU.PEFM 51 %SBGU51 %SBGU

    5PLFOT "WH*OQVU 5PLFOT "WH0VUQVU 5PLFOT "DDFQUBODF 3BUF 5ISPVHIQVU *NQSPWFNF #    # #        #    # #        #    # #       
  11. "HFOU 2য়ט౸Үզॿঌ۰઻ "દ࣠೤פ׮݅ ੷חपदрզॿ੿ࠁܳઁҕೡࣻহणפ׮о੢੿ ഛೠ੿ࠁܳ঳ਵ۰ݶզॿজ੉աਢࢎ੉౟ܳഛੋೞदӝ߄ۉפ׮ 2য়ט౸Үզॿঌ۰઻ "֙ਘੌ౸Ү੄զॿח؀୓۽൒ܻҊ࠺оղ۷णפ׮ ࢲ਎੄୭੷ӝৡ਷ب৓ਵݴ ୭Ҋӝৡ਷ب۽৘࢚غ঻णפ׮ ࠺۽ੋ೧оदѢܻоૣই૑Ҋj

    - ࢎਊೡ ో: search_web - ో ഐ୹ ੿ࠁ { "rephrased_query": "2024֙ 10ਘ 1ੌ ౸Ү զॿ", "analyze_result_of_user_query_intent_in_1_line" : "ࢎਊ੗ח 2024֙ 10ਘ 1ੌ ౸Ү੄ զॿܳ ঌҊ रয೤פ - Ѩ࢝ Ѿҗ [{ "content": "2024֙ 10ਘ 1ੌ ചਃੌ য়ט զॿ …” } …] 1VSF--. "HFOU--.
  12.  ੸੺൤HSBQITUBUFীݏѱ4ZTUFNQSPNQU ܳ߸҃ೡࣻ੓যঠೣ  ࢚ടীݏחUPPMਸ઱ੑೞৈMMN੉ࢶఖೡࣻ੉ѱ ೧ঠೣ -> ೨बӝמ  3F

    fl FYJPO৬э੉&WBMVBUPS৬SF fl FDUJPO NPEVMF੉ઓ੤ೞৈ੗न੄਽׹ਸಣоೞҊ׮਺ "DUJPOਸࢶఖೡࣻ੓যঠೣ 3FGFSFODF-BOHHSBQI  3FGFSFODF3F fl YJPO "HFOU
  13. -BOHBVHF5SBOTGFSJO'VODUJPO$BMM  'VODUJPO$BMM࢚ടद 0QFO"*$PNQBUJCMF"1*۽UPPM੉ֈয৳׮ݶ Ҋ੿੸ੋ4ZTUFNQSPNQU ܳ"QQFOEೞҊӒ࢚ടਸ೟णೠ׮ &Y MMBNBDIBUUFNQMBUF :PVIBWFBDDFTTUPUIFGPMMPXJOHGVODUJPOT5PDBMMBGVODUJPO QMFBTFSFTQPOEXJUI+40/GPSBGVODUJPODBMMj

     'VODUJPO"SHVNFOUHFOFSBUJPOѾҗীೠӖਸನೣೠؘ੉ఠܳࢸ҅ೠ׮  'VODUJPO1SPNQU৬'VODUJPO/BNF਷৔য۽ࢸ҅ೣ &Y GVODUJPOFYBNQMF@GVODUJPO@OBNF\=uFYBNQMF@OBNF=u=uೞաلࣇച੉౴=u^GVODUJPO  ୭ઙ3FTQPOTFীࢲೠӖ੉ನೣػؘ੉ఠܳࢸ҅ೠ׮
  14. ଵҊޙ೴  'FSOBOEP'FSOBOEFT/FUP %BWJE(PMDIJOGBSBOE&SJD)BSUGPSE0QUJNJ[JOH-BSHF-BOHVBHF.PEFMT6TJOH-BZFS - 4FMFDUJWF3BOL3FEVDUJPOBOE3BOEPN.BUSJY5IFPSZ   (PEEBSE $IBSMFT

    FUBM"SDFFT.FSHF,JU"5PPMLJUGPS.FSHJOH-BSHF-BOHVBHF.PEFMTBS9JWQSFQSJOUBS9JW    MBOHDIBJO t-BOH(SBQIu ֙ਘੌ IUUQT XXXMBOHDIBJODPNMBOHHSBQI  4IJOO /PBI FUBM3F fl FYJPO-BOHVBHFBHFOUTXJUIWFSCBMSFJOGPSDFNFOUMFBSOJOH"EWBODFTJO/FVSBM*OGPSNBUJPO1SPDFTTJOH4ZTUFNT  
  15. 2"