카카오뱅크 MLOps

$PQZSJHIU,BLBP$PSQ"MMSJHIUTSFTFSWFE3FEJTUSJCVUJPOPSQVCMJDEJTQMBZJTOPUQFSNJUUFEXJUIPVUXSJUUFOQFSNJTTJPOGSPN,BLBP ஠஠য়ߛ௼.-0QT JG LBLBP ஠஠য়ߛ௼ ӣթӝ5PCZL

.-0QTрۚࣗѐ .-0QTইఃఫ୊ҳࢿਃࣗ߂ౠ૚ য়೑ࣗझبੑী؀ೠҊ޹ .-0QT੉೧ҙ҅੗ഈস ஠ߛ .-0QTदझమҳࢿ߂઱ਃஹನք౟ ஠ߛ .-ࢲ࠺झ҃೷ ݛ਺݈

.-0QTрۚࣗѐ

.-0QTۆ .-0QTژח.-0QTח਍৔ജ҃ীࢲ.-ݽ؛ਸউ੿੸੉Ҋബਯ੸ਵ۽ߓನೞҊਬ૑ҙܻೞחѪਸݾ಴۽ ೞחੌ۲੄ҙ೯ੑפ׮ 8JLJQFEJB .-0QTח.-दझమѐߊ %FW җ.-दझమ਍৔ 0QT ਸా೤ೞחѪਸݾ಴۽ೞח.-ূ૑פয݂ޙച ߂ҙ೯ੑפ׮
(PPHMF$MPVE ੉޷૑୹୊.-0QTWT%FW0QT

ࣗ೐౟ਝযѐߊ੗р੄ࣗాҗഈসਸъઑೞחѐߊޙച %4৬ূ૑פযр੄ࣗాҗഈসਸъઑೞח.-ূ૑פয݂ޙച ѐߊ੗ܳਤೠ੗زച೐۽ࣁझੋ૑ࣘ੸ੋా೤ਸ੄޷ ௏٘୽جޙઁ೧Ѿ ؘ੉ఠ ؘ੉ఠझః݃ ݽ؛ী؀೧పझ౟ೞҊѨૐ ૑ࣘ੸ੋࢲ࠺झઁҕ߂૑ࣘ੸ੋߓನܳ੄޷ ݽ؛೟ण߂৘ஏࢲ࠺झܳ੗زਵ۽ߓನ೧ঠೞחदझమ
.-दझమҊਬࣘࢿ ݽ؛ਸ੗زਵ۽੤೟णदఃҊࢲ࠺झઁҕ %FW0QT৬.-0QT੄࠺Ү • CT (지속적 학습) • CD DEVOps • 정의 • CI .-0QT • 정의 • CI • CD • CT (지속적 학습) • CD • CI • CI • CD • 정의 DEVOps • 정의 .-0QT ※ DevOps의 CT(Continuous Testing)과 MLOps의 CM(Continuous Monitoring)은 추가적으로 설명하지 않음. ※ DevOps와 MLOps의 차이는 서비스를 운영하는 목적의 차이가 있을 뿐, 상당히 유사하다고 볼 수 있음.

&YQFSJNFOUBUJPO %FWFMPQNFOU 5SBJOJOH1JQFMJOF $*$% $POUJOVPVT 5SBJOJOH .PEFM %FQMPZNFOU 4FSWJOH
.POJUPSJOH .-0QTݾ಴ 더 빠른 Iterations

.-0QTইఃఫ୊ҳࢿਃࣗ߂ౠ૚

.-0QTইఃఫ୊ (PPHMF$MPVE "84 "[VSF .-0QTইఃఫ୊੄ҳࢿਃࣗ߂ౠ૚ਸ੿ഛ൤౵ঈ೧ঠ ਋ܻ݅੄.-0QTইఃఫ୊ܳҳࢿೡࣻ੓਺

.-0QTইఃఫ୊ Google Cloud ୹୊(PPHMF$MPVE

.-0QTইఃఫ୊ AWS ୹୊AWS ইఃఫ୊

୹୊Azure ইఃఫ୊ .-0QTইఃఫ୊ Azure

.-0QTইఃఫ୊ Google MLOps - 수준 0 ✘$*$% ✘ࢿמݽפఠ݂ ୹୊(PPHMF$MPVE

.-0QTইఃఫ୊ Google MLOps - 수준 1 ୹୊(PPHMF$MPVE

.-0QTইఃఫ୊ Google MLOps - 수준 2 ୹୊(PPHMF$MPVE

஠஠য়ߛ௼.-0QTইఃఫ୊ (PPHMF.-0QTࣻળ_ױ҅੄ҳࢿਃࣗ৬ౠ૚ਸળࣻೞৈ ஠஠য়ߛ௼.-0QTইఃఫ୊ܳҳࢿೞ৓਺ %BUB FYUSBDUJPO %BUB QSFQBSBUJPO .PEFM USBJOJOH .PEFM
FWBMVBUJPO .PEFM WBMJEBUJPO 4PVSDF DPEF %BUB BOBMZTJT %BUB FYUSBDUJPO %BUB WBMJEBUJPO %BUB QSFQBSBUJPO .PEFM USBJOJOH .PEFM FWBMVBUJPO .PEFM WBMJEBUJPO .-NFUBEBUBTUPSF (JUMBC $* .PEFM SFHJTUSZ 'FBUVSF 4UPSF %BUB.BSU &%8 1FSGPSNBODF .POJUPSJOH $% .PEFMTFSWJOH 1SFEJDUJPO TFSWJDF %BUBTPVSDF NFUBTZTUFN 5SJHHFS &DBNT $% "VUPNBUFE1JQFMJOF 0SDIFTUSBUFE&YQFSJNFOU FYQFSJNFOUBUJPOUFTU QSPEVDUJPO #BUDI 'FUDIJOH .-0QT ஠஠য়ߛ௼.-0QTইఃఫ୊ח_ױ҅۽ա־য૓೯ೞ৓Ҋ ݃૑݄ױ҅੄ইఃఫ୊݅ҕਬ೤פ׮

오픈소스 도입에 대한 고민

য়೑ࣗझܻࢲ஖

য়೑ࣗझبੑী؀ೠҊ޹ ؘ੉ఠҗ೟੗ ূ૑פয Әਲ਼ӏઁ $%48 +VQZUFS 4"4੊ࣼೠജ҃ әѺೠ࠙ࢳജ҃߸ചী؀ೠ਋۰ "JSGMPX਍৔҃೷ য়೑ࣗझبੑীٮܲࠗ׸
ࠁউࢿब੄ ѐߊపझ౟਍৔੄ݎрܻ࠙

য়೑ࣗझࢶఖ t.-0QTࣁ҅оޖ঺ੋ૑ੜঌ૑ޅೞח࢚కীࢲ ъ۱ೠয়೑ࣗझܳبੑೞݶ ਋ܻ੄ࢤпҗदঠоગই૓׮ u ؘ੉ఠҗ೟੗ীѱ੗ਬ۽਍࠙ࢳജ҃ઁҕ .-౵੉೐ۄੋ૒੽ѐߊ .-ী؀ೠبݫੋ૑धਸևഃࠁ੗
दр੸ࠗ׸হ੉߄۽द੘ೡࣻ੓חജ҃ "JSGMPXѐߊী؀ೠ੗नх

MLOps 이해관계자 협업

.-0QT੉೧ҙ҅੗рழޭפா੉࣌߂ഈস੄઺ਃࢿ ؘ੉ఠҗ೟੗ ؘ੉ఠূ૑פয 48ূ૑פয সޖী؀ೠ੉೧৬ߓ۰ ؊݆਷ழޭפா੉࣌ ഈস

જ਷.-0QTޙചٜ݅ӝ .-0QTޙച ؘ੉ఠҗ೟੗ ؘ੉ఠূ૑פয 48ূ૑פয

MLOps 시스템구성 및 주요 컴포넌트

.-0QTदझమҳࢿب ҃৔ݎ Әਲ਼ѐߊݎ Әਲ਼਍৔ݎ FOE - VTFS .-4FSWJOH4FSWFS .PEFM3FQP %PDLFS
3FHJTUSZ .-1JQFMJOF 0OMJOF'4 0GGMJOF'4 (JU-BC ߓ஖ࢲࡂ (JU-BC पदрࢲࡂ .PEFM#VJME4FSWFS .FUB%BUB 4FSWJDF 4FSWJDF 4FSWJDF 경영망 금융개발망 금융운영망 .PEFM3FQP (JU-BC .PEFM#VJME4FSWFS (JU-BC %PDLFS 3FHJTUSZ .-1JQFMJOF 0GGMJOF'4 0OMJOF'4 .-4FSWJOH4FSWFS

'FBUVSF4UPSF 0GGMJOF'4 0OMJOF'4 %BUB4PVSDF $IBOOFM &%8 $PSF#BOLJOH 4USFBNJOH #BUDI LBGLB
'FBUVSF4UPSF .PEFM 3FBM - UJNF %BUB-BLF .PEFM 4FSWF .PEFM 5SBJOJOH 1SPDFTTJOH ,VEV)%'4 %BUB.BSU $IBOOFM $PSF#BOLJOH &%8 %BUB.BSU $IBOOFM 0OMJOF'4 0GGMJOF'4 .PEFM 4FSWF .PEFM 5SBJOJOH

.PEFM SFHJTUSZ .-ݽ؛౵੉೐ۄੋమ೒݁ .-ݽ؛౵੉೐ۄੋਸమ೒݁ചೞৈ ੗زചػ.-౵੉೐ۄੋਸऔҊࡅܰѱࢲ࠺झী੸ਊೣ %BUB FYUSBDUJPO %BUB WBMJEBUJPO %BUB
QSFQBSBUJPO .PEFM USBJOJOH .PEFM FWBMVBUJPO .PEFM WBMJEBUJPO 1SFEJDUJPO TFSWJDF झ௏য14* MJGU BDDVSBDZj 1FSGPSNBODF .POJUPSJOH $5 /P :FT ೖ୊14* ৘ஏчࢤࢿ -JGUj 005j .-'SBNFXPSLT ݽ؛ۨನ੷੢ ೖ୊ࢤࢿ ೖ୊੐ߓ٬ ݽ؛೟ण ૑ࣘ੸೟ण 14* 1PQVMBUJPO4UBCJMJUZ*OEFY ݽ૘ױ੄উ੿ࢿਸաఋղח૑಴ -JGU ೱ࢚ب ੹୓߈਽ܫ؀࠺ ೧׼࠙ਤ੄߈਽ܫ࠺Ү 005 0VUPG5JNF ݽ؛ѐߊীࢎਊػؘ੉ఠ৬৮੹൤׮ܲӝр੄ ؘ੉ఠ۽Ѩૐ %BUB1SPDFTTJOH

.-ݽ؛ۨನ ML 모델 목록

.-ݽ؛ۨನ ML모델 등록 ML모델 상세

.-ݽ؛ۨನ ML모델 빌드 및 배포

.-ݽ؛ۨನ ML모델 성능 모니터링 -JGU "DDVSBDZ 1SFDJTJPO 3FDBMMj

ؘ੉ఠҗ೟੗৬ূ૑פযр੄ழޭפா੉࣌ਸ׸׼ ੌҙػ௏٘7JFX ؘ੉ఠҗ೟੗оࢎਊೞח௏٘ܳ਍৔ࢲ࠺झী߄۽੸ਊ ࠛ೙ਃೠ௏٘߸ജ .-ݽ؛ࢲࡂਸࠁ׮औҊࡅܰѱ ௏٘੄न܉ࢿ .-0QTۄ੉࠳۞ܻ 역할 주요기능 'FBUVSF.BOBHFNFOU
%FGJOJUJPO 'JOE 'FBUVSF0QFSBUPST $SFBUF6QEBUF%FMFUF7JFX "VUPNBUJD'FBUVSF"OBMZTJT &%" $PSSFMBUJPO .PEFM.BOBHFNFOU %FGJOJUJPO 'JOE .PEFM0QFSBUPST $SFBUF6QEBUF%FMFUF7JFX 1JQFMJOF"QQMJFS • • • • • • • • • 1JQFMJOF)BOEMFS 7BMJEBUJPO 1BDLJOH 'FBUVSF .PEFM 1JQFMJOF 'FBUVSF .PEFM 1JQFMJOF

ML 서비스 경험

.-0QT૑ࣘ੸೟ण $5 $POUJOVPVT5SBJOJOH ੉޷૑୹୊JHVB[JP ஠஠য়ߛ௼"5.੉ਊѤࣻ .-0QT૑ࣘ੸೟ण « पઁ «
৘ஏ ª ஠஠য়ߛ௼"5.੉ਊѤࣻ .-0QT૑ࣘ੸೟ण $PODFQU%SJGUदр੉૑թীٮۄؘ੉ఠ੄ా҅੸ౠࢿ੉߸ೞחѪ

؀୹פૉझ௏য૑ࣘ੸೟ण $5 ੸ਊѾҗ .PEFM FWBMVBUJPO %BUB FYUSBDUJPO %BUB QSFQBSBUJPO .PEFM
USBJOJOH .PEFM WBMJEBUJPO .-౵੉೐ۄੋ ೖ୊ࢤࢿ ఋѶчࢤࢿ ׹উ૑ USBJOUFTUؘ੉ఠࣇ ੐߬٬୊ܻ ݽ؛೟ण ݽ؛౵ੌ੷੢ ݽ؛ಣо NFUSJD੿ࠁ੷੢ ݽ؛Ѩૐ 005 0VUPG5JNF ӝઓݽ؛җ$5ݽ؛ࢿמ࠺Ү $5ݽ؛ࢿמݽפఠ݂ ѐਘ ઱੄ $5ݽ؛੄Ѿҗо೦࢚જ਷Ѫ਷ইש ¬ $5ݽ؛੸ਊ «$5ݽ؛ «ӝઓݽ؛ ӝઓݽ؛җ$5ݽ؛ࢿמ࠺Ү $5ݽ؛ࢿמݽפఠ݂ ѐਘ

.-0QTۄ੉࠳۞ܻ઱ਃӝמ Date Parameters - DOU୭Ӕѐਘ࢚ಿѐࣻ - HJKVO@ZNӝળ੉غחਘ .
- UY@ZN@NPC.ਘ - UY@ZN@NPC.ਘ   4&-&$5"DVTUPNFS@JE \HJKVO@ZN^BTHJKVO@ZN $06/5 %*45*/$5CQSPEVDU@OBNF BTDOU '30.EFGBVMUPSEFST"-&'5065&3+0*/ EFGBVMUQSPEVDUT# 0/"QSPE@JE#QSPE@JE 8)&3&46#453 "PSEFS@EBUF \UY@ZN@NPC^s "/%46#453 "PSEFS@EBUF \UY@ZN@NPC^s "/%"PSEFS@JUFN@DPEFs "/%"PSEFS@TUBUVT ߓ࣠৮ܐ (3061#:"DVTUPNFS@JE - DOU୭Ӕѐਘ࢚ಿѐࣻ - CBTF@ZNӝળ੉غחਘ . - CG@N@ZN.ਘ - CG@N@ZN.ਘ   4&-&$5"DVTUPNFS@JE r\CBTF@ZN^BTHJKVO@ZN $06/5 %*45*/$5CQSPEVDU@OBNF BTDOU '30.EFGBVMUPSEFST"-&'5065&3+0*/ EFGBVMUQSPEVDUT# 0/"QSPE@JE#QSPE@JE 8)&3&46#453 "PSEFS@EBUF r\CG@N@ZN^s "/%46#453 "PSEFS@EBUF \CG@N@ZN^s "/%"PSEFS@JUFN@DPEFs "/%"PSEFS@TUBUVT ߓ࣠৮ܐ (3061#:"DVTUPNFS@JE { "base_ym": "202210", "thmm_ym": "202210", "bf_1m_ym": "202209", "bf_2m_ym": "202208", "bf_3m_ym": "202207", "bf_4m_ym": "202206", "bf_5m_ym": "202205", "bf_1m_dt": "20220901", "bf_2m_dt": "20220801", "bf_3m_dt": "20220701", "af_1m_ym": "202211", "af_2m_ym": "202212", "af_3m_ym": "202301” … } .-0QT-JC

.-0QTۄ੉࠳۞ܻ઱ਃӝמ Pandas Dataframe to Impala Table */4&35EFGBVMUVTFSJOGP 7"-6&4 rUPCZs rUPCZLs
rIPNFUPCZs VOBNF GVMMOBNF IEJS BCSU FUDBCSU BEN BEN WBSBEN BWBIJ - BVUPJQE CJO WBSMJCBWBIJ CJO CJO CJO DMPVEFSB IPNFDMPVEFSB DMPVEFSBj $MPVEFSB VTSMJCDMPVEFSB j pandas hue pandas hue

def save_pandas_to_table(self, pandas_df: PandasDataFrame, table_name: str, table_creation: Optional[bool] = False)
-> bool:   """   Pandas DataFrame 데이터를 지정한 테이블에 적재합니다.   :param PandasDataFrame pandas_df: Pandas DataFrame   :param str table_name: 데이터를 적재할 테이블명   :param bool table_creation: 테이블 미존재 시 테이블을 생성합니다. (default = False)     >>> pandas_df   base_dt col1 col2   0 202206 1 2   1 202206 1 2   >>> fs.save_pandas_to_table(pandas_df=pandas_df, table_name='tmp.test_table_name')   or   >>> fs.save_pandas_to_table(pandas_df=pandas_df, table_name='tmp.test_table_name', table_creation=True)     .. note:: 데이터를 적재할 테이블이 존재할 경우 테이블과 Pandas Dataframe 컬럼명과 동일해야 합니다.   """   if isinstance(pandas_df, PandasDataFrame):   if not self.meta_data.table_exists(table_name=table_name):   self._create_table_if_ture(pandas_df=pandas_df, table_name=table_name, table_creation= table_creation)   return self._save_pandas_to_table(pandas_df=pandas_df, table_name=table_name)   raise InputTypeException(f"Please input data in pandas dataframe type. type={type(pandas_df)}") .-0QTۄ੉࠳۞ܻ઱ਃӝמ Pandas Dataframe to Impala Table %PDTUSJOH ప੉࠶੗زࢤࢿ 0QUJPOBM

पदр.-ࢲࡂࢲ࠺झѐߊ 1SFEJDUJWF.PEFM.BSLVQ-BOHVBHF

ઁോࢎ؀୹୶ୌಕ੉૑ ࢚ಿಕ੉૑ पदр.-ࢲࡂࢲ࠺झѐߊ 추천 알고리즘 ML모델 서비스 $VTUPNFS ࠗѾ ࠗѾ
ઁോࢎ؀୹୶ୌಕ੉૑ ࢚ಿಕ੉૑

{ "responses": [ { "affiNo": "100000001", "affiCode": "CODE01", "score": 0.1182257
}, { "affiNo": "100000002", "affiCode": "CODE02", "score": 0.61977172 }, … { "affiNo": "100000015", "affiCode": "CODE15", "score": 0.3917292 } ] } { "requests": [ { "customId": "00000000001", "affiNo": "100000001", "affiCode": "CODE01", "feature01": 5.0E7, "feature02": 3.0E7, "feature03": 1.0E3, "feature04": 60, "feature05": 10, "feature06": 2.0, "feature07": 0.0, "feature08": "N", "feature09": "Y", "feature10": "Y" }, … ] } पदр.-ࢲࡂࢲ࠺झѐߊ 추천 알고리즘 ML모델 Inference 과정 3FRVFTU 3FTQPOTF %BUB1SFQBSBUJPO ୭ઙѐ*OQVU ౵ࢤ߸ࣻ୶о ѐ ୭؀ ѐ Request Data Preparation ୭ઙѐ*OQVU ౵ࢤ߸ࣻ୶о Response ↟↟↟

पदр.-ࢲࡂࢲ࠺झѐߊ BentoML 1차 성능 테스트 పझ౟ജ҃ 7.ࢲߡ $16DPSF .FNPSZH ؀
߮ష.-੄8PSLFSѐࣻ ௏যѐࣻ $16ࢎਊ۝_ 314 .FBO$OUT ૑োदр UIQDU _NT ࢿמపझ౟חѐౣ݂ (BUMJOH ਵ۽૓೯ೣ 314 3FRVFTUT1FS4FDPOE 1차 성능 테스트 결과

पदр.-ࢲࡂࢲ࠺झѐߊ BentoML 2차 성능 테스트 పझ౟ജ҃ ର৬زੌ 7.ࢲߡ $16DPSF .FNPSZH
؀ ߮ష.-੄8PSLFSѐࣻ ௏যѐࣻ $16ࢎਊ۝_ 314 .FBO$OUT ૑োदр UIQDU _NT 2차 성능 테스트 결과 ѐࢶࢎ೦ 1ZUIPO௏٘ࢿמѐࢶ ؀ۚߓ 314 3FRVFTUT1FS4FDPOE

1..-WFSTJPO )FBEFSDPQZSJHIU,/*.& "QQMJDBUJPOOBNF,/*.&j )FBEFS %BUB'JFMEEBUB5ZQFTUSJOHu OBNF&NQMPZNFOUu PQUZQFDBUFHPSJDBM 7BMVFWBMVF1SJWBUF 7BMVFWBMVF$POTVMUBOU 7BMVFWBMVF4FMG&NQ
7BMVFWBMVF14-PDBM j 7BMVFWBMVF/" 7BMVFWBMVF7PMVOUFFS %BUB'JFME j %BUB%JDUJPOBSZ j 1..- पदр.-ࢲࡂࢲ࠺झѐߊ PMML (Predictive Model Markup Language) ୹୊+BWB1..-"1* PMML File Sample

पदр.-ࢲࡂࢲ࠺झѐߊ PMML 모델 생성 코드 (데이터 과학자) GSPNMJHIUHCNJNQPSU-(#.$MBTTJGJFS   GSPNTLMFBSOQNNMQJQFMJOFJNQPSU1..-1JQFMJOF
  GSPNTLMFBSOQNNMQSFQSPDFTTJOHMJHIUHCNJNQPSUNBLF@MJHIUHCN@EBUBGSBNF@NBQQFS   GSPNTLMFBSOQNNMJNQPSUTLMFBSOQNNM     tuu1..-౵੉೐ۄੋࢤࢿtuu   NBQQFSNBLF@MJHIUHCN@EBUBGSBNF@NBQQFS 9@UFTUEUZQFT NJTTJOH@WBMVF@BXBSF5SVF   DMBTTJGJFS-(#.$MBTTJGJFS QBSBNT GFBUVSF@OBNF9@USBJODPMVNOTUPMJTU j   QJQFMJOF1..-1JQFMJOF < rNBQQFSs NBQQFS DMBTTJGJFS DMBTTJGJFS >     ݽ؛೟ण   QJQFMJOFGJU 9@USBJO :@USBJO DMBTTJGJFS@@FWBM@TFUFWBMT j     tuuݽ؛੷੢tuu   TLMFBSOQNNM QJQFMJOF GNPEFM@GJMF\NPEFM@GJMF@OBNF^

पदр.-ࢲࡂࢲ࠺झѐߊ ML모델 서빙 애플리케이션 코드 (엔지니어) @Bean  public Evaluator evaluator(ModelProperties
properties) throws JAXBException, SAXException, IOException {  File modelPmmlFile = new File(properties.getPmmlLocation());  return new LoadingModelEvaluatorBuilder().setLocatable(false)  .setVisitors(new DefaultModelEvaluatorBattery())  .load(modelPmmlFile)  .build();  } private LinkLoanResponse applyModelScore(LinkEmergencyLoanFeature linkEmergencyLoanFeature) {  Map<FieldName, ?> score = this.modelEvaluator.evaluate(linkEmergencyLoanFeature);  return new LinkLoanResponse(linkEmergencyLoanFeature.getIntr_ltmn_no(), linkEmergencyLoanFeature.getLkl_inst_cd(), (Double) score.get(this.modelEvaluator.getProbability()));  } PMML Model Loading PMML Model Evaluate

पदр.-ࢲࡂࢲ࠺झѐߊ PMML 3차 성능 테스트 పझ౟ജ҃ 7.ࢲߡ $16DPSF .FNPSZH ؀
੉੹ࢿמపझ౟ח؀ైੑ $16ࢎਊ۝੉ೞ 314 .FBO$OUT ૑োदр UIQDU _NT 3차 성능 테스트 결과 ؀ۚߓ 314 3FRVFTUT1FS4FDPOE

맺음말

જ਷.-0QTޙചٜܳ݅যࠁ੗ .-0QTޙച ؘ੉ఠҗ೟੗ ؘ੉ఠূ૑פয 48ূ૑פয

카카오뱅크 MLOps

카카오뱅크 MLOps

More Decks by kakao

Other Decks in Programming

Featured

Transcript