Google Cloud Platform에서 시간과 비용과 성능 세 마리 토끼를 잡는 daily batch 구축기

ࣗѐ %BUB1MBUGPSNࢎসप%BUB1MBUGPSNѐߊ౱%BUB౵౟ ߊ಴੗

(PPHMF$MPVE1MBUGPSNীࢲ%BJMZCBUDIQSPDFTTJOHҳ୷ೞӝ दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ ࢎ۹߽۳ࢿਸ୭؀ചೞৈઁೠदрউী୭؀ബਯղӝ ࢎ۹ؘ੉ఠ୊ܻҗ੿ীࢲয়ܨܳѺܻೞৈ৔ೱب୭ࣗചೞӝ ࢎ۹"MMPZ%#۽ੋݫݽܻ%#ࢿמਸٮۄ੟ӝ

ߊ಴੹ীঌݶજ਷ѐ֛ٜ )FBMUIDBSFEBUB3FTFBSDI4VJUF )34 "QBDIF4QBSL "MMPZ%# "QBDIF"JS
fl PX %JSFDUFE"DZDMJD(SBQI %"( &YUSBDU5SBOTGPSN-PBE &5- %FMUB-BLF

(PPHMF$MPVE1MBUGPSNীࢲ %BJMZCBUDIQSPDFTTJOHҳ୷ೞӝ

ؘ੉ఠۨ੉௼ ߽ਗ (PPHMF$MPVE1MBUGPSNীࢲ%BJMZCBUDIQSPDFTTJOHҳ୷ೞӝ ؘ੉ఠਝযೞ਋झ )34 (PPHMF$MPVE1MBUGPSN "11-BZFS %8-BZFS %--BZFS 42-
5IF"QBDIF"JS fl PX UIF"QBDIF4QBSLMPHPTBSFBUSBEFNBSLPGUIF"QBDIF4PGUXBSF'PVOEBUJPO"MMSJHIUTUPUIJTMPHPBSFPXOFECZUIF"QBDIF4PGUXBSF'PVOEBUJPO 5IF%FMUB-BLFMPHPJTBUSBEFNBSLPGUIF%FMUB-BLFQSPKFDU"MMSJHIUTUPUIJTMPHPBSFPXOFECZUIF-JOVY'PVOEBUJPO 5IF(PPHMF$MPVE1MBUGPSNMPHPJTBUSBEFNBSLPG(PPHMF--$"MMSJHIUTUPUIJTMPHPBSFPXOFECZ(PPHMF--$

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ )FBMUIDBSFEBUB3FTFBSDI4VJUF )34 ੄޷࣌ ৈ۞ӝҙ੄ؘ੉ఠܳэ਷ߑधਵ۽ઑഥೡࣻ੓ѱೠ׮ ׮নೠઑѤਵ۽Ѩ࢝ೞ؊ۄبࡅܰѱѾҗܳઁҕೠ׮
যઁө૑੄ؘ੉ఠܳোҳীࢎਊೡࣻ੓ѱઁҕೠ׮

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ ӒܻҊݽٚѐߊ੗੄җઁ ӝೠղীࢲ࠺झبয়೑ೞҊ تبইՙҊ সޖࢤ࢑ࢿب֫੉Ҋ
ୡҗӔޖب઴੉Ҋ ࠺ਊ दр ࢿמ ੉޷૑୹୊IUUQT IPSJ[POLJBTSFLS

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ ੉ઙؘ੉ఠ߬੉झܳ؀਽ ߽ਗؘ੉ఠ߬੉झীࠗೞ୭ࣗച बঠ_࢜߷दр؀ীઙܐ &5-ҳ୷ୡӝח੤प೯࠼بо֫਺
ؘ੉ఠۨ੉௼ ߽ਗ (PPHMF$MPVE1MBUGPSN %--BZFS

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ ߽ਗ݃׮׮ܲݽন੄ؘ੉ఠܳزੌೠݽ؛۽оҕ &5-ҳ୷ୡӝח੤प೯࠼بо֫਺ ؘ੉ఠۨ੉௼ ؘ੉ఠਝযೞ਋झ (PPHMF$MPVE1MBUGPSN
%8-BZFS

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ইঠ೮؍੉ਬ ؀ਊ۝ప੉࠶ٜ੄ઑੋ ࠂ੟ೠѨ࢝ઑѤ ؘ੉ఠ࠙ನ߸ച ؘ੉ఠਝযೞ਋झ
)34 (PPHMF$MPVE1MBUGPSN "11-BZFS 42-

दр ࠺ਊ ࢿמ੄୭੸੼ਸ଺ӝਤೠ पୌ

য়ܨѺܻ ߽۳ച ੗ਗ ࠺ਊ दр য়ܨܳߊࢤ૑੼ীݠޖܰѱ ਗӝৠ ೡࣻ੓חݽٚѪਸೞӝ

ࢎ۹ ߽۳ࢿਸ୭؀ചೞৈ ઁೠदрউী୭؀ബਯղӝ

߽۳ചযڌѱ೧ࠁदѷযਃ ઁডઑѤ ੉ઙؘ੉ఠ߬੉झ؀਽ 4"1)"/" 0SBDMF 5JCFSP .TTRM 7FSUJDB١
೧ঠೠ׮ݶ ߽ਗؘ੉ఠ߬੉झࠗೞ୭ࣗച೧ঠೠ׮ݶ बঠ_࢜߷दр؀উীઙܐ೧ঠೠ׮ݶ ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ ߽ਗҗ(PPHMF$MPVE1MBUGPSNр੄؀৉಩୭؀ೠഝਊ ৘ ঠрNCQT ઱рNCQT ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ ߽ਗؘ੉ఠ߬੉झ੄ࢎਊ۝߂ࢎনҊ۰ೠزद੽ࣘࣁ࣌౵ঈ ߽ਗؘ੉ఠ߬੉झ੄$16.&.03:%*4,ࢎਊ۝ী࠺۹ೞৈزद੽ࣘࣁ࣌ࣻഛੋ ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ ࣗझప੉࠶੄ౠࢿীݏח࠙ೡ੸੤੹ۚࣻ݀ զ૞ژחं੗ஸۢ੄ؘ੉ఠ࠙ನبઑࢎറੌ੿ҳрਵ۽աׇ୶୹ ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ ୶୹ઑѤীݏחࣗझప੉࠶੄ੋؙझઑࢎ ઑѤীݏחੋؙझоহਸ҃਋ ࢎ੹ীࢤࢿਃ୒ ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ Ҵࣗ੸ੋ߽۳ࢿઑ੺ ୶୹ೡؘ੉ఠ௼ӝীݏח4QBSLܻࣗझ೐۽౵ੌࢎਊ TNBMMNFEJVNMBSHFY - MBSHF ؘ੉ఠۨ੉௼
߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ ੹৉੸ੋ߽۳ࢿઑ੺ "JS fl PX1PPMT۽زदীप೯ೡ%"(ࣻઑ੿ ؘ੉ఠۨ੉௼ ߽ਗ %--BZFS

߽۳ച੉ۧѱ೧ࠌणפ׮ Ӓޖ঺ࠁ׮ب઺ਃೠ۽Ӧ ୶୹ оҕ ੷੢ যוҳрীࢲदрਸױ୷ೡࣻ੓ਸ૑ঌࣻ੓חо੢୐ߣ૩ߑߨ ؘ੉ఠۨ੉௼
߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ Ӓޖ঺ࠁ׮ب઺ਃೠ۽Ӧ ߽ਗҗэ਷ઁ੄दझమীࠗೞܳ઱૑ঋח׮חѪਸૐݺೡࣻ੓חѪب۽Ӧ

߽۳ച੉ۧѱ೧ࠌणפ׮ #JH5BCMF 5PQ ୨Ѥࣻ ಣӐੌߓ஖Ѥࣻ ੌੌ߸҃࠺ਯ ੸੤ࣗਃदр ࠙ ୶୹ࣗਃदр ࠙
୶୹दр࠺ਯ 5 5 5 5 5

߽۳ചযڌѱ೧ࠁदѷযਃ ઁডઑѤ ߽ਗ૑੼߹ؘ੉ఠ߬੉झоޛܻ੸ਵ۽ٮ۽੓׮ݶ झః݃חزੌೞ׮ݶ ؘ੉ఠۨ੉௼ ߽ਗ

߽۳ച੉ۧѱ೧ࠌणפ׮ п૑੼߹۽ة݀੸ਵ۽प೯غب۾4QBSL௿۞झఠࢤࢿ

߽۳ച੉ۧѱ೧ࠌणפ׮ п૑੼߹۽ߓ஖о৮ܐغݶ௿۞झఠח߄۽࢏ઁ

߽۳ച੉ۧѱ೧ࠌणפ׮ ೞա੄ࣗझ௏٘۽૑੼߹प೯оמೞѱ౵ۄ޷ఠച hrs_start >> reset_manual_var for hospital_code in hospital_code_set:
( reset_manual_var >> create_cluster_task_dict[ f"check_create_cluster_{hospital_code}" ] >> [ create_cluster_task_dict[ f"create_cluster_{hospital_code}" ], create_cluster_task_dict[ f"skip_create_cluster{hospital_code}" ], ]

߽۳ച੉ۧѱ೧ࠌणפ׮ ߽۳ചܳాೠ4QBSL௿۞झఠ6UJMJ[BUJPOҗࣗਃदр୭੸ച

#FGPSF "GUFS ߽۳ച੉ۧѱ೧ࠌणפ׮

߽۳ച੉ۧѱ೧ࠌणפ׮ 4QBSL௿۞झఠݠनࢎন9ؘ੉ఠন9߽۳ࢿ੄ઑ೤୭੸੄ࣗਃदр୭ࣗ੄࠺ਊ

߽۳ച੉ۧѱ೧ࠌणפ׮ प೷ :"3/ .BDIJOF5ZQF $16 .&. (# PG/PEFT 1SJDF QFSIPVS
%VSBUJPO IPVS 'JOBM1SJDF EBZ #BTFMJOF OTUBOEBSE OIJHINFN OTUBOEBSE FTUBOEBSE FTUBOEBSE ZBSOTDIFEVMFSDBQBDJUZNBYJVN - BN - SFTPVSDF - QFSDFOU EFGBVMU NBTUFS OXPSLFST

߽۳ച੉ۧѱ೧ࠌणפ׮ ೠߣ୭੸ച೧֬਷ѱӒ؀۽ਬ૑ؼܻহભ

ࢎ۹ ؘ੉ఠ୊ܻҗ੿ীࢲ য়ܨܳѺܻೞৈ ৔ೱب୭ࣗചೞӝ

য়ܨѺܻযڌѱ೧ࠁदѷযਃ ઁডઑѤ ప੉࠶ױਤ੄&5-੘সೠ׮ݶ ݽ؛р੄੄ઓࢿҗࠂ੟ࢿ੉੓׮ݶ ݒੌप೯غחࣻୌѐ੄%"(о੓׮ݶ ؘ੉ఠۨ੉௼
߽ਗ ؘ੉ఠਝযೞ਋झ

য়ܨѺܻ੉ۧѱ೧ࠌणפ׮ पઁ&5-੘সਸࣻ೯ೞחо੢੘਷ױਤ੄%"(ҳࢿ %"(߂&5-۽૒਷ݵ١ࢿਬ૑ ղࠗযڃకझ௼оपಁೞ؊ۄب %"(੤प೯दزੌೠѾҗࠁ੢

য়ܨѺܻ੉ۧѱ೧ࠌणפ׮ ࣻୌѐֈח%"(ܳױ҅ച߂Ӓܛച ؘ੉ఠۨ੉௼ ߽ਗ ؘ੉ఠਝযೞ਋झ %8-BZFS %--BZFS

য়ܨѺܻ੉ۧѱ೧ࠌणפ׮ 5SJHHFS%"(ਵ۽-FBG%"(Tഐ୹ઁয߂प೯࢚కҙ଴ %BUB-BLF 5SJHHFS%"( %BUB8BSFIPVTF 5SJHHFS%"( ୭࢚ਤ%"( %"( %"(
%"( %"(

য়ܨѺܻ੉ۧѱ೧ࠌणפ׮ যڃ%"(оपಁೞ؊ۄب৔ೱب੹౵୭ࣗച पಁ૑੼ࣻ੿റীաݠ૑प೯ೞৈࠛ೙ਃೠप೯दрױ୷ ؘ੉ఠۨ੉௼ ߽ਗ ؘ੉ఠਝযೞ਋झ %8-BZFS %--BZFS

য়ܨѺܻ੉ۧѱ೧ࠌणפ׮ 5SJHHFS%"(р੄प೯੄ઓࢿࢸ੿ %BUB-BLF 5SJHHFS%"( %BUB8BSFIPVTF 5SJHHFS%"( ୭࢚ਤ%"( %"( %"(
%"( %"(

ࢎ۹ "MMPZ%#۽ ੋݫݽܻ%#ࢿמٮۄ੟ӝ

ࢿמٮۄ੟ӝযڌѱ೧ࠁदѷযਃ ઁডઑѤ ౵౟ցࢎח࠺ਊ੺хਸਤ೧ੋݫݽܻ%#ܳ1PTUHSF42-۽؀୓ೞӡਗೠ׮ݶ ੋݫݽؘܻ੉ఠ߬੉झ੄ࢿמҗӔ੽ೞӡӝ؀ೠ׮ݶ

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ Ѩ࢝؀࢚߂ઑѤ ୶୹೯ࣻ ੋݫݽܻ%# ࣗਃदр 1PTUHSF42- ࣗਃदр 1PTUHSF42-୭੸ച
ࣗਃदр ୨ਊ۝5# Ѩ࢝ਊ۝(# Ѩ࢝ӝрড֙஖ Ѩ࢝௏٘ѐ _ ੋݫݽܻ%#ࢎন$165ISFBE.FNPSZ5#4UPSBHF5# 1PTUHSF42-%#ࢎন$165ISFBE.FNPSZ5#4UPSBHF5#/7.&

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ *OEFY - POMZTDBO DPWFSJOHJOEFY ੋؙझܳѐࢶೞৈझபࣘبܳೱ࢚ Ѩ࢝ীࢎਊغחtੋؙझ৬Ѩ࢝Ѿҗੋtಕ੉۽٘ܳҳ࠙ೞৈੋؙझࢤࢿ
select y from tab where x = 'key'; create index tab_x_y on tab(x) include (y);

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ QH@QSFXBSNFYUFOTJPO ੗઱ࢎਊغחੋؙझоݫݽܻীࢲ޻۰ա૑ঋѱ׮दநय ݒੌ_दزউदр݃׮QSFXBSNࣻ೯

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ *OUFSNFEJBUF.BSU ੗઱ࢎਊೞחஸٜۢਸݽইױੌప੉࠶۽ҳࢿ ઑੋപࣻхࣗªप೯दрױ୷ ݫݽܻநयਊ۝хࣗª੗઱ࢎਊೞח׮ؘܲ੉ఠܳݫݽܻীৢܾࣻ੓਺

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ (PPHMF$MPVE1MBUGPSNীࢲח"MMPZ%# ࠙ࢳ௪ܻী؀೧಴ળ1PTUHSF42-ࠁ׮୭؀ߓࡅܴ ஸۢূ૓੉ప੉࠶झப ૘҅ ੿۳
ઑੋীࢿמ࢚੉੼ઁҕ IUUQT DMPVEHPPHMFDPNBMMPZECCFOF fi UT

ࢿמٮۄ੟ӝ੉ۧѱ೧ࠌणפ׮ ா੉झ दաܻয় ୨ਊ۝ 5# ݾ಴ 1PTUHSF42- "MMPZ%#
Ѩ࢝ਊ۝(# Ѩ࢝ӝрড֙஖ Ѩ࢝௏٘ѐ 6OEFSTFDPOET Ѩ࢝ਊ۝(# Ѩ࢝ӝрড֙஖ Ѩ࢝௏٘ѐ Ѩ࢝ਊ۝(# Ѩ࢝ӝрড֙஖ Ѩ࢝௏٘ѐ 1PTUHSF42-୭੸ചXJUIPVUQH@QSFXBSN "MMPZ%#ࢎন$165ISFBE.FNPSZ(#

աоݴ ୭੸ചܳਤೠपୌ߽۳ച৬য়ܨѺܻ ӒܻҊೡࣻ੓חݽٚѪਸೞӝ ୭੸ചҗ੿઺ীपઁ۽ೠѪ౱ਗٜҗೣԋप೷ೞҊѐࢶೞӝܳ߈ࠂೞӝ ୭੸ച۽঳਷Ѫ౱੉ೣԋೠ୭੸ച҃೷੉Ѫ਷҅ࣘࠂܻ۽جই২פ׮

Google Cloud Platform에서 시간과 비용과 성능 세 마리 토끼를 잡는 ...

Google Cloud Platform에서 시간과 비용과 성능 세 마리 토끼를 잡는 daily batch 구축기

More Decks by kakao

Other Decks in Programming

Featured

Transcript