Share
Embed
【初心者向け勉強会#9】MLOpsの基本 ~構築から運用まで~ / MLOps Basics: From Development to Operations
Ogata Katsuya
March 08, 2026
Transcript
53 slides
-
ࣗݾհ • ໊લ: ॹํ ࠀ࠸ (͓͕ͨ ͔ͭ) • ग़: ٶ࡚ݝখྛࢢ
• ॴଐ: େࡕେֶ ใՊֶݚڀՊ B4 • झຯ: ւ֎ཱྀߦ / ϙʔΧʔ / ొࢁ • X: @ogata_katsuya • Homepage: www.ogatakatsuya.com 2 ݩͱ࣮ՈͱʹΌΜ͜ -
ຊߨ࠲ʹ͍ͭͯ 3 • ର • MLOpsʹ͍ͭͯΓ͍ͨ • ϓϩμΫτʹMLΛऔΓೖΕ͍ͨ • ༰
• MLOpsͱʁ • MLOpsʹ͍ͭͯͷجૅతͳ༰ -
͓ॻ͖ Agenda 1. MLOpsͱʁ (10) 2. MLOpsͷߏཁૉ (25) 3. LLMOpsͱʁ(5)
4. MLOpsΛମݧͯ͠ΈΑ͏ (30) 4 -
MLOpsͱʁ Machine Learning × Operations 6 Machine Learning Operations ×
DevOps for Machine Learning The techniques for operating the system with machine learning modules. -
ػցֶशͷྺ࢙ 2010͝Ζ͔Βίϯελϯτʹ͞Ε࢝ΊΔ 8 DeepLearning (2006) AlexNetͷੜ (2012) ResNetͷੜ (2015) AlphaGo
(2016) Transformer (2017) GPT3 (2020) -
MLOpsͷྺ࢙ 2020͝Ζ͔Β͞Ε͍ͯΔ👀 10 ੨ઢ: MLOps ઢ: DevOps ʮHidden Technical Debt
in Machine Learning Systemsʯ ͷެ։ (2015) Google Cloud NextͰ MLOpsͱ͍͏ݴ༿͕ॳΊͯΘΕΔ -
MLOpsͷྺ࢙ MLOpsͷࢥతੜ 11 Hidden Technical Debt in Machine Learning Systems
(NeurlIPS2015) ΑΓҾ༻ -
MLOpsͱʁ MLͱϓϩμΫτΛܨ͙Ξμϓλ 13 • MLΛ࣮ϓϩμΫτͰσϦόϦʔ͢ΔͨΊͷٕज़ • MLͱϓϩμΫτͷσϦόϦʔͱͷΛ͢Δ • MLνʔϜϞσϧͷվળʹूதͰ͖ΔΑ͏ʹ͢Δ •
ͦͷதͰMLʹΑΔٕज़తෛ࠴ΛஷΊ͍͔ͯͳ͍ͨΊͷٕज़ -
MLOpsͷߏཁૉ ͬ͘͘͢͟͝ΓͱMLOpsͷશମ૾ 16 Preprocessing Model development Training Inference Data/Model Registry
(versioning) loop Experimentation Watch -
࣮ݧج൫ MLͷ࠶ݱੑͱ։ൃੜ࢈ੑΛߴΊΔ • ։ൃڥΛվળ͢Δ • ύοέʔδϚωʔδϟʔͷಋೖ (uv, poetry, etc.) •
ڥͷίϯςφԽ • ࣮ݧ༻ͷڥΛ༻ҙ͢Δ (Jupyter notebookͳͲͷNaaS) • ࣮ݧཧπʔϧͷಋೖ (Weights and Bias, TensorBoard, etc.) 18 -
ֶशύΠϓϥΠϯ લॲཧ͔ΒֶशɾσϦόϦʔ·ͰͷϑϩʔΛඋ͢Δ • ֶशύΠϓϥΠϯ • σʔλऔಘ -> લॲཧ -> ֶश
-> ݕূ -> σϓϩΠ • ͜ͷύΠϓϥΠϯΛ͋Β͔͡Ίߏ͓ͯ͘͜͠ͱʹΑΓɺ҆શʹ͔ͭߴʹ ߴ࣭ͳMLϞδϡʔϧΛఏڙͰ͖Δ • MLνʔϜϞσϧͷ։ൃͷΈʹूதͰ͖Δ 19 -
ਪج൫ ຊ൪ͷτϥϑΟοΫʹ͑͏Δج൫Λ༻ҙ͢Δ • ML WorkloadҰൠతͳCPUॲཧͱҟͳΓɺҙ͕ଟ͍ • CPUͱผʹGPUͷ༻Λߟ͑Δඞཁ͕͋Δ • 1ϦΫΤετ͋ͨΓͷϦιʔεΛ༗͢Δ͕͍࣌ؒ •
2ͭͷਪํ๏͕͋Δ • όονਪ • ΦϯϥΠϯਪ 20 -
ਪج൫ ΦϯϥΠϯਪ / όονਪ 21 Users Server Data Store Server
ML Model ML Model Online inference Batch inference -
ਪج൫ ͦΕͧΕͷਪํ๏ͷPros/Cons • ΦϯϥΠϯਪ • Pros: ϦΞϧλΠϜʹ༧ଌ݁ՌΛฦ͢͜ͱ͕Ͱ͖Δ • Cons: αʔόʔΛৗʹཱͯͯɺϦΫΤετΛࡹ͚Δঢ়ଶʹ͓ͯ͘͠ඞཁ͕͋Δ
• όονਪ • Pros: ߴεϧʔϓοτͰඅ༻ରޮՌ͕ߴ͍ • Cons: ϢʔβʔͷϦΫΤετΠϕϯτʹରͯ͠ଈ࠲ʹਪͰ͖ͳ͍ 22 -
ਪج൫ ͦΕͧΕͷਪํ๏ͷϢʔεέʔε • ΦϯϥΠϯਪ • ΫϨδοτΧʔυͷෆਖ਼༧ଌ • ࠂαʔόʔͷ͓͢͢Ί༧ଌ • όονਪ
• γϣοϐϯάαΠτͷ͓͢͢Ί༧ଌ • إೝূͷຒΊࠐΈੜ (ϦΫΤετ͕དྷͨΒ͢Ͱʹ͋ΔͷͱݟൺΔ͚ͩ) • RAGͷߏங (υΩϡϝϯτͱΠϯσοΫεͷಉظεέδϡʔϧ͢Δ) 23 -
σʔλɾϞσϧͷόʔδϣϯཧ σʔλͱϞσϧ͕MLͷ؊ • ҰൠతͳϓϩμΫτ։ൃͱMLΛؚΉϓϩμΫτ։ൃͷҧ͍ • σʔλͱϞσϧ͕͋Δ͔Ͳ͏͔ • ίʔυͷόʔδϣϯཧGitHubͰߦ͏͜ͱ͕Ͱ͖Δ • MLͷڍಈΛܾఆ͚ͮΔͷʁʁ
• ܇࿅ʹ༻͞ΕΔσʔλͱ࣮ࡍʹਪ͢ΔϞσϧ • ͜ΕΒόʔδϣϯཧ͓͔ͯ͠ͳ͍ͱσάϨͰ͖ͳ͘ͳͬͯ͠·͏ 24 -
σʔλɾϞσϧͷόʔδϣϯཧ σʔλͱϞσϧ͕MLͷ؊ • جຊతʹόʔδϣϯʹ૬͢Δͷ • v1,v2ͱ͔ɺGitͰ͍͏ͱ͜Ζͷcommit hashͱσʔλɾϞσϧΛؔ࿈͚ͯετϨʔδ ʹอଘͰ͖Εྑ͍ͱࢥ͏ • MLʹಛԽͨ͠πʔϧҎԼ
• DVC (Data Version Control) • Gitͱಉ͡ײ͡ͷίϚϯυͰετϨʔδʹpushͰ͖Δ • MLFlow 25 -
ܧଓతֶश σʔλͷมԽʹؤ݈ͳϞσϧΛܧଓతʹ࡞Δ • CI/CD (Continuous Integration, Continuous Deployment/Delivery)ͱಉ͡Α͏ ʹɺCT (Continuous
Training) ͱݺΕͨΓ͢Δ • ͳͥ͜ͷΈ͕ඞཁ͔ʁ • ࣮ϓϩμΫτͰσʔληοτͷมΘΓ͏Δ • e.g. ϢʔβʔΠϕϯτ/ ੈͷதͷ / ग़͞ΕΔɾΧςΰϦ • ͜ΕΒͷมԽʹਵ͢ΔͨΊʹఆظతʹֶशΛ͢Δඞཁ͕͋Δ 26 -
ࢹج൫ ෆ҆ఆͳMLͷෆ۩߹ʹؾ͚ͮΔج൫Λ࡞Δ • MLϞσϧ֬తͳڍಈΛ͢Δͷ͋Δ • શͯͷೖྗͱग़ྗΛᘳʹςετ͢Δ͜ͱͰ͖ͳ͍ • ઌड़ͷ௨Γɺεϧʔϓοτෆ҆ఆʹͳΓ͕ͪ • MLෆ۩߹Ͱ໌ࣔతͳΤϥʔΛు͍ͯ͘Εͳ͍
• ͙͢ʹෆ۩߹όάʹؾ͚ͮΔΑ͏ʹࢹج൫Λ͓࣋ͬͯ͘͜ͱ͕ॏཁ 27 -
ࢹج൫ ओͳࢹ߲ • Ϟσϧͷ༧ଌ࣭ͷϝτϦΫε • σʔλυϦϑτ • όΠΞεͱެฏੑ • ӡ༻ɾιϑτΣΞϝτϦΫε
28 Ҿ༻: https://www.evidentlyai.com/ml-in-production/model-monitoring -
৫ͱMLOps ͦͷMLOpsຊʹඞཁͰ͔͢ʁ • MLOpsʹཧίετ͕͔͔ΔͨΊɺಋೖஈ֊త͔ͭ৻ॏʹߦ͏ • ৫ͷنͱϏδωεతͳ؍ΛؑΈͯຊʹඞཁ͔ߟ͑Δ • ·ͣ࠷ॳखಈͰ܇࿅Λߦ͍ɺຖճखಈͰσϓϩΠͰͳ͍ • ࠓճͷ߹ɺϋοΧιϯͰ࣮ӡ༻ʹ͍ͭͯͦ͜·Ͱߟ͑Δඞཁͳ͍
• ͔͠͠ɺ࣮ࡍͷӡ༻Λݟӽ࣮ͯ͢͠Δ͜ͱେมษڧʹͳΔ • ྫ͑ɺֶशύΠϓϥΠϯΛ࡞ͨ͜͠ͱͰΑΓՁఏڙʹྗͰ͖ΔΑ͏ʹͳͬͨΒ࠷ߴ🙆 31 -
LLMOpsͱʁ Large Language Model × Operations 34 Large Language Model
Operations × DevOps for Large Language Model -
LLMOpsͱʁ ͓͢͢Ίͷษڧձ 37 MLOps/LLMOps/AgentOpsษڧձ • MLOps/LLMOpsͷੜ͖ͨφϨοδاۀͷ ࣮ϓϩμΫτʹଟʹଘࡏ͢Δ • ಛʹLLOps, AgentOpsʹؔͯ͠୭ਖ਼ղΛ
͓࣋ͬͯΒͣɺࡧஈ֊ • (࠶ܝ) ςοΫϒϩάษڧձʹࢀՃ͍ͯ͠ ͖·͠ΐ͏ • ٯʹݴ͑ίϛϡχςΟʹد༩͢Δνϟϯε -
LLMOpsͱʁ ࣄྫհ (ςετ/࣭อূ) 38 • גࣜձࣾIVRy͞Μ • LLMΛ༻͍ͨࣗಈిରԠαʔϏε • ϓϩόΠμʹڧ͘ґଘ͍ͯ͠ΔҎ্ɺ
ϑΥʔϧόοΫઓུΛߟ͑Δඞཁ͕͋Δ • Ϟσϧ͕ߋ৽͞Εͨ࣌ɾϓϩϯϓτΛमਖ਼ ͨ࣌͠ͷ࣭Λ୲อ͢ΔςετΛͲ͏͢Δ ͔ • LLM as a Judge ࣮ӡ༻ͰֶΜͩԻରγεςϜͷධՁͱςετ LLM APIΛ2ؒຊ൪ӡ༻ͯۤ͠࿑ͨ͠ -
LLMOpsͱʁ ࣄྫհ (ࢹج൫) 39 • גࣜձࣾαΠόʔΤʔδΣϯτ͞Μ • LLMͷ࣭མͪΔՄೳੑ͕ଟʹ͋Δ • ໌ࣔతͳΤϥʔͳ͘མͪΔͷͰɺؔ͠ج൫͕ॏཁ
• LLMͷදతͳࢹج൫ • LangFuse • PromptOpsతͳϓϩϯϓτͷόʔδϣχϯά • LLM as a judgeΛ༻͍ͨࣗಈςετ • LLMͷೖྗɾग़ྗͷࢹ Langfuseͷߏங -
LLMOpsͱʁ ࣄྫհ (ࣗಈԽ) 40 • גࣜձࣾαΠόʔΤʔδΣϯτ͞Μ • גࣜձࣾLayerX͞Μ • ϓϩϯϓτͷࣗಈ࠷దԽ
• ͜ΕLLMOpsͷҰछͩͱࢥ͏ • ϓϩϯϓτΛίωίω͢ΔͷଐਓతʹͳΓ͕͔ͪͭɺ ܾ·ͬͨܕ͕͋ΔͷͰࣗಈԽͰ͖ͦ͏ • LLMϓϩϯϓτͱ͍͏Ϣʔβʔ͕ૢ࡞Ͱ͖Δॊೈͳύ ϥϝʔλ͕͋ΔͷͰ܇࿅ͳ͠ͰύʔιφϥΠζͰ͖Δ • In-Context Learning LLMͷϕϯνϚʔΫείΞΛ7ɺ100ԁͰ͋͛Δ AI Agent࣌ʹ͓͚Δʮ͑͏΄Ͳݡ͘ͳΔAIػೳʯͷ։ൃ -
(ؓٳ) LLMΞϓϦέʔγϣϯͷצॴ ϑϥΠϗΠʔϧ 41 • ϑϥΠϗΠʔϧ • AI͕ϢʔβʔͷߦಈᅂΛֶश͢Δ͜ ͱʹΑΓɺ͑͏͚ͩݡ͘ͳ͍ͬͯ ͘॥
• ઌड़ͷ௨ΓɺϓϩϯϓτΛ༻͍Ε͍ΖΜ ͳ͜ͱ͕Ͱ͖ͦ͏ • DeNA͞ΜͷYouTubeϓϩμΫτ͕ ษڧʹͳΓ·͢ Edge -
ࠓճ࡞͢ΔύΠϓϥΠϯ YOLOΛ༻͍ͨը૾ೝࣝ 45 VertexAI Cloud Storage Cloud Build Artifact Registry
Pipelines trigger Training Deployment Cloud Run Custom Jobs GitHub trigger Weights & Bias -
Vertex AIͱʁ اۀ͚ͷ౷߹ܕAI/ػցֶशϓϥοτϑΥʔϜ • ओͳػೳ • Gemini API • ADKΛ༻͍ͨAI
Agentͷߏங • Vertex AI Search (RAGͷΠϯσοΫεͱݕࡧ) • Ϟσϧ։ൃ • τϨʔχϯάɺσʔληοτɺFeature Storeɺςετ • τϨʔχϯάͰಠࣗͷύΠϓϥΠϯߏங͕Մೳ • ϞσϧͷσϓϩΠϝϯτ (ΤϯυϙΠϯτ) 46 -
Vertex AIͱʁ اۀ͚ͷ౷߹ܕAI/ػցֶशϓϥοτϑΥʔϜ 47 VertexAI Pipelines Training Deployment Custom Jobs
• ύΠϓϥϯػೳΛ༻͍ͯ ֶशύΠϓϥΠϯΛඋ • Custom jobΛఆٛͰ͖ΔͷͰ ਪαʔόʔͷϞσϧͷσϓϩΠ·Ͱ • +α • σϓϩΠલʹϞσϧͷݕূΛߦ͏ • σʔλͷલॲཧΛߦ͏ -
όʔδϣχϯάͱ࣮ݧཧ Cloud Storage / Weights and Bias 48 Cloud Storage
Weights & Bias • Cloud StorageΛ༻͍ͯσʔληοτɾϞσϧͷ όʔδϣχϯάΛߦ͏ • σʔληοτࠓճखಈͰόʔδϣχϯάϞσϧ Cloud BuildͷBuildIDͱରԠ͚ͮͯόʔδϣϯχϯά • +α • DVCΛಋೖͯ͠ΈΔ • Vertex AIͷσʔληοτɾFeature StoreػೳΛ ͬͯΈΔ -
Vertex AIͱʁ GitHub͔ΒͷࣗಈτϦΨʔ / CloudRunͷσϓϩΠ 49 • GitHubͷ܇࿅ઃఆϑΝΠϧͷpushΛτϦΨʔʹֶशύΠϓ ϥΠϯશମΛτϦΨʔ •
CloudBuildΛ༻͍ͯ܇࿅ͱਪ༻ͷdocker imageΛϏϧυ • ਪج൫ʹCloud RunΛ࠾༻ • ࠷ۙ GPU on Cloud Run͕GAʹ • +α • GitHub ActionsͰtest/linter/formatterΛೖΕͯΈΔ • Vertex AI EndpointsΛͬͯΈΔ Cloud Run Cloud Build GitHub trigger -
ϋϯζΦϯ YOLOΛͬͨΩϟϥΫλʔͷը૾ೝࣝ 50 • ͪ͜ΒͷϨϙδτϦΛ༻͍ͯϋϯζΦϯΛߦ͍·͢ • https://github.com/ogatakatsuya/mlops-distribution • ຊҰॹʹਐΊ͍ͯ͘༧ఆͰ͕ͨ͠ɺ ܇࿅ʹࢥͬͨΑΓ͕͔͔࣌ؒΔͷͰɺ࣮ࡍʹ
खΛಈ͔͔͢Ͳ͏͔ࣗݾஅͰ͓ئ͍͠·͢🙏 • ͍ʹͳΔ͘Β͍܇࿅͠Α͏ͱࢥ͏ͱ ͔ͳΓ͕͔͔࣌ؒΓ·͢⚠ -
ϋοΧιϯʹ͚ͯ 51 • MLΛϓϩμΫτʹΈࠐΉͱΕΔ͜ͱ͕͕Γ·͢ • ಛʹίϯϐϡʔλϏδϣϯܥ(YOLO)ΕΔ͜ͱ͕ଟ͍Ͱ͢ • ߟ͑Δ͜ͱ • σʔληοτΛߏஙͰ͖Δ͔(܇࿅͠ͳ͍ͳΒ͍Βͳ͍)
• ӡ༻Ͱ͖Δ͔ (CPU্Ͱಈ͘ͳΒͳ͓ྑ͍) • ࣮ͷΠϝʔδ͕͔ͭ͘ • MLOpsతͳ؍ΛೖΕΔ͔Ͳ͏͔৻ॏʹɻɻ • ͰҙࣝͰ͖ͯΔͱධՁ͕ྑͦ͞͏ʁ • ϓϨθϯʹ͔ͬ͠ΓΈࠐΈ·͠ΐ͏ -
;Γ͔͑Γ 52 • MLOpsͱʁ • Machine Learning Operations / DevOps
for ML • MLOpsͷߏཁૉ • ֶशύΠϓϥΠϯɾόʔδϣχϯάɾࢹج൫ɾܧଓతֶशɾɾɾ • LLMOpsͱʁ • LLM Operations / DevOps for LLM • ϋοΧιϯؤு͍ͬͯͩ͘͞💪