ػցֶशʹΑΔϚʔέοτ݈શԽࢪࡦΛࢧ͑Δٕज़Mercari Server Side Tech Talk Vol.2 ʙCREφΠτʙ hnakagawa
View Slide
ࣗݾհ• Hirofumi Nakagawa (hnakagawa)• 20177݄ೖࣾ• ॴଐSRE• σόΠευϥΠό։ൃ͔ΒϑϩϯτΤϯυ։ൃ·ͰΔԿͰ• NOT MLΤϯδχΞ• https://github.com/hnakagawa
͓ࣄ• ML Platform։ൃ• MLΤϯδχΞͱSREͷεΩϧΪϟοϓΛຒΊΔ• ML Reliability, SysML?, MLOps?• SREͷཱ͔ΒCREνʔϜΛॿ͚Δ
ML Platform• ͷML Platform• kubernetesϕʔε• ϩʔΧϧڥͱΫϥελڥͷࠩΛநԽ͢Δ• ศརAPI܈• طଘͷML FrameworkΛ༻͠؆୯ʹTraining/ServingΛߦ͏ڥΛఏڙ
ͦͷ͏ͪOSSͰެ։༧ఆ(ଟ
ࠓͷAgenda ϦΞϧλΠϜࢹγεςϜ
ϦΞϧλΠϜࢹγεςϜ• ௨শ Lovemachine• ML Platform্ʹ࣮͞Ε͍ͯΔ.-1MBUGPSN USBJOJOHDMVTUFS-PWFNBDIJOF($4GKEPubSub.-1MBUGPSN TFSWJOHDMVTUFS-PWFNBDIJOF
ML ModelͷServing….?
Model Serving APIͷߏྫ5FOTPS'MPX 4FSWJOH5'.PEFM5'.PEFM'MBTL4, .PEFM4, .PEFM4, .PEFMgRPC.FSDBSJ"1*RESTFlaskͰલॲཧΛߦ͍ ཪͷTensorFlow Servingʹ͍͛ͯΔ
Model Serving API Streaming ver ͷߏྫ5FOTPS'MPX 4FSWJOH5'.PEFM5'.PEFM.-1MBUGPSN'SBNFXPSL PS "QBDIF#FBN 4, .PEFM4, .PEFM4, .PEFMgRPCPubSub
TensorFlow Serving• TensorFlow project͕ఏڙ͍ͯ͠ΔServingڥ• PythonॲཧܥΛհͣ͞ʹTFͷmodelΛservingͰ͖Δ• ඪ४ͷ࣮ͰgRPCͰAPIΛఏڙ
ModelͱίϯςφɾΠϝʔδ• ڊେͳML ModelΛίϯςφɾΠϝʔδʹؚΊΔ͔൱͔• ؚΊͳ͍ͷͰ͋ΕԿॲʹஔ͢Δ͔• ϙʔλϏϦςΟੑͱϩʔυ࣌ؒͷτϨʔυΦϑ• ྑ͍ΞΠσΟΞ͕͋Εڭ͑ͯԼ͍͞…
௨ৗͷAPIͱҧ͏• ѻ͏ϦιʔεɺModelαΠζ͕େ͖͘ͳΔ߹͕ଟ͍(ඦMBʙGB)• CPUɾϝϞϦϦιʔεͷফඅ͕ܹ͍͠• ߹ʹΑͬͯGPU͏
ϝϞϦফඅ• LovemachineͷPython࣮෦࣮ߦ࣌ʹ2GBϝϞϦΛফඅ͢Δˠࠓޙ͞Βʹ૿͑Δ༧ఆ͋Δ• Scikit-learnͰهड़͞ΕͨTF-IDFͷલॲཧ෦͕େ͖͘ͳΔࣄ͕ଟ͍
Pythonͱฒྻੑ• વThread͕͑ͳ͍(GILͷͨΊ)• ϓϩηεຖʹModelΛϩʔυ͢ΔͱඞཁͳϝϞϦαΠζ͕େ͖͘ͳΔˠ Blue-GreenDeployͷোʹͳΔ
ਖ਼PythonͰͷServing Πϯϑϥతʹਏ͍ࣄ͕ଟ͍…
ϝϞϦΛݡ͘͏• fork͢ΔલʹmodelΛϩʔυ͠Copy on WriteΛޮ͔͢• k8sͷone process per containerηΦϦ͋͑ͯഁ͍ͬͯΔ
Copy On Writeͷ෮शϝϞϦϓϩηε ࢠϓϩηε2.fork1BHF"1.allocation ಉ͡ྖҬΛࢀর
ϓϩηε͕ϝϞϦͷ༰Λ ॻ͖͑Δͱ…ϝϞϦϓϩηε ࢠϓϩηε1BHF" 1BHF#OS͕ผͷྖҬΛAllocationͯ͠ݩσʔλΛίϐʔ͢ΔผͷྖҬΛࢀর
Current Issues• Mercari APIͱͷͭͳ͗ࠐΈʹۤ࿑ ˠ Ұ௨Γ࡞Εޙ࠶ར༻Ͱ͖Δͣ• ਓؒͷߦಈΛ૬खʹ͍ͯ͠Δҝɺσʔλͷ͕มΘΓ͔ͬͨ͢Γɺ༧֎ͷ͕ൃੜͨ͠Γͯ͠ɺରԠ͠ଓ͚Δඞཁ͕͋Δ ˠ ML Model࡞ऀʹෛ୲ֻ͕͔Γଓ͚Δ ˠ SREͱͯࣗ͠ಈԽΛؚΜͩΈͰղܾ͍ͨ͠
Future Plans• ࣾͷσʔλ͔ΒಛྔΛநग़͢Δͯ͠Embedding͢Δ൚༻ͷΈ ˠదͳྨثͱΈ߹ΘͤΕɺ୭Ͱͦͦ͜͜ͷྨϞσϧΛ࡞Ͱ͖Δ? →FBLearner Flowతͳͭ?• ࣾͷղܾʹಛԽͨ͠ઐ༻ͷAutoMLతͳԿ͔?
·ͱΊ• ML ModelͷServingʹɺগ͠௨ৗͱҧ͏Πϯϑϥ͕ඞཁʹͳΔ →·ͩϕετɾϓϥΫςΟε͔Βͳ͍• ਓͷߦಈΛ૬खʹ͢Δͷେม• ͦͦMLͳػೳΛຊ֨ӡ༻͠Α͏ͱ͢Δͱɺେ෯ͳࣗಈԽɾΈԽΛਐΊͳ͍ͱ্ख͘ߦ͔ͳ͍
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠!!
We are Hiring!!
SRE ML Reliability• SysML? MLOps? ৽͍͠Job description• SREεΩϧ+MLͷجૅࣝ• MLΠϯϑϥͷࣗಈԽɾΈԽΛਪ͠ਐΊͯ͘ΕΔਓࡐ• ͪΖΜଞͷ৬छઈࢍืूத!!
ৄࡉͪ͜Β https://careers.mercari.com/