Share
Embed
フリマアプリ「メルカリ」AIチームの サービスライフサイクルと、 メルカリにおけるフルサイクルエンジニア / bitvalley2020-devdays-shopetan
BIT VALLEY 2020 9/10(木) Developers Days
Track 2 14:55-15:15 @ss_shopetan
Shopetan
September 10, 2020
Transcript
64 slides
-
1 ϑϦϚΞϓϦʮϝϧΧϦʯAIνʔϜͷ αʔϏεϥΠϑαΠΫϧͱɺ ϝϧΧϦʹ͓͚ΔϑϧαΠΫϧΤϯδχΞ BIT VALLEY 2020 9/10() Developers Days
Track 2 14:55-15:15 ٠ ฏ (Twitter: @ss_shopetan) -
2 ࣗݾհ !TIPQFUBO !TT@TIPQFUBO .FSDBSJ1FSTPOBMJ[BUJPO$BNQ3FDPNNFOEBUJPO5FBN &YQFSJFODF *OUFSOTIJQ.-&OHJOFFS
.FSDBSJ 1SFTFOU4PGUXBSF&OHJOFFS .FSDBSJ -JLF &TQPSUT8PSLPVU,BSBPLF$PODFSU -
3 ͜ͷϓϨθϯͰͷΰʔϧ Non-Goals Goals ಛఆͷٕज़తͳղܾͱͦ ͷΞϓϩʔνΛΔ ಇ͖ํʹؔ͢Δ ྑ͠ѱ͠Λٞ͢Δ͜ͱ ਪનυϝΠϯΛ௨ͨ͡ ͓٬༷ͷαʔϏε
ϥΠϑαΠΫϧΛΔ ϑϧαΠΫϧΤϯδχΞʹ ͍ͭͯΔ -
5 5 ςΫϊϩδʔͷྗͰՁަͷ͋ΓํΛม͑ΔαʔϏε ͜Ε·Ͱ େྔੜ࢈ɾେྔফඅ ୭͔ʹͱͬͯඞཁͳͷͰ ࣺͯΒΕͯ͠·͏ ඞཁͳͷ͕ඞཁͳਓʹ ඞཁͳ͚ͩಧ͘ ͜Ε͔Β
ͷ ! ςΫϊϩδʔʹΑͬͯͰ͖Δ҆৺ɾ҆શ ◦ ΤεΫϩʔܾࡁ ◦ AIٕज़Λۦͨ͠ҧݕγεςϜ ! ʮചΔ͜ͱΛۭؾʹ͢ΔʯUXͱςΫϊϩδʔ ◦ όʔίʔυग़ ◦ AIը૾ೝࣝ ϝϧΧϦͱ͍͏CtoCϚʔέοτϓϨΠε -
10 ϝϧΧϦͷAIͰղܾ͖ͯͨ͠ྖҬʮग़ʯ Πϯετʔϧ/ ձһొ ͢Δ ചΓͨ͘ͳΔ ௐࠪ͢Δ [͚ͳͲ] ങ͍ͨ͘ͳΔ ݕࡧ/ൺֱ͢Δ
ग़࡞ۀ͢Δ ަব/ߪೖ͢Δ Attention Search/ Compare Interest Action ૹ࡞ۀ͢Δ डऔධՁ͢Δ Ship/ Evaluate ɾAIग़ ɾΧλϩάϚοϐϯά ɾՁ֨αδΣετ ɾݕࡧΞϧΰϦζϜ ɾಉٛޠ֦ு ɾࣸਅݕࡧػೳ ɾҧग़ͷࢹ ɾҟৗߦಈͷݕ ɾૹαδΣετ ɾॏྔਪఆ -
12 ͓٬͞·ͷʮಈػ͚ͮʯΛࢧԉ͢Δ Πϯετʔϧ/ ձһొ ͢Δ ചΓͨ͘ͳΔ ௐࠪ͢Δ [͚ͳͲ] ങ͍ͨ͘ͳΔ ݕࡧ/ൺֱ͢Δ
ग़࡞ۀ͢Δ ަব/ߪೖ͢Δ Attention Search/ Compare Interest Action ૹ࡞ۀ͢Δ डऔධՁ͢Δ Ship/ Evaluate τϨϯυ / ϥϯΩϯά ͷਪન ͓٬༷ͷߦಈʹԠͨ͡ ਪન ϨίϝϯυΛ௨ͯ͡ ʮಈػ͚ʯΛڧԽ͢Δ -
13 Ϩίϝϯυ͕ͨΒ͢٬؍తͳΠϯύΫτ Alibaba͕ӡӦ͢ΔC2CαʔϏεຊ ԁͰ2400ԯԁنͷऩӹ ͦͷ͏ͪϨίϝϯσʔγϣϯػೳ͕ऩӹ ͱτϥϑΟοΫͷେ෦ΛΊ͍ͯΔ ϝϧΧϦͷϨίϝϯσʔγϣϯػೳશ ମτϥϑΟοΫͷ͏ͪ10-20%ڧɹɹ Big ImpactΛͨΒ͢ྖҬ
#JMMJPOTDBMF$PNNPEJUZ&NCFEEJOHGPS&DPNNFSDF3FDPNNFOEBUJPOJO"MJCBCB -
14 ϨίϝϯσʔγϣϯʹऔΓΉ·Ͱͷ γεςϜత ৫త ϞϊϦεͳ ΞϓϦέʔγϣϯͰӡ༻ վળίετ͕ߴ͍ Β͘ϝϯςφϯε ͞Ε͍ͯͳ͍ ϨίϝϯυΛઐͱ͠
Productૌٻ͢Δ৫ମ੍ ͕ແ͔ͬͨ AIνʔϜʹ ڠۀମ੍͕͔ͬͨ͠ -
20 ։ൃνʔϜͷMicroserviceԽͱ NJDSPTFSWJDFTQMBUGPSNBUNFSDBSJ Test Deploy Operate Backend Team QA team
SRE team Develop Service B Develop Service A Develop Service C -
21 ։ൃνʔϜͷMicroserviceԽͱ NJDSPTFSWJDFTQMBUGPSNBUNFSDBSJ Test Deploy Operate Backend Team QA team
SRE team Develop Service B Develop Service A Develop Service C -
22 ։ൃνʔϜͷMicroserviceԽͱ NJDSPTFSWJDFTQMBUGPSNBUNFSDBSJ Test Deploy Operate Backend Team QA team
SRE team Develop Service B Develop Service A Develop Service C -
23 ։ൃνʔϜͷMicroserviceԽͱ NJDSPTFSWJDFTQMBUGPSNBUNFSDBSJ Develop Service B Test Deploy Operate Service
B Team Develop Service A Test Deploy Operate Service A Team Develop Service C Test Deploy Operate Service C Team -
24 ։ൃνʔϜͷMicroserviceԽͱ NJDSPTFSWJDFTQMBUGPSNBUNFSDBSJ Develop Service B Test Deploy Operate Service
B Team Develop Service A Test Deploy Operate Service A Team Develop Service C Test Deploy Operate Service C Team ։ൃ͔Βӡ༻·ͰϑϧαΠΫϧʹಇ͘ -
27 AIνʔϜͰͷαʔϏεϦϦʔε·ͰͷϥΠϑαΠΫϧ Test Deploy Operate ML Engineer MLOps team Develop
Model B Develop Model A Develop Model C -
28 AIνʔϜ֤υϝΠϯ͝ͱʹνʔϜׂͨ͠ Develop Recommend Platform Test Deploy Operate Recommendation Team
Develop Image search Test Deploy Operate Image search Team ML Engineer MLOps Team Search Team -
33 ϨίϝϯσʔγϣϯνʔϜͷαʔϏεϥΠϑαΠΫϧ Develop Recommend Platform Test Deploy Operate Recommendation Team
Act as ML, BI, PM Survey Analyze Business Impact Modeling Experiment Design Develop Platform -
34 ϨίϝϯσʔγϣϯνʔϜͷαʔϏεϥΠϑαΠΫϧ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform ։ൃνʔϜ͕ SWE/SET/SRE/ML/BI/PM ΛϑϧαΠΫϧʹ୲͢Δ -
37 MicroserviceҠߦͯ͠ಘΒΕͨ͜ͱ γεςϜత ৫త Scalability Availability ‐‐‐ Complexity վળ Velocity,
Agility‐ ਅʹ࣮ݱ͍ͨ͜͠ͱ ʮΤϯδχΞϦϯάΛ௨͓ͨ͡٬༷ͷܧଓతͳՁఏڙʯ -
43 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Microservice Migrate Done -
45 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Scalability, Availability ⬆ -
46 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Scalability, Availability ⬆ ͜Εࣗମຊ࣭Ͱͳ͍ -
47 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform -
48 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform -
49 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform νʔϜ͕End-to-EndͰResponsibilityΛ࣋ͯΔ -
50 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform E2EͰResponsibilityΛ࣋ͯΔࣄͰ ϥΠϑαΠΫϧશମͷVelocity্͕͕Δ -
51 αʔϏεϥΠϑαΠΫϧʹΦʔφʔγοϓΛ࣋ͯΔͱ Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform ʮΤϯδχΞϦϯάΛ௨͓ͨ͡٬༷ͷܧଓతͳՁఏڙʯ Λߴʹ࣮ݱ͠ଓ͚Δ͜ͱ͕ՄೳʹͳΔ -
52 ͱ͍͑… Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform -
53 ͱ͍͑… Develop Recommend Platform Test Deploy Operate Recommendation Team
Survey Analyze Business Impact Modeling Experiment Design Develop Platform ։ൃνʔϜ͕ SWE/SET/SRE/ML/BI/PM ΛϑϧαΠΫϧʹ୲͢Δ 4VQFS)BSEͰʜ -
55 ϨίϝϯσʔγϣϯνʔϜͱͯ࣌͠ͷTL͕ܾΊͨ͜ͱ ! AI͔Βͱ͍͏ઢΛΊΔ ◦ MLΤϯδχΞͰ͋ΔҎલʹιϑτΣΞΤϯδχΞͰ͋Δ ◦ ͋͘·ͰखஈͳͷͰඞཁͳλΠϛϯάͰར༻͢Δ ◦ අ༻ରޮՌɼ൚༻ੑΛҙࣝ͠ΠϯύΫτͷ͋Δ෦͔ΒMVPʹ࣮ݱ
◦ Decide Everything Scientifically ! طଘͷׂͷڥքΛͳͨ͘͠νʔϜΛ࡞Δ ◦ ݩͷॴଐҧ͑Ͳతಉ͡ ◦ ڥքෆ৹ΛੜΉ ◦ ઓ͍ͨ͠ਓ͕ઓ͍ͨ͠ྖҬʹऔΓΊΔνʔϜʹ͢Δ ◦ ಘҙྖҬͱνϟϨϯδͷόϥϯεΛอͪͳ͕Β։ൃͨ͠ -
56 ͱ͍͑ࣗʹೳྗ͕͋ͬͨͷ͔? ! ݩʑMLΤϯδχΞͱͯ͠Πϯλʔϯ -> ೖࣾ ◦ SWEɼSETɼSREͱͯ͠ͷҰൠతͳڭཆ͢Βͳ͔ͬͨ ◦ MicroserviceͷMigrationવ͕ͩೖࣾޙॳΊͯܞΘͬͨ
! ϝϧΧϦͷจԽͱνʔϜͷํ͕ࣗΛٹͬͯ͘Εͨ ◦ GoBoldʹະܦݧͷྖҬʹઓͯ͠ྑ͍ڥͩͬͨ ◦ ͓٬༷ͷՁఏڙΛୈҰʹߟ͑ͯɼ Ұ࿈ͷαʔϏεϥΠϑαΠΫϧΛҰਓͰ୲Ͱ͖ΔΑ͏ʹͳͬͨ -
57 ΦʔφʔγοϓΛ࣋ͯͯΑ͔ͬͨ͜ͱ(Team) ! Ϩίϝϯσʔγϣϯʹؔ͢ΔશͯͷυϝΠϯΛվળ͢Δͱݖݶ͕͋Δ ◦ طଘͷϨίϝϯσʔγϣϯʹؔ࿈͢ΔશͯͷAPI͕ ϨίϝϯσʔγϣϯνʔϜͷϓϥοτϑΥʔϜͱܨ͕͍ͬͯΔ ◦ ܧଓతʹ࣮ݧͱվળΛ܁ΓฦͤΔڥ !
ఏى͔ΒϦϦʔε·ͰͷվળαΠΫϧʹ֎෦ґଘ͕গͳ͍ ◦ ϨίϝϯσʔγϣϯνʔϜͰطʹӡ༻͍ͯ͠ΔAPIܭը͔Β ϦϦʔεஅ·ͰνʔϜ෦ͷΈͰ݁͢Δ ◦ ৽نͷAPIଟͯ͘1-2νʔϜͱͷௐ͚ͩͰຊ൪ϦϦʔε͕Ͱ͖Δ -
58 ΦʔφʔγοϓΛ࣋ͯͯΑ͔ͬͨ͜ͱ(ݸਓ) ! MicroserviceԽ/Platform։ൃʹΑΔSWEͱͯ͠ͷٕज़ྖҬͷ֦େ ◦ Manage Elasticsearch on GKE ◦
Develop middleware ! PlatformΛӡ༻͢Δ͜ͱʹΑΔSREͱͯ͠ͷҙࣝ ◦ Monitor any metrics ◦ Consider SLI/SLO for Recommendation ◦ Loadtest with Real-Scenario ◦ Consider Cluster’s constitution ◦ Apply CI/CD ◦ Infrastructure as Code ◦ υϝΠϯશମͰͷCostཧ -
59 ϑϧαΠΫϧʹಇ͚ΔΑ͏ʹͳ͕ͬͨ… ! Q. ࣮ࡍʹϑϧαΠΫϧͳϑϩʔͰಇ͍͍ͯΔ? ◦ A. Yes! ◦ ϦϦʔεӡ༻·ͰҰ؏ͯ͠୲͢Δ͜ͱ͕Մೳ
! Q. νʔϜͰϑϧαΠΫϧҎ֎ͷಇ͖ํ͋Δ? ◦ A. Yes! ◦ ࣗࣗɼڵຯͷ͋ΔྖҬ͕αΠΫϧࣗମͷվળʹ͋Δ ▪ Πϯελϯεͷࣗಈఀࢭ/GKE NodeࣗಈఀࢭͳͲͷίετΧοτ ▪ K8s্Ͱͷscalable, ࠶ݱੑͷߴ͍Loadtest clientͷ࣮ݱ ▪ ElasticsearchΛ͡Ίͱͨ͠ϓϥοτϑΥʔϜͷࢹػߏͷՃ ▪ ͜ΕΒͷIaCରԠͳͲ -
60 αʔϏεϥΠϑαΠΫϧΛࢧԉ͢Δ ྫ͑ : AI Platform(πʔϧ) - ModelཧPipelineΛࢧԉ - ABςετࢧԉ
ྫ͑: Embedded SRE(ׂ) - υϝΠϯʹಛԽͨ͠SRE ϥΠϑαΠΫϧʹؔΘΔ༷ʑͳ ΛղܾΛ͢Δ͜ͱ͕ࣄ -
61 ఆ͑͠Δ࣭ ! MLͷSpecialistͱͯ͠ࣄͯͨ͠ͷʹશͯͷϑΣʔζΛΒͳ͍ͱμϝ? ◦ No. ͦΜͳ͜ͱͳ͍ ◦ ֤ʑʹಘҙͳྖҬ͕͋ΔͷવͳͷͰํνʔϜ͝ͱʹҙࢥܾఆ ◦
νʔϜશମͰe2eʹResponsibilityΛ࣋ͯΔ͜ͱʹՁ͕͋Δ ! ClientFrontendͳͲͷྖҬ͕ඞཁͳ࣌Ͳ͏͢Δͷ? ◦ ϝϦΧϦͰCampͱ͍͏୯ҐͰ։ൃΛ͍ͯ͠Δ ▪ ؔ࿈͢ΔྖҬʹΛ࣋ͭνʔϜ͕ू߹͍ͯ͠Δ ! Include all tech stack ▪ Q͝ͱͷඪΛCampͱ͍͏୯ҐͰڞ༗&CampͰεΫϥϜ ! ༏ઌΛἧ͑ͯ։ൃ͢ΔͷͰϦϦʔεεϐʔυ͕อূ ͓ؾ͖ͮͰ͔͢ʁύʔιφϥΠθʔγϣϯ͔Βੜ·ΕͨϝϧΧϦػೳվળ·ͱΊ -
62 ·ͱΊ ! ਪનυϝΠϯͷαʔϏεϥΠϑαΠΫϧΛհ ◦ جຊతʹMLʹؔ͢ΔProductͷ্ཱͪ͛ͱมΘΒͳ͍ ◦ MicroserviceԽͷརe2eʹΦʔφʔγοϓΛ࣋ͯΔࣄ ▪ νʔϜͰఏى͔Βҙࢥܾఆ·Ͱ݁͢Δ
▪ AIྖҬʹ͓͍ͯҰ࿈ͷTry&ErrorͷߴԽʹՁ͕͋Δ ! ϑϧαΠΫϧΤϯδχΞͱͯ͠ಇ͍͍ͯΔνʔϜΛհ ◦ ֤ΤϯδχΞ͕શͯͷྖҬΛۃΊͳ͍ͱ͍͚ͳ͍༁Ͱͳ͍ ◦ GoBoldʹઓ͍ͨ͠ਓ͕ઓ͍ͨ͠ྖҬʹऔΓΊΔΑ͏ʹܾΊͨ ◦ αΠΫϧࣗମΛվળ͢Δ͜ͱʹՁ͕͋Δ ◦ ϑϧαΠΫϧʹϝϧΧϦͷϨίϝϯυΛΑ͍͖ͯ͘͠·͢ -
63 #BDLFOE J04 "OESPJE 'SPOUFOE 4JUF3FMJBCJMJUZ .BDIJOF-FBSOJOH ৽ଔ࠾༻ ৽ଔΤϯδχΞืूϙδγϣϯ
.JDSPTFSWJDFT 1MBUGPSN *ΠϯλʔϯγοϓͷΈืू -
64 .BDIJOF-FBSOJOH 4JUF3FMJBCJMJUZ .JDSPTFSWJDFT 1MBUGPSN 8FC1MBUGPSN 4JUF3FMJBCJMJUZ &.HS %JHJUBM.BSLFUJOH
4QFDJBMJTU த్࠾༻ த్࠾༻ืूϙδγϣϯ *5#13 .HS 4JUF3FMJBCJMJUZ &.HS