料理きろくと機械学習

by uiureo

Slide 1

Slide 1 text

ྉཧ͖Ζ͘ͱػցֶश ਿຊ෩ే 2017/09/28, 29

Slide 2

Slide 2 text

ਿຊ෩ే • Kazato Sugimoto • @uiureo • αʔϏε։ൃ෦ΤϯδχΞ

Slide 3

Slide 3 text

ΫοΫύουͰͷ࢓ࣄ • ػցֶशΛ׆༻ͨ͠αʔϏε։ൃ • ྉཧ͖Ζ͘, Ϩίϝϯσʔγϣϯ • ػցֶशϞσϧҎ֎ͷ։ൃΛओʹ୲౰ • Ruby, Python, Redshift, Android

Slide 4

Slide 4 text

࿩͢͜ͱ • ػցֶशΛαʔϏεͰ׆༻͢ΔͨΊʹͲ͏͢Δ͔ • αʔϏε։ൃΛࢧ͑Δٕज़ج൫ͷ঺հ • ֶश → ਪଌ • ֶशϑΣʔζͰͳ͘ɺਪଌϑΣʔζΛத৺ʹ

Slide 5

Slide 5 text

๻͕ػցֶशʹؔΘΔཧ༝

Slide 6

Slide 6 text

ศརͳ΋ͷΛ࡞Γ͍ͨ • લ৬Ͱ͸UI/UXΤϯδχΞ • Gyazo / Gyazo GIF • ϧʔϧϕʔεͷΞϓϩʔνʹݶքΛײ͡Δ • Google PhotosΛݟͯিܸ

Slide 7

Slide 7 text

ศར͞ͷ૿Ճ෼ (ࠓޙ3~5೥) ػցֶश > ώϡʔϦεςΟΫε

Slide 8

Slide 8 text

UI͸ಁ໌Խ͍ͯ͘͠ • ࣗಈԽʹΑͬͯೖྗUIͷଘࡏ͕رബʹͳ͍ͬͯ͘ • ྫ: Ϩίϝϯσʔγϣϯ, ࣗಈӡస netﬂix.com ϗʔϜը໘ ࣗಈӡసं

Slide 9

Slide 9 text

ྉཧ͖Ζ͘ • ྉཧͷࣸਅΛࣗಈͰه࿥ • ྉཧΛָ͘͠ৼΓฦΔ • ྉཧࣸਅྦྷܭ 1000ສຕ௒  (2017/09/26 ࣌఺)

Slide 10

Slide 10 text

ྉཧ͖Ζ͘Ͱ͸ ػցֶशΛ ੵۃతʹ׆༻͍ͯ͠·͢

Slide 11

Slide 11 text

ྉཧ൑ผ ࣸਅ͕ྉཧ͔Ͳ͏͔Λ൑ఆ͢Δ ྉཧ͖Ζ͘ʹ͓͚Δྉཧ/ඇྉཧ൑ผϞσϧͷৄࡉ  http://techlife.cookpad.com/entry/2017/09/14/161756

Slide 12

Slide 12 text

࡞ͬͨϨγϐ൑ఆ Ӿཡͨ͠ϨγϐΛ࣮ࡍʹௐཧ͔ͨ͠ਪଌ͢Δ

Slide 13

Slide 13 text

࡞ͬͨϨγϐΛࣗಈͰه࿥

Slide 14

Slide 14 text

Ϩγϐબ୒ը໘ͰͷαδΣετ

Slide 15

Slide 15 text

ັྗ౓ͷਪఆ ྉཧը૾ͷັྗ౓ (1~5఺) Λਪఆ͢Δ ※ ਿຊ࡞ ໺ࡊᖱΊ

Slide 16

Slide 16 text

ັྗ౓ͷਪఆ: push௨஌Ͱͷ׆༻

Slide 17

Slide 17 text

ྉཧࣸਅͷϝχϡʔ෼ྨ ྉཧը૾Λ50ΧςΰϦʹ෼ྨ ʮύελʯ ʮΦ Ϝ ϥΠεʯ

Slide 18

Slide 18 text

ϝχϡʔΛ࣠ʹৼΓฦΔػೳ (։ൃத)

Slide 19

Slide 19 text

ػցֶश → ݱ࣮ੈք Ͱͷ՝୊ • ܭࢉίετͷߴ͞ • ಛʹσΟʔϓϥʔχϯά • ܭࢉ؀ڥͷࠩ • Ruby, CPU <-> Python, GPU • ͲͷΑ͏ʹσϓϩΠ͢Δ͔ʁ

Slide 20

Slide 20 text

ػցֶशϞσϧͷσϓϩΠ • ࠓճ͸ϦΞϧλΠϜੑ͕ඞཁͳ͍৔߹ͷ࿩Λ͠·͢ • ྉཧ൑ఆͰͷΞʔΩςΫνϟʹ͍ͭͯ͸: ΫοΫύουͷػցֶशΛࢧ͑Δج൫ͷͭ͘Γ͔ͨ https://speakerdeck.com/ayemos/machine-learning-ops-at-cookpad-1

Slide 21

Slide 21 text

ػցֶशόονͷΞʔΩςΫνϟ S3 S3 ECS MySQL Kuroko2 Redshift Rails AWSΛྲྀΕΔσʔλͷؾ࣋ͪʹͳΓ·͠ΐ͏

Slide 22

Slide 22 text

DB ߦಈϩά ػցֶशόονͷΞʔΩςΫνϟ AWSΛྲྀΕΔσʔλͷؾ࣋ͪʹͳΓ·͠ΐ͏ S3 ECS Redshift

Slide 23

Slide 23 text

ػցֶशόονͷΞʔΩςΫνϟ S3 S3 ECS MySQL Kuroko2 Redshift Rails AWSΛྲྀΕΔσʔλͷؾ࣋ͪʹͳΓ·͠ΐ͏

Slide 24

Slide 24 text

S3 MySQL Rails ػցֶशόονͷΞʔΩςΫνϟ AWSΛྲྀΕΔσʔλͷؾ࣋ͪʹͳΓ·͠ΐ͏

Slide 25

Slide 25 text

Redshift • σʔλ෼ੳج൫ • ࣾ಺ͷ͢΂ͯͷϩά΍σʔλ͕ू໿ • SQLͰૉૣ͘σʔλ෼ੳͰ͖Δ

Slide 26

Slide 26 text

Kuroko2 • δϣϒ؅ཧγεςϜ • δϣϒͷϑϩʔΛ੍ޚ͍ͯ͠Δ • Web UIܦ༝ͰδϣϒΛ࣮ߦ͢Δ ΫοΫύουͷδϣϒ؅ཧγεςϜ kuroko2 ͷ঺հ http://techlife.cookpad.com/entry/2015/12/07/195732

Slide 27

Slide 27 text

ECS • AWSͷίϯςφ࣮ߦ؀ڥ • hako: DockerͷσϓϩΠπʔϧ • ymlͰఆٛΛॻ͚Δ • ྫ: DockerﬁleͰఆٛͨ͠Python / Tensorﬂowͷ࣮ߦ؀ڥ Λg2.2xlargeͷΫϥελʹσϓϩΠ ECS Λར༻ͨ͠ΦϑϥΠϯδϣϒͷ࣮ߦ؀ڥ http://techlife.cookpad.com/entry/2016/09/09/235007

Slide 28

Slide 28 text

hakoͰܭࢉ؀ڥΛఆٛ

Slide 29

Slide 29 text

ػցֶशόονͷΞʔΩςΫνϟ S3 S3 ECS MySQL Kuroko2 Redshift Rails AWSΛྲྀΕΔσʔλͷؾ࣋ͪʹͳΓ·͠ΐ͏

Slide 30

Slide 30 text

Kuroko2Ͱͷσʔλϑϩʔ੍ޚ

Slide 31

Slide 31 text

Kuroko2Ͱͷδϣϒ࣮ߦ

Slide 32

Slide 32 text

Kuroko2Ͱͷδϣϒ࣮ߦ ఆظ࣮ߦ

Slide 33

Slide 33 text

Kuroko2Ͱͷδϣϒ࣮ߦ ࣮ߦ݁Ռ ఆظ࣮ߦ

Slide 34

Slide 34 text

࡞ͬͨϨγϐ൑ఆͷσϓϩΠ • RedshiftͰಛ௃நग़ • ಛ௃: Ӿཡ࣌ؒ, ϒοΫϚʔΫ͔ͨ͠ͳͲ • xgboostͰͷϞσϧͰʮ࡞ͬͨʯʮ࡞ͬͯͳ͍ʯΛ൑ఆ

Slide 35

Slide 35 text

࡞ͬͨϨγϐ൑ఆͷσϓϩΠ RedshiftͰϩά͔Βಛ௃நग़ time user_id recipe_id 17:00 1 123 18:00 1 234 17:01 2 345 17:02 2 456 ϨγϐӾཡϩά (ΫιͰ͔͍) user_id recipe_id ྦྷܭӾཡ ࣌ؒ 1 123 10෼ 1 234 1෼ 2 345 5ඵ 2 456 20෼ ಛ௃

Slide 36

Slide 36 text

S3 S3 ECS Redshift ࡞ͬͨ ಛ௃நग़ Ϟσϧ൑ఆ xgboost ࡞ͬͯͳ͍ ࡞ͬͨ ࡞ͬͯͳ͍ 10෼ 1෼ 5ඵ 20෼ ࡞ͬͨϨγϐ൑ఆͷσϓϩΠ

Slide 37

Slide 37 text

ػցֶशόονͰ͸ڞ௨͢Δॲཧ͕ଟ͍ S3 S3 ECS unload python import python python python tsvΛdownload → ܭࢉ → tsvΛupload

Slide 38

Slide 38 text

ml-batch • ػցֶशόονͷڞ௨ॲཧΛ؆୯Խ͢ΔpythonϥΠϒϥ Ϧ • ฒྻԽΛ͍͍ײ͡ʹ • ϩΪϯάΛ͍͍ײ͡ʹ • ࣮ߦ࣌ؒͷ౷ܭ৘ใͷग़ྗͳͲ

Slide 39

Slide 39 text

ml-batch

Slide 40

Slide 40 text

ϝχϡʔ෼ྨͰͷ՝୊ • ܭࢉίετ͕ߴ͍ • ը૾ 1݅ 200ms (GPUͰͷ࠷దԽޙ) • ϞσϧΛߋ৽͢Δͨͼʹɺ໿1000ສ݅ʹλά෇͚ • Πϯελϯε1୆Ͱ ໿20೔ • ੒௕͠ଓ͚ΔͷͰɺը૾਺͸૿͑Δ

Slide 41

Slide 41 text

ϝχϡʔ෼ྨͰͷ޻෉ • σʔλΛฒྻʹѻ͑Δ • 40୆Ͱ࣮ߦ͢Ε͹ ໿0.5೔ Ͱ׬ྃ͢Δ

Slide 42

Slide 42 text

ฒྻ࣮ߦʹΑΔߴ଎Խ S3 S3 ͨ͘͞Μ૿΍͢ unload python import python python python

Slide 43

Slide 43 text

Kuroko2Ͱͷฒྻ࣮ߦ 40ίฒྻʹhakoͰδϣϒΛىಈ

Slide 44

Slide 44 text

ฒྻ࣮ߦͷ࢓૊Έ • Redshift͔Βunload͢Δͱ෼ׂͯ͠S3ʹೖྗΛஔ͘ • Kuroko2 + hako ͸NݸδϣϒΛσϓϩΠ • ฒྻ౓ͱԿ൪໨͔ͷ؀ڥม਺Ληοτ͢Δ • ml-batch͸؀ڥม਺Λݟͯ୲౰෼ͷೖྗΛॲཧ

Slide 45

Slide 45 text

·ͱΊ • ྉཧ͖Ζ͘Ͱ͸ػցֶशΛੵۃతʹ׆༻͍ͯ͠Δ • ΫοΫύουͰͷٕज़ج൫Λϑϧ׆༻࣮ͯ͠ݱ͍ͯ͠Δ