Tweet
Tweet

Slide 1

Slide 1 text

Multi-Cloud Architecture Supporting Machine Learning and Data Analysis ػցֶशͱσʔλ෼ੳΛࢧ͑ΔϚϧνΫϥ΢υΞʔΩςΫνϟͷ঺հ VASILY,Inc. @kyuns 2016.12.9 / Minami Aoyama Night #1

Slide 2

Slide 2 text

ࠓଜխ޾ @kyuns / Ωϡϯ VASILY,Inc औక໾CTO / Co-Founder ೥:BIPP+"1"/ʹೖࣾɺ:BIPP'"4)*0/΍ 9#3"/%ͳͲͷαʔϏεͷ্ཱͪ͛ͷ։ൃΛ୲౰ɻ ϦίϝϯσʔγϣϯͷಛڐͳͲΛऔಘޙɺ ೥ʹಠཱɺ7"4*-:Λ૑ۀɻऔక໾$50ʹब೚

Slide 3

Slide 3 text

ઃཱ೥݄ ॴࡏ஍౦ژ౎඼઒۠੢ޒ൓ాδχΞεϏϧ' ैۀһ໊ ࢿຊۚԯ
 ୅දऔక໾ۚࢁ༟थ
 औక໾ࠓଜխ޾ઍ༿େี 4ZOάϧʔϓ
 ϑΝογϣϯʷςΫϊϩδʔ ΛڧΈͱͨ͠ձࣾ

Slide 4

Slide 4 text

೔ຊதͷϑΝογϣϯ ECαΠτͷ঎඼σʔλΛܝࡌ ೔ຊ࠷େڃͷঁͷࢠͷͨΊͷ ϑΝογϣϯΞϓϦʮΞΠίϯʯ ձһ਺ 250ສਓ Google Play ϕετɾΞϓϦ 300Ҏ্ͷ&$αΠτ͔Β ສ఺ͷΞΠςϜܝࡌ

Slide 5

Slide 5 text

೥࿈ଓϕετΞϓϦड৆ (PPHMF1MBZϕετΠϊϕʔςΟϒΞϓϦେ৆ड৆

Slide 6

Slide 6 text

ػցֶशͱσʔλ෼ੳΛࢧ͑Δ ϚϧνΫϥ΢υΞʔΩςΫνϟʔͷ঺հ ຊ೔ͷΞδΣϯμ ֤Ϋϥ΢υͷಛੑΛ׆͔ͯ͠
 ༻్ผʹ࠷దͳ΋ͷΛར༻͢Δ

Slide 7

Slide 7 text

ฐࣾͷσʔλΛ׆༻ͨ͠࠷ۙͷऔΓ૊Έ

Slide 8

Slide 8 text

1.σʔλ෼ੳ ΞϓϦ಺ߦಈϩά΍Ϣʔβʔͷσʔλͷ෼ੳ Λ௒͑Δ7JFXͰͦΕͧΕͷࢪࡦͷ,1*Λ֬ೝ

Slide 9

Slide 9 text

2.ػցֶश ສਓ͇ສ఺ͷΞΠςϜ͔Β Ϣʔβʔͷ޷ΈΛֶशͯ͠ΞΠςϜΛਪન͢ΔϨίϝϯυΤϯδϯ

Slide 10

Slide 10 text

2.ػցֶशʢσΟʔϓϥʔχϯάʣ ը૾͔ΒΞΠςϜݕग़ྨࣅը૾ݕࡧ IUUQUFDIWBTJMZKQFOUSZEFUFDUJPO@BOE@SFUSJFWBM

Slide 11

Slide 11 text

2.ػցֶशʢσΟʔϓϥʔχϯάʣ 7"&ͱ("/Λར༻ͨ͠ϑΝογϣϯΞΠςϜݕࡧ *#*4 IUUQUFDIWBTJMZKQFOUSZSFUSJFWBM@VTJOH@WBF@HBO

Slide 12

Slide 12 text

ϚϧνΫϥ΢υ αʔϏεͷϝΠϯͱͳΔΠϯϑϥ σʔλ෼ੳج൫ͱͯ͠ͷΠϯϑϥ

Slide 13

Slide 13 text

σʔλ෼ੳ

Slide 14

Slide 14 text

RDS/Aurora Log Aggregater GoogleDataProc MongoDB GoogleDrive (SpreadSheet) API Server Web Server fluentd fluentd fluentd iQONͷϩάσʔλ·ΘΓͷΞʔΩςΫνϟ iOS/Android Puree AWS GCP BigQuery ΦϯϓϨ Tesla K Series / GTX1080

Slide 15

Slide 15 text

σʔλ෼ੳʹඞཁͳϩάऩू ΞϓϦ "OESPJE͸1VSFFͱJ04͸ಠࣗͷ4%,Λ௨ͯ͠ߦಈϩάΛऔಘ αʔόʔ Πϕϯτϩά͸શͯqVFOUEܦ༝Ͱ#JH2VFSZʹ4USFBNJOH*OTFSU 8FC΍"1*ͷ/HJOYͷϩά΋શͯ4USFBNJOH*OTFSU Ϛελσʔλ 3%4 .POHP%#ʹೖ͍ͬͯΔσʔλ΋͢΂ͯ#JH2VFSZʹίϐʔ IUUQTHJUIVCDPNDPPLQBEQVSFFBOESPJE

Slide 16

Slide 16 text

(PPHMF͕ఏڙ͢ΔϑϧϚωʔδυ%8)
 ѹ౗తʹ؆୯଎͍͍҆ J20/Ͱ͸ҎԼͷΑ͏ͳσʔλશͯΛอଘ w ΞϓϦ1$εϚϗͷߦಈϩά w Ϛελʔ%#σʔλ GSPN3%4 w Ϋϩʔϥʔͷશͯͷϩά w ϦίϝϯυΤϯδϯͷܭࢉ݁Ռ w σΟʔϓϥʔχϯά༻σʔλ BigQuery

Slide 17

Slide 17 text

೔ςʔϒϧ͝ͱʹશͯͷϩάΛอଘ IUUQUFDIWBTJMZKQFOUSZCJHRVFSZ@EBUB@QMBUGPSN ྫ:ΞϓϦߦಈϩά

Slide 18

Slide 18 text

࠷ۙͷBigQueryͷΞοϓσʔτ w 4UBOEBSE42-ରԠ w */4&3561%"5&%&-&5&΋Մೳʹ w Ұ࣌7JFX഑ྻߏ଄ମαϒΫΤϦͷॻ͚Δ৔ॴ ͕૿͑ͨɺͳͲ༷ʑͳػೳ͕ར༻Մೳʹͳͬͨ w ଞͷػೳมߋͳͲ͸ҎԼʹ·ͱΊͯ͋Γ·͢
 IUUQUFDIWBTJMZKQFOUSZCJHRVFSZTUBOEBSETRM

Slide 19

Slide 19 text

ؾʹͳΔྉۚ w ετϨʔδ(#
 ετϦʔϛϯάΠϯαʔτ(# w ΫΤϦྔ5#
 
 ஫ҙࣄ߲
 ೥݄͔Β͸ΫΤϦͷ಺༰ʹΑͬͯ஋ஈ͕มΘΔ
 NBYJNVN#JMMJOH5JFSΛઃఆਃ੥͢Δඞཁ͕͋Δ ࢀߟJ20/ͷ݄ؒར༻ঢ়گ ೥݄ 
 ෼ੳΫΤϦྔ5JC 5#

Slide 20

Slide 20 text

σʔλ෼ੳ͸Tableau #JH2VFSZ΍3%4΋σʔλιʔεͱͯ͠ϦΞϧλΠϜʹѻ͑Δ
 ΠϯλϥΫςΟϒͳ෼ੳ΍μογϡϘʔυͷ࡞੒͕Մೳ ࡉ͔͍σʔλ෼ੳ͸5BCMFBV%FTLUPQ ࣾ಺Ͱͷσʔλڞ༗ʹ͸5BCMFBV4FSWFS

Slide 21

Slide 21 text

μογϡϘʔυྫ:ΫϩʔϥʔՔಇ཰

Slide 22

Slide 22 text

෼ੳྫ: ΰδϥ(※λϒϩʔͷαϯϓϧͰ͢)

Slide 23

Slide 23 text

ཚཱ͢ΔBIπʔϧ ҰମͲΕΛ࢖͑͹͍͍ͷ͔ʜʁ

Slide 24

Slide 24 text

BIπʔϧൺֱ ྉۚ #JH2VFSZ ୳ࡧత෼ੳ ߴػೳ ར఺ ܽ఺ 5BCMFBV%FTLUPQ 4FSWFS ༗ྉ ˓ ˓ ˓ େ͖ͳσʔλ΋ಡΈ ࠐΊΔ ߴ͍ %BUB4UVEJP ແྉ ˓ ✕ ✕ ͍҆ ࡞ΕΔϘʔυ͸ͭ· Ͱ FYQMPSBUPSZ ༗ྉ ˚ ˓ ˓ LNFBOT47%͕6*Ͱ Ͱ͖Δ όά͕ଟ͍ɺେ͖͍ σʔλಡΈࠐΊͳ͍ SFEBTI ແྉ ˓ ✕ ˚ ͍҆ όά͕ଟ͍ 2VJDL4JHIU ༗ྉ ✕ ✕ ˚ "84ͱͷ਌࿨ੑ #2ະରԠ 4VQFSTFU ແྉ ✕ ✕ ˚ ͍҆ #2ະରԠ 1PXFS#* ༗ྉ ✕ ˓ ˓ .4ͱͷ਌࿨ੑ #2ະରԠ %PNP ༗ྉ ˓ ˓ ˓ ೔ຊͷେखۀͷಋೖ ࣄྫଟ਺ ߴ͍ ࡉ͔͍ൺֱ͸ޙ೔هࣄʹͯ঺հ༧ఆ

Slide 25

Slide 25 text

ݸਓతBIπʔϧબͼํͷίπ ࣾ಺ʹ#*ʹৄ͍͠ਓ͕͍Δ͓ۚΛଟগ͔͚ͯ΋͍͍ ˠ5BCMFBV%FTLUPQ5BCMFBV4FSWFS ˞σʔλ෼ੳ͸ࢪࡦΛ൑அ͢ΔͷʹඇৗʹॏཁͳͷͰɺπʔϧࣗମʹόά͕͋Δ ͜ͱࣗମΫϦςΟΧϧ ͱΓ͓͋͑ͣۚΛ͔͚ͣʹ͍͍ײ͡ʹϨϙʔτΛΈ͍ͨ
 ˠ3FEBTIPS(PPHMF%BUB4UVEJP

Slide 26

Slide 26 text

όονॲཧ "JS'MPX "JSCOC͕ࣾ։ൃͨ͠όονॲཧ࣮ߦϑϨʔϜϫʔΫ ෳ਺ͷδϣϒͷґଘؔ܎ͷղܾ΍ϦτϥΠॲཧͳͲΛ͏·͘΍ͬͯ͘ΕΔ https://github.com/airbnb/airflow -VJHJ 4QPUJGZ͕ࣾ։ൃͨ͠όονॲཧ࣮ߦϑϨʔϜϫʔΫ "JSqPXΑΓ΋γϯϓϧ https://github.com/spotify/luigi

Slide 27

Slide 27 text

ػցֶश

Slide 28

Slide 28 text

ػցֶश·ΘΓͷΞʔΩςΫνϟ جຊํ਑ ɾ"84ͰͰ͖Δ͜ͱ͸"84Ͱ΍Δ ɾ#JH2VFSZʹؔ࿈͢Δ෦෼1ZUIPOͷ෦෼͸($1 ɾ(16ͳͲΫϥ΢υͷ஋ஈ͕ߴͯ͘ɺݕূதʹϦιʔεΛৗ࣌େ ྔফඅ͢Δ΋ͷ͸ΦϯϓϨ
 ɾܭࢉࡁ݁Ռ͸3FEJTʹอଘͯ͠"1*͔Βݺͼग़͢ ɾ֤Ϋϥ΢υؒͷ࣮૷͸#JH2VFSZΛத৺ͱͯ͠ૄ݁߹Λҙࣝ͢Δ

Slide 29

Slide 29 text

ྨࣅը૾APIͷΞʔΩςΫνϟ " 8 4 ( $ 1 Φ ϯ ϓ Ϩ ಛఆͷΞΠςϜʹྨࣅ͍ͯ͠ΔΞΠςϜ͕දࣔ͞ΕΔ ಛ௃ྔநग़εςοϓ ྨࣅ౓ܭࢉεςοϓ ΞΠςϜදࣔεςοϓ

Slide 30

Slide 30 text

ྨࣅը૾APIͷΞʔΩςΫνϟ "1*4FSWFS &$
 "1*3BJMT BigQuery Tesla / GTX1080 / Chainer 2.ը૾ΛDLɺಛ௃ྔநग़ GCP Batch / Python ֶश༻σʔλऔಘ " 8 4 ( $ 1 Φ ϯ ϓ Ϩ ը૾ͷಛ௃ྔΛอଘ ಛ௃ྔσʔλऔಘ ྨࣅ౓ܭࢉ ྨࣅը૾ܭࢉ݁Ռอଘ &$ #BUDI3VCZ 3FEJT ܭࢉࡁΈσʔλऔಘ Ωϟογϡ࡞੒ ྨࣅը૾σʔλऔಘ ಛఆͷΞΠςϜʹྨࣅ͍ͯ͠ΔΞΠςϜ͕දࣔ͞ΕΔ

Slide 31

Slide 31 text

ϨίϝϯυΤϯδϯͷΞʔΩςΫνϟ(όονॲཧ൛) &$
 "1*3BJMT BigQuery GCP Batch / Python ߦಈσʔλऔಘ ਪન಺༰ܭࢉ ਪન݁Ռอଘ &$ #BUDI3VCZ 3FEJT ਪન݁Ռऔಘ Ωϟογϡ࡞੒ ਪન݁Ռऔಘ " 8 4 ( $ 1 ೔ճɺϢʔβʔͷᅂ޷ʹ͋ͬͨΞΠςϜ͕දࣔ͞ΕΔ

Slide 32

Slide 32 text

ϨίϝϯυΤϯδϯͷΞʔΩςΫνϟ(ΦϯϥΠϯॲཧ൛) &$
 "1*3BJMT BigQuery GCP Batch / Python " 8 4 ( $ 1 ߦಈσʔλऔಘ ਪન಺༰ܭࢉ ਪન݁Ռอଘ GCP ߋ৽API / Flask ਪન݁Ռܭࢉˍߋ৽ ߦಈ݁Ռૹ৴ 3&45 ਪન݁Ռऔಘ ਪન݁Ռऔಘ 3&45 Ϣʔβʔ͕ΞΠςϜΛݟΔͨͼʹΦεεϝΞΠςϜ͕มΘͬͯදࣔ͞ΕΔ 3FEJT

Slide 33

Slide 33 text

·ͱΊ w σʔλ෼ੳ
 #JH2VFSZ5BCMFBVͷ૊Έ߹Θ͕ͤͱͯ΋ྑ͍ w ػցֶश
 ಛ௃ྔநग़ͳͲͷॏ͍ॲཧͷաఔ͸ΦϯϓϨͷ(16͕ Φεεϝ
 ͋ͱ͸($1 "84ͷಛੑΛ׆͔ͯ͠ΞʔΩςΫνϟΛ ߟ͑Δ
 #JH2VFSZΛத৺ʹ͢Δͱઃܭ͠΍ͯ͘͢ӡ༻͠΍͍͢

Slide 34

Slide 34 text

We’re Hiring! ͋Γ͕ͱ͏͍͟͝·ͨ͠ 7"4*-:Ͱ͸ ϑΝογϣϯʷςΫϊϩδʔʹ ڵຯͷ͋ΔΤϯδχΞΛืू͍ͯ͠·͢ JOGP!WBTJMZKQ