Slide 1

Slide 1 text

"NB[PO&$4Λ׆༻ͨ͠ "84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ แΈӅͣ͞ղઆ "84ࣄۀຊ෦ɹϓϩμΫτάϧʔϓ ҏ౻঵!TIPJUP

Slide 2

Slide 2 text

εϥΠυ͸ޙͰެ։͠·͢ͷͰ ൃදதͷ಺༰ΛϝϞ͢Δඞཁ͸͋Γ·ͤΜɻ ࣸਅࡱӨΛ͢Δ৔߹͸ ϑϥογϡɾγϟολʔԻ͕ग़ͳ͍Α͏ʹ͝഑ྀ͍ͩ͘͞ Attention http://bit.ly/inside-opswitch

Slide 3

Slide 3 text

ϋογϡλά DNEFWJP DNEFWJP

Slide 4

Slide 4 text

ηογϣϯ֓ཁΠϕϯταΠτΑΓ࠶ܝ "NB[PO&$4Λ׆༻ͨ͠"84ӡ༻ࣗಈԽαʔϏε ʮPQTXJUDIʯɺͦͷཪଆͰ͸ɺ%PDLFSίϯςφ΍ϫʔ ΫϑϩʔΤϯδϯ"QBDIF"JSqPXɺαʔό؅ཧɾ؂ࢹ αʔϏε.BDLFSFMɺίϥϘϨʔγϣϯϋϒ4MBDLͳͲ ͕׆༻͞Ε͍ͯ·͢ɻ࣮ࡍͷαʔϏεͷΞʔΩςΫ νϟɺ։ൃ؀ڥɺσϓϩΠɺ؂ࢹ·ͰɺแΈӅͣ͞ղઆ ͠·͢ɻ

Slide 5

Slide 5 text

ηογϣϯͷ໨తئ๬ ঺հ͢Δଟ͘ͷ৘ใͷத͔ΒɺҰͭͰ΋ ։ൃɾӡ༻ͷώϯτʹͳΔ΋ͷΛ͝ఏڙ͢Δ

Slide 6

Slide 6 text

ࣗݾ঺հ ҏ౻঵!TIPJUP ιϑτ΢ΣΞΤϯδχΞ 4BB4ϓϩμΫτͷاըɾ։ൃɾӡ༻ Ϋϥεϝιουೖ͔ࣾΒ໿೥ܦա ޷͖ͳ"84αʔϏε ⿣ "NB[PO&$4 ⿣ "NQMJGZ$POTPMF ⿣ "NB[PO&MBTUJDTFBSDI4FSWJDF

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

"NB[PO&$4Λ׆༻ͨ͠ "84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ แΈӅͣ͞ղઆ

Slide 9

Slide 9 text

"NB[PO&$4Λ׆༻ͨ͠ "84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ แΈӅͣ͞ղઆ

Slide 10

Slide 10 text

"NB[PO&$4Λ׆༻ͨ͠ "84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ แΈӅͣ͞ղઆ

Slide 11

Slide 11 text

"NB[PO&$4Λ׆༻ͨ͠ "84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ แΈӅͣ͞ղઆ

Slide 12

Slide 12 text

ΞδΣϯμ ✦ "84ӡ༻ͱ՝୊ ✦ τΠϧͱӡ༻ࣗಈԽ ✦ "84ӡ༻ࣗಈԽαʔϏεͷཪଆ ✦ ·ͱΊ

Slide 13

Slide 13 text

ΞδΣϯμ ✦ "84ӡ༻ͱ՝୊ ✦ τΠϧͱӡ༻ࣗಈԽ ✦ "84ӡ༻ࣗಈԽαʔϏεͷཪଆ ✦ ·ͱΊ

Slide 14

Slide 14 text

"84ӡ༻ ఆৗ࡞ۀ ো֐ରԠ "84ΠϯϑϥΛ҆શͰ҆ఆͨ͠؀ڥʹҡ࣋ αʔόɺσʔλϕʔεɺωοτϫʔΫɺ"84Ϛωδϝϯτ ίϯιʔϧͳͲΛखॱॻʹԊͬͯૢ࡞ ࣌ؒ೔ͷো֐ରԠΛߦ͍ɺ҆৺ɾ҆શͳ
 ؀ڥͷҡ࣋ ௨஌΍࠶ىಈͱ͍ͬͨো֐Ұ࣍ରԠ ϩάղੳ΍ݪҼௐࠪɺো֐෮چͱ͍ͬͨೋ࣍ରԠ IUUQTDMBTTNFUIPEKQTFSWJDFTNFNCFSTBXTPQFSBUJOH

Slide 15

Slide 15 text

"84ӡ༻ ఆৗ࡞ۀ ো֐ରԠ "84ΠϯϑϥΛ҆શͰ҆ఆͨ͠؀ڥʹҡ࣋ αʔόɺσʔλϕʔεɺωοτϫʔΫɺ"84Ϛωδϝϯτ ίϯιʔϧͳͲΛखॱॻʹԊͬͯૢ࡞ ࣌ؒ೔ͷো֐ରԠΛߦ͍ɺ҆৺ɾ҆શͳ
 ؀ڥͷҡ࣋ ௨஌΍࠶ىಈͱ͍ͬͨো֐Ұ࣍ରԠ ϩάղੳ΍ݪҼௐࠪɺো֐෮چͱ͍ͬͨೋ࣍ରԠ IUUQTDMBTTNFUIPEKQTFSWJDFTNFNCFSTBXTPQFSBUJOH

Slide 16

Slide 16 text

ఆৗ࡞ۀͷྫ w&$3%4όοΫΞοϓ w&$3%4Πϯελϯεͷى ಈɾఀࢭ w&$ΠϯελϯελΠϓͷม ߋ wηΩϡϦςΟάϧʔϓͷ࡞੒ɾ มߋɾ࡟আ wηΩϡϦςΟνΣοΫ wύονద༻ w"$.ূ໌ॻͷखಈߋ৽ wυϝΠϯͷखಈߋ৽ w3*ͷߪೖ w&-#1SF8BSNJOHਃ੥ w

Slide 17

Slide 17 text

ӡ༻ͷࠔΓ͝ͱͷྫ ӡ༻ʹ๩ࡴ͞Εɺকདྷʹ޲͚ͨվળʹ࣌ؒΛׂ͚ͳ͍ wӡ༻୲౰ऀͷ࡞ۀෛՙ͕ߴ͘ɺ͍ͭ΋࢒ۀɺٳ೔ग़ۈ͕ൃੜ wӡ༻υΩϡϝϯτ͕͋ͬͯ΋ϝϯςφϯε͞Ε͍ͯͳ͍ w୲౰ऀʹґଘͨ͠ۀ຿͕͋Γɺͦͷਓʹෛՙ͕ूத wπʔϧ΍ࣗಈԽʹίετ͕͔͚ΒΕͣʹɺզຫͯ͠࢖͍ଓ͚͍ͯΔ wखॱॻʹैͬͯ࡞ۀΛ͢Δ͕Φϖϛε͕ൃੜ͠ɺ࠶ൃ๷ࢭͷνΣοΫʹ࣌ ͕͔͔ؒΔ wʜ IUUQTDMBTTNFUIPEKQTFSWJDFTNFNCFSTBXTPQFSBUJOH

Slide 18

Slide 18 text

ΞδΣϯμ ✦ "84ӡ༻ͱ՝୊ ✦ τΠϧͱӡ༻ࣗಈԽ ✦ "84ӡ༻ࣗಈԽαʔϏεͷཪଆ ✦ ·ͱΊ

Slide 19

Slide 19 text

ʮτΠϧʯͷఆٛ ϓϩμΫγϣϯαʔϏεΛಈ࡞ͤ͞Δ͜ͱʹؔ܎͢ΔҎԼͷ࡞ۀ wख࡞ۀͰ͋Δ͜ͱ w܁Γฦ͞ΕΔ͜ͱ wࣗಈԽͰ͖Δ͜ͱ wઓज़తͰ͋Δ͜ͱ w௕ظతͳՁ஋Λ࣋ͨͳ͍͜ͱ wαʔϏεͷ੒௕ʹରͯ͠0 O Ͱ͋Δ͜ͱ ˞ষʮτΠϧʯͷ๾໓ΑΓҾ༻

Slide 20

Slide 20 text

ষʮτΠϧʯͷ๾໓ z௨ৗӡ༻தͷγεςϜʹਓख͕ඞཁͳΒɺ
 ͦΕ͸όάͩɻ ௨ৗͷఆٛ͸ɺγεςϜͷ੒௕ͱڞʹมԽ͢Δɻ ŠŠ$BSMB(FJTTFSɺ(PPHMF43&z

Slide 21

Slide 21 text

ӡ༻ࣗಈԽ ࿑ಇ࣌ؒ୹ॖɾ඼࣭޲্ɾίετ࡟ݮ

Slide 22

Slide 22 text

ࣗಈԽͷ೉͠͞ w࣮ߦεέδϡʔϦϯά w࣮ߦঢ়ଶɾ࣮ߦ݁ՌͷՄࢹԽ wࣦഊ࣌ͷ࠶ࢼߦ ϦτϥΠ wλΠϜΞ΢τॲཧ wґଘؔ܎ͷ͋ΔλεΫ wҟৗܥϑϩʔ wϩΪϯά wฒྻԽ w੒ޭɾࣦഊͷ݁Ռ௨஌ wʜ ର৅ͷपลͰߟྀ͢΂͖ࣄ߲͕ଟ͍

Slide 23

Slide 23 text

ΞδΣϯμ ✦ "84ӡ༻ͱ՝୊ ✦ τΠϧͱӡ༻ࣗಈԽ ✦ "84ӡ༻ࣗಈԽαʔϏεͷཪଆ ✦ ·ͱΊ

Slide 24

Slide 24 text

PQTXJUDI Φϓε΢Οον

Slide 25

Slide 25 text

ࣗಈԽͰ͖ΔλεΫ܈ w &$όοΫΞοϓͷ࡞੒ w &$Πϯελϯεͷىಈɾఀࢭ w &$ΠϯελϯελΠϓͷมߋ w &$όοΫΞοϓͷϦʔδϣϯؒίϐʔ w εφοϓγϣοτͷ࡞੒ w εφοϓγϣοτͷϦʔδϣϯؒίϐʔ w 3%4εφοϓγϣοτͷ࡞੒ w 3%4Πϯελϯεͷ࡟আ w 3%4εφοϓγϣοτ͔Βͷ෮ݩ w 3%4Πϯελϯεͷىಈɾఀࢭ w ϦιʔεͷࢭΊ๨ΕνΣοΫ w ηΩϡϦςΟνΣοΫ w ʜਵ࣌௥Ճத

Slide 26

Slide 26 text

δϣϒػೳ ༻్ʹԠͯ͡ॊೈʹࢦఆՄೳͳ࣮ߦεέδϡʔϧ &$όοΫΞοϓͱ εφοϓγϣοτɺ 3%4ͷ%#εφοϓγϣοτ࡞੒ ͳͲɺࣗ༝ͳλεΫͷ૊Έ߹Θͤ δϣϒͷ࣮ߦཤྺɺ ॲཧ࣌ؒɺϩάͷ֬ೝ

Slide 27

Slide 27 text

"84ӡ༻ࣗಈԽαʔϏεͷཪଆ αʔϏεͷϕʔεͱͳΔ
 "NB[PO&$4ͱ"QBDIF"JSqPX

Slide 28

Slide 28 text

"NB[PO&$4 &MBTUJD$POUBJOFS4FSWJDF %PDLFSίϯςφΛ؆୯ʹӡ༻ɾ؅ཧͰ͖ΔίϯςφΦʔέετϨʔγϣϯαʔϏε IUUQTBXTBNB[PODPNKQFDT

Slide 29

Slide 29 text

"QBDIF"JSqPX w044ͷ1ZUIPO੡ϫʔΫϑϩʔΤϯδϯ "QBDIF-JDFOTF w1ZUIPOίʔυͰදݱͨ͠ϫʔΫϑϩʔ %"( ΛεέδϡʔϦϯά ˍϞχλϦϯά͢ΔϓϥοτϑΥʔϜ
 8PSLqPXBTB$PEF wෳ਺ͷλεΫ 0QFSBUPS ͷ࣮ߦॱংΛఆٛ͢ΔϫʔΫϑϩʔͷ࡞ ੒ɺεέδϡʔϦϯάɺϞχλϦϯάɺϦτϥΠɺ௨஌ͳͲ͕Մೳ "JSqPX͕ࣗಈԽͷෳࡶ͞Λαϙʔτ͢ΔͨΊɺ
 ɹɹࢲͨͪ͸ۀ຿ϩδοΫͷ։ൃʹ஫ྗͰ͖Δ

Slide 30

Slide 30 text

"QBDIF"JSqPX4BB4 $MPVE$PNQPTFS ϑϧϚωʔδυ"JSqPX "TUSPOPNFS$MPVE&OUFSQSJTF "JSqPXBTB4FSWJDF

Slide 31

Slide 31 text

"QBDIF"JSqPXͷߏ੒ཁૉ 5BTL" 5BTL# 5BTL$ 5BTL% 5BTL& 5BTL' %"( w0QFSBUPS
 λεΫͷςϯϓϨʔτ w5BTL
 0QFSBUPS͕Πϯελϯ εԽ͞Εͨ΋ͷ w%"(
 ϫʔΫϑϩʔɺλεΫ ͷґଘؔ܎
 1ZUIPOϑΝΠϧͰఆٛ ˞%"(%JSFDUFE"DZDMJD(SBQI

Slide 32

Slide 32 text

"QBDIF"JSqPXͷߏ੒ཁૉ

Slide 33

Slide 33 text

"QBDIF"JSqPXͷཁૉ Web UI / ؅ཧίϯιʔϧΛఏڙ͢Δ

Slide 34

Slide 34 text

"QBDIF"JSqPXͷཁૉ δϣϒ(DAG)ͷεέδϡʔϧΛ࣮ߦ͢Δ

Slide 35

Slide 35 text

"QBDIF"JSqPXͷཁૉ DAGͰఆٛ͞ΕͨλεΫΛ࣮ߦ͢Δ

Slide 36

Slide 36 text

αʔϏεͷཪଆΛ঺հ͢Δʹ͋ͨΓେࣄͳ͜ͱ ͜Ε͕ϕετͱ͍͏Θ͚Ͱ͸͋Γ·ͤΜɻ ઃܭ౰࣌ͷʮཁ݅ʯʮঢ়گʯʮ੍໿ʯͳͲʹ ΑΓબ୒͞Εͨ΋ͷͰ͋Γվળͷ༨஍͕͋Δ ΋ͷͰ͢ɻ

Slide 37

Slide 37 text

શମΞʔΩςΫνϟ

Slide 38

Slide 38 text

શମΞʔΩςΫνϟ

Slide 39

Slide 39 text

શମΞʔΩςΫνϟ

Slide 40

Slide 40 text

ίΞ෦෼ʹߜͬͨΞʔΩςΫνϟ

Slide 41

Slide 41 text

σϓϩΠͷ࢓૊Έ

Slide 42

Slide 42 text

%"(ϑΝΠϧετϨʔδͱͯ͠ͷ4 "JSqPX༻4όέοτ wEBHT w૊৫" wEBHϑΝΠϧ" wEBHϑΝΠϧ# w૊৫# wEBHϑΝΠϧ$ wEBHϑΝΠϧ% wEBHϑΝΠϧ& wʜ ӬଓԽͱϊʔυؒͰͷڞ༗ͷͨΊ4΁

Slide 43

Slide 43 text

%"(ϑΝΠϧͷ࣮૷ # -*- coding: utf-8 -*- import time from pprint import pprint import airflow from airflow.models import DAG from airflow.operators.python_operator import PythonOperator, PythonVirtualenvOperator args = { 'owner': 'Airflow', 'start_date': airflow.utils.dates.days_ago(2), } dag = DAG( dag_id='example_python_operator', default_args=args, schedule_interval=None, ) def print_context(ds, **kwargs): pprint(kwargs) print(ds) return 'Whatever you return gets printed in the logs' run_this = PythonOperator( task_id='print_the_context', python_callable=print_context, dag=dag, ) def my_sleeping_function(random_base): time.sleep(random_base) for i in range(5): task = PythonOperator( task_id='sleep_for_' + str(i), python_callable=my_sleeping_function, op_kwargs={'random_base': float(i) / 10}, dag=dag, ) run_this >> task … https://github.com/apache/airflow/blob/master/airflow/example_dags/example_python_operator.py

Slide 44

Slide 44 text

֓೦σʔλϞσϧ

Slide 45

Slide 45 text

"84ӡ༻δϣϒͷ࣮ߦ"84ΞΧ΢ϯτ࿈ܞ "844FDVSJUZ5PLFO4FSWJDF 454 "TTVNF3PMF"1* Ұ࣌తͳηΩϡϦςΟೝূ৘ใΛऔಘɻ
 ͜ΕΒΛ༻͍ͯɺڐՄ͞ΕͨผͷΞΧ΢ϯτ ͷϦιʔεʹΞΫηεՄೳɻ w ΞΫηεΩʔ*% w γʔΫϨοτΞΫηεΩʔ w ηογϣϯτʔΫϯ

Slide 46

Slide 46 text

"84ΞΧ΢ϯτ࿈ܞʹΑΔΞΫηε ଞͷ"84ΞΧ΢ϯτ͔Β͸ "TTVNF3PMFͰ͖·ͤΜɻ ֎෦*%͕ඞཁͳͷͰɺͦΕΛ ࣋ͨͳ͍ଞͷ"84ΞΧ΢ϯτ ͸"TTVNF3PMF͸Ͱ͖·ͤΜɻ δϣϒͷ࣮ߦʹඞཁͳϦιʔε΁ͷΞΫηεݖݶͷΈ ෇༩͠·͢ɻ4΍σʔλϕʔεʹ֨ೲ͞ΕͨσʔλΛ Ӿཡ͢Δ͜ͱ͸Ͱ͖·ͤΜɻ

Slide 47

Slide 47 text

ϝʔϧ഑৴ "NB[PO4&4 "NB[PO1JOQPJOU ݸผૹ৴ w Ϣʔβʔొ࿥ w Ϣʔβʔট଴ w δϣϒ࣮ߦ௨஌ w ʜ ෳ਺ૹ৴ w ͓஌Βͤ഑৴ w ϝϯςφϯε௨஌ w ʜ

Slide 48

Slide 48 text

"NB[PO1JOQPJOU ෳ਺ͷϝοηʔδϯάνϟωϧΛ௨ͯ͡ސ٬ͱίϛϡχέʔγϣϯ͢ΔαʔϏε IUUQTBXTBNB[PODPNKQQJOQPJOU

Slide 49

Slide 49 text

ϝʔϧ഑৴ͱό΢ϯεϝʔϧରԠ

Slide 50

Slide 50 text

γεςϜϞχλϦϯά

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

γεςϜϞχλϦϯά.BDLFSFMμογϡϘʔυ

Slide 53

Slide 53 text

γεςϜϞχλϦϯά.BDLFSFMμογϡϘʔυ

Slide 54

Slide 54 text

γεςϜϞχλϦϯά.BDLFSFM4MBDL௨஌

Slide 55

Slide 55 text

γεςϜϞχλϦϯά಺੡ӡ༻πʔϧ

Slide 56

Slide 56 text

γεςϜϞχλϦϯά಺੡ӡ༻πʔϧ

Slide 57

Slide 57 text

γεςϜϞχλϦϯά಺੡ӡ༻πʔϧ

Slide 58

Slide 58 text

,1*ϞχλϦϯά "NB[PO2VJDL4JHIUͰՄࢹԽˠ

Slide 59

Slide 59 text

,1*ϞχλϦϯά ि࣍Ͱ4MBDL΁ϨϙʔςΟϯάˠ

Slide 60

Slide 60 text

Πϯϑϥ؀ڥ܈ ։ൃ؀ڥ ෛՙςετ
 ؀ڥ εςʔδϯά
 ؀ڥ ຊ൪؀ڥ ϩʔΧϧ
 ։ൃ؀ڥ
 EPDLFSDPNQPTF "84ΞΧ΢ϯτ9 "84ΞΧ΢ϯτ: "84ΞΧ΢ϯτ;

Slide 61

Slide 61 text

ϩʔΧϧ։ൃ؀ڥͷ࢓૊Έ $ docker-compose up -d

Slide 62

Slide 62 text

%PDLFS$PNQPTF WFSTJPO TFSWJDFT SFEJT JNBHFSFEJT QPTUHSFT JNBHFQPTUHSFT XFCTFSWFS CVJME DPOUFYU\18%^ EPDLFSpMF\18%^%PDLFSpMF JNBHFPQTXJUDI DPNNBOEXFCTFSWFS TDIFEVMFS JNBHFPQTXJUDI DPNNBOETDIFEVMFS XPSLFS JNBHFPQTXJUDI DPNNBOEXPSLFS WPMVNFT ECEBUB docker-compose.yml ෳ਺ͷίϯςφΛ࢖͏ DockerΞϓϦέʔγϣϯΛఆٛ͠ɺ ૢ࡞͢Δπʔϧ

Slide 63

Slide 63 text

·ͱΊ

Slide 64

Slide 64 text

͓࿩ͨ͜͠ͱ wഎܠͱͳΔ"84ӡ༻ͱ՝୊ wτΠϧͷ๾໓ɺࣗಈԽͷͨΊͷߟྀ఺ͷଟ͞ w"84ӡ༻ࣗಈԽαʔϏεͷཪଆΛ࣌ؒ಺ͰՄೳͳݶΓ঺հ wࣗಈԽͷෳࡶ͞ʹରԠ͢ΔͨΊͷ"NB[PO&$4ͱ"QBDIF "JSqPXΛ૊Έ߹Θͤ wΞʔΩςΫνϟ͔ΒσϓϩΠɺϝʔϧ഑৴ɺϞχλϦϯάɺ ։ൃ·Ͱ

Slide 65

Slide 65 text

ηογϣϯͷ໨తئ๬ ঺հ͢Δଟ͘ͷ৘ใͷத͔ΒɺҰͭͰ΋ ։ൃɾӡ༻ͷώϯτʹͳΔ΋ͷΛ͝ఏڙ͢Δ

Slide 66

Slide 66 text

No content