推薦のためのdata pipelineのモニタリングとロギング - Speaker Deck

推薦のためのdata pipelineのモニタリングとロギング

by Hazumi Ichijo

Speaker Deck

Tweet

Tweet

Slide 1

Slide 1 text

©2019 Wantedly, Inc. ਪનͷͨΊͷdata pipelineͷ ϞχλϦϯάͱϩΪϯά Apr 16, 2019 - Hazumi Ichijo Data Pipeline Casual Talk #2

Slide 2

Slide 2 text

©2018 Wantedly, Inc. ࣗݾ঺հ Ұᑍ୺੅!IB[VNJSS!SFSPTU %BUB&OHJOFFSJO8BOUFEMZ (P 3BJMT 3FBDUʜ ๺ւಓग़਎

Slide 3

Slide 3 text

©2019 Wantedly, Inc. ͳͥϞχλϦϯάɾϩΪϯά͢Δ͔ ԿΛϞχλϦϯάɾϩΪϯά͢΂͖͔ ͲΕ͘Β͍कΔ΂͖͔ ԿΛ͢΂͖͔

Slide 4

Slide 4 text

©2019 Wantedly, Inc. +PC +PC +PC +PC +PC σʔλ σʔλ σʔλ αʔϏε

Slide 5

Slide 5 text

©2019 Wantedly, Inc. ໰୊͕ൃੜͨ͠ͱ͖Ͳ͏͠·͢ʁ

Slide 6

Slide 6 text

©2019 Wantedly, Inc. +PC +PC +PC +PC +PC +PC +PC +PC σʔλ σʔλ σʔλ

Slide 7

Slide 7 text

©2019 Wantedly, Inc. +PC +PC +PC +PC +PC +PC +PC +PC σʔλ σʔλ σʔλ

Slide 8

Slide 8 text

©2019 Wantedly, Inc. +PC +PC +PC +PC +PC +PC +PC +PC σʔλ σʔλ σʔλ ʁ ʁ ʁ

Slide 9

Slide 9 text

©2019 Wantedly, Inc. ͦ΋ͦ΋ͳ͍ ζϨΔʢྫݹ͍σʔλΛݩʹͨ͠ਪન͕ग़Δʣ ग़ͯ͘Δ঱ঢ় ਪનͷͨΊͷσʔλ͕

Slide 10

Slide 10 text

©2019 Wantedly, Inc. ͦ΋ͦ΋ͳ͍ ζϨΔʢྫݹ͍σʔλΛݩʹͨ͠ਪન͕ग़Δʣ ग़ͯ͘Δ঱ঢ় ਪનͷͨΊͷσʔλ͕ ؾ͖ͮʹ͍͘

Slide 11

Slide 11 text

©2019 Wantedly, Inc. ໰୊͕ൃੜͨ͠ࡍʹؾ͖ͮʹ͍͘ ໰୊͕ൃੜͨ࣌͠ͷௐࠪํ๏͕໘౗ ͳͥϞχλϦϯάɾϩΪϯάΛ͢Δ͔ ਪનͷͨΊͷσʔλ͸

Slide 12

Slide 12 text

©2019 Wantedly, Inc. ໰୊͕ൃੜͨ͠ࡍʹؾ͖ͮʹ͍͘ ໰୊͕ൃੜͨ࣌͠ͷௐࠪํ๏͕໘౗ ͳͥϞχλϦϯάɾϩΪϯάΛ͢Δ͔ ਪનͷͨΊͷσʔλ͸ ϞχλϦϯάɾϩΪϯάΛͯ͠໰୊ൃੜ࣌ʹඋ͑Δ

Slide 13

Slide 13 text

©2019 Wantedly, Inc. ԿΛϞχλϦϯάɾϩΪϯά͢Δʁ

Slide 14

Slide 14 text

©2019 Wantedly, Inc. ຖճܾΊΒΕͨ৔ॴʹσʔλ͕ग़ྗ͞ΕΔ΂͖ ग़ྗ͞ΕΔσʔλ͕ਖ਼͍͠ඞཁ͕͋Δ w ґଘ͢Δσʔλ͕࠷৽ͷঢ়ଶ w ܾΊΒΕͨ࣌ࠁʹ࣮ߦ͞Εͯ΄͍͠ w FUD ԿΛϞχλϦϯάɾϩΪϯά͢΂͖͔ ਪનͷͨΊͷσʔλ͸

Slide 15

Slide 15 text

©2019 Wantedly, Inc. ຖճܾΊΒΕͨ৔ॴʹσʔλ͕ग़ྗ͞ΕΔ΂͖ ग़ྗ͞ΕΔσʔλ͕ਖ਼͍͠ඞཁ͕͋Δ w ґଘ͢Δσʔλ͕࠷৽ͷঢ়ଶ w ܾΊΒΕͨ࣌ࠁʹ࣮ߦ͞Εͯ΄͍͠ w FUD ԿΛϞχλϦϯάɾϩΪϯά͢΂͖͔ ਪનͷͨΊͷσʔλ͸ ϞχλϦϯάɾϩΪϯάΛͯ͠कΕΔΑ͏ʹ͢Δ

Slide 16

Slide 16 text

©2019 Wantedly, Inc. ΞϓϦέʔγϣϯ w δϣϒ࣮ߦ࣌ͷ࣮ߦঢ়گͷ஋ʢྫֶश཰ʣ w ΦϯϥΠϯධՁͷ஋ w ॻ͖ग़͠ઌʹσʔλ͕ॻ͖ࠐ·Ε͔ͨ w Τϥʔ௨஌ w Կʹ͕͔͔࣌ؒΔ͔ Πϯϑϥ w δϣϒ͕੒ޭ͔ͨ͠ɾࣦഊ͔ͨ͠ɾ࣮ߦ͞Ε͔ͨ w ઃఆ͍ͯ͠Δ࣌ؒʹ࣮ߦ͞Ε͔ͨ ԿΛϞχλϦϯάɾϩΪϯά͢΂͖͔ ϞχλϦϯά

Slide 17

Slide 17 text

©2019 Wantedly, Inc. w ॲཧͷ಺༰͕Θ͔Δ΋ͷʢྫ42-ʣ w δϣϒͷ։࢝ɾऴྃ w ॻ͖ग़ͨ͠σʔλ w FUD ԿΛϞχλϦϯάɾϩΪϯά͢΂͖͔ ϩΪϯά

Slide 18

Slide 18 text

©2019 Wantedly, Inc. ͲΕ͘Β͍कΒΕΔ΂͖΋ͷʁ

Slide 19

Slide 19 text

©2019 Wantedly, Inc. αʔϏεશମ͕࢖͑ͳ͘ͳΔ ,1*ʹ99Өڹ͕ͰΔ Ұ෦ͷػೳ͕࢖͑ͳ͘ͳΔ ͲΕ͘Β͍कΔ΂͖͔ ͋Δਪનʹ໰୊͕͋Δͱɾɾɾ

Slide 20

Slide 20 text

©2019 Wantedly, Inc. αʔϏεશମ͕࢖͑ͳ͘ͳΔ ,1*ʹ99Өڹ͕ͰΔ Ұ෦ͷػೳ͕࢖͑ͳ͘ͳΔ ͲΕ͘Β͍कΔ΂͖͔ ͋Δਪનʹ໰୊͕͋Δͱɾɾɾ ͦ΋ͦ΋ͦͷσʔλͷॏཁ౓͕Θ͔Δඞཁ͕͋Δ

Slide 21

Slide 21 text

©2019 Wantedly, Inc. ԿΛ͢΂͖͔

Slide 22

Slide 22 text

©2019 Wantedly, Inc. ॏཁ౓ΛͰ͖Δ͚ͩ໌֬ʹ͓ͯ͘͠ ϞχλϦϯάɾϩΪϯάͷ߲໨ચ͍ग़͠ ΠϯϑϥνʔϜͱڠྗͯ͠ɺ༧Ί w ॏཁ౓Λ఻͑Δ w ϞχλϦϯάɾϩΪϯάͷλεΫͷׂΓৼΓΛߦ͏ w ໰୊͕ൃੜͨ࣌͠ɺͲ͏͍ͬͨϑϩʔͰ֬ೝ͢Δ͔ɺͲ͏ରॲ ͢Δ͔Λ఻͑ΔඞཁͰ͋Ε͹࣮ࡍʹϦϋʔαϧΛߦ͏ ԿΛ͢΂͖͔