Finagleを使った広告配信基盤

by Tomohisa Omagari

Slide 1

Slide 1 text

FinagleΛ࢖ͬͨ ޿ࠂ഑৴ج൫

Slide 2

Slide 2 text

ࣗݾ঺հ େۂஐٱ COMPANY SERVICE GitHub oomatomo

Slide 3

Slide 3 text

࿩͢಺༰ ▸ ࣮ࡍʹFinagleΛ࢖ͬͯΈͯɺؾ͍ͮͨTipsతͳ಺༰ ▸ Finagleͷઆ໌͸ɺ͋·Γ͠ͳ͍ɻ

Slide 4

Slide 4 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 5

Slide 5 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 6

Slide 6 text

Finagle ▸ ϑΟωʔάϧ ▸ Twitter ͕։ൃͨ͠OSS ▸ RPCγεςϜ  RPCͷ࣮૷Ͱ͸ͳ͍ ▸ https://twitter.github.io/ﬁnagle/ ▸ https://monkey.org/~marius/talks/twittersystems/#1

Slide 7

Slide 7 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 8

Slide 8 text

޿ࠂ഑৴ͷྲྀΕ

Slide 9

Slide 9 text

޿ࠂ഑৴ͷྲྀΕ ഑৴෦෼

Slide 10

Slide 10 text

޿ࠂ഑৴ͷྲྀΕ ूܭ෦෼

Slide 11

Slide 11 text

ࠓճ͸ ഑৴෦෼ʹ͍ͭͯ ࿩͠·͢

Slide 12

Slide 12 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 13

Slide 13 text

഑৴ج൫ʹؔͯ͠

Slide 14

Slide 14 text

഑৴ج൫ʹؔͯ͠

Slide 15

Slide 15 text

σʔϞϯ༻αʔό ഑৴ج൫ʹؔͯ͠

Slide 16

Slide 16 text

σʔλΛγϦΞϥΠζͯ͠ Redisʹอଘ͢Δ ഑৴ج൫ʹؔͯ͠

Slide 17

Slide 17 text

ActorΛεέδϡʔϦϯάͰ ಈ͔ͯ͠σʔϞϯͱͯ͠ར༻ ͜͜ͷσʔϞϯͷ࣮૷ͷ࢓ํ͸ɺผͷൃදࢿྉʹ͋Γ·͢ɻ  https://speakerdeck.com/oomatomo/16-adtech-x-scala-meetup-falselt ഑৴ج൫ʹؔͯ͠

Slide 18

Slide 18 text

޿ࠂ഑৴༻αʔό ഑৴ج൫ʹؔͯ͠

Slide 19

Slide 19 text

޿ࠂͷϦΫΤετ͸  ͢΂ͯ޿ࠂ഑৴༻αʔόͷ1୆Ͱ  ׬݁͢Δ ഑৴ج൫ʹؔͯ͠

Slide 20

Slide 20 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 21

Slide 21 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠

Slide 22

Slide 22 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ϏδωεϩδοΫͷஔ͖৔ Service͔Βݺ͹ΕΔ  ݱঢ়͸ίʔυྔ͸ଟ͘͸ͳ͍ͷͰ Redis͔Βͷσʔλͷऔಘͷॲཧͱ͔ ΋ॻ͍͍ͯΔ

Slide 23

Slide 23 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ Ϟσϧͷஔ͖৔ Logic΍ServiceͰར༻͢ΔΫϥε͕  ॻ͍ͯ͋Δ

Slide 24

Slide 24 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ FinagleͷServiceͷஔ͖৔ Play FrameworkͰݴ͏  ControllerΈ͍ͨͳ΋ͷ

Slide 25

Slide 25 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠

Slide 26

Slide 26 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠

Slide 27

Slide 27 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ڞ௨ॲཧͷஔ͖৔ Service΍LogicͰར༻͢Δ  ศརܥͷॲཧ͕͜͜ʹ͋Δ

Slide 28

Slide 28 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ϝΠϯΫϥε Router΍HttpαʔόͷىಈͳͲΛ  ߦ͍ͬͯΔ

Slide 29

Slide 29 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠

Slide 30

Slide 30 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ graceful stop ͸  ࣮૷͍ͯ͠ͳ͍ͷͰ  ࠶ىಈ࣌͸ϩʔυόϥϯαʔ͔Β  ֎ͯ͠࠶ىಈ͍ͯ͠Δ

Slide 31

Slide 31 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ Unitςετ

Slide 32

Slide 32 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ Unitςετ https://github.com/twitter/finagle/blob/finagle-6.35.0/finagle-redis/src/test/scala/com/twitter/finagle/redis/RedisTest.scala finagle-redisͷςετίʔυΛࢀߟʹ࡞੒ͨ͠

Slide 33

Slide 33 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ Unitςετ

Slide 34

Slide 34 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ E2Eςετ

Slide 35

Slide 35 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ E2Eςετ

Slide 36

Slide 36 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ E2Eςετ ςετ༻ʹαʔόͱΫϥΠΞϯτΛੜ੒͢Δ

Slide 37

Slide 37 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ E2Eςετ ϦΫΤετΛੜ੒࣮͠ߦ͢Δ

Slide 38

Slide 38 text

Finagleͷίʔυͷߏ੒ʹؔͯ͠ ▸ E2Eςετ ϨεϙϯεΛνΣοΫ͢Δ

Slide 39

Slide 39 text

ΞδΣϯμ ▸ Finagleͱ͸ʁ ▸ ޿ࠂ഑৴ͷྲྀΕ ▸ ഑৴ج൫ʹؔͯ͠ ▸ Finagleͷίʔυͷߏ੒ ▸ FinagleͷνϡʔχϯάͰͷؾ͖ͮ

Slide 40

Slide 40 text

ͱΓ͋͑ͣɺಈ͘΋ͷ࡞ͬͯ ෛՙςετΛ࣮ࢪͨ͠ɻ

Slide 41

Slide 41 text

500 QPS ΠϯελϯελΠϓ: c4large

Slide 42

Slide 42 text

▸ RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ Future(ඇಉظ)Λ׆͔ͨ͠ઃܭ ▸ Routerͷॻ͖ํ࣍ୈͰ  ϦΫΤετͷͨͼʹΠϯελϯε͕ੜ੒͞ΕΔ໰୊

Slide 43

Slide 43 text

RedisͰ͋ΔҰఆҎ্ ίωΫγϣϯ͕ ૿͑ͳ͍໰୊

Slide 44

Slide 44 text

RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ ௨ৗͷredisͷClientͷ࡞Γํ

Slide 45

Slide 45 text

RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ ௨ৗͷredisͷClientͷ࡞Γํ

Slide 46

Slide 46 text

RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ ௨ৗͷredisͷClientͷ࡞Γํ https://github.com/twitter/finagle/blob/finagle-6.35.0/finagle-redis/src/main/scala/com/twitter/finagle/redis/Client.scala ▸ ࣮ࡍʹClientΛੜ੒͍ͯ͠Δίʔυ(Finagleͷόʔδϣϯ͸ɺ6.35.0)

Slide 47

Slide 47 text

Slide 48

Slide 48 text

RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ ΧελϚΠζͨ͠Clientͷ࡞Γํ ( 1 -> 10000 )

Slide 49

Slide 49 text

500 QPS → 1350 QPS ΠϯελϯελΠϓ: c4large

Slide 50

Slide 50 text

RedisͰ͋ΔҰఆҎ্ίωΫγϣϯ͕૿͑ͳ͍໰୊ ▸ ΧελϚΠζͨ͠Clientͷ࡞Γํ ( 1 -> 10000 ) ࠷৽൛(6.42.0)Ͱ͸  Clientͷੜ੒ͷํ๏͕มߋ͞Ε͍ͯΔͨΊ  ͜ͷॻ͖ํ͸ग़དྷͳ͍ʂʂʂ

Slide 51

Slide 51 text

Future(ඇಉظ)Λ ׆͔ͨ͠ઃܭ

Slide 52

Slide 52 text

Future(ඇಉظ)Λ׆͔ͨ͠ઃܭ ▸ ࠷ॳͷઃܭ

Slide 53

Slide 53 text

Future(ඇಉظ)Λ׆͔ͨ͠ઃܭ ▸ ࠷ॳͷઃܭ

Slide 54

Slide 54 text

Future(ඇಉظ)Λ׆͔ͨ͠ઃܭ ▸ ฒྻͰ͖ΔΑ͏ʹΩʔΛมߋ

Slide 55

Slide 55 text

Future(ඇಉظ)Λ׆͔ͨ͠ઃܭ ▸ ฒྻͰ͖ΔΑ͏ʹΩʔΛมߋ

Slide 56

Slide 56 text

1350 QPS → 1750 QPS ΠϯελϯελΠϓ: c4large

Slide 57

Slide 57 text

Routerͷॻ͖ํ࣍ୈͰ  ϦΫΤετͷͨͼʹ Πϯελϯε͕ ੜ੒͞ΕΔ໰୊

Slide 58

Slide 58 text

Routerͷॻ͖ํ࣍ୈͰϦΫΤετͷͨͼʹΠϯελϯε͕ੜ੒͞ΕΔ໰୊ ▸ /user/1 ͷ࣮૷(Before)

Slide 59

Slide 59 text

Routerͷॻ͖ํ࣍ୈͰϦΫΤετͷͨͼʹΠϯελϯε͕ੜ੒͞ΕΔ໰୊ ▸ /user/1 ͷ࣮૷(Before) ͜ͷॻ͖ํΛߦ͏ͱຖճϦΫΤετͷ౓ʹ  UserService͕ੜ੒͞Ε͓ͯΓແବʹܨ͕Δ

Slide 60

Slide 60 text

Routerͷॻ͖ํ࣍ୈͰϦΫΤετͷͨͼʹΠϯελϯε͕ੜ੒͞ΕΔ໰୊ ▸ /user/1 ͷ࣮૷ (After) RouterଆͰidͷऔಘΛఘΊͯ  ϦΫΤετͷURL͔Βਖ਼نදݱΛ ࢖ͬͯऔಘ͢ΔΑ͏ʹ͢Δ

Slide 61

Slide 61 text

Routerͷॻ͖ํ࣍ୈͰϦΫΤετͷͨͼʹΠϯελϯε͕ੜ੒͞ΕΔ໰୊ ▸ /user/1 ͷ࣮૷ (After) ࣄલʹServiceΫϥεΛnew͓ͯ͘͠ͱ  ϦΫΤετͷ౓ʹΠϯελϯε͕ੜ੒͞Εͳ͍

Slide 62

Slide 62 text

1750 QPS → 1600 QPS ΠϯελϯελΠϓ: c4large

Slide 63

Slide 63 text

1750 QPS → 1600 QPS ΠϯελϯελΠϓ: c4large Լ͕ͬͨʂʂʂʂʂʂʂʂʂ ਖ਼نදݱͷॲཧͷํ͕ॏ͔ͬͨͷ͔ͳʁʁ ͜ͷमਖ਼͸ෆ࠾༻ʹ͠·ͨ͠ʂ