Scalable Architecture with RabbitMQ

IDCFクラウドのバックエンドをMicroservicesで作って、各コンポーネント間の通信にRabbitMQを使っているという話。

Transcript

1. Scalable Architecture with RabbitMQ IDCϑϩϯςΟΞ UX։ൃ෦ ౉ᬑঘޗ 2015/11/24
2. Slides’ URI https://speakerdeck.com/nownabe/scalable- architecture-with-rabbitmq
3. Who? ౉ᬑঘޗ @nownabe Programmer Drummer
4. History 2013೥4݄ IDCFೖࣾ 2013೥7݄ ΧελϚʔαʔϏεຊ෦ϓϥοτϑΥʔϜαʔϏε෦Ϋϥ΢υάϧʔϓ • Ϋϥ΢υج൫ӡ༻ɾݕূ • αόϑΣε্ཱͪ͛ • MBaaSαʔϏε։ൃ 2014೥4݄ ٕज़։ൃຊ෦R&Dࣨ • MBaaSαʔϏε։ൃ • RDBαʔϏε։ൃ 2015೥1݄ ٕज़։ൃຊ෦UX։ൃ෦ • RDBαʔϏε։ൃ • ϩʔυόϥϯαʔαʔϏε։ൃ (now!)
5. Favorites Ruby Itamae Ruboty React esa Daemon Chain Drive (double) Aquiles Priester
6. Scalable Architecture with RabbitMQ
7. ͸΍Γͷ Publish / Subscribe ͱ Microservices ΍ͬͯΈͨ
8. Which service? ݱࡏ։ൃத IDCF Cloudͷ৽αʔϏε ϩʔυόϥϯαʔαʔϏεͷόοΫΤϯυ ͦΖͦΖʁϦϦʔε༧ఆ
9. What service? 7. 7. 7. … 4DBMBCMF
   #BDLFOE -# -# … *%$'
   $MPVE
   $MPVE4UBDL ࡞ͬͨΓফͨ͠Γ ૿΍ͨ͠ΓݮΒͨ͠Γ ؂ࢹͨ͠Γम෮ͨ͠Γ
10. 8PSLFS … $POUSPMMFS … "1* … .POJUPS … -#
    OPEF -#
    OPEF ……… Scalable architecture
11. 8PSLFS … $POUSPMMFS … "1* … .POJUPS … -#
    OPEF -#
    OPEF ……… Scalable architecture 3&45
    "1*ड͚Δ΍ͭ ؂ࢹɾ՝ۚ͢Δ΍ͭ ࡞ͬͨΓফͨ͠Γ͢Δ΍ͭ ϩʔυόϥϯαʔຊମ ͍Ζ͍Ζ΍Δ΍ͭ
12. Development style Ruby Github Qiita:Team Slack
13. Today’s topics ࿩͢ RabbitMQ (Publish / Subscribe) Microservices ࿩͞ͳ͍ RabbitMQͷෛՙ෼ࢄ/৑௕Խ Monitoring (Sensu)
14. RabbitMQ
15. What is RabbitMQ? ϝοηʔδͷड৴ͱ഑৴ Advanced Message Queuing Protocol OSS࣮૷ Erlang
16. Why RabbitMQ? MicroservicesԽͰશ෦REST API͸͖͍ͭ ߴػೳ ୯ମͰಈ࡞͢Δ υΩϡϝϯτ͕ॆ࣮ͯͨ͠
17. Publish / Subscribe Producer (Publisher) Queueʹ഑৴ Consumer (Subscriber) Queue͔Βߪಡ Queue ✉ ✉ ✉ ✉ ✉ ✉ ✉ ✉ ✉
18. Exchange ϝοηʔδΛQueueʹ഑ୡ͢Δਓ ͍ͭ͜ͷ͓͔͛Ͱ഑৴ܦ࿏͕ॊೈ
19. Example: Fanout ✉ ✉ ✉ ✉ ✉ &YDIBOHF ✉ ඥͮ͘͢΂ͯͷ Queueʹ഑ୡ
20. Example: Roundrobin ✉ &YDIBOHF ✉ ✉ ✉ ✉ ✉ ʮՋͳਓʹ഑৴ʯ΋Մ
21. Example: RPC ✉ ✉ Request/Response (Exchange͸লུ) ฦ৴༻Queue
22. More complecated routings ͔ͳΓෳࡶͳϧʔςΟϯά΋Մೳ
23. Microservices
24. 8PSLFS … $POUSPMMFS … "1* … .POJUPS … -#
    OPEF -#
    OPEF ……… Components 3&45
    "1*ड͚Δ΍ͭ ؂ࢹɾ՝ۚ͢Δ΍ͭ ࡞ͬͨΓফͨ͠Γ͢Δ΍ͭ ϩʔυόϥϯαʔຊମ ͍Ζ͍Ζ΍Δ΍ͭ
25. Why microservices? ScalableͳઃܭΛٻΊΒΕͨ • CloudStack͕ҰຕؠͰεέʔϧ͠ʹ͍͘ઃܭ ֤ίϯϙʔωϯτͷ໾ׂ෼ׂ͕໌֬ͩͬͨ • ͋Δఔ౓Ͳ͏෼͚Ε͹͏·͍͖ͦ͘͏͔෼͔͍ͬͯͨ ΞδϟΠϧʹ׳Ε͍ͯͳ͍ • ΢ΥʔλʔϑΥʔϧͬΆ͘ॎׂΓ։ൃ
26. Our Microservices
27. API gateway ֎෦ϦΫΤετΛ࠷ॳʹॲཧ͢Δड෇৖ REST APIΛఏڙ Ϣʔβͷೝূ౳ ControllerʹϦΫΤετΛ఻͑Δ
28. Controller ͍Ζ͍Ζ൑அͯ͠ࢦࣔΛग़͢Ϙε ϝΠϯDBͱ΍ΓͱΓͰ͖Δ
29. Worker ࣮ࡍʹಇ͘࿑ಇऀ Controller͔ΒͷࢦࣔͰಇ͘ ඇಉظͰ͕͔͔࣌ؒΔॲཧΛ͜ͳ͢ ऴΘͬͨΒControllerʹใࠂ
30. Node agent LB nodeʹજΉΤʔδΣϯτ LB nodeͷॳظઃఆ Worker͔ΒͷࢦࣔͰίϚϯυΛ࣮ߦ ֤छϝτϦΫεΛMonitorʹใࠂ
31. Monitor ؂ࢹ΍՝ۚॲཧΛߦ͏ݟுΓ໾ Agent͔ΒͷϝτϦΫεΛूܭ Կ͔͋Ε͹Controllerʹใࠂ
32. Via RabbitMQ શίϯϙʔωϯτ͕RabbitMQܦ༝Ͱ΍ΓͱΓ 8PSLFS $POUSPMMFS /PEF
    BHFOU ࢦࣔ ࢦࣔ/ใࠂ ใࠂ ඇಉظ ಉظ
33. Good
34. Scalable! εέʔϧΞ΢τ͕؆୯ • ৽͍͠ίϯϙʔωϯτ͕ͭͳ͗ʹ͍͚ͩ͘ RabbitMQ͕ϘτϧωοΫʹͳΓಘΔʁ • ͠͹Β͘৺഑ͳͦ͞͏
35. RabbitMQ is nice ϧʔςΟϯά͕ॊೈ ͱͯ΋ָ: Pub/Sub΍ඇಉظॲཧ ͱͯ΋ָ: ෛՙ෼ࢄ • Producer/Consumer͸ࣗ෼Ͱͭͳ͗ʹ͘Δ • LBͰ෼ࢄ͠Α͏ͱ͢Δͱ…
36. Nice microservices ྑ͍ײ͡ʹίϯϙʔωϯτ෼ׂͰ͖ͨ • ໾ׂ΍ෛՙ ֤୲౰ऀͰҰؾʹฒߦ։ൃͰ͖ͨ 1୆ڞ௨ͷRabbitMQ͕͋Ε͹݁߹ςετ΋؆୯ • ϩʔΧϧͱϩʔΧϧΛ؆୯ʹͭͳ͛Δ
37. Sensu ·͞ʹΫϥ΢υ࣌୅ͷ؂ࢹπʔϧ ͍ͭ͜΋RabbitMQΛ࢖͏ Pub/SubͰ؂ࢹ
38. Not Good
39. Bad microservices ͏·͘Θ͚ΒΕͳ͔ͬͨ෦෼΋ • MonitorͱControllerͰಉ͡DBΈͯΔ ίϯϙʔωϯτԣஅͰΈΔͱDRY͡Όͳ͍ ΠϯϑϥίετͰ͔͍ • ࣍͸ίϯςφԽ͍ͨ͠
40. RPC is not good RPC͍ͬͺ͍࢖ͬͯΔ RabbitMQ͸ຊདྷRPC͢Δ΋ͷ͡Όͳ͍ ͦΕREST APIͰͰ͖ͳ͍ͷʁ • ͸͍ɺͰ͖·͢
41. Insufficient information ৘ใ͕গͳ͍ ެࣜυΩϡϝϯτ͸ॆ࣮ • ӳޠΦϯϦʔ
42. Summary MicroservicesͱPub/SubͰScalableʹʂ✨ MicroservicesͱPub/Subͷ૬ੑ͸ΠΠʂ શ෦RabbitMQܦ༝Ͱ͸ͳ͘దࡐదॴʹʂʂ
43. Thank you for your attention MicroservicesͱPub/SubͰScalableʹʂ✨ MicroservicesͱPub/Subͷ૬ੑ͸ΠΠʂ શ෦RabbitMQܦ༝Ͱ͸ͳ͘దࡐదॴʹʂʂ