Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Djangoアプリケーションにおけるトイル撲滅戦記.pdf

443954f10bd971cbb1dd4599cf6e1099?s=47 Yasuaki Matsuda
September 18, 2018

 Djangoアプリケーションにおけるトイル撲滅戦記.pdf

443954f10bd971cbb1dd4599cf6e1099?s=128

Yasuaki Matsuda

September 18, 2018
Tweet

Transcript

  1. DjangoΞϓϦέʔγϣϯʹ͓͚Δ τΠϧ๾໓ઓه

  2. ࣗݾ঺հ দా ߁ষ (Yasuaki Matsuda) @yacchin0101 גࣜձࣾΞΠϦοδ αʔόαΠυΤϯδχΞ

  3. ࣗݾ঺հ

  4. ͲΜͳਓ͕ର৅ʁ τΠϧ๾໓ʹڵຯ͕͋Δ ݱ৔ͰDjangoΛ࢖ͬͯΔ ೔ʑͷӡ༻ۀ຿Ͱർฐ͍ͯ͠Δਓ

  5. ໨࣍ લ࢙ τΠϧ๾໓ τΠϧͱ͸ ޮՌ ·ͱΊ

  6. େมਃ͠༁͍͟͝·ͤΜ τΠϧΛ๾໓ͨ͠Θ͚Ͱ͸ͳ͍

  7. େมਃ͠༁͍͟͝·ͤΜ SRE͸ʮτΠϧͷ๾໓ʯ͕ϝΠϯ

  8. τΠϧͱ͸ τΠϧʹ͍ͭͯͷ঺հ photo by Yamato Fukui

  9. SREͱ͸ Google͕ࣾఏএɾ࣮ફ͍ͯ͠Δ γεςϜ؅ཧͱαʔϏεӡ༻ͷํ๏࿦

  10. SREͱ͸ IUUQTMBOEJOHHPPHMFDPNTSFCPPLJOEFYIUNM 5ষ τΠϧͷ๾໓ τΧήʁ SRE αΠτϦϥΠΞϏϦςΟ ΤϯδχΞϦϯά

  11. τΠϧͱ͸ ຊ൪αʔϏεʹؔ͢Δ࡞ۀͰ ख࡞ۀͰ܁Γฦ͠ߦΘΕ ࣗಈԽ͢Δ͜ͱ͕ՄೳͰ͋Γ ઓज़తͰ௕ظతͳՁ஋Λ࣋ͨͣ ࡞ۀྔ͕αʔϏεͷ੒௕ʹൺྫ͢Δ

  12. ৯ثͷखચ͍ ਎ۙͳτΠϧ

  13. ਎ۙͳτΠϧ ख࡞ۀͰ͋Δ ܁Γฦ͞ΕΔ ࣗಈԽͰ͖Δ ઓज़తͰͳ͍ ௕ظతͳՁ஋Λ࣋ͨͳ͍ αʔϏε੒௕ʹରͯ͠O(n)Ͱͳ͍ ৯ثͷखચ͍

  14. ਎ۙͳτΠϧ શࣗಈ৯ચػ

  15. τΠϧ͕΋ͨΒ͢΋ͷ ҰॠͰՔಇΛຒΊਚ͘͞ΕΔ ϔΠτΛు͘Α͏ʹͳΔ ΩϟϦΞ͕ఀ଺͢Δ ͳͲ

  16. ·ͱΊ τΠϧΛ๾໓ͯ͠ PythonͰΠϊϕʔςΟϒͳ ࢓ࣄΛ͠Α͏ʂ

  17. લ࢙ τΠϧ๾໓લͷDjangoΞϓϦέʔγϣϯʹ͍ͭͯͷ঺հ photo by Yamato Fukui

  18. લ࢙ Online to Offline

  19. લ࢙ དྷళൢଅࢪࡦ Push௨஌ ϙΠϯτ Ϋʔϙϯ ήʔϜ

  20. Push௨஌ ͓ɺ෰ങ͍ʹ ߦ͜͏͔ͳɻ ΫϦΞϥϯεηʔϧ ։࠵தʂ

  21. ϙΠϯτ ϙΠϯτ໯͑Δ͠ ߦ͜͏͔ͳɻ དྷళϙΠϯτ Ωϟϯϖʔϯ࣮ࢪதʂ

  22. Ϋʔϙϯ ͋ͷళͷΫʔϙϯ ͋Δ͡ΌΜɻ 10%OFFΫʔϙϯ ഑෍தʂ

  23. ήʔϜ ΍ͬͨͥ ͓ΊͰͱ͏ʂ 100PTήοτʂ

  24. લ࢙ hogeΞϓϦ Push௨஌ ϙΠϯτ Ϋʔϙϯ ήʔϜ ܾࡁ

  25. Offline ͓ళͷηʔϧ৘ใΛݟΔ ϙΠϯτ࢒ߴΛ֬ೝ͢Δ ήʔϜΛ࣮ࢪͯ͠ϙΠϯτ֫ಘ

  26. Online ͓ళͰΫʔϙϯΛ࢖͏ ΞϓϦͰܾࡁ͢Δ

  27. લ࢙ hogeΞϓϦͱࢲͷؔ܎ αʔόαΠυػೳ࣮૷ Πϯϑϥߏங ӡ༻ɾอक୲౰ऀ

  28. લ࢙ CPU࢖༻཰ Push௨஌

  29. લ࢙ ֎஫࣌୅ ಺੡࣌୅ τΠϧ๾໓࣌୅ 2015೥ 2016೥ 2018೥

  30. ֎஫࣌୅ ελʔτΞοϓاۀ ։ൃऀෆ଍ ֎஫

  31. ʢ2015.x.xʣ ߏ੒ App & Batch Server Admin Server Coupon Server

    ELB ELB
  32. ֎஫࣌୅ ຯΘ͍ਂ͍ߏ੒ DBಉډ໰୊ SSLূ໌ॻ௚ஔ͖໰୊ σϓϩΠ: SCP ͳͲ

  33. ઌํʮΞϓϦͷػೳ֦ு͍ͨ͠ʯ ֎஫࣌୅ Ӧۀʮঝ஌͍ͨ͠·ͨ͠ʂʯ ೖࣾͨ͠দాʮΑΖ͓͘͠Ͷ͕͍͠·͢ʂʯ PMʮࠓޙͷ֦ுΛߟ͑ͯ಺੡͢Δ͔ʯ

  34. ಺੡࣌୅ ࣾ಺։ൃ Python / Django Ͱ։ൃ GitLabͰιʔε؅ཧ AWSͰߏங

  35. ʢ2016.x.xʣ ߏ੒ App Server Admin Server Coupon Server RDS ELB

    ELB Batch Server
  36. ಺੡࣌୅ ଟগ͸վળ͞Εͨ RDS(ϚωʔδυαʔϏε) PHP͔ΒDjango΁Ҡߦ σϓϩΠ: Git Pull ͳͲ

  37. ϩʔϯνޙ ઌํʮ͜͜ͷจݴमਖ਼Λ͓ئ͍Ͱ͖·͢ʁʯ PMʮঝ஌͠·ͨ͠ɻদా͞Μ͓ئ͍͠·͢ɻʯ দాʮ͸͍ɻʯ

  38. ಺੡࣌୅ ͋Εʁ ʮ͜ͷલ΋ґཔ͞ΕͨΑ͏ͳʯ ʮ࠷ۙɺ։ൃͰ͖͍ͯͳ͍ʯ ʮਓ͕଍Γͳ͍ʯ

  39. ಺੡࣌୅ τΠϧʹ૶ૡ͖ۤ͠Ή࣌୅ͷ౸དྷ

  40. τΠϧ๾໓ τΠϧ๾໓ͷաఔʹ͍ͭͯ঺հ photo by Yamato Fukui

  41. ํ਑ Twelve-Factor AppΛ Managed ServiceΛ CI / CD

  42. ํ਑

  43. Twelve-Factor App ࣌ؒɾίετͷ࠷খԽ Ҡ২ੑ޲্ Πϯϑϥ؅ཧ͔Βͷ։์ ܧଓతσϓϩΠ εέʔϧΞοϓ IUUQTGBDUPSOFU

  44. Managed Service Πϯϑϥ؅ཧෆཁ ӡ༻ϦεΫ௿ݮ ܧଓతվળ

  45. CI / CD ख࡞ۀ͔Βͷ։์ ඼࣭޲্

  46. τΠϧ๾໓ Case 1

  47. τΠϧ๾໓લͷঢ়گ ؀ڥߏங࡞ۀ ৽͍͠ΤϯδχΞ͕ࢀը README௨Γʹ؀ڥߏங ಈ͔ͳ͍ ৬ਓͷग़ਞ

  48. ػೳ։ൃ։࢝·Ͱʹ͔͔࣌ؒΔ ৬ਓͷՔಇ͕औΒΕΔ τΠϧ๾໓લͷঢ়گ ՝୊

  49. τΠϧ๾໓ઓུ Docker Compose

  50. τΠϧ๾໓ઓུ ίϯςφܕԾ૝؀ڥ ؀ڥࠩҟ͕খ͘͞ͳΔ Ҡ২ੑ͕ߴ͍ Docker

  51. τΠϧ๾໓ઓུ Docker Compose ෳ਺ίϯςφΛఆٛ ෳ਺ίϯςφΛ࣮ߦ docker-compose up ͷΈ

  52. ͙͢։ൃͰ͖ΔΑ͏ʹͳͬͨ ৬ਓͷग़ਞ͕཈͑ΒΕͨ ؀ڥࠩҟʹΑΔόά͕཈͑ΒΕͨ τΠϧ๾໓ޙ վળޮՌ

  53. τΠϧ๾໓ޙ

  54. τΠϧ๾໓ Case 2

  55. खॱॻͷ࡞੒ 1୆ͣͭॖୀɾߋ৽ νΣοΧʔͱෳ਺ਓମ੍ σϓϩΠ࡞ۀ τΠϧ๾໓લͷঢ়گ

  56. σϓϩΠखॱ൥ࡶʹΑΔ ώϡʔϚϯΤϥʔϦεΫ σϓϩΠʹର͢Δ޻਺͕ߴΊ τΠϧ๾໓લͷঢ়گ ՝୊

  57. τΠϧ๾໓ઓུ GitLab CI Amazon ECS

  58. τΠϧ๾໓ઓུ ίϯςφΦʔέετϨʔγϣϯ αʔό؅ཧෆཁ εέδϡʔϧ࣮ߦ Amazon ECS

  59. τΠϧ๾໓ઓུ GitϦϙδτϦ CI / CDػೳ issue؅ཧ ίʔυϨϏϡʔ GitLab

  60. RDS ELB X-Ray ECS App Server Service X-Ray Batch Task

    ELB ELB ɾɾɾ X-Ray X-Ray Coupon Server Service ɾɾɾ X-Ray X-Ray Admin Server Service ɾɾɾ X-Ray CloudWatch Logs S3 τΠϧ๾໓ઓུ
  61. ιʔεमਖ਼ ϨϏϡʔ ϙνΔ τΠϧ๾໓ޙ σϓϩΠखॱ

  62. ਓҝతϛε͕ൃੜ͠ͳ͘ͳͬͨ σϓϩΠʹର͢Δ޻਺͕ݮͬͨ վળޮՌ τΠϧ๾໓ޙ

  63. /hoge/ͰPOST࣌͸IP੍ݶ͍ͨ͠ τΠϧ๾໓ޙ

  64. τΠϧ๾໓ޙ SQLΫΤϦվળ

  65. ECR X-Ray APP Server Service X-Ray New Task Old Task

    1. Image Build & Push New Image 2. Create New Task Definition 3. Update Service 1. Push Source Code X-Ray App Task: 1 X-Ray App Task: 2 GitLab 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR τΠϧ๾໓ޙ
  66. τΠϧ๾໓ ͓·͚

  67. τΠϧ๾໓લͷঢ়گ ੩తϑΝΠϧσϓϩΠ ιʔεमਖ਼ ϨϏϡʔ ϙνΔ

  68. ɻΛ෇༩͍ͨ͠ τΠϧ๾໓ޙ

  69. 1. npm install & build 2. Put Artifacts to S3

    Bucket 1. Push Source Code GitLab 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR AWS CLI τΠϧ๾໓ޙ
  70. τΠϧ๾໓ ߋʹ͓·͚

  71. Ϣχοτςετ ίʔσΟϯάن໿֬ೝ ෳࡶ౓֬ೝ τΠϧ๾໓લͷঢ়گ ιʔείʔυϨϏϡ࡞ۀ

  72. 1. Unit Test 2. Flake8 3. Radon GitLab ςετ࣮ࢪʹΑΔ඼࣭୲อ ίʔσΟϯάن໿ͷ֬ೝ

    ίʔυͷෳࡶ౓Λ֬ೝ 1. Push Source Code 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR τΠϧ๾໓ޙ
  73. τΠϧ๾໓ޙ

  74. τΠϧ๾໓ ͓·͚ (͜ΕͰ࠷ޙ)

  75. τΠϧ๾໓લͷঢ়گ όοναʔόෆཁ ECSͰఆظతʹ࣮ߦͰ͖Δ ࢖ͬͨ෼͚ͩͷྉۚ

  76. τΠϧ๾໓ޙ

  77. τΠϧ๾໓ Case 3

  78. τΠϧ๾໓લͷঢ়گ ֤αʔό͔ΒΞΫηεϩάऔಘ grepɾawk৬ਓ ಛఆϢʔβͷΞΫηεௐࠪ

  79. ௐࠪʹ͕͔͔࣌ؒΔ ώϡʔϚϯΤϥʔʹΑΔϦεΫ τΠϧ๾໓લͷঢ়گ ՝୊

  80. τΠϧ๾໓ઓུ CloudWatch Logs

  81. τΠϧ๾໓ઓུ ϩάͷ؂ࢹ ϩάͷอଘ ϩά΁ͷΞΫηε CloudWatch Logs

  82. τΠϧ๾໓ઓུ settings.py

  83. τΠϧ๾໓ઓུ docker-compose.yml

  84. ਺෼Ͱௐࠪ׬ྃ ਓҝతϛεʹΑΔαʔϏεఀࢭ͕ͳ͍ ඇΤϯδχΞͰ΋ௐࠪՄೳ τΠϧ๾໓ վળޮՌ

  85. τΠϧ๾໓ޙ ϩάΛҰՕॴʹू໿ લํҰகݕࡧ ೔࣌ݕࡧ

  86. τΠϧ๾໓ޙ

  87. τΠϧ๾໓ Case 4

  88. τΠϧ๾໓લͷঢ়گ RDSεέʔϧΞοϓ࡞ۀ ࣮ߦ͞ΕΔSQL͕ѱ͍ CPU࢖༻཰100%ுΓ෇͖ αʔϏεμ΢ϯൃੜ ۚͰղܾ

  89. αʔϏεμ΢ϯͷՄೳੑ αʔϏεҡ࣋ʹίετ͕͔͔Δ ࣮૷༧ఆͩͬͨ΋ͷ͕஗Ԇ τΠϧ๾໓લͷঢ়گ ՝୊

  90. τΠϧ๾໓ઓུ Django QuerySet

  91. σʔλϕʔεͷ҆ఆՔಇ ༨ܭͳεέʔϧΞοϓΛ཈͑Δ վળޮՌ τΠϧ๾໓ޙ

  92. ήʔϜ࣮ࢪՄ൱API ήʔϜ͸1೔1ճ·Ͱ ΞϓϦશମͰ1೔ʹ10,000ճ τΠϧ๾໓ޙ

  93. τΠϧ๾໓ޙ

  94. τΠϧ๾໓ޙ

  95. PushͰήʔϜར༻ऀ૿Ճ ϨεϙϯεྼԽ ΠϯελϯεεέʔϧΞοϓ τΠϧ๾໓ޙ

  96. τΠϧ๾໓ޙ

  97. τΠϧ๾໓ޙ

  98. τΠϧ๾໓ޙ

  99. τΠϧ๾໓ Case 5

  100. τΠϧ๾໓લͷঢ়گ ؅ཧαʔόʹϩάΠϯͰ͖ͳ͍໰୊ͷௐࠪ ϩάௐࠪ ϩάʹు͔Ε͍ͯͳ͍ ιʔεΛݟΔ

  101. ໰୊ղܾʹ͕͔͔࣌ؒΔ τΠϧ๾໓લͷঢ়گ ՝୊

  102. τΠϧ๾໓લͷঢ়گ

  103. τΠϧ๾໓લͷঢ়گ Effective Python
 Item 14: Prefer Exceptions to returning None

    νϟϦ Effective Python —
  104. τΠϧ๾໓લͷঢ়گ ྑ͘ͳ͍఺ except۟ͰϩΪϯά͍ͯ͠ͳ͍ except۟ͰNoneΛฦ٫

  105. τΠϧ๾໓ઓུ Django Logging

  106. ϩάΛݟΔ͚ͩͰঢ়گ೺Ѳ τΠϧ๾໓ վળޮՌ

  107. τΠϧ๾໓ޙ

  108. ྑ͘ͳͬͨ఺ ϩάͰঢ়گΛ೺ѲͰ͖Δ ಛผͳྫ֎Λ౤͛Δ͜ͱͰ ΍Γ͍ͨ͜ͱΛ໌֬ʹͨ͠ τΠϧ๾໓ޙ

  109. τΠϧ๾໓ Case 6

  110. τΠϧ๾໓લͷঢ়گ ֤APIͷϨΠςϯγௐࠪ ֤αʔό͔ΒΞΫηεϩάऔಘ grepɾawk৬ਓ

  111. ͕͔͔࣌ؒΔ ϦΞϧλΠϜʹΘ͔Βͳ͍ τΠϧ๾໓લͷঢ়گ ՝୊

  112. τΠϧ๾໓ઓུ AWS X-Ray

  113. τΠϧ๾໓ઓུ ϦΫΤετʹؔ͢Δσʔλऩू σʔλͷදࣔɾϑΟϧλϦϯά தؒͷαʔϏεৄࡉ΋දࣔ AWS X-Ray

  114. τΠϧ๾໓ઓུ X-Ray X-Ray Daemon

  115. τΠϧ๾໓ઓུ settings.py

  116. τΠϧ๾໓ઓུ Dockerfile

  117. ຊ൪؀ڥͷঢ়گ͕೺ѲՄೳ ύϑΥʔϚϯενϡʔχϯά˕ ো֐ରԠͰ໾ʹཱͭ վળޮՌ τΠϧ๾໓ޙ

  118. τΠϧ๾໓ޙ

  119. τΠϧ๾໓ޙ

  120. τΠϧ๾໓ޙ

  121. τΠϧ๾໓ޙ

  122. ޮՌ τΠϧ๾໓ʹΑΔޮՌͷ঺հ photo by Yamato Fukui

  123. ޮՌ ো֐ରԠ͕ݮͬͨ ख࡞ۀݮʹ൐͏ώϡʔϚϯΤϥʔ͕গͳ͘ͳͬͨ͜ͱ͕େ͖͍ ΤϯδχΞϦϯάʹׂ͚Δ͕࣌ؒ૿͑ͨ DjangoΞϓϦέʔγϣϯ։ൃʹ஫ྗͰ͖ΔΑ͏ʹͳͬͨ ΩϟϦΞΞοϓͰ͖ͨ ϦʔμʹͳΓ·ͨ͠

  124. ·ͱΊ photo by Yamato Fukui

  125. ·ͱΊ ʮͦΕτΠϧ͡ΌͶʁʯͷਫ਼ਆ τΠϧΛݮΒͤ͹ΤϯδχΞϦϯάʹूதͰ͖Δ ܧଓͯ͠վળͰ͖Δঢ়ଶΛ࡞Δ τΠϧ͸ແݶʹ༙͖ଓ͚Δ Managed ServiceɾCI / CDͷ׆༻ ؅ཧର৅ΛݮΒ͠ɺࣗಈԽΛਪਐ͢Δ

  126. ·ͱΊ τΠϧΛ๾໓ͯ͠ PythonͰΠϊϕʔςΟϒͳ ࢓ࣄΛ͠Α͏ʂ

  127. ࢀߟจݙ O'Reilly Japan - SRE αΠτϦϥΠΞϏϦςΟΤϯδχΞϦϯά O'Reilly Japan - Effective

    Python