Djangoアプリケーションにおけるトイル撲滅戦記.pdf

Transcript

1. DjangoΞϓϦέʔγϣϯʹ͓͚Δ τΠϧ๾໓ઓه

2. ࣗݾ঺հ দా ߁ষ (Yasuaki Matsuda) @yacchin0101 גࣜձࣾΞΠϦοδ αʔόαΠυΤϯδχΞ

3. ࣗݾ঺հ

4. ͲΜͳਓ͕ର৅ʁ τΠϧ๾໓ʹڵຯ͕͋Δ ݱ৔ͰDjangoΛ࢖ͬͯΔ ೔ʑͷӡ༻ۀ຿Ͱർฐ͍ͯ͠Δਓ

5. ໨࣍ લ࢙ τΠϧ๾໓ τΠϧͱ͸ ޮՌ ·ͱΊ

6. େมਃ͠༁͍͟͝·ͤΜ τΠϧΛ๾໓ͨ͠Θ͚Ͱ͸ͳ͍

7. େมਃ͠༁͍͟͝·ͤΜ SRE͸ʮτΠϧͷ๾໓ʯ͕ϝΠϯ

8. τΠϧͱ͸ τΠϧʹ͍ͭͯͷ঺հ photo by Yamato Fukui

9. SREͱ͸ Google͕ࣾఏএɾ࣮ફ͍ͯ͠Δ γεςϜ؅ཧͱαʔϏεӡ༻ͷํ๏࿦

10. SREͱ͸ IUUQTMBOEJOHHPPHMFDPNTSFCPPLJOEFYIUNM 5ষ τΠϧͷ๾໓ τΧήʁ SRE αΠτϦϥΠΞϏϦςΟ ΤϯδχΞϦϯά

11. τΠϧͱ͸ ຊ൪αʔϏεʹؔ͢Δ࡞ۀͰ ख࡞ۀͰ܁Γฦ͠ߦΘΕ ࣗಈԽ͢Δ͜ͱ͕ՄೳͰ͋Γ ઓज़తͰ௕ظతͳՁ஋Λ࣋ͨͣ ࡞ۀྔ͕αʔϏεͷ੒௕ʹൺྫ͢Δ

12. ৯ثͷखચ͍ ਎ۙͳτΠϧ

13. ਎ۙͳτΠϧ ख࡞ۀͰ͋Δ ܁Γฦ͞ΕΔ ࣗಈԽͰ͖Δ ઓज़తͰͳ͍ ௕ظతͳՁ஋Λ࣋ͨͳ͍ αʔϏε੒௕ʹରͯ͠O(n)Ͱͳ͍ ৯ثͷखચ͍

14. ਎ۙͳτΠϧ શࣗಈ৯ચػ

15. τΠϧ͕΋ͨΒ͢΋ͷ ҰॠͰՔಇΛຒΊਚ͘͞ΕΔ ϔΠτΛు͘Α͏ʹͳΔ ΩϟϦΞ͕ఀ଺͢Δ ͳͲ

16. ·ͱΊ τΠϧΛ๾໓ͯ͠ PythonͰΠϊϕʔςΟϒͳ ࢓ࣄΛ͠Α͏ʂ

17. લ࢙ τΠϧ๾໓લͷDjangoΞϓϦέʔγϣϯʹ͍ͭͯͷ঺հ photo by Yamato Fukui

18. લ࢙ Online to Offline

19. લ࢙ དྷళൢଅࢪࡦ Push௨஌ ϙΠϯτ Ϋʔϙϯ ήʔϜ

20. Push௨஌ ͓ɺ෰ങ͍ʹ ߦ͜͏͔ͳɻ ΫϦΞϥϯεηʔϧ ։࠵தʂ

21. ϙΠϯτ ϙΠϯτ໯͑Δ͠ ߦ͜͏͔ͳɻ དྷళϙΠϯτ Ωϟϯϖʔϯ࣮ࢪதʂ

22. Ϋʔϙϯ ͋ͷళͷΫʔϙϯ ͋Δ͡ΌΜɻ 10%OFFΫʔϙϯ ഑෍தʂ

23. ήʔϜ ΍ͬͨͥ ͓ΊͰͱ͏ʂ 100PTήοτʂ

24. લ࢙ hogeΞϓϦ Push௨஌ ϙΠϯτ Ϋʔϙϯ ήʔϜ ܾࡁ

25. Offline ͓ళͷηʔϧ৘ใΛݟΔ ϙΠϯτ࢒ߴΛ֬ೝ͢Δ ήʔϜΛ࣮ࢪͯ͠ϙΠϯτ֫ಘ

26. Online ͓ళͰΫʔϙϯΛ࢖͏ ΞϓϦͰܾࡁ͢Δ

27. લ࢙ hogeΞϓϦͱࢲͷؔ܎ αʔόαΠυػೳ࣮૷ Πϯϑϥߏங ӡ༻ɾอक୲౰ऀ

28. લ࢙ CPU࢖༻཰ Push௨஌

29. લ࢙ ֎஫࣌୅ ಺੡࣌୅ τΠϧ๾໓࣌୅ 2015೥ 2016೥ 2018೥

30. ֎஫࣌୅ ελʔτΞοϓاۀ ։ൃऀෆ଍ ֎஫

31. ʢ2015.x.xʣ ߏ੒ App & Batch Server Admin Server Coupon Server

    ELB ELB

32. ֎஫࣌୅ ຯΘ͍ਂ͍ߏ੒ DBಉډ໰୊ SSLূ໌ॻ௚ஔ͖໰୊ σϓϩΠ: SCP ͳͲ

33. ઌํʮΞϓϦͷػೳ֦ு͍ͨ͠ʯ ֎஫࣌୅ Ӧۀʮঝ஌͍ͨ͠·ͨ͠ʂʯ ೖࣾͨ͠দాʮΑΖ͓͘͠Ͷ͕͍͠·͢ʂʯ PMʮࠓޙͷ֦ுΛߟ͑ͯ಺੡͢Δ͔ʯ

34. ಺੡࣌୅ ࣾ಺։ൃ Python / Django Ͱ։ൃ GitLabͰιʔε؅ཧ AWSͰߏங

35. ʢ2016.x.xʣ ߏ੒ App Server Admin Server Coupon Server RDS ELB

    ELB Batch Server

36. ಺੡࣌୅ ଟগ͸վળ͞Εͨ RDS(ϚωʔδυαʔϏε) PHP͔ΒDjango΁Ҡߦ σϓϩΠ: Git Pull ͳͲ

37. ϩʔϯνޙ ઌํʮ͜͜ͷจݴमਖ਼Λ͓ئ͍Ͱ͖·͢ʁʯ PMʮঝ஌͠·ͨ͠ɻদా͞Μ͓ئ͍͠·͢ɻʯ দాʮ͸͍ɻʯ

38. ಺੡࣌୅ ͋Εʁ ʮ͜ͷલ΋ґཔ͞ΕͨΑ͏ͳʯ ʮ࠷ۙɺ։ൃͰ͖͍ͯͳ͍ʯ ʮਓ͕଍Γͳ͍ʯ

39. ಺੡࣌୅ τΠϧʹ૶ૡ͖ۤ͠Ή࣌୅ͷ౸དྷ

40. τΠϧ๾໓ τΠϧ๾໓ͷաఔʹ͍ͭͯ঺հ photo by Yamato Fukui

41. ํ਑ Twelve-Factor AppΛ Managed ServiceΛ CI / CD

42. ํ਑

43. Twelve-Factor App ࣌ؒɾίετͷ࠷খԽ Ҡ২ੑ޲্ Πϯϑϥ؅ཧ͔Βͷ։์ ܧଓతσϓϩΠ εέʔϧΞοϓ IUUQTGBDUPSOFU

44. Managed Service Πϯϑϥ؅ཧෆཁ ӡ༻ϦεΫ௿ݮ ܧଓతվળ

45. CI / CD ख࡞ۀ͔Βͷ։์ ඼࣭޲্

46. τΠϧ๾໓ Case 1

47. τΠϧ๾໓લͷঢ়گ ؀ڥߏங࡞ۀ ৽͍͠ΤϯδχΞ͕ࢀը README௨Γʹ؀ڥߏங ಈ͔ͳ͍ ৬ਓͷग़ਞ

48. ػೳ։ൃ։࢝·Ͱʹ͔͔࣌ؒΔ ৬ਓͷՔಇ͕औΒΕΔ τΠϧ๾໓લͷঢ়گ ՝୊

49. τΠϧ๾໓ઓུ Docker Compose

50. τΠϧ๾໓ઓུ ίϯςφܕԾ૝؀ڥ ؀ڥࠩҟ͕খ͘͞ͳΔ Ҡ২ੑ͕ߴ͍ Docker

51. τΠϧ๾໓ઓུ Docker Compose ෳ਺ίϯςφΛఆٛ ෳ਺ίϯςφΛ࣮ߦ docker-compose up ͷΈ

52. ͙͢։ൃͰ͖ΔΑ͏ʹͳͬͨ ৬ਓͷग़ਞ͕཈͑ΒΕͨ ؀ڥࠩҟʹΑΔόά͕཈͑ΒΕͨ τΠϧ๾໓ޙ վળޮՌ

53. τΠϧ๾໓ޙ

54. τΠϧ๾໓ Case 2

55. खॱॻͷ࡞੒ 1୆ͣͭॖୀɾߋ৽ νΣοΧʔͱෳ਺ਓମ੍ σϓϩΠ࡞ۀ τΠϧ๾໓લͷঢ়گ

56. σϓϩΠखॱ൥ࡶʹΑΔ ώϡʔϚϯΤϥʔϦεΫ σϓϩΠʹର͢Δ޻਺͕ߴΊ τΠϧ๾໓લͷঢ়گ ՝୊

57. τΠϧ๾໓ઓུ GitLab CI Amazon ECS

58. τΠϧ๾໓ઓུ ίϯςφΦʔέετϨʔγϣϯ αʔό؅ཧෆཁ εέδϡʔϧ࣮ߦ Amazon ECS

59. τΠϧ๾໓ઓུ GitϦϙδτϦ CI / CDػೳ issue؅ཧ ίʔυϨϏϡʔ GitLab

60. RDS ELB X-Ray ECS App Server Service X-Ray Batch Task

    ELB ELB ɾɾɾ X-Ray X-Ray Coupon Server Service ɾɾɾ X-Ray X-Ray Admin Server Service ɾɾɾ X-Ray CloudWatch Logs S3 τΠϧ๾໓ઓུ

61. ιʔεमਖ਼ ϨϏϡʔ ϙνΔ τΠϧ๾໓ޙ σϓϩΠखॱ

62. ਓҝతϛε͕ൃੜ͠ͳ͘ͳͬͨ σϓϩΠʹର͢Δ޻਺͕ݮͬͨ վળޮՌ τΠϧ๾໓ޙ

63. /hoge/ͰPOST࣌͸IP੍ݶ͍ͨ͠ τΠϧ๾໓ޙ

64. τΠϧ๾໓ޙ SQLΫΤϦվળ

65. ECR X-Ray APP Server Service X-Ray New Task Old Task

    1. Image Build & Push New Image 2. Create New Task Definition 3. Update Service 1. Push Source Code X-Ray App Task: 1 X-Ray App Task: 2 GitLab 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR τΠϧ๾໓ޙ

66. τΠϧ๾໓ ͓·͚

67. τΠϧ๾໓લͷঢ়گ ੩తϑΝΠϧσϓϩΠ ιʔεमਖ਼ ϨϏϡʔ ϙνΔ

68. ɻΛ෇༩͍ͨ͠ τΠϧ๾໓ޙ

69. 1. npm install & build 2. Put Artifacts to S3

    Bucket 1. Push Source Code GitLab 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR AWS CLI τΠϧ๾໓ޙ

70. τΠϧ๾໓ ߋʹ͓·͚

71. Ϣχοτςετ ίʔσΟϯάن໿֬ೝ ෳࡶ౓֬ೝ τΠϧ๾໓લͷঢ়گ ιʔείʔυϨϏϡ࡞ۀ

72. 1. Unit Test 2. Flake8 3. Radon GitLab ςετ࣮ࢪʹΑΔ඼࣭୲อ ίʔσΟϯάن໿ͷ֬ೝ

    ίʔυͷෳࡶ౓Λ֬ೝ 1. Push Source Code 2. Create Merge Request(MR) 3. Review & LGTM & Accept MR τΠϧ๾໓ޙ

73. τΠϧ๾໓ޙ

74. τΠϧ๾໓ ͓·͚ (͜ΕͰ࠷ޙ)

75. τΠϧ๾໓લͷঢ়گ όοναʔόෆཁ ECSͰఆظతʹ࣮ߦͰ͖Δ ࢖ͬͨ෼͚ͩͷྉۚ

76. τΠϧ๾໓ޙ

77. τΠϧ๾໓ Case 3

78. τΠϧ๾໓લͷঢ়گ ֤αʔό͔ΒΞΫηεϩάऔಘ grepɾawk৬ਓ ಛఆϢʔβͷΞΫηεௐࠪ

79. ௐࠪʹ͕͔͔࣌ؒΔ ώϡʔϚϯΤϥʔʹΑΔϦεΫ τΠϧ๾໓લͷঢ়گ ՝୊

80. τΠϧ๾໓ઓུ CloudWatch Logs

81. τΠϧ๾໓ઓུ ϩάͷ؂ࢹ ϩάͷอଘ ϩά΁ͷΞΫηε CloudWatch Logs

82. τΠϧ๾໓ઓུ settings.py

83. τΠϧ๾໓ઓུ docker-compose.yml

84. ਺෼Ͱௐࠪ׬ྃ ਓҝతϛεʹΑΔαʔϏεఀࢭ͕ͳ͍ ඇΤϯδχΞͰ΋ௐࠪՄೳ τΠϧ๾໓ վળޮՌ

85. τΠϧ๾໓ޙ ϩάΛҰՕॴʹू໿ લํҰகݕࡧ ೔࣌ݕࡧ

86. τΠϧ๾໓ޙ

87. τΠϧ๾໓ Case 4

88. τΠϧ๾໓લͷঢ়گ RDSεέʔϧΞοϓ࡞ۀ ࣮ߦ͞ΕΔSQL͕ѱ͍ CPU࢖༻཰100%ுΓ෇͖ αʔϏεμ΢ϯൃੜ ۚͰղܾ

89. αʔϏεμ΢ϯͷՄೳੑ αʔϏεҡ࣋ʹίετ͕͔͔Δ ࣮૷༧ఆͩͬͨ΋ͷ͕஗Ԇ τΠϧ๾໓લͷঢ়گ ՝୊

90. τΠϧ๾໓ઓུ Django QuerySet

91. σʔλϕʔεͷ҆ఆՔಇ ༨ܭͳεέʔϧΞοϓΛ཈͑Δ վળޮՌ τΠϧ๾໓ޙ

92. ήʔϜ࣮ࢪՄ൱API ήʔϜ͸1೔1ճ·Ͱ ΞϓϦશମͰ1೔ʹ10,000ճ τΠϧ๾໓ޙ

93. τΠϧ๾໓ޙ

94. τΠϧ๾໓ޙ

95. PushͰήʔϜར༻ऀ૿Ճ ϨεϙϯεྼԽ ΠϯελϯεεέʔϧΞοϓ τΠϧ๾໓ޙ

96. τΠϧ๾໓ޙ

97. τΠϧ๾໓ޙ

98. τΠϧ๾໓ޙ

99. τΠϧ๾໓ Case 5

100. τΠϧ๾໓લͷঢ়گ ؅ཧαʔόʹϩάΠϯͰ͖ͳ͍໰୊ͷௐࠪ ϩάௐࠪ ϩάʹు͔Ε͍ͯͳ͍ ιʔεΛݟΔ

101. ໰୊ղܾʹ͕͔͔࣌ؒΔ τΠϧ๾໓લͷঢ়گ ՝୊

102. τΠϧ๾໓લͷঢ়گ

103. τΠϧ๾໓લͷঢ়گ Effective Python
 Item 14: Prefer Exceptions to returning None

     νϟϦ Effective Python —

104. τΠϧ๾໓લͷঢ়گ ྑ͘ͳ͍఺ except۟ͰϩΪϯά͍ͯ͠ͳ͍ except۟ͰNoneΛฦ٫

105. τΠϧ๾໓ઓུ Django Logging

106. ϩάΛݟΔ͚ͩͰঢ়گ೺Ѳ τΠϧ๾໓ վળޮՌ

107. τΠϧ๾໓ޙ

108. ྑ͘ͳͬͨ఺ ϩάͰঢ়گΛ೺ѲͰ͖Δ ಛผͳྫ֎Λ౤͛Δ͜ͱͰ ΍Γ͍ͨ͜ͱΛ໌֬ʹͨ͠ τΠϧ๾໓ޙ

109. τΠϧ๾໓ Case 6

110. τΠϧ๾໓લͷঢ়گ ֤APIͷϨΠςϯγௐࠪ ֤αʔό͔ΒΞΫηεϩάऔಘ grepɾawk৬ਓ

111. ͕͔͔࣌ؒΔ ϦΞϧλΠϜʹΘ͔Βͳ͍ τΠϧ๾໓લͷঢ়گ ՝୊

112. τΠϧ๾໓ઓུ AWS X-Ray

113. τΠϧ๾໓ઓུ ϦΫΤετʹؔ͢Δσʔλऩू σʔλͷදࣔɾϑΟϧλϦϯά தؒͷαʔϏεৄࡉ΋දࣔ AWS X-Ray

114. τΠϧ๾໓ઓུ X-Ray X-Ray Daemon

115. τΠϧ๾໓ઓུ settings.py

116. τΠϧ๾໓ઓུ Dockerfile

117. ຊ൪؀ڥͷঢ়گ͕೺ѲՄೳ ύϑΥʔϚϯενϡʔχϯά˕ ো֐ରԠͰ໾ʹཱͭ վળޮՌ τΠϧ๾໓ޙ

118. τΠϧ๾໓ޙ

119. τΠϧ๾໓ޙ

120. τΠϧ๾໓ޙ

121. τΠϧ๾໓ޙ

122. ޮՌ τΠϧ๾໓ʹΑΔޮՌͷ঺հ photo by Yamato Fukui

123. ޮՌ ো֐ରԠ͕ݮͬͨ ख࡞ۀݮʹ൐͏ώϡʔϚϯΤϥʔ͕গͳ͘ͳͬͨ͜ͱ͕େ͖͍ ΤϯδχΞϦϯάʹׂ͚Δ͕࣌ؒ૿͑ͨ DjangoΞϓϦέʔγϣϯ։ൃʹ஫ྗͰ͖ΔΑ͏ʹͳͬͨ ΩϟϦΞΞοϓͰ͖ͨ ϦʔμʹͳΓ·ͨ͠

124. ·ͱΊ photo by Yamato Fukui

125. ·ͱΊ ʮͦΕτΠϧ͡ΌͶʁʯͷਫ਼ਆ τΠϧΛݮΒͤ͹ΤϯδχΞϦϯάʹूதͰ͖Δ ܧଓͯ͠վળͰ͖Δঢ়ଶΛ࡞Δ τΠϧ͸ແݶʹ༙͖ଓ͚Δ Managed ServiceɾCI / CDͷ׆༻ ؅ཧର৅ΛݮΒ͠ɺࣗಈԽΛਪਐ͢Δ

126. ·ͱΊ τΠϧΛ๾໓ͯ͠ PythonͰΠϊϕʔςΟϒͳ ࢓ࣄΛ͠Α͏ʂ

127. ࢀߟจݙ O'Reilly Japan - SRE αΠτϦϥΠΞϏϦςΟΤϯδχΞϦϯά O'Reilly Japan - Effective

     Python