A Pragmatic Approach to Rails Deployments and Operations

Transcript

1. Lessons learned from deploying and managing hundreds of Rails applications

   A Pragmatic Approach to Rails Deployments and Operations Monday, May 28, 12

2. Lessons learned from deploying and managing hundreds of Rails applications

   Rails Deployments and Operations Monday, May 28, 12

3. o hai, im josh Monday, May 28, 12

4. •@techpickles •http://github.com/technicalpickles •http://technicalpickles.com Monday, May 28, 12

5. Monday, May 28, 12

6. Awesomeness Engineer of Supreme Versatility III Monday, May 28, 12

7. Managed hosting and operations Monday, May 28, 12

8. I’m talking about Rails Operations Monday, May 28, 12

9. WTF is Rails Operations? Monday, May 28, 12

10. Development and maintenance of a production Rails application Monday, May

    28, 12

11. Before we dig in too deep... Monday, May 28, 12

12. Monday, May 28, 12

13. Does development generate revenue? Question Monday, May 28, 12

14. Monday, May 28, 12

15. •Takes place on laptops, desktop machines, staging servers Monday, May

    28, 12

16. •Takes place on laptops, desktop machines, staging servers •No real

    users Monday, May 28, 12

17. •Takes place on laptops, desktop machines, staging servers •No real

    users •Unknown if it truly works Monday, May 28, 12

18. •Takes place on laptops, desktop machines, staging servers •No real

    users •Unknown if it truly works •Tests are green, but... Monday, May 28, 12

19. Monday, May 28, 12

20. •Step 1: Development Monday, May 28, 12

21. •Step 1: Development •Step 2: Operations Monday, May 28, 12

22. •Step 1: Development •Step 2: Operations •Step 3: ...... Monday,

    May 28, 12

23. •Step 1: Development •Step 2: Operations •Step 3: ...... •Step

    4: PROFIT! Monday, May 28, 12

24. NO Monday, May 28, 12

25. but it CREATES potential revenue Monday, May 28, 12

26. Monday, May 28, 12

27. •Step 1: Development Monday, May 28, 12

28. •Step 1: Development •Step 2: ....... Monday, May 28, 12

29. •Step 1: Development •Step 2: ....... •Step 3: PROFIT Monday,

    May 28, 12

30. Does operations generate revenue? Question Monday, May 28, 12

31. Monday, May 28, 12

32. •Lives on servers located in data centers and clouds Monday,

    May 28, 12

33. •Lives on servers located in data centers and clouds •Real

    users Monday, May 28, 12

34. •Lives on servers located in data centers and clouds •Real

    users •Either code works, or it doesn’t Monday, May 28, 12

35. •Lives on servers located in data centers and clouds •Real

    users •Either code works, or it doesn’t •Either the application is available or not Monday, May 28, 12

36. NO Monday, May 28, 12

37. but it PRESERVES potential revenue Monday, May 28, 12

38. Uh, what generates revenue? Question Monday, May 28, 12

39. Million Dollar Question Monday, May 28, 12

40. Monday, May 28, 12

41. •Working features (or at least that work enough) Monday, May

    28, 12

42. •Working features (or at least that work enough) •Infrastructure to

    keep the application up and running (or at least up enough) Monday, May 28, 12

43. •Working features (or at least that work enough) •Infrastructure to

    keep the application up and running (or at least up enough) •A business model that works (or at least works enough) Monday, May 28, 12

44. •Working features (or at least that work enough) •Infrastructure to

    keep the application up and running (or at least up enough) •A business model that works (or at least works enough) •??????????????????????? Monday, May 28, 12

45. Lessons learned Monday, May 28, 12

46. Common threads Monday, May 28, 12

47. Monday, May 28, 12

48. Monday, May 28, 12

49. Monday, May 28, 12

50. Pattern Configuration Management Monday, May 28, 12

51. duh. It’s about managing configuration. Monday, May 28, 12

52. You write code that manages your servers’ configuration Monday, May

    28, 12

53. Monday, May 28, 12

54. •apache package is installed Monday, May 28, 12

55. •apache package is installed •apache service is running Monday, May

    28, 12

56. •apache package is installed •apache service is running •deploy user

    exists Monday, May 28, 12

57. •apache package is installed •apache service is running •deploy user

    exists •cron jobs Monday, May 28, 12

58. •apache package is installed •apache service is running •deploy user

    exists •cron jobs •etc Monday, May 28, 12

59. •Puppet •Chef •Moonshine Monday, May 28, 12

60. Automation Monday, May 28, 12

61. Monday, May 28, 12

62. Monday, May 28, 12

63. Pattern Staging Environments Monday, May 28, 12

64. Preproduction servers Monday, May 28, 12

65. Gives you a place to ensure correctness of deploy Monday,

    May 28, 12

66. configuration management + staging servers = VERY YES Monday, May

    28, 12

67. Monday, May 28, 12

68. Monday, May 28, 12

69. Repeat after me Monday, May 28, 12

70. Repeat after me • development Monday, May 28, 12

71. Repeat after me • development • staging Monday, May 28,

    12

72. Repeat after me • development • staging • production Monday,

    May 28, 12

73. capistrano-gitflow Monday, May 28, 12

74. Pattern Deploy early, deploy often Monday, May 28, 12

75. Although technically, I guess it’s the same A play on

    release early, release often. Monday, May 28, 12

76. Monday, May 28, 12

77. Monday, May 28, 12

78. Antipattern Feeling Driven Development Monday, May 28, 12

79. Monday, May 28, 12

80. The front page feels slow Monday, May 28, 12

81. The primary key seems like it’s increasing rapidly Monday, May

    28, 12

82. IO seems high Monday, May 28, 12

83. What does it even mean? Monday, May 28, 12

84. Counterpattern Science Driven Development Monday, May 28, 12

85. Monday, May 28, 12

86. Metrics everywhere! Monday, May 28, 12

87. •New Relic - http://newrelic.com •Scout - http://scoutapp.com Monday, May 28,

    12

88. The front page is taking 10 seconds to load, but

    we really need it to be loading in under 1 second The front page feels slow Monday, May 28, 12

89. The primary key is at 90% of it’s maximum, up

    from 80% yesterday, and looks like it’ll run out overnight. The primary key seems like it’s increasing rapidly Monday, May 28, 12

90. IO fluctuates up to 90% sometimes, but doesn’t appear to

    have a negative effect IO seems high Monday, May 28, 12

91. Monday, May 28, 12

92. Topic Monitoring Monday, May 28, 12

93. How do you know when everything is awful? Monday, May

    28, 12

94. How would you prefer to know? Monday, May 28, 12

95. How would you prefer to know? •A late night phone

    call? Monday, May 28, 12

96. How would you prefer to know? •A late night phone

    call? •Angry tweets that you are down? Monday, May 28, 12

97. How would you prefer to know? •A late night phone

    call? •Angry tweets that you are down? •You personally checking everything all the time? Monday, May 28, 12

98. How would you prefer to know? •A late night phone

    call? •Angry tweets that you are down? •You personally checking everything all the time? •An automated system to let you know? Monday, May 28, 12

99. •Nagios •Scout •New Relic •Pingdom Monday, May 28, 12

100. What to monitor Monday, May 28, 12

101. Define urgency Monday, May 28, 12

102. Must be actionable Monday, May 28, 12

103. Single point of contact Monday, May 28, 12

104. Monday, May 28, 12

105. Pattern Vertical scaling Monday, May 28, 12

106. Now what? Your app is slow Monday, May 28, 12

107. Monday, May 28, 12

108. Resources are (relatively) cheap Monday, May 28, 12

109. Developers are (relatively) expensive Monday, May 28, 12

110. Monday, May 28, 12

111. Monday, May 28, 12

112. Antipattern Hipster Stack Monday, May 28, 12

113. Monday, May 28, 12

114. “I read a blog post about how mongo is totally

     web scale” Monday, May 28, 12

115. Cargo cult operations Monday, May 28, 12

116. Monday, May 28, 12

117. Monday, May 28, 12

118. Monday, May 28, 12

119. If you’re still going to go hipster... •experiment in branches

     •understand operational impact •Staging! Monday, May 28, 12

120. .... wait, what? Test in production Monday, May 28, 12

121. Fin. Monday, May 28, 12

122. Further Reading •Web Operations - John Allspaw and Jesse Robins

     •Continuous Delivery - Jez Humble and David Farley •“Web Operations for Developers 101” Monday, May 28, 12

123. find me here [email protected] @techpickles Want to talk ops? Monday,

     May 28, 12