Designing systems to scale

Talk given at PHPNW12 in Manchester, England

Transcript

1. Designing Systems To Scale
   

   View Slide

2. I’m Michael
   

   View Slide

3. I’m @mheap
   I’m Michael
   

   View Slide

4. I’m absolutely terrified
   I’m @mheap
   I’m Michael
   

   View Slide

5. Designing Systems To Scale
   

   View Slide

6. Designing Systems To Scale
   

   View Slide

7. Designing Systems To Scale
   

   View Slide

8. Write programs that do one thing
   and do it well.
   Write programs to work together.
   Write programs to handle text
   streams, because that is a universal
   interface.
   - Doug McIlroy
   

   View Slide

9. Do one
   thing, and
   do it well
   

   View Slide

10. View Slide

11. V1
    

    View Slide

12. V1
    • Codeigniter Application
    

    View Slide

13. V1
    • Codeigniter Application
    • No (working) indices on the database
    

    View Slide

14. V1
    • Codeigniter Application
    • No (working) indices on the database
    • Used the REST API and pulled data on demand
    

    View Slide

15. V1
    • Codeigniter Application
    • No (working) indices on the database
    • Used the REST API and pulled data on demand
    • PHP for realtime parsing
    

    View Slide

16. V1
    • Codeigniter Application
    • No (working) indices on the database
    • Used the REST API and pulled data on demand
    • PHP for realtime parsing
    • All stored in one git repo
    

    View Slide

17. 24 Users
    

    View Slide

18. V1
    • Codeigniter Application
    • No (working) indices on the database
    • Used the REST API and pulled data on demand
    • PHP for realtime parsing
    • All stored in one git repo
    

    View Slide

19. V2
    

    View Slide

20. V2
    • Decoupled
    

    View Slide

21. V2
    • Decoupled
    • Made up of five “apps”
    

    View Slide

22. V2
    • Decoupled
    • Made up of five “apps”
    • Easier to scale
    

    View Slide

23. V2
    • Decoupled
    • Made up of five “apps”
    • Easier to scale
    • More fault tolerant
    

    View Slide

24. MySQL
    Twitter
    Redis
    API
    Bucket
    Parser
    API
    Website Mobile App
    

    View Slide

25. Do one
    thing, and
    do it well
    

    View Slide

26. ... is
    rule #1
    

    View Slide

27. If all you
    have is a
    hammer...
    

    View Slide

28. ... everything
    looks like a
    nail
    

    View Slide

29. Rule #2
    

    View Slide

30. Use the
    right tool
    for the job
    

    View Slide

31. LNMPNRM
    

    View Slide

32. LNMPNRM
    • Linux
    

    View Slide

33. LNMPNRM
    • Linux
    • nginx
    

    View Slide

34. LNMPNRM
    • Linux
    • nginx
    • MySQL
    

    View Slide

35. LNMPNRM
    • Linux
    • nginx
    • MySQL
    • PHP
    

    View Slide

36. LNMPNRM
    • Linux
    • nginx
    • MySQL
    • PHP
    • NodeJS
    

    View Slide

37. LNMPNRM
    • Linux
    • nginx
    • MySQL
    • PHP
    • NodeJS
    • Redis
    

    View Slide

38. LNMPNRM
    • Linux
    • nginx
    • MySQL
    • PHP
    • NodeJS
    • Redis
    • Mongo DB
    

    View Slide

39. LNMPNRMR?
    • Linux
    • nginx
    • MySQL
    • PHP
    • NodeJS
    • Redis
    • Mongo DB
    • Ruby
    

    View Slide

40. Rule #3
    

    View Slide

41. Stand on the
    shoulders
    of giants
    

    View Slide

42. Do one thing, and do it well
    Use the right tool for the job
    Stand on the shoulders of giants
    

    View Slide

43. Now for the
    technical
    stuff...
    

    View Slide

44. Service
    Oriented
    Architecture
    

    View Slide

45. Shared
    Nothing
    Architecture
    

    View Slide

46. ... Except your
    database
    

    View Slide

47. Make it
    stateless
    

    View Slide

48. Make it
    event driven
    

    View Slide

49. API Driven
    Design
    

    View Slide

50. Models
    talk to the
    Database
    

    View Slide

51. Models
    talk to the
    API
    

    View Slide

52. API = Data
    

    View Slide

53. Website
    

    View Slide

54. Mobile App
    

    View Slide

55. Raspberry Pi
    

    View Slide

56. API = Data
    

    View Slide

57. Things
    I’ve
    learned
    

    View Slide

58. Expose
    JSON
    

    View Slide

59. REST is
    good
    

    View Slide

60. REST is
    good
    Except when it’s not
    

    View Slide

61. Databases
    

    View Slide

62. We use
    MySQL
    

    View Slide

63. We use
    InnoDB
    

    View Slide

64. MySQL 5.6
    EXPLAIN ALL
    THE THINGS!
    

    View Slide

65. View Slide

66. pt-online-schema-change
    xtrabackup
    pt-archive
    pt-query-digest
    pt-query-advisor
    pt-show-grants
    pt-fingerprint
    

    View Slide

67. Things
    I’ve
    learned
    

    View Slide

68. Indexes go
    from left to
    right
    

    View Slide

69. User (email, first, last, dob)
    name_index(first, last)
    email_index(email)
    user_index(last, dob)
    

    View Slide

70. InnoDB
    uses
    primary key
    in indexes
    

    View Slide

71. Speed up SELECT
    Slow down INSERT
    Indices
    

    View Slide

72. One connection
    means
    One query
    

    View Slide

73. ORM’s
    are evil
    

    View Slide

74. Denormalisation
    is ok
    

    View Slide

75. Backups
    

    View Slide

76. mysqldump
    

    View Slide

77. InnoDB Hot
    Backup
    

    View Slide

78. Percona
    Xtrabackup
    

    View Slide

79. 1,000,001
    

    View Slide

80. Recovery
    

    View Slide

81. Choose
    your battles
    

    View Slide

82. Servers
    

    View Slide

83. Servers
    (in the cloud?)
    

    View Slide

84. Learn to set up
    a server
    

    View Slide

85. Leave it to the
    professionals
    

    View Slide

86. Virtualisation
    

    View Slide

87. Get a VPS with
    Amazon AWS
    

    View Slide

88. Get a VPS with
    Rackspace
    

    View Slide

89. Get a VPS with
    Azure
    

    View Slide

90. Get a VPS with
    $vpsCompany
    

    View Slide

91. Leeds Hack
    

    View Slide

92. Things
    I’ve
    learned
    

    View Slide

93. £££
    

    View Slide

94. Multiple small
    beats
    one large
    

    View Slide

95. Dedicated
    is good too
    

    View Slide

96. Host your own
    database
    

    View Slide

97. Documentation
    

    View Slide

98. Not just about
    writing your own
    

    View Slide

99. View Slide

100. You should also
     write your own
     

     View Slide

101. View Slide

102. Easy
     Can we still deploy?
     

     View Slide

103. Easy
     Can we still deploy?
     Harder
     What if we need a new server firing up?
     

     View Slide

104. Easy
     Can we still deploy?
     Harder
     What if we need a new server firing up?
     Even Harder
     What if $customComponent breaks?
     

     View Slide

105. Make yourself
     dispensable
     

     View Slide

106. Always code as if the guy who ends
     up maintaining your code will be a
     violent psychopath who knows
     where you live
     - John f. Woods
     

     View Slide

107. Things
     I’ve
     learned
     

     View Slide

108. Smart
     Defaults
     

     View Slide

109. Dependency
     Management
     

     View Slide

110. Just use
     Composer
     or rubygems/npm/pip... etc
     

     View Slide

111. Things
     I’ve
     learned
     

     View Slide

112. Lock your
     dependecies
     

     View Slide

113. v 1.4.2
     

     View Slide

114. v 1.4.2
     Bug Fix
     

     View Slide

115. v 1.4.2
     Feature release
     

     View Slide

116. v 1.4.2
     All hell breaks loose
     

     View Slide

117. Lock your
     dependecies
     

     View Slide

118. Control your
     dependecies
     

     View Slide

119. View Slide

120. Control your
     dependecies
     

     View Slide

121. Automation
     

     View Slide

122. Predictability
     

     View Slide

123. It saves time
     

     View Slide

124. Choose one
     [MRJC]ake
     

     View Slide

125. forever stop /home/user/parser/parser.js &&
     APP_ENV=production forever start -a -l ~/parser-logs/
     parser.log -o ~/parser-logs/out.log -e ~/parser-logs/
     parser.err.log --minUptime 10000 --spinSleepTime 8000 ~/
     parser/parser.js
     

     View Slide

126. forever stop /home/user/parser/parser.js &&
     APP_ENV=production forever start -a -l ~/parser-logs/
     parser.log -o ~/parser-logs/out.log -e ~/parser-logs/
     parser.err.log --minUptime 10000 --spinSleepTime 8000 ~/
     parser/parser.js
     parser_control restart
     

     View Slide

127. Automating
     scary things
     

     View Slide

128. Push data
     not pull
     

     View Slide

129. Things
     I’ve
     learned
     

     View Slide

130. Vagrant
     

     View Slide

131. Puppet/Chef
     

     View Slide

132. Devops!
     

     View Slide

133. Logging
     

     View Slide

134. Log
     *everything*
     

     View Slide

135. Aggregated
     logs
     

     View Slide

136. Monolog
     

     View Slide

137. Make it
     an option
     

     View Slide

138. Things
     I’ve
     learned
     

     View Slide

139. Nothing... yet
     

     View Slide

140. Outsourcing
     

     View Slide

141. Things like
     hosting repos
     

     View Slide

142. Things like
     sending emails
     

     View Slide

143. Things like
     *insert task*
     

     View Slide

144. Outsourcing
     validation
     

     View Slide

145. Things
     I’ve
     learned
     

     View Slide

146. You don’t
     have to do
     everything
     

     View Slide

147. Testing
     

     View Slide

148. Testing is for
     peace of mind
     

     View Slide

149. It is not a
     design process
     

     View Slide

150. Integration tests
     over unit tests
     

     View Slide

151. Unit tests are
     nice too
     

     View Slide

152. Things
     I’ve
     learned
     

     View Slide

153. Test only
     what you
     need
     

     View Slide

154. Behat +
     Selenium
     

     View Slide

155. Being Human
     

     View Slide

156. You’re your own
     worst enemy
     

     View Slide

157. Logging queries
     to MongoDB
     

     View Slide

158. Node.js’
     process.nextTick
     

     View Slide

159. Something I
     haven’t even
     realised
     

     View Slide

160. Things
     I’ve
     learned
     

     View Slide

161. Learn to fail
     

     View Slide

162. People make
     mistakes
     

     View Slide

163. Believe in
     yourself
     

     View Slide

164. I’m Michael
     

     View Slide

165. I’m @mheap
     I’m Michael
     

     View Slide

166. I hope you enjoyed this
     talk
     I’m @mheap
     I’m Michael
     

     View Slide