Upgrade to Pro — share decks privately, control downloads, hide ads and more …

To a thousand servers and beyond: scaling a massive PHP application

75fb365927cb3f5f7b677682d6249406?s=47 Nikolay Bachiyski
February 25, 2012
Programming
3
920

To a thousand servers and beyond: scaling a massive PHP application

Stories of abstraction and scaling WordPress.com. And what they have in common.

75fb365927cb3f5f7b677682d6249406?s=128

Nikolay Bachiyski

February 25, 2012
Tweet

More Decks by Nikolay Bachiyski

See All by Nikolay Bachiyski
75fb365927cb3f5f7b677682d6249406?s=48 nb
0
170
75fb365927cb3f5f7b677682d6249406?s=48 nb
3
420
75fb365927cb3f5f7b677682d6249406?s=48 nb
2
710
75fb365927cb3f5f7b677682d6249406?s=48 nb
2
290
75fb365927cb3f5f7b677682d6249406?s=48 nb
4
6.8k
75fb365927cb3f5f7b677682d6249406?s=48 nb
2
120
75fb365927cb3f5f7b677682d6249406?s=48 nb
3
160
75fb365927cb3f5f7b677682d6249406?s=48 nb
0
92

Other Decks in Programming

See All in Programming
B8403d102456248570005ee7fb2ba0f7?s=48 tooppoo
1
460
88ad4f75d7c84fcf560bb6205c52e8c1?s=48 dnskimo
8
1.5k
88ad4f75d7c84fcf560bb6205c52e8c1?s=48 dnskimo
7
1.4k
9184d05c3493636df20a38b8bcfd66df?s=48 kazuhei0108
3
950
E0e036f89c14b3e59640318eedf9670b?s=48 bkuhlmann
6
710
E505897a79eede1a676f92740261e8f8?s=48 kubode
2
190
C90bac78c0fb61105cfd8239767f903d?s=48 o0h
PRO
3
1.6k
Acd2f6f7498ea41881e161e191aa7c02?s=48 yattom
32
11k
7028eb7f1d5bf62b440e4e6894c4922c?s=48 bosshawk
1
270
828bebe205894e1aa5f75a1fc6935a9c?s=48 ganchan11
0
160
5155c9bc5b30d9e2a8fd677695887861?s=48 kaonash
2
1.9k
7f408bc67dc9ae3b288ee92d16d3c4c2?s=48 deepu105
0
230

Featured

See All Featured
1b8ad785acdd1ce1c99914b1c2a4e10e?s=48 sugarenia
233
880k
0fe2973fa8d27d96547e43322341183a?s=48 swwweet
206
6.9k
C1daf20e5c49ff910745198ef9869ac2?s=48 hatefulcrawdad
257
17k
E4f5d494ebdc9e7cce1aecf3ce3e8bc1?s=48 shpigford
166
19k
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
34
1.6k
5b6a2bd86ef643786a381a212e0ef2f1?s=48 geoffreycrofte
25
1k
Ae14cc4491ac334f9cd23f9f93b4305e?s=48 orderedlist
PRO
330
36k
C620790ae5bf5b50c245b2e0ef95f338?s=48 deanohume
294
28k
9cde37f47e4a800ea081ea42de8d749a?s=48 dougneiner
56
5.4k
0bce06bd06ee7a2c4f5500f25dcf7f18?s=48 paulrobertlloyd
71
3.7k
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
498
130k
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
27
2.1k

Transcript

  1. To 1000 servers and beyond Nikolay Bachiyski PHPUK 2012

  2. http://extrapolate.me/ @nikolayb

  3. None
  4. None
  5. None

  6. Stinson Beach, July 2007

  7. Budapest, Oct 2011

  8. Jul, 2007 65MM Feb, 2011 800MM PV

  9. 2 stories

  10. Story #1

  11. Deep Pink Meet

  12. Don’t like?

  13. Beer Beheader =

  14. None
  15. None

  16. Abstraction

  17. None

  18. Beer Theory

  19. None

  20. Want to be better?

  21. Get to know the abstraction below

  22. Servers are below

  23. Story #2 Scaling WordPress.com

  24. None

  25. X

  26. Deep Pink SAD

  27. None

  28. X

  29. Deep Pink SAD

  30. None
  31. None

  32. Users Blogs Blogs

  33. No joins :(

  34. None

  35. SELECT  *  FROM  wp_users  WHERE  id=43315 SELECT  *  FROM  wp_posts

     LIMIT  10

  36. HyperDB

  37. None

  38. SELECT  *  FROM  wp_users  WHERE  id=43315

  39. Query parsing

  40. add_db_table('user',  'wp_users') Dataset Table name

  41. add_db_server('user',  0,  1,  0,            

    'db42.wordpress.com',…) Dataset Partition Read Write Slave

  42. Failover X

  43. Can work without

  44. http://wordpress.org/extend/plugins/hyperdb/

  45. None
  46. None

  47. C C C C

  48. $count  =  wp_cache_get('posts',  'counts'); if  (  false  !==  $count  )

           return  $count; $count  =  $wpdb-­‐>get_results($complex_query);

  49. Output caching

  50. 10% 90% reads writes

  51. C C C C

  52. C C C C

  53. Deep Pink SAD

  54. Datacenter

  55. C C sane sane

  56. C C crazy sane Update

  57. C C crazy sane Stale!

  58. C C crazy crazy

  59. Deploy

  60. trunk/

  61. Branch in code

  62. ~100 commits a day

  63. ~100 deploys a day

  64. to ~1000 servers

  65. One-click

  66. None

  67. Fast!

  68. wpdev@nikolay:~$  deploy  wpcom Going  to  update  from  47127  to  47128

    Syncing  wpcom  SVN  Mirrors   ORD  (0s)...   LUV  (0s)...   SAT  (0s)... Deploying  wpcom    revision  47128   ORD  (3s)...   SAT  (5s)...   LUV  (6s)... Revision  47128  successfully  deployed  (8s)
  69. None
  70. None

  71. Yeah, immediately!

  72. Push to twitter

  73. Push to Facebok

  74. Stats

  75. Spam checks

  76. Index for search

  77. Invalidate caches

  78. Publish the post

  79. ASYNC JOBS

  80. None

  81. $job_id  =  queue_async_job($data,  'encode_video');

  82. Priorities

  83. Deferred jobs

  84. None

  85. PHP & MySQL

  86. http://code.trac.wordpress.org/wiki/JobsDocs http://code.trac.wordpress.org/browser/jobs

  87. Integration

  88. Hacking

  89. None