Upgrade to Pro — share decks privately, control downloads, hide ads and more …

To a thousand servers and beyond: scaling a massive PHP application

To a thousand servers and beyond: scaling a massive PHP application

Stories of abstraction and scaling WordPress.com. And what they have in common.

75fb365927cb3f5f7b677682d6249406?s=128

Nikolay Bachiyski

February 25, 2012
Tweet

Transcript

  1. To 1000 servers and beyond Nikolay Bachiyski PHPUK 2012

  2. http://extrapolate.me/ @nikolayb

  3. None
  4. None
  5. None
  6. Stinson Beach, July 2007

  7. Budapest, Oct 2011

  8. Jul, 2007 65MM Feb, 2011 800MM PV

  9. 2 stories

  10. Story #1

  11. Deep Pink Meet

  12. Don’t like?

  13. Beer Beheader =

  14. None
  15. None
  16. Abstraction

  17. None
  18. Beer Theory

  19. None
  20. Want to be better?

  21. Get to know the abstraction below

  22. Servers are below

  23. Story #2 Scaling WordPress.com

  24. None
  25. X

  26. Deep Pink SAD

  27. None
  28. X

  29. Deep Pink SAD

  30. None
  31. None
  32. Users Blogs Blogs

  33. No joins :(

  34. None
  35. SELECT  *  FROM  wp_users  WHERE  id=43315 SELECT  *  FROM  wp_posts

     LIMIT  10
  36. HyperDB

  37. None
  38. SELECT  *  FROM  wp_users  WHERE  id=43315

  39. Query parsing

  40. add_db_table('user',  'wp_users') Dataset Table name

  41. add_db_server('user',  0,  1,  0,            

    'db42.wordpress.com',…) Dataset Partition Read Write Slave
  42. Failover X

  43. Can work without

  44. http://wordpress.org/extend/plugins/hyperdb/

  45. None
  46. None
  47. C C C C

  48. $count  =  wp_cache_get('posts',  'counts'); if  (  false  !==  $count  )

           return  $count; $count  =  $wpdb-­‐>get_results($complex_query);
  49. Output caching

  50. 10% 90% reads writes

  51. C C C C

  52. C C C C

  53. Deep Pink SAD

  54. Datacenter

  55. C C sane sane

  56. C C crazy sane Update

  57. C C crazy sane Stale!

  58. C C crazy crazy

  59. Deploy

  60. trunk/

  61. Branch in code

  62. ~100 commits a day

  63. ~100 deploys a day

  64. to ~1000 servers

  65. One-click

  66. None
  67. Fast!

  68. wpdev@nikolay:~$  deploy  wpcom Going  to  update  from  47127  to  47128

    Syncing  wpcom  SVN  Mirrors   ORD  (0s)...   LUV  (0s)...   SAT  (0s)... Deploying  wpcom    revision  47128   ORD  (3s)...   SAT  (5s)...   LUV  (6s)... Revision  47128  successfully  deployed  (8s)
  69. None
  70. None
  71. Yeah, immediately!

  72. Push to twitter

  73. Push to Facebok

  74. Stats

  75. Spam checks

  76. Index for search

  77. Invalidate caches

  78. Publish the post

  79. ASYNC JOBS

  80. None
  81. $job_id  =  queue_async_job($data,  'encode_video');

  82. Priorities

  83. Deferred jobs

  84. None
  85. PHP & MySQL

  86. http://code.trac.wordpress.org/wiki/JobsDocs http://code.trac.wordpress.org/browser/jobs

  87. Integration

  88. Hacking

  89. None