To a thousand servers and beyond: scaling a massive PHP application

Stories of abstraction and scaling WordPress.com. And what they have in common.

Transcript

1. To 1000 servers and beyond Nikolay Bachiyski PHPUK 2012
2. http://extrapolate.me/ @nikolayb
6. Stinson Beach, July 2007
7. Budapest, Oct 2011
8. Jul, 2007 65MM Feb, 2011 800MM PV
9. 2 stories
10. Story #1
11. Deep Pink Meet
12. Don’t like?
13. Beer Beheader =
16. Abstraction
18. Beer Theory
20. Want to be better?
21. Get to know the abstraction below
22. Servers are below
23. Story #2 Scaling WordPress.com
25. X
26. Deep Pink SAD
28. X
29. Deep Pink SAD
32. Users Blogs Blogs
33. No joins :(
35. SELECT  *  FROM  wp_users  WHERE  id=43315 SELECT  *  FROM  wp_posts  LIMIT  10
36. HyperDB
38. SELECT  *  FROM  wp_users  WHERE  id=43315
39. Query parsing
40. add_db_table('user',  'wp_users') Dataset Table name
41. add_db_server('user',  0,  1,  0,             'db42.wordpress.com',…) Dataset Partition Read Write Slave
42. Failover X
43. Can work without
44. http://wordpress.org/extend/plugins/hyperdb/
47. C C C C
48. $count  =  wp_cache_get('posts',  'counts'); if  (  false  !==  $count  )        return  $count; $count  =  $wpdb-­‐>get_results($complex_query);
49. Output caching
50. 10% 90% reads writes
51. C C C C
52. C C C C
53. Deep Pink SAD
54. Datacenter
55. C C sane sane
56. C C crazy sane Update
57. C C crazy sane Stale!
58. C C crazy crazy
59. Deploy
60. trunk/
61. Branch in code
62. ~100 commits a day
63. ~100 deploys a day
64. to ~1000 servers
65. One-click
67. Fast!
68. wpdev@nikolay:~$  deploy  wpcom Going  to  update  from  47127  to  47128 Syncing  wpcom  SVN  Mirrors   ORD  (0s)...   LUV  (0s)...   SAT  (0s)... Deploying  wpcom    revision  47128   ORD  (3s)...   SAT  (5s)...   LUV  (6s)... Revision  47128  successfully  deployed  (8s)
71. Yeah, immediately!
72. Push to twitter
73. Push to Facebok
74. Stats
75. Spam checks
76. Index for search
77. Invalidate caches
78. Publish the post
79. ASYNC JOBS
81. $job_id  =  queue_async_job($data,  'encode_video');
82. Priorities
83. Deferred jobs
85. PHP & MySQL
86. http://code.trac.wordpress.org/wiki/JobsDocs http://code.trac.wordpress.org/browser/jobs
87. Integration
88. Hacking