To a thousand servers and beyond: scaling a massive PHP application

Transcript

1. To 1000 servers and beyond Nikolay Bachiyski PHPUK 2012

2. http://extrapolate.me/ @nikolayb

3. None
4. None
5. None

6. Stinson Beach, July 2007

7. Budapest, Oct 2011

8. Jul, 2007 65MM Feb, 2011 800MM PV

9. 2 stories

10. Story #1

11. Deep Pink Meet

12. Don’t like?

13. Beer Beheader =

14. None
15. None

16. Abstraction

17. None

18. Beer Theory

19. None

20. Want to be better?

21. Get to know the abstraction below

22. Servers are below

23. Story #2 Scaling WordPress.com

24. None

25. X

26. Deep Pink SAD

27. None

28. X

29. Deep Pink SAD

30. None
31. None

32. Users Blogs Blogs

33. No joins :(

34. None

35. SELECT  *  FROM  wp_users  WHERE  id=43315 SELECT  *  FROM  wp_posts

     LIMIT  10

36. HyperDB

37. None

38. SELECT  *  FROM  wp_users  WHERE  id=43315

39. Query parsing

40. add_db_table('user',  'wp_users') Dataset Table name

41. add_db_server('user',  0,  1,  0,            

    'db42.wordpress.com',…) Dataset Partition Read Write Slave

42. Failover X

43. Can work without

44. http://wordpress.org/extend/plugins/hyperdb/

45. None
46. None

47. C C C C

48. $count  =  wp_cache_get('posts',  'counts'); if  (  false  !==  $count  )

           return  $count; $count  =  $wpdb-­‐>get_results($complex_query);

49. Output caching

50. 10% 90% reads writes

51. C C C C

52. C C C C

53. Deep Pink SAD

54. Datacenter

55. C C sane sane

56. C C crazy sane Update

57. C C crazy sane Stale!

58. C C crazy crazy

59. Deploy

60. trunk/

61. Branch in code

62. ~100 commits a day

63. ~100 deploys a day

64. to ~1000 servers

65. One-click

66. None

67. Fast!

68. wpdev@nikolay:~$  deploy  wpcom Going  to  update  from  47127  to  47128

    Syncing  wpcom  SVN  Mirrors   ORD  (0s)...   LUV  (0s)...   SAT  (0s)... Deploying  wpcom    revision  47128   ORD  (3s)...   SAT  (5s)...   LUV  (6s)... Revision  47128  successfully  deployed  (8s)
69. None
70. None

71. Yeah, immediately!

72. Push to twitter

73. Push to Facebok

74. Stats

75. Spam checks

76. Index for search

77. Invalidate caches

78. Publish the post

79. ASYNC JOBS

80. None

81. $job_id  =  queue_async_job($data,  'encode_video');

82. Priorities

83. Deferred jobs

84. None

85. PHP & MySQL

86. http://code.trac.wordpress.org/wiki/JobsDocs http://code.trac.wordpress.org/browser/jobs

87. Integration

88. Hacking

89. None