Upgrade to Pro — share decks privately, control downloads, hide ads and more …

To a thousand servers and beyond: scaling a massive PHP application

To a thousand servers and beyond: scaling a massive PHP application

Stories of abstraction and scaling WordPress.com. And what they have in common.

Nikolay Bachiyski

February 25, 2012
Tweet

More Decks by Nikolay Bachiyski

Other Decks in Programming

Transcript

  1. To 1000
    servers and
    beyond
    Nikolay Bachiyski
    PHPUK 2012

    View full-size slide

  2. http://extrapolate.me/
    @nikolayb

    View full-size slide

  3. Stinson Beach, July 2007

    View full-size slide

  4. Budapest, Oct 2011

    View full-size slide

  5. Jul, 2007
    65MM
    Feb, 2011
    800MM PV

    View full-size slide

  6. Deep Pink
    Meet

    View full-size slide

  7. Don’t like?

    View full-size slide

  8. Beer Beheader
    =

    View full-size slide

  9. Want to be
    better?

    View full-size slide

  10. Get to know
    the abstraction
    below

    View full-size slide

  11. Servers are
    below

    View full-size slide

  12. Story #2
    Scaling
    WordPress.com

    View full-size slide

  13. Deep Pink
    SAD

    View full-size slide

  14. Deep Pink
    SAD

    View full-size slide

  15. Users Blogs Blogs

    View full-size slide

  16. SELECT  *  FROM  wp_users  WHERE  id=43315
    SELECT  *  FROM  wp_posts  LIMIT  10

    View full-size slide

  17. SELECT  *  FROM  wp_users  WHERE  id=43315

    View full-size slide

  18. Query parsing

    View full-size slide

  19. add_db_table('user',  'wp_users')
    Dataset
    Table name

    View full-size slide

  20. add_db_server('user',  0,  1,  0,            
    'db42.wordpress.com',…)
    Dataset Partition Read
    Write
    Slave

    View full-size slide

  21. Can work without

    View full-size slide

  22. http://wordpress.org/extend/plugins/hyperdb/

    View full-size slide

  23. $count  =  wp_cache_get('posts',  'counts');
    if  (  false  !==  $count  )
           return  $count;
    $count  =  $wpdb-­‐>get_results($complex_query);

    View full-size slide

  24. Output caching

    View full-size slide

  25. 10%
    90%
    reads writes

    View full-size slide

  26. Deep Pink
    SAD

    View full-size slide

  27. C C
    sane sane

    View full-size slide

  28. C C
    crazy sane
    Update

    View full-size slide

  29. C C
    crazy sane
    Stale!

    View full-size slide

  30. C C
    crazy crazy

    View full-size slide

  31. Branch in code

    View full-size slide

  32. ~100 commits a day

    View full-size slide

  33. ~100 deploys a day

    View full-size slide

  34. to ~1000 servers

    View full-size slide

  35. wpdev@nikolay:~$  deploy  wpcom
    Going  to  update  from  47127  to  47128
    Syncing  wpcom  SVN  Mirrors
      ORD  (0s)...
      LUV  (0s)...
      SAT  (0s)...
    Deploying  wpcom    revision  47128
      ORD  (3s)...
      SAT  (5s)...
      LUV  (6s)...
    Revision  47128  successfully  deployed  (8s)

    View full-size slide

  36. Yeah, immediately!

    View full-size slide

  37. Push to twitter

    View full-size slide

  38. Push to Facebok

    View full-size slide

  39. Index for search

    View full-size slide

  40. Invalidate caches

    View full-size slide

  41. Publish the post

    View full-size slide

  42. $job_id  =  queue_async_job($data,  'encode_video');

    View full-size slide

  43. Deferred jobs

    View full-size slide

  44. http://code.trac.wordpress.org/wiki/JobsDocs
    http://code.trac.wordpress.org/browser/jobs

    View full-size slide