$30 off During Our Annual Pro Sale. View Details »

Breaking News & Breaking Software, SyncConf 2013

Breaking News & Breaking Software, SyncConf 2013

How the Guardian build web sites - by software architect, Andy Hume.

Andy Hume

March 29, 2013
Tweet

More Decks by Andy Hume

Other Decks in Technology

Transcript

  1. SOFTWARE
    Andy Hume
    SyncConf, 2013
    NEWS
    Breaking
    Breaking
    Sunday, December 29, 13

    View Slide

  2. PLANES?
    Breaking
    http://www.youtube.com/watch?v=Ai2HmvAXcU0
    Sunday, December 29, 13

    View Slide

  3. PLANES?
    Breaking
    http://www.youtube.com/watch?v=Ai2HmvAXcU0
    Sunday, December 29, 13

    View Slide

  4. Why so happy?
    http://www.youtube.com/watch?v=Ai2HmvAXcU0
    Sunday, December 29, 13

    View Slide

  5. WEBSITES
    Breaking
    Sunday, December 29, 13

    View Slide

  6. http://www.flickr.com/photos/disinfeqt/7908154990/
    Process
    Sunday, December 29, 13

    View Slide

  7. Text
    Sunday, December 29, 13

    View Slide

  8. Sunday, December 29, 13

    View Slide

  9. THE ARCHITECTURE
    Guardian APIs
    EC2 / ELB / S3 / Cloudformation
    Scala / Play 2
    AMD JavaScript / requirejs / LESS
    Sunday, December 29, 13

    View Slide

  10. THE TEAM
    6 Developers (2 client-side)
    1 QA
    1 Product Manager
    (me)
    Sunday, December 29, 13

    View Slide

  11. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13

    View Slide

  12. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    Sunday, December 29, 13

    View Slide

  13. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    Sunday, December 29, 13

    View Slide

  14. I think somethings awry ... the graph is
    implying that there have been about 100
    deploys in 17 days.
    ose graphs don't lie.
    Matt, Development Lead
    Michael, Operations Manager
    Sunday, December 29, 13

    View Slide

  15. WORKFLOW
    Github
    http://github.com/guardian/frontend
    Sunday, December 29, 13

    View Slide

  16. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  17. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  18. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  19. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  20. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  21. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13

    View Slide

  22. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    Sunday, December 29, 13

    View Slide

  23. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    Sunday, December 29, 13

    View Slide

  24. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    Sunday, December 29, 13

    View Slide

  25. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13

    View Slide

  26. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13

    View Slide

  27. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13

    View Slide

  28. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    Sunday, December 29, 13

    View Slide

  29. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    Sunday, December 29, 13

    View Slide

  30. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    PERFORMANCE TESTS?
    Sunday, December 29, 13

    View Slide

  31. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    PERFORMANCE TESTS?
    Sunday, December 29, 13

    View Slide

  32. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    EMPOWERMENT
    ADD VALUE
    TAKE OWNERSHIP
    Sunday, December 29, 13

    View Slide

  33. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13

    View Slide

  34. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13

    View Slide

  35. “If it moves, graph it”
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13

    View Slide

  36. Real time performance
    Sunday, December 29, 13

    View Slide

  37. Real time performance
    Sunday, December 29, 13

    View Slide

  38. Real time performance
    Sunday, December 29, 13

    View Slide

  39. FIREFOX LOAD
    CHROME LOAD
    Sunday, December 29, 13

    View Slide

  40. GANGLIA METRICS
    Sunday, December 29, 13

    View Slide

  41. Sunday, December 29, 13

    View Slide

  42. REAL USER METRICS
    (RUM)
    Sunday, December 29, 13

    View Slide

  43. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13

    View Slide

  44. http://www.flickr.com/photos/spacemanbob/1084139169/
    Failing
    Sunday, December 29, 13

    View Slide

  45. http://www.flickr.com/photos/spacemanbob/1084139169/
    Failing
    Sunday, December 29, 13

    View Slide

  46. RESILIENCE
    VS
    RELIABILITY
    Sunday, December 29, 13

    View Slide

  47. Sunday, December 29, 13

    View Slide

  48. Up me is not binary
    Mike Brittain, Etsy
    Sunday, December 29, 13

    View Slide

  49. Sunday, December 29, 13

    View Slide

  50. Sunday, December 29, 13

    View Slide

  51. Sunday, December 29, 13

    View Slide

  52. Sunday, December 29, 13

    View Slide

  53. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13

    View Slide

  54. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13

    View Slide

  55. Sunday, December 29, 13

    View Slide

  56. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13

    View Slide

  57. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13

    View Slide

  58. Sunday, December 29, 13

    View Slide

  59. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    Sunday, December 29, 13

    View Slide

  60. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    Sunday, December 29, 13

    View Slide

  61. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    CACHE
    Sunday, December 29, 13

    View Slide

  62. GOOGLE PAGE SPEED
    Sunday, December 29, 13

    View Slide

  63. GOOGLE PAGE SPEED
    Sunday, December 29, 13

    View Slide

  64. GOOGLE PAGE SPEED
    DOMContentReady event
    Sunday, December 29, 13

    View Slide

  65. GOOGLE PAGE SPEED
    DOMContentReady event Load event
    Sunday, December 29, 13

    View Slide

  66. GOOGLE PAGE SPEED
    P1
    DOMContentReady event Load event
    Sunday, December 29, 13

    View Slide

  67. GOOGLE PAGE SPEED
    P1 P2
    DOMContentReady event Load event
    Sunday, December 29, 13

    View Slide

  68. GOOGLE PAGE SPEED
    P1 P2 P3
    DOMContentReady event Load event
    Sunday, December 29, 13

    View Slide

  69. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13

    View Slide

  70. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13

    View Slide

  71. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13

    View Slide

  72. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13

    View Slide

  73. Thank-you!
    http://lanyrd.com/sccxwy
    @andyhume
    Creative Commons Licensed
    Attribution, Non-Commercial, Share Alike
    cc
    Sunday, December 29, 13

    View Slide

  74. STUFF
    AWS AWS Cloudformation
    Scala/Play Framework
    Github
    Hudson
    Jenkins
    Ganglia
    Graphite
    Alerta
    Google Page Speed Insights
    Webpagetest
    Varnish
    AWS Cloudformation
    Sunday, December 29, 13

    View Slide