Breaking News & Breaking Software, SyncConf 2013

How the Guardian build web sites - by software architect, Andy Hume.

Transcript

1. SOFTWARE
   Andy Hume
   SyncConf, 2013
   NEWS
   Breaking
   Breaking
   Sunday, December 29, 13
   

   View Slide

2. PLANES?
   Breaking
   http://www.youtube.com/watch?v=Ai2HmvAXcU0
   Sunday, December 29, 13
   

   View Slide

3. PLANES?
   Breaking
   http://www.youtube.com/watch?v=Ai2HmvAXcU0
   Sunday, December 29, 13
   

   View Slide

4. Why so happy?
   http://www.youtube.com/watch?v=Ai2HmvAXcU0
   Sunday, December 29, 13
   

   View Slide

5. WEBSITES
   Breaking
   Sunday, December 29, 13
   

   View Slide

6. http://www.flickr.com/photos/disinfeqt/7908154990/
   Process
   Sunday, December 29, 13
   

   View Slide

7. Text
   Sunday, December 29, 13
   

   View Slide

8. Sunday, December 29, 13
   

   View Slide

9. THE ARCHITECTURE
   Guardian APIs
   EC2 / ELB / S3 / Cloudformation
   Scala / Play 2
   AMD JavaScript / requirejs / LESS
   Sunday, December 29, 13
   

   View Slide

10. THE TEAM
    6 Developers (2 client-side)
    1 QA
    1 Product Manager
    (me)
    Sunday, December 29, 13
    

    View Slide

11. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13
    

    View Slide

12. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    Sunday, December 29, 13
    

    View Slide

13. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    Sunday, December 29, 13
    

    View Slide

14. I think somethings awry ... the graph is
    implying that there have been about 100
    deploys in 17 days.
    ose graphs don't lie.
    Matt, Development Lead
    Michael, Operations Manager
    Sunday, December 29, 13
    

    View Slide

15. WORKFLOW
    Github
    http://github.com/guardian/frontend
    Sunday, December 29, 13
    

    View Slide

16. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

17. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

18. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

19. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

20. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

21. Pull request
    https://github.com/guardian/frontend/pull/388
    Sunday, December 29, 13
    

    View Slide

22. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    Sunday, December 29, 13
    

    View Slide

23. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    Sunday, December 29, 13
    

    View Slide

24. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    Sunday, December 29, 13
    

    View Slide

25. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13
    

    View Slide

26. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13
    

    View Slide

27. RUNS THE BUILD
    CONTINUOUS
    INTEGRATION
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    DEPLOYS TO STAGE
    RUNS THE BUILD
    RUNS UNIT TESTS
    RUNS INTEGRATION TESTS
    RUNS INTEGRATION TESTS
    Sunday, December 29, 13
    

    View Slide

28. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    Sunday, December 29, 13
    

    View Slide

29. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    Sunday, December 29, 13
    

    View Slide

30. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    PERFORMANCE TESTS?
    Sunday, December 29, 13
    

    View Slide

31. SANITY CHECK
    DEPLOY TO
    PRODUCTION
    SOME QA REGRESSION TESTS
    PERFORMANCE TESTS
    DEPLOY TO PROD
    SOME QA REGRESSION TESTS ?
    PERFORMANCE TESTS?
    Sunday, December 29, 13
    

    View Slide

32. http://www.flickr.com/photos/spacemanbob/1084139169/
    Continuous deployment
    EMPOWERMENT
    ADD VALUE
    TAKE OWNERSHIP
    Sunday, December 29, 13
    

    View Slide

33. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13
    

    View Slide

34. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13
    

    View Slide

35. “If it moves, graph it”
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13
    

    View Slide

36. Real time performance
    Sunday, December 29, 13
    

    View Slide

37. Real time performance
    Sunday, December 29, 13
    

    View Slide

38. Real time performance
    Sunday, December 29, 13
    

    View Slide

39. FIREFOX LOAD
    CHROME LOAD
    Sunday, December 29, 13
    

    View Slide

40. GANGLIA METRICS
    Sunday, December 29, 13
    

    View Slide

41. Sunday, December 29, 13
    

    View Slide

42. REAL USER METRICS
    (RUM)
    Sunday, December 29, 13
    

    View Slide

43. Metrics monitoring
    http://www.flickr.com/photos/kayaker1204/4319542459/
    Sunday, December 29, 13
    

    View Slide

44. http://www.flickr.com/photos/spacemanbob/1084139169/
    Failing
    Sunday, December 29, 13
    

    View Slide

45. http://www.flickr.com/photos/spacemanbob/1084139169/
    Failing
    Sunday, December 29, 13
    

    View Slide

46. RESILIENCE
    VS
    RELIABILITY
    Sunday, December 29, 13
    

    View Slide

47. Sunday, December 29, 13
    

    View Slide

48. Up me is not binary
    Mike Brittain, Etsy
    Sunday, December 29, 13
    

    View Slide

49. Sunday, December 29, 13
    

    View Slide

50. Sunday, December 29, 13
    

    View Slide

51. Sunday, December 29, 13
    

    View Slide

52. Sunday, December 29, 13
    

    View Slide

53. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13
    

    View Slide

54. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13
    

    View Slide

55. Sunday, December 29, 13
    

    View Slide

56. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13
    

    View Slide

57. USER READING NEWS
    CONTENT API ZEITGEIST
    SOULMATES API
    SCALA WEB APP
    Sunday, December 29, 13
    

    View Slide

58. Sunday, December 29, 13
    

    View Slide

59. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    Sunday, December 29, 13
    

    View Slide

60. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    Sunday, December 29, 13
    

    View Slide

61. CONTENT API ZEITGEIST
    SCALA WEB APP
    USER READING NEWS
    SOULMATES API
    CACHE
    Sunday, December 29, 13
    

    View Slide

62. GOOGLE PAGE SPEED
    Sunday, December 29, 13
    

    View Slide

63. GOOGLE PAGE SPEED
    Sunday, December 29, 13
    

    View Slide

64. GOOGLE PAGE SPEED
    DOMContentReady event
    Sunday, December 29, 13
    

    View Slide

65. GOOGLE PAGE SPEED
    DOMContentReady event Load event
    Sunday, December 29, 13
    

    View Slide

66. GOOGLE PAGE SPEED
    P1
    DOMContentReady event Load event
    Sunday, December 29, 13
    

    View Slide

67. GOOGLE PAGE SPEED
    P1 P2
    DOMContentReady event Load event
    Sunday, December 29, 13
    

    View Slide

68. GOOGLE PAGE SPEED
    P1 P2 P3
    DOMContentReady event Load event
    Sunday, December 29, 13
    

    View Slide

69. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13
    

    View Slide

70. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13
    

    View Slide

71. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13
    

    View Slide

72. FAILING
    CONTINUOUS DEPLOYMENT
    METRICS MONITORING
    Sunday, December 29, 13
    

    View Slide

73. Thank-you!
    http://lanyrd.com/sccxwy
    @andyhume
    Creative Commons Licensed
    Attribution, Non-Commercial, Share Alike
    cc
    Sunday, December 29, 13
    

    View Slide

74. STUFF
    AWS AWS Cloudformation
    Scala/Play Framework
    Github
    Hudson
    Jenkins
    Ganglia
    Graphite
    Alerta
    Google Page Speed Insights
    Webpagetest
    Varnish
    AWS Cloudformation
    Sunday, December 29, 13
    

    View Slide