[ZendCon EU 2013] The Evolution of DevOps

With the rapid adoption of new software development methodologies and infrastructure as a service (IaaS), we’ve seen a new role emerge, DevOps.

The DevOp is a master of this new domain: The Cloud. But just as quickly as we’ve seen DevOps emerge, we’re seeing the role evolve as advanced platforms and services enter on to the scene.

During this talk, we’ll explore how DevOps is evolving, what that means to you, and your role in the ever changing landscape of development and the cloud.

Transcript

1. The Evolution of DevOps
   Where Developers Meet the Cloud
   

   View Slide

2. •Community Engineer at Engine Yard
   •Author of Zend PHP 5 Certification
   Study Guide, Sitepoints PHP
   Anthology: 101 Essential Tips, Tricks
   & Hacks & PHP Master: Write
   Cutting Edge Code
   •A contributor to Zend Framework 1
   & 2, phpdocs, and PHP internals
   • @dshafik
   Davey Shafik
   

   View Slide

3. Apologies For the Accent(s)
   There many be 2 or 3…
   

   View Slide

4. Community
   

   View Slide

5. Community Engineer
   PHPMentoring.org
   #phpmentoring on
   irc.freenode.net
   

   View Slide

6. Community Engineer
   PHPWomen.org
   #phpwomen on irc.freenode.net
   

   View Slide

7. Community Engineer
   #phpc on irc.freenode.net
   

   View Slide

8. What do you do for a living?
   

   View Slide

9. What do you do for a living?
   I build the internet
   

   View Slide

10. We don’t build web pages
    anymore
    

    View Slide

11. We don’t build web pages
    anymore
    We build infrastructures
    

    View Slide

12. Scriptable Hardware
    

    View Slide

13. Scriptable Hardware
    Scriptable “Hardware”
    

    View Slide

14. Scriptable Hardware
    Sysadmin roles are changing
    Scriptable “Hardware”
    

    View Slide

15. Scriptable Hardware
    Sysadmin roles are changing
    Developer roles are changing
    Scriptable “Hardware”
    

    View Slide

16. View Slide

17. Developers
    

    View Slide

18. Developers
    Application
    
    Features
    

    View Slide

19. Developers
    Application
    
    Features
    Application
    
    Architecture
    

    View Slide

20. Developers Sysadmins
    Application
    
    Features
    Application
    
    Architecture
    

    View Slide

21. Developers Sysadmins
    Application
    
    Features
    Application
    
    Architecture
    System
    
    Software
    

    View Slide

22. Developers Sysadmins
    Application
    
    Features
    Application
    
    Architecture
    System
    
    Software
    OS Level
    

    View Slide

23. Developers Sysadmins
    Application
    
    Features
    Application
    
    Architecture
    System
    
    Software
    OS Level
    Deployment
    

    View Slide

24. Developers Sysadmins
    Application
    
    Features
    Application
    
    Architecture
    System
    
    Software
    OS Level
    DevOps
    Deployment
    

    View Slide

25. DevOps Starts at 127.0.0.1
    

    View Slide

26. DevOps Starts at 127.0.0.1
    Where Developers Are In Control
    

    View Slide

27. Vagrant
    vagrantup.com
    

    View Slide

28. PuPHPet
    PuPHPet.com
    

    View Slide

29. View Slide

30. Deploy to Production
    

    View Slide

31. Deploy to Production
    Suddenly: You’re responsible for production!
    

    View Slide

32. Congratulations!
    You’re now a DevOp!
    

    View Slide

33. Deployment
    

    View Slide

34. Deployment
    It’s more than pushing bits to boxes
    

    View Slide

35. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl
    

    View Slide

36. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl
    

    View Slide

37. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl
    Both happen to be written in Ruby
    

    View Slide

38. DevOps are Polyglot
    

    View Slide

39. DevOps are Polyglot
    By Necessity
    

    View Slide

40. Rise of the “As A Service”
    *aaS
    

    View Slide

41. Application
    

    View Slide

42. Platform
    Application
    

    View Slide

43. Platform
    Varnish
    Application
    

    View Slide

44. Platform
    Nginx
    Varnish
    Application
    

    View Slide

45. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    Application
    

    View Slide

46. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    MySQL Riak Cassandra
    Application
    

    View Slide

47. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

48. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

49. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

50. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

51. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

52. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

53. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

54. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

55. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

56. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application
    

    View Slide

57. Infrastructure As A Service (IaaS)
    !
    Application
    Platform
    

    View Slide

58. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !
    

    View Slide

59. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !
    

    View Slide

60. Application
    

    View Slide

61. Application As A Service
    

    View Slide

62. Application As A Service
    AKA: Service Oriented Architecture (SOA)
    

    View Slide

63. Service-oriented architecture (SOA) is
    […] based on discrete pieces of software
    providing application functionality as
    services to other applications. […] It is
    independent of any vendor, product or
    technology.
    Source: Wikipedia!
    (Emphasis Mine)
    

    View Slide

64. SOA = APIs
    

    View Slide

65. APIs = REST
    

    View Slide

66. Separation of Concerns (SoC)
    

    View Slide

67. Separation of concerns (SoC) is a design principle
    for separating a computer program into distinct
    sections […]. A concern is a set of information that
    affects the code of a computer program.!
    !
    The value of separation of concerns is simplifying
    development and maintenance of computer programs.
    When concerns are well separated, individual sections
    can be developed and updated independently. Of
    especial value is the ability to later improve or modify
    one section of code without having to know the details
    of other sections, and without having to make
    corresponding changes to those sections.
    Source: Wikipedia!
    (Emphasis Mine)
    

    View Slide

68. SoC = Encapsulating Processes
    

    View Slide

69. Application
    Message Queues
    Telephony E-Mail
    E-Commerce Logging Translation
    

    View Slide

70. Application
    Authentication
    

    View Slide

71. View Slide

72. View Slide

73. Associates
    Associates
    

    View Slide

74. Associates
    Associates
    Search
    Search
    

    View Slide

75. Associates
    Associates
    Search
    Search
    Deals
    Deals
    

    View Slide

76. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    

    View Slide

77. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    

    View Slide

78. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Wish
    List
    Wish
    List
    

    View Slide

79. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List
    

    View Slide

80. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List
    Ads
    Ads
    

    View Slide

81. View Slide

82. View Slide

83. Content
    Service
    Content
    Service
    

    View Slide

84. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service
    

    View Slide

85. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service
    That's 22°C btw...
    That's 22°C btw...
    

    View Slide

86. Five Rules of SOA
    

    View Slide

87. Isolation
    • Object Oriented Design at the Systems Level
    • Process Encapsulation
    • System Isolation
    Services should be independent of each other
    

    View Slide

88. Isolation
    • Plan well defined Boundaries
    • Code
    • Resources
    • Features
    • Use separate resource for each service, e.g. database
    • Use separate URLs for each
    • e.g. accounts.api.example.org & reviews.api.example.org
    • This allows you to eventually use separate servers and eventually,
    separate clusters for each
    !42
    

    View Slide

89. github.com/Netflix/SimianArmy
    

    View Slide

90. Reusable
    • Use the same service to get the same data for different purposes
    • e.g. Getting an items rating, and getting it’s reviews
    Services should encapsulate a concern entirely
    

    View Slide

91. Reusable
    • Services can be used like libraries
    • Expose as Public APIs
    !45
    

    View Slide

92. Interoperable
    Services should work with each other
    • Loosely couples integration from implementation
    • Allows mixing technology to meet your needs
    

    View Slide

93. Interoperable
    • Similar data should be consistent
    • Similar actions should be consistent
    • HTTP + JSON are supported everywhere
    !47
    

    View Slide

94. Scalable
    Services Allow for Easier Scaling
    • Independently scale each service
    • Scale teams
    

    View Slide

95. Scalable
    • Separated Resources
    • Similar to sharding your database
    !49
    

    View Slide

96. Robust
    Cache Everything
    • Loosely Coupled
    • Refactor without impacting clients
    

    View Slide

97. Robust
    • Consistent and informative errors
    • Treat all services the same as third-party
    • Trust nothing
    • Assume problems
    • Cache Everything
    !51
    

    View Slide

98. Performance
    All that extra HTTP traffic isn’t cheap
    

    View Slide

99. Distributed Resources
    Horizontal Scaling is Easier
    

    View Slide

100. Parallel HTTP Requests
     pecl/http or curl_multi_*
     

     View Slide

101. SPDY/HTTP 2.0
     

     View Slide

102. Thick Frontends
     Client-Side Applications Love SOA
     

     View Slide

103. Mobile is Easy
     

     View Slide

104. But I have Legacy Apps!
     (who doesn’t?)
     

     View Slide

105. Application Lifecycle
     

     View Slide

106. Application Lifecycle
     80%
     Maintenance
     

     View Slide

107. Application Lifecycle
     20%
     80%
     Design, Building,
     QA, etc
     Maintenance
     

     View Slide

108. Maintaining Code
     

     View Slide

109. Maintaining Code
     60% New Features
     

     View Slide

110. Maintaining Code
     17% 60%
     Bug Fixes
     New Features
     

     View Slide

111. Maintaining Code
     23%
     17% 60%
     Other
     (QA, Docs,
     Deployment)
     Bug Fixes
     New Features
     

     View Slide

112. BULLET
     • Bullet point
     
     • Bullet point
     
     • Bullet point
     BULLET
     • Bullet point
     
     • Bullet point
     
     • Bullet point
     BULLET
     • Bullet point
     
     • Bullet point
     
     • Bullet point
     BULLET
     • Bullet point
     
     • Bullet point
     
     • Bullet point
     Reuse
     (What you want to)
     Refactor
     (What you can)
     Rewrite
     (What you need to)
     The 3 R’s of Software Development
     

     View Slide

113. Reuse
     • Models
     • Business Logic
     • Libraries
     • Data sources
     • Almost anything but your views
     Pros: Quickly piggy-back off your current code.
     Cons: Will likely lead to REST-like (or RESTy/RESTful)
     APIs that are RPC in reality.
     

     View Slide

114. Reuse: Best Practices for APIs
     • Design your API first
     • APIs are like URIs: They should be permanent
     • Don’t let bad legacy decision drive new development
     Do: Pretend your API is a re-write. Create the best API
     you can for your application. Then implement it using
     legacy code any way you can. Even if it sucks.
     Don’t: Make compromises based on old code. Instead,
     plan to later refactor the old code.
     

     View Slide

115. Refactor
     • Two types of refactoring:
     • Refactor your app to use your services instead of legacy code.
     Then make the APIs better by:
     • Refactor the underlying implementation of your APIs to
     migrate away from legacy code
     Do: Test. Unit Tests and Integration Tests.
     Don’t: change your API! If you designed it right during
     the Reuse phase, this should be easy!
     

     View Slide

116. Rewrite
     • Rewriting should be your last step
     • You now have separation of concerns, so you can re-write your
     application piece meal
     Do: Discard any and all bad legacy code. Treat it as a
     clean slate.
     Don’t: Rewrite just to rewrite! Legacy code isn’t
     inherently bad code. Focus on new features. Build them
     as services!
     

     View Slide

117. Step-By-Step
     1. Design your API
     2. Build by Reusing legacy code (Do whatever it
     takes!)
     3. Release
     4. Refactor Application to use API instead of Legacy
     code
     5. Refactor (or Rewrite) API to not use Legacy code.
     6. Reuse API in Legacy application Rewrite (or Reuse
     in an entirely new application!)
     

     View Slide

118. Where does that leave
     Developers?
     

     View Slide

119. Start Using VMs
     Vagrant is Awesome
     

     View Slide

120. Learn Chef or Puppet
     

     View Slide

121. Deep Dive into HTTP
     

     View Slide

122. Caching
     • E-Tags/If-None-Match
     • Last-Modified/If-Modified-Since
     • Cache-Control
     

     View Slide

123. Status Codes
     

     View Slide

124. Semantic Requests
     HTTP Requests with Meaning
     • HTTP Verbs (GET/POST/PUT/DELETE/OPTIONS/HEAD/PATCH)

     +
     • HTTP Status Codes (e.g. 405 Method Not Allowed)
     

     View Slide

125. Idempotent Requests
     Doing the same thing multiple times always has the
     same result
     • Easy To Test!
     

     View Slide

126. You Should Check it Out
     

     View Slide

127. Focus on Features
     Build Cool Stuff
     Leave as much work to others as you want
     

     View Slide

128. What about Sysadmins?
     

     View Slide

129. What about Sysadmins?
     Time to learn some Ruby…
     

     View Slide

130. `
     Feedback:
     https://joind.in/9916
     Twitter: @dshafik
     Email: davey@engineyard.com
     !
     Slides:
     http://daveyshafik.com/slides
     

     View Slide