Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[ZendCon EU 2013] The Evolution of DevOps

Davey Shafik
November 20, 2013

[ZendCon EU 2013] The Evolution of DevOps

With the rapid adoption of new software development methodologies and infrastructure as a service (IaaS), we’ve seen a new role emerge, DevOps.

The DevOp is a master of this new domain: The Cloud. But just as quickly as we’ve seen DevOps emerge, we’re seeing the role evolve as advanced platforms and services enter on to the scene.

During this talk, we’ll explore how DevOps is evolving, what that means to you, and your role in the ever changing landscape of development and the cloud.

Davey Shafik

November 20, 2013
Tweet

More Decks by Davey Shafik

Other Decks in Programming

Transcript

  1. The Evolution of DevOps
    Where Developers Meet the Cloud

    View Slide

  2. •Community Engineer at Engine Yard
    •Author of Zend PHP 5 Certification
    Study Guide, Sitepoints PHP
    Anthology: 101 Essential Tips, Tricks
    & Hacks & PHP Master: Write
    Cutting Edge Code
    •A contributor to Zend Framework 1
    & 2, phpdocs, and PHP internals
    • @dshafik
    Davey Shafik

    View Slide

  3. Apologies For the Accent(s)
    There many be 2 or 3…

    View Slide

  4. Community

    View Slide

  5. Community Engineer
    PHPMentoring.org
    #phpmentoring on
    irc.freenode.net

    View Slide

  6. Community Engineer
    PHPWomen.org
    #phpwomen on irc.freenode.net

    View Slide

  7. Community Engineer
    #phpc on irc.freenode.net

    View Slide

  8. What do you do for a living?

    View Slide

  9. What do you do for a living?
    I build the internet

    View Slide

  10. We don’t build web pages
    anymore

    View Slide

  11. We don’t build web pages
    anymore
    We build infrastructures

    View Slide

  12. Scriptable Hardware

    View Slide

  13. Scriptable Hardware
    Scriptable “Hardware”

    View Slide

  14. Scriptable Hardware
    Sysadmin roles are changing
    Scriptable “Hardware”

    View Slide

  15. Scriptable Hardware
    Sysadmin roles are changing
    Developer roles are changing
    Scriptable “Hardware”

    View Slide

  16. View Slide

  17. Developers

    View Slide

  18. Developers
    Application

    Features

    View Slide

  19. Developers
    Application

    Features
    Application

    Architecture

    View Slide

  20. Developers Sysadmins
    Application

    Features
    Application

    Architecture

    View Slide

  21. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software

    View Slide

  22. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level

    View Slide

  23. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level
    Deployment

    View Slide

  24. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level
    DevOps
    Deployment

    View Slide

  25. DevOps Starts at 127.0.0.1

    View Slide

  26. DevOps Starts at 127.0.0.1
    Where Developers Are In Control

    View Slide

  27. Vagrant
    vagrantup.com

    View Slide

  28. PuPHPet
    PuPHPet.com

    View Slide

  29. View Slide

  30. Deploy to Production

    View Slide

  31. Deploy to Production
    Suddenly: You’re responsible for production!

    View Slide

  32. Congratulations!
    You’re now a DevOp!

    View Slide

  33. Deployment

    View Slide

  34. Deployment
    It’s more than pushing bits to boxes

    View Slide

  35. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl

    View Slide

  36. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl

    View Slide

  37. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl
    Both happen to be written in Ruby

    View Slide

  38. DevOps are Polyglot

    View Slide

  39. DevOps are Polyglot
    By Necessity

    View Slide

  40. Rise of the “As A Service”
    *aaS

    View Slide

  41. Application

    View Slide

  42. Platform
    Application

    View Slide

  43. Platform
    Varnish
    Application

    View Slide

  44. Platform
    Nginx
    Varnish
    Application

    View Slide

  45. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    Application

    View Slide

  46. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    MySQL Riak Cassandra
    Application

    View Slide

  47. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  48. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  49. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  50. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  51. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  52. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  53. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  54. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  55. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  56. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View Slide

  57. Infrastructure As A Service (IaaS)
    !
    Application
    Platform

    View Slide

  58. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !

    View Slide

  59. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !

    View Slide

  60. Application

    View Slide

  61. Application As A Service

    View Slide

  62. Application As A Service
    AKA: Service Oriented Architecture (SOA)

    View Slide

  63. Service-oriented architecture (SOA) is
    […] based on discrete pieces of software
    providing application functionality as
    services to other applications. […] It is
    independent of any vendor, product or
    technology.
    Source: Wikipedia!
    (Emphasis Mine)

    View Slide

  64. SOA = APIs

    View Slide

  65. APIs = REST

    View Slide

  66. Separation of Concerns (SoC)

    View Slide

  67. Separation of concerns (SoC) is a design principle
    for separating a computer program into distinct
    sections […]. A concern is a set of information that
    affects the code of a computer program.!
    !
    The value of separation of concerns is simplifying
    development and maintenance of computer programs.
    When concerns are well separated, individual sections
    can be developed and updated independently. Of
    especial value is the ability to later improve or modify
    one section of code without having to know the details
    of other sections, and without having to make
    corresponding changes to those sections.
    Source: Wikipedia!
    (Emphasis Mine)

    View Slide

  68. SoC = Encapsulating Processes

    View Slide

  69. Application
    Message Queues
    Telephony E-Mail
    E-Commerce Logging Translation

    View Slide

  70. Application
    Authentication

    View Slide

  71. View Slide

  72. View Slide

  73. Associates
    Associates

    View Slide

  74. Associates
    Associates
    Search
    Search

    View Slide

  75. Associates
    Associates
    Search
    Search
    Deals
    Deals

    View Slide

  76. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations

    View Slide

  77. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account

    View Slide

  78. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Wish
    List
    Wish
    List

    View Slide

  79. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List

    View Slide

  80. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List
    Ads
    Ads

    View Slide

  81. View Slide

  82. View Slide

  83. Content
    Service
    Content
    Service

    View Slide

  84. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service

    View Slide

  85. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service
    That's 22°C btw...
    That's 22°C btw...

    View Slide

  86. Five Rules of SOA

    View Slide

  87. Isolation
    • Object Oriented Design at the Systems Level
    • Process Encapsulation
    • System Isolation
    Services should be independent of each other

    View Slide

  88. Isolation
    • Plan well defined Boundaries
    • Code
    • Resources
    • Features
    • Use separate resource for each service, e.g. database
    • Use separate URLs for each
    • e.g. accounts.api.example.org & reviews.api.example.org
    • This allows you to eventually use separate servers and eventually,
    separate clusters for each
    !42

    View Slide

  89. github.com/Netflix/SimianArmy

    View Slide

  90. Reusable
    • Use the same service to get the same data for different purposes
    • e.g. Getting an items rating, and getting it’s reviews
    Services should encapsulate a concern entirely

    View Slide

  91. Reusable
    • Services can be used like libraries
    • Expose as Public APIs
    !45

    View Slide

  92. Interoperable
    Services should work with each other
    • Loosely couples integration from implementation
    • Allows mixing technology to meet your needs

    View Slide

  93. Interoperable
    • Similar data should be consistent
    • Similar actions should be consistent
    • HTTP + JSON are supported everywhere
    !47

    View Slide

  94. Scalable
    Services Allow for Easier Scaling
    • Independently scale each service
    • Scale teams

    View Slide

  95. Scalable
    • Separated Resources
    • Similar to sharding your database
    !49

    View Slide

  96. Robust
    Cache Everything
    • Loosely Coupled
    • Refactor without impacting clients

    View Slide

  97. Robust
    • Consistent and informative errors
    • Treat all services the same as third-party
    • Trust nothing
    • Assume problems
    • Cache Everything
    !51

    View Slide

  98. Performance
    All that extra HTTP traffic isn’t cheap

    View Slide

  99. Distributed Resources
    Horizontal Scaling is Easier

    View Slide

  100. Parallel HTTP Requests
    pecl/http or curl_multi_*

    View Slide

  101. SPDY/HTTP 2.0

    View Slide

  102. Thick Frontends
    Client-Side Applications Love SOA

    View Slide

  103. Mobile is Easy

    View Slide

  104. But I have Legacy Apps!
    (who doesn’t?)

    View Slide

  105. Application Lifecycle

    View Slide

  106. Application Lifecycle
    80%
    Maintenance

    View Slide

  107. Application Lifecycle
    20%
    80%
    Design, Building,
    QA, etc
    Maintenance

    View Slide

  108. Maintaining Code

    View Slide

  109. Maintaining Code
    60% New Features

    View Slide

  110. Maintaining Code
    17% 60%
    Bug Fixes
    New Features

    View Slide

  111. Maintaining Code
    23%
    17% 60%
    Other
    (QA, Docs,
    Deployment)
    Bug Fixes
    New Features

    View Slide

  112. BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    Reuse
    (What you want to)
    Refactor
    (What you can)
    Rewrite
    (What you need to)
    The 3 R’s of Software Development

    View Slide

  113. Reuse
    • Models
    • Business Logic
    • Libraries
    • Data sources
    • Almost anything but your views
    Pros: Quickly piggy-back off your current code.
    Cons: Will likely lead to REST-like (or RESTy/RESTful)
    APIs that are RPC in reality.

    View Slide

  114. Reuse: Best Practices for APIs
    • Design your API first
    • APIs are like URIs: They should be permanent
    • Don’t let bad legacy decision drive new development
    Do: Pretend your API is a re-write. Create the best API
    you can for your application. Then implement it using
    legacy code any way you can. Even if it sucks.
    Don’t: Make compromises based on old code. Instead,
    plan to later refactor the old code.

    View Slide

  115. Refactor
    • Two types of refactoring:
    • Refactor your app to use your services instead of legacy code.
    Then make the APIs better by:
    • Refactor the underlying implementation of your APIs to
    migrate away from legacy code
    Do: Test. Unit Tests and Integration Tests.
    Don’t: change your API! If you designed it right during
    the Reuse phase, this should be easy!

    View Slide

  116. Rewrite
    • Rewriting should be your last step
    • You now have separation of concerns, so you can re-write your
    application piece meal
    Do: Discard any and all bad legacy code. Treat it as a
    clean slate.
    Don’t: Rewrite just to rewrite! Legacy code isn’t
    inherently bad code. Focus on new features. Build them
    as services!

    View Slide

  117. Step-By-Step
    1. Design your API
    2. Build by Reusing legacy code (Do whatever it
    takes!)
    3. Release
    4. Refactor Application to use API instead of Legacy
    code
    5. Refactor (or Rewrite) API to not use Legacy code.
    6. Reuse API in Legacy application Rewrite (or Reuse
    in an entirely new application!)

    View Slide

  118. Where does that leave
    Developers?

    View Slide

  119. Start Using VMs
    Vagrant is Awesome

    View Slide

  120. Learn Chef or Puppet

    View Slide

  121. Deep Dive into HTTP

    View Slide

  122. Caching
    • E-Tags/If-None-Match
    • Last-Modified/If-Modified-Since
    • Cache-Control

    View Slide

  123. Status Codes

    View Slide

  124. Semantic Requests
    HTTP Requests with Meaning
    • HTTP Verbs (GET/POST/PUT/DELETE/OPTIONS/HEAD/PATCH)

    +
    • HTTP Status Codes (e.g. 405 Method Not Allowed)

    View Slide

  125. Idempotent Requests
    Doing the same thing multiple times always has the
    same result
    • Easy To Test!

    View Slide

  126. You Should Check it Out

    View Slide

  127. Focus on Features
    Build Cool Stuff
    Leave as much work to others as you want

    View Slide

  128. What about Sysadmins?

    View Slide

  129. What about Sysadmins?
    Time to learn some Ruby…

    View Slide

  130. `
    Feedback:
    https://joind.in/9916
    Twitter: @dshafik
    Email: [email protected]
    !
    Slides:
    http://daveyshafik.com/slides

    View Slide