Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[ZendCon EU 2013] The Evolution of DevOps

Davey Shafik
November 20, 2013

[ZendCon EU 2013] The Evolution of DevOps

With the rapid adoption of new software development methodologies and infrastructure as a service (IaaS), we’ve seen a new role emerge, DevOps.

The DevOp is a master of this new domain: The Cloud. But just as quickly as we’ve seen DevOps emerge, we’re seeing the role evolve as advanced platforms and services enter on to the scene.

During this talk, we’ll explore how DevOps is evolving, what that means to you, and your role in the ever changing landscape of development and the cloud.

Davey Shafik

November 20, 2013
Tweet

More Decks by Davey Shafik

Other Decks in Programming

Transcript

  1. The Evolution of DevOps
    Where Developers Meet the Cloud

    View full-size slide

  2. •Community Engineer at Engine Yard
    •Author of Zend PHP 5 Certification
    Study Guide, Sitepoints PHP
    Anthology: 101 Essential Tips, Tricks
    & Hacks & PHP Master: Write
    Cutting Edge Code
    •A contributor to Zend Framework 1
    & 2, phpdocs, and PHP internals
    • @dshafik
    Davey Shafik

    View full-size slide

  3. Apologies For the Accent(s)
    There many be 2 or 3…

    View full-size slide

  4. Community Engineer
    PHPMentoring.org
    #phpmentoring on
    irc.freenode.net

    View full-size slide

  5. Community Engineer
    PHPWomen.org
    #phpwomen on irc.freenode.net

    View full-size slide

  6. Community Engineer
    #phpc on irc.freenode.net

    View full-size slide

  7. What do you do for a living?

    View full-size slide

  8. What do you do for a living?
    I build the internet

    View full-size slide

  9. We don’t build web pages
    anymore

    View full-size slide

  10. We don’t build web pages
    anymore
    We build infrastructures

    View full-size slide

  11. Scriptable Hardware

    View full-size slide

  12. Scriptable Hardware
    Scriptable “Hardware”

    View full-size slide

  13. Scriptable Hardware
    Sysadmin roles are changing
    Scriptable “Hardware”

    View full-size slide

  14. Scriptable Hardware
    Sysadmin roles are changing
    Developer roles are changing
    Scriptable “Hardware”

    View full-size slide

  15. Developers
    Application

    Features

    View full-size slide

  16. Developers
    Application

    Features
    Application

    Architecture

    View full-size slide

  17. Developers Sysadmins
    Application

    Features
    Application

    Architecture

    View full-size slide

  18. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software

    View full-size slide

  19. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level

    View full-size slide

  20. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level
    Deployment

    View full-size slide

  21. Developers Sysadmins
    Application

    Features
    Application

    Architecture
    System

    Software
    OS Level
    DevOps
    Deployment

    View full-size slide

  22. DevOps Starts at 127.0.0.1

    View full-size slide

  23. DevOps Starts at 127.0.0.1
    Where Developers Are In Control

    View full-size slide

  24. Vagrant
    vagrantup.com

    View full-size slide

  25. PuPHPet
    PuPHPet.com

    View full-size slide

  26. Deploy to Production

    View full-size slide

  27. Deploy to Production
    Suddenly: You’re responsible for production!

    View full-size slide

  28. Congratulations!
    You’re now a DevOp!

    View full-size slide

  29. Deployment
    It’s more than pushing bits to boxes

    View full-size slide

  30. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl

    View full-size slide

  31. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl

    View full-size slide

  32. chef
    puppet
    apache
    aws
    azure
    bash
    bower
    cassandra
    composer
    config
    ec2
    elastic ip
    elb
    fabric grunt
    ha
    haproxy
    heartbeat
    ini
    javascript
    json
    linux
    mariadb
    memcached
    mongodb
    mysql
    nginx
    node.js
    openstack
    oracle
    pear
    phing
    php php-fpm
    postgresql
    python
    redis
    riak
    ruby
    sass
    shell
    squid
    storage
    terramark
    vagrant varnish
    yaml
    pecl
    Both happen to be written in Ruby

    View full-size slide

  33. DevOps are Polyglot

    View full-size slide

  34. DevOps are Polyglot
    By Necessity

    View full-size slide

  35. Rise of the “As A Service”
    *aaS

    View full-size slide

  36. Platform
    Application

    View full-size slide

  37. Platform
    Varnish
    Application

    View full-size slide

  38. Platform
    Nginx
    Varnish
    Application

    View full-size slide

  39. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    Application

    View full-size slide

  40. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    MySQL Riak Cassandra
    Application

    View full-size slide

  41. Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  42. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  43. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  44. Infrastructure
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  45. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  46. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  47. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  48. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  49. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  50. Infrastructure As A Service (IaaS)
    !
    Platform
    Nginx
    PHP Ruby/Rails Node.js
    Varnish
    PostgreSQL MongoDB
    Other: Memcached, Gearman, etc.
    MySQL Riak Cassandra
    Application

    View full-size slide

  51. Infrastructure As A Service (IaaS)
    !
    Application
    Platform

    View full-size slide

  52. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !

    View full-size slide

  53. Application
    Platform
    Platform
    Application
    Infrastructure As A Service (IaaS)
    !

    View full-size slide

  54. Application As A Service

    View full-size slide

  55. Application As A Service
    AKA: Service Oriented Architecture (SOA)

    View full-size slide

  56. Service-oriented architecture (SOA) is
    […] based on discrete pieces of software
    providing application functionality as
    services to other applications. […] It is
    independent of any vendor, product or
    technology.
    Source: Wikipedia!
    (Emphasis Mine)

    View full-size slide

  57. Separation of Concerns (SoC)

    View full-size slide

  58. Separation of concerns (SoC) is a design principle
    for separating a computer program into distinct
    sections […]. A concern is a set of information that
    affects the code of a computer program.!
    !
    The value of separation of concerns is simplifying
    development and maintenance of computer programs.
    When concerns are well separated, individual sections
    can be developed and updated independently. Of
    especial value is the ability to later improve or modify
    one section of code without having to know the details
    of other sections, and without having to make
    corresponding changes to those sections.
    Source: Wikipedia!
    (Emphasis Mine)

    View full-size slide

  59. SoC = Encapsulating Processes

    View full-size slide

  60. Application
    Message Queues
    Telephony E-Mail
    E-Commerce Logging Translation

    View full-size slide

  61. Application
    Authentication

    View full-size slide

  62. Associates
    Associates

    View full-size slide

  63. Associates
    Associates
    Search
    Search

    View full-size slide

  64. Associates
    Associates
    Search
    Search
    Deals
    Deals

    View full-size slide

  65. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations

    View full-size slide

  66. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account

    View full-size slide

  67. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Wish
    List
    Wish
    List

    View full-size slide

  68. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List

    View full-size slide

  69. Associates
    Associates
    Search
    Search
    Deals
    Deals
    Recommendations
    Recommendations
    Account
    Account
    Cart
    Cart
    Wish
    List
    Wish
    List
    Ads
    Ads

    View full-size slide

  70. Content
    Service
    Content
    Service

    View full-size slide

  71. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service

    View full-size slide

  72. Related Content Service
    Related Content Service
    Content
    Service
    Content
    Service
    That's 22°C btw...
    That's 22°C btw...

    View full-size slide

  73. Five Rules of SOA

    View full-size slide

  74. Isolation
    • Object Oriented Design at the Systems Level
    • Process Encapsulation
    • System Isolation
    Services should be independent of each other

    View full-size slide

  75. Isolation
    • Plan well defined Boundaries
    • Code
    • Resources
    • Features
    • Use separate resource for each service, e.g. database
    • Use separate URLs for each
    • e.g. accounts.api.example.org & reviews.api.example.org
    • This allows you to eventually use separate servers and eventually,
    separate clusters for each
    !42

    View full-size slide

  76. github.com/Netflix/SimianArmy

    View full-size slide

  77. Reusable
    • Use the same service to get the same data for different purposes
    • e.g. Getting an items rating, and getting it’s reviews
    Services should encapsulate a concern entirely

    View full-size slide

  78. Reusable
    • Services can be used like libraries
    • Expose as Public APIs
    !45

    View full-size slide

  79. Interoperable
    Services should work with each other
    • Loosely couples integration from implementation
    • Allows mixing technology to meet your needs

    View full-size slide

  80. Interoperable
    • Similar data should be consistent
    • Similar actions should be consistent
    • HTTP + JSON are supported everywhere
    !47

    View full-size slide

  81. Scalable
    Services Allow for Easier Scaling
    • Independently scale each service
    • Scale teams

    View full-size slide

  82. Scalable
    • Separated Resources
    • Similar to sharding your database
    !49

    View full-size slide

  83. Robust
    Cache Everything
    • Loosely Coupled
    • Refactor without impacting clients

    View full-size slide

  84. Robust
    • Consistent and informative errors
    • Treat all services the same as third-party
    • Trust nothing
    • Assume problems
    • Cache Everything
    !51

    View full-size slide

  85. Performance
    All that extra HTTP traffic isn’t cheap

    View full-size slide

  86. Distributed Resources
    Horizontal Scaling is Easier

    View full-size slide

  87. Parallel HTTP Requests
    pecl/http or curl_multi_*

    View full-size slide

  88. SPDY/HTTP 2.0

    View full-size slide

  89. Thick Frontends
    Client-Side Applications Love SOA

    View full-size slide

  90. Mobile is Easy

    View full-size slide

  91. But I have Legacy Apps!
    (who doesn’t?)

    View full-size slide

  92. Application Lifecycle

    View full-size slide

  93. Application Lifecycle
    80%
    Maintenance

    View full-size slide

  94. Application Lifecycle
    20%
    80%
    Design, Building,
    QA, etc
    Maintenance

    View full-size slide

  95. Maintaining Code

    View full-size slide

  96. Maintaining Code
    60% New Features

    View full-size slide

  97. Maintaining Code
    17% 60%
    Bug Fixes
    New Features

    View full-size slide

  98. Maintaining Code
    23%
    17% 60%
    Other
    (QA, Docs,
    Deployment)
    Bug Fixes
    New Features

    View full-size slide

  99. BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    BULLET
    • Bullet point

    • Bullet point

    • Bullet point
    Reuse
    (What you want to)
    Refactor
    (What you can)
    Rewrite
    (What you need to)
    The 3 R’s of Software Development

    View full-size slide

  100. Reuse
    • Models
    • Business Logic
    • Libraries
    • Data sources
    • Almost anything but your views
    Pros: Quickly piggy-back off your current code.
    Cons: Will likely lead to REST-like (or RESTy/RESTful)
    APIs that are RPC in reality.

    View full-size slide

  101. Reuse: Best Practices for APIs
    • Design your API first
    • APIs are like URIs: They should be permanent
    • Don’t let bad legacy decision drive new development
    Do: Pretend your API is a re-write. Create the best API
    you can for your application. Then implement it using
    legacy code any way you can. Even if it sucks.
    Don’t: Make compromises based on old code. Instead,
    plan to later refactor the old code.

    View full-size slide

  102. Refactor
    • Two types of refactoring:
    • Refactor your app to use your services instead of legacy code.
    Then make the APIs better by:
    • Refactor the underlying implementation of your APIs to
    migrate away from legacy code
    Do: Test. Unit Tests and Integration Tests.
    Don’t: change your API! If you designed it right during
    the Reuse phase, this should be easy!

    View full-size slide

  103. Rewrite
    • Rewriting should be your last step
    • You now have separation of concerns, so you can re-write your
    application piece meal
    Do: Discard any and all bad legacy code. Treat it as a
    clean slate.
    Don’t: Rewrite just to rewrite! Legacy code isn’t
    inherently bad code. Focus on new features. Build them
    as services!

    View full-size slide

  104. Step-By-Step
    1. Design your API
    2. Build by Reusing legacy code (Do whatever it
    takes!)
    3. Release
    4. Refactor Application to use API instead of Legacy
    code
    5. Refactor (or Rewrite) API to not use Legacy code.
    6. Reuse API in Legacy application Rewrite (or Reuse
    in an entirely new application!)

    View full-size slide

  105. Where does that leave
    Developers?

    View full-size slide

  106. Start Using VMs
    Vagrant is Awesome

    View full-size slide

  107. Learn Chef or Puppet

    View full-size slide

  108. Deep Dive into HTTP

    View full-size slide

  109. Caching
    • E-Tags/If-None-Match
    • Last-Modified/If-Modified-Since
    • Cache-Control

    View full-size slide

  110. Status Codes

    View full-size slide

  111. Semantic Requests
    HTTP Requests with Meaning
    • HTTP Verbs (GET/POST/PUT/DELETE/OPTIONS/HEAD/PATCH)

    +
    • HTTP Status Codes (e.g. 405 Method Not Allowed)

    View full-size slide

  112. Idempotent Requests
    Doing the same thing multiple times always has the
    same result
    • Easy To Test!

    View full-size slide

  113. You Should Check it Out

    View full-size slide

  114. Focus on Features
    Build Cool Stuff
    Leave as much work to others as you want

    View full-size slide

  115. What about Sysadmins?

    View full-size slide

  116. What about Sysadmins?
    Time to learn some Ruby…

    View full-size slide

  117. `
    Feedback:
    https://joind.in/9916
    Twitter: @dshafik
    Email: [email protected]
    !
    Slides:
    http://daveyshafik.com/slides

    View full-size slide