Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Monoliths to Microservices - Practical Tips For CI, CD and DevOps in the Microservices world

Dave Kerr
October 25, 2017

Monoliths to Microservices - Practical Tips For CI, CD and DevOps in the Microservices world

Video: https://www.youtube.com/watch?v=NVb7aljfKYo&feature=youtu.be&t=6657

As the popularity of microservice architecture grows, CI and CD becomes more and more critical. Systems which contain dozens of services and diverse databases can get very challenging to manage. In this presentation I’ll take you through some hands on patterns which you can use to help you better manage your microservices!

Dave Kerr

October 25, 2017
Tweet

More Decks by Dave Kerr

Other Decks in Technology

Transcript

  1. Monoliths to Microservices
    Practical Tips For CI, CD and DevOps in the Microservices world
    @dwmkerr
    Dave Kerr
    Senior Expert, McKinsey & Company

    View full-size slide

  2. Who am I?
    @dwmkerr

    View full-size slide

  3. I am a technology consultant.
    @dwmkerr

    View full-size slide

  4. Which is closer to this…
    @dwmkerr

    View full-size slide

  5. What am I going to talk about?
    @dwmkerr

    View full-size slide

  6. The Project.
    @dwmkerr
    …or ‘come back WebSphere, all is forgiven’.

    View full-size slide

  7. 2016 – new digital banking
    application launched to critical
    acclaim
    @dwmkerr

    View full-size slide

  8. Digital garage setup,
    experienced product team,
    streets paved with gold.
    @dwmkerr

    View full-size slide

  9. An effective product
    team will rapidly surface
    technological and
    organisational issues.
    @dwmkerr

    View full-size slide

  10. I once caught a
    monolith thiiiiiiiis big

    View full-size slide

  11. Solution
    Rebuild
    Stack
    Refresh
    105 people
    60 devs
    5 tech leads
    2 architects
    9 months
    React Native
    Node.js
    Kubernetes
    GraphQL
    DevOps 2.0
    Hiring
    Overhaul
    New HR processes
    > 50 new hires
    Sometimes there are no easy answers…
    Extensive
    Training
    Backend
    Frontend
    Scrum / Agile
    PMO
    Let’s talk about this…
    @dwmkerr

    View full-size slide

  12. Our vision is to replace large complicated
    systems with small and simple units
    Mobile Web
    Monolith
    Integrator ESB
    SC
    CI/CD
    Artefacts
    APM
    Logs
    Core
    Card management
    Transfers
    Fraud
    ATM switch
    Interbank switch
    File Management
    …etc…
    Map/
    Reduce
    …etc…
    Spark
    Data lake
    Internal Gateway
    API Gateway
    Bill payments
    Profile &
    preferences
    Referrals
    P2P payments
    2FA, TX
    signing, soft-
    Token
    Contacts &
    groups
    Statements
    Device
    management
    Cards
    Identity
    management
    Notifications &
    inbox
    Accounts
    & transfers
    Mobile Web
    Existing server applications
    Enterprise integration
    Core platforms & databases
    Service zones
    Devops zone
    Analytics zone
    Existing
    Retired New
    Don’t forget to mix in some

    View full-size slide

  13. People were nervous about all
    of the changes…
    @dwmkerr

    View full-size slide

  14. Hic sunt dracones.
    …practical tips for devops in the microservice world…

    View full-size slide

  15. Tip: Keep pipelines simple
    @dwmkerr

    View full-size slide

  16. Tip: Keep pipelines simple.
    Build: turn code into artifacts
    Deploy: put artifacts
    somewhere
    @dwmkerr

    View full-size slide

  17. Source
    http://www.dwmkerr.com/simple-continuous-integration-for-docker-images/

    View full-size slide

  18. Tip: Keep pipelines simple... by embracing make.
    Step 1: rip out logic from your build system to create
    primitive commands
    @dwmkerr

    View full-size slide

  19. Tip: Keep pipelines simple... by embracing make.
    Step 2: call primitive commands from your build system
    @dwmkerr

    View full-size slide

  20. Tip: Keep pipelines simple... by embracing make.
    Step 3: when things get complicated, put them into ./scripts
    @dwmkerr

    View full-size slide

  21. Tip: Keep pipelines simple... by embracing make.
    Demo
    github.com/dwmkerr/beautifully-simple-app-ci
    @dwmkerr

    View full-size slide

  22. Tip: Craft Dockerfiles for your
    projects
    @dwmkerr

    View full-size slide

  23. Tip: Craft Dockerfiles for your projects
    A project’s baseline image for a Node 8 microservice. Handles: build-time proxies, NPM registry, non-root
    users, version tracking and allows the base OS to be changed. This Dockerfile is sneaky – no file system
    changes, only instructions for later builds!

    View full-size slide

  24. Tip: Craft Dockerfiles for your projects

    View full-size slide

  25. Tip: Craft Dockerfiles for your projects
    Demo
    github.com/dwmkerr/docker-dynamodb
    @dwmkerr

    View full-size slide

  26. @dwmkerr
    Tip: Use the sandbox
    …and share your toys!

    View full-size slide

  27. Tip: Use the sandbox
    Code Sample
    github.com/dwmkerr/terraform-aws-openshift
    @dwmkerr

    View full-size slide

  28. Tip: Use the sandbox
    Demo
    github.com/dwmkerr/terraform-aws-openshift
    @dwmkerr

    View full-size slide

  29. @dwmkerr
    Don’t forget to share your toys!
    …it’ll help you find bugs, get more ideas and build a
    group to collaborate with.

    View full-size slide

  30. @dwmkerr
    …one more thing…

    View full-size slide

  31. @dwmkerr
    Don’t forget to fall madly in
    love with Terraform!

    View full-size slide

  32. @dwmkerr
    Tip: Make a microservice
    manifest
    …say that fast five times…

    View full-size slide

  33. Tip: Make a microservice manifest

    View full-size slide

  34. Tip: Make a microservice manifest

    View full-size slide

  35. Tip: The developer’s machine
    comes first
    @dwmkerr

    View full-size slide

  36. Tip: The developer’s machine comes first
    • Never sacrifice developer experience
    • Use docker-compose liberally
    • You should be able to deploy to production from anyone’s machine,
    as long as they have credentials
    @dwmkerr

    View full-size slide

  37. Tip: Co-locate code and
    deployment configuration
    @dwmkerr

    View full-size slide

  38. Tip: Co-locate microservice code & deployment
    config
    • jenkins / circle / gitlab / whatever
    • k8s / openshift / swarm / whatever
    • src / test
    • README!
    @dwmkerr

    View full-size slide

  39. Tip: Co-locate microservice code & deployment
    config
    If I change the logic for my
    healthcheck, does my config
    change?
    @dwmkerr

    View full-size slide

  40. Tip: Love the shell
    @dwmkerr

    View full-size slide

  41. @dwmkerr
    npm install -g tldr
    …if I can leave you with one gift
    github.com/dwmkerr/effective-shell

    View full-size slide

  42. Thanks!
    A lot of this stuff is on my blog and GitHub.
    @dwmkerr
    dwmkerr.com
    github.com/dwmkerr
    linkedin.com/in/dwmkerr

    View full-size slide