Monoliths to Microservices - Practical Tips For CI, CD and DevOps in the Microservices world

Video: https://www.youtube.com/watch?v=NVb7aljfKYo&feature=youtu.be&t=6657

As the popularity of microservice architecture grows, CI and CD becomes more and more critical. Systems which contain dozens of services and diverse databases can get very challenging to manage. In this presentation I’ll take you through some hands on patterns which you can use to help you better manage your microservices!

Transcript

1. Monoliths to Microservices
   Practical Tips For CI, CD and DevOps in the Microservices world
   @dwmkerr
   Dave Kerr
   Senior Expert, McKinsey & Company
   

   View Slide

2. Who am I?
   @dwmkerr
   

   View Slide

3. I am a technology consultant.
   @dwmkerr
   

   View Slide

4. View Slide

5. Which is closer to this…
   @dwmkerr
   

   View Slide

6. View Slide

7. What am I going to talk about?
   @dwmkerr
   

   View Slide

8. The Project.
   @dwmkerr
   …or ‘come back WebSphere, all is forgiven’.
   

   View Slide

9. 2016 – new digital banking
   application launched to critical
   acclaim
   @dwmkerr
   

   View Slide

10. Digital garage setup,
    experienced product team,
    streets paved with gold.
    @dwmkerr
    

    View Slide

11. An effective product
    team will rapidly surface
    technological and
    organisational issues.
    @dwmkerr
    

    View Slide

12. I once caught a
    monolith thiiiiiiiis big
    

    View Slide

13. Solution
    Rebuild
    Stack
    Refresh
    105 people
    60 devs
    5 tech leads
    2 architects
    9 months
    React Native
    Node.js
    Kubernetes
    GraphQL
    DevOps 2.0
    Hiring
    Overhaul
    New HR processes
    > 50 new hires
    Sometimes there are no easy answers…
    Extensive
    Training
    Backend
    Frontend
    Scrum / Agile
    PMO
    Let’s talk about this…
    @dwmkerr
    

    View Slide

14. Our vision is to replace large complicated
    systems with small and simple units
    Mobile Web
    Monolith
    Integrator ESB
    SC
    CI/CD
    Artefacts
    APM
    Logs
    Core
    Card management
    Transfers
    Fraud
    ATM switch
    Interbank switch
    File Management
    …etc…
    Map/
    Reduce
    …etc…
    Spark
    Data lake
    Internal Gateway
    API Gateway
    Bill payments
    Profile &
    preferences
    Referrals
    P2P payments
    2FA, TX
    signing, soft-
    Token
    Contacts &
    groups
    Statements
    Device
    management
    Cards
    Identity
    management
    Notifications &
    inbox
    Accounts
    & transfers
    Mobile Web
    Existing server applications
    Enterprise integration
    Core platforms & databases
    Service zones
    Devops zone
    Analytics zone
    Existing
    Retired New
    Don’t forget to mix in some
    

    View Slide

15. People were nervous about all
    of the changes…
    @dwmkerr
    

    View Slide

16. @dwmkerr
    

    View Slide

17. Hic sunt dracones.
    …practical tips for devops in the microservice world…
    

    View Slide

18. Tip: Keep pipelines simple
    @dwmkerr
    

    View Slide

19. Tip: Keep pipelines simple.
    Build: turn code into artifacts
    Deploy: put artifacts
    somewhere
    @dwmkerr
    

    View Slide

20. Source
    http://www.dwmkerr.com/simple-continuous-integration-for-docker-images/
    

    View Slide

21. Tip: Keep pipelines simple... by embracing make.
    Step 1: rip out logic from your build system to create
    primitive commands
    @dwmkerr
    

    View Slide

22. Tip: Keep pipelines simple... by embracing make.
    Step 2: call primitive commands from your build system
    @dwmkerr
    

    View Slide

23. Tip: Keep pipelines simple... by embracing make.
    Step 3: when things get complicated, put them into ./scripts
    @dwmkerr
    

    View Slide

24. Tip: Keep pipelines simple... by embracing make.
    Demo
    github.com/dwmkerr/beautifully-simple-app-ci
    @dwmkerr
    

    View Slide

25. Tip: Craft Dockerfiles for your
    projects
    @dwmkerr
    

    View Slide

26. View Slide

27. Tip: Craft Dockerfiles for your projects
    A project’s baseline image for a Node 8 microservice. Handles: build-time proxies, NPM registry, non-root
    users, version tracking and allows the base OS to be changed. This Dockerfile is sneaky – no file system
    changes, only instructions for later builds!
    

    View Slide

28. Tip: Craft Dockerfiles for your projects
    

    View Slide

29. Tip: Craft Dockerfiles for your projects
    Demo
    github.com/dwmkerr/docker-dynamodb
    @dwmkerr
    

    View Slide

30. @dwmkerr
    Tip: Use the sandbox
    …and share your toys!
    

    View Slide

31. Tip: Use the sandbox
    Code Sample
    github.com/dwmkerr/terraform-aws-openshift
    @dwmkerr
    

    View Slide

32. Tip: Use the sandbox
    Demo
    github.com/dwmkerr/terraform-aws-openshift
    @dwmkerr
    

    View Slide

33. @dwmkerr
    Don’t forget to share your toys!
    …it’ll help you find bugs, get more ideas and build a
    group to collaborate with.
    

    View Slide

34. @dwmkerr
    …one more thing…
    

    View Slide

35. @dwmkerr
    Don’t forget to fall madly in
    love with Terraform!
    

    View Slide

36. @dwmkerr
    Tip: Make a microservice
    manifest
    …say that fast five times…
    

    View Slide

37. Tip: Make a microservice manifest
    

    View Slide

38. Tip: Make a microservice manifest
    

    View Slide

39. Tip: The developer’s machine
    comes first
    @dwmkerr
    

    View Slide

40. Tip: The developer’s machine comes first
    • Never sacrifice developer experience
    • Use docker-compose liberally
    • You should be able to deploy to production from anyone’s machine,
    as long as they have credentials
    @dwmkerr
    

    View Slide

41. Tip: Co-locate code and
    deployment configuration
    @dwmkerr
    

    View Slide

42. Tip: Co-locate microservice code & deployment
    config
    • jenkins / circle / gitlab / whatever
    • k8s / openshift / swarm / whatever
    • src / test
    • README!
    @dwmkerr
    

    View Slide

43. Tip: Co-locate microservice code & deployment
    config
    If I change the logic for my
    healthcheck, does my config
    change?
    @dwmkerr
    

    View Slide

44. Tip: Love the shell
    @dwmkerr
    

    View Slide

45. @dwmkerr
    npm install -g tldr
    …if I can leave you with one gift
    github.com/dwmkerr/effective-shell
    

    View Slide

46. Thanks!
    A lot of this stuff is on my blog and GitHub.
    @dwmkerr
    dwmkerr.com
    github.com/dwmkerr
    linkedin.com/in/dwmkerr
    

    View Slide