Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ITT 2018 - Amir Chaudhry - Engineering for a more distributed future

ITT 2018 - Amir Chaudhry - Engineering for a more distributed future

In the last few years we've had several waves of change from microservices, to containerisation, and more recently serverless and unikernels. Even the notion of an 'application' has changed over the years, as we now have to consider mobile apps, back-end AI algorithms, and even remote devices like IoT and autonomous vehicles.

The general idea of these approaches point in the same direction — that of applications becoming a collection of components that perform discrete tasks. However, since the architecture of such applications is more distributed, there are many trade-offs and new considerations. Including security implications, orchestration, monitoring, and much more. These are things that engineers need to become more aware of when creating and maintaining such systems. This talk will cover some of the issues that engineers should be aware of and we'll also discuss some of the tools and technologies that might be useful to consider.

Istanbul Tech Talks

April 17, 2018
Tweet

More Decks by Istanbul Tech Talks

Other Decks in Programming

Transcript

  1. ENGINEERING FOR A
    DISTRIBUTED FUTURE
    AMIR CHAUDHRY
    Istanbul Tech Talk

    17 Apr 2018

    View Slide

  2. Amir Chaudhry


    @amirmc

    View Slide

  3. ENGINEERING FOR THE FUTURE
    OVERVIEW
    ▸ Software today
    ▸ A brief history
    ▸ Fresh approaches
    ▸ Unikernels
    ▸ New considerations
    ▸ Discussion

    View Slide

  4. SOFTWARE TODAY…

    View Slide

  5. SOFTWARE TODAY…
    … is an application…
    ENGINEERING FOR THE FUTURE

    View Slide

  6. SOFTWARE TODAY…
    … is an application…
    … on top of an
    Operating System.
    ENGINEERING FOR THE FUTURE

    View Slide

  7. A BRIEF HISTORY

    View Slide

  8. FIRST PRODUCTION COMPUTER

    View Slide

  9. The EDSAC
    Actual customers
    3 Nobel Prizes

    View Slide

  10. Full programs
    Punched tape
    Self contained

    View Slide

  11. The subroutine
    Reusable code
    Efficient programs

    View Slide

  12. TOWARDS THE OS

    View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. Carried forward

    legacy assumptions

    View Slide

  17. ENGINEERING FOR THE FUTURE
    SOFTWARE TODAY…
    … is an application…
    … on top of an
    Operating System.

    View Slide

  18. SOFTWARE TODAY…
    ▸ Multiple users?
    ▸ Multiple apps?
    ▸ System services?
    ▸ Physical hardware?
    ENGINEERING FOR THE FUTURE

    View Slide

  19. Code you

    care about
    Code the OS

    insists you need

    View Slide

  20. Code you

    care about
    Code the OS

    insists you need

    View Slide

  21. STILL MORE COMPLICATED

    View Slide

  22. SOFTWARE TODAY…
    … is built locally…
    ENGINEERING FOR THE FUTURE

    View Slide

  23. SOFTWARE TODAY…
    … but deployed remotely…
    … is built locally…
    ENGINEERING FOR THE FUTURE

    View Slide

  24. SOFTWARE TODAY…
    … is built locally…
    … but deployed remotely…
    … very remotely.
    ENGINEERING FOR THE FUTURE

    View Slide

  25. View Slide

  26. View Slide

  27. SOFTWARE TODAY…
    … is complex!
    Even though we
    simplify aspects of it
    ENGINEERING FOR THE FUTURE

    View Slide

  28. THINGS ARE GETTING EASIER

    View Slide

  29. THINGS ARE GETTING EASIER…
    ENGINEERING FOR THE FUTURE

    View Slide

  30. View Slide

  31. Carry forward 

    legacy assumptions

    View Slide

  32. View Slide

  33. SUMMARY SO FAR
    ▸ World looks very different:
    ▸ Model of deployment is changing
    ▸ How we write apps is changing
    ▸ But underlying assumptions are still baked in
    ENGINEERING FOR THE FUTURE

    View Slide

  34. FRESH APPROACHES

    View Slide

  35. FRESH(ISH) APPROACHES
    ▸ Move to microservices

    Cattle vs Pets

    Multi-repo apps

    Containers

    Unikernels
    ▸ More distributed approaches
    ▸ Immutable artefacts
    ▸ Design systems not just apps

    View Slide

  36. FRESH APPROACHES
    A SPECTRUM OF OPTIONS
    Monolith Microservices

    View Slide

  37. UNIKERNELS?

    View Slide

  38. UNIKERNELS
    ABOUT UNIKERNELS
    ▸ Represent an extreme view
    ▸ Constructed from Library OSs
    ▸ OS functionality available as modular components
    ▸ Link only the OS functionality your app needs
    ▸ Target alternative platforms from single codebase

    View Slide

  39. UNIKERNEL PROJECTS
    ▸ ClickOS
    ▸ Clive
    ▸ Drawbridge
    ▸ HaLVM
    ▸ IncludeOS
    ▸ LING
    ▸ MirageOS
    ▸ OSv
    ▸ Rumprun
    ▸ runtime.js
    UNIKERNELS

    View Slide

  40. TWO APPROACHES
    Consider

    Legacy
    Clean

    Slate
    UNIKERNELS

    View Slide

  41. TWO APPROACHES
    Consider

    Legacy
    UNIKERNELS

    View Slide

  42. TWO APPROACHES
    Clean

    Slate
    UNIKERNELS

    View Slide

  43. MIRAGEOS
    unikernel
    }
    UNIKERNELS

    View Slide

  44. EXAMPLE: STATIC WEBSITE

    View Slide

  45. EXAMPLE: STATIC WEBSITE
    UNIKERNELS

    View Slide

  46. EXAMPLE: STATIC WEBSITE
    UNIKERNELS

    View Slide

  47. EXAMPLE: STATIC WEBSITE
    UNIKERNELS

    View Slide

  48. EXAMPLE: STATIC WEBSITE
    ~200MB

    2560 kloc
    UNIKERNELS

    View Slide

  49. EXAMPLE: STATIC WEBSITE
    ~200MB

    2560 kloc
    ~8.2MB

    102 kloc
    UNIKERNELS

    View Slide

  50. EXAMPLE: STATIC WEBSITE
    ▸ Highly specialised
    ▸ Immutable by default
    ▸ Well defined interfaces
    ▸ Minimal attack surface
    UNIKERNELS

    View Slide

  51. View Slide

  52. REST OF THE SYSTEM?
    ▸ CI/CD Pipelines?
    ▸ Orchestration?
    ▸ Monitoring?
    ▸ Logging?
    UNIKERNELS

    View Slide

  53. View Slide

  54. SIMILAR IDEAS
    UNIKERNELS

    View Slide

  55. NEW CONSIDERATIONS

    View Slide

  56. NEW CONSIDERATIONS
    ▸ Pipelines

    From Dev to Production

    Staged roll-outs

    Tool building

    View Slide

  57. NEW CONSIDERATIONS
    ▸ Pipelines

    From Dev to Production

    Staged roll-outs

    Tool building
    ▸ Security

    New threat models

    Maintenance burden

    Dependency tracking

    View Slide

  58. NEW CONSIDERATIONS
    ▸ Pipelines

    From Dev to Production

    Staged roll-outs

    Tool building
    ▸ Orchestration

    Beyond ‘cloud native’

    Embedded devices
    ▸ Security

    New threat models

    Maintenance burden

    Dependency tracking

    View Slide

  59. NEW CONSIDERATIONS
    ▸ Pipelines

    From Dev to Production

    Staged roll-outs

    Tool building
    ▸ Orchestration

    Beyond ‘cloud native’

    Embedded devices
    ▸ Security

    New threat models

    Maintenance burden

    Dependency tracking
    ▸ Designing systems

    Logging strategies

    Monitoring systems

    No longer just ‘apps’

    No one-size-fits-all

    View Slide

  60. NEW CONSIDERATIONS … SOME THINGS EXIST

    View Slide

  61. NEW CONSIDERATIONS
    We are all

    Systems Thinkers

    now

    View Slide

  62. SUMMARY

    View Slide

  63. SUMMARY
    ENGINEERING FOR THE FUTURE

    View Slide

  64. SUMMARY
    ▸ Software is eating the world

    Cloud, Embedded devices. etc
    ENGINEERING FOR THE FUTURE

    View Slide

  65. SUMMARY
    ▸ Software is eating the world

    Cloud, Embedded devices. etc
    ▸ Must revisit legacy assumptions

    So we don’t carry forward tech debt
    ENGINEERING FOR THE FUTURE

    View Slide

  66. SUMMARY
    ▸ Software is eating the world

    Cloud, Embedded devices. etc
    ▸ Must revisit legacy assumptions

    So we don’t carry forward tech debt
    ▸ Everyone will be a systems dev

    At least for a while :)
    ENGINEERING FOR THE FUTURE

    View Slide

  67. SUMMARY
    ▸ Software is eating the world

    Cloud, Embedded devices. etc
    ▸ Must revisit legacy assumptions

    So we don’t carry forward tech debt
    ▸ Everyone will be a systems dev

    At least for a while :)
    ▸ Exciting time to be a dev!
    ENGINEERING FOR THE FUTURE

    View Slide

  68. THANK YOU!

    @amirmc

    View Slide

  69. THANK YOU!

    DISCUSS?
    @amirmc

    View Slide