ITT 2018 - Amir Chaudhry - Engineering for a more distributed future

In the last few years we've had several waves of change from microservices, to containerisation, and more recently serverless and unikernels. Even the notion of an 'application' has changed over the years, as we now have to consider mobile apps, back-end AI algorithms, and even remote devices like IoT and autonomous vehicles.

The general idea of these approaches point in the same direction — that of applications becoming a collection of components that perform discrete tasks. However, since the architecture of such applications is more distributed, there are many trade-offs and new considerations. Including security implications, orchestration, monitoring, and much more. These are things that engineers need to become more aware of when creating and maintaining such systems. This talk will cover some of the issues that engineers should be aware of and we'll also discuss some of the tools and technologies that might be useful to consider.

Transcript

1. ENGINEERING FOR A DISTRIBUTED FUTURE AMIR CHAUDHRY Istanbul Tech Talk
 17 Apr 2018
2. Amir Chaudhry
 
 @amirmc
3. ENGINEERING FOR THE FUTURE OVERVIEW ▸ Software today ▸ A brief history ▸ Fresh approaches ▸ Unikernels ▸ New considerations ▸ Discussion
4. SOFTWARE TODAY…
5. SOFTWARE TODAY… … is an application… ENGINEERING FOR THE FUTURE
6. SOFTWARE TODAY… … is an application… … on top of an Operating System. ENGINEERING FOR THE FUTURE
7. A BRIEF HISTORY
8. FIRST PRODUCTION COMPUTER
9. The EDSAC Actual customers 3 Nobel Prizes
10. Full programs Punched tape Self contained
11. The subroutine Reusable code Efficient programs
12. TOWARDS THE OS
16. Carried forward
 legacy assumptions
17. ENGINEERING FOR THE FUTURE SOFTWARE TODAY… … is an application… … on top of an Operating System.
18. SOFTWARE TODAY… ▸ Multiple users? ▸ Multiple apps? ▸ System services? ▸ Physical hardware? ENGINEERING FOR THE FUTURE
19. Code you
 care about Code the OS
 insists you need
20. Code you
 care about Code the OS
 insists you need
21. STILL MORE COMPLICATED
22. SOFTWARE TODAY… … is built locally… ENGINEERING FOR THE FUTURE
23. SOFTWARE TODAY… … but deployed remotely… … is built locally… ENGINEERING FOR THE FUTURE
24. SOFTWARE TODAY… … is built locally… … but deployed remotely… … very remotely. ENGINEERING FOR THE FUTURE
27. SOFTWARE TODAY… … is complex! Even though we simplify aspects of it ENGINEERING FOR THE FUTURE
28. THINGS ARE GETTING EASIER
29. THINGS ARE GETTING EASIER… ENGINEERING FOR THE FUTURE
31. Carry forward 
 legacy assumptions
33. SUMMARY SO FAR ▸ World looks very different: ▸ Model of deployment is changing ▸ How we write apps is changing ▸ But underlying assumptions are still baked in ENGINEERING FOR THE FUTURE
34. FRESH APPROACHES
35. FRESH(ISH) APPROACHES ▸ Move to microservices
 Cattle vs Pets
 Multi-repo apps
 Containers
 Unikernels ▸ More distributed approaches ▸ Immutable artefacts ▸ Design systems not just apps
36. FRESH APPROACHES A SPECTRUM OF OPTIONS Monolith Microservices
37. UNIKERNELS?
38. UNIKERNELS ABOUT UNIKERNELS ▸ Represent an extreme view ▸ Constructed from Library OSs ▸ OS functionality available as modular components ▸ Link only the OS functionality your app needs ▸ Target alternative platforms from single codebase
39. UNIKERNEL PROJECTS ▸ ClickOS ▸ Clive ▸ Drawbridge ▸ HaLVM ▸ IncludeOS ▸ LING ▸ MirageOS ▸ OSv ▸ Rumprun ▸ runtime.js UNIKERNELS
40. TWO APPROACHES Consider
 Legacy Clean
 Slate UNIKERNELS
41. TWO APPROACHES Consider
 Legacy UNIKERNELS
42. TWO APPROACHES Clean
 Slate UNIKERNELS
43. MIRAGEOS unikernel } UNIKERNELS
44. EXAMPLE: STATIC WEBSITE
45. EXAMPLE: STATIC WEBSITE UNIKERNELS
46. EXAMPLE: STATIC WEBSITE UNIKERNELS
47. EXAMPLE: STATIC WEBSITE UNIKERNELS
48. EXAMPLE: STATIC WEBSITE ~200MB
 2560 kloc UNIKERNELS
49. EXAMPLE: STATIC WEBSITE ~200MB
 2560 kloc ~8.2MB
 102 kloc UNIKERNELS
50. EXAMPLE: STATIC WEBSITE ▸ Highly specialised ▸ Immutable by default ▸ Well defined interfaces ▸ Minimal attack surface UNIKERNELS
52. REST OF THE SYSTEM? ▸ CI/CD Pipelines? ▸ Orchestration? ▸ Monitoring? ▸ Logging? UNIKERNELS
54. SIMILAR IDEAS UNIKERNELS
55. NEW CONSIDERATIONS
56. NEW CONSIDERATIONS ▸ Pipelines
 From Dev to Production
 Staged roll-outs
 Tool building
57. NEW CONSIDERATIONS ▸ Pipelines
 From Dev to Production
 Staged roll-outs
 Tool building ▸ Security
 New threat models
 Maintenance burden
 Dependency tracking
58. NEW CONSIDERATIONS ▸ Pipelines
 From Dev to Production
 Staged roll-outs
 Tool building ▸ Orchestration
 Beyond ‘cloud native’
 Embedded devices ▸ Security
 New threat models
 Maintenance burden
 Dependency tracking
59. NEW CONSIDERATIONS ▸ Pipelines
 From Dev to Production
 Staged roll-outs
 Tool building ▸ Orchestration
 Beyond ‘cloud native’
 Embedded devices ▸ Security
 New threat models
 Maintenance burden
 Dependency tracking ▸ Designing systems
 Logging strategies
 Monitoring systems
 No longer just ‘apps’
 No one-size-fits-all
60. NEW CONSIDERATIONS … SOME THINGS EXIST
61. NEW CONSIDERATIONS We are all
 Systems Thinkers
 now
62. SUMMARY
63. SUMMARY ENGINEERING FOR THE FUTURE
64. SUMMARY ▸ Software is eating the world
 Cloud, Embedded devices. etc ENGINEERING FOR THE FUTURE
65. SUMMARY ▸ Software is eating the world
 Cloud, Embedded devices. etc ▸ Must revisit legacy assumptions
 So we don’t carry forward tech debt ENGINEERING FOR THE FUTURE
66. SUMMARY ▸ Software is eating the world
 Cloud, Embedded devices. etc ▸ Must revisit legacy assumptions
 So we don’t carry forward tech debt ▸ Everyone will be a systems dev
 At least for a while :) ENGINEERING FOR THE FUTURE
67. SUMMARY ▸ Software is eating the world
 Cloud, Embedded devices. etc ▸ Must revisit legacy assumptions
 So we don’t carry forward tech debt ▸ Everyone will be a systems dev
 At least for a while :) ▸ Exciting time to be a dev! ENGINEERING FOR THE FUTURE
68. THANK YOU!
 @amirmc
69. THANK YOU!
 DISCUSS? @amirmc