Slide 1

Slide 1 text

Eberhard Wolff Fellow INNOQ @ewolff http://ewolff.com

Slide 2

Slide 2 text

http://continuous-delivery-book.com/

Slide 3

Slide 3 text

http://microservices-book.com/

Slide 4

Slide 4 text

http://microservices-book.com/ primer.html FREE!!!!

Slide 5

Slide 5 text

http://practical-microservices.com/

Slide 6

Slide 6 text

http://practical-microservices.com/ recipes.html FREE!!!!

Slide 7

Slide 7 text

Continuous Delivery – Why Do I Even Care?

Slide 8

Slide 8 text

Faster Feedback Implementation Production Deployment

Slide 9

Slide 9 text

Lower Risk • Each deployment contains less changes • Lower risk of a bug • Easier to fall back • …or add other safeguards Quarterly Release Daily Release

Slide 10

Slide 10 text

Continuous Delivery A Proven Success! • 2017 State of DevOps Reports by DORA based on 27.000 surveys • Failure rate multiple deploys a day: 0-15% • Failure rate once a week to once a month: 31-45% https://puppet.com/resources/whitepaper/state-of-devops-report

Slide 11

Slide 11 text

Higher Reliability • Automated deployment and tests • …easy to reproduce • ...faster • ...executed frequently Commit Stage Automated Acceptance Testing Automated Capacity Testing Manual Explorative Testing Release ❌ ✅ ❌ ✅ ❌ ✅

Slide 12

Slide 12 text

Principles Agile Manifesto Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Slide 13

Slide 13 text

Continuous Delivery: Why Do I Even Care? • Faster Feedback • Lower Risk • Higher Reliability • Value to the customer

Slide 14

Slide 14 text

Architecture

Slide 15

Slide 15 text

Architecture • Structure • Goal often: Easy changeability

Slide 16

Slide 16 text

Continuous Delivery FTW! • Usually changeability = well-structured software • What is easier to change? • …well-structured software? • …or software with extensive automated tests? ❌ ✅

Slide 17

Slide 17 text

Continuous Delivery FTW! • Usually changeability = well-structured software • What would you rather change? • …well-structured software • …or software that goes in production reliably and fully automated?

Slide 18

Slide 18 text

Architecture Continuous Delivery CD supports architecture goals like changeability

Slide 19

Slide 19 text

Implementing Continuous Delivery

Slide 20

Slide 20 text

Customer Scenario • Quarterly releases • 10 weeks of testing • Release two days over the weekend • Goal: Several deployments a day Development Testing Release Testing + Release + Development Time

Slide 21

Slide 21 text

Customer Scenario: Automation • Development faster – smaller batches • Assumption: Test automation = 100x speedup for test • 4h Tests Development Testing Release Testing

Slide 22

Slide 22 text

Customer Scenario: Automation • Assumption: Deployment automation = 4x speedup • 2h Deployment Release Testing

Slide 23

Slide 23 text

Customer Scenario: Automation • 2h Deployment + 4h tests = 6 hours = 1 deployment per day Need another threefold improvement Release Testing

Slide 24

Slide 24 text

Automation • Automation gives limited speedup • Necessary for reliability • Is automation the only knob?

Slide 25

Slide 25 text

Architecture • What are the modules? • Make modules independently deployable! • …continuous delivery pipelines are much simpler. • …deployment is less risky. • Microservices!

Slide 26

Slide 26 text

Microservices • Another knob to tune deployment • Don’t ignore this option!

Slide 27

Slide 27 text

Architecture Continuous Delivery CD supports architecture goals like changeability Architecture must support Continuous Delivery

Slide 28

Slide 28 text

Microservices

Slide 29

Slide 29 text

Microservices are not a Magic Wand

Slide 30

Slide 30 text

Independent Systems Architecture: ISA 9 Microservices Best Prac:ces Creator: INNOQ | http://isa-principles.org

Slide 31

Slide 31 text

1 | Modules Creator: INNOQ | h5p://isa-principles.org

Slide 32

Slide 32 text

Creator: INNOQ | h/p://isa-principles.org >Reuse “Module” ideas: >High cohesion, low coupling, >SeparaCon of concerns, >Single Responsibility … 1 | Modules

Slide 33

Slide 33 text

2 | Container Creator: INNOQ / h3p://isa-principles.org 2 | Container Creator: INNOQ / h3p://isa-principles.org

Slide 34

Slide 34 text

Creator: INNOQ | http://isa-principles.org >Modules = containers (or VMs, processes …) 2 | Container

Slide 35

Slide 35 text

6 | Independent Con,nuous Delivery Pipeline Creator: INNOQ | h

Slide 36

Slide 36 text

Creator: INNOQ | h/p://isa-principles.org >Microservices can only be deployed independently … >... if pipelines are independent 6 | Independent ConBnuous Delivery Pipeline

Slide 37

Slide 37 text

Making Microservices Actually Work

Slide 38

Slide 38 text

Interfaces

Slide 39

Slide 39 text

Interfaces • Microservice have interfaces to other microservices • Changing an interface means two microservices need to be changed

Slide 40

Slide 40 text

Invoice Uses Interface Version 42 Order Provides Interface Version 42 Invoice V43 Order V43 Deployment of Order and Invoice

Slide 41

Slide 41 text

Invoice Uses Interface version 42 Order Provides Interface version 42 Invoice V43 Order V43 Deployment of order and invoice Dependent Deployment Pipelines!

Slide 42

Slide 42 text

Invoice Uses V42 Order Provides V42 Invoice Uses V42 Order Provides V43 Backwards Compatible Invoice Uses V43 Order Provides V43 Deployment of Order Deployment of Invoice

Slide 43

Slide 43 text

What about not backwards compatible changes?

Slide 44

Slide 44 text

Invoice Uses V42 Order Provides V42 Invoice Uses V42 Order Provides V43 & V42 Invoice Uses V43 Order Provides V43 & V42 Deployment of Order Deployment of Invoice Invoice Uses V43 Order Provides V43 Deployment of Order

Slide 45

Slide 45 text

But… • …this is hard! • …but otherwise: no independent deployment • Worst case: everything must be deployed in one step. • Deployment monolith

Slide 46

Slide 46 text

Tests

Slide 47

Slide 47 text

Tests • It is not enough to separate microservices • For independent deployment pipelines, tests must be separated, too.

Slide 48

Slide 48 text

Order Continuous Delivery Pipeline Integration Tests Invoice Continuous Delivery Pipeline Search Continuous Delivery Pipeline Production

Slide 49

Slide 49 text

Order Continuous Delivery Pipeline Integration Tests Invoice Continuous Delivery Pipeline Search Continuous Delivery Pipeline Production

Slide 50

Slide 50 text

Integration Test = Bottleneck

Slide 51

Slide 51 text

Invoice (Consumer) Order (Provider) Request Reply

Slide 52

Slide 52 text

Invoice (Consumer) Stub Request Reply

Slide 53

Slide 53 text

Stubs • Does the calling module (invoice) work as expected? • Use stubs to replace called module

Slide 54

Slide 54 text

Order Continuous Delivery Pipeline Integration Tests Invoice Continuous Delivery Pipeline Search Continuous Delivery Pipeline Production

Slide 55

Slide 55 text

Invoice (Consumer) Order (Provider) Request Reply

Slide 56

Slide 56 text

Consumer- Driven Contract Test Order (Provider) Request Reply

Slide 57

Slide 57 text

Consumer-driven Contract Test • Does the called module (order) work as expected? • Use consumer-driven contract test to replace calling module • Easy to test changes to interfaces

Slide 58

Slide 58 text

Consumer-driven Contract Test • A pattern: Invoice provides a test for Order • …might be written with any testing framework • Pact test can be written in one programming language • …recorded as JSON • …and can be executed with a different language • Technology independence • https://github.com/mvitz/pact-example

Slide 59

Slide 59 text

Order Continuous Delivery Pipeline Integration Tests Invoice Continuous Delivery Pipeline Search Continuous Delivery Pipeline Production

Slide 60

Slide 60 text

Database

Slide 61

Slide 61 text

Database Changes Are Hard • Schema changes • Converting existing data • Potential lots of data • Takes long • Might fail • Hard or even impossible to rollback

Slide 62

Slide 62 text

Dealing with Database Changes • Backwards compatible changes only • Support previous version schema • Default value for new columns • Incompatible changes after software changes • Remove column only if not used any more • …

Slide 63

Slide 63 text

Keep Database Schema Stable • Change database before microservices use it • Change database seldom • E.g. once a week • i.e. schema changes not part of deployment • No rollback necessary

Slide 64

Slide 64 text

Invoice Uses Less Columns Invoice Invoice Uses New Columns Database Schema V24 Database Schema V23 Database Schema V24 Weekly Database Deployment Invoice Uses Database Schema V23 Database Schema V23 Deployment of Invoice Deployment of Invoice

Slide 65

Slide 65 text

Domain Architecture

Slide 66

Slide 66 text

Invoice Order • New features need changes to both microservices. • Deployment fully decoupled • …but that is useless • …as both will be deployed anyways • Also: Order too complex

Slide 67

Slide 67 text

Order Shipping address Tracking # Items Item Categories Priority shipping Customs # Account # ... Credit card # Order #

Slide 68

Slide 68 text

Bounded Context • Domain model is only valid for one context • There is no universal data model! • See all failed SOA attempts

Slide 69

Slide 69 text

Order Shipping address Tracking # Items Item Categories Priority shipping Customs # Account # ... Credit card # Order # Customs Order Recommen- dations Order Tracking Order Shipping address Tracking # Item Categories Priority shipping Customs # Invoice Order Account # Credit card #

Slide 70

Slide 70 text

Bounded Context • Decoupled domain architecture • Features need changes to one bounded context only. • Implement bounded contexts as microservices • …to gain independent microservices • …and really independent deployment.

Slide 71

Slide 71 text

Conclusion

Slide 72

Slide 72 text

Architecture Continuous Delivery CD supports architecture goals like changeability Architecture must support Continuous Delivery

Slide 73

Slide 73 text

Architecture for Continuous Delivery • Separately deployable modules • Aka microservices • Decouple deployments for interface changes! • Decouple tests! • Decouple database changes! • Use bounded context for decoupled domain architecture!

Slide 74

Slide 74 text

Architecture and Continuous Delivery • Architecture and continuous delivery • …two battles at the same time? • Split apart a microservice! • Easy to build a continuous delivery pipeline for a new microservice! • So: Synergy when introducing microservices and continuous delivery at the same time.