Tweet
Tweet

Slide 1

Slide 1 text

@y3key linkedin.com/in/stefan-friese-12205 05/31/2018 https://speakerdeck.com/y3key/ continuous-testing

Slide 2

Slide 2 text

Presentation title | Month 00, 2016 2 © 2016 HERE | HERE Internal Use Only Unit + Component Tests API / Service Layer Tests Service-integration Tests GUI Layer Tests Test Pyramid Exploratory Tests

Slide 3

Slide 3 text

Presentation title | Month 00, 2016 4 © 2016 HERE | HERE Internal Use Only Let’s turn it upside down …

Slide 4

Slide 4 text

Presentation title | Month 00, 2016 5 © 2016 HERE | HERE Internal Use Only [More Information: Search for “Ice Cream Anti-Pattern”] Unit Tests Service-level Tests Service-integration Tests E2E Tests

Slide 5

Slide 5 text

Presentation title | Month 00, 2016 6 © 2016 HERE | HERE Internal Use Only Increased test debt  Test debt occurs due to shortcuts related to testing. It is the test dimension of technical debt.

Slide 6

Slide 6 text

Presentation title | Month 00, 2016 7 © 2016 HERE | HERE Internal Use Only The right ratio of tests – with many unit tests as basis - is important! But how can we mature the test execution?

Slide 7

Slide 7 text

Presentation title | Month 00, 2016 8 © 2016 HERE | HERE Internal Use Only Let us return to the Test Pyramid …

Slide 8

Slide 8 text

Presentation title | Month 00, 2016 9 © 2016 HERE | HERE Internal Use Only … but let’s change the Perspective

Slide 9

Slide 9 text

Presentation title | Month 00, 2016 10 © 2016 HERE | HERE Internal Use Only Continuous Integration & Continuous Delivery Faster Feedback More Stable Less Expensive to fix Later Feedback More Fragile More Expensive to fix commit deploy & release [More Information: Search for “CI&CD”, “Shift Left Testing”]

Slide 10

Slide 10 text

Presentation title | Month 00, 2016 12 © 2016 HERE | HERE Internal Use Only The Shift Left is essential. However, we can’t replace all long running tests with unit tests. How can we reduce the test debt further?

Slide 11

Slide 11 text

Presentation title | Month 00, 2016 14 © 2016 HERE | HERE Internal Use Only development stable increased confidence

Slide 12

Slide 12 text

Presentation title | Month 00, 2016 15 © 2016 HERE | HERE Internal Use Only Shift Right commit deploy incremental release & monitoring [More Information: Search for “Test in Production”]

Slide 13

Slide 13 text

Presentation title | Month 00, 2016 16 © 2016 HERE | HERE Internal Use Only We talked about Test Debt: Why should I test in Production?

Slide 14

Slide 14 text

Presentation title | Month 00, 2016 17 © 2016 HERE | HERE Internal Use Only How well can you predict how your application will be used in production?

Slide 15

Slide 15 text

Presentation title | Month 00, 2016 18 © 2016 HERE | HERE Internal Use Only Let us return to the Test Pyramid …

Slide 16

Slide 16 text

Presentation title | Month 00, 2016 19 © 2016 HERE | HERE Internal Use Only … but let’s change (again) the Perspective

Slide 17

Slide 17 text

Presentation title | Month 00, 2016 20 © 2016 HERE | HERE Internal Use Only Bug Filter (Semantic) Monitoring Some Integration and E2E Tests small risk high risk that bugs are not detected Many Unit Tests! Comprehensive Acceptance Test Set Incremental Release A/B Tests Alpha/Beta Releases Controlled Experiments (e.g. “ChaosMonkey”) Contract Tests Pre- Production Tests Production Tests Based on: https://t.co/ereKYJzyzM

Slide 18

Slide 18 text

Presentation title | Month 00, 2016 21 © 2016 HERE | HERE Internal Use Only Based on: https://danashby.co.uk/2016/10/19/continuous-testing-in-devops Dev Ops Test Here Test Here Test Here

Slide 19

Slide 19 text

Presentation title | Month 00, 2016 22 © 2016 HERE | HERE Internal Use Only That’s Continuous Testing ☺

Slide 20

Slide 20 text

Presentation title | Month 00, 2016 23 © 2016 HERE | HERE Internal Use Only Shadow Deployment Tests* Load Tests Canary Release* Feature Flagging Monitoring A/B Test – lean experimentation Alpha/Beta Tests Chaos Engineering Synthetic Monitoring Pre-Release Production Traffic Tests Release Tests in Production Post-Release Tests in Production See: https://www.infoq.com/articles/architecturally-aligned-testing

Slide 21

Slide 21 text

Presentation title | Month 00, 2016 24 © 2016 HERE | HERE Internal Use Only production instances test instance production- like instance frontend end users load balancer See: https://www.infoq.com/articles/performance-tests Shadow Deployment

Slide 22

Slide 22 text

Presentation title | Month 00, 2016 26 © 2016 HERE | HERE Internal Use Only production instances frontend end users load balancer new version (v1.1) current release (v1.0) current release (v1.0) current release (v1.0) canary instance

Slide 23

Slide 23 text

Presentation title | Month 00, 2016 27 © 2016 HERE | HERE Internal Use Only Shift in Mindset Shift Left • Early and fast feedback. • Comprehensive set of automated pre-production tests. Shift Right • Release often and early. • Expose new features early in a controlled way. • Execute some of the tests in Production.

Slide 24

Slide 24 text

Presentation title | Month 00, 2016 28 © 2016 HERE | HERE Internal Use Only More information: • Dan Ashby (3 May 2018): A Better Testing Pyramid - https://danashby.co.uk/2018/05/03/a-better-testing-pyramid • Ham Vocke (26 Feb 2018): The Practical Test Pyramid - https://martinfowler.com/articles/practical-test-pyramid.html • Noah Sussman (28 Feb 2017): Abandoning the Pyramid Of Testing in favor of a Band- Pass model of risk management - https://twitter.com/noahsussman/status/836612175707930625 • Katrin Clockie (2017): A Practical Guide to Testing in DevOps. Leanpub. • Stefan Friese (2016): Testing the Unexpected: A Shift Right in DevOps Testing. https://www.stickyminds.com/article/testing-unexpected-shift-right-devops-testing • Stefan Friese (2018): https://speakerdeck.com/y3key/continuous-testing

Slide 25

Slide 25 text

Presentation title | Month 00, 2016 29 © 2016 HERE | HERE Internal Use Only