exactpro.com 1 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com Agile-Testing and Guinea Pigs Iosif Itkin CEO and co-founder, Exactpro

exactpro.com 2 BUILD SOFTWARE TO TEST SOFTWARE ● https://www.linkedin.com/in/iosifitkin/ ● Co-chair of FIX Trading Community MOST Working Group (Monitoring, Onboarding and Software Testing) ● Exactpro representative at the Zero Outage Industry Standard association ● Head of LOST (Laboratory of Software Testing) ● Co-founder of Tools & Methods of Program Analysis Conference ● ISTQB CFTL, A4Q AI and Software Testing About me

exactpro.com 3 BUILD SOFTWARE TO TEST SOFTWARE ● Software Testing Definition ● Build vs. Test ● What is Agility? ● How to Evaluate Software Testing? ● Agile Testing Definition ● Financial Market Infrastructures ● HP LaserJet Case Study ● About Exactpro Contents

exactpro.com 4 BUILD SOFTWARE TO TEST SOFTWARE What Do These Terms Have In Common? Agile Testing Test Driven Development Unit Tests

exactpro.com 5 BUILD SOFTWARE TO TEST SOFTWARE Guinea Pig (“Sea” Pig)

exactpro.com 6 BUILD SOFTWARE TO TEST SOFTWARE Guinea Pig (“Sea” Pig)

exactpro.com 7 BUILD SOFTWARE TO TEST SOFTWARE Software Testing is The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of a component or system and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. Software Testing Definition https://www.youtube.com/c/ExactproSystems

exactpro.com 8 BUILD SOFTWARE TO TEST SOFTWARE Software Testing is exploring software with the intent of finding bugs. Software Testing is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software Testing Definition Information Service

exactpro.com 9 BUILD SOFTWARE TO TEST SOFTWARE Software Definition Software is a collection of instructions and data that tell a computer how to work The computer (hardware) is able to execute the software. Each instruction causes the computer to carry out an operation - moving data, carrying out a computation, altering the control flow of instructions or manipulating hardware

exactpro.com 10 BUILD SOFTWARE TO TEST SOFTWARE NO AUTOMATION DRIVER ASSISTANCE PARTIAL AUTOMATION CONDITIONAL AUTOMATION HIGH AUTOMATION FULL AUTOMATION Steering and acceleration/ deceleration Monitoring of driving environment Fallback when automation fails Automated Control: Human driver monitors the road Automated driving system monitors the road SOME DRIVING MODES SOME DRIVING MODES SOME DRIVING MODES SOME DRIVING MODES 0 1 2 3 4 5 Human driver Automated system Autonomy

exactpro.com 11 BUILD SOFTWARE TO TEST SOFTWARE Models Abstract Model = theory Concrete Description = data Software = object of study flow of information observe compare Abstract Model = design concept Concrete Description = specification Software = useful product flow of information design produce Testing Development The Antiparallel Structures of Software Testing and Software Development Software Testing vs Software Development

exactpro.com 12 BUILD SOFTWARE TO TEST SOFTWARE Scout mindset: the motivation to see things as they are, not as you wish they were. Julia Galef ”

exactpro.com 13 BUILD SOFTWARE TO TEST SOFTWARE Agile Testing Definition Wikipedia: Agile testing is a software testing practice that follows the principles of agile software development ISTQB: Testing practice for a project using Agile software development methodologies, incorporating techniques and methods, such as extreme programming (XP), treating development as the customer of testing and emphasizing the test-first design paradigm Lisa Crispin, Janet Gregory: Like a lot of terminology, “agile development” and “agile testing” mean different things to different people… When we say “agile testing” in this book, we’re usually talking about business-facing tests, tests that define the business experts’ desired features and functionality… Agile testing doesn’t just mean testing on an agile project. Some testing approaches, such as exploratory testing, are inherently agile, whether it’s done an agile project or not….

exactpro.com 14 BUILD SOFTWARE TO TEST SOFTWARE Agile Testing Definition Lisa Crispin, Janet Gregory: Chapter 1. What Is Agile Testing, Anyway? Like a lot of terminology, “agile development” and “agile testing” mean different things to different people… When we say “agile testing” in this book, we’re usually talking about business-facing tests, tests that define the business experts’ desired features and functionality… Agile testing doesn’t just mean testing on an agile project. Some testing approaches, such as exploratory testing, are inherently agile, whether it’s done an agile project or not…. Chapter 2. Ten Principles for Agile Testers We define an agile tester this way: a professional tester who embraces change, collaborates well with both technical and business people, and understands the concept of using tests to document requirements and drive development… Who’s an agile tester? She’s a team member who drives agile testing….

exactpro.com 15 BUILD SOFTWARE TO TEST SOFTWARE Agile Testing and More Agile Testing

exactpro.com 16 BUILD SOFTWARE TO TEST SOFTWARE Agile Testing Quadrants

exactpro.com 17 BUILD SOFTWARE TO TEST SOFTWARE Gojko Adzic’s Version of the Agile Testing Quadrants

exactpro.com 18 BUILD SOFTWARE TO TEST SOFTWARE Refactoring the Agile Testing Quadrants https://www.developsense.com/presentations/2019-05-RefactoringtheAgileTestingQuadrants.pdf

exactpro.com 19 BUILD SOFTWARE TO TEST SOFTWARE Refactoring the Agile Testing Quadrants https://www.developsense.com/presentations/2019-05-RefactoringtheAgileTestingQuadrants.pdf

exactpro.com 20 BUILD SOFTWARE TO TEST SOFTWARE ● Accuracy ● Relevance ● Interpretability ● Accessibility Information Quality Characteristics

exactpro.com 21 BUILD SOFTWARE TO TEST SOFTWARE Ironic (Song) a state of affairs or an event that seems deliberately contrary to what was or might be expected; an outcome cruelly, humorously, or strangely at odds with assumptions or expectations an incongruity between the actual result of a sequence of events and the normal or expected result … It's a black fly in your Chardonnay It's a death row pardon two minutes too late And isn't it ironic, don't you think? It's like rain on your wedding day It's a free ride when you've already paid …

exactpro.com 22 BUILD SOFTWARE TO TEST SOFTWARE Relevance of Information in Context

exactpro.com 23 BUILD SOFTWARE TO TEST SOFTWARE Software Testing is Context Dependent

exactpro.com 24 BUILD SOFTWARE TO TEST SOFTWARE About Exactpro

exactpro.com 25 BUILD SOFTWARE TO TEST SOFTWARE

exactpro.com 26 BUILD SOFTWARE TO TEST SOFTWARE MANAGED SUPPORT SERVICES PROVIDER OF THE YEAR In this Risk.net interview, Alexey Zverev, Exactpro co-CEO and co-founder, talks about mitigating risks while driving innovation during the time of high volatility and explains what made Exactpro stand out among the contestants. Watch the interview with Risk.net via the QR code above.

exactpro.com 27 BUILD SOFTWARE TO TEST SOFTWARE Complex Systems

exactpro.com 28 BUILD SOFTWARE TO TEST SOFTWARE Never invest in a business you cannot understand “ Warren Buffett

exactpro.com 29 BUILD SOFTWARE TO TEST SOFTWARE Agility is Responsiveness to Change https://arxiv.org/pdf/1909.10082.pdf

exactpro.com 30 BUILD SOFTWARE TO TEST SOFTWARE ● Accuracy ● Relevance ● Interpretability ● Accessibility Information Quality Characteristics http://mitiq.mit.edu/Documents/Publications/TDQMpub/14_Beyond_Accuracy.pdf

exactpro.com 31 BUILD SOFTWARE TO TEST SOFTWARE Optimize Feedback Loop Manual Using Tools Mechanization Automation A tool is an object used to extend the ability of an individual to modify features of the surrounding environment Automation is the technology by which a process or procedure is performed with minimal human assistance

exactpro.com 32 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software

exactpro.com 33 BUILD SOFTWARE TO TEST SOFTWARE Agile Testing Quadrants

exactpro.com 34 BUILD SOFTWARE TO TEST SOFTWARE Exploratory Testing is Testing https://www.satisfice.com/blog/archives/1509

exactpro.com 35 BUILD SOFTWARE TO TEST SOFTWARE Duplication is Not Necessarily a Bad Thing

exactpro.com 36 BUILD SOFTWARE TO TEST SOFTWARE Back-up Plans https://twitter.com/StanfordGSB/ status/564839950521090048

exactpro.com 37 BUILD SOFTWARE TO TEST SOFTWARE Independent Software Testing Team ● Focus ● Objectivity ● Capacity ● Diversity ● Resilience

exactpro.com 38 BUILD SOFTWARE TO TEST SOFTWARE Defense in Depth

exactpro.com 39 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development Success

exactpro.com 40 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development

exactpro.com 41 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development

exactpro.com 42 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development hp laserjet firmware team 10% - code integration 20% - detailed planning 25% - porting code 25% - current product support 15% - manual testing ~5% - innovation 2% - continuous integration 5% - agile planning 15% - one main branch 10% - one branch cpe 5% - most testing automated ~40% - innovation 2008 2011

exactpro.com 43 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development hp laserjet firmware team 10% - code integration 20% - detailed planning 25% - porting code 25% - current product support 15% - manual testing ~5% - innovation 2% - continuous integration 5% - agile planning 15% - one main branch 10% - one branch cpe 5% - most testing automated ~40% - innovation 2008 2011 100% 77%

exactpro.com 44 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development hp laserjet firmware team 10% - code integration 20% - detailed planning 25% - porting code 25% - current product support 15% - manual testing ~5% - innovation 2% - continuous integration 5% - agile planning 15% - one main branch 10% - one branch cpe 5% - most testing automated ~40% - innovation 2008 2011 100% 77% The remaining 23% on RHS is spent on managing automated tests 23%

exactpro.com 45 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development

exactpro.com 46 BUILD SOFTWARE TO TEST SOFTWARE Large-Scale Agile Development hp laserjet firmware team 10% - code integration 20% - detailed planning 25% - porting code 25% - current product support 15% - manual testing ~5% - innovation 2% - continuous integration 5% - agile planning 15% - one main branch 10% - one branch cpe 5% - most testing automated ~40% - innovation 2008 2011 100% The remaining 23% on RHS is spent on managing automated tests 3% - test tools 10% - scripting checks 10% - failure triage

exactpro.com 47 BUILD SOFTWARE TO TEST SOFTWARE ● CI/CD is Software, TDD/BDD/Agile Testing is Software Development ● Software Testing is an Information Service ● Agility is Responsiveness to Change ● Large-Scale Agile Requires Investment in Software Testing Conclusion

exactpro.com 48 BUILD SOFTWARE TO TEST SOFTWARE FOLLOW US ON SOCIAL MEDIA: @exactprosystems TO #th2net

exactpro.com 49 BUILD SOFTWARE TO TEST SOFTWARE Thank You