Lecture 4 - Foundations of Software Testing

"Software Testing for Complex Intelligent Systems and Autonomous Vehicles" online course.

The lecture is available via the link: https://youtu.be/NiZG8SkV-ps
--
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. 1 Build Software to Test Software exactpro.com Lecture 4 Foundations of Software Testing Software Testing for Complex Intelligent Systems and Autonomous Vehicles Online Course
2. 2 Build Software to Test Software exactpro.com Lecture structure Sources of information about testing What is testing? What is quality, errors, defects, failures? What testing is not: QA, unit testing, BDD, bad testing Principles and types of testing Testing objectives; what constitutes good testing Testing Components: processes, platforms, people The testing process: phases and results Is it the testers' fault that failures occur in production
3. 3 Build Software to Test Software exactpro.com Exactpro The company was founded on Tester Day, September 9, 2009 The company’s headquarters are in London Moscow Kostroma Saratov Obninsk Tbilisi London New York
4. 4 Build Software to Test Software exactpro.com What We Do Our company specializes in development and implementation of high load market infrastructures as well as systems that work with financial tools/platforms. Our solutions are used in more than 20 countries all over the world.
5. 5 Build Software to Test Software exactpro.com Sources of information - Absence-of-errors fallacy and other principles of software testing. - Types and levels of software testing. - Understanding of test objectives and methodologies in different schools of thought in the software testing domain.
6. 6 Build Software to Test Software exactpro.com Sources of Information https://www.developsense.com/blog/2010/05/testers-get-out-of-the-quality-assurance-business/
7. 7 Build Software to Test Software exactpro.com Sources of Information https://youtu.be/jZitXMQaXvE Watch Yegor Bugayenko
8. 8 Build Software to Test Software exactpro.com Sources of Information https://ingophilipp.medium.com/excellent-software-testing-6e0fd8f3e11e
9. 9 Build Software to Test Software exactpro.com Sources of Information http://cartoontester.blogspot.com/ Andy Glover
10. 10 Build Software to Test Software exactpro.com Software Testing: Definition • Testing is about extracting information about software quality • Testing is pain • Testing is relentless learning • Testing is about processes, platforms, people
11. 11 Build Software to Test Software exactpro.com West World TEST WORLD: REBOOT
12. 12 Build Software to Test Software exactpro.com Ray Dalio PAIN + REFLECTION = PROGRESS
13. 13 Build Software to Test Software exactpro.com Ray Dalio
14. 14 Build Software to Test Software exactpro.com Ray Dalio
15. 15 Build Software to Test Software exactpro.com How We Get Information
16. 16 Build Software to Test Software exactpro.com How We Get Information
17. 17 Build Software to Test Software exactpro.com https://blogs.scientificamerican.com/observations/the-line-between-science-and-journali sm-is-getting-blurry-again/ • Journalism is communication of 'what's new'. A journalist is anyone who can say "I’m there, you’re not, let me tell you about it." • Science is communication of 'how the world works'. A scientist is anyone who can say "I understand something about the world, you don't, let me explain it to you". • Neither definition necessitates that what they say is True, just what they know to the best of their ability and understanding.
18. 18 Build Software to Test Software exactpro.com How We Get Information
19. 19 Build Software to Test Software exactpro.com How We Get Information
20. 20 Build Software to Test Software exactpro.com Quality Quality is value to some people Quality is the degree to which a component or system satisfies the stated and implied needs of its various stakeholders. Quality is inherently subjective. Different stakeholders will perceive the same product as having different levels of quality.
21. 21 Build Software to Test Software exactpro.com What is a Bug? Bug is a discrepancy between expectations and reality An attribute of a software product that reduces its value to a stakeholder or increases its value to a competitor. May or may not be a coding error or a functional error Design errors are bugs too.
22. 22 Build Software to Test Software exactpro.com What is a Bug? Error – a human action that produces an incorrect result Defect (bug) – an imperfection or deficiency in a work product where it does not meet its requirements or specifications Failure – an event in which a component or system does not perform a required function within specified limits
23. 23 Build Software to Test Software exactpro.com What is a Bug? 65,586 m/ h (~65.6 km/h) error defect/bug failure
24. 24 Build Software to Test Software exactpro.com What is a Bug? 65,586 m/ h (~65.6 km/h) error defect/bug failure
25. 25 Build Software to Test Software exactpro.com What is a Bug? Breaking path
26. 26 Build Software to Test Software exactpro.com What is a Bug? Breaking path https://cordis.europa.eu/arti cle/id/19509-ariane-5-explo sion-caused-by-fault-in-mai n-engine-cooling-system
27. 27 Build Software to Test Software exactpro.com https://www.snopes.com/fact-check/cone-of-silence PASS FAIL PASS + = = = + + What is a Bug?
28. 28 Build Software to Test Software exactpro.com What is Software Testing? Software Testing is an • empirical • technical • investigation • conducted to provide stakeholders • with information • about the quality • of the product or service under test Software Testing is exploring software with the intent of finding bugs
29. 29 Build Software to Test Software exactpro.com Watch “Software is Eating the World.”
30. 30 Build Software to Test Software exactpro.com “Software is Eating the World.” http://www3.weforum.org/docs/WEF_ The_Global_Risks_Report_2021.pdf
31. 31 Build Software to Test Software exactpro.com Software Testing is not Quality Assurance Testers: Get Out of the Quality Assurance Business Quality Assurance Quality Control Testing Code review Unit testing Audit Process Improvements Change management Early reviews and testing
32. 32 Build Software to Test Software exactpro.com Software Testing is an Information Service
33. 33 Build Software to Test Software exactpro.com Testing is not Quality Assurance Testers: Get Out of the Quality Assurance Business https://www.developsense.com/presentations/2010-06-TestersGetO utOfTheQABusiness.pdf
34. 34 Build Software to Test Software exactpro.com Unit Testing Development of unit tests allows you to isolate individual parts of the program and test their performance - creating tests, executing them and obtaining reports using open source frameworks Pytest and PyUnit.
35. 35 Build Software to Test Software exactpro.com BDD Studying the Behavior Driven Development (BDD) software development methodology Using the Gherkin in the requirements definition process, use cases, and acceptance testing
36. 36 Build Software to Test Software exactpro.com Software Testing Not Software Testing
37. 37 Build Software to Test Software exactpro.com Examples of Obtaining Information
38. 38 Build Software to Test Software exactpro.com Examples of Obtaining Information
39. 39 Build Software to Test Software exactpro.com Examples of obtaining and disseminating information
40. 40 Build Software to Test Software exactpro.com Bad Testing https://youtu.be/2S--Ge3c-yA
41. 41 Build Software to Test Software exactpro.com Software Testing is not... https://mrslavchev.com/2016/03/28/software-testing-not/
42. 42 Build Software to Test Software exactpro.com
43. 43 Build Software to Test Software exactpro.com Testing Principles Principle 1: Testing shows the presence of defects, not their absence Principle 2: Exhaustive testing is impossible Principle 3: Early testing saves time and money Principle 4: Defects cluster together Principle 5: Beware of the pesticide paradox Principle 6: Testing is context dependent Principle 7: Absence-of-errors is a fallacy
44. 44 Build Software to Test Software exactpro.com Testing shows the presence of defects, not their absence
45. 45 Build Software to Test Software exactpro.com Exhaustive testing is impossible
46. 46 Build Software to Test Software exactpro.com Autonomous Vehicle Testing
47. 47 Build Software to Test Software exactpro.com LGSVL Road Simulator - Introduction to the multi-agent road simulator developed by LG's Research and Development division based on the cross-platform environment for creating video games - Unity. - Overview of installation, launch, and user interface.
48. 48 Build Software to Test Software exactpro.com Pareto Principle
49. 49 Build Software to Test Software exactpro.com Beware of the Pesticide Paradox
50. 50 Build Software to Test Software exactpro.com Beware of the Pesticide Paradox
51. 51 Build Software to Test Software exactpro.com Context Driven Testing Consider: • Software development lifecycle model and project methodologies being used • Test levels and test types being considered • Product and project risks • Business domain • Operational constraints, including but not limited to: a. Budgets and resources b. Timescales c. Complexity d. Contractual and regulatory requirements • Organizational policies and practices • Required internal and external standards
52. 52 Build Software to Test Software exactpro.com Context Driven Testing The Seven Basic Principles of the Context-Driven School 1. The value of any practice depends on its context. 2. There are good practices in context, but there are no best practices. 3. People, working together, are the most important part of any project’s context. 4. Projects unfold over time in ways that are often not predictable. 5. The product is a solution. If the problem isn’t solved, the product doesn’t work. 6. Good software testing is a challenging intellectual process. 7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products. https://context-driven-testing.com/
53. 53 Build Software to Test Software exactpro.com
54. 54 Build Software to Test Software exactpro.com Absence-of-errors is a fallacy
55. 55 Build Software to Test Software exactpro.com Verification vs. Validation Verification – checks whether the system meets the specified requirements Validation – checks whether the system will meet user and other stakeholder needs • https://techcrunch.com/2018/06/22/uber-safety-driver-of-fatal-self-driving-crash-was-watchin g-hulu-not-the-road • https://www.ntsb.gov/investigations/AccidentReports/Reports/HWY18MH010-prelim.pdf
56. 56 Build Software to Test Software exactpro.com https://www.ntsb.gov/investigations/AccidentReports/Reports/HWY18MH010-prelim.pdf
57. 57 Build Software to Test Software exactpro.com https://www.ntsb.gov/investigations/AccidentReports/Reports/HWY18MH010-prelim.pdf
58. 58 Build Software to Test Software exactpro.com Types of Software Testing • Functional testing — WHAT • Non-functional testing — HOW • Testing with access to the source code — White Box
59. 59 Build Software to Test Software exactpro.com Software Testing at the confluence Functional Testing Nonfunctional Testing
60. 60 Build Software to Test Software exactpro.com Black Box Testing vs. White Box Testing
61. 61 Build Software to Test Software exactpro.com Testing Components
62. 62 Build Software to Test Software exactpro.com • Test planning – involves defining the objectives of testing and the approach for meeting those objectives within project constraints and contexts • Test monitoring and control – compares actual progress against the plan, takes actions which are necessary to meet the mission and objectives of the project, and/or adjust the plan • Test analysis – determines ‘what to test’ • Test design – addresses the question ‘how to test’ • Test implementation – asks ‘do we now have everything in place to run the tests? • Test execution – runs tests • Test completion – summarizes the results of testing Testing Components
63. 63 Build Software to Test Software exactpro.com Test Objectives 1. To prevent defects by evaluating work products such as requirements, user stories, design, and code 2. To verify whether all specified requirements have been fulfilled 3. To check whether the test object is complete and validate if it works as the users and other stakeholders expect 4. To build confidence in the level of quality of the test object 5. To find defects and failures thus reduce the level of risk of inadequate software quality 6. To provide sufficient information to stakeholders to allow them to make informed decisions, especially regarding the level of quality of the test object 7. To comply with contractual, legal, or regulatory requirements or standards, and/or to verify the test object’s compliance with such requirements or standards
64. 64 Build Software to Test Software exactpro.com
65. 65 Build Software to Test Software exactpro.com Bad Testing Doctor Fauci
66. 66 Build Software to Test Software exactpro.com Testers vs. Activists
67. 67 Build Software to Test Software exactpro.com Test Objectives • To prevent defects by evaluate work products such as requirements, user stories, design, and code • To verify whether all specified requirements have been fulfilled • To check whether the test object is complete and validate if it works as the users and other stakeholders expect • To build confidence in the level of quality of the test object • To find defects and failures thus reduce the level of risk of inadequate software quality • To provide sufficient information to stakeholders to allow them to make informed decisions, especially regarding the level of quality of the test object • To comply with contractual, legal, or regulatory requirements or standards, and/or to verify the test object’s compliance with such requirements or standards
68. 68 Build Software to Test Software exactpro.com 2021 IEEE Autonomous Driving AI Test Challenge http://av-test-challenge.org/
69. 69 Build Software to Test Software exactpro.com Good Testing
70. 70 Build Software to Test Software exactpro.com Test Objectives
71. 71 Build Software to Test Software exactpro.com Testing is not Quality Assurance Testers: Get Out of the Quality Assurance Business Quality Assurance Quality Control Testing Code review Unit testing Audit Process Improvements Change management Early reviews and testing
72. 72 Build Software to Test Software exactpro.com Good Testing
73. 73 Build Software to Test Software exactpro.com Good Testing
74. 74 Build Software to Test Software exactpro.com Good Testing • Ownership – taking care of your company. Treat others the way you would like to be treated. Take responsibility for company's needs. Remember: responsibility is power. It is not granted. It is taken. Extreme ownership is the way to lead and win. • Deep Impact – exitus acta probat. The outcome justifies the effort. Make sure that your work results in a valuable contribution. If you want to go fast, go alone. If you want to go far, go together. Involve others to achieve scalability and increase the impact of our work. • Hard Work – no pain, no gain. Sometimes putting in long hours does not help achieve any result. We do not measure time spent in the office. Having said that, we've never observed extraordinary results without hard work and deliberate effort.
75. 75 Build Software to Test Software exactpro.com • Learning – software testing is relentless learning. Change is the end result of all true learning. The beautiful thing about learning is that nobody can take it away from you. The Mistakes and problems are inevitable. However, it is unacceptable not to learn from them. The best way to learn something is to teach it. • Agility – the only constant is change. We embrace the values reflected in the Manifesto for Agile Software Development. Agility allows us to turn learning into action and action into change. From its inception, Exactpro has been able to adapt to change. It is important to keep this ability as we scale up our business. • Communication – the medium is the message. A relationship is based on communication. We need to make sure others understand us and we understand them. • Objectivity – you have nothing to fear from knowing the truth. It is at the core of our business. Our main service is to build software to test software and provide objective information about it to our clients. Our business benefits from openness. Everyone in our company has the right to understand what makes sense, and no one has the right to hold a critical opinion without speaking up. Good Testing
76. 76 Build Software to Test Software exactpro.com Thank you Exactpro Systems Exactpro Vlog