Lecture 8: Software negligence and testing coverage

"Software Testing for Complex Intelligent Systems and Autonomous Vehicles" online course.

The lecture is available via the link: https://youtu.be/MoYDDHMfHbk
--
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. 1 Build Software to Test Software exactpro.com Lecture 8 Software negligence and test coverage Software Testing for Complex Intelligent Systems and Autonomous Vehicles Online Course
2. 2 Build Software to Test Software exactpro.com Lecture Structure The concepts of professional malpractice and criminal negligence The issue of liability for software malfunction. The principle that exhaustive testing is impossible. Test coverage. Types of test coverage. The relation between coverage and liability for software malfunction
3. 3 Build Software to Test Software exactpro.com Reading Materials Cem Kaner Software Negligence and Testing Coverage SOFTWARE NEGLIGENCE AND TESTING COVERAGE Cem Kaner, kaner@kaner.com Copyright, 1995, all rights reserved. Portions of this paper were originally published in C. Kaner, “Software Negligence & Testing Coverage”, Software QA Quarterly, Vol. 2, #2, p. 18, 1995. 1 1
4. 4 Build Software to Test Software exactpro.com Reading Materials Iosif Itkin for FOW Assuring compliance with pending AI liability regulation
5. 5 Build Software to Test Software exactpro.com Reading Materials ISTQB Foundation Level Syllabus 2018
6. 6 Build Software to Test Software exactpro.com Cem Kaner: Case Study Cem Kaner Software Negligence and Testing Coverage
7. 7 Build Software to Test Software exactpro.com Negligence in Relation to Software MISREPRESENTATION
8. 8 Build Software to Test Software exactpro.com The Question of Responsibility for Test Results VS.
9. 9 Build Software to Test Software exactpro.com The Subject The Field of IT The Field of Law
10. 10 Build Software to Test Software exactpro.com Negligence/Malpractice Intent Adequate Measures Intent Adequate Measures Professionalism VS.
11. 11 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent Adequate Measures Intent Adequate Measures Professionalism VS.
12. 12 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent Adequate Measures Intent Adequate Measures Professionalism VS. Standard?
13. 13 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner)
14. 14 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent Adequate Measures Intent Adequate Measures Professionalism VS. ??? ???
15. 15 Build Software to Test Software exactpro.com Reasonable measures Cem Kaner Software Negligence and Testing Coverage Costs to prevent Cost of the error
16. 16 Build Software to Test Software exactpro.com Reasonable measures Cem Kaner Software Negligence and Testing Coverage Cost of the error Costs to prevent + the cost of finding the bug “What would it have cost to make the testing process thorough enough that you would have found this bug during testing?”
17. 17 Build Software to Test Software exactpro.com Test Coverage
18. 18 Build Software to Test Software exactpro.com Line coverage Line coverage measures the number / percentage of lines of code that have been executed Cem Kaner Software Negligence and Testing Coverage
19. 19 Build Software to Test Software exactpro.com Black box testing vs. white box testing ISTQB Foundation Level Syllabus 2018
20. 20 Build Software to Test Software exactpro.com Cem Kaner: 101 coverage metrics “This appendix lists 101 coverage measures. Coverage measures the amount of testing done of a certain type. Because testing is done to find bugs, coverage is also a measure of your effort to detect a certain class of potential errors. For example, 100% line coverage doesn’t just mean that you’ve executed every line of code; it also means that you’ve tested for every bug that can be revealed by simple execution of a line of code.” Cem Kaner Software Negligence and Testing Coverage
21. 21 Build Software to Test Software exactpro.com Test Coverage: All Message Fields 8=FIXT1.1 | 9=90 | 35=D | 49=User1 | 56=FGW | 34=2 | 55=ACME | 54=1 | 40=2 | 38=40 | 59=0 | 44=100 | 11=ORD1 | 10=197 | Test Coverage in FinTech: Protocol Logs and Discrete Values
22. 22 Build Software to Test Software exactpro.com Test Coverage: All System Events New Ack Working Fill Done Part Fill Canceled Expired Rejected Events: Requests: New/Amend/Cancel Market: Sessions/Trades Time: Triggers/Expiration ISTQB Foundation Level Certification Explained Chapter 4 (Part 1)
23. 23 Build Software to Test Software exactpro.com (Eng) Mind the Gap: Applying Process Mining https://focus.world-exchanges.org/articles/mind-gap-between-testing-and- production-applying-process-mining-test-resilience-exchange-platforms New (100) New (279) New (4784) END NEW Rejected Filled Expired Cancelled START New (38) Reject (3222) (3222) (100) (38) (279) (4784) Cancel (100) Cancel (38) Expired (279) Trade (4784) Trade (38) TradeCancel (38) Test Coverage: All Transitions between System States
24. 24 Build Software to Test Software exactpro.com Test Coverage: Requirement Coverage Test Coverage in FinTech: Protocol Logs and Discrete Values
25. 25 Build Software to Test Software exactpro.com Coverage Matrix: the Illusion of Exhaustive Test Coverage Test Coverage in FinTech: Protocol Logs and Discrete Values
26. 26 Build Software to Test Software exactpro.com Test Coverage: the Importance of Specific Values Fake Testing: "The Pentagon Wars" Film Review
27. 27 Build Software to Test Software exactpro.com AV AI Test Challenge: Simulation Coverage http://av-test-challenge.org/
28. 28 Build Software to Test Software exactpro.com https://www.svlsimulator.com/docs/python-api/python-api/#weather- and-time-of-day-control Test Coverage: Weather Weather: rain, fog, wetness, cloudiness
29. 29 Build Software to Test Software exactpro.com LGSVL - Vehicle Dynamics Test Coverage: Car Dynamics Parameters (the EGO vehicle)
30. 30 Build Software to Test Software exactpro.com https://www.svlsimulator.com/docs/python-api/python-api/#agents Test Coverage: Other Vehicles and Pedestrians NPC vehicles: Sedan, SUV, Jeep, Hatchback, SchoolBus, BoxTruck Pedestrians: Bob, EntrepreneurFemale, Howard, Johny, Pamela, Presley, Red, Robin, Stephen, Zoe
31. 31 Build Software to Test Software exactpro.com Coverage and Liability
32. 32 Build Software to Test Software exactpro.com Exhaustive Testing is Impossible
33. 33 Build Software to Test Software exactpro.com Other Offenses in Software Development </code> VS.
34. 34 Build Software to Test Software exactpro.com Other Offenses in Software Development VS. MISREPRESENTATION
35. 35 Build Software to Test Software exactpro.com Software Responsibility in Artificial Intelligence Systems European Commission - Liability for Artificial Intelligence (2019) FCA and Bank of England - Machine learning in UK financial services (2019) US DoD - AI Principles: Recommendations on the Ethical Use of Artificial Intelligence (2019) US Whitehouse - Guidance for Regulation of Artificial Intelligence Applications (2019) FSB - Artificial intelligence and machine learning in financial services (2017) IOSCO - The use of artificial intelligence and machine learning by market intermediaries and asset managers (2020)
36. 36 Build Software to Test Software exactpro.com Product vs Service Product Service
37. 37 Build Software to Test Software exactpro.com AI Service - There is a duty to take adequate measures to ensure quality - There is a breach of duty - Violation is the cause of the damage service
38. 38 Build Software to Test Software exactpro.com AI Product - Product liability - Strict liability Product Intent Adequate Measures
39. 39 Build Software to Test Software exactpro.com Liability for AI Malfunction - Negligence - Product liability - Strict liability - Misrepresentation https://spectrum.ieee.org/cars-that-think/transportation/self-driving/why- you-shouldnt-worry-about-liability-for-selfdriving-car-accidents
40. 40 Build Software to Test Software exactpro.com Presenting Facts Volvo XC90 Drive Me Autonomous (self-driving) car interface (IntelliSafe Autopilot)
41. 41 Build Software to Test Software exactpro.com Presenting Facts Elon Musk on Twitter - 17 April Tesla (TSLA) Texas Crash: Two Men Dead After Accident With 'No One' Driving - Bloomberg, 18 April
42. 42 Build Software to Test Software exactpro.com Thank you! Join our Telegram chat, meet your peers, ask questions and get new software testing skills with Exactpro! Subscribe to our YouTube channel to see all the new videos!
43. 43 Build Software to Test Software exactpro.com Further Reading Cem Kaner - Legal Issues Related to Software Quality (1997) John Villasenor - Products Liability and Driverless Cars: Issues and Guiding Principles for Legislation (2014) Karen Mercedes Goertzel - Legal Liability for Bad Software (2016) John Kingston - Artificial Intelligence and Legal Liability (2018) Miriam Buiten - Towards Intelligent Regulation of Artificial Intelligence (2019) Darrell M. West - It is time to restore the US Office of Technology Assessment (2021)