Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lecture 8: Software negligence and testing coverage

Lecture 8: Software negligence and testing coverage

"Software Testing for Complex Intelligent Systems and Autonomous Vehicles" online course.

The lecture is available via the link: https://youtu.be/MoYDDHMfHbk
--
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

April 27, 2021
Tweet

Transcript

  1. 1 Build Software to Test Software exactpro.com Lecture 8 Software

    negligence and test coverage Software Testing for Complex Intelligent Systems and Autonomous Vehicles Online Course
  2. 2 Build Software to Test Software exactpro.com Lecture Structure The

    concepts of professional malpractice and criminal negligence The issue of liability for software malfunction. The principle that exhaustive testing is impossible. Test coverage. Types of test coverage. The relation between coverage and liability for software malfunction
  3. 3 Build Software to Test Software exactpro.com Reading Materials Cem

    Kaner Software Negligence and Testing Coverage SOFTWARE NEGLIGENCE AND TESTING COVERAGE Cem Kaner, kaner@kaner.com Copyright, 1995, all rights reserved. Portions of this paper were originally published in C. Kaner, “Software Negligence & Testing Coverage”, Software QA Quarterly, Vol. 2, #2, p. 18, 1995. 1 1
  4. 4 Build Software to Test Software exactpro.com Reading Materials Iosif

    Itkin for FOW Assuring compliance with pending AI liability regulation
  5. 5 Build Software to Test Software exactpro.com Reading Materials ISTQB

    Foundation Level Syllabus 2018
  6. 6 Build Software to Test Software exactpro.com Cem Kaner: Case

    Study Cem Kaner Software Negligence and Testing Coverage
  7. 7 Build Software to Test Software exactpro.com Negligence in Relation

    to Software MISREPRESENTATION
  8. 8 Build Software to Test Software exactpro.com The Question of

    Responsibility for Test Results VS.
  9. 9 Build Software to Test Software exactpro.com The Subject The

    Field of IT The Field of Law
  10. 10 Build Software to Test Software exactpro.com Negligence/Malpractice Intent Adequate

    Measures Intent Adequate Measures Professionalism VS.
  11. 11 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent

    Adequate Measures Intent Adequate Measures Professionalism VS.
  12. 12 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent

    Adequate Measures Intent Adequate Measures Professionalism VS. Standard?
  13. 13 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner)

  14. 14 Build Software to Test Software exactpro.com Negligence/Malpractice (Kaner) Intent

    Adequate Measures Intent Adequate Measures Professionalism VS. ??? ???
  15. 15 Build Software to Test Software exactpro.com Reasonable measures Cem

    Kaner Software Negligence and Testing Coverage Costs to prevent Cost of the error
  16. 16 Build Software to Test Software exactpro.com Reasonable measures Cem

    Kaner Software Negligence and Testing Coverage Cost of the error Costs to prevent + the cost of finding the bug “What would it have cost to make the testing process thorough enough that you would have found this bug during testing?”
  17. 17 Build Software to Test Software exactpro.com Test Coverage

  18. 18 Build Software to Test Software exactpro.com Line coverage Line

    coverage measures the number / percentage of lines of code that have been executed Cem Kaner Software Negligence and Testing Coverage
  19. 19 Build Software to Test Software exactpro.com Black box testing

    vs. white box testing ISTQB Foundation Level Syllabus 2018
  20. 20 Build Software to Test Software exactpro.com Cem Kaner: 101

    coverage metrics “This appendix lists 101 coverage measures. Coverage measures the amount of testing done of a certain type. Because testing is done to find bugs, coverage is also a measure of your effort to detect a certain class of potential errors. For example, 100% line coverage doesn’t just mean that you’ve executed every line of code; it also means that you’ve tested for every bug that can be revealed by simple execution of a line of code.” Cem Kaner Software Negligence and Testing Coverage
  21. 21 Build Software to Test Software exactpro.com Test Coverage: All

    Message Fields 8=FIXT1.1 | 9=90 | 35=D | 49=User1 | 56=FGW | 34=2 | 55=ACME | 54=1 | 40=2 | 38=40 | 59=0 | 44=100 | 11=ORD1 | 10=197 | Test Coverage in FinTech: Protocol Logs and Discrete Values
  22. 22 Build Software to Test Software exactpro.com Test Coverage: All

    System Events New Ack Working Fill Done Part Fill Canceled Expired Rejected Events: Requests: New/Amend/Cancel Market: Sessions/Trades Time: Triggers/Expiration ISTQB Foundation Level Certification Explained Chapter 4 (Part 1)
  23. 23 Build Software to Test Software exactpro.com (Eng) Mind the

    Gap: Applying Process Mining https://focus.world-exchanges.org/articles/mind-gap-between-testing-and- production-applying-process-mining-test-resilience-exchange-platforms New (100) New (279) New (4784) END NEW Rejected Filled Expired Cancelled START New (38) Reject (3222) (3222) (100) (38) (279) (4784) Cancel (100) Cancel (38) Expired (279) Trade (4784) Trade (38) TradeCancel (38) Test Coverage: All Transitions between System States
  24. 24 Build Software to Test Software exactpro.com Test Coverage: Requirement

    Coverage Test Coverage in FinTech: Protocol Logs and Discrete Values
  25. 25 Build Software to Test Software exactpro.com Coverage Matrix: the

    Illusion of Exhaustive Test Coverage Test Coverage in FinTech: Protocol Logs and Discrete Values
  26. 26 Build Software to Test Software exactpro.com Test Coverage: the

    Importance of Specific Values Fake Testing: "The Pentagon Wars" Film Review
  27. 27 Build Software to Test Software exactpro.com AV AI Test

    Challenge: Simulation Coverage http://av-test-challenge.org/
  28. 28 Build Software to Test Software exactpro.com https://www.svlsimulator.com/docs/python-api/python-api/#weather- and-time-of-day-control Test

    Coverage: Weather Weather: rain, fog, wetness, cloudiness
  29. 29 Build Software to Test Software exactpro.com LGSVL - Vehicle

    Dynamics Test Coverage: Car Dynamics Parameters (the EGO vehicle)
  30. 30 Build Software to Test Software exactpro.com https://www.svlsimulator.com/docs/python-api/python-api/#agents Test Coverage:

    Other Vehicles and Pedestrians NPC vehicles: Sedan, SUV, Jeep, Hatchback, SchoolBus, BoxTruck Pedestrians: Bob, EntrepreneurFemale, Howard, Johny, Pamela, Presley, Red, Robin, Stephen, Zoe
  31. 31 Build Software to Test Software exactpro.com Coverage and Liability

  32. 32 Build Software to Test Software exactpro.com Exhaustive Testing is

    Impossible
  33. 33 Build Software to Test Software exactpro.com Other Offenses in

    Software Development </code> VS.
  34. 34 Build Software to Test Software exactpro.com Other Offenses in

    Software Development VS. MISREPRESENTATION
  35. 35 Build Software to Test Software exactpro.com Software Responsibility in

    Artificial Intelligence Systems European Commission - Liability for Artificial Intelligence (2019) FCA and Bank of England - Machine learning in UK financial services (2019) US DoD - AI Principles: Recommendations on the Ethical Use of Artificial Intelligence (2019) US Whitehouse - Guidance for Regulation of Artificial Intelligence Applications (2019) FSB - Artificial intelligence and machine learning in financial services (2017) IOSCO - The use of artificial intelligence and machine learning by market intermediaries and asset managers (2020)
  36. 36 Build Software to Test Software exactpro.com Product vs Service

    Product Service
  37. 37 Build Software to Test Software exactpro.com AI Service -

    There is a duty to take adequate measures to ensure quality - There is a breach of duty - Violation is the cause of the damage service
  38. 38 Build Software to Test Software exactpro.com AI Product -

    Product liability - Strict liability Product Intent Adequate Measures
  39. 39 Build Software to Test Software exactpro.com Liability for AI

    Malfunction - Negligence - Product liability - Strict liability - Misrepresentation https://spectrum.ieee.org/cars-that-think/transportation/self-driving/why- you-shouldnt-worry-about-liability-for-selfdriving-car-accidents
  40. 40 Build Software to Test Software exactpro.com Presenting Facts Volvo

    XC90 Drive Me Autonomous (self-driving) car interface (IntelliSafe Autopilot)
  41. 41 Build Software to Test Software exactpro.com Presenting Facts Elon

    Musk on Twitter - 17 April Tesla (TSLA) Texas Crash: Two Men Dead After Accident With 'No One' Driving - Bloomberg, 18 April
  42. 42 Build Software to Test Software exactpro.com Thank you! Join

    our Telegram chat, meet your peers, ask questions and get new software testing skills with Exactpro! Subscribe to our YouTube channel to see all the new videos!
  43. 43 Build Software to Test Software exactpro.com Further Reading Cem

    Kaner - Legal Issues Related to Software Quality (1997) John Villasenor - Products Liability and Driverless Cars: Issues and Guiding Principles for Legislation (2014) Karen Mercedes Goertzel - Legal Liability for Bad Software (2016) John Kingston - Artificial Intelligence and Legal Liability (2018) Miriam Buiten - Towards Intelligent Regulation of Artificial Intelligence (2019) Darrell M. West - It is time to restore the US Office of Technology Assessment (2021)