Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A4Q World Congress: 2021 IEEE Autonomous Driving AI Test Challenge

Exactpro
April 15, 2021
67

A4Q World Congress: 2021 IEEE Autonomous Driving AI Test Challenge

Elena Treshcheva - IEEE AV AI Test Challenge Organizing Committee,
Researcher, Exactpro

2021 IEEE Autonomous Driving AI Test Challenge
15.04.2021
--
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Exactpro

April 15, 2021
Tweet

Transcript

  1. 2021 IEEE Autonomous Driving AI Test Challenge Elena Treshcheva IEEE

    AV AI Test Challenge Organizing Committee, Researcher, Exactpro
  2. - WHY both industry and academia need initiatives at the

    confluence of AI and software testing - WHAT the 2021 IEEE Autonomous Driving AI Test Challenge is all about - HOW one can explore the knowledge domain of testing complex intelligent systems Overview
  3. WHY: AI and Software Testing as a new field of

    studies Today’s context: - rapid advancements in AI-based technologies, active development of intelligent systems - research on trustworthiness, safety and reliability of AI - professional certifications and educational programs Indicators of the new area being established
  4. AI systems as specific type of the system under test:

    - Non-deterministic - Non-testable with traditional methods AI to support software testing: - Oracle problem: telling right from wrong cannot be delegated to the algorithms - Limited applicability - not to testing per se, but rather to surrounding tasks A4Q AI and Software Testing Foundation Syllabus WHY: AI and Software Testing as a new field of studies
  5. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  6. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  7. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  8. http://www.ieeeaitests.com/html/callForPapers.html AI Test 2021 - Call for Papers: - Regular

    (8 pages) and Short papers (2 pages) - AI Testing In Practice (8 pages) - Tool Demo Track (4 pages) The Third IEEE International Conference on Artificial Intelligence Testing
  9. http://www.ieeeaitests.com/html/callForPapers.html AI Test 2021 - Topics of Interest: - Testing

    AI applications - Applications of AI techniques to software testing - Data quality checking for AI applications The Third IEEE International Conference on Artificial Intelligence Testing
  10. Tutorial: End-to-end video tutorial of LGSVL simulator with Apollo driving

    Recordings and slides from the AV AI Test Challenge training sessions HOW: More on Software testing and AI
  11. Software testing for complex intelligent systems and autonomous vehicles -

    Playlist on the Exactpro YouTube channel Software Testing for Complex Intelligent Systems and Autonomous Vehicles Online Course HOW: More on Software testing and AI
  12. Online course from Exactpro Software Testing for Complex Intelligent Systems

    and Autonomous Vehicles Online Course - an introduction to AI-based software, risks associated with it and the necessity of its comprehensive testing - general knowledge of the software testing basics - an ability to use testing methods on real-life tasks - hands-on experience of using test tools and configuring a test environment - the skill of writing test cases, including automated test cases, and executing them - an ability to analyse and interpret test run results
  13. 1. Autonomous driving and software testing 2. Introduction to self-driving

    cars 3. Application programming interface 4. Foundations of software testing 5. Build Software to Test Software 6. Reliability and safety 7. Software defects and metrics 8. Software negligence and testing coverage Online course from Exactpro Theoretical module Online course on the Exactpro YouTube channel
  14. 1. Launching the LGSVL Simulator 2. Setting up a Python

    IDE 3. Unit Testing 4. Behavior Driven Development 5. Data and Keyword Driven Testing 6. Data Analysis and Reporting 7. Running Apollo 8. Modeling Test Scenarios Online course from Exactpro Practical module Online course on the Exactpro YouTube channel
  15. Takeaways - WHY: we need initiatives at the confluence of

    AI and software testing to work out new software testing techniques addressing sophisticated AI systems - WHAT: IEEE Autonomous Driving AI Test Challenge and AI Test Conference serve as a forum for discussion and development of new approaches to software testing - HOW: We definitely need to prepare software testers to work with intelligent systems. There are resources providing knowledge and training in the intersection of AI and Software Testing domains