Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A4Q World Congress: 2021 IEEE Autonomous Driving AI Test Challenge

5206c19df417b8876825b5561344c1a0?s=47 Exactpro
PRO
April 15, 2021
26

A4Q World Congress: 2021 IEEE Autonomous Driving AI Test Challenge

Elena Treshcheva - IEEE AV AI Test Challenge Organizing Committee,
Researcher, Exactpro

2021 IEEE Autonomous Driving AI Test Challenge
15.04.2021
--
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

April 15, 2021
Tweet

Transcript

  1. 2021 IEEE Autonomous Driving AI Test Challenge Elena Treshcheva IEEE

    AV AI Test Challenge Organizing Committee, Researcher, Exactpro
  2. http://av-test-challenge.org/ 2021 IEEE Autonomous Driving AI Test Challenge

  3. - WHY both industry and academia need initiatives at the

    confluence of AI and software testing - WHAT the 2021 IEEE Autonomous Driving AI Test Challenge is all about - HOW one can explore the knowledge domain of testing complex intelligent systems Overview
  4. WHY: AI and Software Testing as a new field of

    studies Today’s context: - rapid advancements in AI-based technologies, active development of intelligent systems - research on trustworthiness, safety and reliability of AI - professional certifications and educational programs Indicators of the new area being established
  5. WHY: AI and Software Testing as a new field of

    studies
  6. AI systems as specific type of the system under test:

    - Non-deterministic - Non-testable with traditional methods AI to support software testing: - Oracle problem: telling right from wrong cannot be delegated to the algorithms - Limited applicability - not to testing per se, but rather to surrounding tasks A4Q AI and Software Testing Foundation Syllabus WHY: AI and Software Testing as a new field of studies
  7. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  8. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  9. WHY: AI and Software Testing as a new field of

    studies Quality characteristics of AI-based systems: - Ability to learn: the capacity of the system to learn from use for the system itself, or data and events it is exposed to - Ability to generalize: the ability of the system to apply to different and previously unseen scenarios - Trustworthiness: the degree to which the system is trusted by stakeholders A4Q AI and Software Testing Foundation Syllabus
  10. http://av-test-challenge.org/ WHAT: IEEE Autonomous Driving AI Test Challenge

  11. http://ieeeaitests.com/ The Third IEEE International Conference on Artificial Intelligence Testing

  12. http://www.ieeeaitests.com/html/callForPapers.html AI Test 2021 - Call for Papers: - Regular

    (8 pages) and Short papers (2 pages) - AI Testing In Practice (8 pages) - Tool Demo Track (4 pages) The Third IEEE International Conference on Artificial Intelligence Testing
  13. http://www.ieeeaitests.com/html/callForPapers.html AI Test 2021 - Topics of Interest: - Testing

    AI applications - Applications of AI techniques to software testing - Data quality checking for AI applications The Third IEEE International Conference on Artificial Intelligence Testing
  14. http://av-test-challenge.org/ WHAT: IEEE Autonomous Driving AI Test Challenge

  15. Autonomous Driving: Testing by simulation

  16. Autonomous Driving: Testing by simulation

  17. Autonomous Driving: Testing by simulation

  18. Autonomous Driving: Testing by simulation

  19. Autonomous Driving: Testing by simulation

  20. Autonomous Driving: Testing by simulation

  21. Autonomous Driving: Testing by simulation

  22. Autonomous Driving: Testing by simulation

  23. https://arxiv.org/pdf/2005.03778.pdf Rong et al. (2020) - LGSVL Simulator: A high

    fidelity simulator for autonomous driving
  24. https://arxiv.org/pdf/2005.03778.pdf Rong et al. (2020) - LGSVL Simulator: A high

    fidelity simulator for autonomous driving
  25. https://arxiv.org/pdf/2005.03778.pdf Rong et al. (2020) - LGSVL Simulator: A high

    fidelity simulator for autonomous driving
  26. Baidu Apollo Simulator

  27. Tutorial: End-to-end video tutorial of LGSVL simulator with Apollo driving

    Recordings and slides from the AV AI Test Challenge training sessions HOW: More on Software testing and AI
  28. Software testing for complex intelligent systems and autonomous vehicles -

    Playlist on the Exactpro YouTube channel Software Testing for Complex Intelligent Systems and Autonomous Vehicles Online Course HOW: More on Software testing and AI
  29. Online course from Exactpro Software Testing for Complex Intelligent Systems

    and Autonomous Vehicles Online Course - an introduction to AI-based software, risks associated with it and the necessity of its comprehensive testing - general knowledge of the software testing basics - an ability to use testing methods on real-life tasks - hands-on experience of using test tools and configuring a test environment - the skill of writing test cases, including automated test cases, and executing them - an ability to analyse and interpret test run results
  30. 1. Autonomous driving and software testing 2. Introduction to self-driving

    cars 3. Application programming interface 4. Foundations of software testing 5. Build Software to Test Software 6. Reliability and safety 7. Software defects and metrics 8. Software negligence and testing coverage Online course from Exactpro Theoretical module Online course on the Exactpro YouTube channel
  31. 1. Launching the LGSVL Simulator 2. Setting up a Python

    IDE 3. Unit Testing 4. Behavior Driven Development 5. Data and Keyword Driven Testing 6. Data Analysis and Reporting 7. Running Apollo 8. Modeling Test Scenarios Online course from Exactpro Practical module Online course on the Exactpro YouTube channel
  32. Takeaways - WHY: we need initiatives at the confluence of

    AI and software testing to work out new software testing techniques addressing sophisticated AI systems - WHAT: IEEE Autonomous Driving AI Test Challenge and AI Test Conference serve as a forum for discussion and development of new approaches to software testing - HOW: We definitely need to prepare software testers to work with intelligent systems. There are resources providing knowledge and training in the intersection of AI and Software Testing domains
  33. None