Audi Electronics Venture - Fundamentals of Autonomous Driving | ADM#1

Why didn't self-driving cars hit the market yet? What makes it difficult to build them? And which technologies will play an important role?

Although self-driving cars were repeatedly announced in recent years, we still don't see them on streets, apart from occasional tests. While the technology behind some of the first autonomous cars seemed somewhat mysterious, it became subject of public discussion in the meanwhile. MOOC providers and universities are offering courses on the engineering of self-driving systems and a wave of startups has spawned - often just to be bought by traditional car makers soon after [1, 2, 3].

We want to bring the discussion about self-driving cars and the technology to build them to center of the engineer community and share it with everyone who is interested in the topic. This sessions is intended to give an overview of challenges involved with self-driving cars, discusses fundamental technologies. Ideally, the session will not only equip you with the basic understanding of the topic, but also ignite many further discussions.

Transcript

1. Audi Electronics Venture ADM Meetup #1 Fundamentals of Autonomous Driving
2. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 2 Autonomous driving has the opportunity to be worth multiple 1,000,000,000,000 USD. - Morgan Stanley, goo.gl/xdEdQz Opportunity Autonomous Driving for everyone and everywhere? Autonomous Cargo Transportation Autonomous Taxi Fleet Autonomous Car (Individual Ownership) STEP 1 STEP 2
3. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 3 OEMs OEMs Competition Autonomous driving extends the field of competitors Suppliers Tech Companies New BEV OEMs OEMs
4. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 4 Competition Current state of development [miles], logarithmic scale › Autonomous miles on public roads in California | 2016 Source: DMV California, goo.gl/1Kumjm 377,9 550 590 638 673,42 3125,3 4099 9776,03 635868 1 10 100 1000 10000 100000 1000000 Bosch Tesla Motors Ford BMW Mercedes-Benz Delphi Automotive Systems Nissan North Americ GM Cruise Google Auto, Waymo
5. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 5 Autonomous Driving at Audi Audi Q2 deep learning concept, NIPS2016 Audi piloted driving: 550+ miles from Silicon Valley to Las Vegas, CES 2015 ? Internal projects… A8 Piloted drive, Berlinale 2016 Upcoming Audi A8 series car will reach NHTSA Level 3 autonomy
6. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 6 Warnings + Info Acceleration OR Steering Acceleration AND Steering Full driving task Driver intervention not required All environments Level 0  Level 1   Level 2   Level 3   Level 4   Level 5    Levels of Autonomy by NHTSA / SAE Source: SAE, goo.gl/XF7NYe
7. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 7 Deployment Environments Complexity - Restricted area e.g. parking lot, company premises - Highway accidents and constructions sites are difficult - Country road very diverse traffic participants - City high traffic participant density, often chaotic traffic - World
8. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 8 Deployment Scenarios Level 0 Level 1 Level 2 Level 3 Level 4 Level 5 Transportation Taxi Fleet Engineering of private autonomous cars for worldwide deployment is very challenging Autonomous driving might appear earlier in more specific scenarios Otto Uber Google Google Waymo? Current series cars Audi A8
9. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 9 ? Car2X Lidar Simu- lation HD Maps HPC Which Technologies are needed?
10. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 10 Challenges and Solutions = Checklist for autonomous driving? Challenge Occlusion Traffic Policeman Cities Weather Accidents Traffic diversity & complexity Solution > Car2X > Flexible routing > Remote control > 360° sensors view > HPC platform in car > Redundant sensors: Camera, Lidar, Radar, (Ultrasonic) > Simulation > AI Details Single car can’t perceive full environment Some situations are very difficult to handle Chaotic environment, traffic from all directions Weather can disable certain sensors, e.g. sun blinds camera Some situations are too expensive or rare to test and train Participant variety (rickshaw), behaviour patterns Environment awareness > (HD?) Maps Single car can’t perceive the large- scale environment Domain shift > AI; life long learning Both car and environment can change substantially over time
11. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 11 Sensor Layout of current cars Front-Kamera Top-View-Kameras Front Laserscanner Ultraschallsensoren Front-Radar-Sensoren Heck-Radar-Sensoren Heck-Laserscanner Eck-Radar-Sensoren Unlike humans, autonomous cars will have permanent 360° perception
12. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 12 Perception sensor types vs. Distance Measurement (=Lidar, radar, ultrasonic) LIDAR-generated road map* Camera Vision Camera and MobilEye EyeQ2 Chip** Short–term planning collision avoidance Medium–term planning hazard avoidance Sources: *Oregon Transportation Department, **Binarysequence@Wikipedia › Distance measuring sensors are agnostic towards obstacle type; Easy interpretation but limited insight › Knowing the distance to every object in proximity allows to avoid collision with vehicles on the ego-trajectory. › Cameras offer rich information but image interpretation requires advanced methods like Deep Learning › A semantic understanding of the vehicle’s environment allows early anticipation of potential hazards.
13. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 13 Sensor Blinding sunlight, Darkness Rain, Fog, Snow Non-Metal Objects Wind/ High velocity Camera     Lidar     Radar     Ultrasonic     Sensor comparison Robust perception requires sensor redundancy. Factors like price, size, features, etc. must be considered as well. $ $$ $$$ $$ $ Resolu tion +++ +++ ++ + Range +++ ++ +++ +
14. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 15 › Considering vehicle velocity in connection with camera framerate › Considering 30-50cm as useful range  at least 60 Hz must be reached › E.g. 4 cameras  4*60=240 Hz + radar + lidar + xx (102 Hz on AlexNet @Drive PX2) 10 Hz 30 Hz 60 Hz 100 Hz 0.100 s 0.033 s 0.017 s 0.010 s 5 km/h 1.4 m/s 0.1 m 0.14 m 0.05 m 0.02 m 0.01 m 10 km/h 2.8 m/s 0.5 m 0.28 m 0.09 m 0.05 m 0.03 m 20 km/h 5.6 m/s 2.0 m 0.56 m 0.19 m 0.09 m 0.06 m 30 km/h 8.3 m/s 4.5 m 0.83 m 0.28 m 0.14 m 0.08 m 50 km/h 13.9 m/s 12.5 m 1.39 m 0.46 m 0.23 m 0.14 m 100 km/h 27.8 m/s 50.0 m 2.78 m 0.93 m 0.46 m 0.28 m 200 km/h 55.6 m/s 200.0 m 5.56 m 1.85 m 0.93 m 0.56 m 300 km/h 83.3 m/s 450.0 m 8.33 m 2.78 m 1.39 m 0.83 m Data Processing km/h m/s Emergency Breaking Distance Hardware issue is very challenging. Not only perception have to be calculated. More powerful hardware will be needed. Latency and computing power
15. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 16 High Level View for a Data Center Approach compared to Embedded Embedded Data Center/Cloud Use Case Inference Training Batch Size Low, because of latency High, because of throughput Interconnect Multiple hardware instances Less important Very important Framework/Engine requirements > Low latency > Safety > Language: C/C++, (Java) > … > Multiple-machine training > Good scalability > Multi language support > …
16. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 17 End2End Approach Architectures for AI-powered driving Sensory Input DNN Steering Wheel Angle Modular Approach Fusion Image Recognition Depth Recognition Risk Prediction Environment Interpretation A B Trajectory Planning Control
17. Audi Electronics Venture GmbH - Fundamentals of Autonomous Driving - 14.03.2017 18 #TheComeback youtu.be/I2j2-DqcPfM This was just a brief Overview  Many open issues to be discussed in detail