OpenTalks.AI - Алексей Кашевник, Мониторинг водителя транспортного средства с использованием камеры и сенсоров смартфона​

Transcript

1. Driver Monitoring System Based on Image Analysis and Sensors Dr. Alexey Kashevnik St.Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS) 39, 14th Line St.Petersburg, 199178
2. Speaker Dr. Alexey Kashevnik  Senior Researcher of Computer-Aided Integrated Systems Laboratory, SPIIRAS (2008);  Associate Professor in International Laboratory on Intelligent Technologies for Socio- Cyber-Physical Systems, ITMO University (2014).  Member:  Associate Editor in International Journal of Embedded and Real-Time Communication Systems (IJERTCS), IGI Global.  Advisory board member in Association of Open Innovations FRUCT.  Editor in the following jounals:  Sensors, Computers, MTI, Technologies, MDPI  IEEE IoT Journal  Technical program committee member in international conferences:  on Internet of Things and Smart Spaces (ruSMART).  on Advances in Future Internet (AFIN).  Communications, Computation, Networks and Technologies (INNOV).  on Autonomic and Autonomous Systems (ICAS).  Communication Theory, Reliability, and Quality of Service (CTRQ).  Advances in Signal Processing and Artificial Intelligence (ASPAI).
3. Table of Contents  Motivation  Introduction  Driver Monitoring System  Dangerous Event Detection  Driver Data Analysis  Statistics Cloud  Conclusion 3
4. Driver Monitoring System: Motivation 4
5. Driver Monitoring System: General Scheme 5 Internet Vehicle Infotainment System Cloud Trip reports Statistics Consumers Smartphone: Driver Monitoring System Dangerous sate detection Recommendation generation Smartphone Camera-based monitoring Sensor data analysis Driver monitoring statistics Driver behavior patterns Driver personalization module
6. Smartphone Data Utilization for Dangerous Events Identification 6 Drowsiness Distraction High Pulse Rate Aggressive driving Calculated Parameters for single frame Offline Detected Dangerous States Stress Condition Drunk Driving Effects Traffic Accident Interference to Other Drivers Road Damage Non-eco-Friendly Driving Online Detected Dangerous States Raw data Calculated Visual Cues Camera Frame Location Audio Fragment Acceleration (Accelerometer) Light Level Angle (Gyroscope) Speed PERCLOS Head Yaw Angle Head Pitch Angle Mouth Openness Direction (Magnetometer) Yawning Eyes Openness Head Movements Vehicle Maneuvers Vehicle Position Lighting Conditions Forehead Color Speech Rate Speech Loudness Seat Belt
7. Dangerous States Examples 7 Drowsiness Dangerous State Head rotation angle Yawns count Eye openness
9. Driver Data Analysis for Dangerous States Detection Improvement 9 Fleet Manager Update threshold model Recommendation Smartphone Driver view Data Multi-layer perceptron classifier Drowsiness Distraction Training Training Data preprocessing Feature selection Model training drowsiness distraction Cloud computation Drowsiness and distraction based on thresholds Information Aggressive Driving
10. Driver Data Analysis: From Manual to Automatic Head Angles Calibration Line of sight/ Movement direction Yaw calibration angle Line of sight/ Movement direction Pitch calibration angle Accuracy 3 degree: Time, sec 0 200 400 600 800 70%: in less than 50 sec 90%: in ~3 min 99%: in ~10 min
11. Driver Data Analysis: Aggressive Driving Detection with Adaptive Thresholds x y Adaptive threshold for aggressive driving Acceleration on x axis Acceleration on y axis “Safe” maneuvers “Aggressive” maneuvers Driver’s acceleration history Learning a classifier: - Сar model (engine power etc.). - Individual driver’s style. Too sensitive threshold for a particular driver Too coarse threshold for a particular driver ML
12. Driver Data Analysis: Eyes Openness & Lightness Level 12 Sun in the back Sun is in front Eyes openness Lightness Level Time
13. Dispatcher Interface 13
14. Conclusion 14 Drive Safely in Google Play Market: https://play.google.com/store/apps/details?id=ru.igla.drivesafely&hl=ru Drive Safely! 10 000+ downloads
15. Jetson Nano Based Driver Monitoring System 15 Pilot project with: - Gazpromneft company - SKAUT
16. Thank you for Attention! 16 Dr. Alexey Kashevnik St. Petersburg, Russia, E-mail: alexey@iias.spb.su